2024-05-08T20:54:42.325 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T20:54:42.325 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi --owner scheduled_pdonnell@teuthology --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} --archive /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035 -- /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035/orig.config.yaml 2024-05-08T20:54:42.380 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T20:54:42.470 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035 branch: wip-pdonnell-testing-20240508.150423-reef description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} email: pdonnell@redhat.com first_in_suite: false job_id: '7698035' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-pdonnell-testing-20240508.150423-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 kclient: client.0: mount_subvol_num: 1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.150423-reef sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2738 sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 sleep_before_teardown: 0 subset: 17/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.150423-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCQ7ymuXb0i9cwjifNvwaHUkLOp9OADfvpvQ0QQ57ddE+gjC/TSDwowAly67LyMCiwcTUEs+HgbjQektt5i9zMI= smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOvMVu2GLlaSvIsud2+cN0lRvVLorzzDd/UDpzPdb0PEIb1f2jT80EwR3Occ7NzouBRiZPpZsACJPSTufZirOlo= smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5DcUGy306mw7nbZ2rQ4y71zViLfUwC0MEBBbPjwlFSMGjNOyvEOi+rYivNOeV4OYyS8f3fFY63WHVfwjTYBS0= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_19:22:21 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T20:54:42.470 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa; will attempt to use it 2024-05-08T20:54:42.470 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d21e4f961318b56f0d915caa2490c3711bf13c34/qa/tasks 2024-05-08T20:54:42.471 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T20:54:42.472 INFO:teuthology.task.internal:Checking packages... 2024-05-08T20:54:42.496 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'd21e4f961318b56f0d915caa2490c3711bf13c34' 2024-05-08T20:54:42.496 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T20:54:42.496 INFO:teuthology.packaging:ref: None 2024-05-08T20:54:42.496 INFO:teuthology.packaging:tag: None 2024-05-08T20:54:42.496 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.150423-reef 2024-05-08T20:54:42.496 INFO:teuthology.packaging:sha1: d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T20:54:42.497 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pdonnell-testing-20240508.150423-reef 2024-05-08T20:54:42.643 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1322.gd21e4f96 2024-05-08T20:54:42.645 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T20:54:42.651 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T20:54:42.651 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T20:54:42.683 INFO:teuthology.task.internal:Saving configuration 2024-05-08T20:54:42.704 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T20:54:42.726 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T20:54:42.751 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 20:49:33.886645', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCQ7ymuXb0i9cwjifNvwaHUkLOp9OADfvpvQ0QQ57ddE+gjC/TSDwowAly67LyMCiwcTUEs+HgbjQektt5i9zMI='} 2024-05-08T20:54:42.775 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 20:49:33.889421', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOvMVu2GLlaSvIsud2+cN0lRvVLorzzDd/UDpzPdb0PEIb1f2jT80EwR3Occ7NzouBRiZPpZsACJPSTufZirOlo='} 2024-05-08T20:54:42.798 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 20:49:33.888391', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5DcUGy306mw7nbZ2rQ4y71zViLfUwC0MEBBbPjwlFSMGjNOyvEOi+rYivNOeV4OYyS8f3fFY63WHVfwjTYBS0='} 2024-05-08T20:54:42.798 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T20:54:42.805 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-08T20:54:42.805 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-08T20:54:42.805 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-08T20:54:42.805 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T20:54:42.915 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9078afd160>, signals=[15]) 2024-05-08T20:54:42.915 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T20:54:42.922 INFO:teuthology.task.internal:Opening connections... 2024-05-08T20:54:42.922 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T20:54:42.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T20:54:43.000 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-05-08T20:54:43.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T20:54:43.075 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-05-08T20:54:43.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T20:54:43.151 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T20:54:43.157 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-05-08T20:54:43.179 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-05-08T20:54:43.179 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-05-08T20:54:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T20:54:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T20:54:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:ID="rhel" 2024-05-08T20:54:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="fedora" 2024-05-08T20:54:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="8.6" 2024-05-08T20:54:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el8" 2024-05-08T20:54:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T20:54:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-05-08T20:54:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T20:54:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T20:54:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T20:54:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T20:54:43.241 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T20:54:43.241 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T20:54:43.241 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T20:54:43.241 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T20:54:43.241 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T20:54:43.241 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-05-08T20:54:43.264 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-05-08T20:54:43.284 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-05-08T20:54:43.284 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-05-08T20:54:43.343 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T20:54:43.343 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:ID="rhel" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="fedora" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="8.6" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el8" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T20:54:43.344 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T20:54:43.345 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-05-08T20:54:43.363 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-05-08T20:54:43.383 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-05-08T20:54:43.384 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-05-08T20:54:43.443 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="Red Hat Enterprise Linux" 2024-05-08T20:54:43.443 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8.6 (Ootpa)" 2024-05-08T20:54:43.443 INFO:teuthology.orchestra.run.smithi082.stdout:ID="rhel" 2024-05-08T20:54:43.443 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="fedora" 2024-05-08T20:54:43.443 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8.6" 2024-05-08T20:54:43.443 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8" 2024-05-08T20:54:43.443 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-05-08T20:54:43.443 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-05-08T20:54:43.443 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-05-08T20:54:43.443 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://www.redhat.com/" 2024-05-08T20:54:43.444 INFO:teuthology.orchestra.run.smithi082.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-05-08T20:54:43.444 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T20:54:43.444 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T20:54:43.444 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-05-08T20:54:43.444 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-05-08T20:54:43.444 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-05-08T20:54:43.444 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-05-08T20:54:43.444 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-05-08T20:54:43.463 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T20:54:43.478 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T20:54:43.484 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T20:54:43.484 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-05-08T20:54:43.487 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-05-08T20:54:43.489 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-05-08T20:54:43.507 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T20:54:43.513 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T20:54:43.513 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-05-08T20:54:43.547 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-05-08T20:54:43.549 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-05-08T20:54:43.578 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T20:54:43.648 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T20:54:43.664 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-08T20:54:43.664 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-08T20:54:43.664 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T20:54:43.665 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T20:54:43.665 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T20:54:43.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T20:54:43.685 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-05-08T20:54:43.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T20:54:43.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T20:54:43.687 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-05-08T20:54:43.687 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-05-08T20:54:43.745 INFO:teuthology.orchestra.run.smithi082.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T20:54:43.746 INFO:teuthology.task.kernel:Running kernel on smithi082: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T20:54:43.746 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum install -y kernel 2024-05-08T20:54:43.747 INFO:teuthology.orchestra.run.smithi071.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T20:54:43.747 INFO:teuthology.task.kernel:Running kernel on smithi071: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T20:54:43.747 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum install -y kernel 2024-05-08T20:54:43.749 INFO:teuthology.orchestra.run.smithi033.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-05-08T20:54:43.749 INFO:teuthology.task.kernel:Running kernel on smithi033: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T20:54:43.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum install -y kernel 2024-05-08T20:54:45.219 INFO:teuthology.orchestra.run.smithi071.stdout:Updating Subscription Management repositories. 2024-05-08T20:54:45.219 INFO:teuthology.orchestra.run.smithi071.stdout:Unable to read consumer identity 2024-05-08T20:54:45.341 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2024-05-08T20:54:45.341 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2024-05-08T20:54:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T20:54:45.457 INFO:teuthology.orchestra.run.smithi071.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T20:54:45.457 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T20:54:45.509 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2024-05-08T20:54:45.510 INFO:teuthology.orchestra.run.smithi033.stdout:Unable to read consumer identity 2024-05-08T20:54:45.678 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T20:54:45.679 INFO:teuthology.orchestra.run.smithi082.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T20:54:45.679 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T20:54:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T20:54:45.876 INFO:teuthology.orchestra.run.smithi033.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T20:54:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T20:54:47.279 INFO:teuthology.orchestra.run.smithi071.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2024-05-08T20:54:47.425 INFO:teuthology.orchestra.run.smithi082.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2024-05-08T20:54:47.698 INFO:teuthology.orchestra.run.smithi033.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-05-08T20:54:48.553 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2024-05-08T20:54:48.576 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 19 MB/s | 16 MB 00:00 2024-05-08T20:54:48.663 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2024-05-08T20:54:52.109 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 464 kB/s | 24 kB 00:00 2024-05-08T20:54:52.279 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 421 kB/s | 24 kB 00:00 2024-05-08T20:54:52.624 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 353 kB/s | 24 kB 00:00 2024-05-08T20:54:52.861 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-08T20:54:52.862 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-08T20:54:52.933 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-05-08T20:54:52.934 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2024-05-08T20:54:52.934 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-05-08T20:54:53.031 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-08T20:54:53.032 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-08T20:54:53.054 DEBUG:teuthology.orchestra.run.smithi082:> echo no | sudo yum reinstall kernel || true 2024-05-08T20:54:53.106 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-05-08T20:54:53.107 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-05-08T20:54:53.107 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-05-08T20:54:53.232 DEBUG:teuthology.orchestra.run.smithi071:> echo no | sudo yum reinstall kernel || true 2024-05-08T20:54:53.428 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2024-05-08T20:54:53.428 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2024-05-08T20:54:53.444 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-05-08T20:54:53.444 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-05-08T20:54:53.524 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-08T20:54:53.525 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-05-08T20:54:53.525 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-08T20:54:53.609 INFO:teuthology.orchestra.run.smithi071.stdout:Updating Subscription Management repositories. 2024-05-08T20:54:53.609 INFO:teuthology.orchestra.run.smithi071.stdout:Unable to read consumer identity 2024-05-08T20:54:53.695 DEBUG:teuthology.orchestra.run.smithi033:> echo no | sudo yum reinstall kernel || true 2024-05-08T20:54:53.741 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T20:54:53.742 INFO:teuthology.orchestra.run.smithi082.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T20:54:53.742 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T20:54:53.827 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T20:54:53.828 INFO:teuthology.orchestra.run.smithi071.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T20:54:53.828 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T20:54:54.049 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 May 2024 08:54:52 PM UTC. 2024-05-08T20:54:54.083 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2024-05-08T20:54:54.084 INFO:teuthology.orchestra.run.smithi033.stdout:Unable to read consumer identity 2024-05-08T20:54:54.133 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T20:54:54.140 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 May 2024 08:54:52 PM UTC. 2024-05-08T20:54:54.140 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T20:54:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:Error: No packages marked for reinstall. 2024-05-08T20:54:54.228 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum reinstall -y kernel || true 2024-05-08T20:54:54.236 INFO:teuthology.orchestra.run.smithi071.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T20:54:54.236 INFO:teuthology.orchestra.run.smithi071.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T20:54:54.281 INFO:teuthology.orchestra.run.smithi071.stderr:Error: No packages marked for reinstall. 2024-05-08T20:54:54.328 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum reinstall -y kernel || true 2024-05-08T20:54:54.340 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T20:54:54.340 INFO:teuthology.orchestra.run.smithi033.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T20:54:54.340 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T20:54:54.560 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2024-05-08T20:54:54.561 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2024-05-08T20:54:54.664 INFO:teuthology.orchestra.run.smithi071.stdout:Updating Subscription Management repositories. 2024-05-08T20:54:54.664 INFO:teuthology.orchestra.run.smithi071.stdout:Unable to read consumer identity 2024-05-08T20:54:54.705 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:02 ago on Wed 08 May 2024 08:54:52 PM UTC. 2024-05-08T20:54:54.774 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T20:54:54.774 INFO:teuthology.orchestra.run.smithi082.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T20:54:54.774 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T20:54:54.792 INFO:teuthology.orchestra.run.smithi033.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T20:54:54.800 INFO:teuthology.orchestra.run.smithi033.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T20:54:54.840 INFO:teuthology.orchestra.run.smithi033.stderr:Error: No packages marked for reinstall. 2024-05-08T20:54:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T20:54:54.890 INFO:teuthology.orchestra.run.smithi071.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T20:54:54.891 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T20:54:54.897 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum reinstall -y kernel || true 2024-05-08T20:54:55.076 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:03 ago on Wed 08 May 2024 08:54:52 PM UTC. 2024-05-08T20:54:55.138 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T20:54:55.138 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T20:54:55.175 INFO:teuthology.orchestra.run.smithi082.stderr:Error: No packages marked for reinstall. 2024-05-08T20:54:55.191 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:03 ago on Wed 08 May 2024 08:54:52 PM UTC. 2024-05-08T20:54:55.221 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T20:54:55.255 INFO:teuthology.orchestra.run.smithi071.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T20:54:55.255 INFO:teuthology.orchestra.run.smithi071.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T20:54:55.278 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2024-05-08T20:54:55.278 INFO:teuthology.orchestra.run.smithi033.stdout:Unable to read consumer identity 2024-05-08T20:54:55.294 INFO:teuthology.orchestra.run.smithi071.stderr:Error: No packages marked for reinstall. 2024-05-08T20:54:55.341 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T20:54:55.382 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-08T20:54:55.383 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T20:54:55.383 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-08T20:54:55.383 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-05-08T20:54:55.383 DEBUG:teuthology.orchestra.run.smithi082:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T20:54:55.420 INFO:teuthology.orchestra.run.smithi082.stdout:ttyS1 2024-05-08T20:54:55.432 DEBUG:teuthology.parallel:result is None 2024-05-08T20:54:55.502 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-08T20:54:55.503 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T20:54:55.503 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-08T20:54:55.503 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-08T20:54:55.503 DEBUG:teuthology.orchestra.run.smithi071:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T20:54:55.504 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T20:54:55.504 INFO:teuthology.orchestra.run.smithi033.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-05-08T20:54:55.504 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T20:54:55.540 INFO:teuthology.orchestra.run.smithi071.stdout:ttyS1 2024-05-08T20:54:55.553 DEBUG:teuthology.parallel:result is None 2024-05-08T20:54:55.854 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:03 ago on Wed 08 May 2024 08:54:52 PM UTC. 2024-05-08T20:54:55.929 INFO:teuthology.orchestra.run.smithi033.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-05-08T20:54:55.929 INFO:teuthology.orchestra.run.smithi033.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-05-08T20:54:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:Error: No packages marked for reinstall. 2024-05-08T20:54:56.013 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T20:54:56.209 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-05-08T20:54:56.209 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-05-08T20:54:56.209 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-08T20:54:56.209 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-08T20:54:56.210 DEBUG:teuthology.orchestra.run.smithi033:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T20:54:56.248 INFO:teuthology.orchestra.run.smithi033.stdout:ttyS1 2024-05-08T20:54:56.276 DEBUG:teuthology.parallel:result is None 2024-05-08T20:54:56.276 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T20:54:56.283 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T20:54:56.283 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T20:54:56.319 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T20:54:56.321 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T20:54:56.341 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T20:54:56.347 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T20:54:56.353 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T20:54:56.354 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T20:54:56.379 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T20:54:56.381 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T20:54:56.418 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T20:54:56.424 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T20:54:56.425 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T20:54:56.451 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T20:54:56.454 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T20:54:56.480 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T20:54:56.491 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T20:54:56.499 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T20:54:56.508 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T20:54:56.513 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T20:54:56.520 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T20:54:56.521 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T20:54:56.529 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T20:54:56.529 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T20:54:56.531 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T20:54:56.535 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T20:54:56.564 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T20:54:56.573 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T20:54:56.573 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T20:54:56.607 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T20:54:56.609 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T20:54:56.626 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T20:54:56.716 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T20:54:56.772 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T20:54:56.861 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T20:54:56.916 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T20:54:56.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T20:54:56.995 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T20:54:57.043 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T20:54:57.092 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T20:54:57.180 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T20:54:57.221 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-08T20:54:57.221 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T20:54:57.288 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T20:54:57.334 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T20:54:57.387 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T20:54:57.473 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T20:54:57.516 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-08T20:54:57.517 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T20:54:57.585 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-05-08T20:54:57.588 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-05-08T20:54:57.590 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-05-08T20:54:57.629 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T20:54:57.641 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T20:54:57.669 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T20:54:58.116 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T20:54:58.123 INFO:teuthology.task.internal:Starting timer... 2024-05-08T20:54:58.123 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T20:54:58.132 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T20:54:58.141 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-05-08T20:54:58.141 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-05-08T20:54:58.213 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-05-08T20:54:58.215 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-05-08T20:54:58.286 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-05-08T20:54:58.288 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2024-05-08T20:54:58.356 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2024-05-08T20:54:58.358 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T20:54:58.358 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-05-08T20:54:58.406 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-05-08T20:54:58.406 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-05-08T20:54:58.426 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-05-08T20:54:58.426 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2024-05-08T20:54:58.451 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2024-05-08T20:54:58.451 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi071.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi082.front.sepia.ceph.com': 'permissive'} 2024-05-08T20:54:58.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T20:54:58.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T20:54:58.482 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T20:54:58.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T20:54:58.510 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T20:54:58.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T20:54:58.537 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T20:54:58.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-05-08T20:54:58.568 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-05-08T20:54:58.595 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2024-05-08T20:54:58.621 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T20:54:58.631 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T20:54:58.636 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T20:54:58.779 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T20:54:58.795 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T20:54:58.796 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi071.front.sepia.ceph.com,smithi082.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T21:05:57.034 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi082.front.sepia.ceph.com')] 2024-05-08T21:05:57.036 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-08T21:05:57.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:05:57.121 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-05-08T21:05:57.197 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-05-08T21:05:57.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-05-08T21:05:57.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:05:57.277 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-05-08T21:05:57.345 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-05-08T21:05:57.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-05-08T21:05:57.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T21:05:57.422 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-05-08T21:05:57.493 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-05-08T21:05:57.493 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T21:05:57.504 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T21:05:57.504 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T21:05:57.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T21:05:57.507 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T21:05:57.507 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T21:05:57.509 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T21:05:57.509 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T21:05:57.543 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T21:05:57.544 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T21:05:57.560 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T21:05:57.562 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T21:05:57.567 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T21:05:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T21:05:57.633 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-05-08T21:05:57.641 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-05-08T21:05:57.645 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-05-08T21:05:57.657 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-05-08T21:05:57.661 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T21:05:57.669 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2024-05-08T21:05:57.675 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T21:05:57.675 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T21:05:57.681 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2024-05-08T21:05:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T21:05:57.695 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T21:05:57.709 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T21:05:57.746 INFO:teuthology.orchestra.run.smithi071.stderr:bash: ntpq: command not found 2024-05-08T21:05:57.751 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T21:05:57.751 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-05-08T21:05:57.751 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:05:57.751 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:05:57.751 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:05:57.751 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:05:57.785 INFO:teuthology.orchestra.run.smithi033.stderr:bash: ntpq: command not found 2024-05-08T21:05:57.789 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T21:05:57.789 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-05-08T21:05:57.789 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:05:57.789 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:05:57.789 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:05:57.789 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:05:57.793 INFO:teuthology.orchestra.run.smithi082.stderr:bash: ntpq: command not found 2024-05-08T21:05:57.797 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T21:05:57.797 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-05-08T21:05:57.797 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:05:57.797 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:05:57.797 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:05:57.797 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:05:57.798 INFO:teuthology.run_tasks:Running task pexec... 2024-05-08T21:05:57.808 INFO:teuthology.task.pexec:Executing custom commands... 2024-05-08T21:05:57.808 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T21:05:57.808 DEBUG:teuthology.orchestra.run.smithi033:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T21:05:57.808 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi071.front.sepia.ceph.com 2024-05-08T21:05:57.809 DEBUG:teuthology.orchestra.run.smithi071:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T21:05:57.809 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi082.front.sepia.ceph.com 2024-05-08T21:05:57.809 DEBUG:teuthology.orchestra.run.smithi082:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-05-08T21:05:58.184 INFO:teuthology.orchestra.run.smithi071.stdout:Updating Subscription Management repositories. 2024-05-08T21:05:58.215 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2024-05-08T21:05:58.264 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2024-05-08T21:06:00.010 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 54 kB/s | 4.5 kB 00:00 2024-05-08T21:06:00.107 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 kB/s | 4.1 kB 00:00 2024-05-08T21:06:00.450 INFO:teuthology.orchestra.run.smithi033.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 31 kB/s | 4.5 kB 00:00 2024-05-08T21:06:00.487 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 kB/s | 4.5 kB 00:00 2024-05-08T21:06:00.641 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-05-08T21:06:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-05-08T21:06:03.489 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-05-08T21:06:03.490 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-05-08T21:06:03.490 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-05-08T21:06:03.490 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-05-08T21:06:03.490 INFO:teuthology.orchestra.run.smithi071.stdout:Resetting modules: 2024-05-08T21:06:03.490 INFO:teuthology.orchestra.run.smithi071.stdout: container-tools 2024-05-08T21:06:03.491 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T21:06:03.491 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-05-08T21:06:03.491 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-05-08T21:06:03.491 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T21:06:03.628 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-05-08T21:06:03.629 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-05-08T21:06:03.629 INFO:teuthology.orchestra.run.smithi082.stdout: Package Architecture Version Repository Size 2024-05-08T21:06:03.629 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-05-08T21:06:03.630 INFO:teuthology.orchestra.run.smithi082.stdout:Resetting modules: 2024-05-08T21:06:03.630 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools 2024-05-08T21:06:03.630 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:06:03.630 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-05-08T21:06:03.630 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-05-08T21:06:03.630 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:06:04.579 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-08T21:06:04.580 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-08T21:06:04.580 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-05-08T21:06:04.580 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-08T21:06:04.580 INFO:teuthology.orchestra.run.smithi033.stdout:Resetting modules: 2024-05-08T21:06:04.581 INFO:teuthology.orchestra.run.smithi033.stdout: container-tools 2024-05-08T21:06:04.581 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:06:04.581 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-08T21:06:04.581 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-05-08T21:06:04.581 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:06:14.716 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-05-08T21:06:15.216 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2024-05-08T21:06:16.804 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-05-08T21:06:17.249 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-05-08T21:06:17.476 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-08T21:06:17.562 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-05-08T21:06:17.998 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2024-05-08T21:06:18.019 INFO:teuthology.orchestra.run.smithi071.stdout:Updating Subscription Management repositories. 2024-05-08T21:06:19.782 INFO:teuthology.orchestra.run.smithi033.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-05-08T21:06:19.828 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-05-08T21:06:20.297 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-05-08T21:06:20.463 INFO:teuthology.orchestra.run.smithi082.stdout:========================================================================================================== 2024-05-08T21:06:20.463 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-05-08T21:06:20.463 INFO:teuthology.orchestra.run.smithi082.stdout:========================================================================================================== 2024-05-08T21:06:20.463 INFO:teuthology.orchestra.run.smithi082.stdout:Installing group/module packages: 2024-05-08T21:06:20.463 INFO:teuthology.orchestra.run.smithi082.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T21:06:20.463 INFO:teuthology.orchestra.run.smithi082.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T21:06:20.463 INFO:teuthology.orchestra.run.smithi082.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout:Installing weak dependencies: 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout:Installing module profiles: 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools/common 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling module streams: 2024-05-08T21:06:20.464 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools rhel8 2024-05-08T21:06:20.465 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:06:20.465 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-05-08T21:06:20.465 INFO:teuthology.orchestra.run.smithi082.stdout:========================================================================================================== 2024-05-08T21:06:20.465 INFO:teuthology.orchestra.run.smithi082.stdout:Install 13 Packages 2024-05-08T21:06:20.465 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:06:20.465 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 26 M 2024-05-08T21:06:20.465 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 87 M 2024-05-08T21:06:20.465 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-05-08T21:06:20.483 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 22 kB/s | 4.1 kB 00:00 2024-05-08T21:06:20.601 INFO:teuthology.orchestra.run.smithi082.stdout:(1/13): criu-libs-3.18-4.module+el8.9.0+20326+3 292 kB/s | 39 kB 00:00 2024-05-08T21:06:20.606 INFO:teuthology.orchestra.run.smithi033.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 17 kB/s | 4.1 kB 00:00 2024-05-08T21:06:20.626 INFO:teuthology.orchestra.run.smithi082.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 155 kB/s | 25 kB 00:00 2024-05-08T21:06:20.651 INFO:teuthology.orchestra.run.smithi082.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 506 kB/s | 94 kB 00:00 2024-05-08T21:06:20.835 INFO:teuthology.orchestra.run.smithi082.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 4.8 MB/s | 1.0 MB 00:00 2024-05-08T21:06:20.860 INFO:teuthology.orchestra.run.smithi082.stdout:(5/13): yajl-2.1.0-12.el8.x86_64.rpm 158 kB/s | 41 kB 00:00 2024-05-08T21:06:21.027 INFO:teuthology.orchestra.run.smithi082.stdout:(6/13): buildah-1.31.5-1.module+el8.9.0+21697+6 24 MB/s | 8.8 MB 00:00 2024-05-08T21:06:21.053 INFO:teuthology.orchestra.run.smithi082.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.2 MB/s | 239 kB 00:00 2024-05-08T21:06:21.120 INFO:teuthology.orchestra.run.smithi082.stdout:(8/13): cockpit-podman-75-1.module+el8.9.0+2169 2.5 MB/s | 739 kB 00:00 2024-05-08T21:06:21.162 INFO:teuthology.orchestra.run.smithi082.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 1.4 MB/s | 154 kB 00:00 2024-05-08T21:06:21.245 INFO:teuthology.orchestra.run.smithi082.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 17 MB/s | 3.7 MB 00:00 2024-05-08T21:06:21.329 INFO:teuthology.orchestra.run.smithi082.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 15 MB/s | 2.5 MB 00:00 2024-05-08T21:06:21.383 INFO:teuthology.orchestra.run.smithi082.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 361 kB/s | 49 kB 00:00 2024-05-08T21:06:21.512 INFO:teuthology.orchestra.run.smithi082.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 21 MB/s | 8.1 MB 00:00 2024-05-08T21:06:21.513 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:06:21.513 INFO:teuthology.orchestra.run.smithi082.stdout:Total 24 MB/s | 26 MB 00:01 2024-05-08T21:06:21.791 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-05-08T21:06:21.811 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-05-08T21:06:21.812 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-05-08T21:06:22.040 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-05-08T21:06:22.049 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-05-08T21:06:22.814 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-05-08T21:06:23.305 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T21:06:23.359 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-05-08T21:06:23.523 INFO:teuthology.orchestra.run.smithi071.stdout:========================================================================================================== 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout:========================================================================================================== 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout:Installing group/module packages: 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T21:06:23.524 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout:Installing module profiles: 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout: container-tools/common 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling module streams: 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout: container-tools rhel8 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout:========================================================================================================== 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout:Install 13 Packages 2024-05-08T21:06:23.525 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T21:06:23.526 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 26 M 2024-05-08T21:06:23.526 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 87 M 2024-05-08T21:06:23.526 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-05-08T21:06:23.526 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T21:06:23.673 INFO:teuthology.orchestra.run.smithi071.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 169 kB/s | 25 kB 00:00 2024-05-08T21:06:23.679 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T21:06:23.698 INFO:teuthology.orchestra.run.smithi071.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 229 kB/s | 39 kB 00:00 2024-05-08T21:06:23.724 INFO:teuthology.orchestra.run.smithi071.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 477 kB/s | 94 kB 00:00 2024-05-08T21:06:23.807 INFO:teuthology.orchestra.run.smithi071.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 306 kB/s | 41 kB 00:00 2024-05-08T21:06:23.811 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-08T21:06:23.849 INFO:teuthology.orchestra.run.smithi071.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 6.7 MB/s | 1.0 MB 00:00 2024-05-08T21:06:23.864 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T21:06:23.933 INFO:teuthology.orchestra.run.smithi071.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 5.8 MB/s | 739 kB 00:00 2024-05-08T21:06:23.966 INFO:teuthology.orchestra.run.smithi071.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 2.0 MB/s | 239 kB 00:00 2024-05-08T21:06:23.992 INFO:teuthology.orchestra.run.smithi033.stdout:========================================================================================================== 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout:========================================================================================================== 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout:Installing group/module packages: 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: buildah x86_64 1:1.31.5-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: cockpit-podman noarch 75-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 739 k 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: crun x86_64 1.8.7-1.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 239 k 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: netavark x86_64 2:1.7.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: python3-podman noarch 4.6.0-2.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 154 k 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: skopeo x86_64 2:1.13.3-3.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: udica noarch 0.2.6-20.module+el8.9.0+21697+6a5e98e7 rhel-8-for-x86_64-appstream-rpms 49 k 2024-05-08T21:06:23.993 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout:Installing module profiles: 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: container-tools/common 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling module streams: 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: container-tools rhel8 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-08T21:06:23.994 INFO:teuthology.orchestra.run.smithi033.stdout:========================================================================================================== 2024-05-08T21:06:23.995 INFO:teuthology.orchestra.run.smithi033.stdout:Install 13 Packages 2024-05-08T21:06:23.995 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:06:23.995 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 26 M 2024-05-08T21:06:23.995 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 87 M 2024-05-08T21:06:23.995 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-08T21:06:24.059 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T21:06:24.091 INFO:teuthology.orchestra.run.smithi071.stdout:(8/13): buildah-1.31.5-1.module+el8.9.0+21697+6 24 MB/s | 8.8 MB 00:00 2024-05-08T21:06:24.117 INFO:teuthology.orchestra.run.smithi071.stdout:(9/13): python3-podman-4.6.0-2.module+el8.9.0+2 1.0 MB/s | 154 kB 00:00 2024-05-08T21:06:24.203 INFO:teuthology.orchestra.run.smithi071.stdout:(10/13): netavark-1.7.0-2.module+el8.9.0+21697+ 14 MB/s | 3.7 MB 00:00 2024-05-08T21:06:24.229 INFO:teuthology.orchestra.run.smithi033.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 106 kB/s | 25 kB 00:00 2024-05-08T21:06:24.262 INFO:teuthology.orchestra.run.smithi033.stdout:(2/13): criu-libs-3.18-4.module+el8.9.0+20326+3 148 kB/s | 39 kB 00:00 2024-05-08T21:06:24.269 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T21:06:24.287 INFO:teuthology.orchestra.run.smithi033.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 323 kB/s | 94 kB 00:00 2024-05-08T21:06:24.309 INFO:teuthology.orchestra.run.smithi071.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 13 MB/s | 2.5 MB 00:00 2024-05-08T21:06:24.334 INFO:teuthology.orchestra.run.smithi071.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 370 kB/s | 49 kB 00:00 2024-05-08T21:06:24.362 INFO:teuthology.orchestra.run.smithi033.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 308 kB/s | 41 kB 00:00 2024-05-08T21:06:24.475 INFO:teuthology.orchestra.run.smithi033.stdout:(5/13): aardvark-dns-1.7.0-1.module+el8.9.0+216 4.7 MB/s | 1.0 MB 00:00 2024-05-08T21:06:24.521 INFO:teuthology.orchestra.run.smithi033.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2169 4.6 MB/s | 739 kB 00:00 2024-05-08T21:06:24.528 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T21:06:24.543 INFO:teuthology.orchestra.run.smithi071.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 18 MB/s | 8.1 MB 00:00 2024-05-08T21:06:24.543 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:06:24.543 INFO:teuthology.orchestra.run.smithi071.stdout:Total 25 MB/s | 26 MB 00:01 2024-05-08T21:06:24.672 INFO:teuthology.orchestra.run.smithi033.stdout:(7/13): buildah-1.31.5-1.module+el8.9.0+21697+6 23 MB/s | 8.8 MB 00:00 2024-05-08T21:06:24.705 INFO:teuthology.orchestra.run.smithi033.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+21697+6a5e9 1.0 MB/s | 239 kB 00:00 2024-05-08T21:06:24.814 INFO:teuthology.orchestra.run.smithi033.stdout:(9/13): netavark-1.7.0-2.module+el8.9.0+21697+6 13 MB/s | 3.7 MB 00:00 2024-05-08T21:06:24.824 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-05-08T21:06:24.839 INFO:teuthology.orchestra.run.smithi033.stdout:(10/13): python3-podman-4.6.0-2.module+el8.9.0+ 920 kB/s | 154 kB 00:00 2024-05-08T21:06:24.845 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-05-08T21:06:24.845 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-05-08T21:06:25.056 INFO:teuthology.orchestra.run.smithi033.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2169 10 MB/s | 2.5 MB 00:00 2024-05-08T21:06:25.075 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-05-08T21:06:25.081 INFO:teuthology.orchestra.run.smithi033.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+21697+6a 204 kB/s | 49 kB 00:00 2024-05-08T21:06:25.084 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-05-08T21:06:25.215 INFO:teuthology.orchestra.run.smithi033.stdout:(13/13): skopeo-1.13.3-3.module+el8.9.0+21697+6 16 MB/s | 8.1 MB 00:00 2024-05-08T21:06:25.216 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:06:25.216 INFO:teuthology.orchestra.run.smithi033.stdout:Total 21 MB/s | 26 MB 00:01 2024-05-08T21:06:25.424 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:06:25.523 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-08T21:06:25.548 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-08T21:06:25.548 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-08T21:06:25.698 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:06:25.803 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-08T21:06:25.813 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-08T21:06:25.821 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-05-08T21:06:26.112 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T21:06:26.325 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T21:06:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T21:06:26.607 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-08T21:06:26.664 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T21:06:26.808 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T21:06:26.977 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T21:06:26.979 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T21:06:27.167 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 1/13 2024-05-08T21:06:27.191 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T21:06:27.228 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T21:06:27.385 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 2/13 2024-05-08T21:06:27.398 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:06:27.481 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T21:06:27.572 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : yajl-2.1.0-12.el8.x86_64 3/13 2024-05-08T21:06:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 4/13 2024-05-08T21:06:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:06:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T21:06:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T21:06:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T21:06:27.801 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T21:06:27.801 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T21:06:27.801 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T21:06:27.801 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T21:06:27.801 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:06:27.801 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:06:27.801 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T21:06:27.801 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T21:06:27.801 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T21:06:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-05-08T21:06:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-05-08T21:06:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:06:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 7/13 2024-05-08T21:06:28.567 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:06:28.914 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T21:06:29.393 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:06:29.615 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:06:29.789 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T21:06:29.985 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T21:06:30.054 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 10/13 2024-05-08T21:06:30.167 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:06:30.594 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:06:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T21:06:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T21:06:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T21:06:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T21:06:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T21:06:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T21:06:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T21:06:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:06:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:06:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T21:06:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T21:06:30.595 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T21:06:31.238 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 11/13 2024-05-08T21:06:31.438 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 12/13 2024-05-08T21:06:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:06:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 13/13 2024-05-08T21:06:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-05-08T21:06:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-05-08T21:06:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 3/13 2024-05-08T21:06:32.071 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 4/13 2024-05-08T21:06:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98 5/13 2024-05-08T21:06:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x 6/13 2024-05-08T21:06:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7. 7/13 2024-05-08T21:06:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 8/13 2024-05-08T21:06:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x 9/13 2024-05-08T21:06:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98 10/13 2024-05-08T21:06:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x8 11/13 2024-05-08T21:06:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x 12/13 2024-05-08T21:06:37.215 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T21:06:37.215 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2024-05-08T21:06:37.764 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:06:37.764 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-05-08T21:06:37.764 INFO:teuthology.orchestra.run.smithi082.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:37.764 INFO:teuthology.orchestra.run.smithi082.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:37.764 INFO:teuthology.orchestra.run.smithi082.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:06:37.764 INFO:teuthology.orchestra.run.smithi082.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T21:06:37.764 INFO:teuthology.orchestra.run.smithi082.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:37.764 INFO:teuthology.orchestra.run.smithi082.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:37.764 INFO:teuthology.orchestra.run.smithi082.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:06:37.764 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T21:06:37.765 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T21:06:37.765 INFO:teuthology.orchestra.run.smithi082.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:37.765 INFO:teuthology.orchestra.run.smithi082.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:37.765 INFO:teuthology.orchestra.run.smithi082.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:06:37.765 INFO:teuthology.orchestra.run.smithi082.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T21:06:37.765 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:06:37.765 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-05-08T21:06:37.957 DEBUG:teuthology.parallel:result is None 2024-05-08T21:06:43.148 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T21:06:43.148 INFO:teuthology.orchestra.run.smithi071.stdout:Installed products updated. 2024-05-08T21:06:43.708 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T21:06:43.709 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T21:06:43.710 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-05-08T21:06:43.875 DEBUG:teuthology.parallel:result is None 2024-05-08T21:06:45.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarc 13/13 2024-05-08T21:06:45.049 INFO:teuthology.orchestra.run.smithi033.stdout:Installed products updated. 2024-05-08T21:06:45.678 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:06:45.678 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: buildah-1:1.31.5-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: cockpit-podman-75-1.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: crun-1.8.7-1.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: netavark-2:1.7.0-2.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: python3-podman-4.6.0-2.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: skopeo-2:1.13.3-3.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: toolbox-0.0.99.4-5.module+el8.9.0+21697+6a5e98e7.x86_64 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: udica-0.2.6-20.module+el8.9.0+21697+6a5e98e7.noarch 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: yajl-2.1.0-12.el8.x86_64 2024-05-08T21:06:45.679 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:06:45.680 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-08T21:06:45.870 DEBUG:teuthology.parallel:result is None 2024-05-08T21:06:45.870 INFO:teuthology.run_tasks:Running task install... 2024-05-08T21:06:45.879 DEBUG:teuthology.task.install:project ceph 2024-05-08T21:06:45.879 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'}} 2024-05-08T21:06:45.879 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34'} 2024-05-08T21:06:45.879 INFO:teuthology.task.install:Using flavor: default 2024-05-08T21:06:45.889 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T21:06:45.890 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T21:06:45.890 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T21:06:45.890 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:06:45.892 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T21:06:45.893 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:06:45.894 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'tag': None, 'wait_for_package': False} 2024-05-08T21:06:45.894 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:06:46.048 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T21:06:46.048 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T21:06:46.117 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T21:06:46.117 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T21:06:46.121 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/ 2024-05-08T21:06:46.121 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1322.gd21e4f96 2024-05-08T21:06:46.419 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T21:06:46.419 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-08T21:06:46.419 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T21:06:46.438 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T21:06:46.439 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:06:46.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T21:06:46.452 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T21:06:46.453 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T21:06:46.474 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T21:06:46.474 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T21:06:46.524 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T21:06:46.524 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-08T21:06:46.525 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T21:06:46.526 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T21:06:46.548 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T21:06:46.559 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T21:06:46.559 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d21e4f961318b56f0d915caa2490c3711bf13c34/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T21:06:46.607 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T21:06:46.631 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T21:06:46.637 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T21:06:46.669 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-05-08T21:06:46.671 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-05-08T21:06:46.674 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-05-08T21:06:46.676 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-05-08T21:06:46.712 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T21:06:46.780 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2024-05-08T21:06:46.782 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2024-05-08T21:06:47.054 INFO:teuthology.orchestra.run.smithi071.stdout:Updating Subscription Management repositories. 2024-05-08T21:06:47.105 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2024-05-08T21:06:47.159 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2024-05-08T21:06:48.287 INFO:teuthology.orchestra.run.smithi071.stdout:63 files removed 2024-05-08T21:06:48.346 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T21:06:48.350 INFO:teuthology.orchestra.run.smithi082.stdout:63 files removed 2024-05-08T21:06:48.402 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T21:06:48.422 INFO:teuthology.orchestra.run.smithi033.stdout:63 files removed 2024-05-08T21:06:48.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T21:06:48.683 INFO:teuthology.orchestra.run.smithi071.stdout:Updating Subscription Management repositories. 2024-05-08T21:06:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2024-05-08T21:06:48.861 INFO:teuthology.orchestra.run.smithi033.stdout:Updating Subscription Management repositories. 2024-05-08T21:06:51.022 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 101 kB/s | 80 kB 00:00 2024-05-08T21:06:51.099 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 94 kB/s | 80 kB 00:00 2024-05-08T21:06:51.337 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 93 kB/s | 80 kB 00:00 2024-05-08T21:06:51.630 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 20 kB/s | 11 kB 00:00 2024-05-08T21:06:51.841 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 16 kB/s | 11 kB 00:00 2024-05-08T21:06:52.072 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 16 kB/s | 11 kB 00:00 2024-05-08T21:06:52.314 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 2.3 kB/s | 1.5 kB 00:00 2024-05-08T21:06:52.498 INFO:teuthology.orchestra.run.smithi071.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-05-08T21:06:52.533 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 2.2 kB/s | 1.5 kB 00:00 2024-05-08T21:06:52.726 INFO:teuthology.orchestra.run.smithi082.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-05-08T21:06:52.880 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 1.9 kB/s | 1.5 kB 00:00 2024-05-08T21:06:53.047 INFO:teuthology.orchestra.run.smithi033.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-05-08T21:06:54.227 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 16 MB 00:01 2024-05-08T21:06:54.937 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 16 MB 00:02 2024-05-08T21:06:57.865 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 387 kB/s | 24 kB 00:00 2024-05-08T21:06:58.509 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 342 kB/s | 24 kB 00:00 2024-05-08T21:06:58.967 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 8.5 MB/s | 8.8 MB 00:01 2024-05-08T21:06:59.986 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 67 MB 00:01 2024-05-08T21:07:01.852 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 61 MB 00:01 2024-05-08T21:07:05.423 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 16 MB 00:12 2024-05-08T21:07:09.514 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 393 kB/s | 24 kB 00:00 2024-05-08T21:07:10.186 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 14 MB/s | 8.8 MB 00:00 2024-05-08T21:07:10.308 INFO:teuthology.orchestra.run.smithi033.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 12 MB/s | 8.8 MB 00:00 2024-05-08T21:07:13.293 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 67 MB 00:01 2024-05-08T21:07:13.320 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 61 MB 00:01 2024-05-08T21:07:13.742 INFO:teuthology.orchestra.run.smithi033.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 MB/s | 61 MB 00:01 2024-05-08T21:07:26.376 INFO:teuthology.orchestra.run.smithi033.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 67 MB 00:01 2024-05-08T21:07:28.104 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:07:28.106 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:07:28.123 INFO:teuthology.orchestra.run.smithi071.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T21:07:28.124 INFO:teuthology.orchestra.run.smithi071.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T21:07:28.125 INFO:teuthology.orchestra.run.smithi071.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T21:07:28.127 INFO:teuthology.orchestra.run.smithi071.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T21:07:28.129 INFO:teuthology.orchestra.run.smithi071.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T21:07:28.133 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T21:07:28.135 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T21:07:28.357 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-05-08T21:07:28.371 INFO:teuthology.orchestra.run.smithi071.stdout:=================================================================================================================================== 2024-05-08T21:07:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-05-08T21:07:28.371 INFO:teuthology.orchestra.run.smithi071.stdout:=================================================================================================================================== 2024-05-08T21:07:28.371 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-05-08T21:07:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T21:07:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T21:07:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T21:07:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T21:07:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T21:07:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T21:07:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T21:07:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T21:07:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T21:07:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T21:07:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T21:07:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T21:07:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T21:07:28.374 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-05-08T21:07:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T21:07:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T21:07:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T21:07:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T21:07:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T21:07:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T21:07:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T21:07:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T21:07:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:07:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T21:07:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T21:07:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T21:07:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T21:07:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T21:07:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T21:07:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T21:07:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T21:07:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T21:07:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T21:07:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T21:07:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T21:07:28.382 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling module streams: 2024-05-08T21:07:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql 10 2024-05-08T21:07:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T21:07:28.383 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-05-08T21:07:28.383 INFO:teuthology.orchestra.run.smithi071.stdout:=================================================================================================================================== 2024-05-08T21:07:28.383 INFO:teuthology.orchestra.run.smithi071.stdout:Install 130 Packages 2024-05-08T21:07:28.383 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 16 Packages 2024-05-08T21:07:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T21:07:28.383 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 215 M 2024-05-08T21:07:28.383 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-05-08T21:07:28.575 INFO:teuthology.orchestra.run.smithi071.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 53 kB/s | 6.4 kB 00:00 2024-05-08T21:07:28.817 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:07:28.818 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:07:28.838 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T21:07:28.839 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T21:07:28.841 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T21:07:28.842 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T21:07:28.845 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T21:07:28.849 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T21:07:28.851 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T21:07:28.892 INFO:teuthology.orchestra.run.smithi071.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 2.7 MB/s | 860 kB 00:00 2024-05-08T21:07:28.975 INFO:teuthology.orchestra.run.smithi071.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 1.9 MB/s | 166 kB 00:00 2024-05-08T21:07:29.073 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-05-08T21:07:29.084 INFO:teuthology.orchestra.run.smithi071.stdout:(4/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 8.8 MB/s | 5.5 MB 00:00 2024-05-08T21:07:29.087 INFO:teuthology.orchestra.run.smithi082.stdout:=================================================================================================================================== 2024-05-08T21:07:29.087 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-05-08T21:07:29.087 INFO:teuthology.orchestra.run.smithi082.stdout:=================================================================================================================================== 2024-05-08T21:07:29.087 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T21:07:29.088 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T21:07:29.089 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-05-08T21:07:29.090 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T21:07:29.091 INFO:teuthology.orchestra.run.smithi082.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T21:07:29.092 INFO:teuthology.orchestra.run.smithi082.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T21:07:29.093 INFO:teuthology.orchestra.run.smithi082.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T21:07:29.094 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T21:07:29.095 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T21:07:29.096 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T21:07:29.097 INFO:teuthology.orchestra.run.smithi082.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling module streams: 2024-05-08T21:07:29.098 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql 10 2024-05-08T21:07:29.099 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:07:29.099 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-05-08T21:07:29.099 INFO:teuthology.orchestra.run.smithi082.stdout:=================================================================================================================================== 2024-05-08T21:07:29.099 INFO:teuthology.orchestra.run.smithi082.stdout:Install 130 Packages 2024-05-08T21:07:29.099 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 16 Packages 2024-05-08T21:07:29.099 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:07:29.099 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 215 M 2024-05-08T21:07:29.099 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-05-08T21:07:29.251 INFO:teuthology.orchestra.run.smithi071.stdout:(5/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 9.4 MB/s | 1.6 MB 00:00 2024-05-08T21:07:29.751 INFO:teuthology.orchestra.run.smithi071.stdout:(6/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 9.7 MB/s | 4.8 MB 00:00 2024-05-08T21:07:29.785 INFO:teuthology.orchestra.run.smithi082.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 19 kB/s | 6.4 kB 00:00 2024-05-08T21:07:29.793 INFO:teuthology.orchestra.run.smithi071.stdout:(7/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 2.8 MB/s | 2.3 MB 00:00 2024-05-08T21:07:30.344 INFO:teuthology.orchestra.run.smithi071.stdout:(8/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 11 MB/s | 22 MB 00:01 2024-05-08T21:07:30.410 INFO:teuthology.orchestra.run.smithi071.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 366 kB/s | 24 kB 00:00 2024-05-08T21:07:30.744 INFO:teuthology.orchestra.run.smithi082.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 898 kB/s | 860 kB 00:00 2024-05-08T21:07:30.944 INFO:teuthology.orchestra.run.smithi082.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 830 kB/s | 166 kB 00:00 2024-05-08T21:07:31.777 INFO:teuthology.orchestra.run.smithi071.stdout:(10/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 9.1 MB/s | 18 MB 00:02 2024-05-08T21:07:31.877 INFO:teuthology.orchestra.run.smithi071.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 2.6 MB/s | 261 kB 00:00 2024-05-08T21:07:31.969 INFO:teuthology.orchestra.run.smithi071.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 338 kB/s | 31 kB 00:00 2024-05-08T21:07:32.228 INFO:teuthology.orchestra.run.smithi071.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 3.0 MB/s | 782 kB 00:00 2024-05-08T21:07:32.286 INFO:teuthology.orchestra.run.smithi082.stdout:(4/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 1.7 MB/s | 2.3 MB 00:01 2024-05-08T21:07:32.303 INFO:teuthology.orchestra.run.smithi071.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.9 MB/s | 142 kB 00:00 2024-05-08T21:07:32.386 INFO:teuthology.orchestra.run.smithi071.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 1.6 MB/s | 139 kB 00:00 2024-05-08T21:07:32.487 INFO:teuthology.orchestra.run.smithi071.stdout:(16/146): libradosstriper1-18.2.2-1322.gd21e4f9 4.6 MB/s | 474 kB 00:00 2024-05-08T21:07:32.586 INFO:teuthology.orchestra.run.smithi082.stdout:(5/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 6.9 MB/s | 22 MB 00:03 2024-05-08T21:07:33.094 INFO:teuthology.orchestra.run.smithi082.stdout:(6/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 9.5 MB/s | 4.8 MB 00:00 2024-05-08T21:07:33.128 INFO:teuthology.orchestra.run.smithi082.stdout:(7/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 1.9 MB/s | 1.6 MB 00:00 2024-05-08T21:07:33.862 INFO:teuthology.orchestra.run.smithi071.stdout:(17/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 4.8 MB/s | 6.5 MB 00:01 2024-05-08T21:07:33.945 INFO:teuthology.orchestra.run.smithi071.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 538 kB/s | 45 kB 00:00 2024-05-08T21:07:34.029 INFO:teuthology.orchestra.run.smithi071.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 1.5 MB/s | 129 kB 00:00 2024-05-08T21:07:34.121 INFO:teuthology.orchestra.run.smithi071.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 2.1 MB/s | 197 kB 00:00 2024-05-08T21:07:34.254 INFO:teuthology.orchestra.run.smithi071.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 2.7 MB/s | 370 kB 00:00 2024-05-08T21:07:34.320 INFO:teuthology.orchestra.run.smithi082.stdout:(8/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el8 11 MB/s | 13 MB 00:01 2024-05-08T21:07:34.388 INFO:teuthology.orchestra.run.smithi071.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 2.7 MB/s | 365 kB 00:00 2024-05-08T21:07:34.420 INFO:teuthology.orchestra.run.smithi082.stdout:(9/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 1.1 MB/s | 5.5 MB 00:04 2024-05-08T21:07:34.438 INFO:teuthology.orchestra.run.smithi071.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 2.1 MB/s | 107 kB 00:00 2024-05-08T21:07:34.445 INFO:teuthology.orchestra.run.smithi082.stdout:(10/146): ceph-selinux-18.2.2-1322.gd21e4f96.el 194 kB/s | 24 kB 00:00 2024-05-08T21:07:34.512 INFO:teuthology.orchestra.run.smithi082.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 3.8 MB/s | 261 kB 00:00 2024-05-08T21:07:34.562 INFO:teuthology.orchestra.run.smithi082.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 620 kB/s | 31 kB 00:00 2024-05-08T21:07:34.580 INFO:teuthology.orchestra.run.smithi071.stdout:(24/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 2.7 MB/s | 13 MB 00:04 2024-05-08T21:07:34.597 INFO:teuthology.orchestra.run.smithi071.stdout:(25/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 599 kB/s | 95 kB 00:00 2024-05-08T21:07:34.671 INFO:teuthology.orchestra.run.smithi082.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 7.0 MB/s | 782 kB 00:00 2024-05-08T21:07:34.681 INFO:teuthology.orchestra.run.smithi071.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 2.1 MB/s | 181 kB 00:00 2024-05-08T21:07:34.721 INFO:teuthology.orchestra.run.smithi082.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 2.8 MB/s | 142 kB 00:00 2024-05-08T21:07:34.739 INFO:teuthology.orchestra.run.smithi071.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 418 kB/s | 24 kB 00:00 2024-05-08T21:07:34.788 INFO:teuthology.orchestra.run.smithi082.stdout:(15/146): librados-devel-18.2.2-1322.gd21e4f96. 2.0 MB/s | 139 kB 00:00 2024-05-08T21:07:34.806 INFO:teuthology.orchestra.run.smithi071.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 2.0 MB/s | 138 kB 00:00 2024-05-08T21:07:34.863 INFO:teuthology.orchestra.run.smithi082.stdout:(16/146): libradosstriper1-18.2.2-1322.gd21e4f9 6.2 MB/s | 474 kB 00:00 2024-05-08T21:07:35.564 INFO:teuthology.orchestra.run.smithi082.stdout:(17/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 9.4 MB/s | 6.5 MB 00:00 2024-05-08T21:07:35.614 INFO:teuthology.orchestra.run.smithi082.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 900 kB/s | 45 kB 00:00 2024-05-08T21:07:35.664 INFO:teuthology.orchestra.run.smithi082.stdout:(19/146): python3-ceph-common-18.2.2-1322.gd21e 2.5 MB/s | 129 kB 00:00 2024-05-08T21:07:35.681 INFO:teuthology.orchestra.run.smithi071.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 4.0 MB/s | 3.5 MB 00:00 2024-05-08T21:07:35.714 INFO:teuthology.orchestra.run.smithi082.stdout:(20/146): python3-cephfs-18.2.2-1322.gd21e4f96. 3.9 MB/s | 197 kB 00:00 2024-05-08T21:07:35.773 INFO:teuthology.orchestra.run.smithi082.stdout:(21/146): python3-rados-18.2.2-1322.gd21e4f96.e 6.2 MB/s | 370 kB 00:00 2024-05-08T21:07:35.856 INFO:teuthology.orchestra.run.smithi082.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 4.3 MB/s | 365 kB 00:00 2024-05-08T21:07:35.907 INFO:teuthology.orchestra.run.smithi082.stdout:(23/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 2.1 MB/s | 107 kB 00:00 2024-05-08T21:07:35.957 INFO:teuthology.orchestra.run.smithi082.stdout:(24/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.9 MB/s | 95 kB 00:00 2024-05-08T21:07:36.107 INFO:teuthology.orchestra.run.smithi071.stdout:(30/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 8.3 MB/s | 47 MB 00:05 2024-05-08T21:07:36.157 INFO:teuthology.orchestra.run.smithi071.stdout:(31/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 2.2 MB/s | 3.5 MB 00:01 2024-05-08T21:07:36.199 INFO:teuthology.orchestra.run.smithi071.stdout:(32/146): ceph-mgr-modules-core-18.2.2-1322.gd2 2.6 MB/s | 245 kB 00:00 2024-05-08T21:07:36.199 INFO:teuthology.orchestra.run.smithi082.stdout:(25/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 14 MB/s | 3.5 MB 00:00 2024-05-08T21:07:36.224 INFO:teuthology.orchestra.run.smithi071.stdout:(33/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 738 kB/s | 49 kB 00:00 2024-05-08T21:07:36.249 INFO:teuthology.orchestra.run.smithi071.stdout:(34/146): ceph-prometheus-alerts-18.2.2-1322.gd 326 kB/s | 16 kB 00:00 2024-05-08T21:07:36.258 INFO:teuthology.orchestra.run.smithi082.stdout:(26/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 3.1 MB/s | 181 kB 00:00 2024-05-08T21:07:36.293 INFO:teuthology.orchestra.run.smithi071.stdout:(35/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 3.9 MB/s | 263 kB 00:00 2024-05-08T21:07:36.300 INFO:teuthology.orchestra.run.smithi082.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1322.g 583 kB/s | 24 kB 00:00 2024-05-08T21:07:36.317 INFO:teuthology.orchestra.run.smithi071.stdout:(36/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 2.5 MB/s | 170 kB 00:00 2024-05-08T21:07:36.342 INFO:teuthology.orchestra.run.smithi071.stdout:(37/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 360 kB/s | 18 kB 00:00 2024-05-08T21:07:36.350 INFO:teuthology.orchestra.run.smithi082.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 2.7 MB/s | 138 kB 00:00 2024-05-08T21:07:36.609 INFO:teuthology.orchestra.run.smithi082.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 14 MB/s | 3.5 MB 00:00 2024-05-08T21:07:36.717 INFO:teuthology.orchestra.run.smithi071.stdout:(38/146): liboath-2.6.2-3.el8.x86_64.rpm 157 kB/s | 59 kB 00:00 2024-05-08T21:07:36.909 INFO:teuthology.orchestra.run.smithi071.stdout:(39/146): ceph-mgr-diskprediction-local-18.2.2- 6.0 MB/s | 7.4 MB 00:01 2024-05-08T21:07:36.951 INFO:teuthology.orchestra.run.smithi071.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 321 kB/s | 75 kB 00:00 2024-05-08T21:07:36.976 INFO:teuthology.orchestra.run.smithi071.stdout:(41/146): gperftools-libs-2.7-9.el8.x86_64.rpm 465 kB/s | 306 kB 00:00 2024-05-08T21:07:37.075 INFO:teuthology.orchestra.run.smithi082.stdout:(30/146): ceph-mgr-diskprediction-local-18.2.2- 16 MB/s | 7.4 MB 00:00 2024-05-08T21:07:37.085 INFO:teuthology.orchestra.run.smithi071.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 326 kB/s | 44 kB 00:00 2024-05-08T21:07:37.110 INFO:teuthology.orchestra.run.smithi071.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2024-05-08T21:07:37.134 INFO:teuthology.orchestra.run.smithi082.stdout:(31/146): ceph-mgr-modules-core-18.2.2-1322.gd2 4.1 MB/s | 245 kB 00:00 2024-05-08T21:07:37.185 INFO:teuthology.orchestra.run.smithi082.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 985 kB/s | 49 kB 00:00 2024-05-08T21:07:37.235 INFO:teuthology.orchestra.run.smithi082.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 327 kB/s | 16 kB 00:00 2024-05-08T21:07:37.235 INFO:teuthology.orchestra.run.smithi071.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 220 kB/s | 33 kB 00:00 2024-05-08T21:07:37.261 INFO:teuthology.orchestra.run.smithi071.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 109 kB/s | 16 kB 00:00 2024-05-08T21:07:37.318 INFO:teuthology.orchestra.run.smithi082.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 3.1 MB/s | 263 kB 00:00 2024-05-08T21:07:37.369 INFO:teuthology.orchestra.run.smithi082.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 3.3 MB/s | 170 kB 00:00 2024-05-08T21:07:37.418 INFO:teuthology.orchestra.run.smithi082.stdout:(36/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 364 kB/s | 18 kB 00:00 2024-05-08T21:07:37.428 INFO:teuthology.orchestra.run.smithi071.stdout:(46/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.3 MB/s | 384 kB 00:00 2024-05-08T21:07:37.486 INFO:teuthology.orchestra.run.smithi071.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 689 kB/s | 173 kB 00:00 2024-05-08T21:07:37.511 INFO:teuthology.orchestra.run.smithi071.stdout:(48/146): python3-asyncssh-2.7.0-2.el8.noarch.r 791 kB/s | 476 kB 00:00 2024-05-08T21:07:37.553 INFO:teuthology.orchestra.run.smithi071.stdout:(49/146): python3-google-auth-1.1.1-10.el8.noar 649 kB/s | 81 kB 00:00 2024-05-08T21:07:37.595 INFO:teuthology.orchestra.run.smithi071.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2024-05-08T21:07:37.629 INFO:teuthology.orchestra.run.smithi071.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 150 kB/s | 18 kB 00:00 2024-05-08T21:07:37.729 INFO:teuthology.orchestra.run.smithi071.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 364 kB/s | 49 kB 00:00 2024-05-08T21:07:37.754 INFO:teuthology.orchestra.run.smithi071.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 468 kB/s | 59 kB 00:00 2024-05-08T21:07:37.797 INFO:teuthology.orchestra.run.smithi071.stdout:(54/146): python3-kubernetes-11.0.0-6.el8.noarc 4.8 MB/s | 1.2 MB 00:00 2024-05-08T21:07:37.863 INFO:teuthology.orchestra.run.smithi071.stdout:(55/146): python3-natsort-7.1.1-2.el8.noarch.rp 450 kB/s | 60 kB 00:00 2024-05-08T21:07:37.914 INFO:teuthology.orchestra.run.smithi071.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2024-05-08T21:07:37.939 INFO:teuthology.orchestra.run.smithi071.stdout:(57/146): python3-pecan-1.3.2-9.el8.noarch.rpm 1.5 MB/s | 283 kB 00:00 2024-05-08T21:07:37.973 INFO:teuthology.orchestra.run.smithi071.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 309 kB/s | 34 kB 00:00 2024-05-08T21:07:38.040 INFO:teuthology.orchestra.run.smithi071.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2024-05-08T21:07:38.065 INFO:teuthology.orchestra.run.smithi071.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 512 kB/s | 64 kB 00:00 2024-05-08T21:07:38.090 INFO:teuthology.orchestra.run.smithi071.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 160 kB/s | 19 kB 00:00 2024-05-08T21:07:38.149 INFO:teuthology.orchestra.run.smithi071.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2024-05-08T21:07:38.191 INFO:teuthology.orchestra.run.smithi071.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 233 kB/s | 29 kB 00:00 2024-05-08T21:07:38.216 INFO:teuthology.orchestra.run.smithi071.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 219 kB/s | 27 kB 00:00 2024-05-08T21:07:38.244 INFO:teuthology.orchestra.run.smithi082.stdout:(37/146): gperftools-libs-2.7-9.el8.x86_64.rpm 371 kB/s | 306 kB 00:00 2024-05-08T21:07:38.258 INFO:teuthology.orchestra.run.smithi071.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2024-05-08T21:07:38.325 INFO:teuthology.orchestra.run.smithi071.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 564 kB/s | 61 kB 00:00 2024-05-08T21:07:38.350 INFO:teuthology.orchestra.run.smithi071.stdout:(67/146): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2024-05-08T21:07:38.375 INFO:teuthology.orchestra.run.smithi071.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 729 kB/s | 85 kB 00:00 2024-05-08T21:07:38.401 INFO:teuthology.orchestra.run.smithi071.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2024-05-08T21:07:38.426 INFO:teuthology.orchestra.run.smithi071.stdout:(70/146): python3-zc-lockfile-2.0-2.el8.noarch. 228 kB/s | 23 kB 00:00 2024-05-08T21:07:38.528 INFO:teuthology.orchestra.run.smithi082.stdout:(38/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 3.4 MB/s | 18 MB 00:05 2024-05-08T21:07:38.553 INFO:teuthology.orchestra.run.smithi082.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 191 kB/s | 59 kB 00:00 2024-05-08T21:07:38.576 INFO:teuthology.orchestra.run.smithi071.stdout:(71/146): python3-jsonpointer-1.10-11.el8.noarc 138 kB/s | 20 kB 00:00 2024-05-08T21:07:38.601 INFO:teuthology.orchestra.run.smithi071.stdout:(72/146): python3-packaging-16.8-9.el8.noarch.r 241 kB/s | 48 kB 00:00 2024-05-08T21:07:38.686 INFO:teuthology.orchestra.run.smithi071.stdout:(73/146): thrift-0.13.0-2.el8.x86_64.rpm 5.2 MB/s | 1.7 MB 00:00 2024-05-08T21:07:38.718 INFO:teuthology.orchestra.run.smithi071.stdout:(74/146): python3-prettytable-0.7.2-14.el8.noar 309 kB/s | 44 kB 00:00 2024-05-08T21:07:38.819 INFO:teuthology.orchestra.run.smithi071.stdout:(75/146): python3-pyasn1-modules-0.3.7-6.el8.no 507 kB/s | 110 kB 00:00 2024-05-08T21:07:38.869 INFO:teuthology.orchestra.run.smithi071.stdout:(76/146): python3-jsonpatch-1.21-2.el8.noarch.r 146 kB/s | 27 kB 00:00 2024-05-08T21:07:38.903 INFO:teuthology.orchestra.run.smithi071.stdout:(77/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 710 kB/s | 126 kB 00:00 2024-05-08T21:07:38.961 INFO:teuthology.orchestra.run.smithi071.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 3.1 MB/s | 457 kB 00:00 2024-05-08T21:07:39.003 INFO:teuthology.orchestra.run.smithi071.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 3.1 MB/s | 320 kB 00:00 2024-05-08T21:07:39.011 INFO:teuthology.orchestra.run.smithi082.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 155 kB/s | 75 kB 00:00 2024-05-08T21:07:39.045 INFO:teuthology.orchestra.run.smithi071.stdout:(80/146): bison-3.0.4-10.el8.x86_64.rpm 3.8 MB/s | 688 kB 00:00 2024-05-08T21:07:39.129 INFO:teuthology.orchestra.run.smithi071.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 4.1 MB/s | 709 kB 00:00 2024-05-08T21:07:39.136 INFO:teuthology.orchestra.run.smithi082.stdout:(41/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 349 kB/s | 44 kB 00:00 2024-05-08T21:07:39.154 INFO:teuthology.orchestra.run.smithi071.stdout:(82/146): python3-markupsafe-0.23-19.el8.x86_64 258 kB/s | 39 kB 00:00 2024-05-08T21:07:39.171 INFO:teuthology.orchestra.run.smithi071.stdout:(83/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2024-05-08T21:07:39.254 INFO:teuthology.orchestra.run.smithi082.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 679 kB/s | 476 kB 00:00 2024-05-08T21:07:39.305 INFO:teuthology.orchestra.run.smithi071.stdout:(84/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 688 kB/s | 103 kB 00:00 2024-05-08T21:07:39.321 INFO:teuthology.orchestra.run.smithi071.stdout:(85/146): uuid-1.6.2-43.el8.x86_64.rpm 332 kB/s | 64 kB 00:00 2024-05-08T21:07:39.404 INFO:teuthology.orchestra.run.smithi082.stdout:(43/146): python3-cachetools-3.1.1-4.el8.noarch 220 kB/s | 33 kB 00:00 2024-05-08T21:07:39.430 INFO:teuthology.orchestra.run.smithi071.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2024-05-08T21:07:39.472 INFO:teuthology.orchestra.run.smithi071.stdout:(87/146): python3-jinja2-2.10.1-3.el8.noarch.rp 3.1 MB/s | 538 kB 00:00 2024-05-08T21:07:39.521 INFO:teuthology.orchestra.run.smithi082.stdout:(44/146): python3-certifi-2018.10.15-7.el8.noar 139 kB/s | 16 kB 00:00 2024-05-08T21:07:39.530 INFO:teuthology.orchestra.run.smithi071.stdout:(88/146): socat-1.7.4.1-1.el8.x86_64.rpm 3.2 MB/s | 323 kB 00:00 2024-05-08T21:07:39.546 INFO:teuthology.orchestra.run.smithi082.stdout:(45/146): python3-beautifulsoup4-4.6.3-2.el8.1. 453 kB/s | 185 kB 00:00 2024-05-08T21:07:39.581 INFO:teuthology.orchestra.run.smithi071.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.0 MB/s | 225 kB 00:00 2024-05-08T21:07:39.681 INFO:teuthology.orchestra.run.smithi071.stdout:(90/146): python3-numpy-1.14.3-10.el8.x86_64.rp 10 MB/s | 3.7 MB 00:00 2024-05-08T21:07:39.706 INFO:teuthology.orchestra.run.smithi071.stdout:(91/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.0 MB/s | 354 kB 00:00 2024-05-08T21:07:39.773 INFO:teuthology.orchestra.run.smithi071.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2024-05-08T21:07:39.813 INFO:teuthology.orchestra.run.smithi082.stdout:(46/146): python3-cheroot-8.5.2-1.el8.noarch.rp 591 kB/s | 173 kB 00:00 2024-05-08T21:07:39.848 INFO:teuthology.orchestra.run.smithi071.stdout:(93/146): openblas-threads-0.3.15-6.el8.x86_64. 19 MB/s | 5.2 MB 00:00 2024-05-08T21:07:39.957 INFO:teuthology.orchestra.run.smithi071.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 3.8 MB/s | 714 kB 00:00 2024-05-08T21:07:40.021 INFO:teuthology.orchestra.run.smithi082.stdout:(47/146): python3-google-auth-1.1.1-10.el8.noar 390 kB/s | 81 kB 00:00 2024-05-08T21:07:40.032 INFO:teuthology.orchestra.run.smithi071.stdout:(95/146): openblas-0.3.15-6.el8.x86_64.rpm 16 MB/s | 5.0 MB 00:00 2024-05-08T21:07:40.057 INFO:teuthology.orchestra.run.smithi071.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 948 kB/s | 198 kB 00:00 2024-05-08T21:07:40.099 INFO:teuthology.orchestra.run.smithi071.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2024-05-08T21:07:40.138 INFO:teuthology.orchestra.run.smithi082.stdout:(48/146): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2024-05-08T21:07:40.164 INFO:teuthology.orchestra.run.smithi082.stdout:(49/146): python3-cherrypy-18.4.0-1.el8.noarch. 623 kB/s | 384 kB 00:00 2024-05-08T21:07:40.175 INFO:teuthology.orchestra.run.smithi071.stdout:(98/146): python3-pytz-2017.2-11.el8.noarch.rpm 382 kB/s | 54 kB 00:00 2024-05-08T21:07:40.264 INFO:teuthology.orchestra.run.smithi082.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2024-05-08T21:07:40.283 INFO:teuthology.orchestra.run.smithi071.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-05-08T21:07:40.333 INFO:teuthology.orchestra.run.smithi071.stdout:(100/146): postgresql-10.23-4.module+el8.9.0+21 9.5 MB/s | 1.5 MB 00:00 2024-05-08T21:07:40.381 INFO:teuthology.orchestra.run.smithi082.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 418 kB/s | 49 kB 00:00 2024-05-08T21:07:40.523 INFO:teuthology.orchestra.run.smithi082.stdout:(52/146): python3-more-itertools-7.2.0-3.el8.no 414 kB/s | 59 kB 00:00 2024-05-08T21:07:40.534 INFO:teuthology.orchestra.run.smithi071.stdout:(101/146): python3-scipy-1.0.0-21.module+el8.9. 29 MB/s | 14 MB 00:00 2024-05-08T21:07:40.567 INFO:teuthology.orchestra.run.smithi071.stdout:(102/146): postgresql-contrib-10.23-4.module+el 2.8 MB/s | 812 kB 00:00 2024-05-08T21:07:40.657 INFO:teuthology.orchestra.run.smithi082.stdout:(53/146): python3-natsort-7.1.1-2.el8.noarch.rp 452 kB/s | 60 kB 00:00 2024-05-08T21:07:40.668 INFO:teuthology.orchestra.run.smithi071.stdout:(103/146): postgresql-server-10.23-4.module+el8 15 MB/s | 5.1 MB 00:00 2024-05-08T21:07:40.684 INFO:teuthology.orchestra.run.smithi071.stdout:(104/146): libverto-devel-0.3.0-5.el8.x86_64.rp 119 kB/s | 18 kB 00:00 2024-05-08T21:07:40.960 INFO:teuthology.orchestra.run.smithi071.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 109 kB/s | 43 kB 00:00 2024-05-08T21:07:41.019 INFO:teuthology.orchestra.run.smithi071.stdout:(106/146): python3-pycparser-2.14-14.el8.noarch 327 kB/s | 109 kB 00:00 2024-05-08T21:07:41.043 INFO:teuthology.orchestra.run.smithi071.stdout:(107/146): python3-oauthlib-2.1.0-1.el8.noarch. 412 kB/s | 155 kB 00:00 2024-05-08T21:07:41.048 INFO:teuthology.orchestra.run.smithi082.stdout:(54/146): python3-pecan-1.3.2-9.el8.noarch.rpm 722 kB/s | 283 kB 00:00 2024-05-08T21:07:41.077 INFO:teuthology.orchestra.run.smithi071.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 365 kB/s | 43 kB 00:00 2024-05-08T21:07:41.165 INFO:teuthology.orchestra.run.smithi082.stdout:(55/146): python3-portend-2.6-1.el8.noarch.rpm 134 kB/s | 16 kB 00:00 2024-05-08T21:07:41.169 INFO:teuthology.orchestra.run.smithi071.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 124 kB/s | 19 kB 00:00 2024-05-08T21:07:41.186 INFO:teuthology.orchestra.run.smithi071.stdout:(110/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 308 kB/s | 44 kB 00:00 2024-05-08T21:07:41.299 INFO:teuthology.orchestra.run.smithi082.stdout:(56/146): python3-repoze-lru-0.7-6.el8.noarch.r 252 kB/s | 34 kB 00:00 2024-05-08T21:07:41.303 INFO:teuthology.orchestra.run.smithi071.stdout:(111/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2024-05-08T21:07:41.363 INFO:teuthology.orchestra.run.smithi071.stdout:(112/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.1 MB/s | 605 kB 00:00 2024-05-08T21:07:41.387 INFO:teuthology.orchestra.run.smithi071.stdout:(113/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.0 MB/s | 229 kB 00:00 2024-05-08T21:07:41.479 INFO:teuthology.orchestra.run.smithi071.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.1 MB/s | 200 kB 00:00 2024-05-08T21:07:41.529 INFO:teuthology.orchestra.run.smithi071.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 612 kB/s | 87 kB 00:00 2024-05-08T21:07:41.588 INFO:teuthology.orchestra.run.smithi071.stdout:(116/146): keyutils-libs-devel-1.5.10-9.el8.x86 214 kB/s | 48 kB 00:00 2024-05-08T21:07:41.649 INFO:teuthology.orchestra.run.smithi082.stdout:(57/146): python3-routes-2.4.1-12.el8.noarch.rp 561 kB/s | 196 kB 00:00 2024-05-08T21:07:41.655 INFO:teuthology.orchestra.run.smithi071.stdout:(117/146): libcom_err-devel-1.45.6-4.el8.x86_64 308 kB/s | 39 kB 00:00 2024-05-08T21:07:41.680 INFO:teuthology.orchestra.run.smithi071.stdout:(118/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 287 kB/s | 58 kB 00:00 2024-05-08T21:07:41.774 INFO:teuthology.orchestra.run.smithi082.stdout:(58/146): python3-rsa-4.9-2.el8.noarch.rpm 514 kB/s | 64 kB 00:00 2024-05-08T21:07:41.789 INFO:teuthology.orchestra.run.smithi071.stdout:(119/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.7 MB/s | 645 kB 00:00 2024-05-08T21:07:41.805 INFO:teuthology.orchestra.run.smithi071.stdout:(120/146): fuse-2.9.7-17.el8.x86_64.rpm 382 kB/s | 83 kB 00:00 2024-05-08T21:07:41.839 INFO:teuthology.orchestra.run.smithi071.stdout:(121/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.1 MB/s | 172 kB 00:00 2024-05-08T21:07:41.881 INFO:teuthology.orchestra.run.smithi071.stdout:(122/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 965 kB/s | 73 kB 00:00 2024-05-08T21:07:41.883 INFO:teuthology.orchestra.run.smithi082.stdout:(59/146): python3-simplegeneric-0.8.1-17.el8.no 173 kB/s | 19 kB 00:00 2024-05-08T21:07:41.917 INFO:teuthology.orchestra.run.smithi082.stdout:(60/146): python3-kubernetes-11.0.0-6.el8.noarc 681 kB/s | 1.2 MB 00:01 2024-05-08T21:07:41.956 INFO:teuthology.orchestra.run.smithi071.stdout:(123/146): elfutils-libelf-devel-0.189-3.el8.x8 528 kB/s | 62 kB 00:00 2024-05-08T21:07:41.990 INFO:teuthology.orchestra.run.smithi071.stdout:(124/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 5.1 MB/s | 562 kB 00:00 2024-05-08T21:07:42.000 INFO:teuthology.orchestra.run.smithi082.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 205 kB/s | 24 kB 00:00 2024-05-08T21:07:42.015 INFO:teuthology.orchestra.run.smithi071.stdout:(125/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.0 MB/s | 238 kB 00:00 2024-05-08T21:07:42.091 INFO:teuthology.orchestra.run.smithi071.stdout:(126/146): openssl-devel-1.1.1k-12.el8_9.x86_64 17 MB/s | 2.3 MB 00:00 2024-05-08T21:07:42.109 INFO:teuthology.orchestra.run.smithi082.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 152 kB/s | 29 kB 00:00 2024-05-08T21:07:42.124 INFO:teuthology.orchestra.run.smithi071.stdout:(127/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 353 kB/s | 47 kB 00:00 2024-05-08T21:07:42.149 INFO:teuthology.orchestra.run.smithi071.stdout:(128/146): python3-cryptography-3.2.1-7.el8_9.x 4.1 MB/s | 559 kB 00:00 2024-05-08T21:07:42.266 INFO:teuthology.orchestra.run.smithi071.stdout:(129/146): NetworkManager-initscripts-updown-1. 824 kB/s | 144 kB 00:00 2024-05-08T21:07:42.375 INFO:teuthology.orchestra.run.smithi071.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 398 kB/s | 100 kB 00:00 2024-05-08T21:07:42.376 INFO:teuthology.orchestra.run.smithi082.stdout:(63/146): python3-trustme-0.6.0-4.el8.noarch.rp 73 kB/s | 27 kB 00:00 2024-05-08T21:07:42.558 INFO:teuthology.orchestra.run.smithi071.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.2 MB/s | 232 kB 00:00 2024-05-08T21:07:42.618 INFO:teuthology.orchestra.run.smithi082.stdout:(64/146): python3-waitress-1.4.3-1.el8.noarch.r 502 kB/s | 255 kB 00:00 2024-05-08T21:07:42.683 INFO:teuthology.orchestra.run.smithi071.stdout:(132/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 8.6 MB/s | 3.6 MB 00:00 2024-05-08T21:07:42.709 INFO:teuthology.orchestra.run.smithi071.stdout:(133/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.0 MB/s | 303 kB 00:00 2024-05-08T21:07:42.735 INFO:teuthology.orchestra.run.smithi082.stdout:(65/146): python3-websocket-client-0.56.0-5.el8 525 kB/s | 61 kB 00:00 2024-05-08T21:07:42.784 INFO:teuthology.orchestra.run.smithi071.stdout:(134/146): librados2-18.2.2-1322.gd21e4f96.el8. 5.8 MB/s | 3.7 MB 00:00 2024-05-08T21:07:42.826 INFO:teuthology.orchestra.run.smithi071.stdout:(135/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.9 MB/s | 854 kB 00:00 2024-05-08T21:07:42.868 INFO:teuthology.orchestra.run.smithi071.stdout:(136/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.4 MB/s | 711 kB 00:00 2024-05-08T21:07:42.935 INFO:teuthology.orchestra.run.smithi071.stdout:(137/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 9.8 MB/s | 1.5 MB 00:00 2024-05-08T21:07:42.960 INFO:teuthology.orchestra.run.smithi071.stdout:(138/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.4 MB/s | 333 kB 00:00 2024-05-08T21:07:43.002 INFO:teuthology.orchestra.run.smithi082.stdout:(66/146): python3-webtest-2.0.33-1.el8.noarch.r 320 kB/s | 85 kB 00:00 2024-05-08T21:07:43.054 INFO:teuthology.orchestra.run.smithi071.stdout:(139/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.8 MB/s | 1.1 MB 00:00 2024-05-08T21:07:43.102 INFO:teuthology.orchestra.run.smithi071.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 254 kB 00:00 2024-05-08T21:07:43.141 INFO:teuthology.orchestra.run.smithi082.stdout:(67/146): python3-zc-lockfile-2.0-2.el8.noarch. 166 kB/s | 23 kB 00:00 2024-05-08T21:07:43.159 INFO:teuthology.orchestra.run.smithi071.stdout:(141/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 985 kB/s | 221 kB 00:00 2024-05-08T21:07:43.195 INFO:teuthology.orchestra.run.smithi071.stdout:(142/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.6 MB/s | 237 kB 00:00 2024-05-08T21:07:43.261 INFO:teuthology.orchestra.run.smithi071.stdout:(143/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.1 MB/s | 180 kB 00:00 2024-05-08T21:07:43.345 INFO:teuthology.orchestra.run.smithi071.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 534 kB/s | 99 kB 00:00 2024-05-08T21:07:43.415 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:07:43.416 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-05-08T21:07:43.429 INFO:teuthology.orchestra.run.smithi071.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 11 MB/s | 2.5 MB 00:00 2024-05-08T21:07:43.436 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-05-08T21:07:43.438 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-05-08T21:07:43.440 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-05-08T21:07:43.441 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-05-08T21:07:43.444 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-05-08T21:07:43.449 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-05-08T21:07:43.450 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-05-08T21:07:43.462 INFO:teuthology.orchestra.run.smithi071.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 513 kB/s | 103 kB 00:00 2024-05-08T21:07:43.465 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:07:43.466 INFO:teuthology.orchestra.run.smithi071.stdout:Total 14 MB/s | 215 MB 00:15 2024-05-08T21:07:43.698 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-05-08T21:07:43.714 INFO:teuthology.orchestra.run.smithi033.stdout:=================================================================================================================================== 2024-05-08T21:07:43.715 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-05-08T21:07:43.715 INFO:teuthology.orchestra.run.smithi033.stdout:=================================================================================================================================== 2024-05-08T21:07:43.715 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-05-08T21:07:43.715 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-15.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-05-08T21:07:43.715 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-05-08T21:07:43.715 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.4 k 2024-05-08T21:07:43.715 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 5.5 M 2024-05-08T21:07:43.715 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 860 k 2024-05-08T21:07:43.715 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 166 k 2024-05-08T21:07:43.715 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 1.6 M 2024-05-08T21:07:43.715 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 138 k 2024-05-08T21:07:43.715 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 3.5 M 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 7.4 M 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 49 k 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 13 M 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 47 M 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 263 k 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 170 k 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 261 k 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 18 k 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 31 k 2024-05-08T21:07:43.716 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 782 k 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 139 k 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 100 k 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 197 k 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 370 k 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 365 k 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 107 k 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 95 k 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.5 M 2024-05-08T21:07:43.717 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 181 k 2024-05-08T21:07:43.718 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-05-08T21:07:43.718 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-05-08T21:07:43.718 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-05-08T21:07:43.718 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-05-08T21:07:43.718 INFO:teuthology.orchestra.run.smithi033.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-05-08T21:07:43.718 INFO:teuthology.orchestra.run.smithi033.stdout: libblkid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 221 k 2024-05-08T21:07:43.718 INFO:teuthology.orchestra.run.smithi033.stdout: libfdisk x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 254 k 2024-05-08T21:07:43.718 INFO:teuthology.orchestra.run.smithi033.stdout: libmount x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 237 k 2024-05-08T21:07:43.718 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.7 M 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 3.6 M 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: libsmartcols x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 180 k 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 99 k 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-user x86_64 2.32.1-44.el8_9.1 rhel-8-for-x86_64-baseos-rpms 103 k 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 22 M 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 24 k 2024-05-08T21:07:43.719 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 2.3 M 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 245 k 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 4.8 M 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 18 M 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1322.gd21e4f96.el8 ceph-noarch 16 k 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 24 k 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 142 k 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-05-08T21:07:43.720 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 474 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 6.5 M 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-05-08T21:07:43.721 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 45 k 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:18.2.2-1322.gd21e4f96.el8 ceph 129 k 2024-05-08T21:07:43.722 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-05-08T21:07:43.723 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-05-08T21:07:43.724 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-05-08T21:07:43.725 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling module streams: 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql 10 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout:=================================================================================================================================== 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout:Install 130 Packages 2024-05-08T21:07:43.726 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 16 Packages 2024-05-08T21:07:43.727 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:07:43.727 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 215 M 2024-05-08T21:07:43.727 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-05-08T21:07:44.050 INFO:teuthology.orchestra.run.smithi033.stdout:(1/146): ceph-18.2.2-1322.gd21e4f96.el8.x86_64. 35 kB/s | 6.4 kB 00:00 2024-05-08T21:07:44.503 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-05-08T21:07:44.519 INFO:teuthology.orchestra.run.smithi082.stdout:(68/146): python3-webob-1.8.5-1.el8.1.noarch.rp 117 kB/s | 251 kB 00:02 2024-05-08T21:07:44.544 INFO:teuthology.orchestra.run.smithi082.stdout:(69/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2024-05-08T21:07:44.608 INFO:teuthology.orchestra.run.smithi033.stdout:(2/146): ceph-fuse-18.2.2-1322.gd21e4f96.el8.x8 1.5 MB/s | 860 kB 00:00 2024-05-08T21:07:44.678 INFO:teuthology.orchestra.run.smithi082.stdout:(70/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 134 kB/s | 18 kB 00:00 2024-05-08T21:07:44.792 INFO:teuthology.orchestra.run.smithi033.stdout:(3/146): ceph-immutable-object-cache-18.2.2-132 907 kB/s | 166 kB 00:00 2024-05-08T21:07:44.845 INFO:teuthology.orchestra.run.smithi082.stdout:(71/146): python3-jwt-1.6.1-2.el8.noarch.rpm 256 kB/s | 43 kB 00:00 2024-05-08T21:07:44.898 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-05-08T21:07:44.898 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-05-08T21:07:44.937 INFO:teuthology.orchestra.run.smithi082.stdout:(72/146): python3-oauthlib-2.1.0-1.el8.noarch.r 1.7 MB/s | 155 kB 00:00 2024-05-08T21:07:45.026 INFO:teuthology.orchestra.run.smithi033.stdout:(4/146): ceph-base-18.2.2-1322.gd21e4f96.el8.x8 4.8 MB/s | 5.5 MB 00:01 2024-05-08T21:07:45.079 INFO:teuthology.orchestra.run.smithi082.stdout:(73/146): python3-pycparser-2.14-14.el8.noarch. 771 kB/s | 109 kB 00:00 2024-05-08T21:07:45.262 INFO:teuthology.orchestra.run.smithi082.stdout:(74/146): python3-requests-oauthlib-1.0.0-1.el8 233 kB/s | 43 kB 00:00 2024-05-08T21:07:45.379 INFO:teuthology.orchestra.run.smithi082.stdout:(75/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 160 kB/s | 19 kB 00:00 2024-05-08T21:07:45.429 INFO:teuthology.orchestra.run.smithi082.stdout:(76/146): thrift-0.13.0-2.el8.x86_64.rpm 776 kB/s | 1.7 MB 00:02 2024-05-08T21:07:45.496 INFO:teuthology.orchestra.run.smithi082.stdout:(77/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 375 kB/s | 44 kB 00:00 2024-05-08T21:07:45.568 INFO:teuthology.orchestra.run.smithi033.stdout:(5/146): ceph-common-18.2.2-1322.gd21e4f96.el8. 13 MB/s | 22 MB 00:01 2024-05-08T21:07:45.609 INFO:teuthology.orchestra.run.smithi033.stdout:(6/146): ceph-mgr-18.2.2-1322.gd21e4f96.el8.x86 2.7 MB/s | 1.6 MB 00:00 2024-05-08T21:07:45.638 INFO:teuthology.orchestra.run.smithi082.stdout:(78/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.8 MB/s | 605 kB 00:00 2024-05-08T21:07:45.672 INFO:teuthology.orchestra.run.smithi082.stdout:(79/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.3 MB/s | 229 kB 00:00 2024-05-08T21:07:45.735 INFO:teuthology.orchestra.run.smithi033.stdout:(7/146): ceph-mds-18.2.2-1322.gd21e4f96.el8.x86 2.4 MB/s | 2.3 MB 00:00 2024-05-08T21:07:45.781 INFO:teuthology.orchestra.run.smithi082.stdout:(80/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2024-05-08T21:07:45.823 INFO:teuthology.orchestra.run.smithi082.stdout:(81/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.3 MB/s | 200 kB 00:00 2024-05-08T21:07:45.923 INFO:teuthology.orchestra.run.smithi082.stdout:(82/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 339 kB/s | 48 kB 00:00 2024-05-08T21:07:45.968 INFO:teuthology.orchestra.run.smithi033.stdout:(8/146): ceph-mon-18.2.2-1322.gd21e4f96.el8.x86 12 MB/s | 4.8 MB 00:00 2024-05-08T21:07:46.018 INFO:teuthology.orchestra.run.smithi033.stdout:(9/146): ceph-selinux-18.2.2-1322.gd21e4f96.el8 489 kB/s | 24 kB 00:00 2024-05-08T21:07:46.090 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-05-08T21:07:46.173 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-05-08T21:07:46.357 INFO:teuthology.orchestra.run.smithi082.stdout:(83/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 4.0 MB/s | 47 MB 00:11 2024-05-08T21:07:46.382 INFO:teuthology.orchestra.run.smithi082.stdout:(84/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 126 kB/s | 58 kB 00:00 2024-05-08T21:07:46.415 INFO:teuthology.orchestra.run.smithi082.stdout:(85/146): libsepol-devel-2.9-3.el8.x86_64.rpm 147 kB/s | 87 kB 00:00 2024-05-08T21:07:46.466 INFO:teuthology.orchestra.run.smithi082.stdout:(86/146): fuse-2.9.7-17.el8.x86_64.rpm 1.0 MB/s | 83 kB 00:00 2024-05-08T21:07:46.491 INFO:teuthology.orchestra.run.smithi082.stdout:(87/146): libcom_err-devel-1.45.6-4.el8.x86_64. 289 kB/s | 39 kB 00:00 2024-05-08T21:07:46.525 INFO:teuthology.orchestra.run.smithi082.stdout:(88/146): libquadmath-8.5.0-20.el8.x86_64.rpm 2.9 MB/s | 172 kB 00:00 2024-05-08T21:07:46.558 INFO:teuthology.orchestra.run.smithi082.stdout:(89/146): libgfortran-8.5.0-20.el8.x86_64.rpm 4.4 MB/s | 645 kB 00:00 2024-05-08T21:07:46.583 INFO:teuthology.orchestra.run.smithi082.stdout:(90/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 1.2 MB/s | 73 kB 00:00 2024-05-08T21:07:46.640 INFO:teuthology.orchestra.run.smithi082.stdout:(91/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.6 MB/s | 238 kB 00:00 2024-05-08T21:07:46.676 INFO:teuthology.orchestra.run.smithi082.stdout:(92/146): elfutils-libelf-devel-0.189-3.el8.x86 527 kB/s | 62 kB 00:00 2024-05-08T21:07:46.751 INFO:teuthology.orchestra.run.smithi082.stdout:(93/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 3.3 MB/s | 562 kB 00:00 2024-05-08T21:07:46.793 INFO:teuthology.orchestra.run.smithi082.stdout:(94/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 403 kB/s | 47 kB 00:00 2024-05-08T21:07:46.851 INFO:teuthology.orchestra.run.smithi082.stdout:(95/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-05-08T21:07:46.885 INFO:teuthology.orchestra.run.smithi082.stdout:(96/146): python3-cryptography-3.2.1-7.el8_9.x8 4.1 MB/s | 559 kB 00:00 2024-05-08T21:07:46.944 INFO:teuthology.orchestra.run.smithi082.stdout:(97/146): libuuid-devel-2.32.1-44.el8_9.1.x86_6 1.1 MB/s | 100 kB 00:00 2024-05-08T21:07:46.977 INFO:teuthology.orchestra.run.smithi082.stdout:(98/146): NetworkManager-initscripts-updown-1.4 784 kB/s | 144 kB 00:00 2024-05-08T21:07:47.002 INFO:teuthology.orchestra.run.smithi082.stdout:(99/146): python3-packaging-16.8-9.el8.noarch.r 411 kB/s | 48 kB 00:00 2024-05-08T21:07:47.036 INFO:teuthology.orchestra.run.smithi082.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 217 kB/s | 20 kB 00:00 2024-05-08T21:07:47.061 INFO:teuthology.orchestra.run.smithi082.stdout:(101/146): python3-prettytable-0.7.2-14.el8.noa 525 kB/s | 44 kB 00:00 2024-05-08T21:07:47.078 INFO:teuthology.orchestra.run.smithi082.stdout:(102/146): python3-pyasn1-modules-0.3.7-6.el8.n 1.4 MB/s | 110 kB 00:00 2024-05-08T21:07:47.128 INFO:teuthology.orchestra.run.smithi082.stdout:(103/146): python3-jsonpatch-1.21-2.el8.noarch. 288 kB/s | 27 kB 00:00 2024-05-08T21:07:47.170 INFO:teuthology.orchestra.run.smithi082.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.1 MB/s | 126 kB 00:00 2024-05-08T21:07:47.262 INFO:teuthology.orchestra.run.smithi082.stdout:(105/146): python3-werkzeug-0.12.2-4.el8.noarch 2.4 MB/s | 457 kB 00:00 2024-05-08T21:07:47.296 INFO:teuthology.orchestra.run.smithi082.stdout:(106/146): flex-2.6.1-9.el8.x86_64.rpm 2.5 MB/s | 320 kB 00:00 2024-05-08T21:07:47.338 INFO:teuthology.orchestra.run.smithi082.stdout:(107/146): bison-3.0.4-10.el8.x86_64.rpm 3.2 MB/s | 688 kB 00:00 2024-05-08T21:07:47.396 INFO:teuthology.orchestra.run.smithi082.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2024-05-08T21:07:47.438 INFO:teuthology.orchestra.run.smithi082.stdout:(109/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.5 MB/s | 259 kB 00:00 2024-05-08T21:07:47.480 INFO:teuthology.orchestra.run.smithi082.stdout:(110/146): python3-markupsafe-0.23-19.el8.x86_6 210 kB/s | 39 kB 00:00 2024-05-08T21:07:47.530 INFO:teuthology.orchestra.run.smithi082.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 477 kB/s | 64 kB 00:00 2024-05-08T21:07:47.589 INFO:teuthology.orchestra.run.smithi082.stdout:(112/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 689 kB/s | 103 kB 00:00 2024-05-08T21:07:47.790 INFO:teuthology.orchestra.run.smithi082.stdout:(113/146): python3-babel-2.5.1-7.el8.noarch.rpm 16 MB/s | 4.8 MB 00:00 2024-05-08T21:07:47.856 INFO:teuthology.orchestra.run.smithi082.stdout:(114/146): python3-jinja2-2.10.1-3.el8.noarch.r 1.6 MB/s | 538 kB 00:00 2024-05-08T21:07:47.944 INFO:teuthology.orchestra.run.smithi033.stdout:(10/146): ceph-radosgw-18.2.2-1322.gd21e4f96.el 5.9 MB/s | 13 MB 00:02 2024-05-08T21:07:47.973 INFO:teuthology.orchestra.run.smithi082.stdout:(115/146): python3-numpy-1.14.3-10.el8.x86_64.r 9.7 MB/s | 3.7 MB 00:00 2024-05-08T21:07:47.998 INFO:teuthology.orchestra.run.smithi082.stdout:(116/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.5 MB/s | 323 kB 00:00 2024-05-08T21:07:48.024 INFO:teuthology.orchestra.run.smithi082.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.3 MB/s | 225 kB 00:00 2024-05-08T21:07:48.060 INFO:teuthology.orchestra.run.smithi033.stdout:(11/146): cephfs-mirror-18.2.2-1322.gd21e4f96.e 2.2 MB/s | 261 kB 00:00 2024-05-08T21:07:48.152 INFO:teuthology.orchestra.run.smithi033.stdout:(12/146): libcephfs-devel-18.2.2-1322.gd21e4f96 338 kB/s | 31 kB 00:00 2024-05-08T21:07:48.174 INFO:teuthology.orchestra.run.smithi082.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 1.7 MB/s | 354 kB 00:00 2024-05-08T21:07:48.257 INFO:teuthology.orchestra.run.smithi082.stdout:(119/146): openblas-threads-0.3.15-6.el8.x86_64 20 MB/s | 5.2 MB 00:00 2024-05-08T21:07:48.283 INFO:teuthology.orchestra.run.smithi082.stdout:(120/146): python3-mako-1.0.6-14.el8.noarch.rpm 607 kB/s | 157 kB 00:00 2024-05-08T21:07:48.328 INFO:teuthology.orchestra.run.smithi033.stdout:(13/146): libcephfs2-18.2.2-1322.gd21e4f96.el8. 4.4 MB/s | 782 kB 00:00 2024-05-08T21:07:48.420 INFO:teuthology.orchestra.run.smithi033.stdout:(14/146): libcephsqlite-18.2.2-1322.gd21e4f96.e 1.5 MB/s | 142 kB 00:00 2024-05-08T21:07:48.425 INFO:teuthology.orchestra.run.smithi082.stdout:(121/146): openblas-0.3.15-6.el8.x86_64.rpm 20 MB/s | 5.0 MB 00:00 2024-05-08T21:07:48.450 INFO:teuthology.orchestra.run.smithi082.stdout:(122/146): libpq-13.11-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2024-05-08T21:07:48.483 INFO:teuthology.orchestra.run.smithi082.stdout:(123/146): automake-1.16.1-8.el8.noarch.rpm 3.1 MB/s | 714 kB 00:00 2024-05-08T21:07:48.550 INFO:teuthology.orchestra.run.smithi082.stdout:(124/146): jq-1.6-7.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2024-05-08T21:07:48.592 INFO:teuthology.orchestra.run.smithi082.stdout:(125/146): python3-pytz-2017.2-11.el8.noarch.rp 382 kB/s | 54 kB 00:00 2024-05-08T21:07:48.618 INFO:teuthology.orchestra.run.smithi082.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-05-08T21:07:48.645 INFO:teuthology.orchestra.run.smithi033.stdout:(15/146): ceph-osd-18.2.2-1322.gd21e4f96.el8.x8 6.1 MB/s | 18 MB 00:03 2024-05-08T21:07:48.670 INFO:teuthology.orchestra.run.smithi033.stdout:(16/146): librados-devel-18.2.2-1322.gd21e4f96. 556 kB/s | 139 kB 00:00 2024-05-08T21:07:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 11 MB/s | 1.5 MB 00:00 2024-05-08T21:07:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:(128/146): postgresql-contrib-10.23-4.module+el 4.3 MB/s | 812 kB 00:00 2024-05-08T21:07:48.803 INFO:teuthology.orchestra.run.smithi033.stdout:(17/146): libradosstriper1-18.2.2-1322.gd21e4f9 2.9 MB/s | 474 kB 00:00 2024-05-08T21:07:48.904 INFO:teuthology.orchestra.run.smithi033.stdout:(18/146): python3-ceph-argparse-18.2.2-1322.gd2 447 kB/s | 45 kB 00:00 2024-05-08T21:07:49.010 INFO:teuthology.orchestra.run.smithi082.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 26 MB/s | 14 MB 00:00 2024-05-08T21:07:49.202 INFO:teuthology.orchestra.run.smithi082.stdout:(130/146): postgresql-server-10.23-4.module+el8 11 MB/s | 5.1 MB 00:00 2024-05-08T21:07:49.344 INFO:teuthology.orchestra.run.smithi082.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.6 MB/s | 232 kB 00:00 2024-05-08T21:07:49.362 INFO:teuthology.orchestra.run.smithi033.stdout:(19/146): ceph-test-18.2.2-1322.gd21e4f96.el8.x 14 MB/s | 47 MB 00:03 2024-05-08T21:07:49.388 INFO:teuthology.orchestra.run.smithi033.stdout:(20/146): python3-ceph-common-18.2.2-1322.gd21e 266 kB/s | 129 kB 00:00 2024-05-08T21:07:49.438 INFO:teuthology.orchestra.run.smithi033.stdout:(21/146): python3-cephfs-18.2.2-1322.gd21e4f96. 2.6 MB/s | 197 kB 00:00 2024-05-08T21:07:49.486 INFO:teuthology.orchestra.run.smithi082.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.1 MB/s | 303 kB 00:00 2024-05-08T21:07:49.530 INFO:teuthology.orchestra.run.smithi033.stdout:(22/146): python3-rbd-18.2.2-1322.gd21e4f96.el8 3.9 MB/s | 365 kB 00:00 2024-05-08T21:07:49.614 INFO:teuthology.orchestra.run.smithi033.stdout:(23/146): python3-rados-18.2.2-1322.gd21e4f96.e 1.6 MB/s | 370 kB 00:00 2024-05-08T21:07:49.639 INFO:teuthology.orchestra.run.smithi033.stdout:(24/146): python3-rgw-18.2.2-1322.gd21e4f96.el8 982 kB/s | 107 kB 00:00 2024-05-08T21:07:49.669 INFO:teuthology.orchestra.run.smithi082.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.6 MB/s | 854 kB 00:00 2024-05-08T21:07:49.723 INFO:teuthology.orchestra.run.smithi033.stdout:(25/146): librgw2-18.2.2-1322.gd21e4f96.el8.x86 6.2 MB/s | 6.5 MB 00:01 2024-05-08T21:07:49.748 INFO:teuthology.orchestra.run.smithi033.stdout:(26/146): rbd-fuse-18.2.2-1322.gd21e4f96.el8.x8 708 kB/s | 95 kB 00:00 2024-05-08T21:07:49.786 INFO:teuthology.orchestra.run.smithi082.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 5.9 MB/s | 711 kB 00:00 2024-05-08T21:07:49.878 INFO:teuthology.orchestra.run.smithi033.stdout:(27/146): rbd-nbd-18.2.2-1322.gd21e4f96.el8.x86 1.1 MB/s | 181 kB 00:00 2024-05-08T21:07:49.911 INFO:teuthology.orchestra.run.smithi082.stdout:(135/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 4.0 MB/s | 3.6 MB 00:00 2024-05-08T21:07:49.962 INFO:teuthology.orchestra.run.smithi082.stdout:(136/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.4 MB/s | 1.5 MB 00:00 2024-05-08T21:07:50.007 INFO:teuthology.orchestra.run.smithi033.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1322.gd21e4f9 1.1 MB/s | 138 kB 00:00 2024-05-08T21:07:50.071 INFO:teuthology.orchestra.run.smithi082.stdout:(137/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.1 MB/s | 333 kB 00:00 2024-05-08T21:07:50.157 INFO:teuthology.orchestra.run.smithi033.stdout:(29/146): rbd-mirror-18.2.2-1322.gd21e4f96.el8. 6.7 MB/s | 3.5 MB 00:00 2024-05-08T21:07:50.171 INFO:teuthology.orchestra.run.smithi082.stdout:(138/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.1 MB/s | 1.1 MB 00:00 2024-05-08T21:07:50.213 INFO:teuthology.orchestra.run.smithi082.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.5 MB/s | 221 kB 00:00 2024-05-08T21:07:50.215 INFO:teuthology.orchestra.run.smithi033.stdout:(30/146): ceph-grafana-dashboards-18.2.2-1322.g 52 kB/s | 24 kB 00:00 2024-05-08T21:07:50.280 INFO:teuthology.orchestra.run.smithi082.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 2.3 MB/s | 254 kB 00:00 2024-05-08T21:07:50.363 INFO:teuthology.orchestra.run.smithi082.stdout:(141/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.5 MB/s | 237 kB 00:00 2024-05-08T21:07:50.388 INFO:teuthology.orchestra.run.smithi082.stdout:(142/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.6 MB/s | 180 kB 00:00 2024-05-08T21:07:50.531 INFO:teuthology.orchestra.run.smithi082.stdout:(143/146): librados2-18.2.2-1322.gd21e4f96.el8. 2.1 MB/s | 3.7 MB 00:01 2024-05-08T21:07:50.541 INFO:teuthology.orchestra.run.smithi033.stdout:(31/146): ceph-mgr-dashboard-18.2.2-1322.gd21e4 6.6 MB/s | 3.5 MB 00:00 2024-05-08T21:07:50.556 INFO:teuthology.orchestra.run.smithi082.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 515 kB/s | 99 kB 00:00 2024-05-08T21:07:50.623 INFO:teuthology.orchestra.run.smithi082.stdout:(145/146): util-linux-2.32.1-44.el8_9.1.x86_64. 11 MB/s | 2.5 MB 00:00 2024-05-08T21:07:50.658 INFO:teuthology.orchestra.run.smithi033.stdout:(32/146): ceph-mgr-rook-18.2.2-1322.gd21e4f96.e 422 kB/s | 49 kB 00:00 2024-05-08T21:07:50.664 INFO:teuthology.orchestra.run.smithi082.stdout:(146/146): util-linux-user-2.32.1-44.el8_9.1.x8 770 kB/s | 103 kB 00:00 2024-05-08T21:07:50.667 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:07:50.667 INFO:teuthology.orchestra.run.smithi082.stdout:Total 10 MB/s | 215 MB 00:21 2024-05-08T21:07:50.742 INFO:teuthology.orchestra.run.smithi033.stdout:(33/146): ceph-prometheus-alerts-18.2.2-1322.gd 196 kB/s | 16 kB 00:00 2024-05-08T21:07:50.867 INFO:teuthology.orchestra.run.smithi033.stdout:(34/146): ceph-volume-18.2.2-1322.gd21e4f96.el8 2.1 MB/s | 263 kB 00:00 2024-05-08T21:07:50.975 INFO:teuthology.orchestra.run.smithi033.stdout:(35/146): cephadm-18.2.2-1322.gd21e4f96.el8.noa 1.5 MB/s | 170 kB 00:00 2024-05-08T21:07:51.025 INFO:teuthology.orchestra.run.smithi033.stdout:(36/146): ceph-mgr-modules-core-18.2.2-1322.gd2 303 kB/s | 245 kB 00:00 2024-05-08T21:07:51.076 INFO:teuthology.orchestra.run.smithi033.stdout:(37/146): cephfs-top-18.2.2-1322.gd21e4f96.el8. 180 kB/s | 18 kB 00:00 2024-05-08T21:07:51.118 INFO:teuthology.orchestra.run.smithi033.stdout:(38/146): gperftools-libs-2.7-9.el8.x86_64.rpm 3.3 MB/s | 306 kB 00:00 2024-05-08T21:07:51.176 INFO:teuthology.orchestra.run.smithi033.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 588 kB/s | 59 kB 00:00 2024-05-08T21:07:51.260 INFO:teuthology.orchestra.run.smithi033.stdout:(40/146): ceph-mgr-diskprediction-local-18.2.2- 6.7 MB/s | 7.4 MB 00:01 2024-05-08T21:07:51.285 INFO:teuthology.orchestra.run.smithi033.stdout:(41/146): libunwind-1.3.1-3.el8.x86_64.rpm 449 kB/s | 75 kB 00:00 2024-05-08T21:07:51.310 INFO:teuthology.orchestra.run.smithi033.stdout:(42/146): python3-asyncssh-2.7.0-2.el8.noarch.r 3.5 MB/s | 476 kB 00:00 2024-05-08T21:07:51.336 INFO:teuthology.orchestra.run.smithi033.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2024-05-08T21:07:51.361 INFO:teuthology.orchestra.run.smithi033.stdout:(44/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 434 kB/s | 44 kB 00:00 2024-05-08T21:07:51.416 INFO:teuthology.orchestra.run.smithi033.stdout:(45/146): python3-cachetools-3.1.1-4.el8.noarch 314 kB/s | 33 kB 00:00 2024-05-08T21:07:51.445 INFO:teuthology.orchestra.run.smithi033.stdout:(46/146): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2024-05-08T21:07:51.470 INFO:teuthology.orchestra.run.smithi033.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2024-05-08T21:07:51.487 INFO:teuthology.orchestra.run.smithi033.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2024-05-08T21:07:51.512 INFO:teuthology.orchestra.run.smithi033.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 265 kB/s | 11 kB 00:00 2024-05-08T21:07:51.513 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-05-08T21:07:51.537 INFO:teuthology.orchestra.run.smithi033.stdout:(50/146): python3-jaraco-functools-2.0-4.el8.no 349 kB/s | 18 kB 00:00 2024-05-08T21:07:51.571 INFO:teuthology.orchestra.run.smithi033.stdout:(51/146): python3-cherrypy-18.4.0-1.el8.noarch. 3.0 MB/s | 384 kB 00:00 2024-05-08T21:07:51.596 INFO:teuthology.orchestra.run.smithi033.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 829 kB/s | 49 kB 00:00 2024-05-08T21:07:51.622 INFO:teuthology.orchestra.run.smithi033.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2024-05-08T21:07:51.646 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T21:07:51.677 INFO:teuthology.orchestra.run.smithi033.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 752 kB/s | 60 kB 00:00 2024-05-08T21:07:51.706 INFO:teuthology.orchestra.run.smithi033.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2024-05-08T21:07:51.707 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-05-08T21:07:51.731 INFO:teuthology.orchestra.run.smithi033.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 291 kB/s | 16 kB 00:00 2024-05-08T21:07:51.756 INFO:teuthology.orchestra.run.smithi033.stdout:(57/146): python3-repoze-lru-0.7-6.el8.noarch.r 669 kB/s | 34 kB 00:00 2024-05-08T21:07:51.782 INFO:teuthology.orchestra.run.smithi033.stdout:(58/146): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2024-05-08T21:07:51.798 INFO:teuthology.orchestra.run.smithi033.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 1.5 MB/s | 64 kB 00:00 2024-05-08T21:07:51.824 INFO:teuthology.orchestra.run.smithi033.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 446 kB/s | 19 kB 00:00 2024-05-08T21:07:51.849 INFO:teuthology.orchestra.run.smithi033.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 476 kB/s | 24 kB 00:00 2024-05-08T21:07:51.875 INFO:teuthology.orchestra.run.smithi033.stdout:(62/146): python3-tempora-1.14.1-5.el8.noarch.r 583 kB/s | 29 kB 00:00 2024-05-08T21:07:51.879 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:07:51.931 INFO:teuthology.orchestra.run.smithi033.stdout:(63/146): python3-kubernetes-11.0.0-6.el8.noarc 2.8 MB/s | 1.2 MB 00:00 2024-05-08T21:07:51.967 INFO:teuthology.orchestra.run.smithi033.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2024-05-08T21:07:51.992 INFO:teuthology.orchestra.run.smithi033.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 2.1 MB/s | 255 kB 00:00 2024-05-08T21:07:52.017 INFO:teuthology.orchestra.run.smithi033.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 2.9 MB/s | 251 kB 00:00 2024-05-08T21:07:52.043 INFO:teuthology.orchestra.run.smithi033.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 813 kB/s | 61 kB 00:00 2024-05-08T21:07:52.068 INFO:teuthology.orchestra.run.smithi033.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 457 kB/s | 23 kB 00:00 2024-05-08T21:07:52.092 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:07:52.093 INFO:teuthology.orchestra.run.smithi033.stdout:(69/146): python3-webtest-2.0.33-1.el8.noarch.r 848 kB/s | 85 kB 00:00 2024-05-08T21:07:52.100 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-05-08T21:07:52.100 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-05-08T21:07:52.118 INFO:teuthology.orchestra.run.smithi033.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2024-05-08T21:07:52.193 INFO:teuthology.orchestra.run.smithi033.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2024-05-08T21:07:52.244 INFO:teuthology.orchestra.run.smithi033.stdout:(72/146): python3-packaging-16.8-9.el8.noarch.r 320 kB/s | 48 kB 00:00 2024-05-08T21:07:52.271 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T21:07:52.311 INFO:teuthology.orchestra.run.smithi033.stdout:(73/146): python3-jsonpointer-1.10-11.el8.noarc 109 kB/s | 20 kB 00:00 2024-05-08T21:07:52.403 INFO:teuthology.orchestra.run.smithi033.stdout:(74/146): python3-jsonpatch-1.21-2.el8.noarch.r 289 kB/s | 27 kB 00:00 2024-05-08T21:07:52.434 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:07:52.455 INFO:teuthology.orchestra.run.smithi033.stdout:(75/146): python3-prettytable-0.7.2-14.el8.noar 174 kB/s | 44 kB 00:00 2024-05-08T21:07:52.495 INFO:teuthology.orchestra.run.smithi033.stdout:(76/146): python3-pyasn1-modules-0.3.7-6.el8.no 455 kB/s | 110 kB 00:00 2024-05-08T21:07:52.513 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:07:52.528 INFO:teuthology.orchestra.run.smithi033.stdout:(77/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2024-05-08T21:07:52.570 INFO:teuthology.orchestra.run.smithi033.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 3.9 MB/s | 457 kB 00:00 2024-05-08T21:07:52.621 INFO:teuthology.orchestra.run.smithi033.stdout:(79/146): flex-2.6.1-9.el8.x86_64.rpm 3.4 MB/s | 320 kB 00:00 2024-05-08T21:07:52.671 INFO:teuthology.orchestra.run.smithi033.stdout:(80/146): bison-3.0.4-10.el8.x86_64.rpm 3.8 MB/s | 688 kB 00:00 2024-05-08T21:07:52.676 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:07:52.740 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:07:52.746 INFO:teuthology.orchestra.run.smithi033.stdout:(81/146): libtool-2.4.6-25.el8.x86_64.rpm 3.9 MB/s | 709 kB 00:00 2024-05-08T21:07:52.763 INFO:teuthology.orchestra.run.smithi033.stdout:(82/146): python3-markupsafe-0.23-19.el8.x86_64 273 kB/s | 39 kB 00:00 2024-05-08T21:07:52.788 INFO:teuthology.orchestra.run.smithi033.stdout:(83/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2024-05-08T21:07:52.864 INFO:teuthology.orchestra.run.smithi033.stdout:(84/146): uuid-1.6.2-43.el8.x86_64.rpm 547 kB/s | 64 kB 00:00 2024-05-08T21:07:52.923 INFO:teuthology.orchestra.run.smithi033.stdout:(85/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 652 kB/s | 103 kB 00:00 2024-05-08T21:07:52.965 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T21:07:53.014 INFO:teuthology.orchestra.run.smithi033.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2024-05-08T21:07:53.048 INFO:teuthology.orchestra.run.smithi033.stdout:(87/146): python3-jinja2-2.10.1-3.el8.noarch.rp 2.9 MB/s | 538 kB 00:00 2024-05-08T21:07:53.148 INFO:teuthology.orchestra.run.smithi033.stdout:(88/146): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2024-05-08T21:07:53.159 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T21:07:53.216 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T21:07:53.323 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-05-08T21:07:53.333 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-05-08T21:07:53.403 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T21:07:53.407 INFO:teuthology.orchestra.run.smithi033.stdout:(89/146): socat-1.7.4.1-1.el8.x86_64.rpm 824 kB/s | 323 kB 00:00 2024-05-08T21:07:53.510 INFO:teuthology.orchestra.run.smithi033.stdout:(90/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 491 kB/s | 225 kB 00:00 2024-05-08T21:07:53.558 INFO:teuthology.orchestra.run.smithi033.stdout:(91/146): librdkafka-0.11.4-3.el8.x86_64.rpm 865 kB/s | 354 kB 00:00 2024-05-08T21:07:53.576 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:07:53.616 INFO:teuthology.orchestra.run.smithi033.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.4 MB/s | 157 kB 00:00 2024-05-08T21:07:53.725 INFO:teuthology.orchestra.run.smithi033.stdout:(93/146): openblas-threads-0.3.15-6.el8.x86_64. 16 MB/s | 5.2 MB 00:00 2024-05-08T21:07:53.816 INFO:teuthology.orchestra.run.smithi033.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 3.5 MB/s | 714 kB 00:00 2024-05-08T21:07:53.892 INFO:teuthology.orchestra.run.smithi033.stdout:(95/146): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2024-05-08T21:07:53.917 INFO:teuthology.orchestra.run.smithi033.stdout:(96/146): libpq-13.11-1.el8.x86_64.rpm 1.0 MB/s | 198 kB 00:00 2024-05-08T21:07:53.950 INFO:teuthology.orchestra.run.smithi033.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 1.5 MB/s | 202 kB 00:00 2024-05-08T21:07:53.982 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:07:54.009 INFO:teuthology.orchestra.run.smithi033.stdout:(98/146): python3-pytz-2017.2-11.el8.noarch.rpm 463 kB/s | 54 kB 00:00 2024-05-08T21:07:54.076 INFO:teuthology.orchestra.run.smithi033.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-08T21:07:54.151 INFO:teuthology.orchestra.run.smithi033.stdout:(100/146): postgresql-10.23-4.module+el8.9.0+21 11 MB/s | 1.5 MB 00:00 2024-05-08T21:07:54.176 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:07:54.201 INFO:teuthology.orchestra.run.smithi033.stdout:(101/146): postgresql-contrib-10.23-4.module+el 6.3 MB/s | 812 kB 00:00 2024-05-08T21:07:54.360 INFO:teuthology.orchestra.run.smithi033.stdout:(102/146): postgresql-server-10.23-4.module+el8 24 MB/s | 5.1 MB 00:00 2024-05-08T21:07:54.386 INFO:teuthology.orchestra.run.smithi033.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 97 kB/s | 18 kB 00:00 2024-05-08T21:07:54.568 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:07:54.629 INFO:teuthology.orchestra.run.smithi033.stdout:(104/146): python3-scipy-1.0.0-21.module+el8.9. 19 MB/s | 14 MB 00:00 2024-05-08T21:07:54.653 INFO:teuthology.orchestra.run.smithi033.stdout:(105/146): python3-jwt-1.6.1-2.el8.noarch.rpm 146 kB/s | 43 kB 00:00 2024-05-08T21:07:54.678 INFO:teuthology.orchestra.run.smithi033.stdout:(106/146): python3-oauthlib-2.1.0-1.el8.noarch. 530 kB/s | 155 kB 00:00 2024-05-08T21:07:54.759 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:07:54.761 INFO:teuthology.orchestra.run.smithi033.stdout:(107/146): python3-requests-oauthlib-1.0.0-1.el 393 kB/s | 43 kB 00:00 2024-05-08T21:07:54.787 INFO:teuthology.orchestra.run.smithi033.stdout:(108/146): python3-pycparser-2.14-14.el8.noarch 688 kB/s | 109 kB 00:00 2024-05-08T21:07:54.820 INFO:teuthology.orchestra.run.smithi033.stdout:(109/146): libaio-devel-0.3.112-1.el8.x86_64.rp 131 kB/s | 19 kB 00:00 2024-05-08T21:07:54.887 INFO:teuthology.orchestra.run.smithi033.stdout:(110/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.4 MB/s | 229 kB 00:00 2024-05-08T21:07:54.890 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:07:54.921 INFO:teuthology.orchestra.run.smithi033.stdout:(111/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 275 kB/s | 44 kB 00:00 2024-05-08T21:07:54.997 INFO:teuthology.orchestra.run.smithi033.stdout:(112/146): pcre2-devel-10.32-2.el8.x86_64.rpm 2.8 MB/s | 605 kB 00:00 2024-05-08T21:07:55.022 INFO:teuthology.orchestra.run.smithi033.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.6 MB/s | 220 kB 00:00 2024-05-08T21:07:55.080 INFO:teuthology.orchestra.run.smithi033.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2024-05-08T21:07:55.082 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:07:55.156 INFO:teuthology.orchestra.run.smithi033.stdout:(115/146): libsepol-devel-2.9-3.el8.x86_64.rpm 651 kB/s | 87 kB 00:00 2024-05-08T21:07:55.169 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:07:55.189 INFO:teuthology.orchestra.run.smithi033.stdout:(116/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 531 kB/s | 58 kB 00:00 2024-05-08T21:07:55.214 INFO:teuthology.orchestra.run.smithi033.stdout:(117/146): keyutils-libs-devel-1.5.10-9.el8.x86 221 kB/s | 48 kB 00:00 2024-05-08T21:07:55.256 INFO:teuthology.orchestra.run.smithi033.stdout:(118/146): libcom_err-devel-1.45.6-4.el8.x86_64 384 kB/s | 39 kB 00:00 2024-05-08T21:07:55.374 INFO:teuthology.orchestra.run.smithi033.stdout:(119/146): fuse-2.9.7-17.el8.x86_64.rpm 449 kB/s | 83 kB 00:00 2024-05-08T21:07:55.441 INFO:teuthology.orchestra.run.smithi033.stdout:(120/146): libquadmath-8.5.0-20.el8.x86_64.rpm 938 kB/s | 172 kB 00:00 2024-05-08T21:07:55.475 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:07:55.490 INFO:teuthology.orchestra.run.smithi033.stdout:(121/146): libgfortran-8.5.0-20.el8.x86_64.rpm 2.3 MB/s | 645 kB 00:00 2024-05-08T21:07:55.516 INFO:teuthology.orchestra.run.smithi033.stdout:(122/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.6 MB/s | 238 kB 00:00 2024-05-08T21:07:55.566 INFO:teuthology.orchestra.run.smithi033.stdout:(123/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 580 kB/s | 73 kB 00:00 2024-05-08T21:07:55.658 INFO:teuthology.orchestra.run.smithi033.stdout:(124/146): elfutils-libelf-devel-0.189-3.el8.x8 369 kB/s | 62 kB 00:00 2024-05-08T21:07:55.692 INFO:teuthology.orchestra.run.smithi033.stdout:(125/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 3.1 MB/s | 562 kB 00:00 2024-05-08T21:07:55.725 INFO:teuthology.orchestra.run.smithi033.stdout:(126/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 705 kB/s | 47 kB 00:00 2024-05-08T21:07:55.742 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:07:55.826 INFO:teuthology.orchestra.run.smithi033.stdout:(127/146): openssl-devel-1.1.1k-12.el8_9.x86_64 9.0 MB/s | 2.3 MB 00:00 2024-05-08T21:07:55.826 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:07:55.851 INFO:teuthology.orchestra.run.smithi033.stdout:(128/146): NetworkManager-initscripts-updown-1. 1.1 MB/s | 144 kB 00:00 2024-05-08T21:07:55.926 INFO:teuthology.orchestra.run.smithi033.stdout:(129/146): python3-cryptography-3.2.1-7.el8_9.x 2.3 MB/s | 559 kB 00:00 2024-05-08T21:07:55.960 INFO:teuthology.orchestra.run.smithi033.stdout:(130/146): libuuid-devel-2.32.1-44.el8_9.1.x86_ 748 kB/s | 100 kB 00:00 2024-05-08T21:07:56.031 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T21:07:56.147 INFO:teuthology.orchestra.run.smithi033.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.2 MB/s | 232 kB 00:00 2024-05-08T21:07:56.285 INFO:teuthology.orchestra.run.smithi033.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.1 MB/s | 303 kB 00:00 2024-05-08T21:07:56.303 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T21:07:56.435 INFO:teuthology.orchestra.run.smithi033.stdout:(133/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.6 MB/s | 854 kB 00:00 2024-05-08T21:07:56.470 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T21:07:56.611 INFO:teuthology.orchestra.run.smithi033.stdout:(134/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.0 MB/s | 711 kB 00:00 2024-05-08T21:07:56.786 INFO:teuthology.orchestra.run.smithi033.stdout:(135/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 8.4 MB/s | 1.5 MB 00:00 2024-05-08T21:07:56.936 INFO:teuthology.orchestra.run.smithi033.stdout:(136/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 2.2 MB/s | 333 kB 00:00 2024-05-08T21:07:57.095 INFO:teuthology.orchestra.run.smithi033.stdout:(137/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.8 MB/s | 1.1 MB 00:00 2024-05-08T21:07:57.195 INFO:teuthology.orchestra.run.smithi033.stdout:(138/146): librados2-18.2.2-1322.gd21e4f96.el8. 2.8 MB/s | 3.7 MB 00:01 2024-05-08T21:07:57.220 INFO:teuthology.orchestra.run.smithi033.stdout:(139/146): libblkid-2.32.1-44.el8_9.1.x86_64.rp 1.7 MB/s | 221 kB 00:00 2024-05-08T21:07:57.254 INFO:teuthology.orchestra.run.smithi033.stdout:(140/146): libfdisk-2.32.1-44.el8_9.1.x86_64.rp 4.2 MB/s | 254 kB 00:00 2024-05-08T21:07:57.354 INFO:teuthology.orchestra.run.smithi033.stdout:(141/146): librbd1-18.2.2-1322.gd21e4f96.el8.x8 2.5 MB/s | 3.6 MB 00:01 2024-05-08T21:07:57.366 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T21:07:57.380 INFO:teuthology.orchestra.run.smithi033.stdout:(142/146): libsmartcols-2.32.1-44.el8_9.1.x86_6 1.4 MB/s | 180 kB 00:00 2024-05-08T21:07:57.433 INFO:teuthology.orchestra.run.smithi033.stdout:(143/146): libmount-2.32.1-44.el8_9.1.x86_64.rp 1.1 MB/s | 237 kB 00:00 2024-05-08T21:07:57.539 INFO:teuthology.orchestra.run.smithi033.stdout:(144/146): libuuid-2.32.1-44.el8_9.1.x86_64.rpm 540 kB/s | 99 kB 00:00 2024-05-08T21:07:57.606 INFO:teuthology.orchestra.run.smithi033.stdout:(145/146): util-linux-user-2.32.1-44.el8_9.1.x8 598 kB/s | 103 kB 00:00 2024-05-08T21:07:57.637 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T21:07:57.656 INFO:teuthology.orchestra.run.smithi033.stdout:(146/146): util-linux-2.32.1-44.el8_9.1.x86_64. 9.0 MB/s | 2.5 MB 00:00 2024-05-08T21:07:57.659 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:07:57.659 INFO:teuthology.orchestra.run.smithi033.stdout:Total 15 MB/s | 215 MB 00:13 2024-05-08T21:07:57.766 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:07:57.869 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:07:58.016 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:07:58.092 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:07:58.249 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T21:07:58.416 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T21:07:58.566 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T21:07:58.765 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-05-08T21:07:59.066 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-05-08T21:07:59.191 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T21:07:59.200 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T21:07:59.213 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-05-08T21:07:59.213 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-05-08T21:07:59.366 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T21:07:59.423 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:07:59.670 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:07:59.858 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T21:07:59.971 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T21:08:00.079 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:08:00.156 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:08:00.166 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T21:08:00.387 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:08:00.459 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:08:00.549 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-05-08T21:08:00.560 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-05-08T21:08:00.683 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T21:08:00.860 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-05-08T21:08:00.875 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T21:08:01.071 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T21:08:01.147 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T21:08:01.165 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-05-08T21:08:01.288 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T21:08:01.379 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:08:01.436 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T21:08:01.673 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T21:08:01.785 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:08:01.962 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T21:08:02.037 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:08:02.259 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T21:08:02.405 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T21:08:02.471 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:08:02.534 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T21:08:02.687 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:08:02.787 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T21:08:02.817 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:08:03.034 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:08:03.130 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:08:03.470 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:08:03.758 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T21:08:03.770 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:08:03.846 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:08:04.051 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T21:08:04.070 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T21:08:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T21:08:04.356 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T21:08:04.556 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T21:08:04.568 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T21:08:04.951 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T21:08:05.223 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T21:08:05.411 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:08:05.514 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:08:05.719 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:08:05.795 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:08:05.998 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-05-08T21:08:06.302 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T21:08:06.321 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-05-08T21:08:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-05-08T21:08:06.608 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-05-08T21:08:06.629 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T21:08:06.756 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-05-08T21:08:06.829 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T21:08:06.867 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T21:08:06.971 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-05-08T21:08:07.051 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:08:07.066 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T21:08:07.152 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T21:08:07.253 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T21:08:07.270 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-05-08T21:08:07.413 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T21:08:07.457 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-05-08T21:08:07.590 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:08:07.665 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:08:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-05-08T21:08:07.810 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:08:07.865 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:08:07.875 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libuuid-2.32.1-44.el8_9.1.x86_64 3/162 2024-05-08T21:08:07.944 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T21:08:07.959 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:08:08.077 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:08:08.145 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libblkid-2.32.1-44.el8_9.1.x86_64 4/162 2024-05-08T21:08:08.193 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T21:08:08.391 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T21:08:08.395 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-05-08T21:08:08.570 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-05-08T21:08:08.572 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:08:08.602 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T21:08:08.670 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-05-08T21:08:08.752 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T21:08:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-05-08T21:08:08.990 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:08:09.264 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:08:09.281 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:08:09.368 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:08:09.430 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-05-08T21:08:09.468 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-08T21:08:09.660 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-08T21:08:09.692 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-05-08T21:08:09.814 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-05-08T21:08:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-08T21:08:09.907 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-05-08T21:08:10.025 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-08T21:08:10.043 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:08:10.195 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T21:08:10.208 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-08T21:08:10.391 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T21:08:10.477 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-08T21:08:10.509 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:08:10.521 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T21:08:10.693 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:08:10.757 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-08T21:08:10.806 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T21:08:10.818 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:08:10.952 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-08T21:08:11.042 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T21:08:11.051 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:08:11.092 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-08T21:08:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:08:11.334 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-08T21:08:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:08:11.701 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-08T21:08:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:08:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-05-08T21:08:11.999 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-08T21:08:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-05-08T21:08:12.287 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-08T21:08:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-05-08T21:08:12.513 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-08T21:08:12.682 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-08T21:08:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-05-08T21:08:12.774 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-05-08T21:08:12.909 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-08T21:08:13.110 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-05-08T21:08:13.159 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-08T21:08:13.163 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-05-08T21:08:13.365 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-05-08T21:08:13.381 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-08T21:08:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:08:13.574 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-08T21:08:13.646 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:08:13.759 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-08T21:08:13.834 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:08:13.914 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 20/162 2024-05-08T21:08:14.033 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-08T21:08:14.126 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T21:08:14.127 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-05-08T21:08:14.222 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T21:08:14.299 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T21:08:14.302 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-05-08T21:08:14.501 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T21:08:14.505 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-05-08T21:08:14.573 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-05-08T21:08:14.732 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-05-08T21:08:14.788 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-05-08T21:08:14.954 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T21:08:14.961 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T21:08:15.009 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T21:08:15.174 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-05-08T21:08:15.203 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T21:08:15.219 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T21:08:15.375 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T21:08:15.428 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-05-08T21:08:15.448 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T21:08:15.616 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T21:08:15.623 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T21:08:15.831 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-08T21:08:15.834 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:08:15.909 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:08:16.013 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T21:08:16.029 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-05-08T21:08:16.167 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:08:16.201 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T21:08:16.225 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T21:08:16.257 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:08:16.488 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T21:08:16.518 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T21:08:16.727 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T21:08:16.753 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T21:08:16.942 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T21:08:16.967 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:08:16.983 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-05-08T21:08:17.099 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T21:08:17.258 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-05-08T21:08:17.292 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T21:08:17.394 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:08:17.443 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-05-08T21:08:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T21:08:17.632 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-05-08T21:08:17.737 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T21:08:17.767 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:08:17.794 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:08:17.882 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-05-08T21:08:17.949 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T21:08:17.992 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T21:08:18.085 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T21:08:18.090 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-05-08T21:08:18.192 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-05-08T21:08:18.300 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T21:08:18.326 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-05-08T21:08:18.480 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-05-08T21:08:18.508 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T21:08:18.559 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-05-08T21:08:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 31/162 2024-05-08T21:08:18.682 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T21:08:18.767 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T21:08:18.805 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 32/162 2024-05-08T21:08:18.834 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-05-08T21:08:18.941 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T21:08:19.032 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-05-08T21:08:19.091 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 33/162 2024-05-08T21:08:19.182 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-05-08T21:08:19.186 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T21:08:19.352 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T21:08:19.419 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-05-08T21:08:19.620 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T21:08:19.635 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-05-08T21:08:19.854 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-05-08T21:08:20.049 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-05-08T21:08:20.142 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-05-08T21:08:20.255 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-05-08T21:08:20.461 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-05-08T21:08:20.492 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-05-08T21:08:20.653 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-05-08T21:08:20.674 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T21:08:20.839 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-05-08T21:08:20.971 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-05-08T21:08:20.973 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T21:08:21.081 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T21:08:21.160 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T21:08:21.262 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-05-08T21:08:21.468 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T21:08:21.496 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-05-08T21:08:21.678 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-05-08T21:08:21.720 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T21:08:21.876 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-05-08T21:08:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T21:08:22.222 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-05-08T21:08:22.528 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-05-08T21:08:22.656 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T21:08:22.798 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-05-08T21:08:22.903 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:08:22.981 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:08:22.982 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-05-08T21:08:23.044 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-05-08T21:08:23.183 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-05-08T21:08:23.186 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T21:08:23.391 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-05-08T21:08:23.453 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-05-08T21:08:23.661 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-05-08T21:08:23.720 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-05-08T21:08:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-05-08T21:08:23.967 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-05-08T21:08:24.149 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 41/162 2024-05-08T21:08:24.227 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-05-08T21:08:24.301 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 42/162 2024-05-08T21:08:24.426 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-05-08T21:08:24.507 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:08:24.616 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libfdisk-2.32.1-44.el8_9.1.x86_64 43/162 2024-05-08T21:08:24.726 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-05-08T21:08:24.841 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:08:24.902 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libmount-2.32.1-44.el8_9.1.x86_64 44/162 2024-05-08T21:08:24.907 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T21:08:25.136 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T21:08:25.159 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.32.1-44.el8_9.1.x86_64 45/162 2024-05-08T21:08:25.386 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-05-08T21:08:25.446 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T21:08:25.557 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:08:25.689 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T21:08:25.853 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T21:08:26.035 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T21:08:26.074 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libsmartcols-2.32.1-44.el8_9.1.x86_64 47/162 2024-05-08T21:08:26.224 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T21:08:26.441 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:08:26.461 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T21:08:26.463 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: util-linux-2.32.1-44.el8_9.1.x86_64 48/162 2024-05-08T21:08:26.629 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-05-08T21:08:26.681 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T21:08:26.850 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:08:26.850 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-05-08T21:08:26.902 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T21:08:27.038 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T21:08:27.052 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:08:27.188 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-05-08T21:08:27.286 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T21:08:27.413 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-05-08T21:08:27.501 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T21:08:27.596 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-05-08T21:08:27.639 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:08:27.645 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T21:08:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-05-08T21:08:27.952 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T21:08:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-05-08T21:08:28.197 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T21:08:28.397 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T21:08:28.538 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:08:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-05-08T21:08:28.673 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T21:08:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-05-08T21:08:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-05-08T21:08:29.130 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:08:29.335 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:08:29.402 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-05-08T21:08:29.747 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-05-08T21:08:29.779 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T21:08:30.051 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-05-08T21:08:30.118 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T21:08:30.314 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T21:08:30.317 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-05-08T21:08:30.523 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-05-08T21:08:30.632 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T21:08:30.749 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-05-08T21:08:30.890 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T21:08:31.026 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-05-08T21:08:31.139 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T21:08:31.218 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-05-08T21:08:31.451 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-05-08T21:08:31.689 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-05-08T21:08:31.927 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T21:08:32.004 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-05-08T21:08:32.122 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:08:32.192 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:08:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T21:08:32.346 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-05-08T21:08:32.414 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T21:08:32.693 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-05-08T21:08:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-05-08T21:08:33.114 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T21:08:33.190 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-05-08T21:08:33.414 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T21:08:33.598 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-05-08T21:08:33.820 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T21:08:34.054 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-05-08T21:08:34.241 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-05-08T21:08:34.465 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-05-08T21:08:34.812 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-05-08T21:08:35.056 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-05-08T21:08:35.386 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-05-08T21:08:35.585 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-05-08T21:08:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-05-08T21:08:36.021 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-05-08T21:08:36.293 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:08:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-05-08T21:08:36.501 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-05-08T21:08:36.571 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:08:36.706 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-05-08T21:08:36.983 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-05-08T21:08:37.161 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:08:37.219 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-05-08T21:08:37.420 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-05-08T21:08:37.473 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:08:37.563 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:08:37.678 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-05-08T21:08:37.737 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:08:37.923 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-05-08T21:08:38.106 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-05-08T21:08:38.360 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-05-08T21:08:39.590 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-05-08T21:08:39.933 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-05-08T21:08:40.136 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-05-08T21:08:40.905 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-05-08T21:08:41.199 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-05-08T21:08:41.435 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-05-08T21:08:42.274 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-05-08T21:08:42.496 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:08:42.563 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 103/162 2024-05-08T21:08:42.772 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 104/162 2024-05-08T21:08:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:08:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:08:48.037 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 105/162 2024-05-08T21:08:48.353 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:08:48.462 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 106/162 2024-05-08T21:08:48.595 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:08:48.972 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:08:48.972 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-05-08T21:08:48.973 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-05-08T21:08:48.973 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /dev 2024-05-08T21:08:48.973 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /run 2024-05-08T21:08:48.973 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-05-08T21:08:48.973 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-05-08T21:08:48.973 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-05-08T21:08:48.974 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-05-08T21:08:48.974 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /dev 2024-05-08T21:08:48.974 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T21:08:49.331 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:08:50.965 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:08:51.141 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:08:51.360 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:08:51.555 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:08:52.029 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:08:52.197 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:08:52.214 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:08:53.942 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:08:54.085 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:08:54.253 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T21:08:54.464 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T21:08:54.686 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:08:55.717 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:08:55.965 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:08:56.928 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:08:57.104 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:08:57.178 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:08:57.178 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2024-05-08T21:08:57.178 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2024-05-08T21:08:57.179 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /dev 2024-05-08T21:08:57.179 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /run 2024-05-08T21:08:57.179 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2024-05-08T21:08:57.179 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2024-05-08T21:08:57.179 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2024-05-08T21:08:57.179 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2024-05-08T21:08:57.179 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /dev 2024-05-08T21:08:57.179 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:08:57.197 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:08:57.544 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T21:08:57.575 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:08:57.740 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:08:57.778 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:08:57.937 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:08:57.949 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:08:59.212 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:08:59.214 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T21:08:59.373 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:08:59.408 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:08:59.437 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:08:59.604 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:08:59.646 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:08:59.652 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:08:59.803 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:08:59.824 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:09:00.150 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:09:00.299 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:09:00.370 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:09:00.499 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:09:00.516 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:09:02.341 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:09:02.528 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:09:02.639 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T21:09:02.875 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T21:09:03.172 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:09:04.197 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:09:04.492 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:09:04.820 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:09:05.268 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T21:09:05.461 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:09:05.639 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:09:05.644 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T21:09:05.732 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:09:05.888 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T21:09:06.096 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T21:09:06.317 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:09:06.355 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:09:06.572 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T21:09:06.688 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:09:06.699 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:09:06.848 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T21:09:07.093 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T21:09:07.283 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:09:07.512 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T21:09:07.698 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T21:09:07.878 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T21:09:07.958 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T21:09:08.158 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:09:08.192 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:09:08.439 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:09:08.483 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:09:08.657 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:09:09.002 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:09:09.181 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:09:10.164 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:09:10.435 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:09:10.472 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:09:10.921 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T21:09:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 107/162 2024-05-08T21:09:11.190 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-05-08T21:09:11.190 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-05-08T21:09:11.190 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /dev 2024-05-08T21:09:11.191 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /run 2024-05-08T21:09:11.191 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-05-08T21:09:11.191 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-05-08T21:09:11.191 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-05-08T21:09:11.191 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-05-08T21:09:11.191 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /dev 2024-05-08T21:09:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:09:11.283 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T21:09:11.513 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T21:09:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:09:11.667 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T21:09:11.892 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T21:09:12.158 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T21:09:12.419 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T21:09:12.584 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T21:09:12.593 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T21:09:12.803 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T21:09:12.992 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T21:09:13.067 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T21:09:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T21:09:13.280 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T21:09:13.382 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 108/162 2024-05-08T21:09:13.499 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T21:09:13.607 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:09:13.617 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:09:13.669 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T21:09:13.837 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T21:09:13.837 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:09:13.852 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 109/162 2024-05-08T21:09:14.041 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:09:14.061 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:09:14.062 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:09:14.279 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:09:14.279 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:09:14.547 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T21:09:14.548 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:09:14.569 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 110/162 2024-05-08T21:09:14.725 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:09:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 111/162 2024-05-08T21:09:14.785 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T21:09:14.834 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-05-08T21:09:14.960 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:09:15.112 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-05-08T21:09:15.202 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T21:09:15.202 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:09:15.478 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:09:15.478 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:09:15.512 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 129/162 2024-05-08T21:09:15.711 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:09:15.711 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:09:15.856 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-05-08T21:09:15.927 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:09:15.928 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:09:16.142 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T21:09:16.156 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-05-08T21:09:16.210 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:09:16.210 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:09:16.390 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T21:09:16.435 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:09:16.586 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T21:09:16.586 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:09:16.592 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T21:09:16.746 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T21:09:16.926 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:09:17.085 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 112/162 2024-05-08T21:09:17.249 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 113/162 2024-05-08T21:09:17.474 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:09:17.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T21:09:17.476 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T21:09:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T21:09:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T21:09:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T21:09:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T21:09:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T21:09:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T21:09:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T21:09:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T21:09:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T21:09:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T21:09:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T21:09:17.477 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T21:09:17.478 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T21:09:17.479 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-05-08T21:09:17.480 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-05-08T21:09:17.481 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-05-08T21:09:17.482 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-05-08T21:09:17.482 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-05-08T21:09:17.482 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-05-08T21:09:17.482 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-05-08T21:09:17.482 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-05-08T21:09:17.482 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-05-08T21:09:17.482 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-05-08T21:09:17.482 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-05-08T21:09:17.482 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-08T21:09:17.482 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-08T21:09:17.482 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-08T21:09:17.482 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-08T21:09:17.483 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T21:09:17.484 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T21:09:17.485 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:09:17.486 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T21:09:17.495 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 114/162 2024-05-08T21:09:17.745 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:09:18.889 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 115/162 2024-05-08T21:09:19.042 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:09:19.131 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:09:19.387 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:09:19.425 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:09:19.917 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T21:09:20.207 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 116/162 2024-05-08T21:09:20.269 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T21:09:20.532 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T21:09:20.581 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:09:20.695 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 117/162 2024-05-08T21:09:20.753 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T21:09:20.961 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T21:09:21.243 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 118/162 2024-05-08T21:09:21.273 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T21:09:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:09:21.487 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 119/162 2024-05-08T21:09:21.657 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T21:09:21.674 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:09:21.689 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 120/162 2024-05-08T21:09:21.902 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T21:09:21.983 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-05-08T21:09:22.219 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T21:09:22.233 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-05-08T21:09:22.402 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T21:09:22.413 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 145/162 2024-05-08T21:09:22.605 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-05-08T21:09:22.817 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T21:09:22.996 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T21:09:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 121/162 2024-05-08T21:09:23.206 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T21:09:23.206 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:09:23.289 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:09:23.330 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 122/162 2024-05-08T21:09:23.480 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:09:23.480 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:09:23.557 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:09:23.608 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 123/162 2024-05-08T21:09:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:09:23.756 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:09:23.840 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:09:24.042 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:09:24.228 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 124/162 2024-05-08T21:09:24.287 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T21:09:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:09:24.470 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:09:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T21:09:24.737 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:09:25.013 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:09:25.013 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:09:25.288 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:09:25.288 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:09:25.562 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:09:25.563 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:09:25.837 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:09:25.837 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:09:26.096 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:09:26.263 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T21:09:26.263 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:09:27.221 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:09:27.222 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T21:09:27.223 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T21:09:27.224 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T21:09:27.224 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T21:09:27.224 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T21:09:27.224 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T21:09:27.237 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T21:09:27.237 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T21:09:27.237 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T21:09:27.237 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T21:09:27.237 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T21:09:27.237 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T21:09:27.237 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T21:09:27.238 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T21:09:27.239 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T21:09:27.239 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T21:09:27.239 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T21:09:27.239 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T21:09:27.239 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T21:09:27.239 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T21:09:27.239 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T21:09:27.239 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T21:09:27.239 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T21:09:27.239 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T21:09:27.239 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-05-08T21:09:27.240 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 90/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 91/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 92/162 2024-05-08T21:09:27.241 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 93/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 94/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 95/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 96/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 97/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 98/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-05-08T21:09:27.242 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-05-08T21:09:27.243 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-05-08T21:09:27.244 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-05-08T21:09:27.244 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-05-08T21:09:27.244 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-05-08T21:09:27.244 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-05-08T21:09:27.244 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-05-08T21:09:27.244 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-05-08T21:09:27.244 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-05-08T21:09:27.244 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-05-08T21:09:27.244 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-05-08T21:09:27.244 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T21:09:27.245 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:09:27.246 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T21:09:27.247 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:09:27.247 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T21:09:28.979 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T21:09:28.979 INFO:teuthology.orchestra.run.smithi071.stdout:Installed products updated. 2024-05-08T21:09:29.391 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 125/162 2024-05-08T21:09:29.584 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:09:29.585 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:29.586 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.587 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: jq-1.6-7.el8.x86_64 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.588 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T21:09:29.589 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:09:29.590 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T21:09:29.591 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T21:09:29.592 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T21:09:29.593 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T21:09:29.594 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T21:09:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T21:09:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T21:09:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T21:09:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T21:09:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T21:09:29.595 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T21:09:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T21:09:29.596 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-05-08T21:09:29.596 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-05-08T21:09:29.907 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 126/162 2024-05-08T21:09:30.275 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-05-08T21:09:30.500 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T21:09:31.328 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-05-08T21:09:31.661 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-05-08T21:09:31.945 DEBUG:teuthology.parallel:result is None 2024-05-08T21:09:31.956 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-05-08T21:09:32.155 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:09:32.402 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : util-linux-user-2.32.1-44.el8_9.1.x86_64 132/162 2024-05-08T21:09:32.610 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-05-08T21:09:32.814 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 134/162 2024-05-08T21:09:35.519 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:09:35.880 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:09:35.920 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-05-08T21:09:36.383 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-05-08T21:09:36.925 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-05-08T21:09:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-05-08T21:09:37.389 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 139/162 2024-05-08T21:09:37.606 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 140/162 2024-05-08T21:09:37.883 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 141/162 2024-05-08T21:09:38.173 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-05-08T21:09:38.362 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T21:09:38.374 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-15.e 143/162 2024-05-08T21:09:38.554 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-05-08T21:09:38.787 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T21:09:38.875 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-05-08T21:09:39.070 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T21:09:39.087 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-05-08T21:09:39.300 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-05-08T21:09:39.514 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-05-08T21:09:39.666 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-05-08T21:09:39.667 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:09:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-05-08T21:09:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:09:40.241 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-05-08T21:09:40.241 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:09:40.520 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-05-08T21:09:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-05-08T21:09:41.092 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:09:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-05-08T21:09:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:09:41.606 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:09:41.607 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:09:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-05-08T21:09:41.865 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:09:42.115 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:09:42.115 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:09:42.372 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T21:09:42.372 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2024-05-08T21:09:42.381 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-05-08T21:09:42.381 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:09:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:09:42.823 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-05-08T21:09:42.823 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:09:42.989 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:09:42.989 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-05-08T21:09:42.989 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T21:09:42.989 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T21:09:42.989 INFO:teuthology.orchestra.run.smithi082.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T21:09:42.989 INFO:teuthology.orchestra.run.smithi082.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:42.989 INFO:teuthology.orchestra.run.smithi082.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:42.989 INFO:teuthology.orchestra.run.smithi082.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:42.989 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.990 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.990 INFO:teuthology.orchestra.run.smithi082.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:42.990 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:42.990 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:09:42.990 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:09:42.990 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:42.990 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:42.990 INFO:teuthology.orchestra.run.smithi082.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T21:09:42.991 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T21:09:42.991 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-05-08T21:09:42.991 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T21:09:42.991 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T21:09:42.991 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T21:09:42.991 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.991 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.991 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.991 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.992 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:42.992 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.992 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.992 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.992 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:42.992 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:42.992 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:42.992 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:42.992 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:42.992 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.992 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.992 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: jq-1.6-7.el8.x86_64 2024-05-08T21:09:42.993 INFO:teuthology.orchestra.run.smithi082.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T21:09:42.994 INFO:teuthology.orchestra.run.smithi082.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T21:09:42.994 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T21:09:42.994 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.994 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.994 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.994 INFO:teuthology.orchestra.run.smithi082.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T21:09:42.994 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T21:09:42.994 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T21:09:42.994 INFO:teuthology.orchestra.run.smithi082.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T21:09:42.994 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T21:09:42.994 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T21:09:42.995 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.995 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.995 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T21:09:42.995 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.995 INFO:teuthology.orchestra.run.smithi082.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T21:09:42.995 INFO:teuthology.orchestra.run.smithi082.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T21:09:42.995 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T21:09:42.995 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T21:09:42.995 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:42.995 INFO:teuthology.orchestra.run.smithi082.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T21:09:42.996 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T21:09:42.997 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T21:09:42.998 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T21:09:42.998 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T21:09:42.998 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T21:09:42.998 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T21:09:42.998 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T21:09:42.998 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T21:09:42.998 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T21:09:42.998 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T21:09:42.998 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T21:09:42.998 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:42.999 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T21:09:43.000 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T21:09:43.001 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T21:09:43.001 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T21:09:43.001 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T21:09:43.001 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:43.001 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:43.001 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:43.001 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T21:09:43.001 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T21:09:43.002 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T21:09:43.002 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T21:09:43.002 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T21:09:43.002 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T21:09:43.002 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:09:43.002 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-05-08T21:09:43.222 DEBUG:teuthology.parallel:result is None 2024-05-08T21:09:43.890 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-05-08T21:09:43.890 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 1/162 2024-05-08T21:09:43.893 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2/162 2024-05-08T21:09:43.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 3/162 2024-05-08T21:09:43.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 4/162 2024-05-08T21:09:43.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1322.gd21e4 5/162 2024-05-08T21:09:43.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 6/162 2024-05-08T21:09:43.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 7/162 2024-05-08T21:09:43.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 8/162 2024-05-08T21:09:43.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 9/162 2024-05-08T21:09:43.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 10/162 2024-05-08T21:09:43.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 11/162 2024-05-08T21:09:43.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 12/162 2024-05-08T21:09:43.895 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 13/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_ 14/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 15/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 16/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_6 17/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86 18/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 19/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el 20/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8. 21/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_6 22/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 23/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 24/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 25/162 2024-05-08T21:09:43.896 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 26/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 27/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 28/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96. 29/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noa 30/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.n 31/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21 32/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el 33/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 34/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.e 35/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 36/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 37/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 38/162 2024-05-08T21:09:43.897 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-05-08T21:09:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-05-08T21:09:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-05-08T21:09:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-05-08T21:09:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-05-08T21:09:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-05-08T21:09:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-05-08T21:09:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-05-08T21:09:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-05-08T21:09:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-05-08T21:09:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-05-08T21:09:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-05-08T21:09:43.898 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-05-08T21:09:43.899 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-05-08T21:09:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-05-08T21:09:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-05-08T21:09:43.902 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-05-08T21:09:43.903 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 122/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 123/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 124/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 125/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 127/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 128/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-15.e 129/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.32.1-44.el8_9.1.x86_64 130/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 131/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 133/162 2024-05-08T21:09:43.904 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 139/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 140/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 141/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 142/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 143/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 144/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 145/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 146/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 147/162 2024-05-08T21:09:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 148/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libblkid-2.32.1-44.el8_9.1.x86_64 149/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 150/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libfdisk-2.32.1-44.el8_9.1.x86_64 151/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 152/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libmount-2.32.1-44.el8_9.1.x86_64 153/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 154/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsmartcols-2.32.1-44.el8_9.1.x86_64 155/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 156/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-2.32.1-44.el8_9.1.x86_64 157/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 158/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-2.32.1-44.el8_9.1.x86_64 159/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 160/162 2024-05-08T21:09:43.906 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-user-2.32.1-44.el8_9.1.x86_64 161/162 2024-05-08T21:09:55.935 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 162/162 2024-05-08T21:09:55.935 INFO:teuthology.orchestra.run.smithi033.stdout:Installed products updated. 2024-05-08T21:09:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:09:56.610 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-05-08T21:09:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-05-08T21:09:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: libblkid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: libfdisk-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: libmount-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: libsmartcols-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-user-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-05-08T21:09:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-initscripts-updown-1:1.40.16-15.el8_9.noarch 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.1-8.el8.noarch 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.0.4-10.el8.x86_64 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:56.612 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:18.2.2-1322.gd21e4f96.el8.noarch 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.1-9.el8.x86_64 2024-05-08T21:09:56.613 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.7-17.el8.x86_64 2024-05-08T21:09:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-05-08T21:09:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: jq-1.6-7.el8.x86_64 2024-05-08T21:09:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-05-08T21:09:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-05-08T21:09:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-05-08T21:09:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-05-08T21:09:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-05-08T21:09:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.2-3.el8.x86_64 2024-05-08T21:09:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: libpq-13.11-1.el8.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-25.el8.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.3.1-3.el8.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.32.1-44.el8_9.1.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-05-08T21:09:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.15-6.el8.x86_64 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-05-08T21:09:56.616 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.5.1-7.el8.noarch 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-6.2-6.el8.noarch 2024-05-08T21:09:56.617 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.0.6-14.el8.noarch 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-05-08T21:09:56.618 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging-16.8-9.el8.noarch 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-2.6-1.el8.noarch 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.14-14.el8.noarch 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2017.2-11.el8.noarch 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.619 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.4.1-12.el8.noarch 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el8.noarch 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:18.2.2-1322.gd21e4f96.el8.x86_64 2024-05-08T21:09:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-1.el8.x86_64 2024-05-08T21:09:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.13.0-2.el8.x86_64 2024-05-08T21:09:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-43.el8.x86_64 2024-05-08T21:09:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-05-08T21:09:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-05-08T21:09:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-05-08T21:09:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:09:56.621 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-05-08T21:09:56.842 DEBUG:teuthology.parallel:result is None 2024-05-08T21:09:56.842 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:09:57.063 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T21:09:57.094 INFO:teuthology.orchestra.run.smithi033.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T21:09:57.095 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T21:09:57.095 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T21:09:57.097 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:09:57.344 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T21:09:57.372 INFO:teuthology.orchestra.run.smithi071.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T21:09:57.373 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T21:09:57.373 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T21:09:57.375 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:09:57.519 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T21:09:57.548 INFO:teuthology.orchestra.run.smithi082.stdout:18.2.2-1322.gd21e4f96.el8 2024-05-08T21:09:57.549 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1322.gd21e4f96.el8 2024-05-08T21:09:57.549 INFO:teuthology.task.install:The correct ceph version 18.2.2-1322.gd21e4f96 is installed. 2024-05-08T21:09:57.550 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T21:09:57.551 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:09:57.551 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T21:09:57.605 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-08T21:09:57.605 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T21:09:57.655 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-08T21:09:57.655 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T21:09:57.703 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T21:09:57.703 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:09:57.704 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T21:09:57.739 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T21:09:57.814 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-08T21:09:57.814 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T21:09:57.847 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T21:09:57.919 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-08T21:09:57.919 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T21:09:57.949 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T21:09:58.021 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T21:09:58.021 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:09:58.021 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T21:09:58.055 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T21:09:58.131 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-08T21:09:58.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T21:09:58.163 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T21:09:58.234 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-08T21:09:58.235 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T21:09:58.267 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T21:09:58.338 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T21:09:58.338 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:09:58.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T21:09:58.372 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T21:09:58.447 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-08T21:09:58.448 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T21:09:58.479 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T21:09:58.551 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-08T21:09:58.551 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T21:09:58.582 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T21:09:58.656 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-08T21:09:58.770 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-08T21:09:58.770 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:09:58.771 INFO:tasks.cephadm:Cluster fsid is 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:09:58.771 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-08T21:09:58.771 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.33', 'mon.b': '172.21.15.71', 'mon.c': '172.21.15.82'} 2024-05-08T21:09:58.771 INFO:tasks.cephadm:First mon is mon.a on smithi033 2024-05-08T21:09:58.771 INFO:tasks.cephadm:First mgr is x 2024-05-08T21:09:58.771 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-08T21:09:58.771 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname $(hostname -s) 2024-05-08T21:09:58.807 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-05-08T21:09:58.841 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname $(hostname -s) 2024-05-08T21:09:58.873 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-08T21:09:58.873 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:09:59.034 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79222/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:11:17.581213', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['arm64', 'source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd21e4f961318b56f0d915caa2490c3711bf13c34', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/79220/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1322-gd21e4f96', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1322.gd21e4f96'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 18:26:58.979379', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.150423-reef', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-08T21:09:59.198 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240508.150423-reef, sha1 d21e4f961318b56f0d915caa2490c3711bf13c34 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:09:59.199 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm 2024-05-08T21:09:59.199 DEBUG:teuthology.orchestra.run.smithi033:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T21:10:00.019 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 21:10 /home/ubuntu/cephtest/cephadm 2024-05-08T21:10:00.020 DEBUG:teuthology.orchestra.run.smithi071:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T21:10:00.390 INFO:teuthology.orchestra.run.smithi071.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 21:10 /home/ubuntu/cephtest/cephadm 2024-05-08T21:10:00.390 DEBUG:teuthology.orchestra.run.smithi082:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.150423-reef/d21e4f961318b56f0d915caa2490c3711bf13c34/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T21:10:00.761 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 May 8 21:10 /home/ubuntu/cephtest/cephadm 2024-05-08T21:10:00.761 DEBUG:teuthology.orchestra.run.smithi033:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T21:10:00.785 DEBUG:teuthology.orchestra.run.smithi071:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T21:10:00.806 DEBUG:teuthology.orchestra.run.smithi082:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T21:10:00.843 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 on all hosts... 2024-05-08T21:10:00.843 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T21:10:00.845 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T21:10:00.849 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 pull 2024-05-08T21:10:01.789 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:10:01.847 INFO:teuthology.orchestra.run.smithi082.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:10:02.885 INFO:teuthology.orchestra.run.smithi033.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:10:31.845 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:10:31.845 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T21:10:31.845 INFO:teuthology.orchestra.run.smithi033.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T21:10:31.846 INFO:teuthology.orchestra.run.smithi033.stdout: "repo_digests": [ 2024-05-08T21:10:31.846 INFO:teuthology.orchestra.run.smithi033.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T21:10:31.846 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T21:10:31.846 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:10:51.359 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-05-08T21:10:51.360 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T21:10:51.360 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T21:10:51.360 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-05-08T21:10:51.360 INFO:teuthology.orchestra.run.smithi071.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T21:10:51.360 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-05-08T21:10:51.360 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-05-08T21:10:53.031 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-08T21:10:53.031 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T21:10:53.031 INFO:teuthology.orchestra.run.smithi082.stdout: "image_id": "af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c", 2024-05-08T21:10:53.031 INFO:teuthology.orchestra.run.smithi082.stdout: "repo_digests": [ 2024-05-08T21:10:53.031 INFO:teuthology.orchestra.run.smithi082.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e8ec72da4fcdc307fe9b94614a1c07d7ebae386ad20c4998c8d80e9b80a48985" 2024-05-08T21:10:53.032 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-05-08T21:10:53.032 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-08T21:10:53.050 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph 2024-05-08T21:10:53.091 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-05-08T21:10:53.131 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph 2024-05-08T21:10:53.162 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /etc/ceph 2024-05-08T21:10:53.196 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-05-08T21:10:53.227 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /etc/ceph 2024-05-08T21:10:53.254 INFO:tasks.cephadm:Writing seed config... 2024-05-08T21:10:53.255 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-08T21:10:53.255 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-08T21:10:53.255 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-08T21:10:53.255 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-08T21:10:53.256 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-08T21:10:53.257 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-08T21:10:53.258 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-08T21:10:53.258 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-08T21:10:53.258 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-08T21:10:53.258 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-08T21:10:53.258 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-08T21:10:53.258 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-08T21:10:53.258 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-08T21:10:53.258 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-08T21:10:53.258 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-08T21:10:53.259 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:10:53.259 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-08T21:10:53.281 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 53ae3a5a-0d7f-11ef-bc97-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-08T21:10:53.281 DEBUG:teuthology.orchestra.run.smithi033:mon.a> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.a.service 2024-05-08T21:10:53.324 DEBUG:teuthology.orchestra.run.smithi033:mgr.x> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.x.service 2024-05-08T21:10:53.358 INFO:journalctl@ceph.mon.a.smithi033.stdout:-- Logs begin at Wed 2024-05-08 20:53:21 UTC. -- 2024-05-08T21:10:53.359 INFO:tasks.cephadm:Bootstrapping... 2024-05-08T21:10:53.359 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 -v bootstrap --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.33 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:10:53.391 INFO:journalctl@ceph.mgr.x.smithi033.stdout:-- Logs begin at Wed 2024-05-08 20:53:21 UTC. -- 2024-05-08T21:10:53.548 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-05-08T21:10:53.548 INFO:teuthology.orchestra.run.smithi033.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34', '-v', 'bootstrap', '--fsid', '53ae3a5a-0d7f-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.33', '--skip-admin-label'] 2024-05-08T21:10:53.583 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.6.1 2024-05-08T21:10:53.583 INFO:teuthology.orchestra.run.smithi033.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-08T21:10:53.583 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying podman|docker is present... 2024-05-08T21:10:53.614 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.6.1 2024-05-08T21:10:53.614 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying lvm2 is present... 2024-05-08T21:10:53.614 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying time synchronization is in place... 2024-05-08T21:10:53.622 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T21:10:53.622 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T21:10:53.628 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T21:10:53.629 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-08T21:10:53.635 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-05-08T21:10:53.642 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-05-08T21:10:53.642 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-05-08T21:10:53.642 INFO:teuthology.orchestra.run.smithi033.stdout:Repeating the final host check... 2024-05-08T21:10:53.680 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.6.1 2024-05-08T21:10:53.680 INFO:teuthology.orchestra.run.smithi033.stdout:podman (/bin/podman) version 4.6.1 is present 2024-05-08T21:10:53.680 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl is present 2024-05-08T21:10:53.680 INFO:teuthology.orchestra.run.smithi033.stdout:lvcreate is present 2024-05-08T21:10:53.687 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T21:10:53.687 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T21:10:53.693 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T21:10:53.694 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-08T21:10:53.700 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-05-08T21:10:53.707 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-05-08T21:10:53.707 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-05-08T21:10:53.707 INFO:teuthology.orchestra.run.smithi033.stdout:Host looks OK 2024-05-08T21:10:53.707 INFO:teuthology.orchestra.run.smithi033.stdout:Cluster fsid: 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:10:53.707 INFO:teuthology.orchestra.run.smithi033.stdout:Acquiring lock 139761956882920 on /run/cephadm/53ae3a5a-0d7f-11ef-bc97-c7b262605968.lock 2024-05-08T21:10:53.708 INFO:teuthology.orchestra.run.smithi033.stdout:Lock 139761956882920 acquired on /run/cephadm/53ae3a5a-0d7f-11ef-bc97-c7b262605968.lock 2024-05-08T21:10:53.708 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 3300 ... 2024-05-08T21:10:53.708 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 6789 ... 2024-05-08T21:10:53.709 INFO:teuthology.orchestra.run.smithi033.stdout:Base mon IP(s) is [172.21.15.33:3300, 172.21.15.33:6789], mon addrv is [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-05-08T21:10:53.713 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2024-05-08T21:10:53.713 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.33 metric 100 2024-05-08T21:10:53.716 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-08T21:10:53.716 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-05-08T21:10:53.716 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1776sec hoplimit 64 pref medium 2024-05-08T21:10:53.719 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-08T21:10:53.719 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-08T21:10:53.719 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T21:10:53.719 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-08T21:10:53.719 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-05-08T21:10:53.719 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T21:10:53.720 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-05-08T21:10:53.720 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-05-08T21:10:53.720 INFO:teuthology.orchestra.run.smithi033.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-08T21:10:53.721 INFO:teuthology.orchestra.run.smithi033.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-08T21:10:53.722 INFO:teuthology.orchestra.run.smithi033.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:10:54.095 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T21:10:54.095 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34... 2024-05-08T21:10:54.095 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Getting image source signatures 2024-05-08T21:10:54.095 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:7850648fa4707607dddc5eec7674b8c9296086cf87f20a8b1acf1b014b952e9f 2024-05-08T21:10:54.095 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-08T21:10:54.095 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying config sha256:af23193ff691dd58bee78ea954b569105e41c41def20b81a25baa1d30500ad1c 2024-05-08T21:10:54.095 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-08T21:10:54.919 INFO:teuthology.orchestra.run.smithi033.stdout:ceph: stdout ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T21:10:54.920 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph version: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable) 2024-05-08T21:10:54.920 INFO:teuthology.orchestra.run.smithi033.stdout:Extracting ceph user uid/gid from container image... 2024-05-08T21:10:55.525 INFO:teuthology.orchestra.run.smithi033.stdout:stat: stdout 167 167 2024-05-08T21:10:55.526 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial keys... 2024-05-08T21:10:57.633 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQBf6jtmLaocLxAAiOW80efshGTFzuvPUyMkpw== 2024-05-08T21:11:00.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQBi6jtmVx8XMhAA4FQohPk9UCSAjRWPpag2EA== 2024-05-08T21:11:03.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQBl6jtmmuNJGxAAnthbI1acHsRQCdqC9JSgSw== 2024-05-08T21:11:03.280 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial monmap... 2024-05-08T21:11:05.412 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T21:11:05.413 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-05-08T21:11:05.413 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:11:05.413 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T21:11:05.413 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool for a [v2:172.21.15.33:3300,v1:172.21.15.33:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T21:11:05.413 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = pacific 2024-05-08T21:11:05.413 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: set fsid to 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:11:05.414 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T21:11:05.414 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:11:05.414 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mon... 2024-05-08T21:11:06.120 INFO:teuthology.orchestra.run.smithi033.stdout:create mon.a on 2024-05-08T21:11:06.272 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-05-08T21:11:06.369 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-08T21:11:06.513 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968.target. 2024-05-08T21:11:06.513 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968.target. 2024-05-08T21:11:06.715 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.a 2024-05-08T21:11:06.715 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-08T21:11:06.814 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968.target.wants/ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@.service. 2024-05-08T21:11:07.192 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T21:11:07.193 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-05-08T21:11:07.201 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T21:11:07.201 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-08T21:11:07.201 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-05-08T21:11:07.202 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T21:11:07.202 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon to start... 2024-05-08T21:11:07.202 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon... 2024-05-08T21:11:07.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:07 smithi033 ceph-mon[42043]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T21:11:08.093 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout cluster: 2024-05-08T21:11:08.093 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout id: 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:11:08.093 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout services: 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.526882s) 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout data: 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pgs: 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:08.094 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.648+0000 7fe4f0cbc700 1 Processor -- start 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.648+0000 7fe4f0cbc700 1 -- start start 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.649+0000 7fe4f0cbc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 0x7fe4ec1093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.649+0000 7fe4f0cbc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec079e40 con 0x7fe4ec107000 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.649+0000 7fe4ea59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 0x7fe4ec1093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.649+0000 7fe4ea59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 0x7fe4ec1093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41842/0 (socket says 172.21.15.33:41842) 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.649+0000 7fe4ea59c700 1 -- 172.21.15.33:0/4156185232 learned_addr learned my addr 172.21.15.33:0/4156185232 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.650+0000 7fe4ea59c700 1 -- 172.21.15.33:0/4156185232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec109930 con 0x7fe4ec107000 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.650+0000 7fe4ea59c700 1 --2- 172.21.15.33:0/4156185232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 0x7fe4ec1093f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0098a0 tx=0x7fe4dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbff8074169e6c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.650+0000 7fe4e959a700 1 -- 172.21.15.33:0/4156185232 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4dc004160 con 0x7fe4ec107000 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.650+0000 7fe4e959a700 1 -- 172.21.15.33:0/4156185232 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4dc0042c0 con 0x7fe4ec107000 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.650+0000 7fe4e959a700 1 -- 172.21.15.33:0/4156185232 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4dc0045a0 con 0x7fe4ec107000 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.650+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/4156185232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 msgr2=0x7fe4ec1093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.650+0000 7fe4f0cbc700 1 --2- 172.21.15.33:0/4156185232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 0x7fe4ec1093f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0098a0 tx=0x7fe4dc009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:08.095 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.651+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/4156185232 shutdown_connections 2024-05-08T21:11:08.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.651+0000 7fe4f0cbc700 1 --2- 172.21.15.33:0/4156185232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 0x7fe4ec1093f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:08.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.651+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/4156185232 >> 172.21.15.33:0/4156185232 conn(0x7fe4ec100a10 msgr2=0x7fe4ec102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:08.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.651+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/4156185232 shutdown_connections 2024-05-08T21:11:08.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.651+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/4156185232 wait complete. 2024-05-08T21:11:08.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.652+0000 7fe4f0cbc700 1 Processor -- start 2024-05-08T21:11:08.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.652+0000 7fe4f0cbc700 1 -- start start 2024-05-08T21:11:08.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.652+0000 7fe4f0cbc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 0x7fe4ec199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:08.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.652+0000 7fe4f0cbc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec199ed0 con 0x7fe4ec107000 2024-05-08T21:11:08.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.652+0000 7fe4ea59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 0x7fe4ec199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:08.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.652+0000 7fe4ea59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 0x7fe4ec199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41852/0 (socket says 172.21.15.33:41852) 2024-05-08T21:11:08.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.652+0000 7fe4ea59c700 1 -- 172.21.15.33:0/2133566005 learned_addr learned my addr 172.21.15.33:0/2133566005 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:08.096 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.652+0000 7fe4ea59c700 1 -- 172.21.15.33:0/2133566005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc009550 con 0x7fe4ec107000 2024-05-08T21:11:08.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.652+0000 7fe4ea59c700 1 --2- 172.21.15.33:0/2133566005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 0x7fe4ec199990 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec079280 tx=0x7fe4dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:08.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.653+0000 7fe4db7fe700 1 -- 172.21.15.33:0/2133566005 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4dc011630 con 0x7fe4ec107000 2024-05-08T21:11:08.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.653+0000 7fe4db7fe700 1 -- 172.21.15.33:0/2133566005 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4dc011790 con 0x7fe4ec107000 2024-05-08T21:11:08.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.653+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/2133566005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ec19a0d0 con 0x7fe4ec107000 2024-05-08T21:11:08.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.653+0000 7fe4db7fe700 1 -- 172.21.15.33:0/2133566005 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4dc011a30 con 0x7fe4ec107000 2024-05-08T21:11:08.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.653+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/2133566005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ec19a570 con 0x7fe4ec107000 2024-05-08T21:11:08.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.653+0000 7fe4db7fe700 1 -- 172.21.15.33:0/2133566005 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fe4dc021070 con 0x7fe4ec107000 2024-05-08T21:11:08.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.653+0000 7fe4db7fe700 1 -- 172.21.15.33:0/2133566005 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe4dc01b440 con 0x7fe4ec107000 2024-05-08T21:11:08.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.654+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/2133566005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ec1937c0 con 0x7fe4ec107000 2024-05-08T21:11:08.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.656+0000 7fe4db7fe700 1 -- 172.21.15.33:0/2133566005 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fe4dc01a460 con 0x7fe4ec107000 2024-05-08T21:11:08.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.733+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/2133566005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fe4ec109d60 con 0x7fe4ec107000 2024-05-08T21:11:08.097 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.733+0000 7fe4db7fe700 1 -- 172.21.15.33:0/2133566005 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fe4dc01a600 con 0x7fe4ec107000 2024-05-08T21:11:08.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.734+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/2133566005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 msgr2=0x7fe4ec199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:08.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.734+0000 7fe4f0cbc700 1 --2- 172.21.15.33:0/2133566005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 0x7fe4ec199990 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec079280 tx=0x7fe4dc004750 comp rx=0 tx=0).stop 2024-05-08T21:11:08.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.734+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/2133566005 shutdown_connections 2024-05-08T21:11:08.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.734+0000 7fe4f0cbc700 1 --2- 172.21.15.33:0/2133566005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ec107000 0x7fe4ec199990 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:08.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.734+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/2133566005 >> 172.21.15.33:0/2133566005 conn(0x7fe4ec100a10 msgr2=0x7fe4ec102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:08.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.734+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/2133566005 shutdown_connections 2024-05-08T21:11:08.098 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:07.734+0000 7fe4f0cbc700 1 -- 172.21.15.33:0/2133566005 wait complete. 2024-05-08T21:11:08.098 INFO:teuthology.orchestra.run.smithi033.stdout:mon is available 2024-05-08T21:11:08.098 INFO:teuthology.orchestra.run.smithi033.stdout:Assimilating anything we can from ceph.conf... 2024-05-08T21:11:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:08 smithi033 ceph-mon[42043]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T21:11:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:08 smithi033 ceph-mon[42043]: monmap e1: 1 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:11:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:08 smithi033 ceph-mon[42043]: fsmap 2024-05-08T21:11:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:08 smithi033 ceph-mon[42043]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T21:11:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:08 smithi033 ceph-mon[42043]: mgrmap e1: no daemons active 2024-05-08T21:11:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:08 smithi033 ceph-mon[42043]: from='client.? 172.21.15.33:0/2133566005' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-08T21:11:08.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:08.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T21:11:08.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:11:08.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-05-08T21:11:08.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T21:11:08.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T21:11:08.944 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.491+0000 7ff99a691700 1 Processor -- start 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.491+0000 7ff99a691700 1 -- start start 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.492+0000 7ff99a691700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 0x7ff994109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.492+0000 7ff99a691700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff994079ea0 con 0x7ff994106fa0 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.492+0000 7ff993fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 0x7ff994109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.492+0000 7ff993fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 0x7ff994109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41866/0 (socket says 172.21.15.33:41866) 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.492+0000 7ff993fff700 1 -- 172.21.15.33:0/4033062807 learned_addr learned my addr 172.21.15.33:0/4033062807 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.492+0000 7ff993fff700 1 -- 172.21.15.33:0/4033062807 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9941098d0 con 0x7ff994106fa0 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.492+0000 7ff993fff700 1 --2- 172.21.15.33:0/4033062807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 0x7ff994109390 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff9840098a0 tx=0x7ff984009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90692fc591dc3f43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.493+0000 7ff992ffd700 1 -- 172.21.15.33:0/4033062807 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff984004160 con 0x7ff994106fa0 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.493+0000 7ff992ffd700 1 -- 172.21.15.33:0/4033062807 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9840042c0 con 0x7ff994106fa0 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.493+0000 7ff992ffd700 1 -- 172.21.15.33:0/4033062807 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9840045a0 con 0x7ff994106fa0 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.493+0000 7ff99a691700 1 -- 172.21.15.33:0/4033062807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 msgr2=0x7ff994109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:08.945 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.493+0000 7ff99a691700 1 --2- 172.21.15.33:0/4033062807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 0x7ff994109390 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff9840098a0 tx=0x7ff984009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:08.946 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.493+0000 7ff99a691700 1 -- 172.21.15.33:0/4033062807 shutdown_connections 2024-05-08T21:11:08.946 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.493+0000 7ff99a691700 1 --2- 172.21.15.33:0/4033062807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 0x7ff994109390 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:08.946 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.493+0000 7ff99a691700 1 -- 172.21.15.33:0/4033062807 >> 172.21.15.33:0/4033062807 conn(0x7ff994100a10 msgr2=0x7ff994102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:08.946 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.493+0000 7ff99a691700 1 -- 172.21.15.33:0/4033062807 shutdown_connections 2024-05-08T21:11:08.946 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.493+0000 7ff99a691700 1 -- 172.21.15.33:0/4033062807 wait complete. 2024-05-08T21:11:08.946 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.494+0000 7ff99a691700 1 Processor -- start 2024-05-08T21:11:08.946 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.494+0000 7ff99a691700 1 -- start start 2024-05-08T21:11:08.946 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.494+0000 7ff99a691700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 0x7ff9941999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:08.946 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.494+0000 7ff99a691700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff994199ef0 con 0x7ff994106fa0 2024-05-08T21:11:08.946 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.495+0000 7ff993fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 0x7ff9941999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:08.946 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.495+0000 7ff993fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 0x7ff9941999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41874/0 (socket says 172.21.15.33:41874) 2024-05-08T21:11:08.946 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.495+0000 7ff993fff700 1 -- 172.21.15.33:0/343569228 learned_addr learned my addr 172.21.15.33:0/343569228 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.495+0000 7ff993fff700 1 -- 172.21.15.33:0/343569228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff984009550 con 0x7ff994106fa0 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.495+0000 7ff993fff700 1 --2- 172.21.15.33:0/343569228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 0x7ff9941999b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff9940792e0 tx=0x7ff984004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.495+0000 7ff9917fa700 1 -- 172.21.15.33:0/343569228 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff984011630 con 0x7ff994106fa0 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.495+0000 7ff9917fa700 1 -- 172.21.15.33:0/343569228 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff984011790 con 0x7ff994106fa0 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.495+0000 7ff99a691700 1 -- 172.21.15.33:0/343569228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99419a0f0 con 0x7ff994106fa0 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.495+0000 7ff9917fa700 1 -- 172.21.15.33:0/343569228 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff984011a30 con 0x7ff994106fa0 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.495+0000 7ff99a691700 1 -- 172.21.15.33:0/343569228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99419a590 con 0x7ff994106fa0 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.495+0000 7ff9917fa700 1 -- 172.21.15.33:0/343569228 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff984021070 con 0x7ff994106fa0 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.495+0000 7ff9917fa700 1 -- 172.21.15.33:0/343569228 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff98401b440 con 0x7ff994106fa0 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.496+0000 7ff99a691700 1 -- 172.21.15.33:0/343569228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9941937c0 con 0x7ff994106fa0 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.499+0000 7ff9917fa700 1 -- 172.21.15.33:0/343569228 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7ff98401a460 con 0x7ff994106fa0 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.571+0000 7ff99a691700 1 -- 172.21.15.33:0/343569228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff994109b80 con 0x7ff994106fa0 2024-05-08T21:11:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.576+0000 7ff9917fa700 1 -- 172.21.15.33:0/343569228 <== mon.0 v2:172.21.15.33:3300/0 7 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff98401a600 con 0x7ff994106fa0 2024-05-08T21:11:08.948 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.576+0000 7ff9917fa700 1 -- 172.21.15.33:0/343569228 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7ff9840334e0 con 0x7ff994106fa0 2024-05-08T21:11:08.948 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.576+0000 7ff99a691700 1 -- 172.21.15.33:0/343569228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 msgr2=0x7ff9941999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:08.948 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.576+0000 7ff99a691700 1 --2- 172.21.15.33:0/343569228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 0x7ff9941999b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff9940792e0 tx=0x7ff984004750 comp rx=0 tx=0).stop 2024-05-08T21:11:08.948 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.577+0000 7ff99a691700 1 -- 172.21.15.33:0/343569228 shutdown_connections 2024-05-08T21:11:08.948 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.577+0000 7ff99a691700 1 --2- 172.21.15.33:0/343569228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff994106fa0 0x7ff9941999b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:08.948 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.577+0000 7ff99a691700 1 -- 172.21.15.33:0/343569228 >> 172.21.15.33:0/343569228 conn(0x7ff994100a10 msgr2=0x7ff994102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:08.948 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.577+0000 7ff99a691700 1 -- 172.21.15.33:0/343569228 shutdown_connections 2024-05-08T21:11:08.948 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:08.577+0000 7ff99a691700 1 -- 172.21.15.33:0/343569228 wait complete. 2024-05-08T21:11:08.948 INFO:teuthology.orchestra.run.smithi033.stdout:Generating new minimal ceph.conf... 2024-05-08T21:11:09.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:09 smithi033 ceph-mon[42043]: from='client.? 172.21.15.33:0/343569228' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T21:11:09.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:09 smithi033 ceph-mon[42043]: from='client.? 172.21.15.33:0/343569228' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-08T21:11:09.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.346+0000 7fcb4d1e5700 1 Processor -- start 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.346+0000 7fcb4d1e5700 1 -- start start 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.346+0000 7fcb4d1e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 0x7fcb481068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.346+0000 7fcb4d1e5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb48106eb0 con 0x7fcb481064c0 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.347+0000 7fcb46d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 0x7fcb481068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.347+0000 7fcb46d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 0x7fcb481068e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41876/0 (socket says 172.21.15.33:41876) 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.347+0000 7fcb46d9d700 1 -- 172.21.15.33:0/2056975817 learned_addr learned my addr 172.21.15.33:0/2056975817 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.347+0000 7fcb46d9d700 1 -- 172.21.15.33:0/2056975817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb481076c0 con 0x7fcb481064c0 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.347+0000 7fcb46d9d700 1 --2- 172.21.15.33:0/2056975817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 0x7fcb481068e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0098a0 tx=0x7fcb3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b298f8ebea1568 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.347+0000 7fcb45d9b700 1 -- 172.21.15.33:0/2056975817 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c004160 con 0x7fcb481064c0 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.347+0000 7fcb45d9b700 1 -- 172.21.15.33:0/2056975817 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fcb3c0042c0 con 0x7fcb481064c0 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.347+0000 7fcb45d9b700 1 -- 172.21.15.33:0/2056975817 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c011940 con 0x7fcb481064c0 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.348+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/2056975817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 msgr2=0x7fcb481068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:09.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.348+0000 7fcb4d1e5700 1 --2- 172.21.15.33:0/2056975817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 0x7fcb481068e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0098a0 tx=0x7fcb3c009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.348+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/2056975817 shutdown_connections 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.348+0000 7fcb4d1e5700 1 --2- 172.21.15.33:0/2056975817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 0x7fcb481068e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.348+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/2056975817 >> 172.21.15.33:0/2056975817 conn(0x7fcb48101a60 msgr2=0x7fcb48103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.348+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/2056975817 shutdown_connections 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.348+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/2056975817 wait complete. 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.349+0000 7fcb4d1e5700 1 Processor -- start 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.349+0000 7fcb4d1e5700 1 -- start start 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.350+0000 7fcb4d1e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 0x7fcb4819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.350+0000 7fcb4d1e5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4819a650 con 0x7fcb481064c0 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.350+0000 7fcb46d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 0x7fcb4819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.350+0000 7fcb46d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 0x7fcb4819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41892/0 (socket says 172.21.15.33:41892) 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.350+0000 7fcb46d9d700 1 -- 172.21.15.33:0/849594963 learned_addr learned my addr 172.21.15.33:0/849594963 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:09.792 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.350+0000 7fcb46d9d700 1 -- 172.21.15.33:0/849594963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c009550 con 0x7fcb481064c0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.350+0000 7fcb46d9d700 1 --2- 172.21.15.33:0/849594963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 0x7fcb4819a110 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c003f20 tx=0x7fcb3c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.350+0000 7fcb2ffff700 1 -- 172.21.15.33:0/849594963 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c0036a0 con 0x7fcb481064c0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.351+0000 7fcb2ffff700 1 -- 172.21.15.33:0/849594963 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fcb3c003c70 con 0x7fcb481064c0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.351+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/849594963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4819a850 con 0x7fcb481064c0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.351+0000 7fcb2ffff700 1 -- 172.21.15.33:0/849594963 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c01be80 con 0x7fcb481064c0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.351+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/849594963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4819acf0 con 0x7fcb481064c0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.351+0000 7fcb2ffff700 1 -- 172.21.15.33:0/849594963 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fcb3c0237e0 con 0x7fcb481064c0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.351+0000 7fcb2ffff700 1 -- 172.21.15.33:0/849594963 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcb3c02cd70 con 0x7fcb481064c0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.352+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/849594963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb480625d0 con 0x7fcb481064c0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.354+0000 7fcb2ffff700 1 -- 172.21.15.33:0/849594963 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fcb3c01b440 con 0x7fcb481064c0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.428+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/849594963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fcb4810a8f0 con 0x7fcb481064c0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.428+0000 7fcb2ffff700 1 -- 172.21.15.33:0/849594963 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fcb3c01b720 con 0x7fcb481064c0 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.429+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/849594963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 msgr2=0x7fcb4819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:09.793 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.429+0000 7fcb4d1e5700 1 --2- 172.21.15.33:0/849594963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 0x7fcb4819a110 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c003f20 tx=0x7fcb3c011f00 comp rx=0 tx=0).stop 2024-05-08T21:11:09.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.429+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/849594963 shutdown_connections 2024-05-08T21:11:09.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.429+0000 7fcb4d1e5700 1 --2- 172.21.15.33:0/849594963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb481064c0 0x7fcb4819a110 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:09.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.429+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/849594963 >> 172.21.15.33:0/849594963 conn(0x7fcb48101a60 msgr2=0x7fcb481024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:09.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.429+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/849594963 shutdown_connections 2024-05-08T21:11:09.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:09.429+0000 7fcb4d1e5700 1 -- 172.21.15.33:0/849594963 wait complete. 2024-05-08T21:11:09.794 INFO:teuthology.orchestra.run.smithi033.stdout:Restarting the monitor... 2024-05-08T21:11:09.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:09 smithi033 systemd[1]: Stopping Ceph mon.a for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:11:10.360 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:10 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-a[42020]: 2024-05-08T21:11:10.010+0000 7fcb28054700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:11:10.360 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:10 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-a[42020]: 2024-05-08T21:11:10.010+0000 7fcb28054700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-08T21:11:10.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:10 smithi033 bash[42368]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-a 2024-05-08T21:11:10.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:10 smithi033 systemd[1]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.a.service: Succeeded. 2024-05-08T21:11:10.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:10 smithi033 systemd[1]: Stopped Ceph mon.a for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:11:10.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:10 smithi033 systemd[1]: Starting Ceph mon.a for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:11:11.060 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 bash[42489]: 5f32ed17a96fadc31f7c027e526f1015afb22e2f7372c344a9844afaa899add7 2024-05-08T21:11:11.060 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T21:11:11.062 INFO:teuthology.orchestra.run.smithi033.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-08T21:11:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable), process ceph-mon, pid 7 2024-05-08T21:11:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: pidfile_write: ignore empty --pid-file 2024-05-08T21:11:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 systemd[1]: Started Ceph mon.a for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:11:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: load: jerasure load: lrc 2024-05-08T21:11:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: RocksDB version: 7.9.2 2024-05-08T21:11:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Git sha 0 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Compile date 2024-05-08 17:55:50 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: DB SUMMARY 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: DB Session ID: SD6E89UYPNKB0RQ4BZZ0 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: CURRENT file: CURRENT 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 96434 ; 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.error_if_exists: 0 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.create_if_missing: 0 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.paranoid_checks: 1 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T21:11:11.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.env: 0x558ea871b820 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.info_log: 0x558eabbe3520 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.statistics: (nil) 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.use_fsync: 0 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_log_file_size: 0 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.allow_fallocate: 1 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T21:11:11.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.use_direct_reads: 0 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.db_log_dir: 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.wal_dir: 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T21:11:11.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.write_buffer_manager: 0x558eaae78320 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.unordered_write: 0 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.row_cache: None 2024-05-08T21:11:11.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.wal_filter: None 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.two_write_queues: 0 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.wal_compression: 0 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.atomic_flush: 0 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.log_readahead_size: 0 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T21:11:11.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_background_jobs: 2 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_background_compactions: -1 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_subcompactions: 1 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_open_files: -1 2024-05-08T21:11:11.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_background_flushes: -1 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Compression algorithms supported: 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: kZSTD supported: 0 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: kXpressCompression supported: 0 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: kZlibCompression supported: 1 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: kSnappyCompression supported: 1 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: kLZ4Compression supported: 1 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: kBZip2Compression supported: 0 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T21:11:11.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.merge_operator: 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_filter: None 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558eabbe3620) 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks: 1 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_top_level_index_and_filter: 1 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_type: 0 2024-05-08T21:11:11.467 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_index_type: 0 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_shortening: 1 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: checksum: 4 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: no_block_cache: 0 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache: 0x558eaaefb1f0 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_name: BinnedLRUCache 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_options: 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: capacity : 536870912 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_shard_bits : 4 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: strict_capacity_limit : 0 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: high_pri_pool_ratio: 0.000 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_compressed: (nil) 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: persistent_cache: (nil) 2024-05-08T21:11:11.468 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size: 4096 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size_deviation: 10 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_restart_interval: 16 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_block_restart_interval: 1 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: metadata_block_size: 4096 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: partition_filters: 0 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: use_delta_encoding: 1 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: filter_policy: bloomfilter 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: whole_key_filtering: 1 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: verify_compression: 0 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: read_amp_bytes_per_bit: 0 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: format_version: 5 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_index_compression: 1 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_align: 0 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: max_auto_readahead_size: 262144 2024-05-08T21:11:11.469 INFO:journalctl@ceph.mon.a.smithi033.stdout: prepopulate_block_cache: 0 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout: initial_auto_readahead_size: 8192 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compression: NoCompression 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.num_levels: 7 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T21:11:11.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T21:11:11.471 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T21:11:11.471 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T21:11:11.471 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T21:11:11.471 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T21:11:11.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T21:11:11.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.table_properties_collectors: 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.inplace_update_support: 0 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T21:11:11.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.bloom_locality: 0 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.max_successive_merges: 0 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.ttl: 2592000 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.enable_blob_files: false 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.min_blob_size: 0 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T21:11:11.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 938e5abf-d64a-43e1-bb62-b5c7f7368fb5 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715202671066636, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715202671068461, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91923, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 303, "table_properties": {"data_size": 89945, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13340, "raw_average_key_size": 50, "raw_value_size": 82672, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 266, "num_filter_entries": 266, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715202671, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "938e5abf-d64a-43e1-bb62-b5c7f7368fb5", "db_session_id": "SD6E89UYPNKB0RQ4BZZ0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715202671068594, "job": 1, "event": "recovery_finished"} 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T21:11:11.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558eaaf98000 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: DB pointer 0x558eaaf86000 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** DB Stats ** 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T21:11:11.477 INFO:journalctl@ceph.mon.a.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: L0 2/0 91.60 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 79.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: Sum 2/0 91.60 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 79.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 79.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 79.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T21:11:11.478 INFO:journalctl@ceph.mon.a.smithi033.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T21:11:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T21:11:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T21:11:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T21:11:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T21:11:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative compaction: 0.00 GB write, 15.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T21:11:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval compaction: 0.00 GB write, 15.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T21:11:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T21:11:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache BinnedLRUCache@0x558eaaefb1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 5.2e-05 secs_since: 0 2024-05-08T21:11:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-05-08T21:11:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-08T21:11:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T21:11:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: starting mon.a rank 0 at public addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] at bind addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: mon.a@-1(???) e1 preinit fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: mon.a@-1(???).mds e1 new map 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: mon.a@-1(???).mds e1 print_map 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout: e1 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout: legacy client fscid: -1 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout: No filesystems configured 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T21:11:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: monmap e1: 1 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:11:11.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: fsmap 2024-05-08T21:11:11.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T21:11:11.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:11 smithi033 ceph-mon[42528]: mgrmap e1: no daemons active 2024-05-08T21:11:11.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.454+0000 7fee7a1df700 1 Processor -- start 2024-05-08T21:11:11.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.455+0000 7fee7a1df700 1 -- start start 2024-05-08T21:11:11.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.455+0000 7fee7a1df700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 0x7fee74104bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:11.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.455+0000 7fee7a1df700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee741051a0 con 0x7fee741047b0 2024-05-08T21:11:11.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.455+0000 7fee737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 0x7fee74104bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:11.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.455+0000 7fee737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 0x7fee74104bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50888/0 (socket says 172.21.15.33:50888) 2024-05-08T21:11:11.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.455+0000 7fee737fe700 1 -- 172.21.15.33:0/877502736 learned_addr learned my addr 172.21.15.33:0/877502736 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:11.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.456+0000 7fee737fe700 1 -- 172.21.15.33:0/877502736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee74107740 con 0x7fee741047b0 2024-05-08T21:11:11.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.456+0000 7fee737fe700 1 --2- 172.21.15.33:0/877502736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 0x7fee74104bd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fee640098a0 tx=0x7fee64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53345c6e58709f63 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:11.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.456+0000 7fee727fc700 1 -- 172.21.15.33:0/877502736 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee64004160 con 0x7fee741047b0 2024-05-08T21:11:11.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.456+0000 7fee727fc700 1 -- 172.21.15.33:0/877502736 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fee640042c0 con 0x7fee741047b0 2024-05-08T21:11:11.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.456+0000 7fee727fc700 1 -- 172.21.15.33:0/877502736 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee64011940 con 0x7fee741047b0 2024-05-08T21:11:11.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.457+0000 7fee7a1df700 1 -- 172.21.15.33:0/877502736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 msgr2=0x7fee74104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:11.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.457+0000 7fee7a1df700 1 --2- 172.21.15.33:0/877502736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 0x7fee74104bd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fee640098a0 tx=0x7fee64009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:11.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.457+0000 7fee7a1df700 1 -- 172.21.15.33:0/877502736 shutdown_connections 2024-05-08T21:11:11.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.457+0000 7fee7a1df700 1 --2- 172.21.15.33:0/877502736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 0x7fee74104bd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:11.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.457+0000 7fee7a1df700 1 -- 172.21.15.33:0/877502736 >> 172.21.15.33:0/877502736 conn(0x7fee741000e0 msgr2=0x7fee74102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:11.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.457+0000 7fee7a1df700 1 -- 172.21.15.33:0/877502736 shutdown_connections 2024-05-08T21:11:11.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.457+0000 7fee7a1df700 1 -- 172.21.15.33:0/877502736 wait complete. 2024-05-08T21:11:11.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.458+0000 7fee7a1df700 1 Processor -- start 2024-05-08T21:11:11.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.458+0000 7fee7a1df700 1 -- start start 2024-05-08T21:11:11.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.458+0000 7fee7a1df700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 0x7fee74106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:11.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.458+0000 7fee7a1df700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee741074b0 con 0x7fee741047b0 2024-05-08T21:11:11.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.458+0000 7fee737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 0x7fee74106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:11.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.458+0000 7fee737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 0x7fee74106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50896/0 (socket says 172.21.15.33:50896) 2024-05-08T21:11:11.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.458+0000 7fee737fe700 1 -- 172.21.15.33:0/586404627 learned_addr learned my addr 172.21.15.33:0/586404627 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:11.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.459+0000 7fee737fe700 1 -- 172.21.15.33:0/586404627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee64009550 con 0x7fee741047b0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.459+0000 7fee737fe700 1 --2- 172.21.15.33:0/586404627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 0x7fee74106f70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fee64006af0 tx=0x7fee640119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.459+0000 7fee70ff9700 1 -- 172.21.15.33:0/586404627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee64003af0 con 0x7fee741047b0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.459+0000 7fee70ff9700 1 -- 172.21.15.33:0/586404627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fee64003c50 con 0x7fee741047b0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.459+0000 7fee7a1df700 1 -- 172.21.15.33:0/586404627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee741056f0 con 0x7fee741047b0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.459+0000 7fee70ff9700 1 -- 172.21.15.33:0/586404627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee64023620 con 0x7fee741047b0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.459+0000 7fee7a1df700 1 -- 172.21.15.33:0/586404627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee74105b90 con 0x7fee741047b0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.459+0000 7fee70ff9700 1 -- 172.21.15.33:0/586404627 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fee64023b90 con 0x7fee741047b0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.460+0000 7fee70ff9700 1 -- 172.21.15.33:0/586404627 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fee64023e40 con 0x7fee741047b0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.461+0000 7fee7a1df700 1 -- 172.21.15.33:0/586404627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee740625d0 con 0x7fee741047b0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.463+0000 7fee70ff9700 1 -- 172.21.15.33:0/586404627 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7fee64043dc0 con 0x7fee741047b0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.537+0000 7fee7a1df700 1 -- 172.21.15.33:0/586404627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fee74105dd0 con 0x7fee741047b0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.542+0000 7fee70ff9700 1 -- 172.21.15.33:0/586404627 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fee64032070 con 0x7fee741047b0 2024-05-08T21:11:11.936 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.543+0000 7fee7a1df700 1 -- 172.21.15.33:0/586404627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 msgr2=0x7fee74106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:11.937 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.543+0000 7fee7a1df700 1 --2- 172.21.15.33:0/586404627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 0x7fee74106f70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fee64006af0 tx=0x7fee640119c0 comp rx=0 tx=0).stop 2024-05-08T21:11:11.937 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.543+0000 7fee7a1df700 1 -- 172.21.15.33:0/586404627 shutdown_connections 2024-05-08T21:11:11.937 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.543+0000 7fee7a1df700 1 --2- 172.21.15.33:0/586404627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee741047b0 0x7fee74106f70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:11.937 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.543+0000 7fee7a1df700 1 -- 172.21.15.33:0/586404627 >> 172.21.15.33:0/586404627 conn(0x7fee741000e0 msgr2=0x7fee74100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:11.937 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.543+0000 7fee7a1df700 1 -- 172.21.15.33:0/586404627 shutdown_connections 2024-05-08T21:11:11.937 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:11.543+0000 7fee7a1df700 1 -- 172.21.15.33:0/586404627 wait complete. 2024-05-08T21:11:11.937 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-08T21:11:11.949 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:11:11.950 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mgr... 2024-05-08T21:11:11.950 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-08T21:11:11.951 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-08T21:11:12.162 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.x 2024-05-08T21:11:12.162 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-08T21:11:12.270 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968.target.wants/ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@.service. 2024-05-08T21:11:12.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:12 smithi033 systemd[1]: Starting Ceph mgr.x for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:11:12.742 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T21:11:12.742 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-05-08T21:11:12.751 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T21:11:12.751 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-08T21:11:12.751 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-05-08T21:11:12.751 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T21:11:12.761 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T21:11:12.761 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-05-08T21:11:12.770 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T21:11:12.770 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-08T21:11:12.770 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-05-08T21:11:12.770 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-08T21:11:12.770 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr to start... 2024-05-08T21:11:12.770 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr... 2024-05-08T21:11:12.886 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:12 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/586404627' entity='client.admin' 2024-05-08T21:11:12.886 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:12 smithi033 bash[42762]: df72257e92a5c29ea48479de3850838220d8f5deca0a9c56d0d0dac9e5046555 2024-05-08T21:11:12.886 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:12 smithi033 systemd[1]: Started Ceph mgr.x for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:11:13.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:13 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:13.183+0000 7f16218ba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:11:13.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:13 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:13.290+0000 7f16218ba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:11:13.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:13 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:13.393+0000 7f16218ba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:11:13.601 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:13.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:11:13.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:11:13.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T21:11:13.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T21:11:13.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T21:11:13.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T21:11:13.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:13.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T21:11:13.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T21:11:13.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-05-08T21:11:13.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:11:13.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T21:11:13.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T21:11:13.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:11:13.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-08T21:11:13.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T21:11:13.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:13.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T21:11:13.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T21:11:13.610 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:13.611 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T21:11:13.611 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T21:11:13.611 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T21:11:13.611 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T21:11:13.611 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T21:11:13.611 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T21:11:13.611 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T21:11:13.611 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:11:13.611 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:11:13.617 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:13.617 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T21:11:13.617 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:13.617 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T21:11:07.209073+0000", 2024-05-08T21:11:13.617 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:11:13.617 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:13.617 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T21:11:13.617 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:11:13.617 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.145+0000 7fef008e7700 1 Processor -- start 2024-05-08T21:11:13.617 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.145+0000 7fef008e7700 1 -- start start 2024-05-08T21:11:13.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.145+0000 7fef008e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 0x7feefc07a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:13.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.145+0000 7fef008e7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc07a8e0 con 0x7feefc07beb0 2024-05-08T21:11:13.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.146+0000 7feefa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 0x7feefc07a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:13.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.146+0000 7feefa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 0x7feefc07a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50916/0 (socket says 172.21.15.33:50916) 2024-05-08T21:11:13.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.146+0000 7feefa59c700 1 -- 172.21.15.33:0/3610476744 learned_addr learned my addr 172.21.15.33:0/3610476744 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:13.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.146+0000 7feefa59c700 1 -- 172.21.15.33:0/3610476744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc07aa20 con 0x7feefc07beb0 2024-05-08T21:11:13.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.146+0000 7feefa59c700 1 --2- 172.21.15.33:0/3610476744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 0x7feefc07a310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feef00098a0 tx=0x7feef0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba66fa0a93f9cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:13.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.147+0000 7feef959a700 1 -- 172.21.15.33:0/3610476744 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feef0004160 con 0x7feefc07beb0 2024-05-08T21:11:13.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.147+0000 7feef959a700 1 -- 172.21.15.33:0/3610476744 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7feef00042c0 con 0x7feefc07beb0 2024-05-08T21:11:13.618 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.147+0000 7feef959a700 1 -- 172.21.15.33:0/3610476744 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feef0011940 con 0x7feefc07beb0 2024-05-08T21:11:13.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.147+0000 7fef008e7700 1 -- 172.21.15.33:0/3610476744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 msgr2=0x7feefc07a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:13.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.147+0000 7fef008e7700 1 --2- 172.21.15.33:0/3610476744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 0x7feefc07a310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feef00098a0 tx=0x7feef0009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:13.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.147+0000 7fef008e7700 1 -- 172.21.15.33:0/3610476744 shutdown_connections 2024-05-08T21:11:13.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.147+0000 7fef008e7700 1 --2- 172.21.15.33:0/3610476744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 0x7feefc07a310 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:13.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.147+0000 7fef008e7700 1 -- 172.21.15.33:0/3610476744 >> 172.21.15.33:0/3610476744 conn(0x7feefc101a60 msgr2=0x7feefc103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:13.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.148+0000 7fef008e7700 1 -- 172.21.15.33:0/3610476744 shutdown_connections 2024-05-08T21:11:13.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.148+0000 7fef008e7700 1 -- 172.21.15.33:0/3610476744 wait complete. 2024-05-08T21:11:13.619 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.148+0000 7fef008e7700 1 Processor -- start 2024-05-08T21:11:13.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.149+0000 7fef008e7700 1 -- start start 2024-05-08T21:11:13.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.149+0000 7fef008e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 0x7feefc19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:13.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.149+0000 7fef008e7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc19c850 con 0x7feefc07beb0 2024-05-08T21:11:13.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.149+0000 7feefa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 0x7feefc19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:13.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.149+0000 7feefa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 0x7feefc19c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50924/0 (socket says 172.21.15.33:50924) 2024-05-08T21:11:13.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.149+0000 7feefa59c700 1 -- 172.21.15.33:0/201250719 learned_addr learned my addr 172.21.15.33:0/201250719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:13.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.149+0000 7feefa59c700 1 -- 172.21.15.33:0/201250719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef0009550 con 0x7feefc07beb0 2024-05-08T21:11:13.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.149+0000 7feefa59c700 1 --2- 172.21.15.33:0/201250719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 0x7feefc19c310 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feef0003f20 tx=0x7feef0011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:13.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.150+0000 7feeeb7fe700 1 -- 172.21.15.33:0/201250719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feef00036a0 con 0x7feefc07beb0 2024-05-08T21:11:13.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.150+0000 7feeeb7fe700 1 -- 172.21.15.33:0/201250719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7feef0003c70 con 0x7feefc07beb0 2024-05-08T21:11:13.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.150+0000 7fef008e7700 1 -- 172.21.15.33:0/201250719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc19ca50 con 0x7feefc07beb0 2024-05-08T21:11:13.620 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.150+0000 7feeeb7fe700 1 -- 172.21.15.33:0/201250719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feef001be80 con 0x7feefc07beb0 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.150+0000 7fef008e7700 1 -- 172.21.15.33:0/201250719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc19cef0 con 0x7feefc07beb0 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.150+0000 7feeeb7fe700 1 -- 172.21.15.33:0/201250719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7feef0023790 con 0x7feefc07beb0 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.150+0000 7feeeb7fe700 1 -- 172.21.15.33:0/201250719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feef002cd20 con 0x7feefc07beb0 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.151+0000 7fef008e7700 1 -- 172.21.15.33:0/201250719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc10c810 con 0x7feefc07beb0 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.153+0000 7feeeb7fe700 1 -- 172.21.15.33:0/201250719 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7feef001b440 con 0x7feefc07beb0 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.231+0000 7fef008e7700 1 -- 172.21.15.33:0/201250719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7feefc10c9d0 con 0x7feefc07beb0 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.231+0000 7feeeb7fe700 1 -- 172.21.15.33:0/201250719 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7feef002c3f0 con 0x7feefc07beb0 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.232+0000 7fef008e7700 1 -- 172.21.15.33:0/201250719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 msgr2=0x7feefc19c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.232+0000 7fef008e7700 1 --2- 172.21.15.33:0/201250719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 0x7feefc19c310 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feef0003f20 tx=0x7feef0011f00 comp rx=0 tx=0).stop 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.232+0000 7fef008e7700 1 -- 172.21.15.33:0/201250719 shutdown_connections 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.232+0000 7fef008e7700 1 --2- 172.21.15.33:0/201250719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feefc07beb0 0x7feefc19c310 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.232+0000 7fef008e7700 1 -- 172.21.15.33:0/201250719 >> 172.21.15.33:0/201250719 conn(0x7feefc101a60 msgr2=0x7feefc102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.232+0000 7fef008e7700 1 -- 172.21.15.33:0/201250719 shutdown_connections 2024-05-08T21:11:13.621 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:13.232+0000 7fef008e7700 1 -- 172.21.15.33:0/201250719 wait complete. 2024-05-08T21:11:13.622 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (1/15)... 2024-05-08T21:11:13.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:13 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/201250719' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T21:11:13.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:13 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:13.491+0000 7f16218ba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:11:14.525 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:14 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:14.238+0000 7f16218ba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:11:14.927 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:14 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:14.522+0000 7f16218ba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:11:14.927 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:14 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:14.620+0000 7f16218ba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:11:14.927 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:14 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:14.743+0000 7f16218ba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:11:14.927 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:14 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:14.923+0000 7f16218ba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:11:15.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:15 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:15.069+0000 7f16218ba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:11:15.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:15 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:15.165+0000 7f16218ba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:11:15.958 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:15 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:15.591+0000 7f16218ba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:11:16.395 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:16 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4090217194' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T21:11:16.395 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:16 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:16.069+0000 7f16218ba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:11:16.395 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:16 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:16.297+0000 7f16218ba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:11:16.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:16.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:11:16.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:11:16.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T21:11:16.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T21:11:16.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T21:11:16.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T21:11:16.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:16.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T21:11:16.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T21:11:16.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-05-08T21:11:16.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:11:16.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T21:11:16.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T21:11:16.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:11:16.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-05-08T21:11:16.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T21:11:16.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:16.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T21:11:16.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T21:11:16.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:16.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T21:11:16.463 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:16.463 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T21:11:16.463 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T21:11:16.463 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T21:11:16.463 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T21:11:16.463 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T21:11:16.463 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T21:11:16.463 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:16.463 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T21:11:16.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T21:11:07.209073+0000", 2024-05-08T21:11:16.465 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:11:16.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:16.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T21:11:16.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:11:16.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.996+0000 7f06691f2700 1 Processor -- start 2024-05-08T21:11:16.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.996+0000 7f06691f2700 1 -- start start 2024-05-08T21:11:16.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.997+0000 7f06691f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 0x7f0664109410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:16.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.997+0000 7f06691f2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0664079e20 con 0x7f0664107020 2024-05-08T21:11:16.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.997+0000 7f0662d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 0x7f0664109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:16.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.997+0000 7f0662d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 0x7f0664109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50930/0 (socket says 172.21.15.33:50930) 2024-05-08T21:11:16.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.997+0000 7f0662d9d700 1 -- 172.21.15.33:0/3452603404 learned_addr learned my addr 172.21.15.33:0/3452603404 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:16.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.997+0000 7f0662d9d700 1 -- 172.21.15.33:0/3452603404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0664109950 con 0x7f0664107020 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.997+0000 7f0662d9d700 1 --2- 172.21.15.33:0/3452603404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 0x7f0664109410 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f06540098a0 tx=0x7f0654009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d876b32d509f2b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.997+0000 7f0661d9b700 1 -- 172.21.15.33:0/3452603404 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0654004160 con 0x7f0664107020 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.998+0000 7f0661d9b700 1 -- 172.21.15.33:0/3452603404 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f06540042c0 con 0x7f0664107020 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.998+0000 7f0661d9b700 1 -- 172.21.15.33:0/3452603404 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0654011940 con 0x7f0664107020 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.998+0000 7f06691f2700 1 -- 172.21.15.33:0/3452603404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 msgr2=0x7f0664109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.998+0000 7f06691f2700 1 --2- 172.21.15.33:0/3452603404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 0x7f0664109410 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f06540098a0 tx=0x7f0654009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.998+0000 7f06691f2700 1 -- 172.21.15.33:0/3452603404 shutdown_connections 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.998+0000 7f06691f2700 1 --2- 172.21.15.33:0/3452603404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 0x7f0664109410 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.998+0000 7f06691f2700 1 -- 172.21.15.33:0/3452603404 >> 172.21.15.33:0/3452603404 conn(0x7f0664100a10 msgr2=0x7f0664102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.998+0000 7f06691f2700 1 -- 172.21.15.33:0/3452603404 shutdown_connections 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.998+0000 7f06691f2700 1 -- 172.21.15.33:0/3452603404 wait complete. 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.999+0000 7f06691f2700 1 Processor -- start 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:15.999+0000 7f06691f2700 1 -- start start 2024-05-08T21:11:16.467 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.000+0000 7f06691f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 0x7f066419c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.000+0000 7f06691f2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066419c8b0 con 0x7f0664107020 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.000+0000 7f0662d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 0x7f066419c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.000+0000 7f0662d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 0x7f066419c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50946/0 (socket says 172.21.15.33:50946) 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.000+0000 7f0662d9d700 1 -- 172.21.15.33:0/4090217194 learned_addr learned my addr 172.21.15.33:0/4090217194 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.000+0000 7f0662d9d700 1 -- 172.21.15.33:0/4090217194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0654009550 con 0x7f0664107020 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.000+0000 7f0662d9d700 1 --2- 172.21.15.33:0/4090217194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 0x7f066419c370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0654003f20 tx=0x7f0654011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.000+0000 7f064bfff700 1 -- 172.21.15.33:0/4090217194 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06540037d0 con 0x7f0664107020 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.001+0000 7f064bfff700 1 -- 172.21.15.33:0/4090217194 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f065401b440 con 0x7f0664107020 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.001+0000 7f06691f2700 1 -- 172.21.15.33:0/4090217194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066419cab0 con 0x7f0664107020 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.001+0000 7f064bfff700 1 -- 172.21.15.33:0/4090217194 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0654023590 con 0x7f0664107020 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.001+0000 7f06691f2700 1 -- 172.21.15.33:0/4090217194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066419cf50 con 0x7f0664107020 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.001+0000 7f064bfff700 1 -- 172.21.15.33:0/4090217194 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0654023b00 con 0x7f0664107020 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.001+0000 7f064bfff700 1 -- 172.21.15.33:0/4090217194 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f065402cdb0 con 0x7f0664107020 2024-05-08T21:11:16.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.002+0000 7f06691f2700 1 -- 172.21.15.33:0/4090217194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066410a5d0 con 0x7f0664107020 2024-05-08T21:11:16.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.004+0000 7f064bfff700 1 -- 172.21.15.33:0/4090217194 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f065401b5b0 con 0x7f0664107020 2024-05-08T21:11:16.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.082+0000 7f06691f2700 1 -- 172.21.15.33:0/4090217194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f066410a790 con 0x7f0664107020 2024-05-08T21:11:16.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.083+0000 7f064bfff700 1 -- 172.21.15.33:0/4090217194 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f065401b750 con 0x7f0664107020 2024-05-08T21:11:16.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.084+0000 7f06691f2700 1 -- 172.21.15.33:0/4090217194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 msgr2=0x7f066419c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:16.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.084+0000 7f06691f2700 1 --2- 172.21.15.33:0/4090217194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 0x7f066419c370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0654003f20 tx=0x7f0654011f00 comp rx=0 tx=0).stop 2024-05-08T21:11:16.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.084+0000 7f06691f2700 1 -- 172.21.15.33:0/4090217194 shutdown_connections 2024-05-08T21:11:16.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.084+0000 7f06691f2700 1 --2- 172.21.15.33:0/4090217194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107020 0x7f066419c370 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:16.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.084+0000 7f06691f2700 1 -- 172.21.15.33:0/4090217194 >> 172.21.15.33:0/4090217194 conn(0x7f0664100a10 msgr2=0x7f0664101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:16.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.084+0000 7f06691f2700 1 -- 172.21.15.33:0/4090217194 shutdown_connections 2024-05-08T21:11:16.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:16.084+0000 7f06691f2700 1 -- 172.21.15.33:0/4090217194 wait complete. 2024-05-08T21:11:16.469 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (2/15)... 2024-05-08T21:11:16.708 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:16 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:16.404+0000 7f16218ba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:11:17.566 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:17 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:17.284+0000 7f16218ba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:11:17.566 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:17 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:17.393+0000 7f16218ba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:11:17.958 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:17 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:17.563+0000 7f16218ba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:11:18.568 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:18 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:18.287+0000 7f16218ba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:11:18.568 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:18 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:18.497+0000 7f16218ba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:11:18.915 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:18 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:18.726+0000 7f16218ba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:11:19.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:18 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3146907961' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T21:11:19.242 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:19.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:11:19.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:11:19.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T21:11:19.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T21:11:19.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T21:11:19.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T21:11:19.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:19.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T21:11:19.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T21:11:19.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-05-08T21:11:19.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:11:19.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T21:11:19.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T21:11:19.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:11:19.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-08T21:11:19.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T21:11:19.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:19.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T21:11:19.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T21:11:19.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:19.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T21:11:19.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:19.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T21:11:19.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T21:11:19.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T21:11:19.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T21:11:19.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T21:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T21:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T21:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T21:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T21:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T21:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T21:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T21:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T21:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T21:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T21:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:11:19.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T21:11:07.209073+0000", 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.800+0000 7f920cdae700 1 Processor -- start 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.800+0000 7f920cdae700 1 -- start start 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.800+0000 7f920cdae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 0x7f9208106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.800+0000 7f920cdae700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9208106f10 con 0x7f9208106520 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.801+0000 7f920659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 0x7f9208106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.801+0000 7f920659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 0x7f9208106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50952/0 (socket says 172.21.15.33:50952) 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.801+0000 7f920659c700 1 -- 172.21.15.33:0/2538283578 learned_addr learned my addr 172.21.15.33:0/2538283578 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.801+0000 7f920659c700 1 -- 172.21.15.33:0/2538283578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9208107720 con 0x7f9208106520 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.801+0000 7f920659c700 1 --2- 172.21.15.33:0/2538283578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 0x7f9208106940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91f80098a0 tx=0x7f91f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a050949596a194 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.802+0000 7f920559a700 1 -- 172.21.15.33:0/2538283578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91f8004160 con 0x7f9208106520 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.802+0000 7f920559a700 1 -- 172.21.15.33:0/2538283578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f91f80042c0 con 0x7f9208106520 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.802+0000 7f920559a700 1 -- 172.21.15.33:0/2538283578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91f8011940 con 0x7f9208106520 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.802+0000 7f920cdae700 1 -- 172.21.15.33:0/2538283578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 msgr2=0x7f9208106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.802+0000 7f920cdae700 1 --2- 172.21.15.33:0/2538283578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 0x7f9208106940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f91f80098a0 tx=0x7f91f8009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.803+0000 7f920cdae700 1 -- 172.21.15.33:0/2538283578 shutdown_connections 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.803+0000 7f920cdae700 1 --2- 172.21.15.33:0/2538283578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 0x7f9208106940 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.803+0000 7f920cdae700 1 -- 172.21.15.33:0/2538283578 >> 172.21.15.33:0/2538283578 conn(0x7f9208101b00 msgr2=0x7f9208103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.803+0000 7f920cdae700 1 -- 172.21.15.33:0/2538283578 shutdown_connections 2024-05-08T21:11:19.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.803+0000 7f920cdae700 1 -- 172.21.15.33:0/2538283578 wait complete. 2024-05-08T21:11:19.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.803+0000 7f920cdae700 1 Processor -- start 2024-05-08T21:11:19.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.804+0000 7f920cdae700 1 -- start start 2024-05-08T21:11:19.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.804+0000 7f920cdae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 0x7f920807bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:19.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.804+0000 7f920cdae700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920807c050 con 0x7f9208106520 2024-05-08T21:11:19.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.804+0000 7f920659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 0x7f920807bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:19.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.804+0000 7f920659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 0x7f920807bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50954/0 (socket says 172.21.15.33:50954) 2024-05-08T21:11:19.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.804+0000 7f920659c700 1 -- 172.21.15.33:0/3146907961 learned_addr learned my addr 172.21.15.33:0/3146907961 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:19.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.804+0000 7f920659c700 1 -- 172.21.15.33:0/3146907961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f8009550 con 0x7f9208106520 2024-05-08T21:11:19.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.804+0000 7f920659c700 1 --2- 172.21.15.33:0/3146907961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 0x7f920807bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91f8006af0 tx=0x7f91f8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:19.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.805+0000 7f91f77fe700 1 -- 172.21.15.33:0/3146907961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91f8003bf0 con 0x7f9208106520 2024-05-08T21:11:19.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.805+0000 7f91f77fe700 1 -- 172.21.15.33:0/3146907961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f91f801b8b0 con 0x7f9208106520 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.805+0000 7f920cdae700 1 -- 172.21.15.33:0/3146907961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920807a290 con 0x7f9208106520 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.805+0000 7f91f77fe700 1 -- 172.21.15.33:0/3146907961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91f8023830 con 0x7f9208106520 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.805+0000 7f920cdae700 1 -- 172.21.15.33:0/3146907961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920807a730 con 0x7f9208106520 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.805+0000 7f91f77fe700 1 -- 172.21.15.33:0/3146907961 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f91f802c3f0 con 0x7f9208106520 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.805+0000 7f91f77fe700 1 -- 172.21.15.33:0/3146907961 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f91f8019710 con 0x7f9208106520 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.806+0000 7f920cdae700 1 -- 172.21.15.33:0/3146907961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92080625d0 con 0x7f9208106520 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.808+0000 7f91f77fe700 1 -- 172.21.15.33:0/3146907961 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73121 (secure 0 0 0) 0x7f91f801bd70 con 0x7f9208106520 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.888+0000 7f920cdae700 1 -- 172.21.15.33:0/3146907961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f920807ab50 con 0x7f9208106520 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.888+0000 7f91f77fe700 1 -- 172.21.15.33:0/3146907961 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f91f801bad0 con 0x7f9208106520 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.889+0000 7f920cdae700 1 -- 172.21.15.33:0/3146907961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 msgr2=0x7f920807bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.889+0000 7f920cdae700 1 --2- 172.21.15.33:0/3146907961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 0x7f920807bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91f8006af0 tx=0x7f91f8011a50 comp rx=0 tx=0).stop 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.889+0000 7f920cdae700 1 -- 172.21.15.33:0/3146907961 shutdown_connections 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.889+0000 7f920cdae700 1 --2- 172.21.15.33:0/3146907961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9208106520 0x7f920807bb10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:19.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.889+0000 7f920cdae700 1 -- 172.21.15.33:0/3146907961 >> 172.21.15.33:0/3146907961 conn(0x7f9208101b00 msgr2=0x7f9208102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:19.264 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.889+0000 7f920cdae700 1 -- 172.21.15.33:0/3146907961 shutdown_connections 2024-05-08T21:11:19.264 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:18.889+0000 7f920cdae700 1 -- 172.21.15.33:0/3146907961 wait complete. 2024-05-08T21:11:19.264 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (3/15)... 2024-05-08T21:11:20.123 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:19 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:19.800+0000 7f16218ba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:11:20.436 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:20 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:20.120+0000 7f16218ba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:11:20.436 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:20 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:20.219+0000 7f16218ba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:11:20.708 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:20 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:20.433+0000 7f16218ba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:11:20.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: Activating manager daemon x 2024-05-08T21:11:20.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: mgrmap e2: x(active, starting, since 0.00329959s) 2024-05-08T21:11:20.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: from='mgr.14100 172.21.15.33:0/3058488044' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T21:11:20.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: from='mgr.14100 172.21.15.33:0/3058488044' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T21:11:20.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: from='mgr.14100 172.21.15.33:0/3058488044' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T21:11:20.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: from='mgr.14100 172.21.15.33:0/3058488044' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:11:20.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: from='mgr.14100 172.21.15.33:0/3058488044' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T21:11:20.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: Manager daemon x is now available 2024-05-08T21:11:20.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: from='mgr.14100 172.21.15.33:0/3058488044' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T21:11:20.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: from='mgr.14100 172.21.15.33:0/3058488044' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T21:11:20.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: from='mgr.14100 172.21.15.33:0/3058488044' entity='mgr.x' 2024-05-08T21:11:20.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: from='mgr.14100 172.21.15.33:0/3058488044' entity='mgr.x' 2024-05-08T21:11:20.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:20 smithi033 ceph-mon[42528]: from='mgr.14100 172.21.15.33:0/3058488044' entity='mgr.x' 2024-05-08T21:11:22.287 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:22.287 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:11:22.287 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T21:11:22.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:22.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-05-08T21:11:22.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T21:11:22.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:22.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T21:11:22.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:22.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T21:11:22.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T21:11:22.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T21:11:22.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T21:11:22.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T21:11:22.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T21:11:22.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:22.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T21:11:22.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T21:11:22.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T21:11:22.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T21:11:22.299 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T21:11:22.300 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T21:11:22.300 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T21:11:22.300 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T21:11:22.300 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T21:11:22.300 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:22.300 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T21:11:22.300 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:22.300 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T21:11:22.300 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T21:11:22.300 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T21:11:07.209073+0000", 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T21:11:22.301 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.619+0000 7efedfa69700 1 Processor -- start 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.619+0000 7efedfa69700 1 -- start start 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.620+0000 7efedfa69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 0x7efed8109450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.620+0000 7efedfa69700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed8079de0 con 0x7efed8107060 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.620+0000 7efedd805700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 0x7efed8109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.620+0000 7efedd805700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 0x7efed8109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35140/0 (socket says 172.21.15.33:35140) 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.620+0000 7efedd805700 1 -- 172.21.15.33:0/2479627681 learned_addr learned my addr 172.21.15.33:0/2479627681 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.620+0000 7efedd805700 1 -- 172.21.15.33:0/2479627681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed8079f20 con 0x7efed8107060 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.620+0000 7efedd805700 1 --2- 172.21.15.33:0/2479627681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 0x7efed8109450 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efed40098a0 tx=0x7efed4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75598890bcf9ff8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.621+0000 7efedc803700 1 -- 172.21.15.33:0/2479627681 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efed4004160 con 0x7efed8107060 2024-05-08T21:11:22.302 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.621+0000 7efedc803700 1 -- 172.21.15.33:0/2479627681 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7efed40042c0 con 0x7efed8107060 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.621+0000 7efedc803700 1 -- 172.21.15.33:0/2479627681 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efed4011940 con 0x7efed8107060 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.621+0000 7efedfa69700 1 -- 172.21.15.33:0/2479627681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 msgr2=0x7efed8109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.621+0000 7efedfa69700 1 --2- 172.21.15.33:0/2479627681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 0x7efed8109450 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efed40098a0 tx=0x7efed4009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.621+0000 7efedfa69700 1 -- 172.21.15.33:0/2479627681 shutdown_connections 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.621+0000 7efedfa69700 1 --2- 172.21.15.33:0/2479627681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 0x7efed8109450 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.621+0000 7efedfa69700 1 -- 172.21.15.33:0/2479627681 >> 172.21.15.33:0/2479627681 conn(0x7efed8100a10 msgr2=0x7efed8102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.621+0000 7efedfa69700 1 -- 172.21.15.33:0/2479627681 shutdown_connections 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.621+0000 7efedfa69700 1 -- 172.21.15.33:0/2479627681 wait complete. 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.622+0000 7efedfa69700 1 Processor -- start 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.622+0000 7efedfa69700 1 -- start start 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.622+0000 7efedfa69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 0x7efed819a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.622+0000 7efedfa69700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed819a680 con 0x7efed8107060 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.622+0000 7efedd805700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 0x7efed819a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.622+0000 7efedd805700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 0x7efed819a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35156/0 (socket says 172.21.15.33:35156) 2024-05-08T21:11:22.303 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.622+0000 7efedd805700 1 -- 172.21.15.33:0/1200254606 learned_addr learned my addr 172.21.15.33:0/1200254606 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.623+0000 7efedd805700 1 -- 172.21.15.33:0/1200254606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed4009550 con 0x7efed8107060 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.623+0000 7efedd805700 1 --2- 172.21.15.33:0/1200254606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 0x7efed819a140 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efed4003f20 tx=0x7efed4011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.623+0000 7efeceffd700 1 -- 172.21.15.33:0/1200254606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efed40037d0 con 0x7efed8107060 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.623+0000 7efeceffd700 1 -- 172.21.15.33:0/1200254606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7efed401b440 con 0x7efed8107060 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.623+0000 7efedfa69700 1 -- 172.21.15.33:0/1200254606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed819a880 con 0x7efed8107060 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.623+0000 7efeceffd700 1 -- 172.21.15.33:0/1200254606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efed4023590 con 0x7efed8107060 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.623+0000 7efedfa69700 1 -- 172.21.15.33:0/1200254606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed819ad20 con 0x7efed8107060 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.624+0000 7efeceffd700 1 -- 172.21.15.33:0/1200254606 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7efed4023800 con 0x7efed8107060 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.624+0000 7efedfa69700 1 -- 172.21.15.33:0/1200254606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed810a5f0 con 0x7efed8107060 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.624+0000 7efeceffd700 1 --2- 172.21.15.33:0/1200254606 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7efec40385d0 0x7efec403aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.624+0000 7efeceffd700 1 -- 172.21.15.33:0/1200254606 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efed404c510 con 0x7efed8107060 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.625+0000 7efedd004700 1 --2- 172.21.15.33:0/1200254606 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7efec40385d0 0x7efec403aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.625+0000 7efedd004700 1 --2- 172.21.15.33:0/1200254606 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7efec40385d0 0x7efec403aa90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efec8006fd0 tx=0x7efec8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.630+0000 7efeceffd700 1 -- 172.21.15.33:0/1200254606 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efed402b8b0 con 0x7efed8107060 2024-05-08T21:11:22.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.928+0000 7efedfa69700 1 -- 172.21.15.33:0/1200254606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efed810a830 con 0x7efed8107060 2024-05-08T21:11:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.929+0000 7efeceffd700 1 -- 172.21.15.33:0/1200254606 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7efed40314f0 con 0x7efed8107060 2024-05-08T21:11:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.931+0000 7efedfa69700 1 -- 172.21.15.33:0/1200254606 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7efec40385d0 msgr2=0x7efec403aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.931+0000 7efedfa69700 1 --2- 172.21.15.33:0/1200254606 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7efec40385d0 0x7efec403aa90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efec8006fd0 tx=0x7efec8006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.931+0000 7efedfa69700 1 -- 172.21.15.33:0/1200254606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 msgr2=0x7efed819a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.931+0000 7efedfa69700 1 --2- 172.21.15.33:0/1200254606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 0x7efed819a140 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efed4003f20 tx=0x7efed4011f00 comp rx=0 tx=0).stop 2024-05-08T21:11:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.931+0000 7efedfa69700 1 -- 172.21.15.33:0/1200254606 shutdown_connections 2024-05-08T21:11:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.931+0000 7efedfa69700 1 --2- 172.21.15.33:0/1200254606 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7efec40385d0 0x7efec403aa90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.931+0000 7efedfa69700 1 --2- 172.21.15.33:0/1200254606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efed8107060 0x7efed819a140 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.931+0000 7efedfa69700 1 -- 172.21.15.33:0/1200254606 >> 172.21.15.33:0/1200254606 conn(0x7efed8100a10 msgr2=0x7efed81014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.931+0000 7efedfa69700 1 -- 172.21.15.33:0/1200254606 shutdown_connections 2024-05-08T21:11:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:21.931+0000 7efedfa69700 1 -- 172.21.15.33:0/1200254606 wait complete. 2024-05-08T21:11:22.305 INFO:teuthology.orchestra.run.smithi033.stdout:mgr is available 2024-05-08T21:11:22.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:22 smithi033 ceph-mon[42528]: mgrmap e3: x(active, since 1.00775s) 2024-05-08T21:11:22.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:22 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1200254606' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T21:11:23.274 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:23.274 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T21:11:23.274 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:11:23.274 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T21:11:23.274 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T21:11:23.274 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T21:11:23.274 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T21:11:23.274 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:23.274 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.654+0000 7ff68e4e1700 1 Processor -- start 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.654+0000 7ff68e4e1700 1 -- start start 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.655+0000 7ff68e4e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 0x7ff688109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.655+0000 7ff68e4e1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff688079ea0 con 0x7ff688106fa0 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.655+0000 7ff687fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 0x7ff688109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:23.275 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.655+0000 7ff687fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 0x7ff688109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35168/0 (socket says 172.21.15.33:35168) 2024-05-08T21:11:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.655+0000 7ff687fff700 1 -- 172.21.15.33:0/2414163462 learned_addr learned my addr 172.21.15.33:0/2414163462 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.655+0000 7ff687fff700 1 -- 172.21.15.33:0/2414163462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6881098d0 con 0x7ff688106fa0 2024-05-08T21:11:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.655+0000 7ff687fff700 1 --2- 172.21.15.33:0/2414163462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 0x7ff688109390 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff6780098a0 tx=0x7ff678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84e130d1a54e92e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.656+0000 7ff686ffd700 1 -- 172.21.15.33:0/2414163462 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff678004160 con 0x7ff688106fa0 2024-05-08T21:11:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.656+0000 7ff686ffd700 1 -- 172.21.15.33:0/2414163462 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff6780042c0 con 0x7ff688106fa0 2024-05-08T21:11:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.656+0000 7ff686ffd700 1 -- 172.21.15.33:0/2414163462 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff678011940 con 0x7ff688106fa0 2024-05-08T21:11:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.656+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2414163462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 msgr2=0x7ff688109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.656+0000 7ff68e4e1700 1 --2- 172.21.15.33:0/2414163462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 0x7ff688109390 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff6780098a0 tx=0x7ff678009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.656+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2414163462 shutdown_connections 2024-05-08T21:11:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.656+0000 7ff68e4e1700 1 --2- 172.21.15.33:0/2414163462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 0x7ff688109390 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:23.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.656+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2414163462 >> 172.21.15.33:0/2414163462 conn(0x7ff688100a10 msgr2=0x7ff688102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:23.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.656+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2414163462 shutdown_connections 2024-05-08T21:11:23.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.656+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2414163462 wait complete. 2024-05-08T21:11:23.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.657+0000 7ff68e4e1700 1 Processor -- start 2024-05-08T21:11:23.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.657+0000 7ff68e4e1700 1 -- start start 2024-05-08T21:11:23.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.657+0000 7ff68e4e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 0x7ff68819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:23.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.657+0000 7ff68e4e1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68819c870 con 0x7ff688106fa0 2024-05-08T21:11:23.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.658+0000 7ff687fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 0x7ff68819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:23.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.658+0000 7ff687fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 0x7ff68819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35178/0 (socket says 172.21.15.33:35178) 2024-05-08T21:11:23.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.658+0000 7ff687fff700 1 -- 172.21.15.33:0/2152540817 learned_addr learned my addr 172.21.15.33:0/2152540817 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:23.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.658+0000 7ff687fff700 1 -- 172.21.15.33:0/2152540817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff678009550 con 0x7ff688106fa0 2024-05-08T21:11:23.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.658+0000 7ff687fff700 1 --2- 172.21.15.33:0/2152540817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 0x7ff68819c330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff678003f20 tx=0x7ff678011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:23.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.658+0000 7ff6857fa700 1 -- 172.21.15.33:0/2152540817 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6780037d0 con 0x7ff688106fa0 2024-05-08T21:11:23.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.658+0000 7ff6857fa700 1 -- 172.21.15.33:0/2152540817 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7ff67801b440 con 0x7ff688106fa0 2024-05-08T21:11:23.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.658+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2152540817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68819ca70 con 0x7ff688106fa0 2024-05-08T21:11:23.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.658+0000 7ff6857fa700 1 -- 172.21.15.33:0/2152540817 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff678023590 con 0x7ff688106fa0 2024-05-08T21:11:23.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.658+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2152540817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68819cf10 con 0x7ff688106fa0 2024-05-08T21:11:23.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.659+0000 7ff6857fa700 1 -- 172.21.15.33:0/2152540817 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 3) v1 ==== 45424+0+0 (secure 0 0 0) 0x7ff678023850 con 0x7ff688106fa0 2024-05-08T21:11:23.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.659+0000 7ff6857fa700 1 --2- 172.21.15.33:0/2152540817 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7ff670038580 0x7ff67003aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:23.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.659+0000 7ff6857fa700 1 -- 172.21.15.33:0/2152540817 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff67804c610 con 0x7ff688106fa0 2024-05-08T21:11:23.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.659+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2152540817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68810a600 con 0x7ff688106fa0 2024-05-08T21:11:23.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.659+0000 7ff6877fe700 1 --2- 172.21.15.33:0/2152540817 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7ff670038580 0x7ff67003aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:23.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.660+0000 7ff6877fe700 1 --2- 172.21.15.33:0/2152540817 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7ff670038580 0x7ff67003aa40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff67c006fd0 tx=0x7ff67c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:23.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.665+0000 7ff6857fa700 1 -- 172.21.15.33:0/2152540817 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff678023b00 con 0x7ff688106fa0 2024-05-08T21:11:23.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.875+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2152540817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff68819f7a0 con 0x7ff688106fa0 2024-05-08T21:11:23.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.876+0000 7ff6857fa700 1 -- 172.21.15.33:0/2152540817 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7ff67802bb40 con 0x7ff688106fa0 2024-05-08T21:11:23.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.879+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2152540817 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7ff670038580 msgr2=0x7ff67003aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:23.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.879+0000 7ff68e4e1700 1 --2- 172.21.15.33:0/2152540817 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7ff670038580 0x7ff67003aa40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff67c006fd0 tx=0x7ff67c006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:23.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.879+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2152540817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 msgr2=0x7ff68819c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:23.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.879+0000 7ff68e4e1700 1 --2- 172.21.15.33:0/2152540817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 0x7ff68819c330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff678003f20 tx=0x7ff678011f00 comp rx=0 tx=0).stop 2024-05-08T21:11:23.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.879+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2152540817 shutdown_connections 2024-05-08T21:11:23.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.879+0000 7ff68e4e1700 1 --2- 172.21.15.33:0/2152540817 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7ff670038580 0x7ff67003aa40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:23.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.879+0000 7ff68e4e1700 1 --2- 172.21.15.33:0/2152540817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688106fa0 0x7ff68819c330 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:23.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.879+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2152540817 >> 172.21.15.33:0/2152540817 conn(0x7ff688100a10 msgr2=0x7ff688101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:23.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.879+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2152540817 shutdown_connections 2024-05-08T21:11:23.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:22.879+0000 7ff68e4e1700 1 -- 172.21.15.33:0/2152540817 wait complete. 2024-05-08T21:11:23.280 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling cephadm module... 2024-05-08T21:11:23.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:23 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2152540817' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T21:11:24.708 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:24 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: ignoring --setuser ceph since I am not root 2024-05-08T21:11:24.708 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:24 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: ignoring --setgroup ceph since I am not root 2024-05-08T21:11:24.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:24 smithi033 ceph-mon[42528]: mgrmap e4: x(active, since 3s) 2024-05-08T21:11:24.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:24 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2537599484' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-08T21:11:24.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.637+0000 7f00fbe50700 1 Processor -- start 2024-05-08T21:11:24.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.637+0000 7f00fbe50700 1 -- start start 2024-05-08T21:11:24.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.637+0000 7f00fbe50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 0x7f00f41093a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:24.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.637+0000 7f00fbe50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f4079ea0 con 0x7f00f4106fb0 2024-05-08T21:11:24.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.637+0000 7f00f9bec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 0x7f00f41093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:24.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.638+0000 7f00f9bec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 0x7f00f41093a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35186/0 (socket says 172.21.15.33:35186) 2024-05-08T21:11:24.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.638+0000 7f00f9bec700 1 -- 172.21.15.33:0/1966006963 learned_addr learned my addr 172.21.15.33:0/1966006963 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:24.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.638+0000 7f00f9bec700 1 -- 172.21.15.33:0/1966006963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f41098e0 con 0x7f00f4106fb0 2024-05-08T21:11:24.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.638+0000 7f00f9bec700 1 --2- 172.21.15.33:0/1966006963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 0x7f00f41093a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f00f00098a0 tx=0x7f00f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d119dac8ac8917c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:24.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.638+0000 7f00f8bea700 1 -- 172.21.15.33:0/1966006963 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00f0004160 con 0x7f00f4106fb0 2024-05-08T21:11:24.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.638+0000 7f00f8bea700 1 -- 172.21.15.33:0/1966006963 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f00f00042c0 con 0x7f00f4106fb0 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.638+0000 7f00f8bea700 1 -- 172.21.15.33:0/1966006963 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00f0011940 con 0x7f00f4106fb0 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.639+0000 7f00fbe50700 1 -- 172.21.15.33:0/1966006963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 msgr2=0x7f00f41093a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.639+0000 7f00fbe50700 1 --2- 172.21.15.33:0/1966006963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 0x7f00f41093a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f00f00098a0 tx=0x7f00f0009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.639+0000 7f00fbe50700 1 -- 172.21.15.33:0/1966006963 shutdown_connections 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.639+0000 7f00fbe50700 1 --2- 172.21.15.33:0/1966006963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 0x7f00f41093a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.639+0000 7f00fbe50700 1 -- 172.21.15.33:0/1966006963 >> 172.21.15.33:0/1966006963 conn(0x7f00f4100a00 msgr2=0x7f00f4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.639+0000 7f00fbe50700 1 -- 172.21.15.33:0/1966006963 shutdown_connections 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.639+0000 7f00fbe50700 1 -- 172.21.15.33:0/1966006963 wait complete. 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.640+0000 7f00fbe50700 1 Processor -- start 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.640+0000 7f00fbe50700 1 -- start start 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.640+0000 7f00fbe50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 0x7f00f419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.640+0000 7f00fbe50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f419c870 con 0x7f00f4106fb0 2024-05-08T21:11:24.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.640+0000 7f00f9bec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 0x7f00f419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.640+0000 7f00f9bec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 0x7f00f419c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35198/0 (socket says 172.21.15.33:35198) 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.640+0000 7f00f9bec700 1 -- 172.21.15.33:0/2537599484 learned_addr learned my addr 172.21.15.33:0/2537599484 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.640+0000 7f00f9bec700 1 -- 172.21.15.33:0/2537599484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f0009550 con 0x7f00f4106fb0 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.640+0000 7f00f9bec700 1 --2- 172.21.15.33:0/2537599484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 0x7f00f419c330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f00f0003f20 tx=0x7f00f0011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.641+0000 7f00eaffd700 1 -- 172.21.15.33:0/2537599484 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00f00037d0 con 0x7f00f4106fb0 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.641+0000 7f00fbe50700 1 -- 172.21.15.33:0/2537599484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f419ca70 con 0x7f00f4106fb0 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.641+0000 7f00eaffd700 1 -- 172.21.15.33:0/2537599484 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f00f001b440 con 0x7f00f4106fb0 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.641+0000 7f00eaffd700 1 -- 172.21.15.33:0/2537599484 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00f0023590 con 0x7f00f4106fb0 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.641+0000 7f00fbe50700 1 -- 172.21.15.33:0/2537599484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f419cf10 con 0x7f00f4106fb0 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.642+0000 7f00eaffd700 1 -- 172.21.15.33:0/2537599484 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 45530+0+0 (secure 0 0 0) 0x7f00f0023850 con 0x7f00f4106fb0 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.642+0000 7f00eaffd700 1 --2- 172.21.15.33:0/2537599484 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f00e00386b0 0x7f00e003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.642+0000 7f00eaffd700 1 -- 172.21.15.33:0/2537599484 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f00f004c5e0 con 0x7f00f4106fb0 2024-05-08T21:11:24.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.642+0000 7f00fbe50700 1 -- 172.21.15.33:0/2537599484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f410a610 con 0x7f00f4106fb0 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.642+0000 7f00f93eb700 1 --2- 172.21.15.33:0/2537599484 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f00e00386b0 0x7f00e003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.642+0000 7f00f93eb700 1 --2- 172.21.15.33:0/2537599484 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f00e00386b0 0x7f00e003ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f00e4006fd0 tx=0x7f00e4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.648+0000 7f00eaffd700 1 -- 172.21.15.33:0/2537599484 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f00f0023b00 con 0x7f00f4106fb0 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:23.922+0000 7f00fbe50700 1 -- 172.21.15.33:0/2537599484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f00f410a7d0 con 0x7f00f4106fb0 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:24.458+0000 7f00eaffd700 1 -- 172.21.15.33:0/2537599484 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f00f0023ce0 con 0x7f00f4106fb0 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:24.459+0000 7f00eaffd700 1 -- 172.21.15.33:0/2537599484 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f00f004c000 con 0x7f00f4106fb0 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:24.462+0000 7f00fbe50700 1 -- 172.21.15.33:0/2537599484 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f00e00386b0 msgr2=0x7f00e003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:24.462+0000 7f00fbe50700 1 --2- 172.21.15.33:0/2537599484 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f00e00386b0 0x7f00e003ab70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f00e4006fd0 tx=0x7f00e4006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:24.462+0000 7f00fbe50700 1 -- 172.21.15.33:0/2537599484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 msgr2=0x7f00f419c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:24.462+0000 7f00fbe50700 1 --2- 172.21.15.33:0/2537599484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 0x7f00f419c330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f00f0003f20 tx=0x7f00f0011f00 comp rx=0 tx=0).stop 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:24.462+0000 7f00fbe50700 1 -- 172.21.15.33:0/2537599484 shutdown_connections 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:24.462+0000 7f00fbe50700 1 --2- 172.21.15.33:0/2537599484 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f00e00386b0 0x7f00e003ab70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:24.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:24.462+0000 7f00fbe50700 1 --2- 172.21.15.33:0/2537599484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00f4106fb0 0x7f00f419c330 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:24.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:24.462+0000 7f00fbe50700 1 -- 172.21.15.33:0/2537599484 >> 172.21.15.33:0/2537599484 conn(0x7f00f4100a00 msgr2=0x7f00f4101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:24.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:24.463+0000 7f00fbe50700 1 -- 172.21.15.33:0/2537599484 shutdown_connections 2024-05-08T21:11:24.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:24.463+0000 7f00fbe50700 1 -- 172.21.15.33:0/2537599484 wait complete. 2024-05-08T21:11:25.081 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:24 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:24.972+0000 7f04e774c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:11:25.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:25 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:25.078+0000 7f04e774c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:11:25.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:25 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:25.179+0000 7f04e774c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:11:25.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:25 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:25.278+0000 7f04e774c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:11:25.839 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:25 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2537599484' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-08T21:11:25.839 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:25 smithi033 ceph-mon[42528]: mgrmap e5: x(active, since 4s) 2024-05-08T21:11:25.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:11:25.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-08T21:11:25.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T21:11:25.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T21:11:25.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T21:11:25.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:11:25.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.198+0000 7f9272e84700 1 Processor -- start 2024-05-08T21:11:25.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.198+0000 7f9272e84700 1 -- start start 2024-05-08T21:11:25.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.198+0000 7f9272e84700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 0x7f926c1051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:25.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.198+0000 7f9272e84700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c079e60 con 0x7f926c104dd0 2024-05-08T21:11:25.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.198+0000 7f9270c20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 0x7f926c1051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.198+0000 7f9270c20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 0x7f926c1051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35226/0 (socket says 172.21.15.33:35226) 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.198+0000 7f9270c20700 1 -- 172.21.15.33:0/674542560 learned_addr learned my addr 172.21.15.33:0/674542560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.198+0000 7f9270c20700 1 -- 172.21.15.33:0/674542560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c105730 con 0x7f926c104dd0 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.199+0000 7f9270c20700 1 --2- 172.21.15.33:0/674542560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 0x7f926c1051f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92600098a0 tx=0x7f9260009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fc6137b4b42cdb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.199+0000 7f926b7fe700 1 -- 172.21.15.33:0/674542560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9260004160 con 0x7f926c104dd0 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.199+0000 7f926b7fe700 1 -- 172.21.15.33:0/674542560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f92600042c0 con 0x7f926c104dd0 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.199+0000 7f926b7fe700 1 -- 172.21.15.33:0/674542560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9260011940 con 0x7f926c104dd0 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.200+0000 7f9272e84700 1 -- 172.21.15.33:0/674542560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 msgr2=0x7f926c1051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.200+0000 7f9272e84700 1 --2- 172.21.15.33:0/674542560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 0x7f926c1051f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92600098a0 tx=0x7f9260009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.200+0000 7f9272e84700 1 -- 172.21.15.33:0/674542560 shutdown_connections 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.200+0000 7f9272e84700 1 --2- 172.21.15.33:0/674542560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 0x7f926c1051f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.200+0000 7f9272e84700 1 -- 172.21.15.33:0/674542560 >> 172.21.15.33:0/674542560 conn(0x7f926c100a10 msgr2=0x7f926c102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.200+0000 7f9272e84700 1 -- 172.21.15.33:0/674542560 shutdown_connections 2024-05-08T21:11:25.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.200+0000 7f9272e84700 1 -- 172.21.15.33:0/674542560 wait complete. 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.200+0000 7f9272e84700 1 Processor -- start 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.200+0000 7f9272e84700 1 -- start start 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.201+0000 7f9272e84700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 0x7f926c19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.201+0000 7f9272e84700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c19c860 con 0x7f926c104dd0 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.201+0000 7f9270c20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 0x7f926c19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.201+0000 7f9270c20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 0x7f926c19c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35236/0 (socket says 172.21.15.33:35236) 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.201+0000 7f9270c20700 1 -- 172.21.15.33:0/3316350881 learned_addr learned my addr 172.21.15.33:0/3316350881 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.201+0000 7f9270c20700 1 -- 172.21.15.33:0/3316350881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9260009550 con 0x7f926c104dd0 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.201+0000 7f9270c20700 1 --2- 172.21.15.33:0/3316350881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 0x7f926c19c320 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9260003f20 tx=0x7f9260011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.201+0000 7f9269ffb700 1 -- 172.21.15.33:0/3316350881 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92600037d0 con 0x7f926c104dd0 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.201+0000 7f9269ffb700 1 -- 172.21.15.33:0/3316350881 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f926001b440 con 0x7f926c104dd0 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.201+0000 7f9272e84700 1 -- 172.21.15.33:0/3316350881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c19ca60 con 0x7f926c104dd0 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.201+0000 7f9269ffb700 1 -- 172.21.15.33:0/3316350881 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9260023590 con 0x7f926c104dd0 2024-05-08T21:11:25.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.202+0000 7f9272e84700 1 -- 172.21.15.33:0/3316350881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c19cf00 con 0x7f926c104dd0 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.203+0000 7f9269ffb700 1 -- 172.21.15.33:0/3316350881 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f9260023800 con 0x7f926c104dd0 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.203+0000 7f9272e84700 1 -- 172.21.15.33:0/3316350881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c195e20 con 0x7f926c104dd0 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.203+0000 7f9269ffb700 1 --2- 172.21.15.33:0/3316350881 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f925c038750 0x7f925c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.203+0000 7f926bfff700 1 -- 172.21.15.33:0/3316350881 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f925c038750 msgr2=0x7f925c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3475366293 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.203+0000 7f9269ffb700 1 -- 172.21.15.33:0/3316350881 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f926004c650 con 0x7f926c104dd0 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.203+0000 7f926bfff700 1 --2- 172.21.15.33:0/3316350881 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f925c038750 0x7f925c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.209+0000 7f9269ffb700 1 -- 172.21.15.33:0/3316350881 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9260023ab0 con 0x7f926c104dd0 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.404+0000 7f926bfff700 1 -- 172.21.15.33:0/3316350881 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f925c038750 msgr2=0x7f925c03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3475366293 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.404+0000 7f926bfff700 1 --2- 172.21.15.33:0/3316350881 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f925c038750 0x7f925c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.484+0000 7f9272e84700 1 -- 172.21.15.33:0/3316350881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f926c02cfa0 con 0x7f926c104dd0 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.484+0000 7f9269ffb700 1 -- 172.21.15.33:0/3316350881 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f926002bb40 con 0x7f926c104dd0 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.487+0000 7f9272e84700 1 -- 172.21.15.33:0/3316350881 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f925c038750 msgr2=0x7f925c03ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:11:25.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.487+0000 7f9272e84700 1 --2- 172.21.15.33:0/3316350881 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f925c038750 0x7f925c03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:25.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.487+0000 7f9272e84700 1 -- 172.21.15.33:0/3316350881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 msgr2=0x7f926c19c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:25.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.487+0000 7f9272e84700 1 --2- 172.21.15.33:0/3316350881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 0x7f926c19c320 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9260003f20 tx=0x7f9260011f00 comp rx=0 tx=0).stop 2024-05-08T21:11:25.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.487+0000 7f9272e84700 1 -- 172.21.15.33:0/3316350881 shutdown_connections 2024-05-08T21:11:25.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.487+0000 7f9272e84700 1 --2- 172.21.15.33:0/3316350881 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f925c038750 0x7f925c03ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:25.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.487+0000 7f9272e84700 1 --2- 172.21.15.33:0/3316350881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926c104dd0 0x7f926c19c320 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:25.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.487+0000 7f9272e84700 1 -- 172.21.15.33:0/3316350881 >> 172.21.15.33:0/3316350881 conn(0x7f926c100a10 msgr2=0x7f926c1907a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:25.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.487+0000 7f9272e84700 1 -- 172.21.15.33:0/3316350881 shutdown_connections 2024-05-08T21:11:25.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:25.487+0000 7f9272e84700 1 -- 172.21.15.33:0/3316350881 wait complete. 2024-05-08T21:11:25.885 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-05-08T21:11:25.885 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 5... 2024-05-08T21:11:26.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:26 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:26.016+0000 7f04e774c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:11:26.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:26 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3316350881' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T21:11:26.708 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:26 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:26.309+0000 7f04e774c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:11:26.708 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:26 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:26.406+0000 7f04e774c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:11:26.708 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:26 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:26.530+0000 7f04e774c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:11:27.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:26 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:26.710+0000 7f04e774c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:11:27.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:26 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:26.855+0000 7f04e774c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:11:27.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:26 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:26.952+0000 7f04e774c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:11:27.708 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:27 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:27.374+0000 7f04e774c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:11:28.144 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:27 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:27.816+0000 7f04e774c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:11:28.144 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:28 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:28.035+0000 7f04e774c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:11:29.295 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:28 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:28.140+0000 7f04e774c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:11:29.295 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:29 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:29.013+0000 7f04e774c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:11:29.295 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:29 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:29.122+0000 7f04e774c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:11:30.448 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:29 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:29.291+0000 7f04e774c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:11:30.448 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:30 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:30.018+0000 7f04e774c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:11:30.448 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:30 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:30.223+0000 7f04e774c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:11:30.448 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:30 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:30.444+0000 7f04e774c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:11:31.831 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:31 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:31.506+0000 7f04e774c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:11:32.145 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:31 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:31.827+0000 7f04e774c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:11:32.146 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:31 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:31.927+0000 7f04e774c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:11:32.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:32 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:32.142+0000 7f04e774c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:11:32.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:32 smithi033 ceph-mon[42528]: Active manager daemon x restarted 2024-05-08T21:11:32.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:32 smithi033 ceph-mon[42528]: Activating manager daemon x 2024-05-08T21:11:32.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:32 smithi033 ceph-mon[42528]: osdmap e2: 0 total, 0 up, 0 in 2024-05-08T21:11:32.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:32 smithi033 ceph-mon[42528]: mgrmap e6: x(active, starting, since 0.00625383s) 2024-05-08T21:11:32.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:32 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:11:32.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:32 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T21:11:32.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:32 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T21:11:32.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:32 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T21:11:32.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:32 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T21:11:32.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:32 smithi033 ceph-mon[42528]: Manager daemon x is now available 2024-05-08T21:11:32.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:32 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:33.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:33 smithi033 ceph-mon[42528]: Found migration_current of "None". Setting to last migration. 2024-05-08T21:11:33.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:33 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:33.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:33 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:11:33.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:33 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:11:33.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:33 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T21:11:33.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:33 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T21:11:33.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:33 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:33.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:33 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:33.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:33 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:11:33.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:33 smithi033 ceph-mon[42528]: mgrmap e7: x(active, since 1.00736s) 2024-05-08T21:11:33.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:11:33.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-08T21:11:33.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T21:11:33.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:11:33.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.240+0000 7f9b861cc700 1 Processor -- start 2024-05-08T21:11:33.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.240+0000 7f9b861cc700 1 -- start start 2024-05-08T21:11:33.570 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.240+0000 7f9b861cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 0x7f9b80105200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.240+0000 7f9b861cc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80079e40 con 0x7f9b80104de0 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.240+0000 7f9b7f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 0x7f9b80105200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.241+0000 7f9b7f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 0x7f9b80105200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35238/0 (socket says 172.21.15.33:35238) 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.241+0000 7f9b7f7fe700 1 -- 172.21.15.33:0/267945866 learned_addr learned my addr 172.21.15.33:0/267945866 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.241+0000 7f9b7f7fe700 1 -- 172.21.15.33:0/267945866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b80105740 con 0x7f9b80104de0 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.241+0000 7f9b7f7fe700 1 --2- 172.21.15.33:0/267945866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 0x7f9b80105200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9b680098a0 tx=0x7f9b68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e6de1db66721c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.241+0000 7f9b7e7fc700 1 -- 172.21.15.33:0/267945866 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b68004160 con 0x7f9b80104de0 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.241+0000 7f9b7e7fc700 1 -- 172.21.15.33:0/267945866 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9b680042c0 con 0x7f9b80104de0 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.241+0000 7f9b7e7fc700 1 -- 172.21.15.33:0/267945866 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b68011940 con 0x7f9b80104de0 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.242+0000 7f9b861cc700 1 -- 172.21.15.33:0/267945866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 msgr2=0x7f9b80105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.242+0000 7f9b861cc700 1 --2- 172.21.15.33:0/267945866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 0x7f9b80105200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9b680098a0 tx=0x7f9b68009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.242+0000 7f9b861cc700 1 -- 172.21.15.33:0/267945866 shutdown_connections 2024-05-08T21:11:33.571 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.242+0000 7f9b861cc700 1 --2- 172.21.15.33:0/267945866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 0x7f9b80105200 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.242+0000 7f9b861cc700 1 -- 172.21.15.33:0/267945866 >> 172.21.15.33:0/267945866 conn(0x7f9b80100a40 msgr2=0x7f9b80102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.242+0000 7f9b861cc700 1 -- 172.21.15.33:0/267945866 shutdown_connections 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.242+0000 7f9b861cc700 1 -- 172.21.15.33:0/267945866 wait complete. 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.243+0000 7f9b861cc700 1 Processor -- start 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.243+0000 7f9b861cc700 1 -- start start 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.243+0000 7f9b861cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 0x7f9b8019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.243+0000 7f9b861cc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8019c850 con 0x7f9b80104de0 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.244+0000 7f9b7f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 0x7f9b8019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.244+0000 7f9b7f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 0x7f9b8019c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35250/0 (socket says 172.21.15.33:35250) 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.244+0000 7f9b7f7fe700 1 -- 172.21.15.33:0/1785165993 learned_addr learned my addr 172.21.15.33:0/1785165993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.244+0000 7f9b7f7fe700 1 -- 172.21.15.33:0/1785165993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b68009550 con 0x7f9b80104de0 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.244+0000 7f9b7f7fe700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 0x7f9b8019c310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b68006af0 tx=0x7f9b68011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.244+0000 7f9b7cff9700 1 -- 172.21.15.33:0/1785165993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b68003bf0 con 0x7f9b80104de0 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.244+0000 7f9b7cff9700 1 -- 172.21.15.33:0/1785165993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9b6801b8b0 con 0x7f9b80104de0 2024-05-08T21:11:33.572 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.244+0000 7f9b861cc700 1 -- 172.21.15.33:0/1785165993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8019ca50 con 0x7f9b80104de0 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.244+0000 7f9b7cff9700 1 -- 172.21.15.33:0/1785165993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b68023800 con 0x7f9b80104de0 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.244+0000 7f9b861cc700 1 -- 172.21.15.33:0/1785165993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8019cef0 con 0x7f9b80104de0 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.245+0000 7f9b7cff9700 1 -- 172.21.15.33:0/1785165993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f9b68023960 con 0x7f9b80104de0 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.246+0000 7f9b7cff9700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 0x7f9b6c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.246+0000 7f9b861cc700 1 -- 172.21.15.33:0/1785165993 --> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b801064e0 con 0x7f9b6c038700 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.246+0000 7f9b7cff9700 1 -- 172.21.15.33:0/1785165993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9b6804cd10 con 0x7f9b80104de0 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.246+0000 7f9b7effd700 1 -- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 msgr2=0x7f9b6c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3475366293 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.246+0000 7f9b7effd700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 0x7f9b6c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.447+0000 7f9b7effd700 1 -- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 msgr2=0x7f9b6c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3475366293 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.447+0000 7f9b7effd700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 0x7f9b6c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.848+0000 7f9b7effd700 1 -- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 msgr2=0x7f9b6c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3475366293 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:26.848+0000 7f9b7effd700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 0x7f9b6c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:27.649+0000 7f9b7effd700 1 -- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 msgr2=0x7f9b6c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3475366293 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:27.649+0000 7f9b7effd700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 0x7f9b6c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T21:11:33.573 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:29.250+0000 7f9b7effd700 1 -- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 msgr2=0x7f9b6c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3475366293 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:29.250+0000 7f9b7effd700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 0x7f9b6c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:32.160+0000 7f9b7cff9700 1 -- 172.21.15.33:0/1785165993 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 6) v1 ==== 45308+0+0 (secure 0 0 0) 0x7f9b6801ba90 con 0x7f9b80104de0 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:32.161+0000 7f9b7cff9700 1 -- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 msgr2=0x7f9b6c03abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:32.161+0000 7f9b7cff9700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 0x7f9b6c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.160+0000 7f9b7cff9700 1 -- 172.21.15.33:0/1785165993 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7f9b6804e240 con 0x7f9b80104de0 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.160+0000 7f9b7cff9700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f9b6c03be30 0x7f9b6c03e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.160+0000 7f9b7cff9700 1 -- 172.21.15.33:0/1785165993 --> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b801064e0 con 0x7f9b6c03be30 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.161+0000 7f9b7effd700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f9b6c03be30 0x7f9b6c03e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.161+0000 7f9b7effd700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f9b6c03be30 0x7f9b6c03e220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9b70003a10 tx=0x7f9b700092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.162+0000 7f9b7cff9700 1 -- 172.21.15.33:0/1785165993 <== mgr.14120 v2:172.21.15.33:6800/3413344235 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f9b801064e0 con 0x7f9b6c03be30 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.169+0000 7f9b861cc700 1 -- 172.21.15.33:0/1785165993 --> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f9b80106650 con 0x7f9b6c03be30 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.170+0000 7f9b7cff9700 1 -- 172.21.15.33:0/1785165993 <== mgr.14120 v2:172.21.15.33:6800/3413344235 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f9b80106650 con 0x7f9b6c03be30 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.170+0000 7f9b861cc700 1 -- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f9b6c03be30 msgr2=0x7f9b6c03e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.170+0000 7f9b861cc700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f9b6c03be30 0x7f9b6c03e220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9b70003a10 tx=0x7f9b700092b0 comp rx=0 tx=0).stop 2024-05-08T21:11:33.574 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.170+0000 7f9b861cc700 1 -- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 msgr2=0x7f9b8019c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:33.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.170+0000 7f9b861cc700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 0x7f9b8019c310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b68006af0 tx=0x7f9b68011a50 comp rx=0 tx=0).stop 2024-05-08T21:11:33.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.170+0000 7f9b861cc700 1 -- 172.21.15.33:0/1785165993 shutdown_connections 2024-05-08T21:11:33.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.170+0000 7f9b861cc700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f9b6c03be30 0x7f9b6c03e220 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:33.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.170+0000 7f9b861cc700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b80104de0 0x7f9b8019c310 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:33.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.170+0000 7f9b861cc700 1 --2- 172.21.15.33:0/1785165993 >> [v2:172.21.15.33:6800/3475366293,v1:172.21.15.33:6801/3475366293] conn(0x7f9b6c038700 0x7f9b6c03abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:33.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.170+0000 7f9b861cc700 1 -- 172.21.15.33:0/1785165993 >> 172.21.15.33:0/1785165993 conn(0x7f9b80100a40 msgr2=0x7f9b801931a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:33.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.170+0000 7f9b861cc700 1 -- 172.21.15.33:0/1785165993 shutdown_connections 2024-05-08T21:11:33.575 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.171+0000 7f9b861cc700 1 -- 172.21.15.33:0/1785165993 wait complete. 2024-05-08T21:11:33.575 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 5 is available 2024-05-08T21:11:33.575 INFO:teuthology.orchestra.run.smithi033.stdout:Setting orchestrator backend to cephadm... 2024-05-08T21:11:34.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:34 smithi033 ceph-mon[42528]: [08/May/2024:21:11:32] ENGINE Bus STARTING 2024-05-08T21:11:34.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:34 smithi033 ceph-mon[42528]: [08/May/2024:21:11:32] ENGINE Serving on https://172.21.15.33:7150 2024-05-08T21:11:34.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:34 smithi033 ceph-mon[42528]: [08/May/2024:21:11:32] ENGINE Serving on http://172.21.15.33:8765 2024-05-08T21:11:34.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:34 smithi033 ceph-mon[42528]: [08/May/2024:21:11:32] ENGINE Bus STARTED 2024-05-08T21:11:34.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:34 smithi033 ceph-mon[42528]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T21:11:34.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:34 smithi033 ceph-mon[42528]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T21:11:34.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:34 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:34.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.955+0000 7fe9387be700 1 Processor -- start 2024-05-08T21:11:34.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.955+0000 7fe9387be700 1 -- start start 2024-05-08T21:11:34.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.955+0000 7fe9387be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 0x7fe930109400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:34.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.955+0000 7fe9387be700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe930079e20 con 0x7fe930107010 2024-05-08T21:11:34.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.955+0000 7fe93655a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 0x7fe930109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:34.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.955+0000 7fe93655a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 0x7fe930109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34510/0 (socket says 172.21.15.33:34510) 2024-05-08T21:11:34.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.955+0000 7fe93655a700 1 -- 172.21.15.33:0/636329990 learned_addr learned my addr 172.21.15.33:0/636329990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:34.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.956+0000 7fe93655a700 1 -- 172.21.15.33:0/636329990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe930109940 con 0x7fe930107010 2024-05-08T21:11:34.595 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.956+0000 7fe93655a700 1 --2- 172.21.15.33:0/636329990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 0x7fe930109400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0098a0 tx=0x7fe92c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14869162cb85b94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.956+0000 7fe935558700 1 -- 172.21.15.33:0/636329990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe92c004160 con 0x7fe930107010 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.956+0000 7fe935558700 1 -- 172.21.15.33:0/636329990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe92c0042c0 con 0x7fe930107010 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.956+0000 7fe935558700 1 -- 172.21.15.33:0/636329990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe92c011940 con 0x7fe930107010 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.957+0000 7fe9387be700 1 -- 172.21.15.33:0/636329990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 msgr2=0x7fe930109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.957+0000 7fe9387be700 1 --2- 172.21.15.33:0/636329990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 0x7fe930109400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0098a0 tx=0x7fe92c009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.957+0000 7fe9387be700 1 -- 172.21.15.33:0/636329990 shutdown_connections 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.957+0000 7fe9387be700 1 --2- 172.21.15.33:0/636329990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 0x7fe930109400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.957+0000 7fe9387be700 1 -- 172.21.15.33:0/636329990 >> 172.21.15.33:0/636329990 conn(0x7fe930100a40 msgr2=0x7fe930102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.957+0000 7fe9387be700 1 -- 172.21.15.33:0/636329990 shutdown_connections 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.957+0000 7fe9387be700 1 -- 172.21.15.33:0/636329990 wait complete. 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.958+0000 7fe9387be700 1 Processor -- start 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.958+0000 7fe9387be700 1 -- start start 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.958+0000 7fe9387be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 0x7fe93019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:34.596 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.958+0000 7fe9387be700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93019a670 con 0x7fe930107010 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.958+0000 7fe93655a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 0x7fe93019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.958+0000 7fe93655a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 0x7fe93019a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34524/0 (socket says 172.21.15.33:34524) 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.958+0000 7fe93655a700 1 -- 172.21.15.33:0/3398780152 learned_addr learned my addr 172.21.15.33:0/3398780152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.959+0000 7fe93655a700 1 -- 172.21.15.33:0/3398780152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c009550 con 0x7fe930107010 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.959+0000 7fe93655a700 1 --2- 172.21.15.33:0/3398780152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 0x7fe93019a130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe92c003f20 tx=0x7fe92c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.959+0000 7fe9277fe700 1 -- 172.21.15.33:0/3398780152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe92c0037d0 con 0x7fe930107010 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.959+0000 7fe9387be700 1 -- 172.21.15.33:0/3398780152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93019a870 con 0x7fe930107010 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.959+0000 7fe9277fe700 1 -- 172.21.15.33:0/3398780152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe92c01b440 con 0x7fe930107010 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.959+0000 7fe9277fe700 1 -- 172.21.15.33:0/3398780152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe92c023590 con 0x7fe930107010 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.959+0000 7fe9387be700 1 -- 172.21.15.33:0/3398780152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93019ad10 con 0x7fe930107010 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.960+0000 7fe9277fe700 1 -- 172.21.15.33:0/3398780152 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 7) v1 ==== 45435+0+0 (secure 0 0 0) 0x7fe92c003930 con 0x7fe930107010 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.960+0000 7fe9387be700 1 -- 172.21.15.33:0/3398780152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe93010a5c0 con 0x7fe930107010 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.960+0000 7fe9277fe700 1 --2- 172.21.15.33:0/3398780152 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fe91c0385d0 0x7fe91c03aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:34.597 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.960+0000 7fe9277fe700 1 -- 172.21.15.33:0/3398780152 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe92c04c470 con 0x7fe930107010 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.960+0000 7fe935d59700 1 --2- 172.21.15.33:0/3398780152 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fe91c0385d0 0x7fe91c03aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.961+0000 7fe935d59700 1 --2- 172.21.15.33:0/3398780152 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fe91c0385d0 0x7fe91c03aa90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe920006fd0 tx=0x7fe920006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:33.966+0000 7fe9277fe700 1 -- 172.21.15.33:0/3398780152 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe92c00bbd0 con 0x7fe930107010 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.195+0000 7fe9387be700 1 -- 172.21.15.33:0/3398780152 --> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fe930101c80 con 0x7fe91c0385d0 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.205+0000 7fe9277fe700 1 -- 172.21.15.33:0/3398780152 <== mgr.14120 v2:172.21.15.33:6800/3413344235 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe930101c80 con 0x7fe91c0385d0 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.208+0000 7fe9387be700 1 -- 172.21.15.33:0/3398780152 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fe91c0385d0 msgr2=0x7fe91c03aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.208+0000 7fe9387be700 1 --2- 172.21.15.33:0/3398780152 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fe91c0385d0 0x7fe91c03aa90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe920006fd0 tx=0x7fe920006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.208+0000 7fe9387be700 1 -- 172.21.15.33:0/3398780152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 msgr2=0x7fe93019a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.208+0000 7fe9387be700 1 --2- 172.21.15.33:0/3398780152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 0x7fe93019a130 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe92c003f20 tx=0x7fe92c011f00 comp rx=0 tx=0).stop 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.208+0000 7fe9387be700 1 -- 172.21.15.33:0/3398780152 shutdown_connections 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.208+0000 7fe9387be700 1 --2- 172.21.15.33:0/3398780152 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fe91c0385d0 0x7fe91c03aa90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.208+0000 7fe9387be700 1 --2- 172.21.15.33:0/3398780152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe930107010 0x7fe93019a130 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:34.598 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.208+0000 7fe9387be700 1 -- 172.21.15.33:0/3398780152 >> 172.21.15.33:0/3398780152 conn(0x7fe930100a40 msgr2=0x7fe930101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:34.599 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.208+0000 7fe9387be700 1 -- 172.21.15.33:0/3398780152 shutdown_connections 2024-05-08T21:11:34.599 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.208+0000 7fe9387be700 1 -- 172.21.15.33:0/3398780152 wait complete. 2024-05-08T21:11:35.493 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:35 smithi033 ceph-mon[42528]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:11:35.493 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:35 smithi033 ceph-mon[42528]: mgrmap e8: x(active, since 2s) 2024-05-08T21:11:35.493 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:35 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:11:35.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-08T21:11:35.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.979+0000 7fb2e5e45700 1 Processor -- start 2024-05-08T21:11:35.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.979+0000 7fb2e5e45700 1 -- start start 2024-05-08T21:11:35.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.979+0000 7fb2e5e45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 0x7fb2e007a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:35.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.979+0000 7fb2e5e45700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e007a8e0 con 0x7fb2e007beb0 2024-05-08T21:11:35.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.980+0000 7fb2df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 0x7fb2e007a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:35.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.980+0000 7fb2df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 0x7fb2e007a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34530/0 (socket says 172.21.15.33:34530) 2024-05-08T21:11:35.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.980+0000 7fb2df7fe700 1 -- 172.21.15.33:0/352435520 learned_addr learned my addr 172.21.15.33:0/352435520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:35.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.980+0000 7fb2df7fe700 1 -- 172.21.15.33:0/352435520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e007aa20 con 0x7fb2e007beb0 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.980+0000 7fb2df7fe700 1 --2- 172.21.15.33:0/352435520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 0x7fb2e007a310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80098a0 tx=0x7fb2c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e208c3b252b06171 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.980+0000 7fb2de7fc700 1 -- 172.21.15.33:0/352435520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c8004160 con 0x7fb2e007beb0 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.980+0000 7fb2de7fc700 1 -- 172.21.15.33:0/352435520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb2c80042c0 con 0x7fb2e007beb0 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.980+0000 7fb2de7fc700 1 -- 172.21.15.33:0/352435520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c8011940 con 0x7fb2e007beb0 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.981+0000 7fb2e5e45700 1 -- 172.21.15.33:0/352435520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 msgr2=0x7fb2e007a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.981+0000 7fb2e5e45700 1 --2- 172.21.15.33:0/352435520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 0x7fb2e007a310 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80098a0 tx=0x7fb2c8009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.981+0000 7fb2e5e45700 1 -- 172.21.15.33:0/352435520 shutdown_connections 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.981+0000 7fb2e5e45700 1 --2- 172.21.15.33:0/352435520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 0x7fb2e007a310 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.981+0000 7fb2e5e45700 1 -- 172.21.15.33:0/352435520 >> 172.21.15.33:0/352435520 conn(0x7fb2e0101a60 msgr2=0x7fb2e0103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.981+0000 7fb2e5e45700 1 -- 172.21.15.33:0/352435520 shutdown_connections 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.981+0000 7fb2e5e45700 1 -- 172.21.15.33:0/352435520 wait complete. 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.982+0000 7fb2e5e45700 1 Processor -- start 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.982+0000 7fb2e5e45700 1 -- start start 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.982+0000 7fb2e5e45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 0x7fb2e01096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:35.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.982+0000 7fb2e5e45700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e0109c20 con 0x7fb2e007beb0 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.982+0000 7fb2df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 0x7fb2e01096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.982+0000 7fb2df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 0x7fb2e01096e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34536/0 (socket says 172.21.15.33:34536) 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.982+0000 7fb2df7fe700 1 -- 172.21.15.33:0/924163646 learned_addr learned my addr 172.21.15.33:0/924163646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.983+0000 7fb2df7fe700 1 -- 172.21.15.33:0/924163646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8009550 con 0x7fb2e007beb0 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.983+0000 7fb2df7fe700 1 --2- 172.21.15.33:0/924163646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 0x7fb2e01096e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006af0 tx=0x7fb2c8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.983+0000 7fb2dcff9700 1 -- 172.21.15.33:0/924163646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c8003bf0 con 0x7fb2e007beb0 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.983+0000 7fb2dcff9700 1 -- 172.21.15.33:0/924163646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb2c801b8b0 con 0x7fb2e007beb0 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.983+0000 7fb2e5e45700 1 -- 172.21.15.33:0/924163646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e0109e20 con 0x7fb2e007beb0 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.983+0000 7fb2dcff9700 1 -- 172.21.15.33:0/924163646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c8023830 con 0x7fb2e007beb0 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.983+0000 7fb2e5e45700 1 -- 172.21.15.33:0/924163646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e0106270 con 0x7fb2e007beb0 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.984+0000 7fb2dcff9700 1 -- 172.21.15.33:0/924163646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fb2c8023990 con 0x7fb2e007beb0 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.984+0000 7fb2e5e45700 1 -- 172.21.15.33:0/924163646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e010c810 con 0x7fb2e007beb0 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.984+0000 7fb2dcff9700 1 --2- 172.21.15.33:0/924163646 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fb2cc038700 0x7fb2cc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:35.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.984+0000 7fb2dcff9700 1 -- 172.21.15.33:0/924163646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb2c804ccb0 con 0x7fb2e007beb0 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.984+0000 7fb2deffd700 1 --2- 172.21.15.33:0/924163646 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fb2cc038700 0x7fb2cc03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.985+0000 7fb2deffd700 1 --2- 172.21.15.33:0/924163646 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fb2cc038700 0x7fb2cc03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0006fd0 tx=0x7fb2d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:34.990+0000 7fb2dcff9700 1 -- 172.21.15.33:0/924163646 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb2c8023c40 con 0x7fb2e007beb0 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.205+0000 7fb2e5e45700 1 -- 172.21.15.33:0/924163646 --> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2e01069f0 con 0x7fb2cc038700 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.206+0000 7fb2dcff9700 1 -- 172.21.15.33:0/924163646 <== mgr.14120 v2:172.21.15.33:6800/3413344235 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fb2e01069f0 con 0x7fb2cc038700 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.208+0000 7fb2e5e45700 1 -- 172.21.15.33:0/924163646 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fb2cc038700 msgr2=0x7fb2cc03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.208+0000 7fb2e5e45700 1 --2- 172.21.15.33:0/924163646 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fb2cc038700 0x7fb2cc03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0006fd0 tx=0x7fb2d0006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.208+0000 7fb2e5e45700 1 -- 172.21.15.33:0/924163646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 msgr2=0x7fb2e01096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.208+0000 7fb2e5e45700 1 --2- 172.21.15.33:0/924163646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 0x7fb2e01096e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006af0 tx=0x7fb2c8011a50 comp rx=0 tx=0).stop 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.208+0000 7fb2e5e45700 1 -- 172.21.15.33:0/924163646 shutdown_connections 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.208+0000 7fb2e5e45700 1 --2- 172.21.15.33:0/924163646 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fb2cc038700 0x7fb2cc03abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.208+0000 7fb2e5e45700 1 --2- 172.21.15.33:0/924163646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb2e007beb0 0x7fb2e01096e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.208+0000 7fb2e5e45700 1 -- 172.21.15.33:0/924163646 >> 172.21.15.33:0/924163646 conn(0x7fb2e0101a60 msgr2=0x7fb2e0102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.208+0000 7fb2e5e45700 1 -- 172.21.15.33:0/924163646 shutdown_connections 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.209+0000 7fb2e5e45700 1 -- 172.21.15.33:0/924163646 wait complete. 2024-05-08T21:11:35.557 INFO:teuthology.orchestra.run.smithi033.stdout:Generating ssh key... 2024-05-08T21:11:36.589 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:36 smithi033 ceph-mon[42528]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:11:36.958 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: Generating public/private rsa key pair. 2024-05-08T21:11:36.958 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: Your identification has been saved in /tmp/tmp16pfd95m/key. 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: Your public key has been saved in /tmp/tmp16pfd95m/key.pub. 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: The key fingerprint is: 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: SHA256:1F/EEHknyJOC21j6YJRGtNsWgdu64vNjdZzcrtAKZgk ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: The key's randomart image is: 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: +---[RSA 3072]----+ 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: | oo+..oB. | 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: | *oo.*.+ .| 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: | ooOo. o.o | 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: | .Booo . | 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: | E.S+oo.o | 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: | ..oo.= . | 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: | =o.... | 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: | oo+. o . | 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: | ..=... .. | 2024-05-08T21:11:36.959 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: +----[SHA256]-----+ 2024-05-08T21:11:37.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:37 smithi033 ceph-mon[42528]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:11:37.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:37 smithi033 ceph-mon[42528]: Generating ssh key... 2024-05-08T21:11:37.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:37 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:37.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:37 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:38.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.919+0000 7f009e333700 1 Processor -- start 2024-05-08T21:11:38.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.919+0000 7f009e333700 1 -- start start 2024-05-08T21:11:38.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.919+0000 7f009e333700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 0x7f00981068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:38.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.919+0000 7f009e333700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0098106eb0 con 0x7f00981064c0 2024-05-08T21:11:38.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.920+0000 7f0097fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 0x7f00981068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:38.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.920+0000 7f0097fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 0x7f00981068e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34546/0 (socket says 172.21.15.33:34546) 2024-05-08T21:11:38.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.920+0000 7f0097fff700 1 -- 172.21.15.33:0/166332488 learned_addr learned my addr 172.21.15.33:0/166332488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:38.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.920+0000 7f0097fff700 1 -- 172.21.15.33:0/166332488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00981076c0 con 0x7f00981064c0 2024-05-08T21:11:38.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.920+0000 7f0097fff700 1 --2- 172.21.15.33:0/166332488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 0x7f00981068e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f00880098a0 tx=0x7f0088009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74bc8452365ebbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:38.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.920+0000 7f0096ffd700 1 -- 172.21.15.33:0/166332488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0088004160 con 0x7f00981064c0 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.920+0000 7f0096ffd700 1 -- 172.21.15.33:0/166332488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f00880042c0 con 0x7f00981064c0 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.920+0000 7f0096ffd700 1 -- 172.21.15.33:0/166332488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0088011940 con 0x7f00981064c0 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.921+0000 7f009e333700 1 -- 172.21.15.33:0/166332488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 msgr2=0x7f00981068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.921+0000 7f009e333700 1 --2- 172.21.15.33:0/166332488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 0x7f00981068e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f00880098a0 tx=0x7f0088009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.921+0000 7f009e333700 1 -- 172.21.15.33:0/166332488 shutdown_connections 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.921+0000 7f009e333700 1 --2- 172.21.15.33:0/166332488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 0x7f00981068e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.921+0000 7f009e333700 1 -- 172.21.15.33:0/166332488 >> 172.21.15.33:0/166332488 conn(0x7f0098101a60 msgr2=0x7f0098103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.921+0000 7f009e333700 1 -- 172.21.15.33:0/166332488 shutdown_connections 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.921+0000 7f009e333700 1 -- 172.21.15.33:0/166332488 wait complete. 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.922+0000 7f009e333700 1 Processor -- start 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.922+0000 7f009e333700 1 -- start start 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.922+0000 7f009e333700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 0x7f009819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.922+0000 7f009e333700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009819c890 con 0x7f00981064c0 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.922+0000 7f0097fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 0x7f009819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.922+0000 7f0097fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 0x7f009819c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34548/0 (socket says 172.21.15.33:34548) 2024-05-08T21:11:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.922+0000 7f0097fff700 1 -- 172.21.15.33:0/2272291788 learned_addr learned my addr 172.21.15.33:0/2272291788 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.922+0000 7f0097fff700 1 -- 172.21.15.33:0/2272291788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0088009550 con 0x7f00981064c0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.923+0000 7f0097fff700 1 --2- 172.21.15.33:0/2272291788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 0x7f009819c350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0088003f20 tx=0x7f0088011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.923+0000 7f00957fa700 1 -- 172.21.15.33:0/2272291788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00880036a0 con 0x7f00981064c0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.923+0000 7f009e333700 1 -- 172.21.15.33:0/2272291788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009819ca90 con 0x7f00981064c0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.923+0000 7f00957fa700 1 -- 172.21.15.33:0/2272291788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f0088003c70 con 0x7f00981064c0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.923+0000 7f00957fa700 1 -- 172.21.15.33:0/2272291788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f008801be80 con 0x7f00981064c0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.923+0000 7f009e333700 1 -- 172.21.15.33:0/2272291788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009819cf30 con 0x7f00981064c0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.924+0000 7f00957fa700 1 -- 172.21.15.33:0/2272291788 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f00880234e0 con 0x7f00981064c0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.924+0000 7f009e333700 1 -- 172.21.15.33:0/2272291788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00840052f0 con 0x7f00981064c0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.924+0000 7f00957fa700 1 --2- 172.21.15.33:0/2272291788 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f0080038700 0x7f008003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.924+0000 7f00957fa700 1 -- 172.21.15.33:0/2272291788 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f008804c620 con 0x7f00981064c0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.924+0000 7f00977fe700 1 --2- 172.21.15.33:0/2272291788 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f0080038700 0x7f008003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.925+0000 7f00977fe700 1 --2- 172.21.15.33:0/2272291788 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f0080038700 0x7f008003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f008c006fd0 tx=0x7f008c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:35.930+0000 7f00957fa700 1 -- 172.21.15.33:0/2272291788 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f008802bbd0 con 0x7f00981064c0 2024-05-08T21:11:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:36.144+0000 7f009e333700 1 -- 172.21.15.33:0/2272291788 --> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0084000bc0 con 0x7f0080038700 2024-05-08T21:11:38.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:36.593+0000 7f00957fa700 1 -- 172.21.15.33:0/2272291788 <== mgr.14120 v2:172.21.15.33:6800/3413344235 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0084000bc0 con 0x7f0080038700 2024-05-08T21:11:38.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:36.595+0000 7f009e333700 1 -- 172.21.15.33:0/2272291788 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f0080038700 msgr2=0x7f008003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:38.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:36.596+0000 7f009e333700 1 --2- 172.21.15.33:0/2272291788 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f0080038700 0x7f008003abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f008c006fd0 tx=0x7f008c006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:38.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:36.596+0000 7f009e333700 1 -- 172.21.15.33:0/2272291788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 msgr2=0x7f009819c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:38.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:36.596+0000 7f009e333700 1 --2- 172.21.15.33:0/2272291788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 0x7f009819c350 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0088003f20 tx=0x7f0088011f00 comp rx=0 tx=0).stop 2024-05-08T21:11:38.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:36.596+0000 7f009e333700 1 -- 172.21.15.33:0/2272291788 shutdown_connections 2024-05-08T21:11:38.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:36.596+0000 7f009e333700 1 --2- 172.21.15.33:0/2272291788 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f0080038700 0x7f008003abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:38.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:36.596+0000 7f009e333700 1 --2- 172.21.15.33:0/2272291788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00981064c0 0x7f009819c350 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:38.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:36.596+0000 7f009e333700 1 -- 172.21.15.33:0/2272291788 >> 172.21.15.33:0/2272291788 conn(0x7f0098101a60 msgr2=0x7f00981024a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:38.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:36.596+0000 7f009e333700 1 -- 172.21.15.33:0/2272291788 shutdown_connections 2024-05-08T21:11:38.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:36.596+0000 7f009e333700 1 -- 172.21.15.33:0/2272291788 wait complete. 2024-05-08T21:11:38.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzHb55XRxErTdFsuIzj7b1wTMhgpoUt4+Ux3jmSJX8oD+u28CTZV20SVHZS+8GEPvmaGb2KsF4c59pGLWBJmM7DlQ3tIDZLkOw1OTKBm7Qpx/u9xjZyA68XQ+5Ze1bqxIIgwAZ+mc6IqlmRuZYQc7uX1D/e21e04CBMyFyMUwQ90xJh8vMZqjhtdfSpbFdWGbuMGk86K5PGYCPCLSXCjDyjWMFBvpjDsz+kBU8vP3DAALwexAkjNVVf/CjuZxW0o5GWrPYwed772lsAqqEf3bmt87pPb+D5tEI4oVJGj/yetf5/Qmk24c3TTzL3AaTI+1/gGV8d9z7Agtf/aZeXDae3vPQqAAs4f/kzCmuf51Ix6G7OEUV0kDbPtF+neXucbvtmN+YlUFGtDyLag+NdKdUuAqgX7vRN6erPFDvsAtQevKV4lVB66JaPIr0X6HAqLRFknqTg2MvONVyVPZnWnZkgIUvznhmIa4Cz5rYebKVscz4FAz5cJiWqj/YYyjN/Mc= ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:11:38.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.392+0000 7f2a07cbe700 1 Processor -- start 2024-05-08T21:11:38.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.393+0000 7f2a07cbe700 1 -- start start 2024-05-08T21:11:38.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.393+0000 7f2a07cbe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 0x7f2a00104bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:38.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.393+0000 7f2a07cbe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a00105190 con 0x7f2a001047a0 2024-05-08T21:11:38.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.393+0000 7f2a05a5a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 0x7f2a00104bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:38.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.393+0000 7f2a05a5a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 0x7f2a00104bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34554/0 (socket says 172.21.15.33:34554) 2024-05-08T21:11:38.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.393+0000 7f2a05a5a700 1 -- 172.21.15.33:0/1505773504 learned_addr learned my addr 172.21.15.33:0/1505773504 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:38.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.393+0000 7f2a05a5a700 1 -- 172.21.15.33:0/1505773504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a001054c0 con 0x7f2a001047a0 2024-05-08T21:11:38.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.394+0000 7f2a05a5a700 1 --2- 172.21.15.33:0/1505773504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 0x7f2a00104bc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0098a0 tx=0x7f29fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d480663d21722f18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:38.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.394+0000 7f2a04a58700 1 -- 172.21.15.33:0/1505773504 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29fc004160 con 0x7f2a001047a0 2024-05-08T21:11:38.987 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.394+0000 7f2a04a58700 1 -- 172.21.15.33:0/1505773504 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f29fc0042c0 con 0x7f2a001047a0 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.394+0000 7f2a04a58700 1 -- 172.21.15.33:0/1505773504 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29fc011940 con 0x7f2a001047a0 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.394+0000 7f2a07cbe700 1 -- 172.21.15.33:0/1505773504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 msgr2=0x7f2a00104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.394+0000 7f2a07cbe700 1 --2- 172.21.15.33:0/1505773504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 0x7f2a00104bc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0098a0 tx=0x7f29fc009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.395+0000 7f2a07cbe700 1 -- 172.21.15.33:0/1505773504 shutdown_connections 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.395+0000 7f2a07cbe700 1 --2- 172.21.15.33:0/1505773504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 0x7f2a00104bc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.395+0000 7f2a07cbe700 1 -- 172.21.15.33:0/1505773504 >> 172.21.15.33:0/1505773504 conn(0x7f2a001000f0 msgr2=0x7f2a00102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.395+0000 7f2a07cbe700 1 -- 172.21.15.33:0/1505773504 shutdown_connections 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.395+0000 7f2a07cbe700 1 -- 172.21.15.33:0/1505773504 wait complete. 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.395+0000 7f2a07cbe700 1 Processor -- start 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.396+0000 7f2a07cbe700 1 -- start start 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.396+0000 7f2a07cbe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 0x7f2a0019c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.396+0000 7f2a07cbe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0019c860 con 0x7f2a001047a0 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.396+0000 7f2a05a5a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 0x7f2a0019c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.396+0000 7f2a05a5a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 0x7f2a0019c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34566/0 (socket says 172.21.15.33:34566) 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.396+0000 7f2a05a5a700 1 -- 172.21.15.33:0/4086690116 learned_addr learned my addr 172.21.15.33:0/4086690116 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.396+0000 7f2a05a5a700 1 -- 172.21.15.33:0/4086690116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc009550 con 0x7f2a001047a0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.396+0000 7f2a05a5a700 1 --2- 172.21.15.33:0/4086690116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 0x7f2a0019c320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006af0 tx=0x7f29fc011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.397+0000 7f29f6ffd700 1 -- 172.21.15.33:0/4086690116 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29fc003bf0 con 0x7f2a001047a0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.397+0000 7f29f6ffd700 1 -- 172.21.15.33:0/4086690116 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f29fc01b8b0 con 0x7f2a001047a0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.397+0000 7f29f6ffd700 1 -- 172.21.15.33:0/4086690116 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29fc023830 con 0x7f2a001047a0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.397+0000 7f2a07cbe700 1 -- 172.21.15.33:0/4086690116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0019ca60 con 0x7f2a001047a0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.397+0000 7f2a07cbe700 1 -- 172.21.15.33:0/4086690116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0019cf00 con 0x7f2a001047a0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.397+0000 7f29f6ffd700 1 -- 172.21.15.33:0/4086690116 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f29fc023990 con 0x7f2a001047a0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.398+0000 7f29f6ffd700 1 --2- 172.21.15.33:0/4086690116 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f29ec038700 0x7f29ec03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.398+0000 7f29f6ffd700 1 -- 172.21.15.33:0/4086690116 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f29fc04cd70 con 0x7f2a001047a0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.398+0000 7f2a07cbe700 1 -- 172.21.15.33:0/4086690116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a001066e0 con 0x7f2a001047a0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.398+0000 7f2a05259700 1 --2- 172.21.15.33:0/4086690116 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f29ec038700 0x7f29ec03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.398+0000 7f2a05259700 1 --2- 172.21.15.33:0/4086690116 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f29ec038700 0x7f29ec03abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f29f0006fd0 tx=0x7f29f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.404+0000 7f29f6ffd700 1 -- 172.21.15.33:0/4086690116 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f29fc023c40 con 0x7f2a001047a0 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.618+0000 7f2a07cbe700 1 -- 172.21.15.33:0/4086690116 --> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a000613e0 con 0x7f29ec038700 2024-05-08T21:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.618+0000 7f29f6ffd700 1 -- 172.21.15.33:0/4086690116 <== mgr.14120 v2:172.21.15.33:6800/3413344235 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f2a000613e0 con 0x7f29ec038700 2024-05-08T21:11:38.990 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.621+0000 7f2a07cbe700 1 -- 172.21.15.33:0/4086690116 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f29ec038700 msgr2=0x7f29ec03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:38.990 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.621+0000 7f2a07cbe700 1 --2- 172.21.15.33:0/4086690116 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f29ec038700 0x7f29ec03abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f29f0006fd0 tx=0x7f29f0006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:38.990 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.621+0000 7f2a07cbe700 1 -- 172.21.15.33:0/4086690116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 msgr2=0x7f2a0019c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:38.990 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.621+0000 7f2a07cbe700 1 --2- 172.21.15.33:0/4086690116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 0x7f2a0019c320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006af0 tx=0x7f29fc011a50 comp rx=0 tx=0).stop 2024-05-08T21:11:38.990 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.621+0000 7f2a07cbe700 1 -- 172.21.15.33:0/4086690116 shutdown_connections 2024-05-08T21:11:38.990 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.621+0000 7f2a07cbe700 1 --2- 172.21.15.33:0/4086690116 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f29ec038700 0x7f29ec03abc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:38.990 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.621+0000 7f2a07cbe700 1 --2- 172.21.15.33:0/4086690116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a001047a0 0x7f2a0019c320 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:38.990 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.621+0000 7f2a07cbe700 1 -- 172.21.15.33:0/4086690116 >> 172.21.15.33:0/4086690116 conn(0x7f2a001000f0 msgr2=0x7f2a00192b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:38.990 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.621+0000 7f2a07cbe700 1 -- 172.21.15.33:0/4086690116 shutdown_connections 2024-05-08T21:11:38.990 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:38.621+0000 7f2a07cbe700 1 -- 172.21.15.33:0/4086690116 wait complete. 2024-05-08T21:11:38.990 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-08T21:11:38.990 INFO:teuthology.orchestra.run.smithi033.stdout:Adding key to root@localhost authorized_keys... 2024-05-08T21:11:38.991 INFO:teuthology.orchestra.run.smithi033.stdout:Adding host smithi033... 2024-05-08T21:11:39.837 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:39 smithi033 ceph-mon[42528]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:11:40.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:40 smithi033 ceph-mon[42528]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:11:40.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:40 smithi033 ceph-mon[42528]: Deploying cephadm binary to smithi033 2024-05-08T21:11:42.213 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Added host 'smithi033' with addr '172.21.15.33' 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.353+0000 7f57b4e0f700 1 Processor -- start 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.353+0000 7f57b4e0f700 1 -- start start 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.354+0000 7f57b4e0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 0x7f57b0106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.354+0000 7f57b4e0f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b0106ef0 con 0x7f57b0106500 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.354+0000 7f57ae59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 0x7f57b0106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.354+0000 7f57ae59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 0x7f57b0106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34572/0 (socket says 172.21.15.33:34572) 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.354+0000 7f57ae59c700 1 -- 172.21.15.33:0/3328169743 learned_addr learned my addr 172.21.15.33:0/3328169743 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.354+0000 7f57ae59c700 1 -- 172.21.15.33:0/3328169743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b0107700 con 0x7f57b0106500 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.354+0000 7f57ae59c700 1 --2- 172.21.15.33:0/3328169743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 0x7f57b0106920 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57a00098a0 tx=0x7f57a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9896bb00bd45f95a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.355+0000 7f57ad59a700 1 -- 172.21.15.33:0/3328169743 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57a0004160 con 0x7f57b0106500 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.355+0000 7f57ad59a700 1 -- 172.21.15.33:0/3328169743 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f57a00042c0 con 0x7f57b0106500 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.355+0000 7f57ad59a700 1 -- 172.21.15.33:0/3328169743 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57a0011940 con 0x7f57b0106500 2024-05-08T21:11:42.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.355+0000 7f57b4e0f700 1 -- 172.21.15.33:0/3328169743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 msgr2=0x7f57b0106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.355+0000 7f57b4e0f700 1 --2- 172.21.15.33:0/3328169743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 0x7f57b0106920 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57a00098a0 tx=0x7f57a0009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.356+0000 7f57b4e0f700 1 -- 172.21.15.33:0/3328169743 shutdown_connections 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.356+0000 7f57b4e0f700 1 --2- 172.21.15.33:0/3328169743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 0x7f57b0106920 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.356+0000 7f57b4e0f700 1 -- 172.21.15.33:0/3328169743 >> 172.21.15.33:0/3328169743 conn(0x7f57b0101b00 msgr2=0x7f57b0103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.356+0000 7f57b4e0f700 1 -- 172.21.15.33:0/3328169743 shutdown_connections 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.356+0000 7f57b4e0f700 1 -- 172.21.15.33:0/3328169743 wait complete. 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.356+0000 7f57b4e0f700 1 Processor -- start 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.356+0000 7f57b4e0f700 1 -- start start 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.357+0000 7f57b4e0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 0x7f57b019a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.357+0000 7f57b4e0f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b019a6b0 con 0x7f57b0106500 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.357+0000 7f57ae59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 0x7f57b019a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.357+0000 7f57ae59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 0x7f57b019a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34574/0 (socket says 172.21.15.33:34574) 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.357+0000 7f57ae59c700 1 -- 172.21.15.33:0/1624441700 learned_addr learned my addr 172.21.15.33:0/1624441700 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:42.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.357+0000 7f57ae59c700 1 -- 172.21.15.33:0/1624441700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a0009550 con 0x7f57b0106500 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.357+0000 7f57ae59c700 1 --2- 172.21.15.33:0/1624441700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 0x7f57b019a170 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f57a0006af0 tx=0x7f57a0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.357+0000 7f57a77fe700 1 -- 172.21.15.33:0/1624441700 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57a0003bf0 con 0x7f57b0106500 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.357+0000 7f57b4e0f700 1 -- 172.21.15.33:0/1624441700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b019a8b0 con 0x7f57b0106500 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.357+0000 7f57a77fe700 1 -- 172.21.15.33:0/1624441700 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f57a001b8b0 con 0x7f57b0106500 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.357+0000 7f57a77fe700 1 -- 172.21.15.33:0/1624441700 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57a0023830 con 0x7f57b0106500 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.358+0000 7f57b4e0f700 1 -- 172.21.15.33:0/1624441700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b019ad50 con 0x7f57b0106500 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.358+0000 7f57a77fe700 1 -- 172.21.15.33:0/1624441700 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f57a0023aa0 con 0x7f57b0106500 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.359+0000 7f57a77fe700 1 --2- 172.21.15.33:0/1624441700 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f579c0386b0 0x7f579c03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.359+0000 7f57a77fe700 1 -- 172.21.15.33:0/1624441700 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f57a004cec0 con 0x7f57b0106500 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.359+0000 7f57b4e0f700 1 -- 172.21.15.33:0/1624441700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b010a930 con 0x7f57b0106500 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.359+0000 7f57add9b700 1 --2- 172.21.15.33:0/1624441700 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f579c0386b0 0x7f579c03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.359+0000 7f57add9b700 1 --2- 172.21.15.33:0/1624441700 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f579c0386b0 0x7f579c03ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5798006fd0 tx=0x7f5798006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.364+0000 7f57a77fe700 1 -- 172.21.15.33:0/1624441700 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f57a0023d50 con 0x7f57b0106500 2024-05-08T21:11:42.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:39.583+0000 7f57b4e0f700 1 -- 172.21.15.33:0/1624441700 --> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}) v1 -- 0x7f57b0102df0 con 0x7f579c0386b0 2024-05-08T21:11:42.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:41.889+0000 7f57a77fe700 1 -- 172.21.15.33:0/1624441700 <== mgr.14120 v2:172.21.15.33:6800/3413344235 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f57b0102df0 con 0x7f579c0386b0 2024-05-08T21:11:42.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:41.893+0000 7f57b4e0f700 1 -- 172.21.15.33:0/1624441700 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f579c0386b0 msgr2=0x7f579c03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:42.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:41.893+0000 7f57b4e0f700 1 --2- 172.21.15.33:0/1624441700 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f579c0386b0 0x7f579c03ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5798006fd0 tx=0x7f5798006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:42.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:41.893+0000 7f57b4e0f700 1 -- 172.21.15.33:0/1624441700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 msgr2=0x7f57b019a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:42.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:41.893+0000 7f57b4e0f700 1 --2- 172.21.15.33:0/1624441700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 0x7f57b019a170 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f57a0006af0 tx=0x7f57a0011a50 comp rx=0 tx=0).stop 2024-05-08T21:11:42.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:41.893+0000 7f57b4e0f700 1 -- 172.21.15.33:0/1624441700 shutdown_connections 2024-05-08T21:11:42.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:41.893+0000 7f57b4e0f700 1 --2- 172.21.15.33:0/1624441700 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f579c0386b0 0x7f579c03ab70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:42.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:41.893+0000 7f57b4e0f700 1 --2- 172.21.15.33:0/1624441700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b0106500 0x7f57b019a170 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:42.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:41.893+0000 7f57b4e0f700 1 -- 172.21.15.33:0/1624441700 >> 172.21.15.33:0/1624441700 conn(0x7f57b0101b00 msgr2=0x7f57b01026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:42.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:41.893+0000 7f57b4e0f700 1 -- 172.21.15.33:0/1624441700 shutdown_connections 2024-05-08T21:11:42.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:41.893+0000 7f57b4e0f700 1 -- 172.21.15.33:0/1624441700 wait complete. 2024-05-08T21:11:42.217 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mon service... 2024-05-08T21:11:42.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:42 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:42.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:42 smithi033 ceph-mon[42528]: Added host smithi033 2024-05-08T21:11:42.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:42 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:11:42.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:42 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:43.468 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.567+0000 7fc98c6ac700 1 Processor -- start 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.568+0000 7fc98c6ac700 1 -- start start 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.568+0000 7fc98c6ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 0x7fc984106d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.568+0000 7fc98c6ac700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc984107360 con 0x7fc984106970 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.568+0000 7fc98a448700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 0x7fc984106d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.568+0000 7fc98a448700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 0x7fc984106d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44138/0 (socket says 172.21.15.33:44138) 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.569+0000 7fc98a448700 1 -- 172.21.15.33:0/3821086644 learned_addr learned my addr 172.21.15.33:0/3821086644 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.569+0000 7fc98a448700 1 -- 172.21.15.33:0/3821086644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9841055d0 con 0x7fc984106970 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.569+0000 7fc98a448700 1 --2- 172.21.15.33:0/3821086644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 0x7fc984106d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc9800098a0 tx=0x7fc980009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95845731f10a723 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.569+0000 7fc989446700 1 -- 172.21.15.33:0/3821086644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc980004160 con 0x7fc984106970 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.569+0000 7fc989446700 1 -- 172.21.15.33:0/3821086644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fc9800042c0 con 0x7fc984106970 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.569+0000 7fc989446700 1 -- 172.21.15.33:0/3821086644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc980011940 con 0x7fc984106970 2024-05-08T21:11:43.469 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.570+0000 7fc98c6ac700 1 -- 172.21.15.33:0/3821086644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 msgr2=0x7fc984106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.570+0000 7fc98c6ac700 1 --2- 172.21.15.33:0/3821086644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 0x7fc984106d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc9800098a0 tx=0x7fc980009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.570+0000 7fc98c6ac700 1 -- 172.21.15.33:0/3821086644 shutdown_connections 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.570+0000 7fc98c6ac700 1 --2- 172.21.15.33:0/3821086644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 0x7fc984106d90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.570+0000 7fc98c6ac700 1 -- 172.21.15.33:0/3821086644 >> 172.21.15.33:0/3821086644 conn(0x7fc9841000e0 msgr2=0x7fc984102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.570+0000 7fc98c6ac700 1 -- 172.21.15.33:0/3821086644 shutdown_connections 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.570+0000 7fc98c6ac700 1 -- 172.21.15.33:0/3821086644 wait complete. 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.571+0000 7fc98c6ac700 1 Processor -- start 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.571+0000 7fc98c6ac700 1 -- start start 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.571+0000 7fc98c6ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 0x7fc98419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.571+0000 7fc98c6ac700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98419a640 con 0x7fc984106970 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.571+0000 7fc98a448700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 0x7fc98419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.571+0000 7fc98a448700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 0x7fc98419a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44152/0 (socket says 172.21.15.33:44152) 2024-05-08T21:11:43.470 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.571+0000 7fc98a448700 1 -- 172.21.15.33:0/1951514091 learned_addr learned my addr 172.21.15.33:0/1951514091 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.572+0000 7fc98a448700 1 -- 172.21.15.33:0/1951514091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc980009550 con 0x7fc984106970 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.572+0000 7fc98a448700 1 --2- 172.21.15.33:0/1951514091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 0x7fc98419a100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc980011de0 tx=0x7fc980011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.572+0000 7fc97b7fe700 1 -- 172.21.15.33:0/1951514091 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9800036a0 con 0x7fc984106970 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.572+0000 7fc97b7fe700 1 -- 172.21.15.33:0/1951514091 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fc980003c70 con 0x7fc984106970 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.572+0000 7fc98c6ac700 1 -- 172.21.15.33:0/1951514091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98419a840 con 0x7fc984106970 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.572+0000 7fc97b7fe700 1 -- 172.21.15.33:0/1951514091 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc980024d70 con 0x7fc984106970 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.572+0000 7fc98c6ac700 1 -- 172.21.15.33:0/1951514091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98419ace0 con 0x7fc984106970 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.573+0000 7fc98c6ac700 1 -- 172.21.15.33:0/1951514091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc984109810 con 0x7fc984106970 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.574+0000 7fc97b7fe700 1 -- 172.21.15.33:0/1951514091 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fc980024640 con 0x7fc984106970 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.574+0000 7fc97b7fe700 1 --2- 172.21.15.33:0/1951514091 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc970038700 0x7fc97003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.574+0000 7fc97b7fe700 1 -- 172.21.15.33:0/1951514091 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc98004c4c0 con 0x7fc984106970 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.574+0000 7fc989c47700 1 --2- 172.21.15.33:0/1951514091 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc970038700 0x7fc97003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.575+0000 7fc989c47700 1 --2- 172.21.15.33:0/1951514091 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc970038700 0x7fc97003abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc974006fd0 tx=0x7fc974006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.579+0000 7fc97b7fe700 1 -- 172.21.15.33:0/1951514091 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc9800233a0 con 0x7fc984106970 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.808+0000 7fc98c6ac700 1 -- 172.21.15.33:0/1951514091 --> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc9841012f0 con 0x7fc970038700 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.813+0000 7fc97b7fe700 1 -- 172.21.15.33:0/1951514091 <== mgr.14120 v2:172.21.15.33:6800/3413344235 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc9841012f0 con 0x7fc970038700 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.815+0000 7fc98c6ac700 1 -- 172.21.15.33:0/1951514091 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc970038700 msgr2=0x7fc97003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.815+0000 7fc98c6ac700 1 --2- 172.21.15.33:0/1951514091 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc970038700 0x7fc97003abc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc974006fd0 tx=0x7fc974006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.815+0000 7fc98c6ac700 1 -- 172.21.15.33:0/1951514091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 msgr2=0x7fc98419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.815+0000 7fc98c6ac700 1 --2- 172.21.15.33:0/1951514091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 0x7fc98419a100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc980011de0 tx=0x7fc980011ec0 comp rx=0 tx=0).stop 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.816+0000 7fc98c6ac700 1 -- 172.21.15.33:0/1951514091 shutdown_connections 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.816+0000 7fc98c6ac700 1 --2- 172.21.15.33:0/1951514091 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc970038700 0x7fc97003abc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.816+0000 7fc98c6ac700 1 --2- 172.21.15.33:0/1951514091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984106970 0x7fc98419a100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.816+0000 7fc98c6ac700 1 -- 172.21.15.33:0/1951514091 >> 172.21.15.33:0/1951514091 conn(0x7fc9841000e0 msgr2=0x7fc984100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.816+0000 7fc98c6ac700 1 -- 172.21.15.33:0/1951514091 shutdown_connections 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:42.816+0000 7fc98c6ac700 1 -- 172.21.15.33:0/1951514091 wait complete. 2024-05-08T21:11:43.472 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mgr service... 2024-05-08T21:11:44.450 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:44 smithi033 ceph-mon[42528]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:11:44.450 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:44 smithi033 ceph-mon[42528]: Saving service mon spec with placement count:5 2024-05-08T21:11:44.450 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:44 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:44.450 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:44 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:44.450 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:44 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:44.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-08T21:11:44.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.857+0000 7fec402ee700 1 Processor -- start 2024-05-08T21:11:44.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.857+0000 7fec402ee700 1 -- start start 2024-05-08T21:11:44.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.857+0000 7fec402ee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 0x7fec38104bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:44.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.857+0000 7fec402ee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec381051a0 con 0x7fec381047b0 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.858+0000 7fec3e08a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 0x7fec38104bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.858+0000 7fec3e08a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 0x7fec38104bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44160/0 (socket says 172.21.15.33:44160) 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.858+0000 7fec3e08a700 1 -- 172.21.15.33:0/2615839543 learned_addr learned my addr 172.21.15.33:0/2615839543 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.858+0000 7fec3e08a700 1 -- 172.21.15.33:0/2615839543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec38107740 con 0x7fec381047b0 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.858+0000 7fec3e08a700 1 --2- 172.21.15.33:0/2615839543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 0x7fec38104bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fec340098a0 tx=0x7fec34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25aa858b25a1651d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.858+0000 7fec3d088700 1 -- 172.21.15.33:0/2615839543 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec34004160 con 0x7fec381047b0 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.858+0000 7fec3d088700 1 -- 172.21.15.33:0/2615839543 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec340042c0 con 0x7fec381047b0 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.858+0000 7fec3d088700 1 -- 172.21.15.33:0/2615839543 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec340118e0 con 0x7fec381047b0 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.859+0000 7fec402ee700 1 -- 172.21.15.33:0/2615839543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 msgr2=0x7fec38104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.859+0000 7fec402ee700 1 --2- 172.21.15.33:0/2615839543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 0x7fec38104bd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fec340098a0 tx=0x7fec34009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.859+0000 7fec402ee700 1 -- 172.21.15.33:0/2615839543 shutdown_connections 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.859+0000 7fec402ee700 1 --2- 172.21.15.33:0/2615839543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 0x7fec38104bd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.859+0000 7fec402ee700 1 -- 172.21.15.33:0/2615839543 >> 172.21.15.33:0/2615839543 conn(0x7fec38100100 msgr2=0x7fec38102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:44.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.859+0000 7fec402ee700 1 -- 172.21.15.33:0/2615839543 shutdown_connections 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.859+0000 7fec402ee700 1 -- 172.21.15.33:0/2615839543 wait complete. 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.860+0000 7fec402ee700 1 Processor -- start 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.860+0000 7fec402ee700 1 -- start start 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.860+0000 7fec402ee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 0x7fec38106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.860+0000 7fec402ee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec381074b0 con 0x7fec381047b0 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.861+0000 7fec3e08a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 0x7fec38106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.861+0000 7fec3e08a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 0x7fec38106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44164/0 (socket says 172.21.15.33:44164) 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.861+0000 7fec3e08a700 1 -- 172.21.15.33:0/783710511 learned_addr learned my addr 172.21.15.33:0/783710511 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.861+0000 7fec3e08a700 1 -- 172.21.15.33:0/783710511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec34009550 con 0x7fec381047b0 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.861+0000 7fec3e08a700 1 --2- 172.21.15.33:0/783710511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 0x7fec38106f70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fec34003e30 tx=0x7fec34011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.861+0000 7fec2f7fe700 1 -- 172.21.15.33:0/783710511 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec340036a0 con 0x7fec381047b0 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.861+0000 7fec2f7fe700 1 -- 172.21.15.33:0/783710511 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec3401b440 con 0x7fec381047b0 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.861+0000 7fec402ee700 1 -- 172.21.15.33:0/783710511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec381056f0 con 0x7fec381047b0 2024-05-08T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.861+0000 7fec2f7fe700 1 -- 172.21.15.33:0/783710511 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec340193d0 con 0x7fec381047b0 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.861+0000 7fec402ee700 1 -- 172.21.15.33:0/783710511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec38105b90 con 0x7fec381047b0 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.862+0000 7fec2f7fe700 1 -- 172.21.15.33:0/783710511 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7fec34019640 con 0x7fec381047b0 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.863+0000 7fec2f7fe700 1 --2- 172.21.15.33:0/783710511 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fec24038700 0x7fec2403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.863+0000 7fec2f7fe700 1 -- 172.21.15.33:0/783710511 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fec3404c480 con 0x7fec381047b0 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.863+0000 7fec3d889700 1 --2- 172.21.15.33:0/783710511 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fec24038700 0x7fec2403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.863+0000 7fec402ee700 1 -- 172.21.15.33:0/783710511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec380625d0 con 0x7fec381047b0 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.863+0000 7fec3d889700 1 --2- 172.21.15.33:0/783710511 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fec24038700 0x7fec2403abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fec28006fd0 tx=0x7fec28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:43.869+0000 7fec2f7fe700 1 -- 172.21.15.33:0/783710511 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fec34018940 con 0x7fec381047b0 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:44.107+0000 7fec402ee700 1 -- 172.21.15.33:0/783710511 --> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fec38101400 con 0x7fec24038700 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:44.113+0000 7fec2f7fe700 1 -- 172.21.15.33:0/783710511 <== mgr.14120 v2:172.21.15.33:6800/3413344235 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fec38101400 con 0x7fec24038700 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:44.115+0000 7fec402ee700 1 -- 172.21.15.33:0/783710511 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fec24038700 msgr2=0x7fec2403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:44.115+0000 7fec402ee700 1 --2- 172.21.15.33:0/783710511 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fec24038700 0x7fec2403abc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fec28006fd0 tx=0x7fec28006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:44.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:44.115+0000 7fec402ee700 1 -- 172.21.15.33:0/783710511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 msgr2=0x7fec38106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:44.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:44.115+0000 7fec402ee700 1 --2- 172.21.15.33:0/783710511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 0x7fec38106f70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fec34003e30 tx=0x7fec34011e80 comp rx=0 tx=0).stop 2024-05-08T21:11:44.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:44.115+0000 7fec402ee700 1 -- 172.21.15.33:0/783710511 shutdown_connections 2024-05-08T21:11:44.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:44.115+0000 7fec402ee700 1 --2- 172.21.15.33:0/783710511 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fec24038700 0x7fec2403abc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:44.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:44.115+0000 7fec402ee700 1 --2- 172.21.15.33:0/783710511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec381047b0 0x7fec38106f70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:44.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:44.115+0000 7fec402ee700 1 -- 172.21.15.33:0/783710511 >> 172.21.15.33:0/783710511 conn(0x7fec38100100 msgr2=0x7fec38100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:44.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:44.116+0000 7fec402ee700 1 -- 172.21.15.33:0/783710511 shutdown_connections 2024-05-08T21:11:44.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:44.116+0000 7fec402ee700 1 -- 172.21.15.33:0/783710511 wait complete. 2024-05-08T21:11:45.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:45 smithi033 ceph-mon[42528]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:11:45.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:45 smithi033 ceph-mon[42528]: Saving service mgr spec with placement count:2 2024-05-08T21:11:45.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:45 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.042+0000 7f2100e59700 1 Processor -- start 2024-05-08T21:11:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.042+0000 7f2100e59700 1 -- start start 2024-05-08T21:11:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.043+0000 7f2100e59700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 0x7f20fc07a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.043+0000 7f2100e59700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc07a8c0 con 0x7f20fc07be90 2024-05-08T21:11:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.043+0000 7f20fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 0x7f20fc07a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.043+0000 7f20fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 0x7f20fc07a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44178/0 (socket says 172.21.15.33:44178) 2024-05-08T21:11:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.043+0000 7f20fa59c700 1 -- 172.21.15.33:0/2176358786 learned_addr learned my addr 172.21.15.33:0/2176358786 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.043+0000 7f20fa59c700 1 -- 172.21.15.33:0/2176358786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc07aa00 con 0x7f20fc07be90 2024-05-08T21:11:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.043+0000 7f20fa59c700 1 --2- 172.21.15.33:0/2176358786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 0x7f20fc07a2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0098a0 tx=0x7f20ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5583e14034647c76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.044+0000 7f20f959a700 1 -- 172.21.15.33:0/2176358786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20ec004160 con 0x7f20fc07be90 2024-05-08T21:11:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.044+0000 7f20f959a700 1 -- 172.21.15.33:0/2176358786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20ec0042c0 con 0x7f20fc07be90 2024-05-08T21:11:45.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.044+0000 7f20f959a700 1 -- 172.21.15.33:0/2176358786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20ec0118e0 con 0x7f20fc07be90 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.044+0000 7f2100e59700 1 -- 172.21.15.33:0/2176358786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 msgr2=0x7f20fc07a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.044+0000 7f2100e59700 1 --2- 172.21.15.33:0/2176358786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 0x7f20fc07a2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0098a0 tx=0x7f20ec009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.044+0000 7f2100e59700 1 -- 172.21.15.33:0/2176358786 shutdown_connections 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.044+0000 7f2100e59700 1 --2- 172.21.15.33:0/2176358786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 0x7f20fc07a2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.044+0000 7f2100e59700 1 -- 172.21.15.33:0/2176358786 >> 172.21.15.33:0/2176358786 conn(0x7f20fc101ae0 msgr2=0x7f20fc103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.044+0000 7f2100e59700 1 -- 172.21.15.33:0/2176358786 shutdown_connections 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.044+0000 7f2100e59700 1 -- 172.21.15.33:0/2176358786 wait complete. 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.045+0000 7f2100e59700 1 Processor -- start 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.045+0000 7f2100e59700 1 -- start start 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.046+0000 7f2100e59700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 0x7f20fc19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.046+0000 7f2100e59700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19c940 con 0x7f20fc07be90 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.046+0000 7f20fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 0x7f20fc19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.046+0000 7f20fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 0x7f20fc19c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44180/0 (socket says 172.21.15.33:44180) 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.046+0000 7f20fa59c700 1 -- 172.21.15.33:0/3093891296 learned_addr learned my addr 172.21.15.33:0/3093891296 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:45.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.046+0000 7f20fa59c700 1 -- 172.21.15.33:0/3093891296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec009550 con 0x7f20fc07be90 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.046+0000 7f20fa59c700 1 --2- 172.21.15.33:0/3093891296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 0x7f20fc19c400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f20ec003e30 tx=0x7f20ec011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.046+0000 7f20eb7fe700 1 -- 172.21.15.33:0/3093891296 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20ec0036a0 con 0x7f20fc07be90 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.046+0000 7f20eb7fe700 1 -- 172.21.15.33:0/3093891296 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20ec01b440 con 0x7f20fc07be90 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.046+0000 7f2100e59700 1 -- 172.21.15.33:0/3093891296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc19cb40 con 0x7f20fc07be90 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.046+0000 7f20eb7fe700 1 -- 172.21.15.33:0/3093891296 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20ec0193d0 con 0x7f20fc07be90 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.047+0000 7f2100e59700 1 -- 172.21.15.33:0/3093891296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc19cfe0 con 0x7f20fc07be90 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.047+0000 7f20eb7fe700 1 -- 172.21.15.33:0/3093891296 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f20ec019690 con 0x7f20fc07be90 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.047+0000 7f2100e59700 1 -- 172.21.15.33:0/3093891296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20fc10c8e0 con 0x7f20fc07be90 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.048+0000 7f20eb7fe700 1 --2- 172.21.15.33:0/3093891296 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f20e40386b0 0x7f20e403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.048+0000 7f20eb7fe700 1 -- 172.21.15.33:0/3093891296 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f20ec04c6a0 con 0x7f20fc07be90 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.048+0000 7f20f9d9b700 1 --2- 172.21.15.33:0/3093891296 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f20e40386b0 0x7f20e403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.048+0000 7f20f9d9b700 1 --2- 172.21.15.33:0/3093891296 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f20e40386b0 0x7f20e403ab70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f20f0006fd0 tx=0x7f20f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.053+0000 7f20eb7fe700 1 -- 172.21.15.33:0/3093891296 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f20ec018db0 con 0x7f20fc07be90 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.277+0000 7f2100e59700 1 -- 172.21.15.33:0/3093891296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f20fc19f870 con 0x7f20fc07be90 2024-05-08T21:11:45.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.281+0000 7f20eb7fe700 1 -- 172.21.15.33:0/3093891296 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f20ec018350 con 0x7f20fc07be90 2024-05-08T21:11:45.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.283+0000 7f2100e59700 1 -- 172.21.15.33:0/3093891296 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f20e40386b0 msgr2=0x7f20e403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:45.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.283+0000 7f2100e59700 1 --2- 172.21.15.33:0/3093891296 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f20e40386b0 0x7f20e403ab70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f20f0006fd0 tx=0x7f20f0006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:45.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.283+0000 7f2100e59700 1 -- 172.21.15.33:0/3093891296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 msgr2=0x7f20fc19c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:45.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.283+0000 7f2100e59700 1 --2- 172.21.15.33:0/3093891296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 0x7f20fc19c400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f20ec003e30 tx=0x7f20ec011e80 comp rx=0 tx=0).stop 2024-05-08T21:11:45.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.284+0000 7f2100e59700 1 -- 172.21.15.33:0/3093891296 shutdown_connections 2024-05-08T21:11:45.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.284+0000 7f2100e59700 1 --2- 172.21.15.33:0/3093891296 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f20e40386b0 0x7f20e403ab70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:45.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.284+0000 7f2100e59700 1 --2- 172.21.15.33:0/3093891296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20fc07be90 0x7f20fc19c400 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:45.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.284+0000 7f2100e59700 1 -- 172.21.15.33:0/3093891296 >> 172.21.15.33:0/3093891296 conn(0x7f20fc101ae0 msgr2=0x7f20fc102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:45.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.284+0000 7f2100e59700 1 -- 172.21.15.33:0/3093891296 shutdown_connections 2024-05-08T21:11:45.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:45.284+0000 7f2100e59700 1 -- 172.21.15.33:0/3093891296 wait complete. 2024-05-08T21:11:46.554 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:46 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3093891296' entity='client.admin' 2024-05-08T21:11:46.554 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:46 smithi033 ceph-mon[42528]: from='mgr.14120 172.21.15.33:0/37902090' entity='mgr.x' 2024-05-08T21:11:47.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.069+0000 7ff62fd83700 1 Processor -- start 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.069+0000 7ff62fd83700 1 -- start start 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.069+0000 7ff62fd83700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 0x7ff62807a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.069+0000 7ff62fd83700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62807a8e0 con 0x7ff62807beb0 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.069+0000 7ff62db1f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 0x7ff62807a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.069+0000 7ff62db1f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 0x7ff62807a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44182/0 (socket says 172.21.15.33:44182) 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.069+0000 7ff62db1f700 1 -- 172.21.15.33:0/146486520 learned_addr learned my addr 172.21.15.33:0/146486520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.070+0000 7ff62db1f700 1 -- 172.21.15.33:0/146486520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62807aa20 con 0x7ff62807beb0 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.070+0000 7ff62db1f700 1 --2- 172.21.15.33:0/146486520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 0x7ff62807a310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6240098a0 tx=0x7ff624009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2557d17c3b95ab72 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.070+0000 7ff62cb1d700 1 -- 172.21.15.33:0/146486520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff624004160 con 0x7ff62807beb0 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.070+0000 7ff62cb1d700 1 -- 172.21.15.33:0/146486520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff6240042c0 con 0x7ff62807beb0 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.070+0000 7ff62cb1d700 1 -- 172.21.15.33:0/146486520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6240118e0 con 0x7ff62807beb0 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.071+0000 7ff62fd83700 1 -- 172.21.15.33:0/146486520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 msgr2=0x7ff62807a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.071+0000 7ff62fd83700 1 --2- 172.21.15.33:0/146486520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 0x7ff62807a310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6240098a0 tx=0x7ff624009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.071+0000 7ff62fd83700 1 -- 172.21.15.33:0/146486520 shutdown_connections 2024-05-08T21:11:47.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.071+0000 7ff62fd83700 1 --2- 172.21.15.33:0/146486520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 0x7ff62807a310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.071+0000 7ff62fd83700 1 -- 172.21.15.33:0/146486520 >> 172.21.15.33:0/146486520 conn(0x7ff628101b00 msgr2=0x7ff628103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.071+0000 7ff62fd83700 1 -- 172.21.15.33:0/146486520 shutdown_connections 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.071+0000 7ff62fd83700 1 -- 172.21.15.33:0/146486520 wait complete. 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.072+0000 7ff62fd83700 1 Processor -- start 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.072+0000 7ff62fd83700 1 -- start start 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.072+0000 7ff62fd83700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 0x7ff62819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.072+0000 7ff62fd83700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62819c900 con 0x7ff62807beb0 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.072+0000 7ff62db1f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 0x7ff62819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.073+0000 7ff62db1f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 0x7ff62819c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44190/0 (socket says 172.21.15.33:44190) 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.073+0000 7ff62db1f700 1 -- 172.21.15.33:0/1306093645 learned_addr learned my addr 172.21.15.33:0/1306093645 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.073+0000 7ff62db1f700 1 -- 172.21.15.33:0/1306093645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff624009550 con 0x7ff62807beb0 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.073+0000 7ff62db1f700 1 --2- 172.21.15.33:0/1306093645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 0x7ff62819c3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff624003f70 tx=0x7ff6240119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.073+0000 7ff61effd700 1 -- 172.21.15.33:0/1306093645 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6240037d0 con 0x7ff62807beb0 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.073+0000 7ff61effd700 1 -- 172.21.15.33:0/1306093645 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff62401b440 con 0x7ff62807beb0 2024-05-08T21:11:47.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.073+0000 7ff61effd700 1 -- 172.21.15.33:0/1306093645 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6240193d0 con 0x7ff62807beb0 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.073+0000 7ff62fd83700 1 -- 172.21.15.33:0/1306093645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62819cb00 con 0x7ff62807beb0 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.073+0000 7ff62fd83700 1 -- 172.21.15.33:0/1306093645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62819cfa0 con 0x7ff62807beb0 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.075+0000 7ff62fd83700 1 -- 172.21.15.33:0/1306093645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62810c8a0 con 0x7ff62807beb0 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.075+0000 7ff61effd700 1 -- 172.21.15.33:0/1306093645 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7ff62401b860 con 0x7ff62807beb0 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.076+0000 7ff61effd700 1 --2- 172.21.15.33:0/1306093645 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7ff614038700 0x7ff61403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.076+0000 7ff61effd700 1 -- 172.21.15.33:0/1306093645 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff62404c680 con 0x7ff62807beb0 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.079+0000 7ff62d31e700 1 --2- 172.21.15.33:0/1306093645 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7ff614038700 0x7ff61403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.079+0000 7ff62d31e700 1 --2- 172.21.15.33:0/1306093645 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7ff614038700 0x7ff61403abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff618006fd0 tx=0x7ff618006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.081+0000 7ff61effd700 1 -- 172.21.15.33:0/1306093645 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff62400be40 con 0x7ff62807beb0 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.319+0000 7ff62fd83700 1 -- 172.21.15.33:0/1306093645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7ff62810cb10 con 0x7ff62807beb0 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.323+0000 7ff61effd700 1 -- 172.21.15.33:0/1306093645 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7ff62401f0c0 con 0x7ff62807beb0 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.326+0000 7ff62fd83700 1 -- 172.21.15.33:0/1306093645 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7ff614038700 msgr2=0x7ff61403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.326+0000 7ff62fd83700 1 --2- 172.21.15.33:0/1306093645 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7ff614038700 0x7ff61403abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff618006fd0 tx=0x7ff618006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.326+0000 7ff62fd83700 1 -- 172.21.15.33:0/1306093645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 msgr2=0x7ff62819c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:47.013 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.326+0000 7ff62fd83700 1 --2- 172.21.15.33:0/1306093645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 0x7ff62819c3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff624003f70 tx=0x7ff6240119a0 comp rx=0 tx=0).stop 2024-05-08T21:11:47.014 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.326+0000 7ff62fd83700 1 -- 172.21.15.33:0/1306093645 shutdown_connections 2024-05-08T21:11:47.014 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.326+0000 7ff62fd83700 1 --2- 172.21.15.33:0/1306093645 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7ff614038700 0x7ff61403abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:47.014 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.326+0000 7ff62fd83700 1 --2- 172.21.15.33:0/1306093645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff62807beb0 0x7ff62819c3c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:47.014 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.326+0000 7ff62fd83700 1 -- 172.21.15.33:0/1306093645 >> 172.21.15.33:0/1306093645 conn(0x7ff628101b00 msgr2=0x7ff6281026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:47.014 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.326+0000 7ff62fd83700 1 -- 172.21.15.33:0/1306093645 shutdown_connections 2024-05-08T21:11:47.014 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:46.326+0000 7ff62fd83700 1 -- 172.21.15.33:0/1306093645 wait complete. 2024-05-08T21:11:47.014 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling the dashboard module... 2024-05-08T21:11:47.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:47 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1306093645' entity='client.admin' 2024-05-08T21:11:48.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:48 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3558665047' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-08T21:11:48.709 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:48 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: ignoring --setuser ceph since I am not root 2024-05-08T21:11:48.709 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:48 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: ignoring --setgroup ceph since I am not root 2024-05-08T21:11:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.398+0000 7f8506725700 1 Processor -- start 2024-05-08T21:11:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.398+0000 7f8506725700 1 -- start start 2024-05-08T21:11:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.398+0000 7f8506725700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 0x7f850007a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.398+0000 7f8506725700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850007a8e0 con 0x7f850007beb0 2024-05-08T21:11:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.398+0000 7f84fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 0x7f850007a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.398+0000 7f84fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 0x7f850007a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44204/0 (socket says 172.21.15.33:44204) 2024-05-08T21:11:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.398+0000 7f84fffff700 1 -- 172.21.15.33:0/1500699452 learned_addr learned my addr 172.21.15.33:0/1500699452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.398+0000 7f84fffff700 1 -- 172.21.15.33:0/1500699452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850007aa20 con 0x7f850007beb0 2024-05-08T21:11:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.399+0000 7f84fffff700 1 --2- 172.21.15.33:0/1500699452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 0x7f850007a310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f84f00098a0 tx=0x7f84f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c72b7aa2d39d71 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:48.768 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.399+0000 7f84feffd700 1 -- 172.21.15.33:0/1500699452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84f0004160 con 0x7f850007beb0 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.399+0000 7f84feffd700 1 -- 172.21.15.33:0/1500699452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84f00042c0 con 0x7f850007beb0 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.399+0000 7f84feffd700 1 -- 172.21.15.33:0/1500699452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84f00118e0 con 0x7f850007beb0 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.399+0000 7f8506725700 1 -- 172.21.15.33:0/1500699452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 msgr2=0x7f850007a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.399+0000 7f8506725700 1 --2- 172.21.15.33:0/1500699452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 0x7f850007a310 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f84f00098a0 tx=0x7f84f0009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.400+0000 7f8506725700 1 -- 172.21.15.33:0/1500699452 shutdown_connections 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.400+0000 7f8506725700 1 --2- 172.21.15.33:0/1500699452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 0x7f850007a310 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.400+0000 7f8506725700 1 -- 172.21.15.33:0/1500699452 >> 172.21.15.33:0/1500699452 conn(0x7f8500101a60 msgr2=0x7f8500103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.400+0000 7f8506725700 1 -- 172.21.15.33:0/1500699452 shutdown_connections 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.400+0000 7f8506725700 1 -- 172.21.15.33:0/1500699452 wait complete. 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.400+0000 7f8506725700 1 Processor -- start 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.400+0000 7f8506725700 1 -- start start 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.401+0000 7f8506725700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 0x7f850019c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.401+0000 7f8506725700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850019c8f0 con 0x7f850007beb0 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.401+0000 7f84fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 0x7f850019c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:48.769 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.401+0000 7f84fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 0x7f850019c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44214/0 (socket says 172.21.15.33:44214) 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.401+0000 7f84fffff700 1 -- 172.21.15.33:0/3558665047 learned_addr learned my addr 172.21.15.33:0/3558665047 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.401+0000 7f84fffff700 1 -- 172.21.15.33:0/3558665047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f0009550 con 0x7f850007beb0 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.401+0000 7f84fffff700 1 --2- 172.21.15.33:0/3558665047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 0x7f850019c3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84f0003f70 tx=0x7f84f00119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.401+0000 7f84fd7fa700 1 -- 172.21.15.33:0/3558665047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84f00037d0 con 0x7f850007beb0 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.402+0000 7f84fd7fa700 1 -- 172.21.15.33:0/3558665047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84f001b440 con 0x7f850007beb0 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.402+0000 7f8506725700 1 -- 172.21.15.33:0/3558665047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850019caf0 con 0x7f850007beb0 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.402+0000 7f84fd7fa700 1 -- 172.21.15.33:0/3558665047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84f00193d0 con 0x7f850007beb0 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.402+0000 7f8506725700 1 -- 172.21.15.33:0/3558665047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850019cf90 con 0x7f850007beb0 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.402+0000 7f84fd7fa700 1 -- 172.21.15.33:0/3558665047 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45541+0+0 (secure 0 0 0) 0x7f84f0019610 con 0x7f850007beb0 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.403+0000 7f8506725700 1 -- 172.21.15.33:0/3558665047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850010c860 con 0x7f850007beb0 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.403+0000 7f84fd7fa700 1 --2- 172.21.15.33:0/3558665047 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f84e8038700 0x7f84e803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.403+0000 7f84fd7fa700 1 -- 172.21.15.33:0/3558665047 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f84f004c720 con 0x7f850007beb0 2024-05-08T21:11:48.770 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.403+0000 7f84ff7fe700 1 --2- 172.21.15.33:0/3558665047 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f84e8038700 0x7f84e803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.403+0000 7f84ff7fe700 1 --2- 172.21.15.33:0/3558665047 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f84e8038700 0x7f84e803abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f84f4006fd0 tx=0x7f84f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.408+0000 7f84fd7fa700 1 -- 172.21.15.33:0/3558665047 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f84f0019e30 con 0x7f850007beb0 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:47.685+0000 7f8506725700 1 -- 172.21.15.33:0/3558665047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f850010caa0 con 0x7f850007beb0 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:48.330+0000 7f84fd7fa700 1 -- 172.21.15.33:0/3558665047 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f84f001b860 con 0x7f850007beb0 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:48.331+0000 7f84fd7fa700 1 -- 172.21.15.33:0/3558665047 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f84f002c510 con 0x7f850007beb0 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:48.334+0000 7f8506725700 1 -- 172.21.15.33:0/3558665047 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f84e8038700 msgr2=0x7f84e803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:48.334+0000 7f8506725700 1 --2- 172.21.15.33:0/3558665047 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f84e8038700 0x7f84e803abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f84f4006fd0 tx=0x7f84f4006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:48.334+0000 7f8506725700 1 -- 172.21.15.33:0/3558665047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 msgr2=0x7f850019c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:48.334+0000 7f8506725700 1 --2- 172.21.15.33:0/3558665047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 0x7f850019c3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f84f0003f70 tx=0x7f84f00119a0 comp rx=0 tx=0).stop 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:48.335+0000 7f8506725700 1 -- 172.21.15.33:0/3558665047 shutdown_connections 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:48.335+0000 7f8506725700 1 --2- 172.21.15.33:0/3558665047 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f84e8038700 0x7f84e803abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:48.335+0000 7f8506725700 1 --2- 172.21.15.33:0/3558665047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f850007beb0 0x7f850019c3b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:48.335+0000 7f8506725700 1 -- 172.21.15.33:0/3558665047 >> 172.21.15.33:0/3558665047 conn(0x7f8500101a60 msgr2=0x7f8500102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:48.771 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:48.335+0000 7f8506725700 1 -- 172.21.15.33:0/3558665047 shutdown_connections 2024-05-08T21:11:48.772 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:48.335+0000 7f8506725700 1 -- 172.21.15.33:0/3558665047 wait complete. 2024-05-08T21:11:49.052 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:48 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:48.837+0000 7ff7b5e90200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:11:49.052 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:48 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:48.947+0000 7ff7b5e90200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:11:49.332 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:49 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:49.048+0000 7ff7b5e90200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:11:49.332 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:49 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:49.148+0000 7ff7b5e90200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:11:49.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:49 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3558665047' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-08T21:11:49.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:49 smithi033 ceph-mon[42528]: mgrmap e9: x(active, since 16s) 2024-05-08T21:11:49.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:11:49.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-08T21:11:49.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T21:11:49.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T21:11:49.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T21:11:49.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:11:49.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.120+0000 7f7190854700 1 Processor -- start 2024-05-08T21:11:49.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.120+0000 7f7190854700 1 -- start start 2024-05-08T21:11:49.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.120+0000 7f7190854700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 0x7f718c106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:49.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.120+0000 7f7190854700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c106f10 con 0x7f718c106520 2024-05-08T21:11:49.794 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.121+0000 7f718a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 0x7f718c106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.121+0000 7f718a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 0x7f718c106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44234/0 (socket says 172.21.15.33:44234) 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.121+0000 7f718a59c700 1 -- 172.21.15.33:0/621819452 learned_addr learned my addr 172.21.15.33:0/621819452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.121+0000 7f718a59c700 1 -- 172.21.15.33:0/621819452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c107720 con 0x7f718c106520 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.121+0000 7f718a59c700 1 --2- 172.21.15.33:0/621819452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 0x7f718c106940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f717c0098a0 tx=0x7f717c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3438e851375594ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.121+0000 7f718959a700 1 -- 172.21.15.33:0/621819452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f717c004160 con 0x7f718c106520 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.122+0000 7f718959a700 1 -- 172.21.15.33:0/621819452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f717c0042c0 con 0x7f718c106520 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.122+0000 7f718959a700 1 -- 172.21.15.33:0/621819452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f717c0118e0 con 0x7f718c106520 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.122+0000 7f7190854700 1 -- 172.21.15.33:0/621819452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 msgr2=0x7f718c106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.122+0000 7f7190854700 1 --2- 172.21.15.33:0/621819452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 0x7f718c106940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f717c0098a0 tx=0x7f717c009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.122+0000 7f7190854700 1 -- 172.21.15.33:0/621819452 shutdown_connections 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.122+0000 7f7190854700 1 --2- 172.21.15.33:0/621819452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 0x7f718c106940 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.122+0000 7f7190854700 1 -- 172.21.15.33:0/621819452 >> 172.21.15.33:0/621819452 conn(0x7f718c101ae0 msgr2=0x7f718c103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.122+0000 7f7190854700 1 -- 172.21.15.33:0/621819452 shutdown_connections 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.122+0000 7f7190854700 1 -- 172.21.15.33:0/621819452 wait complete. 2024-05-08T21:11:49.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.123+0000 7f7190854700 1 Processor -- start 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.124+0000 7f7190854700 1 -- start start 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.124+0000 7f7190854700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 0x7f718c19a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.124+0000 7f7190854700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c19a760 con 0x7f718c106520 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.124+0000 7f718a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 0x7f718c19a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.124+0000 7f718a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 0x7f718c19a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44240/0 (socket says 172.21.15.33:44240) 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.124+0000 7f718a59c700 1 -- 172.21.15.33:0/1759961155 learned_addr learned my addr 172.21.15.33:0/1759961155 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.124+0000 7f718a59c700 1 -- 172.21.15.33:0/1759961155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c009550 con 0x7f718c106520 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.124+0000 7f718a59c700 1 --2- 172.21.15.33:0/1759961155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 0x7f718c19a220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f717c003e30 tx=0x7f717c011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.125+0000 7f717b7fe700 1 -- 172.21.15.33:0/1759961155 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f717c0036a0 con 0x7f718c106520 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.125+0000 7f7190854700 1 -- 172.21.15.33:0/1759961155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718c19a960 con 0x7f718c106520 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.125+0000 7f717b7fe700 1 -- 172.21.15.33:0/1759961155 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f717c01b440 con 0x7f718c106520 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.125+0000 7f717b7fe700 1 -- 172.21.15.33:0/1759961155 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f717c0193d0 con 0x7f718c106520 2024-05-08T21:11:49.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.125+0000 7f7190854700 1 -- 172.21.15.33:0/1759961155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718c19ae00 con 0x7f718c106520 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.126+0000 7f7190854700 1 -- 172.21.15.33:0/1759961155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718c10a950 con 0x7f718c106520 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.126+0000 7f717b7fe700 1 -- 172.21.15.33:0/1759961155 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f717c019690 con 0x7f718c106520 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.126+0000 7f717b7fe700 1 --2- 172.21.15.33:0/1759961155 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f7174038700 0x7f717403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.126+0000 7f717b7fe700 1 -- 172.21.15.33:0/1759961155 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f717c04c580 con 0x7f718c106520 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.126+0000 7f7189d9b700 1 -- 172.21.15.33:0/1759961155 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f7174038700 msgr2=0x7f717403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3413344235 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.126+0000 7f7189d9b700 1 --2- 172.21.15.33:0/1759961155 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f7174038700 0x7f717403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.131+0000 7f717b7fe700 1 -- 172.21.15.33:0/1759961155 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f717c018db0 con 0x7f718c106520 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.327+0000 7f7189d9b700 1 -- 172.21.15.33:0/1759961155 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f7174038700 msgr2=0x7f717403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3413344235 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.327+0000 7f7189d9b700 1 --2- 172.21.15.33:0/1759961155 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f7174038700 0x7f717403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.407+0000 7f7190854700 1 -- 172.21.15.33:0/1759961155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f718c19d690 con 0x7f718c106520 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.408+0000 7f717b7fe700 1 -- 172.21.15.33:0/1759961155 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f717c045e60 con 0x7f718c106520 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.410+0000 7f7190854700 1 -- 172.21.15.33:0/1759961155 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f7174038700 msgr2=0x7f717403abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.410+0000 7f7190854700 1 --2- 172.21.15.33:0/1759961155 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f7174038700 0x7f717403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.410+0000 7f7190854700 1 -- 172.21.15.33:0/1759961155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 msgr2=0x7f718c19a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.410+0000 7f7190854700 1 --2- 172.21.15.33:0/1759961155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 0x7f718c19a220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f717c003e30 tx=0x7f717c011e80 comp rx=0 tx=0).stop 2024-05-08T21:11:49.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.410+0000 7f7190854700 1 -- 172.21.15.33:0/1759961155 shutdown_connections 2024-05-08T21:11:49.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.410+0000 7f7190854700 1 --2- 172.21.15.33:0/1759961155 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7f7174038700 0x7f717403abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:49.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.410+0000 7f7190854700 1 --2- 172.21.15.33:0/1759961155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c106520 0x7f718c19a220 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:49.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.410+0000 7f7190854700 1 -- 172.21.15.33:0/1759961155 >> 172.21.15.33:0/1759961155 conn(0x7f718c101ae0 msgr2=0x7f718c102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:49.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.410+0000 7f7190854700 1 -- 172.21.15.33:0/1759961155 shutdown_connections 2024-05-08T21:11:49.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:49.410+0000 7f7190854700 1 -- 172.21.15.33:0/1759961155 wait complete. 2024-05-08T21:11:49.798 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-05-08T21:11:49.798 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 9... 2024-05-08T21:11:50.206 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:49 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:49.893+0000 7ff7b5e90200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:11:50.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:50 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1759961155' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T21:11:50.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:50 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:50.202+0000 7ff7b5e90200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:11:50.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:50 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:50.299+0000 7ff7b5e90200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:11:50.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:50 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:50.425+0000 7ff7b5e90200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:11:50.958 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:50 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:50.606+0000 7ff7b5e90200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:11:50.958 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:50 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:50.751+0000 7ff7b5e90200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:11:50.958 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:50 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:50.848+0000 7ff7b5e90200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:11:51.708 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:51 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:51.270+0000 7ff7b5e90200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:11:52.040 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:51 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:51.713+0000 7ff7b5e90200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:11:52.040 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:51 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:51.932+0000 7ff7b5e90200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:11:52.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:52 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:52.037+0000 7ff7b5e90200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:11:53.189 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:52 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:52.908+0000 7ff7b5e90200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:11:53.189 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:53 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:53.017+0000 7ff7b5e90200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:11:53.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:53 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:53.185+0000 7ff7b5e90200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:11:54.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:53 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:53.907+0000 7ff7b5e90200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:11:54.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:54 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:54.113+0000 7ff7b5e90200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:11:54.708 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:54 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:54.333+0000 7ff7b5e90200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:11:55.708 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:55 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:55.393+0000 7ff7b5e90200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:11:56.036 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:55 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:55.716+0000 7ff7b5e90200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:11:56.036 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:55 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:55.816+0000 7ff7b5e90200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:11:56.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:11:56 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:11:56.033+0000 7ff7b5e90200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:11:56.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:56 smithi033 ceph-mon[42528]: Active manager daemon x restarted 2024-05-08T21:11:56.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:56 smithi033 ceph-mon[42528]: Activating manager daemon x 2024-05-08T21:11:56.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:56 smithi033 ceph-mon[42528]: osdmap e3: 0 total, 0 up, 0 in 2024-05-08T21:11:56.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:56 smithi033 ceph-mon[42528]: mgrmap e10: x(active, starting, since 0.00359965s) 2024-05-08T21:11:56.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:11:56.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T21:11:56.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T21:11:56.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T21:11:56.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T21:11:56.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:56 smithi033 ceph-mon[42528]: Manager daemon x is now available 2024-05-08T21:11:56.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:11:57.347 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T21:11:57.347 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T21:11:57.348 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:11:57.348 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:57 smithi033 ceph-mon[42528]: mgrmap e11: x(active, since 1.00866s) 2024-05-08T21:11:57.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-05-08T21:11:57.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-08T21:11:57.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T21:11:57.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-05-08T21:11:57.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.163+0000 7fc39642a700 1 Processor -- start 2024-05-08T21:11:57.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.163+0000 7fc39642a700 1 -- start start 2024-05-08T21:11:57.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.163+0000 7fc39642a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 0x7fc3901068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:57.424 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.163+0000 7fc39642a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc390106eb0 con 0x7fc3901064c0 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.164+0000 7fc38ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 0x7fc3901068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.164+0000 7fc38ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 0x7fc3901068e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44244/0 (socket says 172.21.15.33:44244) 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.164+0000 7fc38ffff700 1 -- 172.21.15.33:0/2418446628 learned_addr learned my addr 172.21.15.33:0/2418446628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.164+0000 7fc38ffff700 1 -- 172.21.15.33:0/2418446628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3901076c0 con 0x7fc3901064c0 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.164+0000 7fc38ffff700 1 --2- 172.21.15.33:0/2418446628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 0x7fc3901068e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc3800098a0 tx=0x7fc380009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3767c9f4a0593a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.164+0000 7fc38effd700 1 -- 172.21.15.33:0/2418446628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc380004160 con 0x7fc3901064c0 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.164+0000 7fc38effd700 1 -- 172.21.15.33:0/2418446628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3800042c0 con 0x7fc3901064c0 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.164+0000 7fc38effd700 1 -- 172.21.15.33:0/2418446628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3800118e0 con 0x7fc3901064c0 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.165+0000 7fc39642a700 1 -- 172.21.15.33:0/2418446628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 msgr2=0x7fc3901068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.165+0000 7fc39642a700 1 --2- 172.21.15.33:0/2418446628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 0x7fc3901068e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc3800098a0 tx=0x7fc380009bb0 comp rx=0 tx=0).stop 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.165+0000 7fc39642a700 1 -- 172.21.15.33:0/2418446628 shutdown_connections 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.165+0000 7fc39642a700 1 --2- 172.21.15.33:0/2418446628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 0x7fc3901068e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.165+0000 7fc39642a700 1 -- 172.21.15.33:0/2418446628 >> 172.21.15.33:0/2418446628 conn(0x7fc390101a60 msgr2=0x7fc390103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.165+0000 7fc39642a700 1 -- 172.21.15.33:0/2418446628 shutdown_connections 2024-05-08T21:11:57.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.165+0000 7fc39642a700 1 -- 172.21.15.33:0/2418446628 wait complete. 2024-05-08T21:11:57.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.166+0000 7fc39642a700 1 Processor -- start 2024-05-08T21:11:57.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.166+0000 7fc39642a700 1 -- start start 2024-05-08T21:11:57.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.166+0000 7fc39642a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 0x7fc39019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:57.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.166+0000 7fc39642a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39019c970 con 0x7fc3901064c0 2024-05-08T21:11:57.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.167+0000 7fc38ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 0x7fc39019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:57.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.167+0000 7fc38ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 0x7fc39019c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44258/0 (socket says 172.21.15.33:44258) 2024-05-08T21:11:57.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.167+0000 7fc38ffff700 1 -- 172.21.15.33:0/632682409 learned_addr learned my addr 172.21.15.33:0/632682409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:57.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.167+0000 7fc38ffff700 1 -- 172.21.15.33:0/632682409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc380009550 con 0x7fc3901064c0 2024-05-08T21:11:57.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.167+0000 7fc38ffff700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 0x7fc39019c430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc380003e30 tx=0x7fc380011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:57.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.167+0000 7fc38d7fa700 1 -- 172.21.15.33:0/632682409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3800036a0 con 0x7fc3901064c0 2024-05-08T21:11:57.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.167+0000 7fc39642a700 1 -- 172.21.15.33:0/632682409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39019cb70 con 0x7fc3901064c0 2024-05-08T21:11:57.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.167+0000 7fc38d7fa700 1 -- 172.21.15.33:0/632682409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc38001b440 con 0x7fc3901064c0 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.167+0000 7fc38d7fa700 1 -- 172.21.15.33:0/632682409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3800193d0 con 0x7fc3901064c0 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.167+0000 7fc39642a700 1 -- 172.21.15.33:0/632682409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39019d010 con 0x7fc3901064c0 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.168+0000 7fc38d7fa700 1 -- 172.21.15.33:0/632682409 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fc380019690 con 0x7fc3901064c0 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.168+0000 7fc38d7fa700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 0x7fc37803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.168+0000 7fc38d7fa700 1 -- 172.21.15.33:0/632682409 --> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc37803b2d0 con 0x7fc378038700 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.168+0000 7fc38d7fa700 1 -- 172.21.15.33:0/632682409 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc38004c560 con 0x7fc3901064c0 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.168+0000 7fc38f7fe700 1 -- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 msgr2=0x7fc37803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3413344235 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.169+0000 7fc38f7fe700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 0x7fc37803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.370+0000 7fc38f7fe700 1 -- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 msgr2=0x7fc37803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3413344235 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.370+0000 7fc38f7fe700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 0x7fc37803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.771+0000 7fc38f7fe700 1 -- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 msgr2=0x7fc37803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3413344235 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:50.771+0000 7fc38f7fe700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 0x7fc37803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T21:11:57.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:51.572+0000 7fc38f7fe700 1 -- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 msgr2=0x7fc37803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3413344235 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:51.572+0000 7fc38f7fe700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 0x7fc37803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:53.173+0000 7fc38f7fe700 1 -- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 msgr2=0x7fc37803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3413344235 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:53.173+0000 7fc38f7fe700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 0x7fc37803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:56.048+0000 7fc38d7fa700 1 -- 172.21.15.33:0/632682409 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 10) v1 ==== 45321+0+0 (secure 0 0 0) 0x7fc3800199e0 con 0x7fc3901064c0 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:56.048+0000 7fc38d7fa700 1 -- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 msgr2=0x7fc37803abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:56.048+0000 7fc38d7fa700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 0x7fc37803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.052+0000 7fc38d7fa700 1 -- 172.21.15.33:0/632682409 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7fc38004dfe0 con 0x7fc3901064c0 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.052+0000 7fc38d7fa700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc37803bff0 0x7fc37803e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.052+0000 7fc38d7fa700 1 -- 172.21.15.33:0/632682409 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc37803b2d0 con 0x7fc37803bff0 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.052+0000 7fc38f7fe700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc37803bff0 0x7fc37803e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.053+0000 7fc38f7fe700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc37803bff0 0x7fc37803e3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc384003a10 tx=0x7fc3840092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.054+0000 7fc38d7fa700 1 -- 172.21.15.33:0/632682409 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fc37803b2d0 con 0x7fc37803bff0 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.060+0000 7fc39642a700 1 -- 172.21.15.33:0/632682409 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc39019d1a0 con 0x7fc37803bff0 2024-05-08T21:11:57.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.061+0000 7fc38d7fa700 1 -- 172.21.15.33:0/632682409 <== mgr.14152 v2:172.21.15.33:6800/2288189532 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fc38400a050 con 0x7fc37803bff0 2024-05-08T21:11:57.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.061+0000 7fc39642a700 1 -- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc37803bff0 msgr2=0x7fc37803e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:57.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.061+0000 7fc39642a700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc37803bff0 0x7fc37803e3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc384003a10 tx=0x7fc3840092b0 comp rx=0 tx=0).stop 2024-05-08T21:11:57.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.061+0000 7fc39642a700 1 -- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 msgr2=0x7fc39019c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:57.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.061+0000 7fc39642a700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 0x7fc39019c430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc380003e30 tx=0x7fc380011e80 comp rx=0 tx=0).stop 2024-05-08T21:11:57.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.061+0000 7fc39642a700 1 -- 172.21.15.33:0/632682409 shutdown_connections 2024-05-08T21:11:57.429 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.061+0000 7fc39642a700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc37803bff0 0x7fc37803e3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:57.430 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.061+0000 7fc39642a700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3901064c0 0x7fc39019c430 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:57.430 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.061+0000 7fc39642a700 1 --2- 172.21.15.33:0/632682409 >> [v2:172.21.15.33:6800/3413344235,v1:172.21.15.33:6801/3413344235] conn(0x7fc378038700 0x7fc37803abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:57.430 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.061+0000 7fc39642a700 1 -- 172.21.15.33:0/632682409 >> 172.21.15.33:0/632682409 conn(0x7fc390101a60 msgr2=0x7fc390102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:57.430 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.061+0000 7fc39642a700 1 -- 172.21.15.33:0/632682409 shutdown_connections 2024-05-08T21:11:57.430 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.061+0000 7fc39642a700 1 -- 172.21.15.33:0/632682409 wait complete. 2024-05-08T21:11:57.430 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 9 is available 2024-05-08T21:11:57.430 INFO:teuthology.orchestra.run.smithi033.stdout:Generating a dashboard self-signed certificate... 2024-05-08T21:11:58.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:58 smithi033 ceph-mon[42528]: [08/May/2024:21:11:57] ENGINE Bus STARTING 2024-05-08T21:11:58.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:58 smithi033 ceph-mon[42528]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T21:11:58.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:58 smithi033 ceph-mon[42528]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T21:11:58.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:58 smithi033 ceph-mon[42528]: [08/May/2024:21:11:57] ENGINE Serving on https://172.21.15.33:7150 2024-05-08T21:11:58.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:58 smithi033 ceph-mon[42528]: [08/May/2024:21:11:57] ENGINE Serving on http://172.21.15.33:8765 2024-05-08T21:11:58.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:58 smithi033 ceph-mon[42528]: [08/May/2024:21:11:57] ENGINE Bus STARTED 2024-05-08T21:11:58.394 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:11:58.641 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-08T21:11:58.641 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.798+0000 7fc1f441f700 1 Processor -- start 2024-05-08T21:11:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.798+0000 7fc1f441f700 1 -- start start 2024-05-08T21:11:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.799+0000 7fc1f441f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 0x7fc1ec109390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.799+0000 7fc1f441f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec079ea0 con 0x7fc1ec106fa0 2024-05-08T21:11:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.799+0000 7fc1f21bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 0x7fc1ec109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.799+0000 7fc1f21bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 0x7fc1ec109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34688/0 (socket says 172.21.15.33:34688) 2024-05-08T21:11:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.799+0000 7fc1f21bb700 1 -- 172.21.15.33:0/891018713 learned_addr learned my addr 172.21.15.33:0/891018713 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.799+0000 7fc1f21bb700 1 -- 172.21.15.33:0/891018713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec1098d0 con 0x7fc1ec106fa0 2024-05-08T21:11:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.799+0000 7fc1f21bb700 1 --2- 172.21.15.33:0/891018713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 0x7fc1ec109390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009e80 tx=0x7fc1dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e430a8c3dc708f36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.800+0000 7fc1f11b9700 1 -- 172.21.15.33:0/891018713 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1dc00e070 con 0x7fc1ec106fa0 2024-05-08T21:11:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.800+0000 7fc1f11b9700 1 -- 172.21.15.33:0/891018713 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1dc00b8c0 con 0x7fc1ec106fa0 2024-05-08T21:11:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.800+0000 7fc1f11b9700 1 -- 172.21.15.33:0/891018713 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1dc004360 con 0x7fc1ec106fa0 2024-05-08T21:11:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.800+0000 7fc1f441f700 1 -- 172.21.15.33:0/891018713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 msgr2=0x7fc1ec109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:58.643 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.800+0000 7fc1f441f700 1 --2- 172.21.15.33:0/891018713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 0x7fc1ec109390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009e80 tx=0x7fc1dc00b330 comp rx=0 tx=0).stop 2024-05-08T21:11:58.643 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.800+0000 7fc1f441f700 1 -- 172.21.15.33:0/891018713 shutdown_connections 2024-05-08T21:11:58.643 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.800+0000 7fc1f441f700 1 --2- 172.21.15.33:0/891018713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 0x7fc1ec109390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:58.643 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.800+0000 7fc1f441f700 1 -- 172.21.15.33:0/891018713 >> 172.21.15.33:0/891018713 conn(0x7fc1ec100a10 msgr2=0x7fc1ec102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:58.643 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.800+0000 7fc1f441f700 1 -- 172.21.15.33:0/891018713 shutdown_connections 2024-05-08T21:11:58.643 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.801+0000 7fc1f441f700 1 -- 172.21.15.33:0/891018713 wait complete. 2024-05-08T21:11:58.643 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.801+0000 7fc1f441f700 1 Processor -- start 2024-05-08T21:11:58.643 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.801+0000 7fc1f441f700 1 -- start start 2024-05-08T21:11:58.643 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.801+0000 7fc1f441f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 0x7fc1ec19a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:58.643 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.801+0000 7fc1f441f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec19a680 con 0x7fc1ec106fa0 2024-05-08T21:11:58.643 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.801+0000 7fc1f21bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 0x7fc1ec19a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:58.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.801+0000 7fc1f21bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 0x7fc1ec19a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34696/0 (socket says 172.21.15.33:34696) 2024-05-08T21:11:58.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.802+0000 7fc1f21bb700 1 -- 172.21.15.33:0/1889536716 learned_addr learned my addr 172.21.15.33:0/1889536716 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:11:58.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.802+0000 7fc1f21bb700 1 -- 172.21.15.33:0/1889536716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1dc009b30 con 0x7fc1ec106fa0 2024-05-08T21:11:58.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.802+0000 7fc1f21bb700 1 --2- 172.21.15.33:0/1889536716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 0x7fc1ec19a140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00b7b0 tx=0x7fc1dc003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:58.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.802+0000 7fc1eb7fe700 1 -- 172.21.15.33:0/1889536716 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1dc00e040 con 0x7fc1ec106fa0 2024-05-08T21:11:58.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.802+0000 7fc1eb7fe700 1 -- 172.21.15.33:0/1889536716 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1dc00bd70 con 0x7fc1ec106fa0 2024-05-08T21:11:58.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.802+0000 7fc1eb7fe700 1 -- 172.21.15.33:0/1889536716 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1dc01c590 con 0x7fc1ec106fa0 2024-05-08T21:11:58.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.802+0000 7fc1f441f700 1 -- 172.21.15.33:0/1889536716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ec19a880 con 0x7fc1ec106fa0 2024-05-08T21:11:58.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.802+0000 7fc1f441f700 1 -- 172.21.15.33:0/1889536716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ec19ad80 con 0x7fc1ec106fa0 2024-05-08T21:11:58.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.803+0000 7fc1eb7fe700 1 -- 172.21.15.33:0/1889536716 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) v1 ==== 45448+0+0 (secure 0 0 0) 0x7fc1dc01c850 con 0x7fc1ec106fa0 2024-05-08T21:11:58.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.803+0000 7fc1eb7fe700 1 --2- 172.21.15.33:0/1889536716 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1d40385c0 0x7fc1d403aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:11:58.644 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.803+0000 7fc1eb7fe700 1 -- 172.21.15.33:0/1889536716 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc1dc04c5f0 con 0x7fc1ec106fa0 2024-05-08T21:11:58.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.803+0000 7fc1e97fa700 1 -- 172.21.15.33:0/1889536716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc0052f0 con 0x7fc1ec106fa0 2024-05-08T21:11:58.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.803+0000 7fc1f19ba700 1 --2- 172.21.15.33:0/1889536716 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1d40385c0 0x7fc1d403aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:11:58.645 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.804+0000 7fc1f19ba700 1 --2- 172.21.15.33:0/1889536716 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1d40385c0 0x7fc1d403aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0006fd0 tx=0x7fc1e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:11:58.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:57.809+0000 7fc1eb7fe700 1 -- 172.21.15.33:0/1889536716 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc1dc017020 con 0x7fc1ec106fa0 2024-05-08T21:11:58.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:58.047+0000 7fc1e97fa700 1 -- 172.21.15.33:0/1889536716 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1cc000bc0 con 0x7fc1d40385c0 2024-05-08T21:11:58.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:58.225+0000 7fc1eb7fe700 1 -- 172.21.15.33:0/1889536716 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc1cc000bc0 con 0x7fc1d40385c0 2024-05-08T21:11:58.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:58.228+0000 7fc1e97fa700 1 -- 172.21.15.33:0/1889536716 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1d40385c0 msgr2=0x7fc1d403aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:58.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:58.228+0000 7fc1e97fa700 1 --2- 172.21.15.33:0/1889536716 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1d40385c0 0x7fc1d403aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0006fd0 tx=0x7fc1e0006e40 comp rx=0 tx=0).stop 2024-05-08T21:11:58.647 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:58.228+0000 7fc1e97fa700 1 -- 172.21.15.33:0/1889536716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 msgr2=0x7fc1ec19a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:11:58.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:58.228+0000 7fc1e97fa700 1 --2- 172.21.15.33:0/1889536716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 0x7fc1ec19a140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc00b7b0 tx=0x7fc1dc003c50 comp rx=0 tx=0).stop 2024-05-08T21:11:58.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:58.228+0000 7fc1e97fa700 1 -- 172.21.15.33:0/1889536716 shutdown_connections 2024-05-08T21:11:58.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:58.228+0000 7fc1e97fa700 1 --2- 172.21.15.33:0/1889536716 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1d40385c0 0x7fc1d403aa80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:58.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:58.228+0000 7fc1e97fa700 1 --2- 172.21.15.33:0/1889536716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1ec106fa0 0x7fc1ec19a140 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:11:58.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:58.228+0000 7fc1e97fa700 1 -- 172.21.15.33:0/1889536716 >> 172.21.15.33:0/1889536716 conn(0x7fc1ec100a10 msgr2=0x7fc1ec101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:11:58.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:58.228+0000 7fc1e97fa700 1 -- 172.21.15.33:0/1889536716 shutdown_connections 2024-05-08T21:11:58.648 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:58.228+0000 7fc1e97fa700 1 -- 172.21.15.33:0/1889536716 wait complete. 2024-05-08T21:11:58.648 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial admin user... 2024-05-08T21:11:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:59 smithi033 ceph-mon[42528]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:11:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:11:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:11:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:11:59 smithi033 ceph-mon[42528]: mgrmap e12: x(active, since 2s) 2024-05-08T21:12:00.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$gaZsg8/nCjJze338V93p7e1nldHHrIHqntq.baVnPJfBAIG/g5HE.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715202719, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-08T21:12:00.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.237+0000 7f7ebaf74700 1 Processor -- start 2024-05-08T21:12:00.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.237+0000 7f7ebaf74700 1 -- start start 2024-05-08T21:12:00.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.237+0000 7f7ebaf74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 0x7f7eb4109370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:00.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.237+0000 7f7ebaf74700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4079e40 con 0x7f7eb4106f80 2024-05-08T21:12:00.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.237+0000 7f7eb8d10700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 0x7f7eb4109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:00.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.237+0000 7f7eb8d10700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 0x7f7eb4109370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34700/0 (socket says 172.21.15.33:34700) 2024-05-08T21:12:00.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.237+0000 7f7eb8d10700 1 -- 172.21.15.33:0/1954629349 learned_addr learned my addr 172.21.15.33:0/1954629349 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:00.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.238+0000 7f7eb8d10700 1 -- 172.21.15.33:0/1954629349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb41098b0 con 0x7f7eb4106f80 2024-05-08T21:12:00.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.238+0000 7f7eb8d10700 1 --2- 172.21.15.33:0/1954629349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 0x7f7eb4109370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40098a0 tx=0x7f7ea4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb1539f070f4a5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:00.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.238+0000 7f7eb37fe700 1 -- 172.21.15.33:0/1954629349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ea4004160 con 0x7f7eb4106f80 2024-05-08T21:12:00.100 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.238+0000 7f7eb37fe700 1 -- 172.21.15.33:0/1954629349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ea40042c0 con 0x7f7eb4106f80 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.238+0000 7f7eb37fe700 1 -- 172.21.15.33:0/1954629349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ea40118e0 con 0x7f7eb4106f80 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.239+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1954629349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 msgr2=0x7f7eb4109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.239+0000 7f7ebaf74700 1 --2- 172.21.15.33:0/1954629349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 0x7f7eb4109370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40098a0 tx=0x7f7ea4009bb0 comp rx=0 tx=0).stop 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.239+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1954629349 shutdown_connections 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.239+0000 7f7ebaf74700 1 --2- 172.21.15.33:0/1954629349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 0x7f7eb4109370 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.239+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1954629349 >> 172.21.15.33:0/1954629349 conn(0x7f7eb4100950 msgr2=0x7f7eb4102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.239+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1954629349 shutdown_connections 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.239+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1954629349 wait complete. 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.239+0000 7f7ebaf74700 1 Processor -- start 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.239+0000 7f7ebaf74700 1 -- start start 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.240+0000 7f7ebaf74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 0x7f7eb419a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.240+0000 7f7ebaf74700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb419a6a0 con 0x7f7eb4106f80 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.240+0000 7f7eb8d10700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 0x7f7eb419a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:00.101 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.240+0000 7f7eb8d10700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 0x7f7eb419a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34704/0 (socket says 172.21.15.33:34704) 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.240+0000 7f7eb8d10700 1 -- 172.21.15.33:0/1527622167 learned_addr learned my addr 172.21.15.33:0/1527622167 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.240+0000 7f7eb8d10700 1 -- 172.21.15.33:0/1527622167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea4009550 con 0x7f7eb4106f80 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.240+0000 7f7eb8d10700 1 --2- 172.21.15.33:0/1527622167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 0x7f7eb419a160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4003e30 tx=0x7f7ea4011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.240+0000 7f7eb1ffb700 1 -- 172.21.15.33:0/1527622167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ea40036a0 con 0x7f7eb4106f80 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.240+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1527622167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb419a8a0 con 0x7f7eb4106f80 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.240+0000 7f7eb1ffb700 1 -- 172.21.15.33:0/1527622167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ea401b440 con 0x7f7eb4106f80 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.240+0000 7f7eb1ffb700 1 -- 172.21.15.33:0/1527622167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ea40193d0 con 0x7f7eb4106f80 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.241+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1527622167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb419ad40 con 0x7f7eb4106f80 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.242+0000 7f7eb1ffb700 1 -- 172.21.15.33:0/1527622167 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f7ea4019690 con 0x7f7eb4106f80 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.242+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1527622167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb410a630 con 0x7f7eb4106f80 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.242+0000 7f7eb1ffb700 1 --2- 172.21.15.33:0/1527622167 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7e9c038700 0x7f7e9c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.242+0000 7f7eb1ffb700 1 -- 172.21.15.33:0/1527622167 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7ea404c880 con 0x7f7eb4106f80 2024-05-08T21:12:00.102 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.242+0000 7f7eb3fff700 1 --2- 172.21.15.33:0/1527622167 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7e9c038700 0x7f7e9c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.242+0000 7f7eb3fff700 1 --2- 172.21.15.33:0/1527622167 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7e9c038700 0x7f7e9c03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8006fd0 tx=0x7f7ea8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.248+0000 7f7eb1ffb700 1 -- 172.21.15.33:0/1527622167 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7ea4018d60 con 0x7f7eb4106f80 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.474+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1527622167 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7eb40623f0 con 0x7f7e9c038700 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.727+0000 7f7eb1ffb700 1 -- 172.21.15.33:0/1527622167 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f7eb40623f0 con 0x7f7e9c038700 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.730+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1527622167 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7e9c038700 msgr2=0x7f7e9c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.730+0000 7f7ebaf74700 1 --2- 172.21.15.33:0/1527622167 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7e9c038700 0x7f7e9c03abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8006fd0 tx=0x7f7ea8006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.730+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1527622167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 msgr2=0x7f7eb419a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.730+0000 7f7ebaf74700 1 --2- 172.21.15.33:0/1527622167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 0x7f7eb419a160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4003e30 tx=0x7f7ea4011e80 comp rx=0 tx=0).stop 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.730+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1527622167 shutdown_connections 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.730+0000 7f7ebaf74700 1 --2- 172.21.15.33:0/1527622167 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7e9c038700 0x7f7e9c03abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.730+0000 7f7ebaf74700 1 --2- 172.21.15.33:0/1527622167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eb4106f80 0x7f7eb419a160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.730+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1527622167 >> 172.21.15.33:0/1527622167 conn(0x7f7eb4100950 msgr2=0x7f7eb41014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.730+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1527622167 shutdown_connections 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:11:59.730+0000 7f7ebaf74700 1 -- 172.21.15.33:0/1527622167 wait complete. 2024-05-08T21:12:00.103 INFO:teuthology.orchestra.run.smithi033.stdout:Fetching dashboard port number... 2024-05-08T21:12:01.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:00 smithi033 ceph-mon[42528]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:01.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:00 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:01.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:00 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1220791564' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-08T21:12:01.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 8443 2024-05-08T21:12:01.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.455+0000 7f18e3c14700 1 Processor -- start 2024-05-08T21:12:01.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.455+0000 7f18e3c14700 1 -- start start 2024-05-08T21:12:01.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.456+0000 7f18e3c14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 0x7f18dc109370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:01.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.456+0000 7f18e3c14700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc079e40 con 0x7f18dc106f80 2024-05-08T21:12:01.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.456+0000 7f18e19b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 0x7f18dc109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:01.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.456+0000 7f18e19b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 0x7f18dc109370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59062/0 (socket says 172.21.15.33:59062) 2024-05-08T21:12:01.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.456+0000 7f18e19b0700 1 -- 172.21.15.33:0/870173792 learned_addr learned my addr 172.21.15.33:0/870173792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.456+0000 7f18e19b0700 1 -- 172.21.15.33:0/870173792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc1098b0 con 0x7f18dc106f80 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.456+0000 7f18e19b0700 1 --2- 172.21.15.33:0/870173792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 0x7f18dc109370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18d80098a0 tx=0x7f18d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605f972fc600ad54 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.456+0000 7f18e09ae700 1 -- 172.21.15.33:0/870173792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18d8004160 con 0x7f18dc106f80 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.457+0000 7f18e09ae700 1 -- 172.21.15.33:0/870173792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18d80042c0 con 0x7f18dc106f80 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.457+0000 7f18e09ae700 1 -- 172.21.15.33:0/870173792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18d80118e0 con 0x7f18dc106f80 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.457+0000 7f18e3c14700 1 -- 172.21.15.33:0/870173792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 msgr2=0x7f18dc109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.457+0000 7f18e3c14700 1 --2- 172.21.15.33:0/870173792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 0x7f18dc109370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f18d80098a0 tx=0x7f18d8009bb0 comp rx=0 tx=0).stop 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.457+0000 7f18e3c14700 1 -- 172.21.15.33:0/870173792 shutdown_connections 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.457+0000 7f18e3c14700 1 --2- 172.21.15.33:0/870173792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 0x7f18dc109370 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.457+0000 7f18e3c14700 1 -- 172.21.15.33:0/870173792 >> 172.21.15.33:0/870173792 conn(0x7f18dc100950 msgr2=0x7f18dc102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.457+0000 7f18e3c14700 1 -- 172.21.15.33:0/870173792 shutdown_connections 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.457+0000 7f18e3c14700 1 -- 172.21.15.33:0/870173792 wait complete. 2024-05-08T21:12:01.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.458+0000 7f18e3c14700 1 Processor -- start 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.458+0000 7f18e3c14700 1 -- start start 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.458+0000 7f18e3c14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 0x7f18dc19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.458+0000 7f18e3c14700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc19c930 con 0x7f18dc106f80 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.458+0000 7f18e19b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 0x7f18dc19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.458+0000 7f18e19b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 0x7f18dc19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59066/0 (socket says 172.21.15.33:59066) 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.458+0000 7f18e19b0700 1 -- 172.21.15.33:0/1220791564 learned_addr learned my addr 172.21.15.33:0/1220791564 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.459+0000 7f18e19b0700 1 -- 172.21.15.33:0/1220791564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d8009550 con 0x7f18dc106f80 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.459+0000 7f18e19b0700 1 --2- 172.21.15.33:0/1220791564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 0x7f18dc19c3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18d80046e0 tx=0x7f18d80119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.459+0000 7f18d2ffd700 1 -- 172.21.15.33:0/1220791564 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18d80037d0 con 0x7f18dc106f80 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.459+0000 7f18d2ffd700 1 -- 172.21.15.33:0/1220791564 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18d801b440 con 0x7f18dc106f80 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.459+0000 7f18e3c14700 1 -- 172.21.15.33:0/1220791564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc19cb30 con 0x7f18dc106f80 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.459+0000 7f18d2ffd700 1 -- 172.21.15.33:0/1220791564 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18d8023400 con 0x7f18dc106f80 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.459+0000 7f18e3c14700 1 -- 172.21.15.33:0/1220791564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc19cfd0 con 0x7f18dc106f80 2024-05-08T21:12:01.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.460+0000 7f18d2ffd700 1 -- 172.21.15.33:0/1220791564 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7f18d8023630 con 0x7f18dc106f80 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.460+0000 7f18e3c14700 1 -- 172.21.15.33:0/1220791564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc10a630 con 0x7f18dc106f80 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.460+0000 7f18d2ffd700 1 --2- 172.21.15.33:0/1220791564 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f18c80387a0 0x7f18c803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.460+0000 7f18d2ffd700 1 -- 172.21.15.33:0/1220791564 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f18d804c870 con 0x7f18dc106f80 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.460+0000 7f18e11af700 1 --2- 172.21.15.33:0/1220791564 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f18c80387a0 0x7f18c803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.461+0000 7f18e11af700 1 --2- 172.21.15.33:0/1220791564 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f18c80387a0 0x7f18c803ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18cc006fd0 tx=0x7f18cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.466+0000 7f18d2ffd700 1 -- 172.21.15.33:0/1220791564 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f18d8018350 con 0x7f18dc106f80 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.677+0000 7f18e3c14700 1 -- 172.21.15.33:0/1220791564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f18dc10a8a0 con 0x7f18dc106f80 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.678+0000 7f18d2ffd700 1 -- 172.21.15.33:0/1220791564 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f18d802bb40 con 0x7f18dc106f80 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.680+0000 7f18e3c14700 1 -- 172.21.15.33:0/1220791564 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f18c80387a0 msgr2=0x7f18c803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.680+0000 7f18e3c14700 1 --2- 172.21.15.33:0/1220791564 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f18c80387a0 0x7f18c803ac60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18cc006fd0 tx=0x7f18cc006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.680+0000 7f18e3c14700 1 -- 172.21.15.33:0/1220791564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 msgr2=0x7f18dc19c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.680+0000 7f18e3c14700 1 --2- 172.21.15.33:0/1220791564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 0x7f18dc19c3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18d80046e0 tx=0x7f18d80119a0 comp rx=0 tx=0).stop 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.680+0000 7f18e3c14700 1 -- 172.21.15.33:0/1220791564 shutdown_connections 2024-05-08T21:12:01.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.680+0000 7f18e3c14700 1 --2- 172.21.15.33:0/1220791564 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f18c80387a0 0x7f18c803ac60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:01.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.680+0000 7f18e3c14700 1 --2- 172.21.15.33:0/1220791564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18dc106f80 0x7f18dc19c3f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:01.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.680+0000 7f18e3c14700 1 -- 172.21.15.33:0/1220791564 >> 172.21.15.33:0/1220791564 conn(0x7f18dc100950 msgr2=0x7f18dc1014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:01.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.680+0000 7f18e3c14700 1 -- 172.21.15.33:0/1220791564 shutdown_connections 2024-05-08T21:12:01.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:00.680+0000 7f18e3c14700 1 -- 172.21.15.33:0/1220791564 wait complete. 2024-05-08T21:12:01.070 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T21:12:01.070 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-05-08T21:12:01.079 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T21:12:01.079 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-05-08T21:12:01.079 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-05-08T21:12:01.079 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-08T21:12:01.081 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph Dashboard is now available at: 2024-05-08T21:12:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:12:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: URL: https://smithi033:8443/ 2024-05-08T21:12:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: User: admin 2024-05-08T21:12:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: Password: tdkzrtql9f 2024-05-08T21:12:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:12:01.082 INFO:teuthology.orchestra.run.smithi033.stdout:Saving cluster configuration to /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/config directory 2024-05-08T21:12:01.083 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling autotune for osd_memory_target 2024-05-08T21:12:02.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.452+0000 7fc20bba3700 1 Processor -- start 2024-05-08T21:12:02.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.453+0000 7fc20bba3700 1 -- start start 2024-05-08T21:12:02.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.453+0000 7fc20bba3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 0x7fc2041068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:02.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.453+0000 7fc20bba3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc204106eb0 con 0x7fc2041064c0 2024-05-08T21:12:02.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.453+0000 7fc20993f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 0x7fc2041068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:02.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.453+0000 7fc20993f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 0x7fc2041068e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59074/0 (socket says 172.21.15.33:59074) 2024-05-08T21:12:02.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.453+0000 7fc20993f700 1 -- 172.21.15.33:0/4099740651 learned_addr learned my addr 172.21.15.33:0/4099740651 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:02.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.453+0000 7fc20993f700 1 -- 172.21.15.33:0/4099740651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2041076c0 con 0x7fc2041064c0 2024-05-08T21:12:02.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.454+0000 7fc20993f700 1 --2- 172.21.15.33:0/4099740651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 0x7fc2041068e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80098a0 tx=0x7fc1f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759ff56f5ad6c1a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:02.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.454+0000 7fc20893d700 1 -- 172.21.15.33:0/4099740651 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1f8004160 con 0x7fc2041064c0 2024-05-08T21:12:02.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.454+0000 7fc20893d700 1 -- 172.21.15.33:0/4099740651 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1f80042c0 con 0x7fc2041064c0 2024-05-08T21:12:02.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.454+0000 7fc20893d700 1 -- 172.21.15.33:0/4099740651 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1f80118e0 con 0x7fc2041064c0 2024-05-08T21:12:02.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.454+0000 7fc20bba3700 1 -- 172.21.15.33:0/4099740651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 msgr2=0x7fc2041068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.454+0000 7fc20bba3700 1 --2- 172.21.15.33:0/4099740651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 0x7fc2041068e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80098a0 tx=0x7fc1f8009bb0 comp rx=0 tx=0).stop 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.455+0000 7fc20bba3700 1 -- 172.21.15.33:0/4099740651 shutdown_connections 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.455+0000 7fc20bba3700 1 --2- 172.21.15.33:0/4099740651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 0x7fc2041068e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.455+0000 7fc20bba3700 1 -- 172.21.15.33:0/4099740651 >> 172.21.15.33:0/4099740651 conn(0x7fc204101a60 msgr2=0x7fc204103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.455+0000 7fc20bba3700 1 -- 172.21.15.33:0/4099740651 shutdown_connections 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.455+0000 7fc20bba3700 1 -- 172.21.15.33:0/4099740651 wait complete. 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.455+0000 7fc20bba3700 1 Processor -- start 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.455+0000 7fc20bba3700 1 -- start start 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.456+0000 7fc20bba3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 0x7fc20419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.456+0000 7fc20bba3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20419c940 con 0x7fc2041064c0 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.456+0000 7fc20993f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 0x7fc20419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.456+0000 7fc20993f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 0x7fc20419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59090/0 (socket says 172.21.15.33:59090) 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.456+0000 7fc20993f700 1 -- 172.21.15.33:0/3060690596 learned_addr learned my addr 172.21.15.33:0/3060690596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.456+0000 7fc20993f700 1 -- 172.21.15.33:0/3060690596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f8009550 con 0x7fc2041064c0 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.456+0000 7fc20993f700 1 --2- 172.21.15.33:0/3060690596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 0x7fc20419c400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8003f70 tx=0x7fc1f80119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:02.043 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.456+0000 7fc1f6ffd700 1 -- 172.21.15.33:0/3060690596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1f80037d0 con 0x7fc2041064c0 2024-05-08T21:12:02.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.456+0000 7fc1f6ffd700 1 -- 172.21.15.33:0/3060690596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1f801b440 con 0x7fc2041064c0 2024-05-08T21:12:02.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.456+0000 7fc1f6ffd700 1 -- 172.21.15.33:0/3060690596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1f80193d0 con 0x7fc2041064c0 2024-05-08T21:12:02.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.456+0000 7fc20bba3700 1 -- 172.21.15.33:0/3060690596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20419cb40 con 0x7fc2041064c0 2024-05-08T21:12:02.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.456+0000 7fc20bba3700 1 -- 172.21.15.33:0/3060690596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20419cfe0 con 0x7fc2041064c0 2024-05-08T21:12:02.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.457+0000 7fc1f6ffd700 1 -- 172.21.15.33:0/3060690596 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) v1 ==== 45554+0+0 (secure 0 0 0) 0x7fc1f801b860 con 0x7fc2041064c0 2024-05-08T21:12:02.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.458+0000 7fc1f6ffd700 1 --2- 172.21.15.33:0/3060690596 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1f00387a0 0x7fc1f003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:02.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.458+0000 7fc20bba3700 1 -- 172.21.15.33:0/3060690596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20410a8f0 con 0x7fc2041064c0 2024-05-08T21:12:02.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.458+0000 7fc1f6ffd700 1 -- 172.21.15.33:0/3060690596 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc1f804c5f0 con 0x7fc2041064c0 2024-05-08T21:12:02.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.458+0000 7fc20913e700 1 --2- 172.21.15.33:0/3060690596 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1f00387a0 0x7fc1f003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:02.044 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.458+0000 7fc20913e700 1 --2- 172.21.15.33:0/3060690596 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1f00387a0 0x7fc1f003ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc200006fd0 tx=0x7fc200006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.464+0000 7fc1f6ffd700 1 -- 172.21.15.33:0/3060690596 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc1f801f080 con 0x7fc2041064c0 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.674+0000 7fc20bba3700 1 -- 172.21.15.33:0/3060690596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fc20419f870 con 0x7fc2041064c0 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.674+0000 7fc1f6ffd700 1 -- 172.21.15.33:0/3060690596 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fc1f8003930 con 0x7fc2041064c0 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.676+0000 7fc20bba3700 1 -- 172.21.15.33:0/3060690596 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1f00387a0 msgr2=0x7fc1f003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.676+0000 7fc20bba3700 1 --2- 172.21.15.33:0/3060690596 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1f00387a0 0x7fc1f003ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc200006fd0 tx=0x7fc200006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.676+0000 7fc20bba3700 1 -- 172.21.15.33:0/3060690596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 msgr2=0x7fc20419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.676+0000 7fc20bba3700 1 --2- 172.21.15.33:0/3060690596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 0x7fc20419c400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8003f70 tx=0x7fc1f80119a0 comp rx=0 tx=0).stop 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.676+0000 7fc20bba3700 1 -- 172.21.15.33:0/3060690596 shutdown_connections 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.676+0000 7fc20bba3700 1 --2- 172.21.15.33:0/3060690596 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1f00387a0 0x7fc1f003ac60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.676+0000 7fc20bba3700 1 --2- 172.21.15.33:0/3060690596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2041064c0 0x7fc20419c400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.677+0000 7fc20bba3700 1 -- 172.21.15.33:0/3060690596 >> 172.21.15.33:0/3060690596 conn(0x7fc204101a60 msgr2=0x7fc204102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.677+0000 7fc20bba3700 1 -- 172.21.15.33:0/3060690596 shutdown_connections 2024-05-08T21:12:02.045 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:01.677+0000 7fc20bba3700 1 -- 172.21.15.33:0/3060690596 wait complete. 2024-05-08T21:12:03.166 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.417+0000 7f93ab31c700 1 Processor -- start 2024-05-08T21:12:03.166 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.417+0000 7f93ab31c700 1 -- start start 2024-05-08T21:12:03.166 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.418+0000 7f93ab31c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 0x7f93a41093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:03.166 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.418+0000 7f93ab31c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a4079e60 con 0x7f93a4106fd0 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.418+0000 7f93a90b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 0x7f93a41093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.418+0000 7f93a90b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 0x7f93a41093c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59102/0 (socket says 172.21.15.33:59102) 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.418+0000 7f93a90b8700 1 -- 172.21.15.33:0/1185577911 learned_addr learned my addr 172.21.15.33:0/1185577911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.418+0000 7f93a90b8700 1 -- 172.21.15.33:0/1185577911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a4109900 con 0x7f93a4106fd0 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.418+0000 7f93a90b8700 1 --2- 172.21.15.33:0/1185577911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 0x7f93a41093c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9394009e80 tx=0x7f939400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d144ddc6be46c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.418+0000 7f93a3fff700 1 -- 172.21.15.33:0/1185577911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f939400e070 con 0x7f93a4106fd0 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.419+0000 7f93a3fff700 1 -- 172.21.15.33:0/1185577911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f939400b8c0 con 0x7f93a4106fd0 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.419+0000 7f93a3fff700 1 -- 172.21.15.33:0/1185577911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9394004360 con 0x7f93a4106fd0 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.419+0000 7f93ab31c700 1 -- 172.21.15.33:0/1185577911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 msgr2=0x7f93a41093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.419+0000 7f93ab31c700 1 --2- 172.21.15.33:0/1185577911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 0x7f93a41093c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9394009e80 tx=0x7f939400b330 comp rx=0 tx=0).stop 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.419+0000 7f93ab31c700 1 -- 172.21.15.33:0/1185577911 shutdown_connections 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.419+0000 7f93ab31c700 1 --2- 172.21.15.33:0/1185577911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 0x7f93a41093c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.419+0000 7f93ab31c700 1 -- 172.21.15.33:0/1185577911 >> 172.21.15.33:0/1185577911 conn(0x7f93a4100a40 msgr2=0x7f93a4102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:03.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.419+0000 7f93ab31c700 1 -- 172.21.15.33:0/1185577911 shutdown_connections 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.419+0000 7f93ab31c700 1 -- 172.21.15.33:0/1185577911 wait complete. 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.420+0000 7f93ab31c700 1 Processor -- start 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.420+0000 7f93ab31c700 1 -- start start 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.420+0000 7f93ab31c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 0x7f93a419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.420+0000 7f93ab31c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a419a6f0 con 0x7f93a4106fd0 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.420+0000 7f93a90b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 0x7f93a419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.420+0000 7f93a90b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 0x7f93a419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59114/0 (socket says 172.21.15.33:59114) 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.420+0000 7f93a90b8700 1 -- 172.21.15.33:0/1144822451 learned_addr learned my addr 172.21.15.33:0/1144822451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.421+0000 7f93a90b8700 1 -- 172.21.15.33:0/1144822451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9394009b30 con 0x7f93a4106fd0 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.421+0000 7f93a90b8700 1 --2- 172.21.15.33:0/1144822451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 0x7f93a419a1b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f939400b7b0 tx=0x7f9394003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.421+0000 7f93a27fc700 1 -- 172.21.15.33:0/1144822451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f939400e040 con 0x7f93a4106fd0 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.421+0000 7f93a27fc700 1 -- 172.21.15.33:0/1144822451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f939400bd70 con 0x7f93a4106fd0 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.421+0000 7f93ab31c700 1 -- 172.21.15.33:0/1144822451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a419a8f0 con 0x7f93a4106fd0 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.421+0000 7f93a27fc700 1 -- 172.21.15.33:0/1144822451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f939401c590 con 0x7f93a4106fd0 2024-05-08T21:12:03.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.421+0000 7f93ab31c700 1 -- 172.21.15.33:0/1144822451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a419adf0 con 0x7f93a4106fd0 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.422+0000 7f93a27fc700 1 -- 172.21.15.33:0/1144822451 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f9394012430 con 0x7f93a4106fd0 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.422+0000 7f93a27fc700 1 --2- 172.21.15.33:0/1144822451 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f938c0387f0 0x7f938c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.422+0000 7f93ab31c700 1 -- 172.21.15.33:0/1144822451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a410a680 con 0x7f93a4106fd0 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.422+0000 7f93a27fc700 1 -- 172.21.15.33:0/1144822451 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f939404c790 con 0x7f93a4106fd0 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.422+0000 7f93a88b7700 1 --2- 172.21.15.33:0/1144822451 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f938c0387f0 0x7f938c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.423+0000 7f93a88b7700 1 --2- 172.21.15.33:0/1144822451 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f938c0387f0 0x7f938c03acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9398006fd0 tx=0x7f9398006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.428+0000 7f93a27fc700 1 -- 172.21.15.33:0/1144822451 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f939401b3a0 con 0x7f93a4106fd0 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.756+0000 7f93ab31c700 1 -- 172.21.15.33:0/1144822451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f93a419d620 con 0x7f93a4106fd0 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.758+0000 7f93a27fc700 1 -- 172.21.15.33:0/1144822451 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f9394017070 con 0x7f93a4106fd0 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.761+0000 7f93ab31c700 1 -- 172.21.15.33:0/1144822451 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f938c0387f0 msgr2=0x7f938c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.761+0000 7f93ab31c700 1 --2- 172.21.15.33:0/1144822451 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f938c0387f0 0x7f938c03acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9398006fd0 tx=0x7f9398006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.761+0000 7f93ab31c700 1 -- 172.21.15.33:0/1144822451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 msgr2=0x7f93a419a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.761+0000 7f93ab31c700 1 --2- 172.21.15.33:0/1144822451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 0x7f93a419a1b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f939400b7b0 tx=0x7f9394003c50 comp rx=0 tx=0).stop 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.761+0000 7f93ab31c700 1 -- 172.21.15.33:0/1144822451 shutdown_connections 2024-05-08T21:12:03.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.761+0000 7f93ab31c700 1 --2- 172.21.15.33:0/1144822451 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f938c0387f0 0x7f938c03acb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.761+0000 7f93ab31c700 1 --2- 172.21.15.33:0/1144822451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93a4106fd0 0x7f93a419a1b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.761+0000 7f93ab31c700 1 -- 172.21.15.33:0/1144822451 >> 172.21.15.33:0/1144822451 conn(0x7f93a4100a40 msgr2=0x7f93a4101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.761+0000 7f93ab31c700 1 -- 172.21.15.33:0/1144822451 shutdown_connections 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-05-08T21:12:02.761+0000 7f93ab31c700 1 -- 172.21.15.33:0/1144822451 wait complete. 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout:Or, if you are only running a single cluster on this host: 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:12:03.170 INFO:teuthology.orchestra.run.smithi033.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-08T21:12:03.171 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:12:03.171 INFO:teuthology.orchestra.run.smithi033.stdout: ceph telemetry on 2024-05-08T21:12:03.171 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:12:03.171 INFO:teuthology.orchestra.run.smithi033.stdout:For more information see: 2024-05-08T21:12:03.171 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:12:03.171 INFO:teuthology.orchestra.run.smithi033.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-08T21:12:03.171 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:12:03.171 INFO:teuthology.orchestra.run.smithi033.stdout:Bootstrap complete. 2024-05-08T21:12:03.213 INFO:tasks.cephadm:Fetching config... 2024-05-08T21:12:03.213 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:12:03.213 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-08T21:12:03.234 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-08T21:12:03.234 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:12:03.234 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-08T21:12:03.293 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-08T21:12:03.293 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:12:03.294 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-08T21:12:03.371 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-08T21:12:03.371 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:12:03.371 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-08T21:12:03.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:03 smithi033 ceph-mon[42528]: mgrmap e13: x(active, since 6s) 2024-05-08T21:12:03.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:03 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1144822451' entity='client.admin' 2024-05-08T21:12:03.412 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-08T21:12:03.412 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzHb55XRxErTdFsuIzj7b1wTMhgpoUt4+Ux3jmSJX8oD+u28CTZV20SVHZS+8GEPvmaGb2KsF4c59pGLWBJmM7DlQ3tIDZLkOw1OTKBm7Qpx/u9xjZyA68XQ+5Ze1bqxIIgwAZ+mc6IqlmRuZYQc7uX1D/e21e04CBMyFyMUwQ90xJh8vMZqjhtdfSpbFdWGbuMGk86K5PGYCPCLSXCjDyjWMFBvpjDsz+kBU8vP3DAALwexAkjNVVf/CjuZxW0o5GWrPYwed772lsAqqEf3bmt87pPb+D5tEI4oVJGj/yetf5/Qmk24c3TTzL3AaTI+1/gGV8d9z7Agtf/aZeXDae3vPQqAAs4f/kzCmuf51Ix6G7OEUV0kDbPtF+neXucbvtmN+YlUFGtDyLag+NdKdUuAqgX7vRN6erPFDvsAtQevKV4lVB66JaPIr0X6HAqLRFknqTg2MvONVyVPZnWnZkgIUvznhmIa4Cz5rYebKVscz4FAz5cJiWqj/YYyjN/Mc= ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T21:12:03.495 INFO:teuthology.orchestra.run.smithi033.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzHb55XRxErTdFsuIzj7b1wTMhgpoUt4+Ux3jmSJX8oD+u28CTZV20SVHZS+8GEPvmaGb2KsF4c59pGLWBJmM7DlQ3tIDZLkOw1OTKBm7Qpx/u9xjZyA68XQ+5Ze1bqxIIgwAZ+mc6IqlmRuZYQc7uX1D/e21e04CBMyFyMUwQ90xJh8vMZqjhtdfSpbFdWGbuMGk86K5PGYCPCLSXCjDyjWMFBvpjDsz+kBU8vP3DAALwexAkjNVVf/CjuZxW0o5GWrPYwed772lsAqqEf3bmt87pPb+D5tEI4oVJGj/yetf5/Qmk24c3TTzL3AaTI+1/gGV8d9z7Agtf/aZeXDae3vPQqAAs4f/kzCmuf51Ix6G7OEUV0kDbPtF+neXucbvtmN+YlUFGtDyLag+NdKdUuAqgX7vRN6erPFDvsAtQevKV4lVB66JaPIr0X6HAqLRFknqTg2MvONVyVPZnWnZkgIUvznhmIa4Cz5rYebKVscz4FAz5cJiWqj/YYyjN/Mc= ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:12:03.509 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzHb55XRxErTdFsuIzj7b1wTMhgpoUt4+Ux3jmSJX8oD+u28CTZV20SVHZS+8GEPvmaGb2KsF4c59pGLWBJmM7DlQ3tIDZLkOw1OTKBm7Qpx/u9xjZyA68XQ+5Ze1bqxIIgwAZ+mc6IqlmRuZYQc7uX1D/e21e04CBMyFyMUwQ90xJh8vMZqjhtdfSpbFdWGbuMGk86K5PGYCPCLSXCjDyjWMFBvpjDsz+kBU8vP3DAALwexAkjNVVf/CjuZxW0o5GWrPYwed772lsAqqEf3bmt87pPb+D5tEI4oVJGj/yetf5/Qmk24c3TTzL3AaTI+1/gGV8d9z7Agtf/aZeXDae3vPQqAAs4f/kzCmuf51Ix6G7OEUV0kDbPtF+neXucbvtmN+YlUFGtDyLag+NdKdUuAqgX7vRN6erPFDvsAtQevKV4lVB66JaPIr0X6HAqLRFknqTg2MvONVyVPZnWnZkgIUvznhmIa4Cz5rYebKVscz4FAz5cJiWqj/YYyjN/Mc= ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T21:12:03.554 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzHb55XRxErTdFsuIzj7b1wTMhgpoUt4+Ux3jmSJX8oD+u28CTZV20SVHZS+8GEPvmaGb2KsF4c59pGLWBJmM7DlQ3tIDZLkOw1OTKBm7Qpx/u9xjZyA68XQ+5Ze1bqxIIgwAZ+mc6IqlmRuZYQc7uX1D/e21e04CBMyFyMUwQ90xJh8vMZqjhtdfSpbFdWGbuMGk86K5PGYCPCLSXCjDyjWMFBvpjDsz+kBU8vP3DAALwexAkjNVVf/CjuZxW0o5GWrPYwed772lsAqqEf3bmt87pPb+D5tEI4oVJGj/yetf5/Qmk24c3TTzL3AaTI+1/gGV8d9z7Agtf/aZeXDae3vPQqAAs4f/kzCmuf51Ix6G7OEUV0kDbPtF+neXucbvtmN+YlUFGtDyLag+NdKdUuAqgX7vRN6erPFDvsAtQevKV4lVB66JaPIr0X6HAqLRFknqTg2MvONVyVPZnWnZkgIUvznhmIa4Cz5rYebKVscz4FAz5cJiWqj/YYyjN/Mc= ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:12:03.567 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzHb55XRxErTdFsuIzj7b1wTMhgpoUt4+Ux3jmSJX8oD+u28CTZV20SVHZS+8GEPvmaGb2KsF4c59pGLWBJmM7DlQ3tIDZLkOw1OTKBm7Qpx/u9xjZyA68XQ+5Ze1bqxIIgwAZ+mc6IqlmRuZYQc7uX1D/e21e04CBMyFyMUwQ90xJh8vMZqjhtdfSpbFdWGbuMGk86K5PGYCPCLSXCjDyjWMFBvpjDsz+kBU8vP3DAALwexAkjNVVf/CjuZxW0o5GWrPYwed772lsAqqEf3bmt87pPb+D5tEI4oVJGj/yetf5/Qmk24c3TTzL3AaTI+1/gGV8d9z7Agtf/aZeXDae3vPQqAAs4f/kzCmuf51Ix6G7OEUV0kDbPtF+neXucbvtmN+YlUFGtDyLag+NdKdUuAqgX7vRN6erPFDvsAtQevKV4lVB66JaPIr0X6HAqLRFknqTg2MvONVyVPZnWnZkgIUvznhmIa4Cz5rYebKVscz4FAz5cJiWqj/YYyjN/Mc= ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T21:12:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCzHb55XRxErTdFsuIzj7b1wTMhgpoUt4+Ux3jmSJX8oD+u28CTZV20SVHZS+8GEPvmaGb2KsF4c59pGLWBJmM7DlQ3tIDZLkOw1OTKBm7Qpx/u9xjZyA68XQ+5Ze1bqxIIgwAZ+mc6IqlmRuZYQc7uX1D/e21e04CBMyFyMUwQ90xJh8vMZqjhtdfSpbFdWGbuMGk86K5PGYCPCLSXCjDyjWMFBvpjDsz+kBU8vP3DAALwexAkjNVVf/CjuZxW0o5GWrPYwed772lsAqqEf3bmt87pPb+D5tEI4oVJGj/yetf5/Qmk24c3TTzL3AaTI+1/gGV8d9z7Agtf/aZeXDae3vPQqAAs4f/kzCmuf51Ix6G7OEUV0kDbPtF+neXucbvtmN+YlUFGtDyLag+NdKdUuAqgX7vRN6erPFDvsAtQevKV4lVB66JaPIr0X6HAqLRFknqTg2MvONVyVPZnWnZkgIUvznhmIa4Cz5rYebKVscz4FAz5cJiWqj/YYyjN/Mc= ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:12:03.627 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-08T21:12:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:12:05.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.191+0000 7f65cfcb5700 1 -- 172.21.15.33:0/3308019344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c80ff830 msgr2=0x7f65c80ffc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:05.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.191+0000 7f65cfcb5700 1 --2- 172.21.15.33:0/3308019344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c80ff830 0x7f65c80ffc50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009910 tx=0x7f65c4009c20 comp rx=0 tx=0).stop 2024-05-08T21:12:05.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.191+0000 7f65cfcb5700 1 -- 172.21.15.33:0/3308019344 shutdown_connections 2024-05-08T21:12:05.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.191+0000 7f65cfcb5700 1 --2- 172.21.15.33:0/3308019344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c80ff830 0x7f65c80ffc50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:05.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.191+0000 7f65cfcb5700 1 -- 172.21.15.33:0/3308019344 >> 172.21.15.33:0/3308019344 conn(0x7f65c80fadd0 msgr2=0x7f65c80fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:05.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.191+0000 7f65cfcb5700 1 -- 172.21.15.33:0/3308019344 shutdown_connections 2024-05-08T21:12:05.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.191+0000 7f65cfcb5700 1 -- 172.21.15.33:0/3308019344 wait complete. 2024-05-08T21:12:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.192+0000 7f65cfcb5700 1 Processor -- start 2024-05-08T21:12:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.192+0000 7f65cfcb5700 1 -- start start 2024-05-08T21:12:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.192+0000 7f65cfcb5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c80ff830 0x7f65c8195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.192+0000 7f65cfcb5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8196250 con 0x7f65c80ff830 2024-05-08T21:12:05.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.192+0000 7f65cda51700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c80ff830 0x7f65c8195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.192+0000 7f65cda51700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c80ff830 0x7f65c8195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59142/0 (socket says 172.21.15.33:59142) 2024-05-08T21:12:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.192+0000 7f65cda51700 1 -- 172.21.15.33:0/4259977258 learned_addr learned my addr 172.21.15.33:0/4259977258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.192+0000 7f65cda51700 1 -- 172.21.15.33:0/4259977258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c40095f0 con 0x7f65c80ff830 2024-05-08T21:12:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.192+0000 7f65cda51700 1 --2- 172.21.15.33:0/4259977258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c80ff830 0x7f65c8195d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f65c4005790 tx=0x7f65c40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.193+0000 7f65beffd700 1 -- 172.21.15.33:0/4259977258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c400b960 con 0x7f65c80ff830 2024-05-08T21:12:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.193+0000 7f65beffd700 1 -- 172.21.15.33:0/4259977258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f65c400f460 con 0x7f65c80ff830 2024-05-08T21:12:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.193+0000 7f65beffd700 1 -- 172.21.15.33:0/4259977258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c40175d0 con 0x7f65c80ff830 2024-05-08T21:12:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.193+0000 7f65cfcb5700 1 -- 172.21.15.33:0/4259977258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c8196450 con 0x7f65c80ff830 2024-05-08T21:12:05.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.193+0000 7f65cfcb5700 1 -- 172.21.15.33:0/4259977258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c81968f0 con 0x7f65c80ff830 2024-05-08T21:12:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.194+0000 7f65beffd700 1 -- 172.21.15.33:0/4259977258 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f65c400f5d0 con 0x7f65c80ff830 2024-05-08T21:12:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.194+0000 7f65beffd700 1 --2- 172.21.15.33:0/4259977258 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65b40387a0 0x7f65b403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.194+0000 7f65cfcb5700 1 -- 172.21.15.33:0/4259977258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c80625d0 con 0x7f65c80ff830 2024-05-08T21:12:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.194+0000 7f65beffd700 1 -- 172.21.15.33:0/4259977258 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f65c404c890 con 0x7f65c80ff830 2024-05-08T21:12:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.194+0000 7f65cd250700 1 --2- 172.21.15.33:0/4259977258 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65b40387a0 0x7f65b403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:05.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.194+0000 7f65cd250700 1 --2- 172.21.15.33:0/4259977258 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65b40387a0 0x7f65b403ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006fd0 tx=0x7f65b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:05.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.200+0000 7f65beffd700 1 -- 172.21.15.33:0/4259977258 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f65c402b5a0 con 0x7f65c80ff830 2024-05-08T21:12:05.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.421+0000 7f65cfcb5700 1 -- 172.21.15.33:0/4259977258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f65c818f450 con 0x7f65c80ff830 2024-05-08T21:12:05.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.424+0000 7f65beffd700 1 -- 172.21.15.33:0/4259977258 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f65c4014ae0 con 0x7f65c80ff830 2024-05-08T21:12:05.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.427+0000 7f65cfcb5700 1 -- 172.21.15.33:0/4259977258 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65b40387a0 msgr2=0x7f65b403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:05.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.427+0000 7f65cfcb5700 1 --2- 172.21.15.33:0/4259977258 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65b40387a0 0x7f65b403ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006fd0 tx=0x7f65b8006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:05.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.427+0000 7f65cfcb5700 1 -- 172.21.15.33:0/4259977258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c80ff830 msgr2=0x7f65c8195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:05.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.427+0000 7f65cfcb5700 1 --2- 172.21.15.33:0/4259977258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c80ff830 0x7f65c8195d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f65c4005790 tx=0x7f65c40047b0 comp rx=0 tx=0).stop 2024-05-08T21:12:05.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.427+0000 7f65cfcb5700 1 -- 172.21.15.33:0/4259977258 shutdown_connections 2024-05-08T21:12:05.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.427+0000 7f65cfcb5700 1 --2- 172.21.15.33:0/4259977258 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65b40387a0 0x7f65b403ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:05.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.427+0000 7f65cfcb5700 1 --2- 172.21.15.33:0/4259977258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65c80ff830 0x7f65c8195d10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:05.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.427+0000 7f65cfcb5700 1 -- 172.21.15.33:0/4259977258 >> 172.21.15.33:0/4259977258 conn(0x7f65c80fadd0 msgr2=0x7f65c80689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:05.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.428+0000 7f65cfcb5700 1 -- 172.21.15.33:0/4259977258 shutdown_connections 2024-05-08T21:12:05.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:05.428+0000 7f65cfcb5700 1 -- 172.21.15.33:0/4259977258 wait complete. 2024-05-08T21:12:05.821 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-08T21:12:05.821 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-08T21:12:06.078 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:12:06.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:06.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:12:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:05 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4259977258' entity='client.admin' 2024-05-08T21:12:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:07.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.215+0000 7fae54dcf700 1 -- 172.21.15.33:0/217322659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501013c0 msgr2=0x7fae501037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:07.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.215+0000 7fae54dcf700 1 --2- 172.21.15.33:0/217322659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501013c0 0x7fae501037b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fae40009910 tx=0x7fae40009c20 comp rx=0 tx=0).stop 2024-05-08T21:12:07.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.215+0000 7fae54dcf700 1 -- 172.21.15.33:0/217322659 shutdown_connections 2024-05-08T21:12:07.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.215+0000 7fae54dcf700 1 --2- 172.21.15.33:0/217322659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501013c0 0x7fae501037b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:07.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.215+0000 7fae54dcf700 1 -- 172.21.15.33:0/217322659 >> 172.21.15.33:0/217322659 conn(0x7fae500fadd0 msgr2=0x7fae500fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.215+0000 7fae54dcf700 1 -- 172.21.15.33:0/217322659 shutdown_connections 2024-05-08T21:12:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.215+0000 7fae54dcf700 1 -- 172.21.15.33:0/217322659 wait complete. 2024-05-08T21:12:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.216+0000 7fae54dcf700 1 Processor -- start 2024-05-08T21:12:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.216+0000 7fae54dcf700 1 -- start start 2024-05-08T21:12:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.216+0000 7fae54dcf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501013c0 0x7fae50100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.216+0000 7fae54dcf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae50101130 con 0x7fae501013c0 2024-05-08T21:12:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.216+0000 7fae4e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501013c0 0x7fae50100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.216+0000 7fae4e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501013c0 0x7fae50100bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59160/0 (socket says 172.21.15.33:59160) 2024-05-08T21:12:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.216+0000 7fae4e59c700 1 -- 172.21.15.33:0/4286269401 learned_addr learned my addr 172.21.15.33:0/4286269401 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.217+0000 7fae4e59c700 1 -- 172.21.15.33:0/4286269401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae400095f0 con 0x7fae501013c0 2024-05-08T21:12:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.217+0000 7fae4e59c700 1 --2- 172.21.15.33:0/4286269401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501013c0 0x7fae50100bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae4000b5c0 tx=0x7fae40005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.217+0000 7fae3f7fe700 1 -- 172.21.15.33:0/4286269401 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae4000b960 con 0x7fae501013c0 2024-05-08T21:12:07.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.217+0000 7fae54dcf700 1 -- 172.21.15.33:0/4286269401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae500ff370 con 0x7fae501013c0 2024-05-08T21:12:07.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.217+0000 7fae3f7fe700 1 -- 172.21.15.33:0/4286269401 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fae4000f460 con 0x7fae501013c0 2024-05-08T21:12:07.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.217+0000 7fae3f7fe700 1 -- 172.21.15.33:0/4286269401 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae40017440 con 0x7fae501013c0 2024-05-08T21:12:07.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.217+0000 7fae54dcf700 1 -- 172.21.15.33:0/4286269401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae500ff810 con 0x7fae501013c0 2024-05-08T21:12:07.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.218+0000 7fae3f7fe700 1 -- 172.21.15.33:0/4286269401 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fae40017700 con 0x7fae501013c0 2024-05-08T21:12:07.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.218+0000 7fae3f7fe700 1 --2- 172.21.15.33:0/4286269401 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fae380387a0 0x7fae3803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:07.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.218+0000 7fae54dcf700 1 -- 172.21.15.33:0/4286269401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae50106e30 con 0x7fae501013c0 2024-05-08T21:12:07.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.218+0000 7fae3f7fe700 1 -- 172.21.15.33:0/4286269401 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fae4004c820 con 0x7fae501013c0 2024-05-08T21:12:07.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.218+0000 7fae4dd9b700 1 --2- 172.21.15.33:0/4286269401 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fae380387a0 0x7fae3803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:07.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.219+0000 7fae4dd9b700 1 --2- 172.21.15.33:0/4286269401 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fae380387a0 0x7fae3803ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fae44006fd0 tx=0x7fae44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:07.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.224+0000 7fae3f7fe700 1 -- 172.21.15.33:0/4286269401 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fae4002c440 con 0x7fae501013c0 2024-05-08T21:12:07.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.439+0000 7fae54dcf700 1 -- 172.21.15.33:0/4286269401 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fae500ffa50 con 0x7fae380387a0 2024-05-08T21:12:07.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.443+0000 7fae3f7fe700 1 -- 172.21.15.33:0/4286269401 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fae500ffa50 con 0x7fae380387a0 2024-05-08T21:12:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.445+0000 7fae54dcf700 1 -- 172.21.15.33:0/4286269401 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fae380387a0 msgr2=0x7fae3803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.445+0000 7fae54dcf700 1 --2- 172.21.15.33:0/4286269401 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fae380387a0 0x7fae3803ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fae44006fd0 tx=0x7fae44006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.445+0000 7fae54dcf700 1 -- 172.21.15.33:0/4286269401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501013c0 msgr2=0x7fae50100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.445+0000 7fae54dcf700 1 --2- 172.21.15.33:0/4286269401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501013c0 0x7fae50100bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fae4000b5c0 tx=0x7fae40005810 comp rx=0 tx=0).stop 2024-05-08T21:12:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.445+0000 7fae54dcf700 1 -- 172.21.15.33:0/4286269401 shutdown_connections 2024-05-08T21:12:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.445+0000 7fae54dcf700 1 --2- 172.21.15.33:0/4286269401 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fae380387a0 0x7fae3803ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.445+0000 7fae54dcf700 1 --2- 172.21.15.33:0/4286269401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae501013c0 0x7fae50100bf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.445+0000 7fae54dcf700 1 -- 172.21.15.33:0/4286269401 >> 172.21.15.33:0/4286269401 conn(0x7fae500fadd0 msgr2=0x7fae500fb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.445+0000 7fae54dcf700 1 -- 172.21.15.33:0/4286269401 shutdown_connections 2024-05-08T21:12:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:07.445+0000 7fae54dcf700 1 -- 172.21.15.33:0/4286269401 wait complete. 2024-05-08T21:12:07.855 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi071 2024-05-08T21:12:07.855 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-08T21:12:07.855 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.conf 2024-05-08T21:12:07.876 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-08T21:12:07.876 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:12:07.935 INFO:tasks.cephadm:Adding host smithi071 to orchestrator... 2024-05-08T21:12:07.935 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch host add smithi071 2024-05-08T21:12:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:12:08.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:08 smithi033 ceph-mon[42528]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:08.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:08 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:08.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:08 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:08.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:08 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:08.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:08 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:08.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:08 smithi033 ceph-mon[42528]: Updating smithi033:/etc/ceph/ceph.conf 2024-05-08T21:12:08.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:08 smithi033 ceph-mon[42528]: Updating smithi033:/var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T21:12:08.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:08 smithi033 ceph-mon[42528]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:12:08.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:08 smithi033 ceph-mon[42528]: Updating smithi033:/var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T21:12:09.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.397+0000 7f882d028700 1 -- 172.21.15.33:0/3539822391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88281058e0 msgr2=0x7f8828105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:09.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.397+0000 7f882d028700 1 --2- 172.21.15.33:0/3539822391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88281058e0 0x7f8828105cc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8818009910 tx=0x7f8818009c20 comp rx=0 tx=0).stop 2024-05-08T21:12:09.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.397+0000 7f882d028700 1 -- 172.21.15.33:0/3539822391 shutdown_connections 2024-05-08T21:12:09.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.397+0000 7f882d028700 1 --2- 172.21.15.33:0/3539822391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88281058e0 0x7f8828105cc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:09.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.397+0000 7f882d028700 1 -- 172.21.15.33:0/3539822391 >> 172.21.15.33:0/3539822391 conn(0x7f88280faa00 msgr2=0x7f88280fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:09.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.397+0000 7f882d028700 1 -- 172.21.15.33:0/3539822391 shutdown_connections 2024-05-08T21:12:09.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.397+0000 7f882d028700 1 -- 172.21.15.33:0/3539822391 wait complete. 2024-05-08T21:12:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.398+0000 7f882d028700 1 Processor -- start 2024-05-08T21:12:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.399+0000 7f882d028700 1 -- start start 2024-05-08T21:12:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.399+0000 7f882d028700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88281058e0 0x7f8828198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.399+0000 7f882d028700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8828198790 con 0x7f88281058e0 2024-05-08T21:12:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.399+0000 7f8826d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88281058e0 0x7f8828198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.399+0000 7f8826d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88281058e0 0x7f8828198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59176/0 (socket says 172.21.15.33:59176) 2024-05-08T21:12:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.399+0000 7f8826d9d700 1 -- 172.21.15.33:0/2046743792 learned_addr learned my addr 172.21.15.33:0/2046743792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.399+0000 7f8826d9d700 1 -- 172.21.15.33:0/2046743792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88180095f0 con 0x7f88281058e0 2024-05-08T21:12:09.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.399+0000 7f8826d9d700 1 --2- 172.21.15.33:0/2046743792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88281058e0 0x7f8828198250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8818005660 tx=0x7f8818005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:09.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.400+0000 7f8824d99700 1 -- 172.21.15.33:0/2046743792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f881801d070 con 0x7f88281058e0 2024-05-08T21:12:09.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.400+0000 7f8824d99700 1 -- 172.21.15.33:0/2046743792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f881800b750 con 0x7f88281058e0 2024-05-08T21:12:09.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.400+0000 7f882d028700 1 -- 172.21.15.33:0/2046743792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8828198990 con 0x7f88281058e0 2024-05-08T21:12:09.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.400+0000 7f8824d99700 1 -- 172.21.15.33:0/2046743792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88180037d0 con 0x7f88281058e0 2024-05-08T21:12:09.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.400+0000 7f882d028700 1 -- 172.21.15.33:0/2046743792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8828198db0 con 0x7f88281058e0 2024-05-08T21:12:09.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.401+0000 7f8824d99700 1 -- 172.21.15.33:0/2046743792 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f8818003a40 con 0x7f88281058e0 2024-05-08T21:12:09.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.401+0000 7f8824d99700 1 --2- 172.21.15.33:0/2046743792 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f88140387f0 0x7f881403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:09.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.401+0000 7f8824d99700 1 -- 172.21.15.33:0/2046743792 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f881804c4b0 con 0x7f88281058e0 2024-05-08T21:12:09.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.401+0000 7f882d028700 1 -- 172.21.15.33:0/2046743792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8828102020 con 0x7f88281058e0 2024-05-08T21:12:09.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.401+0000 7f881e59c700 1 --2- 172.21.15.33:0/2046743792 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f88140387f0 0x7f881403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:09.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.401+0000 7f881e59c700 1 --2- 172.21.15.33:0/2046743792 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f88140387f0 0x7f881403acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8810006fd0 tx=0x7f8810006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:09.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.407+0000 7f8824d99700 1 -- 172.21.15.33:0/2046743792 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8818026070 con 0x7f88281058e0 2024-05-08T21:12:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:09.624+0000 7f882d028700 1 -- 172.21.15.33:0/2046743792 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}) v1 -- 0x7f8828199600 con 0x7f88140387f0 2024-05-08T21:12:09.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:09 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:09.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:09 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:09.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:09 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:10.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:10 smithi033 ceph-mon[42528]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:10.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:10 smithi033 ceph-mon[42528]: Deploying cephadm binary to smithi071 2024-05-08T21:12:11.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:11.937+0000 7f8824d99700 1 -- 172.21.15.33:0/2046743792 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f8828199600 con 0x7f88140387f0 2024-05-08T21:12:11.940 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi071' with addr '172.21.15.71' 2024-05-08T21:12:11.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:11.939+0000 7f882d028700 1 -- 172.21.15.33:0/2046743792 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f88140387f0 msgr2=0x7f881403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:11.939+0000 7f882d028700 1 --2- 172.21.15.33:0/2046743792 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f88140387f0 0x7f881403acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8810006fd0 tx=0x7f8810006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:11.939+0000 7f882d028700 1 -- 172.21.15.33:0/2046743792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88281058e0 msgr2=0x7f8828198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:11.939+0000 7f882d028700 1 --2- 172.21.15.33:0/2046743792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88281058e0 0x7f8828198250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8818005660 tx=0x7f8818005740 comp rx=0 tx=0).stop 2024-05-08T21:12:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:11.940+0000 7f882d028700 1 -- 172.21.15.33:0/2046743792 shutdown_connections 2024-05-08T21:12:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:11.940+0000 7f882d028700 1 --2- 172.21.15.33:0/2046743792 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f88140387f0 0x7f881403acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:11.940+0000 7f882d028700 1 --2- 172.21.15.33:0/2046743792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88281058e0 0x7f8828198250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:11.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:11.940+0000 7f882d028700 1 -- 172.21.15.33:0/2046743792 >> 172.21.15.33:0/2046743792 conn(0x7f88280faa00 msgr2=0x7f88281085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:11.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:11.940+0000 7f882d028700 1 -- 172.21.15.33:0/2046743792 shutdown_connections 2024-05-08T21:12:11.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:11.940+0000 7f882d028700 1 -- 172.21.15.33:0/2046743792 wait complete. 2024-05-08T21:12:12.322 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T21:12:12.575 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:12:13.091 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:13.091 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:12 smithi033 ceph-mon[42528]: Added host smithi071 2024-05-08T21:12:13.091 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:13.091 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:13.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.698+0000 7f497e420700 1 -- 172.21.15.33:0/370207077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4978105820 msgr2=0x7f4978105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.698+0000 7f497e420700 1 --2- 172.21.15.33:0/370207077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4978105820 0x7f4978105c00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4968009910 tx=0x7f4968009c20 comp rx=0 tx=0).stop 2024-05-08T21:12:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.698+0000 7f497e420700 1 -- 172.21.15.33:0/370207077 shutdown_connections 2024-05-08T21:12:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.698+0000 7f497e420700 1 --2- 172.21.15.33:0/370207077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4978105820 0x7f4978105c00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.698+0000 7f497e420700 1 -- 172.21.15.33:0/370207077 >> 172.21.15.33:0/370207077 conn(0x7f4978075350 msgr2=0x7f4978075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.698+0000 7f497e420700 1 -- 172.21.15.33:0/370207077 shutdown_connections 2024-05-08T21:12:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.698+0000 7f497e420700 1 -- 172.21.15.33:0/370207077 wait complete. 2024-05-08T21:12:13.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.698+0000 7f497e420700 1 Processor -- start 2024-05-08T21:12:13.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.698+0000 7f497e420700 1 -- start start 2024-05-08T21:12:13.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.699+0000 7f497e420700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4978105820 0x7f49781961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:13.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.699+0000 7f497e420700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49781966e0 con 0x7f4978105820 2024-05-08T21:12:13.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.699+0000 7f4977fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4978105820 0x7f49781961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:13.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.699+0000 7f4977fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4978105820 0x7f49781961a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44242/0 (socket says 172.21.15.33:44242) 2024-05-08T21:12:13.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.699+0000 7f4977fff700 1 -- 172.21.15.33:0/1873888418 learned_addr learned my addr 172.21.15.33:0/1873888418 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:13.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.699+0000 7f4977fff700 1 -- 172.21.15.33:0/1873888418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49680095f0 con 0x7f4978105820 2024-05-08T21:12:13.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.699+0000 7f4977fff700 1 --2- 172.21.15.33:0/1873888418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4978105820 0x7f49781961a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49680098e0 tx=0x7f4968005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:13.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.699+0000 7f49757fa700 1 -- 172.21.15.33:0/1873888418 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f496801d070 con 0x7f4978105820 2024-05-08T21:12:13.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.700+0000 7f49757fa700 1 -- 172.21.15.33:0/1873888418 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4968009e60 con 0x7f4978105820 2024-05-08T21:12:13.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.700+0000 7f497e420700 1 -- 172.21.15.33:0/1873888418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49781968e0 con 0x7f4978105820 2024-05-08T21:12:13.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.700+0000 7f49757fa700 1 -- 172.21.15.33:0/1873888418 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4968022c10 con 0x7f4978105820 2024-05-08T21:12:13.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.700+0000 7f497e420700 1 -- 172.21.15.33:0/1873888418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4978196d00 con 0x7f4978105820 2024-05-08T21:12:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.701+0000 7f497e420700 1 -- 172.21.15.33:0/1873888418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49780ff3f0 con 0x7f4978105820 2024-05-08T21:12:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.701+0000 7f49757fa700 1 -- 172.21.15.33:0/1873888418 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f4968022470 con 0x7f4978105820 2024-05-08T21:12:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.701+0000 7f49757fa700 1 --2- 172.21.15.33:0/1873888418 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f49600387f0 0x7f496003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.701+0000 7f49757fa700 1 -- 172.21.15.33:0/1873888418 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f496804d060 con 0x7f4978105820 2024-05-08T21:12:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.701+0000 7f49777fe700 1 --2- 172.21.15.33:0/1873888418 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f49600387f0 0x7f496003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.702+0000 7f49777fe700 1 --2- 172.21.15.33:0/1873888418 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f49600387f0 0x7f496003acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f496c006fd0 tx=0x7f496c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:13.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.707+0000 7f49757fa700 1 -- 172.21.15.33:0/1873888418 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f496802b590 con 0x7f4978105820 2024-05-08T21:12:13.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.924+0000 7f497e420700 1 -- 172.21.15.33:0/1873888418 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49781975f0 con 0x7f49600387f0 2024-05-08T21:12:13.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.925+0000 7f49757fa700 1 -- 172.21.15.33:0/1873888418 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f49781975f0 con 0x7f49600387f0 2024-05-08T21:12:13.928 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:12:13.928 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}] 2024-05-08T21:12:13.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.927+0000 7f497e420700 1 -- 172.21.15.33:0/1873888418 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f49600387f0 msgr2=0x7f496003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:13.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.927+0000 7f497e420700 1 --2- 172.21.15.33:0/1873888418 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f49600387f0 0x7f496003acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f496c006fd0 tx=0x7f496c006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:13.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.927+0000 7f497e420700 1 -- 172.21.15.33:0/1873888418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4978105820 msgr2=0x7f49781961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:13.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.927+0000 7f497e420700 1 --2- 172.21.15.33:0/1873888418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4978105820 0x7f49781961a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49680098e0 tx=0x7f4968005710 comp rx=0 tx=0).stop 2024-05-08T21:12:13.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.927+0000 7f497e420700 1 -- 172.21.15.33:0/1873888418 shutdown_connections 2024-05-08T21:12:13.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.927+0000 7f497e420700 1 --2- 172.21.15.33:0/1873888418 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f49600387f0 0x7f496003acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:13.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.927+0000 7f497e420700 1 --2- 172.21.15.33:0/1873888418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4978105820 0x7f49781961a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:13.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.927+0000 7f497e420700 1 -- 172.21.15.33:0/1873888418 >> 172.21.15.33:0/1873888418 conn(0x7f4978075350 msgr2=0x7f49780680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:13.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.927+0000 7f497e420700 1 -- 172.21.15.33:0/1873888418 shutdown_connections 2024-05-08T21:12:13.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:13.927+0000 7f497e420700 1 -- 172.21.15.33:0/1873888418 wait complete. 2024-05-08T21:12:14.329 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi082 2024-05-08T21:12:14.329 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-08T21:12:14.329 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/etc/ceph/ceph.conf 2024-05-08T21:12:14.350 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-08T21:12:14.350 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:12:14.410 INFO:tasks.cephadm:Adding host smithi082 to orchestrator... 2024-05-08T21:12:14.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch host add smithi082 2024-05-08T21:12:14.662 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:12:14.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:14 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:15.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:15 smithi033 ceph-mon[42528]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:12:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.693+0000 7f99c76d0700 1 -- 172.21.15.33:0/360301228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0105820 msgr2=0x7f99c0105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.693+0000 7f99c76d0700 1 --2- 172.21.15.33:0/360301228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0105820 0x7f99c0105c00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009910 tx=0x7f99bc009c20 comp rx=0 tx=0).stop 2024-05-08T21:12:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.693+0000 7f99c76d0700 1 -- 172.21.15.33:0/360301228 shutdown_connections 2024-05-08T21:12:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.693+0000 7f99c76d0700 1 --2- 172.21.15.33:0/360301228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0105820 0x7f99c0105c00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.693+0000 7f99c76d0700 1 -- 172.21.15.33:0/360301228 >> 172.21.15.33:0/360301228 conn(0x7f99c0075350 msgr2=0x7f99c0075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.693+0000 7f99c76d0700 1 -- 172.21.15.33:0/360301228 shutdown_connections 2024-05-08T21:12:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.693+0000 7f99c76d0700 1 -- 172.21.15.33:0/360301228 wait complete. 2024-05-08T21:12:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.694+0000 7f99c76d0700 1 Processor -- start 2024-05-08T21:12:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.694+0000 7f99c76d0700 1 -- start start 2024-05-08T21:12:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.694+0000 7f99c76d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0105820 0x7f99c0196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.694+0000 7f99c76d0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01966a0 con 0x7f99c0105820 2024-05-08T21:12:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.694+0000 7f99c546c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0105820 0x7f99c0196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.694+0000 7f99c546c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0105820 0x7f99c0196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44264/0 (socket says 172.21.15.33:44264) 2024-05-08T21:12:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.694+0000 7f99c546c700 1 -- 172.21.15.33:0/4206910772 learned_addr learned my addr 172.21.15.33:0/4206910772 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.694+0000 7f99c546c700 1 -- 172.21.15.33:0/4206910772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc0095f0 con 0x7f99c0105820 2024-05-08T21:12:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.695+0000 7f99c546c700 1 --2- 172.21.15.33:0/4206910772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0105820 0x7f99c0196160 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0098e0 tx=0x7f99bc005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.695+0000 7f99b67fc700 1 -- 172.21.15.33:0/4206910772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99bc01d070 con 0x7f99c0105820 2024-05-08T21:12:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.695+0000 7f99c76d0700 1 -- 172.21.15.33:0/4206910772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c01968a0 con 0x7f99c0105820 2024-05-08T21:12:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.695+0000 7f99b67fc700 1 -- 172.21.15.33:0/4206910772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99bc009e60 con 0x7f99c0105820 2024-05-08T21:12:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.695+0000 7f99b67fc700 1 -- 172.21.15.33:0/4206910772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99bc022c10 con 0x7f99c0105820 2024-05-08T21:12:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.695+0000 7f99c76d0700 1 -- 172.21.15.33:0/4206910772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c0196cc0 con 0x7f99c0105820 2024-05-08T21:12:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.696+0000 7f99b67fc700 1 -- 172.21.15.33:0/4206910772 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f99bc00f460 con 0x7f99c0105820 2024-05-08T21:12:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.696+0000 7f99c76d0700 1 -- 172.21.15.33:0/4206910772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c00ff3c0 con 0x7f99c0105820 2024-05-08T21:12:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.696+0000 7f99b67fc700 1 --2- 172.21.15.33:0/4206910772 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f99ac0387a0 0x7f99ac03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.696+0000 7f99b67fc700 1 -- 172.21.15.33:0/4206910772 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f99bc04cba0 con 0x7f99c0105820 2024-05-08T21:12:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.696+0000 7f99c4c6b700 1 --2- 172.21.15.33:0/4206910772 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f99ac0387a0 0x7f99ac03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:15.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.697+0000 7f99c4c6b700 1 --2- 172.21.15.33:0/4206910772 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f99ac0387a0 0x7f99ac03ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f99b0006fd0 tx=0x7f99b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.702+0000 7f99b67fc700 1 -- 172.21.15.33:0/4206910772 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f99bc020960 con 0x7f99c0105820 2024-05-08T21:12:15.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:15.920+0000 7f99c76d0700 1 -- 172.21.15.33:0/4206910772 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi082", "target": ["mon-mgr", ""]}) v1 -- 0x7f99c01099e0 con 0x7f99ac0387a0 2024-05-08T21:12:16.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:16.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:16.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:16.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:16.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:12:16.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:16.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:16.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:16 smithi033 ceph-mon[42528]: Updating smithi071:/etc/ceph/ceph.conf 2024-05-08T21:12:16.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:16 smithi033 ceph-mon[42528]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi082", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:16.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:16 smithi033 ceph-mon[42528]: Updating smithi071:/var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T21:12:16.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:16 smithi033 ceph-mon[42528]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:16.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:16 smithi033 ceph-mon[42528]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:12:18.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:18.190+0000 7f99b67fc700 1 -- 172.21.15.33:0/4206910772 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f99c01099e0 con 0x7f99ac0387a0 2024-05-08T21:12:18.195 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi082' with addr '172.21.15.82' 2024-05-08T21:12:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:18.194+0000 7f99c76d0700 1 -- 172.21.15.33:0/4206910772 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f99ac0387a0 msgr2=0x7f99ac03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:18.194+0000 7f99c76d0700 1 --2- 172.21.15.33:0/4206910772 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f99ac0387a0 0x7f99ac03ac60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f99b0006fd0 tx=0x7f99b0006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:18.194+0000 7f99c76d0700 1 -- 172.21.15.33:0/4206910772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0105820 msgr2=0x7f99c0196160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:18.194+0000 7f99c76d0700 1 --2- 172.21.15.33:0/4206910772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0105820 0x7f99c0196160 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0098e0 tx=0x7f99bc005710 comp rx=0 tx=0).stop 2024-05-08T21:12:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:18.194+0000 7f99c76d0700 1 -- 172.21.15.33:0/4206910772 shutdown_connections 2024-05-08T21:12:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:18.194+0000 7f99c76d0700 1 --2- 172.21.15.33:0/4206910772 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f99ac0387a0 0x7f99ac03ac60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:18.194+0000 7f99c76d0700 1 --2- 172.21.15.33:0/4206910772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99c0105820 0x7f99c0196160 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:18.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:18.194+0000 7f99c76d0700 1 -- 172.21.15.33:0/4206910772 >> 172.21.15.33:0/4206910772 conn(0x7f99c0075350 msgr2=0x7f99c00680f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:18.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:18.194+0000 7f99c76d0700 1 -- 172.21.15.33:0/4206910772 shutdown_connections 2024-05-08T21:12:18.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:18.194+0000 7f99c76d0700 1 -- 172.21.15.33:0/4206910772 wait complete. 2024-05-08T21:12:18.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:17 smithi033 ceph-mon[42528]: Deploying cephadm binary to smithi082 2024-05-08T21:12:18.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:17 smithi033 ceph-mon[42528]: Updating smithi071:/var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T21:12:18.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:17 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:18.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:17 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:18.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:17 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:18.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T21:12:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:12:19.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:19 smithi033 ceph-mon[42528]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:19.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:19.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:19 smithi033 ceph-mon[42528]: Added host smithi082 2024-05-08T21:12:19.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:19.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:19.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.904+0000 7fad8694f700 1 -- 172.21.15.33:0/2199462304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801068c0 msgr2=0x7fad80106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.904+0000 7fad8694f700 1 --2- 172.21.15.33:0/2199462304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801068c0 0x7fad80106ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad70009910 tx=0x7fad70009c20 comp rx=0 tx=0).stop 2024-05-08T21:12:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.904+0000 7fad8694f700 1 -- 172.21.15.33:0/2199462304 shutdown_connections 2024-05-08T21:12:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.904+0000 7fad8694f700 1 --2- 172.21.15.33:0/2199462304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801068c0 0x7fad80106ca0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.904+0000 7fad8694f700 1 -- 172.21.15.33:0/2199462304 >> 172.21.15.33:0/2199462304 conn(0x7fad800fc3f0 msgr2=0x7fad800fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.904+0000 7fad8694f700 1 -- 172.21.15.33:0/2199462304 shutdown_connections 2024-05-08T21:12:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.904+0000 7fad8694f700 1 -- 172.21.15.33:0/2199462304 wait complete. 2024-05-08T21:12:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.905+0000 7fad8694f700 1 Processor -- start 2024-05-08T21:12:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.905+0000 7fad8694f700 1 -- start start 2024-05-08T21:12:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.905+0000 7fad8694f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801068c0 0x7fad80196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.905+0000 7fad8694f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad801965d0 con 0x7fad801068c0 2024-05-08T21:12:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.906+0000 7fad7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801068c0 0x7fad80196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.906+0000 7fad7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801068c0 0x7fad80196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44280/0 (socket says 172.21.15.33:44280) 2024-05-08T21:12:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.906+0000 7fad7ffff700 1 -- 172.21.15.33:0/769957668 learned_addr learned my addr 172.21.15.33:0/769957668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.906+0000 7fad7ffff700 1 -- 172.21.15.33:0/769957668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad700095f0 con 0x7fad801068c0 2024-05-08T21:12:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.906+0000 7fad7ffff700 1 --2- 172.21.15.33:0/769957668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801068c0 0x7fad80196090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fad700098e0 tx=0x7fad70005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.906+0000 7fad7d7fa700 1 -- 172.21.15.33:0/769957668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad7001d070 con 0x7fad801068c0 2024-05-08T21:12:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.906+0000 7fad8694f700 1 -- 172.21.15.33:0/769957668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad80196830 con 0x7fad801068c0 2024-05-08T21:12:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.906+0000 7fad7d7fa700 1 -- 172.21.15.33:0/769957668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad70009e60 con 0x7fad801068c0 2024-05-08T21:12:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.906+0000 7fad7d7fa700 1 -- 172.21.15.33:0/769957668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad70022bb0 con 0x7fad801068c0 2024-05-08T21:12:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.906+0000 7fad8694f700 1 -- 172.21.15.33:0/769957668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad80196cb0 con 0x7fad801068c0 2024-05-08T21:12:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.907+0000 7fad7d7fa700 1 -- 172.21.15.33:0/769957668 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fad7000f460 con 0x7fad801068c0 2024-05-08T21:12:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.907+0000 7fad7d7fa700 1 --2- 172.21.15.33:0/769957668 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad680387a0 0x7fad6803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.907+0000 7fad8694f700 1 -- 172.21.15.33:0/769957668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad80102000 con 0x7fad801068c0 2024-05-08T21:12:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.907+0000 7fad7d7fa700 1 -- 172.21.15.33:0/769957668 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fad7004ca10 con 0x7fad801068c0 2024-05-08T21:12:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.908+0000 7fad7f7fe700 1 --2- 172.21.15.33:0/769957668 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad680387a0 0x7fad6803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.908+0000 7fad7f7fe700 1 --2- 172.21.15.33:0/769957668 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad680387a0 0x7fad6803ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fad74006fd0 tx=0x7fad74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:19.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:19.913+0000 7fad7d7fa700 1 -- 172.21.15.33:0/769957668 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fad70027030 con 0x7fad801068c0 2024-05-08T21:12:20.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:20.134+0000 7fad8694f700 1 -- 172.21.15.33:0/769957668 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad8010aa70 con 0x7fad680387a0 2024-05-08T21:12:20.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:20.135+0000 7fad7d7fa700 1 -- 172.21.15.33:0/769957668 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+238 (secure 0 0 0) 0x7fad8010aa70 con 0x7fad680387a0 2024-05-08T21:12:20.138 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:12:20.139 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}, {"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}] 2024-05-08T21:12:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:20.138+0000 7fad8694f700 1 -- 172.21.15.33:0/769957668 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad680387a0 msgr2=0x7fad6803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:20.138+0000 7fad8694f700 1 --2- 172.21.15.33:0/769957668 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad680387a0 0x7fad6803ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fad74006fd0 tx=0x7fad74006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:20.138+0000 7fad8694f700 1 -- 172.21.15.33:0/769957668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801068c0 msgr2=0x7fad80196090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:20.138+0000 7fad8694f700 1 --2- 172.21.15.33:0/769957668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801068c0 0x7fad80196090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fad700098e0 tx=0x7fad70005710 comp rx=0 tx=0).stop 2024-05-08T21:12:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:20.138+0000 7fad8694f700 1 -- 172.21.15.33:0/769957668 shutdown_connections 2024-05-08T21:12:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:20.138+0000 7fad8694f700 1 --2- 172.21.15.33:0/769957668 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad680387a0 0x7fad6803ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:20.138+0000 7fad8694f700 1 --2- 172.21.15.33:0/769957668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad801068c0 0x7fad80196090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:20.138+0000 7fad8694f700 1 -- 172.21.15.33:0/769957668 >> 172.21.15.33:0/769957668 conn(0x7fad800fc3f0 msgr2=0x7fad801085c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:20.138+0000 7fad8694f700 1 -- 172.21.15.33:0/769957668 shutdown_connections 2024-05-08T21:12:20.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:20.138+0000 7fad8694f700 1 -- 172.21.15.33:0/769957668 wait complete. 2024-05-08T21:12:20.529 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-08T21:12:20.529 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-08T21:12:20.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:20 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:20.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:20 smithi033 ceph-mon[42528]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:20.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:20 smithi033 ceph-mon[42528]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:12:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:12:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.745+0000 7f058e1fa700 1 -- 172.21.15.33:0/821979737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05881068c0 msgr2=0x7f0588106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.745+0000 7f058e1fa700 1 --2- 172.21.15.33:0/821979737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05881068c0 0x7f0588106ca0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f058000b0a0 tx=0x7f058000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:12:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.746+0000 7f058e1fa700 1 -- 172.21.15.33:0/821979737 shutdown_connections 2024-05-08T21:12:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.746+0000 7f058e1fa700 1 --2- 172.21.15.33:0/821979737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05881068c0 0x7f0588106ca0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.746+0000 7f058e1fa700 1 -- 172.21.15.33:0/821979737 >> 172.21.15.33:0/821979737 conn(0x7f0588075350 msgr2=0x7f0588075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.746+0000 7f058e1fa700 1 -- 172.21.15.33:0/821979737 shutdown_connections 2024-05-08T21:12:21.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.746+0000 7f058e1fa700 1 -- 172.21.15.33:0/821979737 wait complete. 2024-05-08T21:12:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.746+0000 7f058e1fa700 1 Processor -- start 2024-05-08T21:12:21.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.747+0000 7f058e1fa700 1 -- start start 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.747+0000 7f058e1fa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05881068c0 0x7f0588195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.747+0000 7f058e1fa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05881964f0 con 0x7f05881068c0 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.747+0000 7f05877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05881068c0 0x7f0588195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.747+0000 7f05877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05881068c0 0x7f0588195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44590/0 (socket says 172.21.15.33:44590) 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.747+0000 7f05877fe700 1 -- 172.21.15.33:0/3082066741 learned_addr learned my addr 172.21.15.33:0/3082066741 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.747+0000 7f05877fe700 1 -- 172.21.15.33:0/3082066741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0580009d00 con 0x7f05881068c0 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.747+0000 7f05877fe700 1 --2- 172.21.15.33:0/3082066741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05881068c0 0x7f0588195fb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0580015040 tx=0x7f05800078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.748+0000 7f0584ff9700 1 -- 172.21.15.33:0/3082066741 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f058000e040 con 0x7f05881068c0 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.748+0000 7f058e1fa700 1 -- 172.21.15.33:0/3082066741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05881966f0 con 0x7f05881068c0 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.748+0000 7f0584ff9700 1 -- 172.21.15.33:0/3082066741 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0580003c10 con 0x7f05881068c0 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.748+0000 7f0584ff9700 1 -- 172.21.15.33:0/3082066741 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f058001d610 con 0x7f05881068c0 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.748+0000 7f058e1fa700 1 -- 172.21.15.33:0/3082066741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0588196b70 con 0x7f05881068c0 2024-05-08T21:12:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.748+0000 7f0584ff9700 1 -- 172.21.15.33:0/3082066741 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f058001d8d0 con 0x7f05881068c0 2024-05-08T21:12:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.749+0000 7f0584ff9700 1 --2- 172.21.15.33:0/3082066741 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f05740387a0 0x7f057403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.749+0000 7f058e1fa700 1 -- 172.21.15.33:0/3082066741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0588102000 con 0x7f05881068c0 2024-05-08T21:12:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.749+0000 7f0584ff9700 1 -- 172.21.15.33:0/3082066741 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f058004cb30 con 0x7f05881068c0 2024-05-08T21:12:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.749+0000 7f0586ffd700 1 --2- 172.21.15.33:0/3082066741 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f05740387a0 0x7f057403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.749+0000 7f0586ffd700 1 --2- 172.21.15.33:0/3082066741 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f05740387a0 0x7f057403ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0570006fd0 tx=0x7f0570006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.754+0000 7f0584ff9700 1 -- 172.21.15.33:0/3082066741 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f058001c5c0 con 0x7f05881068c0 2024-05-08T21:12:21.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:21.975+0000 7f058e1fa700 1 -- 172.21.15.33:0/3082066741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f058810aa60 con 0x7f05881068c0 2024-05-08T21:12:22.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:22.028+0000 7f0584ff9700 1 -- 172.21.15.33:0/3082066741 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0580017070 con 0x7f05881068c0 2024-05-08T21:12:22.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:22.031+0000 7f058e1fa700 1 -- 172.21.15.33:0/3082066741 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f05740387a0 msgr2=0x7f057403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:22.031+0000 7f058e1fa700 1 --2- 172.21.15.33:0/3082066741 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f05740387a0 0x7f057403ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0570006fd0 tx=0x7f0570006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:22.031+0000 7f058e1fa700 1 -- 172.21.15.33:0/3082066741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05881068c0 msgr2=0x7f0588195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:22.031+0000 7f058e1fa700 1 --2- 172.21.15.33:0/3082066741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05881068c0 0x7f0588195fb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0580015040 tx=0x7f05800078d0 comp rx=0 tx=0).stop 2024-05-08T21:12:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:22.031+0000 7f058e1fa700 1 -- 172.21.15.33:0/3082066741 shutdown_connections 2024-05-08T21:12:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:22.031+0000 7f058e1fa700 1 --2- 172.21.15.33:0/3082066741 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f05740387a0 0x7f057403ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:22.031+0000 7f058e1fa700 1 --2- 172.21.15.33:0/3082066741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05881068c0 0x7f0588195fb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:22.031+0000 7f058e1fa700 1 -- 172.21.15.33:0/3082066741 >> 172.21.15.33:0/3082066741 conn(0x7f0588075350 msgr2=0x7f05880fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:22.031+0000 7f058e1fa700 1 -- 172.21.15.33:0/3082066741 shutdown_connections 2024-05-08T21:12:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:22.031+0000 7f058e1fa700 1 -- 172.21.15.33:0/3082066741 wait complete. 2024-05-08T21:12:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-05-08T21:12:22.351 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:22 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3082066741' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T21:12:22.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:22 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:22.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:22 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:22.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:22 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:22.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:22 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:22.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:22 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:12:22.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:22 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:22.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:22 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:22.412 INFO:tasks.cephadm:Adding mon.a on smithi033 2024-05-08T21:12:22.412 INFO:tasks.cephadm:Adding mon.b on smithi071 2024-05-08T21:12:22.412 INFO:tasks.cephadm:Adding mon.c on smithi082 2024-05-08T21:12:22.412 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi033:172.21.15.33=a;smithi071:172.21.15.71=b;smithi082:172.21.15.82=c' 2024-05-08T21:12:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T21:12:23.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:23 smithi033 ceph-mon[42528]: Updating smithi082:/etc/ceph/ceph.conf 2024-05-08T21:12:23.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:23 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3082066741' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T21:12:23.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:23 smithi033 ceph-mon[42528]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:12:23.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:23 smithi033 ceph-mon[42528]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:23.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:23 smithi033 ceph-mon[42528]: Updating smithi082:/var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T21:12:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.617+0000 7fa8c6c69700 1 -- 172.21.15.82:0/4131605633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c00ff020 msgr2=0x7fa8c00ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.617+0000 7fa8c6c69700 1 --2- 172.21.15.82:0/4131605633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c00ff020 0x7fa8c00ff400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009910 tx=0x7fa8b4009c20 comp rx=0 tx=0).stop 2024-05-08T21:12:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.617+0000 7fa8c6c69700 1 -- 172.21.15.82:0/4131605633 shutdown_connections 2024-05-08T21:12:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.617+0000 7fa8c6c69700 1 --2- 172.21.15.82:0/4131605633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c00ff020 0x7fa8c00ff400 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.617+0000 7fa8c6c69700 1 -- 172.21.15.82:0/4131605633 >> 172.21.15.82:0/4131605633 conn(0x7fa8c00faa00 msgr2=0x7fa8c00fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.617+0000 7fa8c6c69700 1 -- 172.21.15.82:0/4131605633 shutdown_connections 2024-05-08T21:12:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.617+0000 7fa8c6c69700 1 -- 172.21.15.82:0/4131605633 wait complete. 2024-05-08T21:12:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.617+0000 7fa8c6c69700 1 Processor -- start 2024-05-08T21:12:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.617+0000 7fa8c6c69700 1 -- start start 2024-05-08T21:12:23.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.617+0000 7fa8c6c69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c00ff020 0x7fa8c01960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:23.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.617+0000 7fa8c6c69700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0196630 con 0x7fa8c00ff020 2024-05-08T21:12:23.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.618+0000 7fa8c4a05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c00ff020 0x7fa8c01960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:23.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.618+0000 7fa8c4a05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c00ff020 0x7fa8c01960f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.82:48244/0 (socket says 172.21.15.82:48244) 2024-05-08T21:12:23.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.618+0000 7fa8c4a05700 1 -- 172.21.15.82:0/465020301 learned_addr learned my addr 172.21.15.82:0/465020301 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-08T21:12:23.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.618+0000 7fa8c4a05700 1 -- 172.21.15.82:0/465020301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b40095f0 con 0x7fa8c00ff020 2024-05-08T21:12:23.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.618+0000 7fa8c4a05700 1 --2- 172.21.15.82:0/465020301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c00ff020 0x7fa8c01960f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4005660 tx=0x7fa8b4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.618+0000 7fa8bdffb700 1 -- 172.21.15.82:0/465020301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8b401d070 con 0x7fa8c00ff020 2024-05-08T21:12:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.618+0000 7fa8c6c69700 1 -- 172.21.15.82:0/465020301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c0196830 con 0x7fa8c00ff020 2024-05-08T21:12:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.619+0000 7fa8bdffb700 1 -- 172.21.15.82:0/465020301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8b400b750 con 0x7fa8c00ff020 2024-05-08T21:12:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.619+0000 7fa8bdffb700 1 -- 172.21.15.82:0/465020301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8b40037d0 con 0x7fa8c00ff020 2024-05-08T21:12:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.619+0000 7fa8c6c69700 1 -- 172.21.15.82:0/465020301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c0196c50 con 0x7fa8c00ff020 2024-05-08T21:12:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.620+0000 7fa8c6c69700 1 -- 172.21.15.82:0/465020301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c01020e0 con 0x7fa8c00ff020 2024-05-08T21:12:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.620+0000 7fa8bdffb700 1 -- 172.21.15.82:0/465020301 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7fa8b4004390 con 0x7fa8c00ff020 2024-05-08T21:12:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.620+0000 7fa8bdffb700 1 --2- 172.21.15.82:0/465020301 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa8b00387a0 0x7fa8b003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.620+0000 7fa8bdffb700 1 -- 172.21.15.82:0/465020301 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa8b404c490 con 0x7fa8c00ff020 2024-05-08T21:12:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.620+0000 7fa8bffff700 1 --2- 172.21.15.82:0/465020301 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa8b00387a0 0x7fa8b003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:23.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.621+0000 7fa8bffff700 1 --2- 172.21.15.82:0/465020301 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa8b00387a0 0x7fa8b003ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac006fd0 tx=0x7fa8ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:23.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.626+0000 7fa8bdffb700 1 -- 172.21.15.82:0/465020301 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa8b400b8c0 con 0x7fa8c00ff020 2024-05-08T21:12:23.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.828+0000 7fa8c6c69700 1 -- 172.21.15.82:0/465020301 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi071:172.21.15.71=b;smithi082:172.21.15.82=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8c010aab0 con 0x7fa8b00387a0 2024-05-08T21:12:23.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.835+0000 7fa8bdffb700 1 -- 172.21.15.82:0/465020301 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa8c010aab0 con 0x7fa8b00387a0 2024-05-08T21:12:23.838 INFO:teuthology.orchestra.run.smithi082.stdout:Scheduled mon update... 2024-05-08T21:12:23.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.837+0000 7fa8c6c69700 1 -- 172.21.15.82:0/465020301 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa8b00387a0 msgr2=0x7fa8b003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:23.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.837+0000 7fa8c6c69700 1 --2- 172.21.15.82:0/465020301 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa8b00387a0 0x7fa8b003ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac006fd0 tx=0x7fa8ac006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:23.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.837+0000 7fa8c6c69700 1 -- 172.21.15.82:0/465020301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c00ff020 msgr2=0x7fa8c01960f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:23.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.837+0000 7fa8c6c69700 1 --2- 172.21.15.82:0/465020301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c00ff020 0x7fa8c01960f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4005660 tx=0x7fa8b4005740 comp rx=0 tx=0).stop 2024-05-08T21:12:23.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.837+0000 7fa8c6c69700 1 -- 172.21.15.82:0/465020301 shutdown_connections 2024-05-08T21:12:23.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.837+0000 7fa8c6c69700 1 --2- 172.21.15.82:0/465020301 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa8b00387a0 0x7fa8b003ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:23.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.837+0000 7fa8c6c69700 1 --2- 172.21.15.82:0/465020301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c00ff020 0x7fa8c01960f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:23.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.837+0000 7fa8c6c69700 1 -- 172.21.15.82:0/465020301 >> 172.21.15.82:0/465020301 conn(0x7fa8c00faa00 msgr2=0x7fa8c01024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:23.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.837+0000 7fa8c6c69700 1 -- 172.21.15.82:0/465020301 shutdown_connections 2024-05-08T21:12:23.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:23.837+0000 7fa8c6c69700 1 -- 172.21.15.82:0/465020301 wait complete. 2024-05-08T21:12:24.290 DEBUG:teuthology.orchestra.run.smithi071:mon.b> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.b.service 2024-05-08T21:12:24.293 DEBUG:teuthology.orchestra.run.smithi082:mon.c> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.c.service 2024-05-08T21:12:24.295 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T21:12:24.295 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T21:12:24.324 INFO:journalctl@ceph.mon.c.smithi082.stdout:-- Logs begin at Wed 2024-05-08 20:53:53 UTC. -- 2024-05-08T21:12:24.329 INFO:journalctl@ceph.mon.b.smithi071.stdout:-- Logs begin at Wed 2024-05-08 20:53:45 UTC. -- 2024-05-08T21:12:24.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:24 smithi033 ceph-mon[42528]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:12:24.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:24 smithi033 ceph-mon[42528]: Updating smithi082:/var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T21:12:24.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:24.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:24.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:24.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:24.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:24.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:24.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:24.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:24.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:12:24.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:24.692 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T21:12:25.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:25 smithi033 ceph-mon[42528]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi071:172.21.15.71=b;smithi082:172.21.15.82=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:25.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:25 smithi033 ceph-mon[42528]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi071:172.21.15.71=b;smithi082:172.21.15.82=c;count:3 2024-05-08T21:12:25.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:25 smithi033 ceph-mon[42528]: Deploying daemon mon.c on smithi082 2024-05-08T21:12:25.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:25 smithi033 ceph-mon[42528]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:26.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.334+0000 7f7c693d2700 1 -- 172.21.15.82:0/556167056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c641068c0 msgr2=0x7f7c64106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:26.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.334+0000 7f7c693d2700 1 --2- 172.21.15.82:0/556167056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c641068c0 0x7f7c64106ca0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009910 tx=0x7f7c58009c20 comp rx=0 tx=0).stop 2024-05-08T21:12:26.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.334+0000 7f7c693d2700 1 -- 172.21.15.82:0/556167056 shutdown_connections 2024-05-08T21:12:26.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.334+0000 7f7c693d2700 1 --2- 172.21.15.82:0/556167056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c641068c0 0x7f7c64106ca0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:26.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.334+0000 7f7c693d2700 1 -- 172.21.15.82:0/556167056 >> 172.21.15.82:0/556167056 conn(0x7f7c64075350 msgr2=0x7f7c64075760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:26.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.334+0000 7f7c693d2700 1 -- 172.21.15.82:0/556167056 shutdown_connections 2024-05-08T21:12:26.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.334+0000 7f7c693d2700 1 -- 172.21.15.82:0/556167056 wait complete. 2024-05-08T21:12:26.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.335+0000 7f7c693d2700 1 Processor -- start 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.335+0000 7f7c693d2700 1 -- start start 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.335+0000 7f7c693d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c641068c0 0x7f7c64196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.335+0000 7f7c693d2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c64196670 con 0x7f7c641068c0 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.335+0000 7f7c62ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c641068c0 0x7f7c64196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.335+0000 7f7c62ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c641068c0 0x7f7c64196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.82:48266/0 (socket says 172.21.15.82:48266) 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.335+0000 7f7c62ffd700 1 -- 172.21.15.82:0/2216058062 learned_addr learned my addr 172.21.15.82:0/2216058062 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.336+0000 7f7c62ffd700 1 -- 172.21.15.82:0/2216058062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c580095f0 con 0x7f7c641068c0 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.336+0000 7f7c62ffd700 1 --2- 172.21.15.82:0/2216058062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c641068c0 0x7f7c64196130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005660 tx=0x7f7c58005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.336+0000 7f7c4bfff700 1 -- 172.21.15.82:0/2216058062 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c5801d070 con 0x7f7c641068c0 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.336+0000 7f7c693d2700 1 -- 172.21.15.82:0/2216058062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c641968d0 con 0x7f7c641068c0 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.336+0000 7f7c4bfff700 1 -- 172.21.15.82:0/2216058062 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c5800b6f0 con 0x7f7c641068c0 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.336+0000 7f7c4bfff700 1 -- 172.21.15.82:0/2216058062 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c580037f0 con 0x7f7c641068c0 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.336+0000 7f7c693d2700 1 -- 172.21.15.82:0/2216058062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c64196d50 con 0x7f7c641068c0 2024-05-08T21:12:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.337+0000 7f7c693d2700 1 -- 172.21.15.82:0/2216058062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c64102000 con 0x7f7c641068c0 2024-05-08T21:12:26.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.337+0000 7f7c4bfff700 1 -- 172.21.15.82:0/2216058062 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f7c58004390 con 0x7f7c641068c0 2024-05-08T21:12:26.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.337+0000 7f7c4bfff700 1 --2- 172.21.15.82:0/2216058062 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c4c0387a0 0x7f7c4c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:26.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.337+0000 7f7c4bfff700 1 -- 172.21.15.82:0/2216058062 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7c5804c3c0 con 0x7f7c641068c0 2024-05-08T21:12:26.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.337+0000 7f7c627fc700 1 --2- 172.21.15.82:0/2216058062 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c4c0387a0 0x7f7c4c03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:26.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.338+0000 7f7c627fc700 1 --2- 172.21.15.82:0/2216058062 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c4c0387a0 0x7f7c4c03ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7c54006fd0 tx=0x7f7c54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:26.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.343+0000 7f7c4bfff700 1 -- 172.21.15.82:0/2216058062 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7c58003e30 con 0x7f7c641068c0 2024-05-08T21:12:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.644+0000 7f7c693d2700 1 -- 172.21.15.82:0/2216058062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7c640625d0 con 0x7f7c641068c0 2024-05-08T21:12:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.645+0000 7f7c4bfff700 1 -- 172.21.15.82:0/2216058062 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f7c58026070 con 0x7f7c641068c0 2024-05-08T21:12:26.649 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:12:26.649 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":1,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","modified":"2024-05-08T21:11:04.684022Z","created":"2024-05-08T21:11:04.684022Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-08T21:12:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.648+0000 7f7c693d2700 1 -- 172.21.15.82:0/2216058062 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c4c0387a0 msgr2=0x7f7c4c03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.648+0000 7f7c693d2700 1 --2- 172.21.15.82:0/2216058062 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c4c0387a0 0x7f7c4c03ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7c54006fd0 tx=0x7f7c54006e40 comp rx=0 tx=0).stop 2024-05-08T21:12:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.648+0000 7f7c693d2700 1 -- 172.21.15.82:0/2216058062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c641068c0 msgr2=0x7f7c64196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.648+0000 7f7c693d2700 1 --2- 172.21.15.82:0/2216058062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c641068c0 0x7f7c64196130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005660 tx=0x7f7c58005740 comp rx=0 tx=0).stop 2024-05-08T21:12:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.649+0000 7f7c693d2700 1 -- 172.21.15.82:0/2216058062 shutdown_connections 2024-05-08T21:12:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.649+0000 7f7c693d2700 1 --2- 172.21.15.82:0/2216058062 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c4c0387a0 0x7f7c4c03ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.649+0000 7f7c693d2700 1 --2- 172.21.15.82:0/2216058062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c641068c0 0x7f7c64196130 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.649+0000 7f7c693d2700 1 -- 172.21.15.82:0/2216058062 >> 172.21.15.82:0/2216058062 conn(0x7f7c64075350 msgr2=0x7f7c640fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.649+0000 7f7c693d2700 1 -- 172.21.15.82:0/2216058062 shutdown_connections 2024-05-08T21:12:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:26.649+0000 7f7c693d2700 1 -- 172.21.15.82:0/2216058062 wait complete. 2024-05-08T21:12:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:dumped monmap epoch 1 2024-05-08T21:12:27.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:27 smithi033 ceph-mon[42528]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:27.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:27 smithi033 ceph-mon[42528]: from='client.? 172.21.15.82:0/2216058062' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T21:12:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 systemd[1]: Started Ceph mon.c for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:12:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T21:12:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c7047f4000 2024-05-08T21:12:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: rocksdb: DB pointer 0x55c7047e2000 2024-05-08T21:12:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T21:12:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T21:12:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout: ** DB Stats ** 2024-05-08T21:12:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T21:12:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T21:12:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T21:12:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T21:12:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T21:12:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T21:12:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: ** Compaction Stats [default] ** 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: ** Compaction Stats [default] ** 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T21:12:27.606 INFO:journalctl@ceph.mon.c.smithi082.stdout: 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: Cumulative compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: Interval compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: Block cache BinnedLRUCache@0x55c7047571f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T21:12:27.607 INFO:journalctl@ceph.mon.c.smithi082.stdout: 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: using public_addr v2:172.21.15.82:0/0 -> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: starting mon.c rank -1 at public addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] at bind addrs [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(???) e0 preinit fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).mds e1 new map 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout: e1 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout: legacy client fscid: -1 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout: 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout: No filesystems configured 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T21:12:27.608 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi071:172.21.15.71=b;smithi082:172.21.15.82=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi071:172.21.15.71=b;smithi082:172.21.15.82=c;count:3 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: Deploying daemon mon.c on smithi082 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:27.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: from='client.? 172.21.15.82:0/2216058062' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T21:12:27.610 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:27 smithi082 ceph-mon[47512]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T21:12:28.013 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T21:12:28.013 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T21:12:28.238 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:12:32.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.405+0000 7f57f80db700 1 -- 172.21.15.82:0/1512506720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57dc0035c0 msgr2=0x7f57dc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:32.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.405+0000 7f57f80db700 1 --2- 172.21.15.82:0/1512506720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57dc0035c0 0x7f57dc005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f57f0068630 tx=0x7f57e0013e90 comp rx=0 tx=0).stop 2024-05-08T21:12:32.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.406+0000 7f57f80db700 1 -- 172.21.15.82:0/1512506720 shutdown_connections 2024-05-08T21:12:32.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.406+0000 7f57f80db700 1 --2- 172.21.15.82:0/1512506720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57dc0035c0 0x7f57dc005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:32.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.406+0000 7f57f80db700 1 -- 172.21.15.82:0/1512506720 >> 172.21.15.82:0/1512506720 conn(0x7f57f0076570 msgr2=0x7f57f00fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:32.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.406+0000 7f57f80db700 1 -- 172.21.15.82:0/1512506720 shutdown_connections 2024-05-08T21:12:32.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.406+0000 7f57f80db700 1 -- 172.21.15.82:0/1512506720 wait complete. 2024-05-08T21:12:32.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f80db700 1 Processor -- start 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f80db700 1 -- start start 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f80db700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc0035c0 0x7f57f00ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f80db700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57f00ffdc0 0x7f57f0100240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f80db700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f0101740 con 0x7f57f00ffdc0 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f80db700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f01a22d0 con 0x7f57dc0035c0 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f5e77700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc0035c0 0x7f57f00ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f5e77700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc0035c0 0x7f57f00ff880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57754/0 (socket says 172.21.15.82:57754) 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f5e77700 1 -- 172.21.15.82:0/3637375739 learned_addr learned my addr 172.21.15.82:0/3637375739 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f5e77700 1 -- 172.21.15.82:0/3637375739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc0035c0 msgr2=0x7f57f00ff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f5e77700 1 -- 172.21.15.82:0/3637375739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc0035c0 msgr2=0x7f57f00ff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f5e77700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc0035c0 0x7f57f00ff880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f5e77700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc0035c0 0x7f57f00ff880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.407+0000 7f57f5676700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57f00ffdc0 0x7f57f0100240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.408+0000 7f57f5676700 1 -- 172.21.15.82:0/3637375739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc0035c0 msgr2=0x7f57f00ff880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.408+0000 7f57f5676700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc0035c0 0x7f57f00ff880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.408+0000 7f57f5676700 1 -- 172.21.15.82:0/3637375739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e00138e0 con 0x7f57f00ffdc0 2024-05-08T21:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.408+0000 7f57f5676700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57f00ffdc0 0x7f57f0100240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00b4d0 tx=0x7f57ec00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.408+0000 7f57e6ffd700 1 -- 172.21.15.82:0/3637375739 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f57ec011630 con 0x7f57f00ffdc0 2024-05-08T21:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.408+0000 7f57f80db700 1 -- 172.21.15.82:0/3637375739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f01a2470 con 0x7f57f00ffdc0 2024-05-08T21:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.408+0000 7f57e6ffd700 1 -- 172.21.15.82:0/3637375739 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57ec011c20 con 0x7f57f00ffdc0 2024-05-08T21:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.408+0000 7f57e6ffd700 1 -- 172.21.15.82:0/3637375739 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f57ec010c60 con 0x7f57f00ffdc0 2024-05-08T21:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.409+0000 7f57f80db700 1 -- 172.21.15.82:0/3637375739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f01a2940 con 0x7f57f00ffdc0 2024-05-08T21:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.409+0000 7f57e6ffd700 1 -- 172.21.15.82:0/3637375739 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f57ec01a400 con 0x7f57f00ffdc0 2024-05-08T21:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.410+0000 7f57f80db700 1 -- 172.21.15.82:0/3637375739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f0068e30 con 0x7f57f00ffdc0 2024-05-08T21:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.410+0000 7f57e6ffd700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f57d40385e0 0x7f57d403aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.410+0000 7f57e6ffd700 1 -- 172.21.15.82:0/3637375739 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f57ec04ada0 con 0x7f57f00ffdc0 2024-05-08T21:12:32.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.410+0000 7f57f5e77700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f57d40385e0 0x7f57d403aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:32.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.411+0000 7f57f5e77700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f57d40385e0 0x7f57d403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57e000f470 tx=0x7f57e000f3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.416+0000 7f57e6ffd700 1 -- 172.21.15.82:0/3637375739 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f57ec017680 con 0x7f57f00ffdc0 2024-05-08T21:12:32.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:12:32.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: mon.a calling monitor election 2024-05-08T21:12:32.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:32.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:32.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:32.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:32.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: mon.c calling monitor election 2024-05-08T21:12:32.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:32.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:32.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:32.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:32.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T21:12:32.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: monmap e2: 2 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:12:32.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: fsmap 2024-05-08T21:12:32.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:12:32.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: mgrmap e13: x(active, since 36s) 2024-05-08T21:12:32.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: overall HEALTH_OK 2024-05-08T21:12:32.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:32.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:32.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:32.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.691+0000 7f57f80db700 1 -- 172.21.15.82:0/3637375739 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f57f00625d0 con 0x7f57f00ffdc0 2024-05-08T21:12:32.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.692+0000 7f57e6ffd700 1 -- 172.21.15.82:0/3637375739 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1002 (secure 0 0 0) 0x7f57ec0172c0 con 0x7f57f00ffdc0 2024-05-08T21:12:32.695 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:12:32.695 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":2,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","modified":"2024-05-08T21:12:27.197176Z","created":"2024-05-08T21:11:04.684022Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T21:12:32.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.694+0000 7f57f80db700 1 -- 172.21.15.82:0/3637375739 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f57d40385e0 msgr2=0x7f57d403aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:32.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.694+0000 7f57f80db700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f57d40385e0 0x7f57d403aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57e000f470 tx=0x7f57e000f3c0 comp rx=0 tx=0).stop 2024-05-08T21:12:32.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.694+0000 7f57f80db700 1 -- 172.21.15.82:0/3637375739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57f00ffdc0 msgr2=0x7f57f0100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:32.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.694+0000 7f57f80db700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57f00ffdc0 0x7f57f0100240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00b4d0 tx=0x7f57ec00b890 comp rx=0 tx=0).stop 2024-05-08T21:12:32.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.694+0000 7f57f80db700 1 -- 172.21.15.82:0/3637375739 shutdown_connections 2024-05-08T21:12:32.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.694+0000 7f57f80db700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f57d40385e0 0x7f57d403aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:32.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.694+0000 7f57f80db700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57dc0035c0 0x7f57f00ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:32.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.694+0000 7f57f80db700 1 --2- 172.21.15.82:0/3637375739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57f00ffdc0 0x7f57f0100240 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:32.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.694+0000 7f57f80db700 1 -- 172.21.15.82:0/3637375739 >> 172.21.15.82:0/3637375739 conn(0x7f57f0076570 msgr2=0x7f57f00fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:32.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.694+0000 7f57f80db700 1 -- 172.21.15.82:0/3637375739 shutdown_connections 2024-05-08T21:12:32.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:32.694+0000 7f57f80db700 1 -- 172.21.15.82:0/3637375739 wait complete. 2024-05-08T21:12:32.698 INFO:teuthology.orchestra.run.smithi082.stderr:dumped monmap epoch 2 2024-05-08T21:12:32.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:12:32.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: mon.a calling monitor election 2024-05-08T21:12:32.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:32.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:32.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:32.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: mon.c calling monitor election 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: monmap e2: 2 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: fsmap 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: mgrmap e13: x(active, since 36s) 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: overall HEALTH_OK 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:32.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:33.307 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:33 smithi082 ceph-mon[47512]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:33.307 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:33 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:12:33.308 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:33 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:33.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:33 smithi082 ceph-mon[47512]: Deploying daemon mon.b on smithi071 2024-05-08T21:12:33.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:33 smithi082 ceph-mon[47512]: from='client.? 172.21.15.82:0/3637375739' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T21:12:33.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:33 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:33.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:33 smithi033 ceph-mon[42528]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:33.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:33 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:12:33.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:33 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:33.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:33 smithi033 ceph-mon[42528]: Deploying daemon mon.b on smithi071 2024-05-08T21:12:33.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:33 smithi033 ceph-mon[42528]: from='client.? 172.21.15.82:0/3637375739' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T21:12:33.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:33 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:34.060 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T21:12:34.060 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T21:12:34.290 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:12:35.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 systemd[1]: Started Ceph mon.b for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:12:35.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T21:12:35.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T21:12:35.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T21:12:35.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T21:12:35.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T21:12:35.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T21:12:35.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T21:12:35.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T21:12:35.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T21:12:35.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T21:12:35.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.table_properties_collectors: 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.inplace_update_support: 0 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T21:12:35.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.bloom_locality: 0 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.max_successive_merges: 0 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.ttl: 2592000 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.enable_blob_files: false 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.min_blob_size: 0 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T21:12:35.326 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8d5e7511-6de0-4c79-85d0-94c50fa002b0 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715202754845023, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715202754845479, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715202754, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8d5e7511-6de0-4c79-85d0-94c50fa002b0", "db_session_id": "P1O78U065G0RNHEMT1JW", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715202754845547, "job": 1, "event": "recovery_finished"} 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T21:12:35.327 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562250108000 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: DB pointer 0x5622500f6000 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout: ** DB Stats ** 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-05-08T21:12:35.328 INFO:journalctl@ceph.mon.b.smithi071.stdout: ** Compaction Stats [default] ** 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: ** Compaction Stats [default] ** 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-05-08T21:12:35.329 INFO:journalctl@ceph.mon.b.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout: Block cache BinnedLRUCache@0x56225006b1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T21:12:35.330 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: using public_addr v2:172.21.15.71:0/0 -> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: starting mon.b rank -1 at public addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] at bind addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(???) e0 preinit fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).mds e1 new map 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout: e1 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout: legacy client fscid: -1 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout: 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout: No filesystems configured 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T21:12:35.331 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi071:172.21.15.71=b;smithi082:172.21.15.82=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi071:172.21.15.71=b;smithi082:172.21.15.82=c;count:3 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: Deploying daemon mon.c on smithi082 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='client.? 172.21.15.82:0/2216058062' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T21:12:35.332 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.a calling monitor election 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.c calling monitor election 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: monmap e2: 2 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: fsmap 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:12:35.333 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mgrmap e13: x(active, since 36s) 2024-05-08T21:12:35.334 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: overall HEALTH_OK 2024-05-08T21:12:35.334 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:35.334 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:35.334 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:35.334 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:35.334 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:12:35.334 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:35.334 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: Deploying daemon mon.b on smithi071 2024-05-08T21:12:35.334 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='client.? 172.21.15.82:0/3637375739' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T21:12:35.334 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:35.334 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:34 smithi071 ceph-mon[47074]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T21:12:40.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:12:40.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: mon.c calling monitor election 2024-05-08T21:12:40.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: mon.a calling monitor election 2024-05-08T21:12:40.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:40.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:40.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:40.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:40.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:40.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:40.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:40.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:40.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:40.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T21:12:40.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: monmap e3: 3 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],b=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:12:40.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: fsmap 2024-05-08T21:12:40.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:12:40.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: mgrmap e13: x(active, since 43s) 2024-05-08T21:12:40.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T21:12:40.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T21:12:40.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T21:12:40.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: mon.b (rank 2) addr [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] is down (out of quorum) 2024-05-08T21:12:40.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:40.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:40.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:40.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:40.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:12:40.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: mon.c calling monitor election 2024-05-08T21:12:40.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: mon.a calling monitor election 2024-05-08T21:12:40.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:40.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:40.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:40.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:40.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:40.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:40.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:40.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: monmap e3: 3 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],b=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: fsmap 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: mgrmap e13: x(active, since 43s) 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: mon.b (rank 2) addr [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] is down (out of quorum) 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:40.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:41.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.233+0000 7feec1cba700 1 -- 172.21.15.82:0/2957426261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea4005c90 msgr2=0x7feea4008120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:41.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.233+0000 7feec1cba700 1 --2- 172.21.15.82:0/2957426261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea4005c90 0x7feea4008120 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feebc100d50 tx=0x7feeac00ff20 comp rx=0 tx=0).stop 2024-05-08T21:12:41.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.233+0000 7feec1cba700 1 -- 172.21.15.82:0/2957426261 shutdown_connections 2024-05-08T21:12:41.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.233+0000 7feec1cba700 1 --2- 172.21.15.82:0/2957426261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea4005c90 0x7feea4008120 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:41.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.233+0000 7feec1cba700 1 -- 172.21.15.82:0/2957426261 >> 172.21.15.82:0/2957426261 conn(0x7feebc076570 msgr2=0x7feebc0fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:41.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.233+0000 7feec1cba700 1 -- 172.21.15.82:0/2957426261 shutdown_connections 2024-05-08T21:12:41.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.233+0000 7feec1cba700 1 -- 172.21.15.82:0/2957426261 wait complete. 2024-05-08T21:12:41.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.234+0000 7feec1cba700 1 Processor -- start 2024-05-08T21:12:41.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feec1cba700 1 -- start start 2024-05-08T21:12:41.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feec1cba700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feea4005c90 0x7feebc197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:41.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feec1cba700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feebc197640 0x7feebc197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:41.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feec1cba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feebc1a0170 0x7feebc1a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:41.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feec1cba700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc1a2c00 con 0x7feebc1a0170 2024-05-08T21:12:41.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feec1cba700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc1a2d70 con 0x7feea4005c90 2024-05-08T21:12:41.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feec1cba700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc1a2f30 con 0x7feebc197640 2024-05-08T21:12:41.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feebc197640 0x7feebc197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:41.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feebc197640 0x7feebc197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57766/0 (socket says 172.21.15.82:57766) 2024-05-08T21:12:41.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebaffd700 1 -- 172.21.15.82:0/2743035626 learned_addr learned my addr 172.21.15.82:0/2743035626 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-08T21:12:41.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebb7fe700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feea4005c90 0x7feebc197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:41.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebaffd700 1 -- 172.21.15.82:0/2743035626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feebc197640 msgr2=0x7feebc197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T21:12:41.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebaffd700 1 -- 172.21.15.82:0/2743035626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feebc197640 msgr2=0x7feebc197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T21:12:41.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebaffd700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feebc197640 0x7feebc197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T21:12:41.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebb7fe700 1 -- 172.21.15.82:0/2743035626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feea4005c90 msgr2=0x7feebc197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-08T21:12:41.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebaffd700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feebc197640 0x7feebc197ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:12:41.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebb7fe700 1 -- 172.21.15.82:0/2743035626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feea4005c90 msgr2=0x7feebc197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebb7fe700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feea4005c90 0x7feebc197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebb7fe700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feea4005c90 0x7feebc197100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.235+0000 7feebbfff700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feebc1a0170 0x7feebc1a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.236+0000 7feebbfff700 1 -- 172.21.15.82:0/2743035626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feebc197640 msgr2=0x7feebc197ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.236+0000 7feebbfff700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feebc197640 0x7feebc197ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.236+0000 7feebbfff700 1 -- 172.21.15.82:0/2743035626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feea4005c90 msgr2=0x7feebc197100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.236+0000 7feebbfff700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feea4005c90 0x7feebc197100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.236+0000 7feebbfff700 1 -- 172.21.15.82:0/2743035626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac00f970 con 0x7feebc1a0170 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.236+0000 7feebbfff700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feebc1a0170 0x7feebc1a2560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feeb400bd50 tx=0x7feeb400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.236+0000 7feeb8ff9700 1 -- 172.21.15.82:0/2743035626 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb400cd10 con 0x7feebc1a0170 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.236+0000 7feeb8ff9700 1 -- 172.21.15.82:0/2743035626 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feeb400ce70 con 0x7feebc1a0170 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.236+0000 7feec1cba700 1 -- 172.21.15.82:0/2743035626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc1a3210 con 0x7feebc1a0170 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.236+0000 7feeb8ff9700 1 -- 172.21.15.82:0/2743035626 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4012710 con 0x7feebc1a0170 2024-05-08T21:12:41.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.236+0000 7feec1cba700 1 -- 172.21.15.82:0/2743035626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc1a3710 con 0x7feebc1a0170 2024-05-08T21:12:41.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.237+0000 7feeb8ff9700 1 -- 172.21.15.82:0/2743035626 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7feeb40128f0 con 0x7feebc1a0170 2024-05-08T21:12:41.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.237+0000 7feec1cba700 1 -- 172.21.15.82:0/2743035626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc101000 con 0x7feebc1a0170 2024-05-08T21:12:41.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.237+0000 7feeb8ff9700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feea8038c70 0x7feea803b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.237+0000 7feeb8ff9700 1 -- 172.21.15.82:0/2743035626 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feeb404acf0 con 0x7feebc1a0170 2024-05-08T21:12:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.237+0000 7feebb7fe700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feea8038c70 0x7feea803b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:41.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.238+0000 7feebb7fe700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feea8038c70 0x7feea803b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feeac00f550 tx=0x7feeac00f4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:41.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.244+0000 7feeb8ff9700 1 -- 172.21.15.82:0/2743035626 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feeb401d3b0 con 0x7feebc1a0170 2024-05-08T21:12:41.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:40 smithi082 ceph-mon[47512]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:41.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:41.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:41.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:41.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:41.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:41.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:41.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:41.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:41.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:40 smithi033 ceph-mon[42528]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:41.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:41.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:41.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:41.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:41.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:41.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:41.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:41.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:41.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.538+0000 7feec1cba700 1 -- 172.21.15.82:0/2743035626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7feebc1074e0 con 0x7feebc1a0170 2024-05-08T21:12:41.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.539+0000 7feeb8ff9700 1 -- 172.21.15.82:0/2743035626 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1266 (secure 0 0 0) 0x7feeb40112a0 con 0x7feebc1a0170 2024-05-08T21:12:41.543 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-05-08T21:12:41.543 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":3,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","modified":"2024-05-08T21:12:34.873180Z","created":"2024-05-08T21:11:04.684022Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T21:12:41.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.541+0000 7feec1cba700 1 -- 172.21.15.82:0/2743035626 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feea8038c70 msgr2=0x7feea803b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.541+0000 7feec1cba700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feea8038c70 0x7feea803b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feeac00f550 tx=0x7feeac00f4a0 comp rx=0 tx=0).stop 2024-05-08T21:12:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.541+0000 7feec1cba700 1 -- 172.21.15.82:0/2743035626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feebc1a0170 msgr2=0x7feebc1a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.541+0000 7feec1cba700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feebc1a0170 0x7feebc1a2560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feeb400bd50 tx=0x7feeb400be30 comp rx=0 tx=0).stop 2024-05-08T21:12:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.541+0000 7feec1cba700 1 -- 172.21.15.82:0/2743035626 shutdown_connections 2024-05-08T21:12:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.541+0000 7feec1cba700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feebc1a0170 0x7feebc1a2560 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.541+0000 7feec1cba700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feea8038c70 0x7feea803b130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.541+0000 7feec1cba700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feea4005c90 0x7feebc197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.541+0000 7feec1cba700 1 --2- 172.21.15.82:0/2743035626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feebc197640 0x7feebc197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.541+0000 7feec1cba700 1 -- 172.21.15.82:0/2743035626 >> 172.21.15.82:0/2743035626 conn(0x7feebc076570 msgr2=0x7feebc1013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.541+0000 7feec1cba700 1 -- 172.21.15.82:0/2743035626 shutdown_connections 2024-05-08T21:12:41.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:41.541+0000 7feec1cba700 1 -- 172.21.15.82:0/2743035626 wait complete. 2024-05-08T21:12:41.547 INFO:teuthology.orchestra.run.smithi082.stderr:dumped monmap epoch 3 2024-05-08T21:12:41.907 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-08T21:12:41.907 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-08T21:12:42.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: mon.c calling monitor election 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: mon.a calling monitor election 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T21:12:42.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: monmap e3: 3 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],b=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: fsmap 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: mgrmap e13: x(active, since 43s) 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: mon.b (rank 2) addr [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] is down (out of quorum) 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:42.324 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:42.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:42.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:42.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:42.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:42.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:42.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:42.325 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:42.404 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:12:42.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: mon.b calling monitor election 2024-05-08T21:12:42.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: Reconfiguring daemon mon.a on smithi033 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: mon.b calling monitor election 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: mon.c calling monitor election 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: mon.a calling monitor election 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: monmap e3: 3 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],b=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: fsmap 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: mgrmap e13: x(active, since 45s) 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: Cluster is now healthy 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: overall HEALTH_OK 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:42.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:42.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:42.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:12:42.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T21:12:42.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:43.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: mon.b calling monitor election 2024-05-08T21:12:43.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: Reconfiguring daemon mon.a on smithi033 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: mon.b calling monitor election 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: mon.c calling monitor election 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: mon.a calling monitor election 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: monmap e3: 3 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],b=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: fsmap 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: mgrmap e13: x(active, since 45s) 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: Cluster is now healthy 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: overall HEALTH_OK 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:43.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:43.073 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:43.073 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:12:43.073 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T21:12:43.073 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: mon.b calling monitor election 2024-05-08T21:12:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T21:12:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: Reconfiguring daemon mon.a on smithi033 2024-05-08T21:12:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: mon.b calling monitor election 2024-05-08T21:12:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: mon.c calling monitor election 2024-05-08T21:12:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: mon.a calling monitor election 2024-05-08T21:12:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T21:12:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: monmap e3: 3 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],b=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-05-08T21:12:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: fsmap 2024-05-08T21:12:43.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T21:12:43.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: mgrmap e13: x(active, since 45s) 2024-05-08T21:12:43.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-05-08T21:12:43.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: Cluster is now healthy 2024-05-08T21:12:43.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: overall HEALTH_OK 2024-05-08T21:12:43.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:43.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:43.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:43.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:12:43.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T21:12:43.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:43.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.571+0000 7f8304d3c700 1 -- 172.21.15.33:0/2472216422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8300068950 msgr2=0x7f8300068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:43.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.571+0000 7f8304d3c700 1 --2- 172.21.15.33:0/2472216422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8300068950 0x7f8300068db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009990 tx=0x7f82f4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:12:43.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.572+0000 7f8304d3c700 1 -- 172.21.15.33:0/2472216422 shutdown_connections 2024-05-08T21:12:43.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.572+0000 7f8304d3c700 1 --2- 172.21.15.33:0/2472216422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83000692f0 0x7f830010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:43.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.572+0000 7f8304d3c700 1 --2- 172.21.15.33:0/2472216422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8300107020 0x7f8300107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:43.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.572+0000 7f8304d3c700 1 --2- 172.21.15.33:0/2472216422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8300068950 0x7f8300068db0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:43.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.572+0000 7f8304d3c700 1 -- 172.21.15.33:0/2472216422 >> 172.21.15.33:0/2472216422 conn(0x7f8300075f80 msgr2=0x7f8300076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.572+0000 7f8304d3c700 1 -- 172.21.15.33:0/2472216422 shutdown_connections 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.572+0000 7f8304d3c700 1 -- 172.21.15.33:0/2472216422 wait complete. 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.572+0000 7f8304d3c700 1 Processor -- start 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f8304d3c700 1 -- start start 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f8304d3c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8300068950 0x7f83001972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f8304d3c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83000692f0 0x7f8300197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f8304d3c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8300107020 0x7f830019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f8304d3c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830019c150 con 0x7f8300107020 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f8304d3c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830019c2c0 con 0x7f8300068950 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f8304d3c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830019c480 con 0x7f83000692f0 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f82fe59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8300068950 0x7f83001972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f82fed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8300107020 0x7f830019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f82fe59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8300068950 0x7f83001972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:53604/0 (socket says 172.21.15.33:53604) 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f82fe59c700 1 -- 172.21.15.33:0/3751830883 learned_addr learned my addr 172.21.15.33:0/3751830883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:43.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f82fdd9b700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83000692f0 0x7f8300197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f82fe59c700 1 -- 172.21.15.33:0/3751830883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83000692f0 msgr2=0x7f8300197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f82fe59c700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83000692f0 0x7f8300197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f82fe59c700 1 -- 172.21.15.33:0/3751830883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8300107020 msgr2=0x7f830019bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f82fe59c700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8300107020 0x7f830019bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f82fe59c700 1 -- 172.21.15.33:0/3751830883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40095f0 con 0x7f8300068950 2024-05-08T21:12:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.573+0000 7f82fed9d700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8300107020 0x7f830019bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:12:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.574+0000 7f82fe59c700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8300068950 0x7f83001972c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f82f000d980 tx=0x7f82f000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.574+0000 7f82ef7fe700 1 -- 172.21.15.33:0/3751830883 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0004220 con 0x7f8300068950 2024-05-08T21:12:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.574+0000 7f82ef7fe700 1 -- 172.21.15.33:0/3751830883 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82f0007640 con 0x7f8300068950 2024-05-08T21:12:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.574+0000 7f82ef7fe700 1 -- 172.21.15.33:0/3751830883 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0005660 con 0x7f8300068950 2024-05-08T21:12:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.574+0000 7f8304d3c700 1 -- 172.21.15.33:0/3751830883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830019c760 con 0x7f8300068950 2024-05-08T21:12:43.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.574+0000 7f8304d3c700 1 -- 172.21.15.33:0/3751830883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83001a3fc0 con 0x7f8300068950 2024-05-08T21:12:43.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.575+0000 7f82ef7fe700 1 -- 172.21.15.33:0/3751830883 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f82f000f690 con 0x7f8300068950 2024-05-08T21:12:43.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.575+0000 7f82ef7fe700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f82e80389d0 0x7f82e803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:43.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.575+0000 7f82ef7fe700 1 -- 172.21.15.33:0/3751830883 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f82f004b060 con 0x7f8300068950 2024-05-08T21:12:43.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.575+0000 7f8304d3c700 1 -- 172.21.15.33:0/3751830883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8300109540 con 0x7f8300068950 2024-05-08T21:12:43.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.575+0000 7f82fdd9b700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f82e80389d0 0x7f82e803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:43.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.575+0000 7f82fdd9b700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f82e80389d0 0x7f82e803ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8300076730 tx=0x7f82f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:43.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.580+0000 7f82ef7fe700 1 -- 172.21.15.33:0/3751830883 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f82f0016070 con 0x7f8300068950 2024-05-08T21:12:43.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.796+0000 7f8304d3c700 1 -- 172.21.15.33:0/3751830883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f83001985d0 con 0x7f8300068950 2024-05-08T21:12:43.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.797+0000 7f82ef7fe700 1 -- 172.21.15.33:0/3751830883 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+271 (secure 0 0 0) 0x7f82f003e090 con 0x7f8300068950 2024-05-08T21:12:43.801 INFO:teuthology.orchestra.run.smithi033.stdout:# minimal ceph.conf for 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:12:43.801 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-05-08T21:12:43.801 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:12:43.801 INFO:teuthology.orchestra.run.smithi033.stdout: mon_host = [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] 2024-05-08T21:12:43.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.799+0000 7f8304d3c700 1 -- 172.21.15.33:0/3751830883 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f82e80389d0 msgr2=0x7f82e803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:43.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.799+0000 7f8304d3c700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f82e80389d0 0x7f82e803ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8300076730 tx=0x7f82f4005fb0 comp rx=0 tx=0).stop 2024-05-08T21:12:43.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.799+0000 7f8304d3c700 1 -- 172.21.15.33:0/3751830883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8300068950 msgr2=0x7f83001972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:43.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.799+0000 7f8304d3c700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8300068950 0x7f83001972c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f82f000d980 tx=0x7f82f000dc90 comp rx=0 tx=0).stop 2024-05-08T21:12:43.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.799+0000 7f8304d3c700 1 -- 172.21.15.33:0/3751830883 shutdown_connections 2024-05-08T21:12:43.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.799+0000 7f8304d3c700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8300107020 0x7f830019bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:43.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.799+0000 7f8304d3c700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f82e80389d0 0x7f82e803ae90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:43.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.799+0000 7f8304d3c700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8300068950 0x7f83001972c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:43.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.799+0000 7f8304d3c700 1 --2- 172.21.15.33:0/3751830883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83000692f0 0x7f8300197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:43.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.799+0000 7f8304d3c700 1 -- 172.21.15.33:0/3751830883 >> 172.21.15.33:0/3751830883 conn(0x7f8300075f80 msgr2=0x7f83000fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:43.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.799+0000 7f8304d3c700 1 -- 172.21.15.33:0/3751830883 shutdown_connections 2024-05-08T21:12:43.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:43.799+0000 7f8304d3c700 1 -- 172.21.15.33:0/3751830883 wait complete. 2024-05-08T21:12:44.016 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:43 smithi082 ceph-mon[47512]: Reconfiguring mon.b (monmap changed)... 2024-05-08T21:12:44.016 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:43 smithi082 ceph-mon[47512]: Reconfiguring daemon mon.b on smithi071 2024-05-08T21:12:44.017 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:44.017 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:44.017 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:44.017 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:12:44.017 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T21:12:44.017 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:44.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:43 smithi071 ceph-mon[47074]: Reconfiguring mon.b (monmap changed)... 2024-05-08T21:12:44.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:43 smithi071 ceph-mon[47074]: Reconfiguring daemon mon.b on smithi071 2024-05-08T21:12:44.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:44.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:44.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:44.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:12:44.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T21:12:44.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:44.098 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:43 smithi033 ceph-mon[42528]: Reconfiguring mon.b (monmap changed)... 2024-05-08T21:12:44.098 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:43 smithi033 ceph-mon[42528]: Reconfiguring daemon mon.b on smithi071 2024-05-08T21:12:44.098 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:12:44.098 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:44.098 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:44.099 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T21:12:44.099 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T21:12:44.099 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:44.190 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-08T21:12:44.190 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:12:44.190 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T21:12:44.228 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:12:44.228 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:12:44.303 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-08T21:12:44.304 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T21:12:44.337 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-08T21:12:44.338 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:12:44.407 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-08T21:12:44.407 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T21:12:44.438 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-08T21:12:44.438 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T21:12:44.506 INFO:tasks.cephadm:Adding mgr.x on smithi033 2024-05-08T21:12:44.506 INFO:tasks.cephadm:Adding mgr.y on smithi071 2024-05-08T21:12:44.506 INFO:tasks.cephadm:Adding mgr.z on smithi082 2024-05-08T21:12:44.507 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi033=x;smithi071=y;smithi082=z' 2024-05-08T21:12:44.757 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:12:44.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:44 smithi082 ceph-mon[47512]: Reconfiguring mon.c (monmap changed)... 2024-05-08T21:12:44.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:44 smithi082 ceph-mon[47512]: Reconfiguring daemon mon.c on smithi082 2024-05-08T21:12:44.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:44 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3751830883' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:44.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:44 smithi082 ceph-mon[47512]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:44.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:44 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:44.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:44 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:44.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:44 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:44.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:44 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:44.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:44 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:44.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:44 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:45.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:44 smithi071 ceph-mon[47074]: Reconfiguring mon.c (monmap changed)... 2024-05-08T21:12:45.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:44 smithi071 ceph-mon[47074]: Reconfiguring daemon mon.c on smithi082 2024-05-08T21:12:45.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:44 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3751830883' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:45.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:44 smithi071 ceph-mon[47074]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:45.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:44 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:45.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:44 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:45.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:44 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:45.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:44 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:45.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:44 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:45.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:44 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:45.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:44 smithi033 ceph-mon[42528]: Reconfiguring mon.c (monmap changed)... 2024-05-08T21:12:45.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:44 smithi033 ceph-mon[42528]: Reconfiguring daemon mon.c on smithi082 2024-05-08T21:12:45.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:44 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3751830883' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:45.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:44 smithi033 ceph-mon[42528]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:45.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:44 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:45.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:44 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:45.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:44 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:45.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:44 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:45.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:44 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:45.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:44 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:45.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 -- 172.21.15.82:0/1070533156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d8101a20 msgr2=0x7ff4d8108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:45.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 --2- 172.21.15.82:0/1070533156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d8101a20 0x7ff4d8108340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000b0a0 tx=0x7ff4d000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:12:45.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 -- 172.21.15.82:0/1070533156 shutdown_connections 2024-05-08T21:12:45.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 --2- 172.21.15.82:0/1070533156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d8101a20 0x7ff4d8108340 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:45.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 --2- 172.21.15.82:0/1070533156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4d80fff00 0x7ff4d8100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:45.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 --2- 172.21.15.82:0/1070533156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d8101060 0x7ff4d81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:45.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 -- 172.21.15.82:0/1070533156 >> 172.21.15.82:0/1070533156 conn(0x7ff4d80fb4c0 msgr2=0x7ff4d80fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:45.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 -- 172.21.15.82:0/1070533156 shutdown_connections 2024-05-08T21:12:45.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 -- 172.21.15.82:0/1070533156 wait complete. 2024-05-08T21:12:45.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 Processor -- start 2024-05-08T21:12:45.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 -- start start 2024-05-08T21:12:45.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d80fff00 0x7ff4d8069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:45.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4d8101060 0x7ff4d819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:45.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d8101a20 0x7ff4d819e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:45.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d819e8e0 con 0x7ff4d8101060 2024-05-08T21:12:45.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d819ea50 con 0x7ff4d8101a20 2024-05-08T21:12:45.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.693+0000 7ff4dedf4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d819ec10 con 0x7ff4d80fff00 2024-05-08T21:12:45.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4d8101060 0x7ff4d819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:45.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4d8101060 0x7ff4d819bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.82:33476/0 (socket says 172.21.15.82:33476) 2024-05-08T21:12:45.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4d7fff700 1 -- 172.21.15.82:0/472040917 learned_addr learned my addr 172.21.15.82:0/472040917 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-08T21:12:45.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4dd391700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d8101a20 0x7ff4d819e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:45.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4dcb90700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d80fff00 0x7ff4d8069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:45.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4d7fff700 1 -- 172.21.15.82:0/472040917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d80fff00 msgr2=0x7ff4d8069cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:45.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4d7fff700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d80fff00 0x7ff4d8069cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:45.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4d7fff700 1 -- 172.21.15.82:0/472040917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d8101a20 msgr2=0x7ff4d819e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:45.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4d7fff700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d8101a20 0x7ff4d819e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:45.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4d7fff700 1 -- 172.21.15.82:0/472040917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d0009d00 con 0x7ff4d8101060 2024-05-08T21:12:45.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4d7fff700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4d8101060 0x7ff4d819bb70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400b530 tx=0x7ff4c400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:45.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4d5ffb700 1 -- 172.21.15.82:0/472040917 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4011830 con 0x7ff4d8101060 2024-05-08T21:12:45.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4dedf4700 1 -- 172.21.15.82:0/472040917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d819eef0 con 0x7ff4d8101060 2024-05-08T21:12:45.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4d5ffb700 1 -- 172.21.15.82:0/472040917 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4c4011e20 con 0x7ff4d8101060 2024-05-08T21:12:45.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.694+0000 7ff4d5ffb700 1 -- 172.21.15.82:0/472040917 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4010db0 con 0x7ff4d8101060 2024-05-08T21:12:45.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.695+0000 7ff4dedf4700 1 -- 172.21.15.82:0/472040917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d819f430 con 0x7ff4d8101060 2024-05-08T21:12:45.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.695+0000 7ff4d5ffb700 1 -- 172.21.15.82:0/472040917 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7ff4c4011990 con 0x7ff4d8101060 2024-05-08T21:12:45.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.695+0000 7ff4dedf4700 1 -- 172.21.15.82:0/472040917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d8107320 con 0x7ff4d8101060 2024-05-08T21:12:45.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.696+0000 7ff4d5ffb700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4c80389d0 0x7ff4c803ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:45.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.696+0000 7ff4d5ffb700 1 -- 172.21.15.82:0/472040917 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff4c404adc0 con 0x7ff4d8101060 2024-05-08T21:12:45.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.696+0000 7ff4dcb90700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4c80389d0 0x7ff4c803ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.696+0000 7ff4dcb90700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4c80389d0 0x7ff4c803ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009a10 tx=0x7ff4cc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:45.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.702+0000 7ff4d5ffb700 1 -- 172.21.15.82:0/472040917 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff4c4017730 con 0x7ff4d8101060 2024-05-08T21:12:45.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.902+0000 7ff4dedf4700 1 -- 172.21.15.82:0/472040917 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi071=y;smithi082=z", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4d802cf00 con 0x7ff4c80389d0 2024-05-08T21:12:45.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.909+0000 7ff4d5ffb700 1 -- 172.21.15.82:0/472040917 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff4d802cf00 con 0x7ff4c80389d0 2024-05-08T21:12:45.912 INFO:teuthology.orchestra.run.smithi082.stdout:Scheduled mgr update... 2024-05-08T21:12:45.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.911+0000 7ff4dedf4700 1 -- 172.21.15.82:0/472040917 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4c80389d0 msgr2=0x7ff4c803ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:45.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.911+0000 7ff4dedf4700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4c80389d0 0x7ff4c803ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009a10 tx=0x7ff4cc009190 comp rx=0 tx=0).stop 2024-05-08T21:12:45.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.911+0000 7ff4dedf4700 1 -- 172.21.15.82:0/472040917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4d8101060 msgr2=0x7ff4d819bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:45.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.911+0000 7ff4dedf4700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4d8101060 0x7ff4d819bb70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400b530 tx=0x7ff4c400b840 comp rx=0 tx=0).stop 2024-05-08T21:12:45.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.911+0000 7ff4dedf4700 1 -- 172.21.15.82:0/472040917 shutdown_connections 2024-05-08T21:12:45.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.911+0000 7ff4dedf4700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4c80389d0 0x7ff4c803ae90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:45.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.911+0000 7ff4dedf4700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4d8101a20 0x7ff4d819e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:45.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.911+0000 7ff4dedf4700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d80fff00 0x7ff4d8069cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:45.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.911+0000 7ff4dedf4700 1 --2- 172.21.15.82:0/472040917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4d8101060 0x7ff4d819bb70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:45.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.911+0000 7ff4dedf4700 1 -- 172.21.15.82:0/472040917 >> 172.21.15.82:0/472040917 conn(0x7ff4d80fb4c0 msgr2=0x7ff4d8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:45.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.911+0000 7ff4dedf4700 1 -- 172.21.15.82:0/472040917 shutdown_connections 2024-05-08T21:12:45.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:12:45.911+0000 7ff4dedf4700 1 -- 172.21.15.82:0/472040917 wait complete. 2024-05-08T21:12:46.287 DEBUG:teuthology.orchestra.run.smithi071:mgr.y> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.y.service 2024-05-08T21:12:46.290 DEBUG:teuthology.orchestra.run.smithi082:mgr.z> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.z.service 2024-05-08T21:12:46.292 INFO:tasks.cephadm:Deploying OSDs... 2024-05-08T21:12:46.292 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:12:46.292 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T21:12:46.316 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T21:12:46.316 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-05-08T21:12:46.321 INFO:journalctl@ceph.mgr.y.smithi071.stdout:-- Logs begin at Wed 2024-05-08 20:53:45 UTC. -- 2024-05-08T21:12:46.323 INFO:journalctl@ceph.mgr.z.smithi082.stdout:-- Logs begin at Wed 2024-05-08 20:53:53 UTC. -- 2024-05-08T21:12:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T21:12:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:12:46.379 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 180692 Links: 1 2024-05-08T21:12:46.379 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:12:46.379 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:12:46.379 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-08 21:12:05.873549250 +0000 2024-05-08T21:12:46.379 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-08 21:05:08.507923089 +0000 2024-05-08T21:12:46.379 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-08 21:05:08.507923089 +0000 2024-05-08T21:12:46.379 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-08T21:12:46.380 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T21:12:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-08T21:12:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-08T21:12:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000234894 s, 2.2 MB/s 2024-05-08T21:12:46.454 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T21:12:46.518 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-05-08T21:12:46.581 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T21:12:46.581 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:12:46.582 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 184649 Links: 1 2024-05-08T21:12:46.582 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:12:46.582 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:12:46.582 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-08 21:12:05.926548430 +0000 2024-05-08T21:12:46.582 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-08 21:05:08.911916556 +0000 2024-05-08T21:12:46.582 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-08 21:05:08.911916556 +0000 2024-05-08T21:12:46.582 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-08T21:12:46.582 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T21:12:46.655 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-08T21:12:46.656 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-08T21:12:46.656 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000236129 s, 2.2 MB/s 2024-05-08T21:12:46.657 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T21:12:46.720 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-05-08T21:12:46.783 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T21:12:46.784 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:12:46.784 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 181784 Links: 1 2024-05-08T21:12:46.784 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:12:46.784 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:12:46.784 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-08 21:12:05.978547625 +0000 2024-05-08T21:12:46.785 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-08 21:05:09.275910669 +0000 2024-05-08T21:12:46.785 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-08 21:05:09.275910669 +0000 2024-05-08T21:12:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-08T21:12:46.785 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T21:12:46.857 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-08T21:12:46.857 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-08T21:12:46.857 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000223185 s, 2.3 MB/s 2024-05-08T21:12:46.858 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T21:12:46.921 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-05-08T21:12:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T21:12:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:12:46.985 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 6h/6d Inode: 175951 Links: 1 2024-05-08T21:12:46.985 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:12:46.985 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:12:46.985 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-08 21:12:06.034546758 +0000 2024-05-08T21:12:46.985 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-08 21:05:09.691903941 +0000 2024-05-08T21:12:46.985 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-08 21:05:09.691903941 +0000 2024-05-08T21:12:46.985 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-08T21:12:46.985 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T21:12:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-05-08T21:12:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-05-08T21:12:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.0002443 s, 2.1 MB/s 2024-05-08T21:12:47.060 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T21:12:47.124 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-05-08T21:12:47.124 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T21:12:47.145 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T21:12:47.145 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-05-08T21:12:47.192 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi071=y;smithi082=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:47.192 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: Saving service mgr spec with placement smithi033=x;smithi071=y;smithi082=z;count:3 2024-05-08T21:12:47.192 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:47.192 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:47.192 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:47.192 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:47.192 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:47.192 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:12:47.192 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T21:12:47.192 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:12:47.192 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:47.193 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: Deploying daemon mgr.z on smithi082 2024-05-08T21:12:47.193 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:46 smithi082 ceph-mon[47512]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:47.206 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T21:12:47.206 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:12:47.207 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 180807 Links: 1 2024-05-08T21:12:47.207 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:12:47.207 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:12:47.207 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-05-08 21:12:34.456247559 +0000 2024-05-08T21:12:47.207 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-05-08 21:04:26.524373290 +0000 2024-05-08T21:12:47.207 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-05-08 21:04:26.524373290 +0000 2024-05-08T21:12:47.207 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-05-08T21:12:47.207 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T21:12:47.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi071=y;smithi082=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:47.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: Saving service mgr spec with placement smithi033=x;smithi071=y;smithi082=z;count:3 2024-05-08T21:12:47.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:47.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:47.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:47.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:47.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:47.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:12:47.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T21:12:47.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:12:47.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:47.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: Deploying daemon mgr.z on smithi082 2024-05-08T21:12:47.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:46 smithi033 ceph-mon[42528]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:47.275 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi071=y;smithi082=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:47.275 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: Saving service mgr spec with placement smithi033=x;smithi071=y;smithi082=z;count:3 2024-05-08T21:12:47.275 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:47.275 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:47.275 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:47.275 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:47.275 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:47.275 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:12:47.275 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T21:12:47.275 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:12:47.275 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:47.276 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: Deploying daemon mgr.z on smithi082 2024-05-08T21:12:47.276 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:46 smithi071 ceph-mon[47074]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:47.277 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-05-08T21:12:47.277 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-05-08T21:12:47.277 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000271909 s, 1.9 MB/s 2024-05-08T21:12:47.279 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T21:12:47.339 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-05-08T21:12:47.398 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T21:12:47.399 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:12:47.399 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 180896 Links: 1 2024-05-08T21:12:47.399 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:12:47.399 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:12:47.399 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-05-08 21:12:34.456247559 +0000 2024-05-08T21:12:47.400 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-05-08 21:04:26.876367097 +0000 2024-05-08T21:12:47.400 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-05-08 21:04:26.876367097 +0000 2024-05-08T21:12:47.400 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-05-08T21:12:47.400 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T21:12:47.468 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-05-08T21:12:47.468 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-05-08T21:12:47.468 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000245386 s, 2.1 MB/s 2024-05-08T21:12:47.469 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T21:12:47.510 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:47 smithi082 systemd[1]: Starting Ceph mgr.z for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:12:47.530 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-05-08T21:12:47.589 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T21:12:47.589 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:12:47.589 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 182370 Links: 1 2024-05-08T21:12:47.589 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:12:47.589 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:12:47.589 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-05-08 21:12:34.456247559 +0000 2024-05-08T21:12:47.590 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-05-08 21:04:27.229360887 +0000 2024-05-08T21:12:47.590 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-05-08 21:04:27.229360887 +0000 2024-05-08T21:12:47.590 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-05-08T21:12:47.590 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T21:12:47.657 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-05-08T21:12:47.657 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-05-08T21:12:47.657 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000201407 s, 2.5 MB/s 2024-05-08T21:12:47.658 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T21:12:47.718 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-05-08T21:12:47.776 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T21:12:47.776 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:12:47.776 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 181728 Links: 1 2024-05-08T21:12:47.776 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:12:47.776 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:12:47.776 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-05-08 21:12:34.456247559 +0000 2024-05-08T21:12:47.777 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-05-08 21:04:27.570354889 +0000 2024-05-08T21:12:47.777 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-05-08 21:04:27.570354889 +0000 2024-05-08T21:12:47.777 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-05-08T21:12:47.777 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T21:12:47.843 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-05-08T21:12:47.843 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-05-08T21:12:47.843 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000230717 s, 2.2 MB/s 2024-05-08T21:12:47.844 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T21:12:47.854 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:47 smithi082 ceph-mgr[49398]: -- 172.21.15.82:0/1735181713 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5641907e21a0 con 0x5641907b9000 2024-05-08T21:12:47.854 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:47 smithi082 bash[49361]: 714d0918feb8af14330c86f22fb472bee5d326c0f69204807aabac2b69d3af05 2024-05-08T21:12:47.854 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:47 smithi082 systemd[1]: Started Ceph mgr.z for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:12:47.903 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-05-08T21:12:47.903 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T21:12:47.921 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T21:12:47.921 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-05-08T21:12:47.980 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T21:12:47.980 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:12:47.980 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 178931 Links: 1 2024-05-08T21:12:47.980 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:12:47.980 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:12:47.980 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-05-08 21:12:47.237885260 +0000 2024-05-08T21:12:47.980 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-05-08 21:04:26.523624353 +0000 2024-05-08T21:12:47.980 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-05-08 21:04:26.523624353 +0000 2024-05-08T21:12:47.980 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2024-05-08T21:12:47.981 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T21:12:48.049 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-05-08T21:12:48.050 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-05-08T21:12:48.050 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000259707 s, 2.0 MB/s 2024-05-08T21:12:48.051 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T21:12:48.109 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-05-08T21:12:48.155 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:48 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:48.055+0000 7f30ef7eb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:12:48.168 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T21:12:48.168 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:12:48.169 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 181906 Links: 1 2024-05-08T21:12:48.169 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:12:48.169 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:12:48.169 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-05-08 21:12:47.237885260 +0000 2024-05-08T21:12:48.169 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-05-08 21:04:26.865617550 +0000 2024-05-08T21:12:48.169 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-05-08 21:04:26.865617550 +0000 2024-05-08T21:12:48.169 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2024-05-08T21:12:48.169 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T21:12:48.235 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-05-08T21:12:48.236 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-05-08T21:12:48.236 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000241685 s, 2.1 MB/s 2024-05-08T21:12:48.237 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T21:12:48.294 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-05-08T21:12:48.334 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:48 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:48.152+0000 7f30ef7eb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:12:48.335 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:48 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:48.244+0000 7f30ef7eb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:12:48.335 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:48 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:48.331+0000 7f30ef7eb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:12:48.351 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T21:12:48.351 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:12:48.351 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 182628 Links: 1 2024-05-08T21:12:48.351 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:12:48.351 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:12:48.352 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-05-08 21:12:47.237885260 +0000 2024-05-08T21:12:48.352 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-05-08 21:04:27.198610925 +0000 2024-05-08T21:12:48.352 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-05-08 21:04:27.198610925 +0000 2024-05-08T21:12:48.352 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2024-05-08T21:12:48.352 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T21:12:48.417 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-05-08T21:12:48.417 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-05-08T21:12:48.418 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000183376 s, 2.8 MB/s 2024-05-08T21:12:48.419 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T21:12:48.476 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-05-08T21:12:48.534 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T21:12:48.534 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T21:12:48.534 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 182717 Links: 1 2024-05-08T21:12:48.534 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:12:48.534 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T21:12:48.535 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-05-08 21:12:47.237885260 +0000 2024-05-08T21:12:48.535 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-05-08 21:04:27.564603644 +0000 2024-05-08T21:12:48.535 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-05-08 21:04:27.564603644 +0000 2024-05-08T21:12:48.535 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2024-05-08T21:12:48.535 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T21:12:48.600 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-05-08T21:12:48.601 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-05-08T21:12:48.601 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000230435 s, 2.2 MB/s 2024-05-08T21:12:48.602 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T21:12:48.659 INFO:tasks.cephadm:Deploying osd.0 on smithi033 with /dev/vg_nvme/lv_4... 2024-05-08T21:12:48.660 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T21:12:48.805 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:48.806 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:48.806 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:48.806 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:12:48.806 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T21:12:48.806 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:12:48.806 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:48.806 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:48 smithi071 ceph-mon[47074]: Deploying daemon mgr.y on smithi071 2024-05-08T21:12:48.806 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:48 smithi071 ceph-mon[47074]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:48.909 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:12:48.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:48.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:48.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:48.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:12:48.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T21:12:48.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:12:48.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:48.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:48 smithi033 ceph-mon[42528]: Deploying daemon mgr.y on smithi071 2024-05-08T21:12:48.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:48 smithi033 ceph-mon[42528]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:48.987 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:48.988 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:48.988 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:48.988 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:12:48.988 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T21:12:48.988 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:12:48.988 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:48.988 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:48 smithi082 ceph-mon[47512]: Deploying daemon mgr.y on smithi071 2024-05-08T21:12:48.988 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:48 smithi082 ceph-mon[47512]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:49.071 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:49 smithi071 systemd[1]: Starting Ceph mgr.y for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:12:49.322 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:48 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:48.984+0000 7f30ef7eb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:12:49.322 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:49 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:49.233+0000 7f30ef7eb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:12:49.322 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:49 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:49.318+0000 7f30ef7eb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:12:49.571 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:49 smithi071 ceph-mgr[48580]: -- 172.21.15.71:0/2255873379 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5603beffe1a0 con 0x5603befd4800 2024-05-08T21:12:49.571 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:49 smithi071 bash[48542]: 31096a25dedca913ca6614c87fb76c2518388f8e8b7d48d4af26c5a5e50f611e 2024-05-08T21:12:49.571 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:49 smithi071 systemd[1]: Started Ceph mgr.y for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:12:49.588 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:49 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:49.427+0000 7f30ef7eb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:12:49.854 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:49 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:49.585+0000 7f30ef7eb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:12:49.854 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:49 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:49.718+0000 7f30ef7eb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:12:49.854 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:49 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:49.812+0000 7f30ef7eb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:12:49.977 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:49 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:49.874+0000 7f4784110200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T21:12:50.171 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:49 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:49.973+0000 7f4784110200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T21:12:50.171 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:50 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:50.075+0000 7f4784110200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T21:12:50.171 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:50 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:50.167+0000 7f4784110200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T21:12:50.386 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T21:12:50.386 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T21:12:50.386 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-08T21:12:50.386 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-08T21:12:50.386 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119681 s, 876 MB/s 2024-05-08T21:12:50.387 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-08T21:12:50.537 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:50 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:50.224+0000 7f30ef7eb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:12:50.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:50.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:50.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:50.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:50.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:50.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:50 smithi033 ceph-mon[42528]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:50.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_4 2024-05-08T21:12:50.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:50.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:50.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:50.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:50.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:50.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:50 smithi071 ceph-mon[47074]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:50.836 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:50 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:50.633+0000 7f30ef7eb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:12:50.836 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:50.836 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:50.836 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:50.836 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:50.836 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:50.836 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:50 smithi082 ceph-mon[47512]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:12:51.104 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:50 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:50.832+0000 7f30ef7eb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:12:51.104 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:50 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:50.930+0000 7f30ef7eb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:12:51.181 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:50 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:50.844+0000 7f4784110200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:12:51.182 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:51 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:51.093+0000 7f4784110200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T21:12:51.447 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:51 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:51.178+0000 7f4784110200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T21:12:51.447 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:51 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:51.288+0000 7f4784110200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T21:12:51.821 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:51 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:51.443+0000 7f4784110200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T21:12:51.821 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:51 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:51.569+0000 7f4784110200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T21:12:51.822 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:51 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:51.654+0000 7f4784110200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T21:12:51.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:51.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:51.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T21:12:51.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:12:51.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:12:51.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:51.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: Reconfiguring daemon mgr.x on smithi033 2024-05-08T21:12:51.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:51 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.854 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:51 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:51.689+0000 7f30ef7eb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:12:51.854 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:51 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:51.785+0000 7f30ef7eb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:12:51.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:51.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:51.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T21:12:51.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:12:51.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:12:51.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:51.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: Reconfiguring daemon mgr.x on smithi033 2024-05-08T21:12:51.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:51 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:51.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:51.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:51.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T21:12:51.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T21:12:51.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T21:12:51.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:51.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: Reconfiguring daemon mgr.x on smithi033 2024-05-08T21:12:51.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:51 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:52.321 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:52 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:52.023+0000 7f4784110200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T21:12:52.353 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:51 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:51.934+0000 7f30ef7eb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:12:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.400+0000 7f480cb05700 1 -- 172.21.15.33:0/3191032672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4808073980 msgr2=0x7f4808073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.400+0000 7f480cb05700 1 --2- 172.21.15.33:0/3191032672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4808073980 0x7f4808073e00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009990 tx=0x7f47fc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:12:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.400+0000 7f480cb05700 1 -- 172.21.15.33:0/3191032672 shutdown_connections 2024-05-08T21:12:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.400+0000 7f480cb05700 1 --2- 172.21.15.33:0/3191032672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4808108f00 0x7f480810b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.400+0000 7f480cb05700 1 --2- 172.21.15.33:0/3191032672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808074f50 0x7f48080733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.400+0000 7f480cb05700 1 --2- 172.21.15.33:0/3191032672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4808073980 0x7f4808073e00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.400+0000 7f480cb05700 1 -- 172.21.15.33:0/3191032672 >> 172.21.15.33:0/3191032672 conn(0x7f48080fc5c0 msgr2=0x7f48080fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:12:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.400+0000 7f480cb05700 1 -- 172.21.15.33:0/3191032672 shutdown_connections 2024-05-08T21:12:52.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.400+0000 7f480cb05700 1 -- 172.21.15.33:0/3191032672 wait complete. 2024-05-08T21:12:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f480cb05700 1 Processor -- start 2024-05-08T21:12:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f480cb05700 1 -- start start 2024-05-08T21:12:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f480cb05700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4808073980 0x7f4808199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f480cb05700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808074f50 0x7f48081995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f480cb05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4808108f00 0x7f48081a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f480cb05700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48081a0ba0 con 0x7f4808108f00 2024-05-08T21:12:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f480cb05700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48081a0ce0 con 0x7f4808074f50 2024-05-08T21:12:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f480cb05700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48081a0e70 con 0x7f4808073980 2024-05-08T21:12:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f480659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4808073980 0x7f4808199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f4806d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4808108f00 0x7f48081a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f4805d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808074f50 0x7f48081995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f480659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4808073980 0x7f4808199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:48606/0 (socket says 172.21.15.33:48606) 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.401+0000 7f480659c700 1 -- 172.21.15.33:0/1491795459 learned_addr learned my addr 172.21.15.33:0/1491795459 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.402+0000 7f480659c700 1 -- 172.21.15.33:0/1491795459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808074f50 msgr2=0x7f48081995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.402+0000 7f480659c700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808074f50 0x7f48081995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.402+0000 7f480659c700 1 -- 172.21.15.33:0/1491795459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4808108f00 msgr2=0x7f48081a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.402+0000 7f480659c700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4808108f00 0x7f48081a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.402+0000 7f480659c700 1 -- 172.21.15.33:0/1491795459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc0095f0 con 0x7f4808073980 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.402+0000 7f4805d9b700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808074f50 0x7f48081995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.402+0000 7f480659c700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4808073980 0x7f4808199060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f47f800e9d0 tx=0x7f47f800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.402+0000 7f47f77fe700 1 -- 172.21.15.33:0/1491795459 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8009790 con 0x7f4808073980 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.402+0000 7f47f77fe700 1 -- 172.21.15.33:0/1491795459 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47f800ee70 con 0x7f4808073980 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.402+0000 7f480cb05700 1 -- 172.21.15.33:0/1491795459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48081a1150 con 0x7f4808073980 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.402+0000 7f47f77fe700 1 -- 172.21.15.33:0/1491795459 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8018420 con 0x7f4808073980 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.402+0000 7f480cb05700 1 -- 172.21.15.33:0/1491795459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48081a1650 con 0x7f4808073980 2024-05-08T21:12:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.403+0000 7f47f77fe700 1 -- 172.21.15.33:0/1491795459 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45597+0+0 (secure 0 0 0) 0x7f47f8018580 con 0x7f4808073980 2024-05-08T21:12:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.403+0000 7f47f77fe700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f47f00389d0 0x7f47f003ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:12:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.403+0000 7f47f77fe700 1 -- 172.21.15.33:0/1491795459 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f47f8014070 con 0x7f4808073980 2024-05-08T21:12:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.403+0000 7f480cb05700 1 -- 172.21.15.33:0/1491795459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4808107d10 con 0x7f4808073980 2024-05-08T21:12:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.403+0000 7f4805d9b700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f47f00389d0 0x7f47f003ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:12:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.404+0000 7f4805d9b700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f47f00389d0 0x7f47f003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f47fc006010 tx=0x7f47fc00b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:12:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.409+0000 7f47f77fe700 1 -- 172.21.15.33:0/1491795459 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f47f801f8e0 con 0x7f4808073980 2024-05-08T21:12:52.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:52.631+0000 7f480cb05700 1 -- 172.21.15.33:0/1491795459 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f48080613e0 con 0x7f47f00389d0 2024-05-08T21:12:52.706 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:52 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:52.417+0000 7f4784110200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T21:12:52.706 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:52 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:52.611+0000 7f4784110200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T21:12:52.706 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:52 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:52.703+0000 7f4784110200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T21:12:52.854 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:52 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:52.563+0000 7f30ef7eb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:12:52.854 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:52 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:52.746+0000 7f30ef7eb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:12:53.354 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:52 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:52.941+0000 7f30ef7eb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:12:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:53 smithi082 ceph-mon[47512]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:53.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:12:53.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:12:53.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:53.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:53 smithi033 ceph-mon[42528]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:53.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:53.410 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:53.410 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:53.410 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:53.410 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:53.410 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:53.410 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:12:53.410 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:12:53.410 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:53.465 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:53 smithi071 ceph-mon[47074]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:53.465 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:53.466 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:53.466 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:12:53.466 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:53.466 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:12:53.466 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:12:53.466 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:12:53.466 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:12:53.466 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:12:53.821 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:53 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:53.462+0000 7f4784110200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T21:12:53.821 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:53 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:53.558+0000 7f4784110200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T21:12:53.821 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:53 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:53.705+0000 7f4784110200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T21:12:54.156 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:53 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:53.870+0000 7f30ef7eb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:12:54.435 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:54 smithi082 ceph-mon[47512]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:54.435 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:54 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:54.155+0000 7f30ef7eb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:12:54.436 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:54 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:54.243+0000 7f30ef7eb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:12:54.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:54 smithi033 ceph-mon[42528]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:54.512 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:54 smithi071 ceph-mon[47074]: from='client.24127 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:12:54.512 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:54 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:54.328+0000 7f4784110200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T21:12:54.822 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:54 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:54.508+0000 7f4784110200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T21:12:54.822 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:54 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:54.702+0000 7f4784110200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T21:12:54.854 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:12:54 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z[49375]: 2024-05-08T21:12:54.432+0000 7f30ef7eb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:12:55.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:55.164+0000 7f47f77fe700 1 -- 172.21.15.33:0/1491795459 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 14) v1 ==== 90707+0+0 (secure 0 0 0) 0x7f47f8017560 con 0x7f4808073980 2024-05-08T21:12:55.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:55 smithi033 ceph-mon[42528]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:55.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:55 smithi033 ceph-mon[42528]: Standby manager daemon z started 2024-05-08T21:12:55.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:55 smithi033 ceph-mon[42528]: from='mgr.? 172.21.15.82:0/1297437111' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T21:12:55.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:55 smithi033 ceph-mon[42528]: from='mgr.? 172.21.15.82:0/1297437111' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T21:12:55.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:55 smithi033 ceph-mon[42528]: from='mgr.? 172.21.15.82:0/1297437111' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T21:12:55.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:55 smithi033 ceph-mon[42528]: from='mgr.? 172.21.15.82:0/1297437111' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T21:12:55.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:55 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/381474680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a37bcd68-6698-4832-8f60-78c1e15a7208"}]: dispatch 2024-05-08T21:12:55.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:55 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/381474680' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a37bcd68-6698-4832-8f60-78c1e15a7208"}]': finished 2024-05-08T21:12:55.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:55 smithi033 ceph-mon[42528]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T21:12:55.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:55 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:12:55.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:55 smithi071 ceph-mon[47074]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:55.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:55 smithi071 ceph-mon[47074]: Standby manager daemon z started 2024-05-08T21:12:55.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:55 smithi071 ceph-mon[47074]: from='mgr.? 172.21.15.82:0/1297437111' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T21:12:55.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:55 smithi071 ceph-mon[47074]: from='mgr.? 172.21.15.82:0/1297437111' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T21:12:55.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:55 smithi071 ceph-mon[47074]: from='mgr.? 172.21.15.82:0/1297437111' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T21:12:55.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:55 smithi071 ceph-mon[47074]: from='mgr.? 172.21.15.82:0/1297437111' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T21:12:55.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:55 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/381474680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a37bcd68-6698-4832-8f60-78c1e15a7208"}]: dispatch 2024-05-08T21:12:55.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:55 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/381474680' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a37bcd68-6698-4832-8f60-78c1e15a7208"}]': finished 2024-05-08T21:12:55.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:55 smithi071 ceph-mon[47074]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T21:12:55.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:55 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:12:55.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:55 smithi082 ceph-mon[47512]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:55.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:55 smithi082 ceph-mon[47512]: Standby manager daemon z started 2024-05-08T21:12:55.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:55 smithi082 ceph-mon[47512]: from='mgr.? 172.21.15.82:0/1297437111' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T21:12:55.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:55 smithi082 ceph-mon[47512]: from='mgr.? 172.21.15.82:0/1297437111' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T21:12:55.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:55 smithi082 ceph-mon[47512]: from='mgr.? 172.21.15.82:0/1297437111' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T21:12:55.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:55 smithi082 ceph-mon[47512]: from='mgr.? 172.21.15.82:0/1297437111' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T21:12:55.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:55 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/381474680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a37bcd68-6698-4832-8f60-78c1e15a7208"}]: dispatch 2024-05-08T21:12:55.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:55 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/381474680' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a37bcd68-6698-4832-8f60-78c1e15a7208"}]': finished 2024-05-08T21:12:55.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:55 smithi082 ceph-mon[47512]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T21:12:55.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:55 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:12:55.913 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:55 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:55.627+0000 7f4784110200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T21:12:55.913 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:55 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:55.909+0000 7f4784110200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T21:12:56.176 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:55 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:55.997+0000 7f4784110200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T21:12:56.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:56 smithi033 ceph-mon[42528]: mgrmap e14: x(active, since 59s), standbys: z 2024-05-08T21:12:56.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T21:12:56.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:56 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1726664577' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:12:56.571 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:12:56 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y[48557]: 2024-05-08T21:12:56.186+0000 7f4784110200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T21:12:56.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:56 smithi071 ceph-mon[47074]: mgrmap e14: x(active, since 59s), standbys: z 2024-05-08T21:12:56.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T21:12:56.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:56 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1726664577' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:12:56.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:56 smithi082 ceph-mon[47512]: mgrmap e14: x(active, since 59s), standbys: z 2024-05-08T21:12:56.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T21:12:56.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:56 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1726664577' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:12:57.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:12:57.184+0000 7f47f77fe700 1 -- 172.21.15.33:0/1491795459 <== mon.1 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f47f8020c40 con 0x7f4808073980 2024-05-08T21:12:57.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:57 smithi033 ceph-mon[42528]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:57.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:57 smithi033 ceph-mon[42528]: Standby manager daemon y started 2024-05-08T21:12:57.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:57 smithi033 ceph-mon[42528]: from='mgr.? 172.21.15.71:0/174496047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T21:12:57.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:57 smithi033 ceph-mon[42528]: from='mgr.? 172.21.15.71:0/174496047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T21:12:57.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:57 smithi033 ceph-mon[42528]: from='mgr.? 172.21.15.71:0/174496047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T21:12:57.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:57 smithi033 ceph-mon[42528]: from='mgr.? 172.21.15.71:0/174496047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T21:12:57.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:57 smithi071 ceph-mon[47074]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:57.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:57 smithi071 ceph-mon[47074]: Standby manager daemon y started 2024-05-08T21:12:57.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:57 smithi071 ceph-mon[47074]: from='mgr.? 172.21.15.71:0/174496047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T21:12:57.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:57 smithi071 ceph-mon[47074]: from='mgr.? 172.21.15.71:0/174496047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T21:12:57.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:57 smithi071 ceph-mon[47074]: from='mgr.? 172.21.15.71:0/174496047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T21:12:57.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:57 smithi071 ceph-mon[47074]: from='mgr.? 172.21.15.71:0/174496047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T21:12:57.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:57 smithi082 ceph-mon[47512]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:57.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:57 smithi082 ceph-mon[47512]: Standby manager daemon y started 2024-05-08T21:12:57.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:57 smithi082 ceph-mon[47512]: from='mgr.? 172.21.15.71:0/174496047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T21:12:57.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:57 smithi082 ceph-mon[47512]: from='mgr.? 172.21.15.71:0/174496047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T21:12:57.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:57 smithi082 ceph-mon[47512]: from='mgr.? 172.21.15.71:0/174496047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T21:12:57.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:57 smithi082 ceph-mon[47512]: from='mgr.? 172.21.15.71:0/174496047' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T21:12:58.447 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:58 smithi033 ceph-mon[42528]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-05-08T21:12:58.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T21:12:58.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:58 smithi071 ceph-mon[47074]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-05-08T21:12:58.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T21:12:58.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:58 smithi082 ceph-mon[47512]: mgrmap e15: x(active, since 61s), standbys: z, y 2024-05-08T21:12:58.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T21:12:59.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:12:59 smithi033 ceph-mon[42528]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:59.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:12:59 smithi071 ceph-mon[47074]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:12:59.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:12:59 smithi082 ceph-mon[47512]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:01.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:01 smithi033 ceph-mon[42528]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:01.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T21:13:01.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:01.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:01 smithi071 ceph-mon[47074]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:01.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T21:13:01.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:01 smithi082 ceph-mon[47512]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T21:13:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:02.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:02 smithi033 ceph-mon[42528]: Deploying daemon osd.0 on smithi033 2024-05-08T21:13:02.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:02 smithi071 ceph-mon[47074]: Deploying daemon osd.0 on smithi033 2024-05-08T21:13:02.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:02 smithi082 ceph-mon[47512]: Deploying daemon osd.0 on smithi033 2024-05-08T21:13:03.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:03 smithi033 ceph-mon[42528]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:03.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:03 smithi071 ceph-mon[47074]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:03.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:03 smithi082 ceph-mon[47512]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:05.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:05 smithi033 ceph-mon[42528]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:05.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:05.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:05.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:05.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:05 smithi071 ceph-mon[47074]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:05.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:05.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:05.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:05 smithi082 ceph-mon[47512]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:05.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:07.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:07 smithi033 ceph-mon[42528]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:07.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:07 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:07.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:07 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:07.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:07 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:07.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:07 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:07.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:07 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:07.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.479+0000 7f47f77fe700 1 -- 172.21.15.33:0/1491795459 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f48080613e0 con 0x7f47f00389d0 2024-05-08T21:13:07.483 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 0 on host 'smithi033' 2024-05-08T21:13:07.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.483+0000 7f480cb05700 1 -- 172.21.15.33:0/1491795459 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f47f00389d0 msgr2=0x7f47f003ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:07.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.483+0000 7f480cb05700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f47f00389d0 0x7f47f003ae90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f47fc006010 tx=0x7f47fc00b610 comp rx=0 tx=0).stop 2024-05-08T21:13:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.483+0000 7f480cb05700 1 -- 172.21.15.33:0/1491795459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4808073980 msgr2=0x7f4808199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.483+0000 7f480cb05700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4808073980 0x7f4808199060 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f47f800e9d0 tx=0x7f47f800ece0 comp rx=0 tx=0).stop 2024-05-08T21:13:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.483+0000 7f480cb05700 1 -- 172.21.15.33:0/1491795459 shutdown_connections 2024-05-08T21:13:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.483+0000 7f480cb05700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4808108f00 0x7f48081a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.483+0000 7f480cb05700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4808073980 0x7f4808199060 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.483+0000 7f480cb05700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f47f00389d0 0x7f47f003ae90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.483+0000 7f480cb05700 1 --2- 172.21.15.33:0/1491795459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808074f50 0x7f48081995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.483+0000 7f480cb05700 1 -- 172.21.15.33:0/1491795459 >> 172.21.15.33:0/1491795459 conn(0x7f48080fc5c0 msgr2=0x7f4808104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:13:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.483+0000 7f480cb05700 1 -- 172.21.15.33:0/1491795459 shutdown_connections 2024-05-08T21:13:07.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:07.483+0000 7f480cb05700 1 -- 172.21.15.33:0/1491795459 wait complete. 2024-05-08T21:13:07.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:07 smithi071 ceph-mon[47074]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:07.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:07 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:07.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:07 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:07.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:07 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:07.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:07 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:07.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:07 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:07 smithi082 ceph-mon[47512]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:07 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:07 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:07 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:07 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:07 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:07.903 DEBUG:teuthology.orchestra.run.smithi033:osd.0> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.0.service 2024-05-08T21:13:07.906 INFO:tasks.cephadm:Deploying osd.1 on smithi071 with /dev/vg_nvme/lv_4... 2024-05-08T21:13:07.906 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T21:13:07.940 INFO:journalctl@ceph.osd.0.smithi033.stdout:-- Logs begin at Wed 2024-05-08 20:53:21 UTC. -- 2024-05-08T21:13:08.134 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:13:08.564 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:08 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:08.564 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:08 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:08.564 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:08 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:08.564 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:08 smithi071 ceph-mon[47074]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:08.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:08 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:08.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:08 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:08.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:08 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:08.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:08 smithi033 ceph-mon[42528]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:08.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:08 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:08.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:08 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:08.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:08 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:08.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:08 smithi082 ceph-mon[47512]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:09.208 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:13:08 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-0[60243]: 2024-05-08T21:13:08.855+0000 7f096d5d27c0 -1 osd.0 0 log_to_monitors true 2024-05-08T21:13:09.433 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T21:13:09.433 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T21:13:09.433 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-05-08T21:13:09.434 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-05-08T21:13:09.434 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112362 s, 933 MB/s 2024-05-08T21:13:09.434 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 2024-05-08T21:13:09.434 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-05-08T21:13:09.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:09 smithi033 ceph-mon[42528]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T21:13:09.716 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:09 smithi071 ceph-mon[47074]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T21:13:09.809 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_4 2024-05-08T21:13:09.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:09 smithi082 ceph-mon[47512]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T21:13:10.024 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:13:10.789 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T21:13:10.789 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T21:13:10.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:10.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:13:10.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:10.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: Detected new or changed devices on smithi033 2024-05-08T21:13:10.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:10.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:10.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:13:10.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi033 to 17118M 2024-05-08T21:13:10.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:10.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:10.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:10.790 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:10.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T21:13:10.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T21:13:10.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:10.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:13:10.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:10.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: Detected new or changed devices on smithi033 2024-05-08T21:13:10.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:10.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:10.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:13:10.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi033 to 17118M 2024-05-08T21:13:10.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:10.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:10.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:10.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:10.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T21:13:10.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T21:13:10.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:10.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:13:10.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:10.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: Detected new or changed devices on smithi033 2024-05-08T21:13:10.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:10.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:10.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:13:10.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi033 to 17118M 2024-05-08T21:13:10.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:10.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:10.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:10.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:10.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.970+0000 7f7e8ce83700 1 -- 172.21.15.71:0/3363032293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e88102200 msgr2=0x7f7e88102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:10.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.970+0000 7f7e8ce83700 1 --2- 172.21.15.71:0/3363032293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e88102200 0x7f7e88102660 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c0099c0 tx=0x7f7e7c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:13:10.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.971+0000 7f7e8ce83700 1 -- 172.21.15.71:0/3363032293 shutdown_connections 2024-05-08T21:13:10.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.971+0000 7f7e8ce83700 1 --2- 172.21.15.71:0/3363032293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e88102ba0 0x7f7e88109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:10.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.971+0000 7f7e8ce83700 1 --2- 172.21.15.71:0/3363032293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e88101000 0x7f7e88101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:10.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.971+0000 7f7e8ce83700 1 --2- 172.21.15.71:0/3363032293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e88102200 0x7f7e88102660 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:10.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.971+0000 7f7e8ce83700 1 -- 172.21.15.71:0/3363032293 >> 172.21.15.71:0/3363032293 conn(0x7f7e880fc5c0 msgr2=0x7f7e880fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:13:10.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.971+0000 7f7e8ce83700 1 -- 172.21.15.71:0/3363032293 shutdown_connections 2024-05-08T21:13:10.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.971+0000 7f7e8ce83700 1 -- 172.21.15.71:0/3363032293 wait complete. 2024-05-08T21:13:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.971+0000 7f7e8ce83700 1 Processor -- start 2024-05-08T21:13:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.971+0000 7f7e8ce83700 1 -- start start 2024-05-08T21:13:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e8ce83700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e88101000 0x7f7e88196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:13:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e8ce83700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e88102200 0x7f7e88197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:13:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e8ce83700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e88102ba0 0x7f7e8819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:13:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e8ce83700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8819e9f0 con 0x7f7e88102200 2024-05-08T21:13:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e8ce83700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8819eb30 con 0x7f7e88102ba0 2024-05-08T21:13:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e8ce83700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8819ecc0 con 0x7f7e88101000 2024-05-08T21:13:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e86d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e88102ba0 0x7f7e8819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:13:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e86d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e88102ba0 0x7f7e8819e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41114/0 (socket says 172.21.15.71:41114) 2024-05-08T21:13:10.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e85d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e88102200 0x7f7e88197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e86d9d700 1 -- 172.21.15.71:0/557752641 learned_addr learned my addr 172.21.15.71:0/557752641 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e8659c700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e88101000 0x7f7e88196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e86d9d700 1 -- 172.21.15.71:0/557752641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e88101000 msgr2=0x7f7e88196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e86d9d700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e88101000 0x7f7e88196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e86d9d700 1 -- 172.21.15.71:0/557752641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e88102200 msgr2=0x7f7e88197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e86d9d700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e88102200 0x7f7e88197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e86d9d700 1 -- 172.21.15.71:0/557752641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c0095f0 con 0x7f7e88102ba0 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e86d9d700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e88102ba0 0x7f7e8819e420 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000b970 tx=0x7f7e8000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e777fe700 1 -- 172.21.15.71:0/557752641 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000d860 con 0x7f7e88102ba0 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e777fe700 1 -- 172.21.15.71:0/557752641 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e8000de50 con 0x7f7e88102ba0 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e777fe700 1 -- 172.21.15.71:0/557752641 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80017400 con 0x7f7e88102ba0 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e8ce83700 1 -- 172.21.15.71:0/557752641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e8819efa0 con 0x7f7e88102ba0 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.972+0000 7f7e8ce83700 1 -- 172.21.15.71:0/557752641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e8819f3c0 con 0x7f7e88102ba0 2024-05-08T21:13:10.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.973+0000 7f7e8ce83700 1 -- 172.21.15.71:0/557752641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e88108580 con 0x7f7e88102ba0 2024-05-08T21:13:10.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.974+0000 7f7e777fe700 1 -- 172.21.15.71:0/557752641 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f7e8000d9c0 con 0x7f7e88102ba0 2024-05-08T21:13:10.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.975+0000 7f7e777fe700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7e700a1110 0x7f7e700a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:13:10.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.975+0000 7f7e777fe700 1 -- 172.21.15.71:0/557752641 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7f7e80006e40 con 0x7f7e88102ba0 2024-05-08T21:13:10.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.975+0000 7f7e8659c700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7e700a1110 0x7f7e700a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:13:10.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.975+0000 7f7e8659c700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7e700a1110 0x7f7e700a35d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7e88102060 tx=0x7f7e7800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:13:10.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:10.979+0000 7f7e777fe700 1 -- 172.21.15.71:0/557752641 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7e80095ba0 con 0x7f7e88102ba0 2024-05-08T21:13:11.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:11.177+0000 7f7e8ce83700 1 -- 172.21.15.71:0/557752641 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e8802ceb0 con 0x7f7e700a1110 2024-05-08T21:13:11.459 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:13:11 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-0[60243]: 2024-05-08T21:13:11.168+0000 7f0962c36700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T21:13:11.459 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:13:11 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-0[60243]: 2024-05-08T21:13:11.173+0000 7f095aa23700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:13:11.728 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:11 smithi071 ceph-mon[47074]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-08T21:13:11.728 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:11 smithi071 ceph-mon[47074]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T21:13:11.728 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:11.728 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:11.728 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:11 smithi071 ceph-mon[47074]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' 2024-05-08T21:13:11.728 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:11 smithi071 ceph-mon[47074]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:13:11.728 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:13:11.728 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:13:11.728 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:11.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:11 smithi082 ceph-mon[47512]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-08T21:13:11.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:11 smithi082 ceph-mon[47512]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T21:13:11.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:11.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:11.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:11 smithi082 ceph-mon[47512]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' 2024-05-08T21:13:11.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:11 smithi082 ceph-mon[47512]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:13:11.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:13:11.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:13:11.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:11.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:11 smithi033 ceph-mon[42528]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-08T21:13:11.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:11 smithi033 ceph-mon[42528]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T21:13:11.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:11.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:11.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:11 smithi033 ceph-mon[42528]: from='osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393]' entity='osd.0' 2024-05-08T21:13:11.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:11 smithi033 ceph-mon[42528]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:13:11.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:13:11.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:13:11.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:12.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:12 smithi071 ceph-mon[47074]: purged_snaps scrub starts 2024-05-08T21:13:12.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:12 smithi071 ceph-mon[47074]: purged_snaps scrub ok 2024-05-08T21:13:12.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:12.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:12 smithi071 ceph-mon[47074]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:12.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:12 smithi071 ceph-mon[47074]: osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393] boot 2024-05-08T21:13:12.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:12 smithi071 ceph-mon[47074]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T21:13:12.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:12.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:12 smithi082 ceph-mon[47512]: purged_snaps scrub starts 2024-05-08T21:13:12.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:12 smithi082 ceph-mon[47512]: purged_snaps scrub ok 2024-05-08T21:13:12.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:12.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:12 smithi082 ceph-mon[47512]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:12.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:12 smithi082 ceph-mon[47512]: osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393] boot 2024-05-08T21:13:12.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:12 smithi082 ceph-mon[47512]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T21:13:12.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:12.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:12 smithi033 ceph-mon[42528]: purged_snaps scrub starts 2024-05-08T21:13:12.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:12 smithi033 ceph-mon[42528]: purged_snaps scrub ok 2024-05-08T21:13:12.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:12.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:12 smithi033 ceph-mon[42528]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T21:13:12.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:12 smithi033 ceph-mon[42528]: osd.0 [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393] boot 2024-05-08T21:13:12.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:12 smithi033 ceph-mon[42528]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T21:13:12.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T21:13:13.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:13 smithi071 ceph-mon[47074]: from='client.? 172.21.15.71:0/1950581377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f789dc73-b4d8-49f3-9a6f-83388b7245da"}]: dispatch 2024-05-08T21:13:13.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:13 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f789dc73-b4d8-49f3-9a6f-83388b7245da"}]: dispatch 2024-05-08T21:13:13.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:13 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f789dc73-b4d8-49f3-9a6f-83388b7245da"}]': finished 2024-05-08T21:13:13.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:13 smithi071 ceph-mon[47074]: osdmap e9: 2 total, 1 up, 2 in 2024-05-08T21:13:13.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:13.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:13 smithi082 ceph-mon[47512]: from='client.? 172.21.15.71:0/1950581377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f789dc73-b4d8-49f3-9a6f-83388b7245da"}]: dispatch 2024-05-08T21:13:13.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:13 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f789dc73-b4d8-49f3-9a6f-83388b7245da"}]: dispatch 2024-05-08T21:13:13.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:13 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f789dc73-b4d8-49f3-9a6f-83388b7245da"}]': finished 2024-05-08T21:13:13.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:13 smithi082 ceph-mon[47512]: osdmap e9: 2 total, 1 up, 2 in 2024-05-08T21:13:13.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:13.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:13 smithi033 ceph-mon[42528]: from='client.? 172.21.15.71:0/1950581377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f789dc73-b4d8-49f3-9a6f-83388b7245da"}]: dispatch 2024-05-08T21:13:13.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:13 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f789dc73-b4d8-49f3-9a6f-83388b7245da"}]: dispatch 2024-05-08T21:13:13.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:13 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f789dc73-b4d8-49f3-9a6f-83388b7245da"}]': finished 2024-05-08T21:13:13.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:13 smithi033 ceph-mon[42528]: osdmap e9: 2 total, 1 up, 2 in 2024-05-08T21:13:13.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:14.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:14 smithi071 ceph-mon[47074]: from='client.? 172.21.15.71:0/1238554719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:13:14.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:14 smithi071 ceph-mon[47074]: pgmap v38: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:14.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:14 smithi082 ceph-mon[47512]: from='client.? 172.21.15.71:0/1238554719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:13:14.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:14 smithi082 ceph-mon[47512]: pgmap v38: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:14.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:14 smithi033 ceph-mon[42528]: from='client.? 172.21.15.71:0/1238554719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:13:14.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:14 smithi033 ceph-mon[42528]: pgmap v38: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:17.402 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:17 smithi071 ceph-mon[47074]: pgmap v39: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:17.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:17 smithi033 ceph-mon[42528]: pgmap v39: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:17.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:17 smithi082 ceph-mon[47512]: pgmap v39: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:19.374 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:19 smithi071 ceph-mon[47074]: pgmap v40: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:19.374 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T21:13:19.375 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:19.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:19 smithi033 ceph-mon[42528]: pgmap v40: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:19.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T21:13:19.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:19.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:19 smithi082 ceph-mon[47512]: pgmap v40: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:19.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T21:13:19.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:20.270 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:20 smithi071 ceph-mon[47074]: Deploying daemon osd.1 on smithi071 2024-05-08T21:13:20.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:20 smithi033 ceph-mon[42528]: Deploying daemon osd.1 on smithi071 2024-05-08T21:13:20.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:20 smithi082 ceph-mon[47512]: Deploying daemon osd.1 on smithi071 2024-05-08T21:13:21.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:21 smithi033 ceph-mon[42528]: pgmap v41: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:21.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:21 smithi071 ceph-mon[47074]: pgmap v41: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:21.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:21 smithi082 ceph-mon[47512]: pgmap v41: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:23.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:23 smithi033 ceph-mon[42528]: pgmap v42: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:23.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:23 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:23.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:23 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:23.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:23 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:23.464 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:23 smithi071 ceph-mon[47074]: pgmap v42: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:23.464 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:23 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:23.464 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:23 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:23.464 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:23 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:23.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:23 smithi082 ceph-mon[47512]: pgmap v42: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:23.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:23 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:23.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:23 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:23.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:23 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:24.642 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 1 on host 'smithi071' 2024-05-08T21:13:24.643 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.638+0000 7f7e777fe700 1 -- 172.21.15.71:0/557752641 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7e8802ceb0 con 0x7f7e700a1110 2024-05-08T21:13:24.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.641+0000 7f7e8ce83700 1 -- 172.21.15.71:0/557752641 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7e700a1110 msgr2=0x7f7e700a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:24.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.641+0000 7f7e8ce83700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7e700a1110 0x7f7e700a35d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7e88102060 tx=0x7f7e7800b410 comp rx=0 tx=0).stop 2024-05-08T21:13:24.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.641+0000 7f7e8ce83700 1 -- 172.21.15.71:0/557752641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e88102ba0 msgr2=0x7f7e8819e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:24.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.641+0000 7f7e8ce83700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e88102ba0 0x7f7e8819e420 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000b970 tx=0x7f7e8000bc80 comp rx=0 tx=0).stop 2024-05-08T21:13:24.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.641+0000 7f7e8ce83700 1 -- 172.21.15.71:0/557752641 shutdown_connections 2024-05-08T21:13:24.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.641+0000 7f7e8ce83700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7e700a1110 0x7f7e700a35d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:24.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.641+0000 7f7e8ce83700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e88102ba0 0x7f7e8819e420 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:24.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.641+0000 7f7e8ce83700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e88101000 0x7f7e88196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:24.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.641+0000 7f7e8ce83700 1 --2- 172.21.15.71:0/557752641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e88102200 0x7f7e88197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:24.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.641+0000 7f7e8ce83700 1 -- 172.21.15.71:0/557752641 >> 172.21.15.71:0/557752641 conn(0x7f7e880fc5c0 msgr2=0x7f7e881056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:13:24.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.641+0000 7f7e8ce83700 1 -- 172.21.15.71:0/557752641 shutdown_connections 2024-05-08T21:13:24.645 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:13:24.641+0000 7f7e8ce83700 1 -- 172.21.15.71:0/557752641 wait complete. 2024-05-08T21:13:24.810 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:24.810 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:24.810 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:24.811 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:24.811 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:24.811 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:24 smithi071 ceph-mon[47074]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:24.811 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:24.811 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:24.811 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:24.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:24.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:24.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:24.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:24.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:24.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:24 smithi033 ceph-mon[42528]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:24.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:24.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:24.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:25.028 DEBUG:teuthology.orchestra.run.smithi071:osd.1> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.1.service 2024-05-08T21:13:25.030 INFO:tasks.cephadm:Deploying osd.2 on smithi082 with /dev/vg_nvme/lv_4... 2024-05-08T21:13:25.030 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T21:13:25.061 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:25.061 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:25.061 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:25.061 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:25.061 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:25.061 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:24 smithi082 ceph-mon[47512]: pgmap v43: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:25.062 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:25.062 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:25.062 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:25.062 INFO:journalctl@ceph.osd.1.smithi071.stdout:-- Logs begin at Wed 2024-05-08 20:53:45 UTC. -- 2024-05-08T21:13:25.256 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:13:26.713 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T21:13:26.713 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T21:13:26.713 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-05-08T21:13:26.713 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-05-08T21:13:26.713 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112037 s, 936 MB/s 2024-05-08T21:13:26.713 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-05-08T21:13:26.728 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:13:26 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-1[52049]: 2024-05-08T21:13:26.455+0000 7fcd4d4587c0 -1 osd.1 0 log_to_monitors true 2024-05-08T21:13:27.051 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_4 2024-05-08T21:13:27.312 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:13:27.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:27 smithi071 ceph-mon[47074]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:27.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:27 smithi071 ceph-mon[47074]: from='osd.1 [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T21:13:27.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:27 smithi071 ceph-mon[47074]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T21:13:27.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:27.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:27.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:13:27.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:27.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:27.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:27.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:27.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:27 smithi082 ceph-mon[47512]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:27.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:27 smithi082 ceph-mon[47512]: from='osd.1 [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T21:13:27.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:27 smithi082 ceph-mon[47512]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T21:13:27.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:27.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:27.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:13:27.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:27.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:27.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:27.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:27.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:27 smithi033 ceph-mon[42528]: pgmap v44: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:27 smithi033 ceph-mon[42528]: from='osd.1 [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T21:13:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:27 smithi033 ceph-mon[42528]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T21:13:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:13:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.360+0000 7fb92e207700 1 -- 172.21.15.82:0/2281790259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb928101580 msgr2=0x7fb9281019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.360+0000 7fb92e207700 1 --2- 172.21.15.82:0/2281790259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb928101580 0x7fb9281019a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb918009990 tx=0x7fb918009ca0 comp rx=0 tx=0).stop 2024-05-08T21:13:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.360+0000 7fb92e207700 1 -- 172.21.15.82:0/2281790259 shutdown_connections 2024-05-08T21:13:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.360+0000 7fb92e207700 1 --2- 172.21.15.82:0/2281790259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb928100b10 0x7fb9281083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.360+0000 7fb92e207700 1 --2- 172.21.15.82:0/2281790259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb928101580 0x7fb9281019a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.360+0000 7fb92e207700 1 --2- 172.21.15.82:0/2281790259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb928100150 0x7fb9281005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.360+0000 7fb92e207700 1 -- 172.21.15.82:0/2281790259 >> 172.21.15.82:0/2281790259 conn(0x7fb9280fac40 msgr2=0x7fb9280fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:13:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.360+0000 7fb92e207700 1 -- 172.21.15.82:0/2281790259 shutdown_connections 2024-05-08T21:13:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.360+0000 7fb92e207700 1 -- 172.21.15.82:0/2281790259 wait complete. 2024-05-08T21:13:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb92e207700 1 Processor -- start 2024-05-08T21:13:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb92e207700 1 -- start start 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb92e207700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb928100150 0x7fb9280ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb92e207700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb928100b10 0x7fb92819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb92e207700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb928101580 0x7fb92819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb92e207700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92819e930 con 0x7fb928101580 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb92e207700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92819eaa0 con 0x7fb928100150 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb92e207700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92819ec60 con 0x7fb928100b10 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb91ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb928100b10 0x7fb92819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb91ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb928100b10 0x7fb92819bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57908/0 (socket says 172.21.15.82:57908) 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb91ffff700 1 -- 172.21.15.82:0/2694466648 learned_addr learned my addr 172.21.15.82:0/2694466648 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb927fff700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb928101580 0x7fb92819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb9277fe700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb928100150 0x7fb9280ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb91ffff700 1 -- 172.21.15.82:0/2694466648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb928100150 msgr2=0x7fb9280ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb91ffff700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb928100150 0x7fb9280ffb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:28.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb91ffff700 1 -- 172.21.15.82:0/2694466648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb928101580 msgr2=0x7fb92819e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:28.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb91ffff700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb928101580 0x7fb92819e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:28.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.361+0000 7fb91ffff700 1 -- 172.21.15.82:0/2694466648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9180095f0 con 0x7fb928100b10 2024-05-08T21:13:28.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.362+0000 7fb91ffff700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb928100b10 0x7fb92819bbe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb91000d980 tx=0x7fb91000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:13:28.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.362+0000 7fb9257fa700 1 -- 172.21.15.82:0/2694466648 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91000bb60 con 0x7fb928100b10 2024-05-08T21:13:28.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.362+0000 7fb9257fa700 1 -- 172.21.15.82:0/2694466648 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb910007640 con 0x7fb928100b10 2024-05-08T21:13:28.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.362+0000 7fb9257fa700 1 -- 172.21.15.82:0/2694466648 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910004d60 con 0x7fb928100b10 2024-05-08T21:13:28.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.362+0000 7fb92e207700 1 -- 172.21.15.82:0/2694466648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92819ef40 con 0x7fb928100b10 2024-05-08T21:13:28.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.362+0000 7fb92e207700 1 -- 172.21.15.82:0/2694466648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92819f3b0 con 0x7fb928100b10 2024-05-08T21:13:28.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.363+0000 7fb92e207700 1 -- 172.21.15.82:0/2694466648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb928107530 con 0x7fb928100b10 2024-05-08T21:13:28.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.364+0000 7fb9257fa700 1 -- 172.21.15.82:0/2694466648 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7fb910004020 con 0x7fb928100b10 2024-05-08T21:13:28.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.364+0000 7fb9257fa700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb9140a1110 0x7fb9140a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:13:28.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.364+0000 7fb9257fa700 1 -- 172.21.15.82:0/2694466648 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2029+0+0 (secure 0 0 0) 0x7fb9100cadd0 con 0x7fb928100b10 2024-05-08T21:13:28.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.365+0000 7fb9277fe700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb9140a1110 0x7fb9140a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:13:28.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.365+0000 7fb9277fe700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb9140a1110 0x7fb9140a35d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb918003780 tx=0x7fb918005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:13:28.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.369+0000 7fb9257fa700 1 -- 172.21.15.82:0/2694466648 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb910095ed0 con 0x7fb928100b10 2024-05-08T21:13:28.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:28 smithi033 ceph-mon[42528]: Detected new or changed devices on smithi071 2024-05-08T21:13:28.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:28 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi071 to 17113M 2024-05-08T21:13:28.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:28 smithi033 ceph-mon[42528]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T21:13:28.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:28 smithi033 ceph-mon[42528]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T21:13:28.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:28 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:28.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:28 smithi033 ceph-mon[42528]: from='osd.1 [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:13:28.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:28 smithi033 ceph-mon[42528]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:13:28.571 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:13:28 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-1[52049]: 2024-05-08T21:13:28.143+0000 7fcd42abc700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T21:13:28.572 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:13:28 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-1[52049]: 2024-05-08T21:13:28.148+0000 7fcd3e8b1700 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:13:28.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:28 smithi071 ceph-mon[47074]: Detected new or changed devices on smithi071 2024-05-08T21:13:28.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:28 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi071 to 17113M 2024-05-08T21:13:28.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:28 smithi071 ceph-mon[47074]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T21:13:28.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:28 smithi071 ceph-mon[47074]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T21:13:28.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:28 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:28.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:28 smithi071 ceph-mon[47074]: from='osd.1 [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:13:28.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:28 smithi071 ceph-mon[47074]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:13:28.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:28.570+0000 7fb92e207700 1 -- 172.21.15.82:0/2694466648 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb92802ceb0 con 0x7fb9140a1110 2024-05-08T21:13:28.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:28 smithi082 ceph-mon[47512]: Detected new or changed devices on smithi071 2024-05-08T21:13:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:28 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi071 to 17113M 2024-05-08T21:13:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:28 smithi082 ceph-mon[47512]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T21:13:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:28 smithi082 ceph-mon[47512]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T21:13:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:28 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:28 smithi082 ceph-mon[47512]: from='osd.1 [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:13:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:28 smithi082 ceph-mon[47512]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:13:29.309 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:29 smithi082 ceph-mon[47512]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:29.309 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:29 smithi082 ceph-mon[47512]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-08T21:13:29.309 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:29 smithi082 ceph-mon[47512]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T21:13:29.309 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:29 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:29.309 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:29 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:29.309 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:29 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:13:29.309 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:29 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:13:29.309 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:29 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:29.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:29 smithi033 ceph-mon[42528]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:29.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:29 smithi033 ceph-mon[42528]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-08T21:13:29.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:29 smithi033 ceph-mon[42528]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T21:13:29.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:29 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:29.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:29 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:29.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:29 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:13:29.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:29 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:13:29.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:29 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:29 smithi071 ceph-mon[47074]: pgmap v46: 0 pgs: ; 0 B data, 30 MiB used, 89 GiB / 89 GiB avail 2024-05-08T21:13:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:29 smithi071 ceph-mon[47074]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-08T21:13:29.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:29 smithi071 ceph-mon[47074]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T21:13:29.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:29 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:29.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:29 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:29.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:29 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:13:29.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:29 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:13:29.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:29 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:30.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:30 smithi033 ceph-mon[42528]: purged_snaps scrub starts 2024-05-08T21:13:30.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:30 smithi033 ceph-mon[42528]: purged_snaps scrub ok 2024-05-08T21:13:30.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:30 smithi033 ceph-mon[42528]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:13:30.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:30 smithi033 ceph-mon[42528]: osd.1 [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237] boot 2024-05-08T21:13:30.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:30 smithi033 ceph-mon[42528]: osdmap e12: 2 total, 2 up, 2 in 2024-05-08T21:13:30.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:30 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:30.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:30 smithi071 ceph-mon[47074]: purged_snaps scrub starts 2024-05-08T21:13:30.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:30 smithi071 ceph-mon[47074]: purged_snaps scrub ok 2024-05-08T21:13:30.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:30 smithi071 ceph-mon[47074]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:13:30.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:30 smithi071 ceph-mon[47074]: osd.1 [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237] boot 2024-05-08T21:13:30.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:30 smithi071 ceph-mon[47074]: osdmap e12: 2 total, 2 up, 2 in 2024-05-08T21:13:30.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:30 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:30.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:30 smithi082 ceph-mon[47512]: purged_snaps scrub starts 2024-05-08T21:13:30.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:30 smithi082 ceph-mon[47512]: purged_snaps scrub ok 2024-05-08T21:13:30.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:30 smithi082 ceph-mon[47512]: from='client.24178 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:13:30.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:30 smithi082 ceph-mon[47512]: osd.1 [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237] boot 2024-05-08T21:13:30.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:30 smithi082 ceph-mon[47512]: osdmap e12: 2 total, 2 up, 2 in 2024-05-08T21:13:30.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:30 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T21:13:31.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:31 smithi033 ceph-mon[42528]: pgmap v49: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:31.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:31 smithi033 ceph-mon[42528]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T21:13:31.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:31 smithi033 ceph-mon[42528]: from='client.? 172.21.15.82:0/1681301934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6c73166-7e6f-41d9-a8ad-713e7ddceb63"}]: dispatch 2024-05-08T21:13:31.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:31 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6c73166-7e6f-41d9-a8ad-713e7ddceb63"}]: dispatch 2024-05-08T21:13:31.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:31 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6c73166-7e6f-41d9-a8ad-713e7ddceb63"}]': finished 2024-05-08T21:13:31.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:31 smithi033 ceph-mon[42528]: osdmap e14: 3 total, 2 up, 3 in 2024-05-08T21:13:31.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:31 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:31.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:31 smithi033 ceph-mon[42528]: from='client.? 172.21.15.82:0/3660533079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:13:31.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:31 smithi071 ceph-mon[47074]: pgmap v49: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:31.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:31 smithi071 ceph-mon[47074]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T21:13:31.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:31 smithi071 ceph-mon[47074]: from='client.? 172.21.15.82:0/1681301934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6c73166-7e6f-41d9-a8ad-713e7ddceb63"}]: dispatch 2024-05-08T21:13:31.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:31 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6c73166-7e6f-41d9-a8ad-713e7ddceb63"}]: dispatch 2024-05-08T21:13:31.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:31 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6c73166-7e6f-41d9-a8ad-713e7ddceb63"}]': finished 2024-05-08T21:13:31.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:31 smithi071 ceph-mon[47074]: osdmap e14: 3 total, 2 up, 3 in 2024-05-08T21:13:31.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:31 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:31.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:31 smithi071 ceph-mon[47074]: from='client.? 172.21.15.82:0/3660533079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:13:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:31 smithi082 ceph-mon[47512]: pgmap v49: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:31 smithi082 ceph-mon[47512]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T21:13:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:31 smithi082 ceph-mon[47512]: from='client.? 172.21.15.82:0/1681301934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6c73166-7e6f-41d9-a8ad-713e7ddceb63"}]: dispatch 2024-05-08T21:13:31.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:31 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6c73166-7e6f-41d9-a8ad-713e7ddceb63"}]: dispatch 2024-05-08T21:13:31.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:31 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6c73166-7e6f-41d9-a8ad-713e7ddceb63"}]': finished 2024-05-08T21:13:31.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:31 smithi082 ceph-mon[47512]: osdmap e14: 3 total, 2 up, 3 in 2024-05-08T21:13:31.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:31 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:31.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:31 smithi082 ceph-mon[47512]: from='client.? 172.21.15.82:0/3660533079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:13:33.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:33 smithi033 ceph-mon[42528]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:33.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:33 smithi071 ceph-mon[47074]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:33.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:33 smithi082 ceph-mon[47512]: pgmap v52: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:35 smithi033 ceph-mon[42528]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:35.500 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:35 smithi082 ceph-mon[47512]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:35.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:35 smithi071 ceph-mon[47074]: pgmap v53: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:37.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:37 smithi082 ceph-mon[47512]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:37.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:37 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T21:13:37.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:37 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:37.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:37 smithi082 ceph-mon[47512]: Deploying daemon osd.2 on smithi082 2024-05-08T21:13:37.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:37 smithi033 ceph-mon[42528]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:37.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:37 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T21:13:37.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:37 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:37.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:37 smithi033 ceph-mon[42528]: Deploying daemon osd.2 on smithi082 2024-05-08T21:13:37.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:37 smithi071 ceph-mon[47074]: pgmap v54: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:37.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:37 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T21:13:37.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:37 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:37.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:37 smithi071 ceph-mon[47074]: Deploying daemon osd.2 on smithi082 2024-05-08T21:13:39.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:39 smithi033 ceph-mon[42528]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:39.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:39 smithi071 ceph-mon[47074]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:39.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:39 smithi082 ceph-mon[47512]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:40 smithi082 ceph-mon[47512]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:41.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:41.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:41.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:41.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:40 smithi033 ceph-mon[42528]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:41.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:40 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:41.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:40 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:41.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:40 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:41.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:40 smithi071 ceph-mon[47074]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:42.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.073+0000 7fb9257fa700 1 -- 172.21.15.82:0/2694466648 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb92802ceb0 con 0x7fb9140a1110 2024-05-08T21:13:42.076 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 2 on host 'smithi082' 2024-05-08T21:13:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.075+0000 7fb92e207700 1 -- 172.21.15.82:0/2694466648 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb9140a1110 msgr2=0x7fb9140a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.075+0000 7fb92e207700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb9140a1110 0x7fb9140a35d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb918003780 tx=0x7fb918005fd0 comp rx=0 tx=0).stop 2024-05-08T21:13:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.075+0000 7fb92e207700 1 -- 172.21.15.82:0/2694466648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb928100b10 msgr2=0x7fb92819bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.075+0000 7fb92e207700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb928100b10 0x7fb92819bbe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb91000d980 tx=0x7fb91000dd40 comp rx=0 tx=0).stop 2024-05-08T21:13:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.075+0000 7fb92e207700 1 -- 172.21.15.82:0/2694466648 shutdown_connections 2024-05-08T21:13:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.075+0000 7fb92e207700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb928101580 0x7fb92819e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.075+0000 7fb92e207700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb9140a1110 0x7fb9140a35d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.075+0000 7fb92e207700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb928100150 0x7fb9280ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.075+0000 7fb92e207700 1 --2- 172.21.15.82:0/2694466648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb928100b10 0x7fb92819bbe0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.075+0000 7fb92e207700 1 -- 172.21.15.82:0/2694466648 >> 172.21.15.82:0/2694466648 conn(0x7fb9280fac40 msgr2=0x7fb928104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:13:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.075+0000 7fb92e207700 1 -- 172.21.15.82:0/2694466648 shutdown_connections 2024-05-08T21:13:42.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:13:42.075+0000 7fb92e207700 1 -- 172.21.15.82:0/2694466648 wait complete. 2024-05-08T21:13:42.272 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.273 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.273 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:42.273 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:42.273 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.273 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.273 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.273 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:42.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:42.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:42.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:42.512 DEBUG:teuthology.orchestra.run.smithi082:osd.2> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.2.service 2024-05-08T21:13:42.514 INFO:tasks.cephadm:Deploying osd.3 on smithi033 with /dev/vg_nvme/lv_3... 2024-05-08T21:13:42.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T21:13:42.544 INFO:journalctl@ceph.osd.2.smithi082.stdout:-- Logs begin at Wed 2024-05-08 20:53:53 UTC. -- 2024-05-08T21:13:42.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:42.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:42.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:42.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:42.771 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:13:43.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:43 smithi033 ceph-mon[42528]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:43.314 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:43 smithi082 ceph-mon[47512]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:43.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:43 smithi071 ceph-mon[47074]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T21:13:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T21:13:44.243 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-08T21:13:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-08T21:13:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0169883 s, 617 MB/s 2024-05-08T21:13:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-08T21:13:44.325 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:13:43 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-2[53308]: 2024-05-08T21:13:43.883+0000 7fbe06f1d7c0 -1 osd.2 0 log_to_monitors true 2024-05-08T21:13:44.326 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:44 smithi082 ceph-mon[47512]: from='osd.2 [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T21:13:44.326 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:44 smithi082 ceph-mon[47512]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T21:13:44.434 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:44 smithi033 ceph-mon[42528]: from='osd.2 [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T21:13:44.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:44 smithi033 ceph-mon[42528]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T21:13:44.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:44 smithi071 ceph-mon[47074]: from='osd.2 [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T21:13:44.585 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:44 smithi071 ceph-mon[47074]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T21:13:44.590 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_3 2024-05-08T21:13:44.857 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:13:45.317 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:45.317 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T21:13:45.317 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T21:13:45.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:45.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: from='osd.2 [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:13:45.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:13:45.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:45.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:45.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:13:45.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:45.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:45.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:45.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:45.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:45.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T21:13:45.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T21:13:45.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:45.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: from='osd.2 [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:13:45.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:13:45.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:45.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:45.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:13:45.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:45.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:45.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:45.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:45.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:45.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T21:13:45.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T21:13:45.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:45.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: from='osd.2 [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:13:45.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:13:45.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:45.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:45.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:13:45.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:45.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:45.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:13:45.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.895+0000 7f3131de1700 1 -- 172.21.15.33:0/4212963513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312c0fff00 msgr2=0x7f312c100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.895+0000 7f3131de1700 1 --2- 172.21.15.33:0/4212963513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312c0fff00 0x7f312c100320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f311c009990 tx=0x7f311c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:13:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.895+0000 7f3131de1700 1 -- 172.21.15.33:0/4212963513 shutdown_connections 2024-05-08T21:13:45.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.895+0000 7f3131de1700 1 --2- 172.21.15.33:0/4212963513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f312c101a20 0x7f312c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.895+0000 7f3131de1700 1 --2- 172.21.15.33:0/4212963513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312c0fff00 0x7f312c100320 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.895+0000 7f3131de1700 1 --2- 172.21.15.33:0/4212963513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f312c101060 0x7f312c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.895+0000 7f3131de1700 1 -- 172.21.15.33:0/4212963513 >> 172.21.15.33:0/4212963513 conn(0x7f312c0fb4a0 msgr2=0x7f312c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:13:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.895+0000 7f3131de1700 1 -- 172.21.15.33:0/4212963513 shutdown_connections 2024-05-08T21:13:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.895+0000 7f3131de1700 1 -- 172.21.15.33:0/4212963513 wait complete. 2024-05-08T21:13:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.896+0000 7f3131de1700 1 Processor -- start 2024-05-08T21:13:45.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.896+0000 7f3131de1700 1 -- start start 2024-05-08T21:13:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3131de1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f312c0fff00 0x7f312c196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:13:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3131de1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312c101060 0x7f312c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:13:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3131de1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f312c101a20 0x7f312c19e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:13:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3131de1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c19e8e0 con 0x7f312c101060 2024-05-08T21:13:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3131de1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c19ea20 con 0x7f312c101a20 2024-05-08T21:13:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3131de1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c19ebe0 con 0x7f312c0fff00 2024-05-08T21:13:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3122ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312c101060 0x7f312c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:13:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3122ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312c101060 0x7f312c197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41194/0 (socket says 172.21.15.33:41194) 2024-05-08T21:13:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3122ffd700 1 -- 172.21.15.33:0/1270110912 learned_addr learned my addr 172.21.15.33:0/1270110912 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:13:45.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f312b7fe700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f312c0fff00 0x7f312c196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f312bfff700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f312c101a20 0x7f312c19e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3122ffd700 1 -- 172.21.15.33:0/1270110912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f312c0fff00 msgr2=0x7f312c196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3122ffd700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f312c0fff00 0x7f312c196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3122ffd700 1 -- 172.21.15.33:0/1270110912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f312c101a20 msgr2=0x7f312c19e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3122ffd700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f312c101a20 0x7f312c19e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3122ffd700 1 -- 172.21.15.33:0/1270110912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c0095f0 con 0x7f312c101060 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f312b7fe700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f312c0fff00 0x7f312c196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f3122ffd700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312c101060 0x7f312c197370 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f311400d950 tx=0x7f311400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.897+0000 7f31297fa700 1 -- 172.21.15.33:0/1270110912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311400f890 con 0x7f312c101060 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.898+0000 7f31297fa700 1 -- 172.21.15.33:0/1270110912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f311400fe80 con 0x7f312c101060 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.898+0000 7f3131de1700 1 -- 172.21.15.33:0/1270110912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312c19eec0 con 0x7f312c101060 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.898+0000 7f31297fa700 1 -- 172.21.15.33:0/1270110912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311400e3b0 con 0x7f312c101060 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.898+0000 7f3131de1700 1 -- 172.21.15.33:0/1270110912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312c19f330 con 0x7f312c101060 2024-05-08T21:13:45.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.899+0000 7f3131de1700 1 -- 172.21.15.33:0/1270110912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312c1073d0 con 0x7f312c101060 2024-05-08T21:13:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.900+0000 7f31297fa700 1 -- 172.21.15.33:0/1270110912 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) v1 ==== 135817+0+0 (secure 0 0 0) 0x7f3114010460 con 0x7f312c101060 2024-05-08T21:13:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.900+0000 7f31297fa700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f31180a1110 0x7f31180a35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:13:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.901+0000 7f312b7fe700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f31180a1110 0x7f31180a35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:13:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.901+0000 7f31297fa700 1 -- 172.21.15.33:0/1270110912 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 2571+0+0 (secure 0 0 0) 0x7f31140caec0 con 0x7f312c101060 2024-05-08T21:13:45.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.901+0000 7f312b7fe700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f31180a1110 0x7f31180a35d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f311c00c010 tx=0x7f311c003720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:13:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:45.906+0000 7f31297fa700 1 -- 172.21.15.33:0/1270110912 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3114095da0 con 0x7f312c101060 2024-05-08T21:13:46.103 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:13:45 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-2[53308]: 2024-05-08T21:13:45.805+0000 7fbdfc581700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T21:13:46.104 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:13:45 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-2[53308]: 2024-05-08T21:13:45.809+0000 7fbdf436e700 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:13:46.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:46.131+0000 7f3131de1700 1 -- 172.21.15.33:0/1270110912 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f312c02ce10 con 0x7f31180a1110 2024-05-08T21:13:46.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:46 smithi082 ceph-mon[47512]: Detected new or changed devices on smithi082 2024-05-08T21:13:46.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:46 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi082 to 17113M 2024-05-08T21:13:46.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:46 smithi082 ceph-mon[47512]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-08T21:13:46.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:46 smithi082 ceph-mon[47512]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T21:13:46.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:46.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:46.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:46 smithi082 ceph-mon[47512]: from='osd.2 ' entity='osd.2' 2024-05-08T21:13:46.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:46 smithi033 ceph-mon[42528]: Detected new or changed devices on smithi082 2024-05-08T21:13:46.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:46 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi082 to 17113M 2024-05-08T21:13:46.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:46 smithi033 ceph-mon[42528]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-08T21:13:46.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:46 smithi033 ceph-mon[42528]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T21:13:46.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:46.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:46.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:46 smithi033 ceph-mon[42528]: from='osd.2 ' entity='osd.2' 2024-05-08T21:13:46.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:46 smithi071 ceph-mon[47074]: Detected new or changed devices on smithi082 2024-05-08T21:13:46.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:46 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi082 to 17113M 2024-05-08T21:13:46.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:46 smithi071 ceph-mon[47074]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-08T21:13:46.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:46 smithi071 ceph-mon[47074]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T21:13:46.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:46.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:46.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:46 smithi071 ceph-mon[47074]: from='osd.2 ' entity='osd.2' 2024-05-08T21:13:47.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:47 smithi033 ceph-mon[42528]: purged_snaps scrub starts 2024-05-08T21:13:47.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:47 smithi033 ceph-mon[42528]: purged_snaps scrub ok 2024-05-08T21:13:47.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:47 smithi033 ceph-mon[42528]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:47.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:47 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:47.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:47 smithi033 ceph-mon[42528]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:13:47.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:47 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:13:47.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:47 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:13:47.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:47 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:47.165 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:47 smithi033 ceph-mon[42528]: osd.2 [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540] boot 2024-05-08T21:13:47.165 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:47 smithi033 ceph-mon[42528]: osdmap e17: 3 total, 3 up, 3 in 2024-05-08T21:13:47.165 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:47 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:47.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:47 smithi071 ceph-mon[47074]: purged_snaps scrub starts 2024-05-08T21:13:47.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:47 smithi071 ceph-mon[47074]: purged_snaps scrub ok 2024-05-08T21:13:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:47 smithi071 ceph-mon[47074]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:47 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:47 smithi071 ceph-mon[47074]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:13:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:47 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:13:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:47 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:13:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:47 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:47 smithi071 ceph-mon[47074]: osd.2 [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540] boot 2024-05-08T21:13:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:47 smithi071 ceph-mon[47074]: osdmap e17: 3 total, 3 up, 3 in 2024-05-08T21:13:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:47 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:47 smithi082 ceph-mon[47512]: purged_snaps scrub starts 2024-05-08T21:13:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:47 smithi082 ceph-mon[47512]: purged_snaps scrub ok 2024-05-08T21:13:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:47 smithi082 ceph-mon[47512]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T21:13:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:47 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:47 smithi082 ceph-mon[47512]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:13:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:47 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:13:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:47 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:13:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:47 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:47 smithi082 ceph-mon[47512]: osd.2 [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540] boot 2024-05-08T21:13:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:47 smithi082 ceph-mon[47512]: osdmap e17: 3 total, 3 up, 3 in 2024-05-08T21:13:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:47 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T21:13:49.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:48 smithi082 ceph-mon[47512]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T21:13:49.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:48 smithi082 ceph-mon[47512]: pgmap v64: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:49.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:13:49.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:48 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/432282258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b61b4d-bbc1-448a-856a-5be6034cb981"}]: dispatch 2024-05-08T21:13:49.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T21:13:49.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:48 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/432282258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1b61b4d-bbc1-448a-856a-5be6034cb981"}]': finished 2024-05-08T21:13:49.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:48 smithi082 ceph-mon[47512]: osdmap e19: 4 total, 3 up, 4 in 2024-05-08T21:13:49.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:13:49.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:13:49.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:48 smithi033 ceph-mon[42528]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T21:13:49.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:48 smithi033 ceph-mon[42528]: pgmap v64: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:49.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:13:49.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:48 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/432282258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b61b4d-bbc1-448a-856a-5be6034cb981"}]: dispatch 2024-05-08T21:13:49.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T21:13:49.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:48 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/432282258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1b61b4d-bbc1-448a-856a-5be6034cb981"}]': finished 2024-05-08T21:13:49.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:48 smithi033 ceph-mon[42528]: osdmap e19: 4 total, 3 up, 4 in 2024-05-08T21:13:49.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:13:49.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:13:49.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:48 smithi071 ceph-mon[47074]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T21:13:49.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:48 smithi071 ceph-mon[47074]: pgmap v64: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:49.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:13:49.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:48 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/432282258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1b61b4d-bbc1-448a-856a-5be6034cb981"}]: dispatch 2024-05-08T21:13:49.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T21:13:49.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:48 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/432282258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1b61b4d-bbc1-448a-856a-5be6034cb981"}]': finished 2024-05-08T21:13:49.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:48 smithi071 ceph-mon[47074]: osdmap e19: 4 total, 3 up, 4 in 2024-05-08T21:13:49.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:13:49.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:13:50.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:49 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2322544281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:13:50.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T21:13:50.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:49 smithi082 ceph-mon[47512]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T21:13:50.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:13:50.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:49 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2322544281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:13:50.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T21:13:50.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:49 smithi033 ceph-mon[42528]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T21:13:50.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:13:50.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:49 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2322544281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:13:50.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T21:13:50.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:49 smithi071 ceph-mon[47074]: osdmap e20: 4 total, 3 up, 4 in 2024-05-08T21:13:50.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:13:50.458 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:13:50 smithi033 sudo[66964]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T21:13:50.458 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:13:50 smithi033 sudo[66964]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:13:50.458 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:13:50 smithi033 sudo[66964]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:13:50.458 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:13:50 smithi033 sudo[66964]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:13:50.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:50 smithi033 ceph-mon[42528]: pgmap v67: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:50.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:50 smithi033 ceph-mon[42528]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T21:13:50.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:13:50.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:50.959 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:13:50 smithi033 sudo[67155]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T21:13:50.959 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:13:50 smithi033 sudo[67155]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:13:50.959 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:13:50 smithi033 sudo[67155]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:13:50.959 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:13:50 smithi033 sudo[67155]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:13:51.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:50 smithi082 ceph-mon[47512]: pgmap v67: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:51.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:50 smithi082 ceph-mon[47512]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T21:13:51.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:13:51.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:51.165 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:50 smithi071 ceph-mon[47074]: pgmap v67: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:51.165 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:50 smithi071 ceph-mon[47074]: osdmap e21: 4 total, 3 up, 4 in 2024-05-08T21:13:51.165 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:13:51.165 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:51.166 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:13:50 smithi071 sudo[56859]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T21:13:51.166 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:13:50 smithi071 sudo[56859]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:13:51.166 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:13:50 smithi071 sudo[56859]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:13:51.166 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:13:50 smithi071 sudo[56859]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:13:51.571 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:13:51 smithi071 sudo[56862]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T21:13:51.571 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:13:51 smithi071 sudo[56862]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:13:51.571 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:13:51 smithi071 sudo[56862]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:13:51.571 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:13:51 smithi071 sudo[56862]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:13:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:13:51.688+0000 7f31297fa700 1 -- 172.21.15.33:0/1270110912 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3114096170 con 0x7f312c101060 2024-05-08T21:13:51.854 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:13:51 smithi082 sudo[58715]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T21:13:51.854 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:13:51 smithi082 sudo[58715]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:13:51.854 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:13:51 smithi082 sudo[58715]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:13:51.854 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:13:51 smithi082 sudo[58715]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:13:51.854 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:13:51 smithi082 sudo[58718]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T21:13:51.854 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:13:51 smithi082 sudo[58718]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:13:51.854 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:13:51 smithi082 sudo[58718]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:13:51.854 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:13:51 smithi082 sudo[58718]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:13:52.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:52 smithi033 sudo[67908]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T21:13:52.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:52 smithi033 sudo[67908]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:13:52.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:52 smithi033 sudo[67908]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:13:52.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:52 smithi033 sudo[67908]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:13:52.897 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:52.897 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:52 smithi082 ceph-mon[47512]: mgrmap e16: x(active, since 115s), standbys: z, y 2024-05-08T21:13:52.897 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:52.897 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:52 smithi082 ceph-mon[47512]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:13:52.897 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:52 smithi082 sudo[58721]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T21:13:52.897 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:52 smithi082 sudo[58721]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:13:52.897 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:52 smithi082 sudo[58721]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:13:52.897 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:52 smithi082 sudo[58721]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:13:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:52 smithi033 sudo[68074]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T21:13:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:52 smithi033 sudo[68074]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:13:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:52 smithi033 sudo[68074]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:13:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:52 smithi033 sudo[68074]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:13:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:52 smithi033 ceph-mon[42528]: mgrmap e16: x(active, since 115s), standbys: z, y 2024-05-08T21:13:52.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:52.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:52 smithi033 ceph-mon[42528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:13:53.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:53.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:52 smithi071 ceph-mon[47074]: mgrmap e16: x(active, since 115s), standbys: z, y 2024-05-08T21:13:53.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:53.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:52 smithi071 ceph-mon[47074]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:13:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 sudo[58724]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T21:13:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 sudo[58724]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:13:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 sudo[58724]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:13:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 sudo[58724]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:13:53.651 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 sudo[56865]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T21:13:53.652 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 sudo[56865]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:13:53.652 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 sudo[56865]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:13:53.652 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 sudo[56865]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:13:53.652 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 sudo[56868]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T21:13:53.652 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 sudo[56868]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T21:13:53.652 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 sudo[56868]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T21:13:53.652 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 sudo[56868]: pam_unix(sudo:session): session closed for user root 2024-05-08T21:13:53.951 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:53.951 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:13:53.951 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:13:53.951 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:13:53.951 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:13:53.951 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:13:53.951 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:13:53.951 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:13:53.951 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:13:53.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:13:53.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:13:53.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:13:53.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:13:53.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:13:53.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:53 smithi033 ceph-mon[42528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:13:54.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:13:54.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:53 smithi071 ceph-mon[47074]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:13:54.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: pgmap v69: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:54.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:13:54.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:13:54.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:13:54.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:13:54.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:13:54.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:13:54.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:13:54.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:13:54.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:13:54.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T21:13:54.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T21:13:54.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T21:13:54.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T21:13:54.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:53 smithi082 ceph-mon[47512]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T21:13:54.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:54 smithi033 ceph-mon[42528]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:55.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:54 smithi071 ceph-mon[47074]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:55.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:54 smithi082 ceph-mon[47512]: pgmap v70: 1 pgs: 1 creating+peering; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:55.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:55 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T21:13:55.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:55 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:55.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:55 smithi033 ceph-mon[42528]: Deploying daemon osd.3 on smithi033 2024-05-08T21:13:56.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:55 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T21:13:56.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:55 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:56.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:55 smithi071 ceph-mon[47074]: Deploying daemon osd.3 on smithi033 2024-05-08T21:13:56.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:55 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T21:13:56.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:55 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:13:56.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:55 smithi082 ceph-mon[47512]: Deploying daemon osd.3 on smithi033 2024-05-08T21:13:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:56 smithi033 ceph-mon[42528]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:57.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:56 smithi071 ceph-mon[47074]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:57.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:56 smithi082 ceph-mon[47512]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:59.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:59 smithi033 ceph-mon[42528]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:59.130 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:59.130 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:59.130 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:13:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:59.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:59 smithi071 ceph-mon[47074]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:59.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:59.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:59.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:13:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:13:59.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:59 smithi082 ceph-mon[47512]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:13:59.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:59.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:13:59.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:13:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:01.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.027+0000 7f31297fa700 1 -- 172.21.15.33:0/1270110912 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f312c02ce10 con 0x7f31180a1110 2024-05-08T21:14:01.031 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 3 on host 'smithi033' 2024-05-08T21:14:01.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.030+0000 7f3131de1700 1 -- 172.21.15.33:0/1270110912 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f31180a1110 msgr2=0x7f31180a35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:01.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.030+0000 7f3131de1700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f31180a1110 0x7f31180a35d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f311c00c010 tx=0x7f311c003720 comp rx=0 tx=0).stop 2024-05-08T21:14:01.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.030+0000 7f3131de1700 1 -- 172.21.15.33:0/1270110912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312c101060 msgr2=0x7f312c197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.030+0000 7f3131de1700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312c101060 0x7f312c197370 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f311400d950 tx=0x7f311400dc60 comp rx=0 tx=0).stop 2024-05-08T21:14:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.030+0000 7f3131de1700 1 -- 172.21.15.33:0/1270110912 shutdown_connections 2024-05-08T21:14:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.030+0000 7f3131de1700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f31180a1110 0x7f31180a35d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.030+0000 7f3131de1700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f312c101a20 0x7f312c19e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.030+0000 7f3131de1700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f312c0fff00 0x7f312c196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.030+0000 7f3131de1700 1 --2- 172.21.15.33:0/1270110912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312c101060 0x7f312c197370 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.030+0000 7f3131de1700 1 -- 172.21.15.33:0/1270110912 >> 172.21.15.33:0/1270110912 conn(0x7f312c0fb4a0 msgr2=0x7f312c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:14:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.030+0000 7f3131de1700 1 -- 172.21.15.33:0/1270110912 shutdown_connections 2024-05-08T21:14:01.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:01.030+0000 7f3131de1700 1 -- 172.21.15.33:0/1270110912 wait complete. 2024-05-08T21:14:01.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:01 smithi033 ceph-mon[42528]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:14:01.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:01.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:01.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:01.470 DEBUG:teuthology.orchestra.run.smithi033:osd.3> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.3.service 2024-05-08T21:14:01.472 INFO:tasks.cephadm:Deploying osd.4 on smithi071 with /dev/vg_nvme/lv_3... 2024-05-08T21:14:01.472 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T21:14:01.506 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:01 smithi071 ceph-mon[47074]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:14:01.506 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.506 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.506 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:01.506 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:01.506 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.506 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.506 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.506 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:01.508 INFO:journalctl@ceph.osd.3.smithi033.stdout:-- Logs begin at Wed 2024-05-08 20:53:21 UTC. -- 2024-05-08T21:14:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:01 smithi082 ceph-mon[47512]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:14:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:01.705 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:14:03.039 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T21:14:03.039 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T21:14:03.039 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-05-08T21:14:03.039 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-05-08T21:14:03.039 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112942 s, 928 MB/s 2024-05-08T21:14:03.039 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-05-08T21:14:03.140 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 08 21:14:02 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-3[68970]: 2024-05-08T21:14:02.837+0000 7fd6fcd367c0 -1 osd.3 0 log_to_monitors true 2024-05-08T21:14:03.404 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:03 smithi071 ceph-mon[47074]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:14:03.404 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:03 smithi071 ceph-mon[47074]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T21:14:03.405 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_3 2024-05-08T21:14:03.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:03 smithi033 ceph-mon[42528]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:14:03.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:03 smithi033 ceph-mon[42528]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T21:14:03.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:03 smithi082 ceph-mon[47512]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:14:03.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:03 smithi082 ceph-mon[47512]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T21:14:03.662 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:14:04.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:04 smithi071 ceph-mon[47074]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T21:14:04.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:04 smithi071 ceph-mon[47074]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T21:14:04.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:04 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:04.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:04 smithi071 ceph-mon[47074]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:14:04.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:04 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:04.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:04 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:04.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:04 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:04.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:04 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:04.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:04 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:04.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:04 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:04.445 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:04 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:04.445 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:04 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:04.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:04 smithi033 ceph-mon[42528]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T21:14:04.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:04 smithi033 ceph-mon[42528]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T21:14:04.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:04 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:04.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:04 smithi033 ceph-mon[42528]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:14:04.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:04 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:04.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:04 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:04.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:04 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:04.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:04 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:04.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:04 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:04.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:04 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:04.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:04 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:04.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:04 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:04.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:04 smithi082 ceph-mon[47512]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T21:14:04.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:04 smithi082 ceph-mon[47512]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T21:14:04.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:04 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:04.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:04 smithi082 ceph-mon[47512]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:14:04.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:04 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:04.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:04 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:04.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:04 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:04.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:04 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:04.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:04 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:04.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:04 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:04.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:04 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:04.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:04 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.602+0000 7fc2be4bb700 1 -- 172.21.15.71:0/3067655132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b8101a90 msgr2=0x7fc2b8103e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.602+0000 7fc2be4bb700 1 --2- 172.21.15.71:0/3067655132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b8101a90 0x7fc2b8103e80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009990 tx=0x7fc2ac009ca0 comp rx=0 tx=0).stop 2024-05-08T21:14:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.602+0000 7fc2be4bb700 1 -- 172.21.15.71:0/3067655132 shutdown_connections 2024-05-08T21:14:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.602+0000 7fc2be4bb700 1 --2- 172.21.15.71:0/3067655132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b8106cf0 0x7fc2b8109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.602+0000 7fc2be4bb700 1 --2- 172.21.15.71:0/3067655132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b8101a90 0x7fc2b8103e80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.602+0000 7fc2be4bb700 1 --2- 172.21.15.71:0/3067655132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b81043c0 0x7fc2b81067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.602+0000 7fc2be4bb700 1 -- 172.21.15.71:0/3067655132 >> 172.21.15.71:0/3067655132 conn(0x7fc2b80fb4a0 msgr2=0x7fc2b80fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:14:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.602+0000 7fc2be4bb700 1 -- 172.21.15.71:0/3067655132 shutdown_connections 2024-05-08T21:14:04.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.602+0000 7fc2be4bb700 1 -- 172.21.15.71:0/3067655132 wait complete. 2024-05-08T21:14:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2be4bb700 1 Processor -- start 2024-05-08T21:14:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2be4bb700 1 -- start start 2024-05-08T21:14:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2be4bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b8101a90 0x7fc2b8196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2be4bb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b81043c0 0x7fc2b8197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2be4bb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b8106cf0 0x7fc2b819e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2be4bb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b819e960 con 0x7fc2b8101a90 2024-05-08T21:14:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2be4bb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b819eaa0 con 0x7fc2b81043c0 2024-05-08T21:14:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2be4bb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b819ec30 con 0x7fc2b8106cf0 2024-05-08T21:14:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2b77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b81043c0 0x7fc2b8197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2b77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b81043c0 0x7fc2b8197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51836/0 (socket says 172.21.15.71:51836) 2024-05-08T21:14:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2b77fe700 1 -- 172.21.15.71:0/2284981339 learned_addr learned my addr 172.21.15.71:0/2284981339 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-05-08T21:14:04.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2b7fff700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b8101a90 0x7fc2b8196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2bca58700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b8106cf0 0x7fc2b819e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2b77fe700 1 -- 172.21.15.71:0/2284981339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b8106cf0 msgr2=0x7fc2b819e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2b77fe700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b8106cf0 0x7fc2b819e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2b77fe700 1 -- 172.21.15.71:0/2284981339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b8101a90 msgr2=0x7fc2b8196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2b77fe700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b8101a90 0x7fc2b8196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.603+0000 7fc2b77fe700 1 -- 172.21.15.71:0/2284981339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac0095f0 con 0x7fc2b81043c0 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.604+0000 7fc2b77fe700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b81043c0 0x7fc2b8197360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800d980 tx=0x7fc2a800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.604+0000 7fc2b57fa700 1 -- 172.21.15.71:0/2284981339 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8004220 con 0x7fc2b81043c0 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.604+0000 7fc2b57fa700 1 -- 172.21.15.71:0/2284981339 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2a8007640 con 0x7fc2b81043c0 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.604+0000 7fc2b57fa700 1 -- 172.21.15.71:0/2284981339 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a80055e0 con 0x7fc2b81043c0 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.604+0000 7fc2be4bb700 1 -- 172.21.15.71:0/2284981339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b819ef10 con 0x7fc2b81043c0 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.604+0000 7fc2be4bb700 1 -- 172.21.15.71:0/2284981339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b819f380 con 0x7fc2b81043c0 2024-05-08T21:14:04.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.605+0000 7fc2be4bb700 1 -- 172.21.15.71:0/2284981339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b80fd020 con 0x7fc2b81043c0 2024-05-08T21:14:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.606+0000 7fc2b57fa700 1 -- 172.21.15.71:0/2284981339 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc2a8004750 con 0x7fc2b81043c0 2024-05-08T21:14:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.606+0000 7fc2b57fa700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2a00a11a0 0x7fc2a00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.607+0000 7fc2b57fa700 1 -- 172.21.15.71:0/2284981339 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3389+0+0 (secure 0 0 0) 0x7fc2a80cb830 con 0x7fc2b81043c0 2024-05-08T21:14:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.607+0000 7fc2b7fff700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2a00a11a0 0x7fc2a00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:04.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.607+0000 7fc2b7fff700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2a00a11a0 0x7fc2a00a3660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005910 tx=0x7fc2ac00b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:14:04.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.610+0000 7fc2b57fa700 1 -- 172.21.15.71:0/2284981339 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc2a8096300 con 0x7fc2b81043c0 2024-05-08T21:14:04.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:04.809+0000 7fc2be4bb700 1 -- 172.21.15.71:0/2284981339 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2b802ceb0 con 0x7fc2a00a11a0 2024-05-08T21:14:05.208 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 08 21:14:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-3[68970]: 2024-05-08T21:14:04.920+0000 7fd6f239a700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T21:14:05.208 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 08 21:14:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-3[68970]: 2024-05-08T21:14:04.926+0000 7fd6ea187700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:14:05.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:05 smithi033 ceph-mon[42528]: Detected new or changed devices on smithi033 2024-05-08T21:14:05.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:05 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi033 to 8559M 2024-05-08T21:14:05.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:05 smithi033 ceph-mon[42528]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:14:05.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:05 smithi033 ceph-mon[42528]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-08T21:14:05.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:05 smithi033 ceph-mon[42528]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T21:14:05.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:05.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:05.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:14:05.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:14:05.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:05.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:05 smithi033 ceph-mon[42528]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' 2024-05-08T21:14:05.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:05 smithi071 ceph-mon[47074]: Detected new or changed devices on smithi033 2024-05-08T21:14:05.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:05 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi033 to 8559M 2024-05-08T21:14:05.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:05 smithi071 ceph-mon[47074]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:14:05.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:05 smithi071 ceph-mon[47074]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-08T21:14:05.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:05 smithi071 ceph-mon[47074]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T21:14:05.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:05.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:05.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:14:05.443 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:14:05.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:05.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:05 smithi071 ceph-mon[47074]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' 2024-05-08T21:14:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:05 smithi082 ceph-mon[47512]: Detected new or changed devices on smithi033 2024-05-08T21:14:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:05 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi033 to 8559M 2024-05-08T21:14:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:05 smithi082 ceph-mon[47512]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:14:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:05 smithi082 ceph-mon[47512]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-08T21:14:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:05 smithi082 ceph-mon[47512]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T21:14:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:14:05.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:14:05.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:05.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:05 smithi082 ceph-mon[47512]: from='osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799]' entity='osd.3' 2024-05-08T21:14:06.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:06 smithi033 ceph-mon[42528]: purged_snaps scrub starts 2024-05-08T21:14:06.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:06 smithi033 ceph-mon[42528]: purged_snaps scrub ok 2024-05-08T21:14:06.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:06 smithi033 ceph-mon[42528]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:14:06.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:06 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:06.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:06 smithi033 ceph-mon[42528]: osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799] boot 2024-05-08T21:14:06.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:06 smithi033 ceph-mon[42528]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T21:14:06.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:06 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:06.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:06 smithi033 ceph-mon[42528]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T21:14:06.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:06 smithi071 ceph-mon[47074]: purged_snaps scrub starts 2024-05-08T21:14:06.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:06 smithi071 ceph-mon[47074]: purged_snaps scrub ok 2024-05-08T21:14:06.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:06 smithi071 ceph-mon[47074]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:14:06.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:06 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:06.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:06 smithi071 ceph-mon[47074]: osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799] boot 2024-05-08T21:14:06.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:06 smithi071 ceph-mon[47074]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T21:14:06.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:06 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:06.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:06 smithi071 ceph-mon[47074]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T21:14:06.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:06 smithi082 ceph-mon[47512]: purged_snaps scrub starts 2024-05-08T21:14:06.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:06 smithi082 ceph-mon[47512]: purged_snaps scrub ok 2024-05-08T21:14:06.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:06 smithi082 ceph-mon[47512]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:14:06.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:06 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:06.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:06 smithi082 ceph-mon[47512]: osd.3 [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799] boot 2024-05-08T21:14:06.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:06 smithi082 ceph-mon[47512]: osdmap e24: 4 total, 4 up, 4 in 2024-05-08T21:14:06.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:06 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T21:14:06.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:06 smithi082 ceph-mon[47512]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T21:14:07.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:07 smithi033 ceph-mon[42528]: pgmap v79: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:14:07.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:07 smithi033 ceph-mon[42528]: from='client.? 172.21.15.71:0/2438424396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a8fee3-514c-4bea-b364-566a8664ee55"}]: dispatch 2024-05-08T21:14:07.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:07 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a8fee3-514c-4bea-b364-566a8664ee55"}]: dispatch 2024-05-08T21:14:07.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:07 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21a8fee3-514c-4bea-b364-566a8664ee55"}]': finished 2024-05-08T21:14:07.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:07 smithi033 ceph-mon[42528]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T21:14:07.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:07 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:07.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:07 smithi071 ceph-mon[47074]: pgmap v79: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:14:07.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:07 smithi071 ceph-mon[47074]: from='client.? 172.21.15.71:0/2438424396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a8fee3-514c-4bea-b364-566a8664ee55"}]: dispatch 2024-05-08T21:14:07.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:07 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a8fee3-514c-4bea-b364-566a8664ee55"}]: dispatch 2024-05-08T21:14:07.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:07 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21a8fee3-514c-4bea-b364-566a8664ee55"}]': finished 2024-05-08T21:14:07.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:07 smithi071 ceph-mon[47074]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T21:14:07.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:07 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:07 smithi082 ceph-mon[47512]: pgmap v79: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-08T21:14:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:07 smithi082 ceph-mon[47512]: from='client.? 172.21.15.71:0/2438424396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a8fee3-514c-4bea-b364-566a8664ee55"}]: dispatch 2024-05-08T21:14:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:07 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21a8fee3-514c-4bea-b364-566a8664ee55"}]: dispatch 2024-05-08T21:14:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:07 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21a8fee3-514c-4bea-b364-566a8664ee55"}]': finished 2024-05-08T21:14:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:07 smithi082 ceph-mon[47512]: osdmap e26: 5 total, 4 up, 5 in 2024-05-08T21:14:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:07 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:08 smithi033 ceph-mon[42528]: from='client.? 172.21.15.71:0/2123418449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:14:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:08 smithi033 ceph-mon[42528]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T21:14:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:08 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:08.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:08 smithi071 ceph-mon[47074]: from='client.? 172.21.15.71:0/2123418449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:14:08.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:08 smithi071 ceph-mon[47074]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T21:14:08.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:08 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:08.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:08 smithi082 ceph-mon[47512]: from='client.? 172.21.15.71:0/2123418449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:14:08.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:08 smithi082 ceph-mon[47512]: osdmap e27: 5 total, 4 up, 5 in 2024-05-08T21:14:08.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:08 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:09.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:09 smithi033 ceph-mon[42528]: pgmap v83: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:14:09.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:09 smithi033 ceph-mon[42528]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T21:14:09.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:09 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:09.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:09 smithi071 ceph-mon[47074]: pgmap v83: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:14:09.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:09 smithi071 ceph-mon[47074]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T21:14:09.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:09 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:09.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:09 smithi082 ceph-mon[47512]: pgmap v83: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:14:09.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:09 smithi082 ceph-mon[47512]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T21:14:09.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:09 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:11.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:11 smithi071 ceph-mon[47074]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T21:14:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:11 smithi033 ceph-mon[42528]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T21:14:11.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:11 smithi082 ceph-mon[47512]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T21:14:13.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:13 smithi033 ceph-mon[42528]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-08T21:14:13.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T21:14:13.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:13.549 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:13 smithi071 ceph-mon[47074]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-08T21:14:13.550 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T21:14:13.550 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:13.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:13 smithi082 ceph-mon[47512]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-08T21:14:13.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T21:14:13.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:14.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:14 smithi033 ceph-mon[42528]: Deploying daemon osd.4 on smithi071 2024-05-08T21:14:14.534 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:14 smithi071 ceph-mon[47074]: Deploying daemon osd.4 on smithi071 2024-05-08T21:14:14.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:14 smithi082 ceph-mon[47512]: Deploying daemon osd.4 on smithi071 2024-05-08T21:14:15.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:15 smithi033 ceph-mon[42528]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-08T21:14:15.541 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:15 smithi071 ceph-mon[47074]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-08T21:14:15.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:15 smithi082 ceph-mon[47512]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-08T21:14:17.277 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:17 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:17.277 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:17 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:17.277 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:17 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:17.277 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:17 smithi071 ceph-mon[47074]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T21:14:17.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:17 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:17.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:17 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:17.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:17 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:17.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:17 smithi082 ceph-mon[47512]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T21:14:17.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:17 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:17.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:17 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:17.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:17 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:17.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:17 smithi033 ceph-mon[42528]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T21:14:18.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.166+0000 7fc2b57fa700 1 -- 172.21.15.71:0/2284981339 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc2b802ceb0 con 0x7fc2a00a11a0 2024-05-08T21:14:18.170 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 4 on host 'smithi071' 2024-05-08T21:14:18.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.169+0000 7fc2be4bb700 1 -- 172.21.15.71:0/2284981339 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2a00a11a0 msgr2=0x7fc2a00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:18.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.169+0000 7fc2be4bb700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2a00a11a0 0x7fc2a00a3660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005910 tx=0x7fc2ac00b610 comp rx=0 tx=0).stop 2024-05-08T21:14:18.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.169+0000 7fc2be4bb700 1 -- 172.21.15.71:0/2284981339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b81043c0 msgr2=0x7fc2b8197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:18.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.169+0000 7fc2be4bb700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b81043c0 0x7fc2b8197360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800d980 tx=0x7fc2a800dc90 comp rx=0 tx=0).stop 2024-05-08T21:14:18.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.169+0000 7fc2be4bb700 1 -- 172.21.15.71:0/2284981339 shutdown_connections 2024-05-08T21:14:18.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.169+0000 7fc2be4bb700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2b8106cf0 0x7fc2b819e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:18.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.169+0000 7fc2be4bb700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2b8101a90 0x7fc2b8196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:18.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.169+0000 7fc2be4bb700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2a00a11a0 0x7fc2a00a3660 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:18.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.169+0000 7fc2be4bb700 1 --2- 172.21.15.71:0/2284981339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b81043c0 0x7fc2b8197360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:18.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.169+0000 7fc2be4bb700 1 -- 172.21.15.71:0/2284981339 >> 172.21.15.71:0/2284981339 conn(0x7fc2b80fb4a0 msgr2=0x7fc2b8103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:14:18.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.169+0000 7fc2be4bb700 1 -- 172.21.15.71:0/2284981339 shutdown_connections 2024-05-08T21:14:18.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:18.169+0000 7fc2be4bb700 1 -- 172.21.15.71:0/2284981339 wait complete. 2024-05-08T21:14:18.499 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.499 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.499 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:18.500 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:18.500 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.500 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.500 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.500 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:18.557 DEBUG:teuthology.orchestra.run.smithi071:osd.4> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.4.service 2024-05-08T21:14:18.559 INFO:tasks.cephadm:Deploying osd.5 on smithi082 with /dev/vg_nvme/lv_3... 2024-05-08T21:14:18.559 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T21:14:18.590 INFO:journalctl@ceph.osd.4.smithi071.stdout:-- Logs begin at Wed 2024-05-08 20:53:45 UTC. -- 2024-05-08T21:14:18.591 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.592 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.592 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:18.592 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:18.592 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.592 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.592 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.592 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:18.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:18.791 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:14:19.394 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:19 smithi071 ceph-mon[47074]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T21:14:19.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:19 smithi082 ceph-mon[47512]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T21:14:19.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:19 smithi033 ceph-mon[42528]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T21:14:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T21:14:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T21:14:20.147 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-05-08T21:14:20.147 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-05-08T21:14:20.148 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112012 s, 936 MB/s 2024-05-08T21:14:20.148 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-05-08T21:14:20.321 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 08 21:14:20 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-4[59968]: 2024-05-08T21:14:20.054+0000 7f30202847c0 -1 osd.4 0 log_to_monitors true 2024-05-08T21:14:20.467 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:20 smithi082 ceph-mon[47512]: from='osd.4 [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T21:14:20.467 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:20 smithi082 ceph-mon[47512]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T21:14:20.485 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_3 2024-05-08T21:14:20.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:20 smithi033 ceph-mon[42528]: from='osd.4 [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T21:14:20.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:20 smithi033 ceph-mon[42528]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T21:14:20.741 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:14:20.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:20 smithi071 ceph-mon[47074]: from='osd.4 [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T21:14:20.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:20 smithi071 ceph-mon[47074]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T21:14:21.590 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T21:14:21.590 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T21:14:21.590 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T21:14:21.590 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:21.590 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: from='osd.4 [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:14:21.591 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:14:21.591 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:21.591 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:21.591 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:21.591 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:21.591 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:21.591 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:21.591 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:21.591 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:21 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:21.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T21:14:21.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T21:14:21.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T21:14:21.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:21.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: from='osd.4 [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:14:21.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:14:21.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:21.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:21.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:21.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:21.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:21.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:21.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:21.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:21 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:21.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.755+0000 7f3bc37c2700 1 -- 172.21.15.82:0/3809730756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc0fff00 msgr2=0x7f3bbc100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:21.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.755+0000 7f3bc37c2700 1 --2- 172.21.15.82:0/3809730756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc0fff00 0x7f3bbc100320 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009990 tx=0x7f3bb8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:14:21.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.755+0000 7f3bc37c2700 1 -- 172.21.15.82:0/3809730756 shutdown_connections 2024-05-08T21:14:21.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.755+0000 7f3bc37c2700 1 --2- 172.21.15.82:0/3809730756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bbc101a20 0x7f3bbc108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:21.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.755+0000 7f3bc37c2700 1 --2- 172.21.15.82:0/3809730756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc0fff00 0x7f3bbc100320 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:21.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.755+0000 7f3bc37c2700 1 --2- 172.21.15.82:0/3809730756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bbc101060 0x7f3bbc1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:21.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.755+0000 7f3bc37c2700 1 -- 172.21.15.82:0/3809730756 >> 172.21.15.82:0/3809730756 conn(0x7f3bbc0fb4a0 msgr2=0x7f3bbc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:14:21.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.755+0000 7f3bc37c2700 1 -- 172.21.15.82:0/3809730756 shutdown_connections 2024-05-08T21:14:21.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.755+0000 7f3bc37c2700 1 -- 172.21.15.82:0/3809730756 wait complete. 2024-05-08T21:14:21.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.755+0000 7f3bc37c2700 1 Processor -- start 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.755+0000 7f3bc37c2700 1 -- start start 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc37c2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bbc0fff00 0x7f3bbc196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc37c2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bbc101060 0x7f3bbc197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc37c2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc101a20 0x7f3bbc19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc37c2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc19ea10 con 0x7f3bbc101060 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc37c2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc19eb50 con 0x7f3bbc0fff00 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc37c2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc19ece0 con 0x7f3bbc101a20 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc1d5f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc101a20 0x7f3bbc19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc1d5f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc101a20 0x7f3bbc19e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43126/0 (socket says 172.21.15.82:43126) 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc1d5f700 1 -- 172.21.15.82:0/2307407321 learned_addr learned my addr 172.21.15.82:0/2307407321 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc155e700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bbc0fff00 0x7f3bbc196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc0d5d700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bbc101060 0x7f3bbc197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:21.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc1d5f700 1 -- 172.21.15.82:0/2307407321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bbc0fff00 msgr2=0x7f3bbc196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:21.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc1d5f700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bbc0fff00 0x7f3bbc196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:21.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc1d5f700 1 -- 172.21.15.82:0/2307407321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bbc101060 msgr2=0x7f3bbc197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:21.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc1d5f700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bbc101060 0x7f3bbc197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:21.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc1d5f700 1 -- 172.21.15.82:0/2307407321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb80095f0 con 0x7f3bbc101a20 2024-05-08T21:14:21.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc155e700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bbc0fff00 0x7f3bbc196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:14:21.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc1d5f700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc101a20 0x7f3bbc19e440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400b9a0 tx=0x7f3bb400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:14:21.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bb27fc700 1 -- 172.21.15.82:0/2307407321 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb400c9b0 con 0x7f3bbc101a20 2024-05-08T21:14:21.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bb27fc700 1 -- 172.21.15.82:0/2307407321 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bb4014440 con 0x7f3bbc101a20 2024-05-08T21:14:21.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bb27fc700 1 -- 172.21.15.82:0/2307407321 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb40075d0 con 0x7f3bbc101a20 2024-05-08T21:14:21.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc37c2700 1 -- 172.21.15.82:0/2307407321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc19efc0 con 0x7f3bbc101a20 2024-05-08T21:14:21.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.756+0000 7f3bc37c2700 1 -- 172.21.15.82:0/2307407321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc19f3e0 con 0x7f3bbc101a20 2024-05-08T21:14:21.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.757+0000 7f3bc37c2700 1 -- 172.21.15.82:0/2307407321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc04ec60 con 0x7f3bbc101a20 2024-05-08T21:14:21.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.758+0000 7f3bb27fc700 1 -- 172.21.15.82:0/2307407321 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3bb40145b0 con 0x7f3bbc101a20 2024-05-08T21:14:21.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.759+0000 7f3bb27fc700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ba80a1120 0x7f3ba80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:21.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.759+0000 7f3bb27fc700 1 -- 172.21.15.82:0/2307407321 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3821+0+0 (secure 0 0 0) 0x7f3bb4006e40 con 0x7f3bbc101a20 2024-05-08T21:14:21.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.759+0000 7f3bc155e700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ba80a1120 0x7f3ba80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:21.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.760+0000 7f3bc155e700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ba80a1120 0x7f3ba80a35e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800b5c0 tx=0x7f3bb8003750 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:14:21.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.763+0000 7f3bb27fc700 1 -- 172.21.15.82:0/2307407321 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3bb4096120 con 0x7f3bbc101a20 2024-05-08T21:14:21.821 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 08 21:14:21 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-4[59968]: 2024-05-08T21:14:21.336+0000 7f30158e8700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T21:14:21.821 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 08 21:14:21 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-4[59968]: 2024-05-08T21:14:21.344+0000 7f30116dd700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:14:21.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-08T21:14:21.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T21:14:21.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T21:14:21.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:21.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: from='osd.4 [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:14:21.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:14:21.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:21.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:21.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:21.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:21.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:21.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:21.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:21.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:21 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:21.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:21.964+0000 7f3bc37c2700 1 -- 172.21.15.82:0/2307407321 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bbc069090 con 0x7f3ba80a1120 2024-05-08T21:14:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:22 smithi082 ceph-mon[47512]: Detected new or changed devices on smithi071 2024-05-08T21:14:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:22 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi071 to 8556M 2024-05-08T21:14:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:22 smithi082 ceph-mon[47512]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-08T21:14:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:22 smithi082 ceph-mon[47512]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T21:14:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:22 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:22 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:22 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:14:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:22 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:14:22.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:22 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:22.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:22 smithi033 ceph-mon[42528]: Detected new or changed devices on smithi071 2024-05-08T21:14:22.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:22 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi071 to 8556M 2024-05-08T21:14:22.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:22 smithi033 ceph-mon[42528]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-08T21:14:22.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:22 smithi033 ceph-mon[42528]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T21:14:22.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:22 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:22.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:22 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:22.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:22 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:14:22.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:22 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:14:22.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:22 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:22.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:22 smithi071 ceph-mon[47074]: Detected new or changed devices on smithi071 2024-05-08T21:14:22.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:22 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi071 to 8556M 2024-05-08T21:14:22.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:22 smithi071 ceph-mon[47074]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-08T21:14:22.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:22 smithi071 ceph-mon[47074]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T21:14:22.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:22 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:22.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:22 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:22.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:22 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:14:22.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:22 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:14:22.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:22 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:23.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:23 smithi082 ceph-mon[47512]: purged_snaps scrub starts 2024-05-08T21:14:23.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:23 smithi082 ceph-mon[47512]: purged_snaps scrub ok 2024-05-08T21:14:23.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:23 smithi082 ceph-mon[47512]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:14:23.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:23 smithi082 ceph-mon[47512]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:14:23.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:23 smithi082 ceph-mon[47512]: osd.4 [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764] boot 2024-05-08T21:14:23.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:23 smithi082 ceph-mon[47512]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T21:14:23.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:23 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:23.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:23 smithi033 ceph-mon[42528]: purged_snaps scrub starts 2024-05-08T21:14:23.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:23 smithi033 ceph-mon[42528]: purged_snaps scrub ok 2024-05-08T21:14:23.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:23 smithi033 ceph-mon[42528]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:14:23.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:23 smithi033 ceph-mon[42528]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:14:23.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:23 smithi033 ceph-mon[42528]: osd.4 [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764] boot 2024-05-08T21:14:23.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:23 smithi033 ceph-mon[42528]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T21:14:23.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:23 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:23.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:23 smithi071 ceph-mon[47074]: purged_snaps scrub starts 2024-05-08T21:14:23.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:23 smithi071 ceph-mon[47074]: purged_snaps scrub ok 2024-05-08T21:14:23.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:23 smithi071 ceph-mon[47074]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:14:23.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:23 smithi071 ceph-mon[47074]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-05-08T21:14:23.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:23 smithi071 ceph-mon[47074]: osd.4 [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764] boot 2024-05-08T21:14:23.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:23 smithi071 ceph-mon[47074]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T21:14:23.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:23 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T21:14:24.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:24 smithi082 ceph-mon[47512]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T21:14:24.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:24 smithi082 ceph-mon[47512]: from='client.? 172.21.15.82:0/3103075940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7669410f-f67f-41a0-9b5b-3cc04a4ab45b"}]: dispatch 2024-05-08T21:14:24.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:24 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7669410f-f67f-41a0-9b5b-3cc04a4ab45b"}]: dispatch 2024-05-08T21:14:24.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:24 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7669410f-f67f-41a0-9b5b-3cc04a4ab45b"}]': finished 2024-05-08T21:14:24.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:24 smithi082 ceph-mon[47512]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T21:14:24.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:24.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:24 smithi033 ceph-mon[42528]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T21:14:24.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:24 smithi033 ceph-mon[42528]: from='client.? 172.21.15.82:0/3103075940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7669410f-f67f-41a0-9b5b-3cc04a4ab45b"}]: dispatch 2024-05-08T21:14:24.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:24 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7669410f-f67f-41a0-9b5b-3cc04a4ab45b"}]: dispatch 2024-05-08T21:14:24.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:24 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7669410f-f67f-41a0-9b5b-3cc04a4ab45b"}]': finished 2024-05-08T21:14:24.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:24 smithi033 ceph-mon[42528]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T21:14:24.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:24.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:24 smithi071 ceph-mon[47074]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T21:14:24.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:24 smithi071 ceph-mon[47074]: from='client.? 172.21.15.82:0/3103075940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7669410f-f67f-41a0-9b5b-3cc04a4ab45b"}]: dispatch 2024-05-08T21:14:24.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:24 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7669410f-f67f-41a0-9b5b-3cc04a4ab45b"}]: dispatch 2024-05-08T21:14:24.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:24 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7669410f-f67f-41a0-9b5b-3cc04a4ab45b"}]': finished 2024-05-08T21:14:24.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:24 smithi071 ceph-mon[47074]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T21:14:24.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:25.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:25 smithi033 ceph-mon[42528]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:25.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:25 smithi033 ceph-mon[42528]: from='client.? 172.21.15.82:0/677254295' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:14:25.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:25 smithi071 ceph-mon[47074]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:25.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:25 smithi071 ceph-mon[47074]: from='client.? 172.21.15.82:0/677254295' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:14:25.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:25 smithi082 ceph-mon[47512]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:25.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:25 smithi082 ceph-mon[47512]: from='client.? 172.21.15.82:0/677254295' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:14:27.704 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:27 smithi082 ceph-mon[47512]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:27.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:27 smithi033 ceph-mon[42528]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:27.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:27 smithi071 ceph-mon[47074]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:29.673 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:29 smithi082 ceph-mon[47512]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:29 smithi033 ceph-mon[42528]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:29.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:29 smithi071 ceph-mon[47074]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:30.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:30 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T21:14:30.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:30 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:30.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:30 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T21:14:30.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:30 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:30.836 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:30 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T21:14:30.836 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:30 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:31.423 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:31 smithi082 ceph-mon[47512]: Deploying daemon osd.5 on smithi082 2024-05-08T21:14:31.423 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:31 smithi082 ceph-mon[47512]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:31 smithi033 ceph-mon[42528]: Deploying daemon osd.5 on smithi082 2024-05-08T21:14:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:31 smithi033 ceph-mon[42528]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:31 smithi071 ceph-mon[47074]: Deploying daemon osd.5 on smithi082 2024-05-08T21:14:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:31 smithi071 ceph-mon[47074]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:33.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:33 smithi082 ceph-mon[47512]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:33.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:33 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:33.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:33 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:33.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:33 smithi033 ceph-mon[42528]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:33.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:33 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:33.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:33 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:33.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:33 smithi071 ceph-mon[47074]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:33.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:33 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:33.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:33 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:34.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:34 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:34.711 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:34 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:34.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:35.572 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:35 smithi082 ceph-mon[47512]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:35.572 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:35 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:35.572 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:35 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:35.572 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:35 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:35.572 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:35 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:35.573 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:35 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:35.673 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 5 on host 'smithi082' 2024-05-08T21:14:35.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.670+0000 7f3bb27fc700 1 -- 172.21.15.82:0/2307407321 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3bbc069090 con 0x7f3ba80a1120 2024-05-08T21:14:35.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.672+0000 7f3bc37c2700 1 -- 172.21.15.82:0/2307407321 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ba80a1120 msgr2=0x7f3ba80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:35.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.672+0000 7f3bc37c2700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ba80a1120 0x7f3ba80a35e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800b5c0 tx=0x7f3bb8003750 comp rx=0 tx=0).stop 2024-05-08T21:14:35.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.672+0000 7f3bc37c2700 1 -- 172.21.15.82:0/2307407321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc101a20 msgr2=0x7f3bbc19e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:35.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.672+0000 7f3bc37c2700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc101a20 0x7f3bbc19e440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400b9a0 tx=0x7f3bb400bd60 comp rx=0 tx=0).stop 2024-05-08T21:14:35.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.672+0000 7f3bc37c2700 1 -- 172.21.15.82:0/2307407321 shutdown_connections 2024-05-08T21:14:35.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.672+0000 7f3bc37c2700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc101a20 0x7f3bbc19e440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:35.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.672+0000 7f3bc37c2700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ba80a1120 0x7f3ba80a35e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:35.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.672+0000 7f3bc37c2700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bbc0fff00 0x7f3bbc196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:35.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.672+0000 7f3bc37c2700 1 --2- 172.21.15.82:0/2307407321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bbc101060 0x7f3bbc197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:35.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.672+0000 7f3bc37c2700 1 -- 172.21.15.82:0/2307407321 >> 172.21.15.82:0/2307407321 conn(0x7f3bbc0fb4a0 msgr2=0x7f3bbc068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:14:35.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.673+0000 7f3bc37c2700 1 -- 172.21.15.82:0/2307407321 shutdown_connections 2024-05-08T21:14:35.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:14:35.673+0000 7f3bc37c2700 1 -- 172.21.15.82:0/2307407321 wait complete. 2024-05-08T21:14:35.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:35 smithi033 ceph-mon[42528]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:35.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:35 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:35.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:35 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:35.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:35 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:35.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:35 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:35.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:35 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:35.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:35 smithi071 ceph-mon[47074]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:35.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:35 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:35.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:35 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:35.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:35 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:35.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:35 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:35.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:35 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:36.146 DEBUG:teuthology.orchestra.run.smithi082:osd.5> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.5.service 2024-05-08T21:14:36.149 INFO:tasks.cephadm:Deploying osd.6 on smithi033 with /dev/vg_nvme/lv_2... 2024-05-08T21:14:36.149 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T21:14:36.179 INFO:journalctl@ceph.osd.5.smithi082.stdout:-- Logs begin at Wed 2024-05-08 20:53:53 UTC. -- 2024-05-08T21:14:36.403 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:14:36.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:36 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:36.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:36 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:36.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:36 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:36.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:36 smithi082 ceph-mon[47512]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:36.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:36 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:36.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:36 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:36.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:36 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:36.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:36 smithi033 ceph-mon[42528]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:37.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:36 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:37.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:36 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:37.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:36 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:37.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:36 smithi071 ceph-mon[47074]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:37.604 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:14:37 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:14:37.346+0000 7fc5c96657c0 -1 osd.5 0 log_to_monitors true 2024-05-08T21:14:37.938 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T21:14:37.939 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T21:14:37.939 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-08T21:14:37.939 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-08T21:14:37.939 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131099 s, 800 MB/s 2024-05-08T21:14:37.939 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-08T21:14:37.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:37 smithi033 ceph-mon[42528]: from='osd.5 [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T21:14:37.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:37 smithi033 ceph-mon[42528]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T21:14:37.977 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:37 smithi082 ceph-mon[47512]: from='osd.5 [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T21:14:37.977 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:37 smithi082 ceph-mon[47512]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T21:14:38.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:37 smithi071 ceph-mon[47074]: from='osd.5 [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T21:14:38.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:37 smithi071 ceph-mon[47074]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T21:14:38.296 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_2 2024-05-08T21:14:38.544 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:14:38.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T21:14:38.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T21:14:38.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:38.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: from='osd.5 [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:14:38.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:14:38.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:38.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: Detected new or changed devices on smithi082 2024-05-08T21:14:38.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:38.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:38.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:38.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:38.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi082 to 8556M 2024-05-08T21:14:38.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:38.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:38.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:38.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:39.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: from='osd.5 [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: Detected new or changed devices on smithi082 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi082 to 8556M 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:39.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:39.073 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: from='osd.5 [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: Detected new or changed devices on smithi082 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi082 to 8556M 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:39.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:39.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:39.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:39.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.531+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/2736210272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e28102200 msgr2=0x7f4e28102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:39.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.531+0000 7f4e2ddc8700 1 --2- 172.21.15.33:0/2736210272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e28102200 0x7f4e28102660 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4e180099c0 tx=0x7f4e18009cd0 comp rx=0 tx=0).stop 2024-05-08T21:14:39.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.531+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/2736210272 shutdown_connections 2024-05-08T21:14:39.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.531+0000 7f4e2ddc8700 1 --2- 172.21.15.33:0/2736210272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e28102ba0 0x7f4e28109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:39.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.531+0000 7f4e2ddc8700 1 --2- 172.21.15.33:0/2736210272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e28101000 0x7f4e28101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:39.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.531+0000 7f4e2ddc8700 1 --2- 172.21.15.33:0/2736210272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e28102200 0x7f4e28102660 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:39.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.531+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/2736210272 >> 172.21.15.33:0/2736210272 conn(0x7f4e280fc5c0 msgr2=0x7f4e280fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:14:39.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.531+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/2736210272 shutdown_connections 2024-05-08T21:14:39.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.532+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/2736210272 wait complete. 2024-05-08T21:14:39.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.532+0000 7f4e2ddc8700 1 Processor -- start 2024-05-08T21:14:39.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.532+0000 7f4e2ddc8700 1 -- start start 2024-05-08T21:14:39.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e2ddc8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e28101000 0x7f4e28196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e2ddc8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e28102200 0x7f4e28197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e2ddc8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e28102ba0 0x7f4e2819e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e2ddc8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819e960 con 0x7f4e28101000 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e2ddc8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819eaa0 con 0x7f4e28102ba0 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e2ddc8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2819ec30 con 0x7f4e28102200 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e28101000 0x7f4e28196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e277fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e28101000 0x7f4e28196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59808/0 (socket says 172.21.15.33:59808) 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e277fe700 1 -- 172.21.15.33:0/3461456496 learned_addr learned my addr 172.21.15.33:0/3461456496 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e26ffd700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e28102200 0x7f4e28197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e27fff700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e28102ba0 0x7f4e2819e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e277fe700 1 -- 172.21.15.33:0/3461456496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e28102200 msgr2=0x7f4e28197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e277fe700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e28102200 0x7f4e28197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:39.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e277fe700 1 -- 172.21.15.33:0/3461456496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e28102ba0 msgr2=0x7f4e2819e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:39.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e277fe700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e28102ba0 0x7f4e2819e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:39.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e277fe700 1 -- 172.21.15.33:0/3461456496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e180095f0 con 0x7f4e28101000 2024-05-08T21:14:39.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e27fff700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e28102ba0 0x7f4e2819e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:14:39.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e277fe700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e28101000 0x7f4e28196e20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000d9b0 tx=0x7f4e1000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:14:39.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e24ff9700 1 -- 172.21.15.33:0/3461456496 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000b950 con 0x7f4e28101000 2024-05-08T21:14:39.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.533+0000 7f4e24ff9700 1 -- 172.21.15.33:0/3461456496 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e10007620 con 0x7f4e28101000 2024-05-08T21:14:39.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.534+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/3461456496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2819ef10 con 0x7f4e28101000 2024-05-08T21:14:39.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.534+0000 7f4e24ff9700 1 -- 172.21.15.33:0/3461456496 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000f3c0 con 0x7f4e28101000 2024-05-08T21:14:39.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.534+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/3461456496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2819f320 con 0x7f4e28101000 2024-05-08T21:14:39.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.535+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/3461456496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e28108480 con 0x7f4e28101000 2024-05-08T21:14:39.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.536+0000 7f4e24ff9700 1 -- 172.21.15.33:0/3461456496 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4e1000bab0 con 0x7f4e28101000 2024-05-08T21:14:39.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.536+0000 7f4e24ff9700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e140a1120 0x7f4e140a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:39.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.537+0000 7f4e26ffd700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e140a1120 0x7f4e140a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:39.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.537+0000 7f4e24ff9700 1 -- 172.21.15.33:0/3461456496 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4253+0+0 (secure 0 0 0) 0x7f4e100cc040 con 0x7f4e28101000 2024-05-08T21:14:39.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.537+0000 7f4e26ffd700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e140a1120 0x7f4e140a35e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009990 tx=0x7f4e18003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:14:39.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.541+0000 7f4e24ff9700 1 -- 172.21.15.33:0/3461456496 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4e10096710 con 0x7f4e28101000 2024-05-08T21:14:39.604 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:14:39 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:14:39.332+0000 7fc5becc9700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T21:14:39.604 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:14:39 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:14:39.342+0000 7fc5b6ab6700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:14:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:39.765+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/3461456496 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e2802ceb0 con 0x7f4e140a1120 2024-05-08T21:14:39.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:39 smithi033 ceph-mon[42528]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-08T21:14:39.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:39 smithi033 ceph-mon[42528]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T21:14:39.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:39.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:39.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:39 smithi033 ceph-mon[42528]: from='osd.5 ' entity='osd.5' 2024-05-08T21:14:39.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:40.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:39 smithi071 ceph-mon[47074]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-08T21:14:40.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:39 smithi071 ceph-mon[47074]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T21:14:40.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:39 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:40.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:39 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:40.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:39 smithi071 ceph-mon[47074]: from='osd.5 ' entity='osd.5' 2024-05-08T21:14:40.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:39 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:40.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:39 smithi082 ceph-mon[47512]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-08T21:14:40.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:39 smithi082 ceph-mon[47512]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T21:14:40.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:40.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:40.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:39 smithi082 ceph-mon[47512]: from='osd.5 ' entity='osd.5' 2024-05-08T21:14:40.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:40.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:40 smithi033 ceph-mon[42528]: purged_snaps scrub starts 2024-05-08T21:14:40.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:40 smithi033 ceph-mon[42528]: purged_snaps scrub ok 2024-05-08T21:14:40.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:40 smithi033 ceph-mon[42528]: osd.5 [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719] boot 2024-05-08T21:14:40.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:40 smithi033 ceph-mon[42528]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T21:14:40.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:40.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:40 smithi033 ceph-mon[42528]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:14:40.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:14:40.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:14:40.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:40.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:40 smithi033 ceph-mon[42528]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:41.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:40 smithi071 ceph-mon[47074]: purged_snaps scrub starts 2024-05-08T21:14:41.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:40 smithi071 ceph-mon[47074]: purged_snaps scrub ok 2024-05-08T21:14:41.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:40 smithi071 ceph-mon[47074]: osd.5 [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719] boot 2024-05-08T21:14:41.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:40 smithi071 ceph-mon[47074]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T21:14:41.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:40 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:41.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:40 smithi071 ceph-mon[47074]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:14:41.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:40 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:14:41.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:40 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:14:41.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:40 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:41.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:40 smithi071 ceph-mon[47074]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:40 smithi082 ceph-mon[47512]: purged_snaps scrub starts 2024-05-08T21:14:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:40 smithi082 ceph-mon[47512]: purged_snaps scrub ok 2024-05-08T21:14:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:40 smithi082 ceph-mon[47512]: osd.5 [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719] boot 2024-05-08T21:14:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:40 smithi082 ceph-mon[47512]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T21:14:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T21:14:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:40 smithi082 ceph-mon[47512]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:14:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:14:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:14:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:41.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:40 smithi082 ceph-mon[47512]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-05-08T21:14:41.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:41 smithi033 ceph-mon[42528]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T21:14:42.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:41 smithi071 ceph-mon[47074]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T21:14:42.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:41 smithi082 ceph-mon[47512]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T21:14:42.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:42 smithi033 ceph-mon[42528]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T21:14:42.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:42 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2966225978' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7ff5abb-f264-461b-8f31-afd52b29bd59"}]: dispatch 2024-05-08T21:14:42.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:42 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2966225978' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7ff5abb-f264-461b-8f31-afd52b29bd59"}]': finished 2024-05-08T21:14:42.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:42 smithi033 ceph-mon[42528]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T21:14:42.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:42.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:42 smithi033 ceph-mon[42528]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:14:42.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:42 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1559846417' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:14:43.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:42 smithi071 ceph-mon[47074]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T21:14:43.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:42 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2966225978' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7ff5abb-f264-461b-8f31-afd52b29bd59"}]: dispatch 2024-05-08T21:14:43.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:42 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2966225978' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7ff5abb-f264-461b-8f31-afd52b29bd59"}]': finished 2024-05-08T21:14:43.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:42 smithi071 ceph-mon[47074]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T21:14:43.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:43.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:42 smithi071 ceph-mon[47074]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:14:43.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:42 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1559846417' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:14:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:42 smithi082 ceph-mon[47512]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T21:14:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:42 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2966225978' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7ff5abb-f264-461b-8f31-afd52b29bd59"}]: dispatch 2024-05-08T21:14:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:42 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2966225978' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7ff5abb-f264-461b-8f31-afd52b29bd59"}]': finished 2024-05-08T21:14:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:42 smithi082 ceph-mon[47512]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T21:14:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:42 smithi082 ceph-mon[47512]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:14:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:42 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1559846417' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:14:44.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:43 smithi082 ceph-mon[47512]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T21:14:44.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:44.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:43 smithi033 ceph-mon[42528]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T21:14:44.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:44.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:43 smithi071 ceph-mon[47074]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T21:14:44.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:45.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:44 smithi033 ceph-mon[42528]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T21:14:45.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:44 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:45.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:44 smithi033 ceph-mon[42528]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:14:45.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:44 smithi071 ceph-mon[47074]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T21:14:45.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:44 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:45.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:44 smithi071 ceph-mon[47074]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:14:45.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:44 smithi082 ceph-mon[47512]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T21:14:45.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:44 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:45.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:44 smithi082 ceph-mon[47512]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:14:47.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:47 smithi033 ceph-mon[42528]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-05-08T21:14:47.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:47 smithi071 ceph-mon[47074]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-05-08T21:14:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:47 smithi082 ceph-mon[47512]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-05-08T21:14:48.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T21:14:48.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:48 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:48.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T21:14:48.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:48 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:48.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T21:14:48.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:48 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:49.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:49 smithi033 ceph-mon[42528]: Deploying daemon osd.6 on smithi033 2024-05-08T21:14:49.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:49 smithi033 ceph-mon[42528]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-08T21:14:49.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:49 smithi071 ceph-mon[47074]: Deploying daemon osd.6 on smithi033 2024-05-08T21:14:49.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:49 smithi071 ceph-mon[47074]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-08T21:14:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:49 smithi082 ceph-mon[47512]: Deploying daemon osd.6 on smithi033 2024-05-08T21:14:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:49 smithi082 ceph-mon[47512]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-08T21:14:51.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:51 smithi033 ceph-mon[42528]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:14:51.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:51 smithi071 ceph-mon[47074]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:14:51.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:51 smithi082 ceph-mon[47512]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:14:53.106 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:53.106 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:53.106 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:53.106 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:53 smithi033 ceph-mon[42528]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-08T21:14:53.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:53.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:53.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:53.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:53 smithi071 ceph-mon[47074]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-08T21:14:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:53.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:53 smithi082 ceph-mon[47512]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-08T21:14:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.561+0000 7f4e24ff9700 1 -- 172.21.15.33:0/3461456496 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4e2802ceb0 con 0x7f4e140a1120 2024-05-08T21:14:54.565 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 6 on host 'smithi033' 2024-05-08T21:14:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.564+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/3461456496 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e140a1120 msgr2=0x7f4e140a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.564+0000 7f4e2ddc8700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e140a1120 0x7f4e140a35e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009990 tx=0x7f4e18003680 comp rx=0 tx=0).stop 2024-05-08T21:14:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.564+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/3461456496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e28101000 msgr2=0x7f4e28196e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.564+0000 7f4e2ddc8700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e28101000 0x7f4e28196e20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000d9b0 tx=0x7f4e1000dcc0 comp rx=0 tx=0).stop 2024-05-08T21:14:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.564+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/3461456496 shutdown_connections 2024-05-08T21:14:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.564+0000 7f4e2ddc8700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e140a1120 0x7f4e140a35e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.564+0000 7f4e2ddc8700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e28102ba0 0x7f4e2819e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.564+0000 7f4e2ddc8700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e28101000 0x7f4e28196e20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.564+0000 7f4e2ddc8700 1 --2- 172.21.15.33:0/3461456496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e28102200 0x7f4e28197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.564+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/3461456496 >> 172.21.15.33:0/3461456496 conn(0x7f4e280fc5c0 msgr2=0x7f4e281055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:14:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.564+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/3461456496 shutdown_connections 2024-05-08T21:14:54.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:14:54.564+0000 7f4e2ddc8700 1 -- 172.21.15.33:0/3461456496 wait complete. 2024-05-08T21:14:54.990 DEBUG:teuthology.orchestra.run.smithi033:osd.6> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.6.service 2024-05-08T21:14:54.992 INFO:tasks.cephadm:Deploying osd.7 on smithi071 with /dev/vg_nvme/lv_2... 2024-05-08T21:14:54.992 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T21:14:55.032 INFO:journalctl@ceph.osd.6.smithi033.stdout:-- Logs begin at Wed 2024-05-08 20:53:21 UTC. -- 2024-05-08T21:14:55.183 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:55 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.183 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:55 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:55 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:55.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:55 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:55.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:55 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:55 smithi033 ceph-mon[42528]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-08T21:14:55.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:55 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:55 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:55 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:55.225 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:14:55.316 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:55 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.317 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:55 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.317 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:55 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:55.317 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:55 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:55.317 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:55 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.317 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:55 smithi071 ceph-mon[47074]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-08T21:14:55.317 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:55 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.317 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:55 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.317 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:55 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:55.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:55 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:55 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:55 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:55.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:55 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:55.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:55 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:55 smithi082 ceph-mon[47512]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-05-08T21:14:55.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:55 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:55 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:55.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:55 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:14:56.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:56 smithi071 ceph-mon[47074]: from='osd.6 [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T21:14:56.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:56 smithi071 ceph-mon[47074]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T21:14:56.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:56 smithi082 ceph-mon[47512]: from='osd.6 [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T21:14:56.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:56 smithi082 ceph-mon[47512]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T21:14:56.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:56 smithi033 ceph-mon[42528]: from='osd.6 [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T21:14:56.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:56 smithi033 ceph-mon[42528]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T21:14:56.459 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:14:56 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:14:56.014+0000 7f4170c937c0 -1 osd.6 0 log_to_monitors true 2024-05-08T21:14:56.491 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T21:14:56.491 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T21:14:56.491 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-05-08T21:14:56.491 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-05-08T21:14:56.491 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125475 s, 836 MB/s 2024-05-08T21:14:56.491 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-05-08T21:14:56.936 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_2 2024-05-08T21:14:57.157 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:14:57.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:57 smithi071 ceph-mon[47074]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T21:14:57.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:57 smithi071 ceph-mon[47074]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T21:14:57.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:57.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:57 smithi071 ceph-mon[47074]: from='osd.6 [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:14:57.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:57 smithi071 ceph-mon[47074]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:14:57.272 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:57 smithi071 ceph-mon[47074]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:14:57.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:57 smithi082 ceph-mon[47512]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T21:14:57.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:57 smithi082 ceph-mon[47512]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T21:14:57.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:57.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:57 smithi082 ceph-mon[47512]: from='osd.6 [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:14:57.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:57 smithi082 ceph-mon[47512]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:14:57.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:57 smithi082 ceph-mon[47512]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:14:57.356 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:57 smithi033 ceph-mon[42528]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T21:14:57.356 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:57 smithi033 ceph-mon[42528]: osdmap e42: 7 total, 6 up, 7 in 2024-05-08T21:14:57.356 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:57.356 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:57 smithi033 ceph-mon[42528]: from='osd.6 [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:14:57.356 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:57 smithi033 ceph-mon[42528]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:14:57.356 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:57 smithi033 ceph-mon[42528]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:14:57.357 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:14:57 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:14:57.065+0000 7f41662f7700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T21:14:57.357 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:14:57 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:14:57.076+0000 7f41620ec700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:14:58.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.153+0000 7f1ac1010700 1 -- 172.21.15.71:0/1301920026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc102ba0 msgr2=0x7f1abc109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:58.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.153+0000 7f1ac1010700 1 --2- 172.21.15.71:0/1301920026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc102ba0 0x7f1abc109450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400b0d0 tx=0x7f1ab400b3e0 comp rx=0 tx=0).stop 2024-05-08T21:14:58.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.153+0000 7f1ac1010700 1 -- 172.21.15.71:0/1301920026 shutdown_connections 2024-05-08T21:14:58.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.153+0000 7f1ac1010700 1 --2- 172.21.15.71:0/1301920026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc102ba0 0x7f1abc109450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.153+0000 7f1ac1010700 1 --2- 172.21.15.71:0/1301920026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc101000 0x7f1abc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.153+0000 7f1ac1010700 1 --2- 172.21.15.71:0/1301920026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1abc102200 0x7f1abc102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.153+0000 7f1ac1010700 1 -- 172.21.15.71:0/1301920026 >> 172.21.15.71:0/1301920026 conn(0x7f1abc0fc5c0 msgr2=0x7f1abc0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.153+0000 7f1ac1010700 1 -- 172.21.15.71:0/1301920026 shutdown_connections 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.153+0000 7f1ac1010700 1 -- 172.21.15.71:0/1301920026 wait complete. 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.153+0000 7f1ac1010700 1 Processor -- start 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1ac1010700 1 -- start start 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1ac1010700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc101000 0x7f1abc196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1ac1010700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1abc102200 0x7f1abc197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1ac1010700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc102ba0 0x7f1abc19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1ac1010700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc19e930 con 0x7f1abc101000 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1ac1010700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc19ea70 con 0x7f1abc102ba0 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1ac1010700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc19ec00 con 0x7f1abc102200 2024-05-08T21:14:58.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1abb59e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc102ba0 0x7f1abc19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1abb59e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc102ba0 0x7f1abc19e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43498/0 (socket says 172.21.15.71:43498) 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1abb59e700 1 -- 172.21.15.71:0/872388380 learned_addr learned my addr 172.21.15.71:0/872388380 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1aba59c700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1abc102200 0x7f1abc197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1abad9d700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc101000 0x7f1abc196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1abb59e700 1 -- 172.21.15.71:0/872388380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1abc102200 msgr2=0x7f1abc197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1abb59e700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1abc102200 0x7f1abc197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1abb59e700 1 -- 172.21.15.71:0/872388380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc101000 msgr2=0x7f1abc196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1abb59e700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc101000 0x7f1abc196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1abb59e700 1 -- 172.21.15.71:0/872388380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab4009d00 con 0x7f1abc102ba0 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.154+0000 7f1abb59e700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc102ba0 0x7f1abc19e360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4007b60 tx=0x7f1ab40094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.155+0000 7f1aa3fff700 1 -- 172.21.15.71:0/872388380 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400e040 con 0x7f1abc102ba0 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.155+0000 7f1aa3fff700 1 -- 172.21.15.71:0/872388380 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ab4004e60 con 0x7f1abc102ba0 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.155+0000 7f1ac1010700 1 -- 172.21.15.71:0/872388380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc19ee80 con 0x7f1abc102ba0 2024-05-08T21:14:58.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.155+0000 7f1aa3fff700 1 -- 172.21.15.71:0/872388380 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4012a40 con 0x7f1abc102ba0 2024-05-08T21:14:58.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.155+0000 7f1ac1010700 1 -- 172.21.15.71:0/872388380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc19f230 con 0x7f1abc102ba0 2024-05-08T21:14:58.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.156+0000 7f1ac1010700 1 -- 172.21.15.71:0/872388380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1abc108360 con 0x7f1abc102ba0 2024-05-08T21:14:58.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.157+0000 7f1aa3fff700 1 -- 172.21.15.71:0/872388380 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1ab4005750 con 0x7f1abc102ba0 2024-05-08T21:14:58.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.157+0000 7f1aa3fff700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1aa40a1120 0x7f1aa40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:14:58.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.157+0000 7f1aa3fff700 1 -- 172.21.15.71:0/872388380 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f1ab40cd130 con 0x7f1abc102ba0 2024-05-08T21:14:58.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.157+0000 7f1abad9d700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1aa40a1120 0x7f1aa40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:14:58.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.158+0000 7f1abad9d700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1aa40a1120 0x7f1aa40a35e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1abc102060 tx=0x7f1ab000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:14:58.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.162+0000 7f1aa3fff700 1 -- 172.21.15.71:0/872388380 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1ab4097660 con 0x7f1abc102ba0 2024-05-08T21:14:58.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-08T21:14:58.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: osdmap e43: 7 total, 6 up, 7 in 2024-05-08T21:14:58.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:58.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:58.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:58.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:58.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:58.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:58.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:58.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:58.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:58.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:58.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:58.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-08T21:14:58.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: osdmap e43: 7 total, 6 up, 7 in 2024-05-08T21:14:58.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:58.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:58.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:58.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:58.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:58.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:58.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:58.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:58.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:58.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:58.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:58.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:14:58.365+0000 7f1ac1010700 1 -- 172.21.15.71:0/872388380 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1abc02cdf0 con 0x7f1aa40a1120 2024-05-08T21:14:58.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-08T21:14:58.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: osdmap e43: 7 total, 6 up, 7 in 2024-05-08T21:14:58.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:58.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:58.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:58.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:58.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:58.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:58.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:14:58.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:58.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:58.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:14:58.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:14:59.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: purged_snaps scrub starts 2024-05-08T21:14:59.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: purged_snaps scrub ok 2024-05-08T21:14:59.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: Detected new or changed devices on smithi033 2024-05-08T21:14:59.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi033 to 5706M 2024-05-08T21:14:59.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:59.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: osd.6 [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506] boot 2024-05-08T21:14:59.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T21:14:59.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:59.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:14:59.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:14:59.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:14:59.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:14:59.323 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:14:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:59.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: purged_snaps scrub starts 2024-05-08T21:14:59.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: purged_snaps scrub ok 2024-05-08T21:14:59.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: Detected new or changed devices on smithi033 2024-05-08T21:14:59.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi033 to 5706M 2024-05-08T21:14:59.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:59.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: osd.6 [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506] boot 2024-05-08T21:14:59.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T21:14:59.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:59.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:14:59.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:14:59.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:14:59.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:14:59.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:14:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:14:59.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: purged_snaps scrub starts 2024-05-08T21:14:59.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: purged_snaps scrub ok 2024-05-08T21:14:59.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: Detected new or changed devices on smithi033 2024-05-08T21:14:59.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi033 to 5706M 2024-05-08T21:14:59.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:59.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: osd.6 [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506] boot 2024-05-08T21:14:59.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: osdmap e44: 7 total, 7 up, 7 in 2024-05-08T21:14:59.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T21:14:59.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-08T21:14:59.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:14:59.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:14:59.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:14:59.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:14:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:00.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:00 smithi082 ceph-mon[47512]: osdmap e45: 7 total, 7 up, 7 in 2024-05-08T21:15:00.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:00 smithi033 ceph-mon[42528]: osdmap e45: 7 total, 7 up, 7 in 2024-05-08T21:15:00.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:00 smithi071 ceph-mon[47074]: osdmap e45: 7 total, 7 up, 7 in 2024-05-08T21:15:01.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:01 smithi082 ceph-mon[47512]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:01.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:01 smithi082 ceph-mon[47512]: from='client.? 172.21.15.71:0/2886506344' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1de3175-dfdd-40ad-a3d3-52a45dec6b2e"}]: dispatch 2024-05-08T21:15:01.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:01 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1de3175-dfdd-40ad-a3d3-52a45dec6b2e"}]: dispatch 2024-05-08T21:15:01.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:01 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1de3175-dfdd-40ad-a3d3-52a45dec6b2e"}]': finished 2024-05-08T21:15:01.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:01 smithi082 ceph-mon[47512]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T21:15:01.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:01.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:01 smithi082 ceph-mon[47512]: from='client.? 172.21.15.71:0/4187241895' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:15:01.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:01 smithi033 ceph-mon[42528]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:01.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:01 smithi033 ceph-mon[42528]: from='client.? 172.21.15.71:0/2886506344' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1de3175-dfdd-40ad-a3d3-52a45dec6b2e"}]: dispatch 2024-05-08T21:15:01.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:01 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1de3175-dfdd-40ad-a3d3-52a45dec6b2e"}]: dispatch 2024-05-08T21:15:01.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:01 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1de3175-dfdd-40ad-a3d3-52a45dec6b2e"}]': finished 2024-05-08T21:15:01.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:01 smithi033 ceph-mon[42528]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T21:15:01.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:01.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:01 smithi033 ceph-mon[42528]: from='client.? 172.21.15.71:0/4187241895' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:15:01.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:01 smithi071 ceph-mon[47074]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:01.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:01 smithi071 ceph-mon[47074]: from='client.? 172.21.15.71:0/2886506344' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1de3175-dfdd-40ad-a3d3-52a45dec6b2e"}]: dispatch 2024-05-08T21:15:01.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:01 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1de3175-dfdd-40ad-a3d3-52a45dec6b2e"}]: dispatch 2024-05-08T21:15:01.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:01 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1de3175-dfdd-40ad-a3d3-52a45dec6b2e"}]': finished 2024-05-08T21:15:01.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:01 smithi071 ceph-mon[47074]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T21:15:01.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:01.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:01 smithi071 ceph-mon[47074]: from='client.? 172.21.15.71:0/4187241895' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:15:03.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:03 smithi082 ceph-mon[47512]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:03.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:03 smithi033 ceph-mon[42528]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:03.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:03 smithi071 ceph-mon[47074]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:05.297 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:05 smithi071 ceph-mon[47074]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:05.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:05 smithi082 ceph-mon[47512]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:05.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:05 smithi033 ceph-mon[42528]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:06.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:06 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T21:15:06.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:06 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:06.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:06 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T21:15:06.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:06 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:06.468 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:06 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T21:15:06.469 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:06 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:07.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:07 smithi082 ceph-mon[47512]: Deploying daemon osd.7 on smithi071 2024-05-08T21:15:07.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:07 smithi082 ceph-mon[47512]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:07.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:07 smithi033 ceph-mon[42528]: Deploying daemon osd.7 on smithi071 2024-05-08T21:15:07.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:07 smithi033 ceph-mon[42528]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:07.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:07 smithi071 ceph-mon[47074]: Deploying daemon osd.7 on smithi071 2024-05-08T21:15:07.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:07 smithi071 ceph-mon[47074]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:09.322 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:09 smithi071 ceph-mon[47074]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:09.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:09 smithi033 ceph-mon[42528]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:09.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:09 smithi082 ceph-mon[47512]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:11.023 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:11.024 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:11.024 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:11.024 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:10 smithi071 ceph-mon[47074]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:11.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:11.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:11.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:11.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:10 smithi082 ceph-mon[47512]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:11.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:11.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:11.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:11.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:10 smithi033 ceph-mon[42528]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:12.074 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 7 on host 'smithi071' 2024-05-08T21:15:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.070+0000 7f1aa3fff700 1 -- 172.21.15.71:0/872388380 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1abc02cdf0 con 0x7f1aa40a1120 2024-05-08T21:15:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.072+0000 7f1ac1010700 1 -- 172.21.15.71:0/872388380 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1aa40a1120 msgr2=0x7f1aa40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.072+0000 7f1ac1010700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1aa40a1120 0x7f1aa40a35e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1abc102060 tx=0x7f1ab000a3b0 comp rx=0 tx=0).stop 2024-05-08T21:15:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.072+0000 7f1ac1010700 1 -- 172.21.15.71:0/872388380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc102ba0 msgr2=0x7f1abc19e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.072+0000 7f1ac1010700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc102ba0 0x7f1abc19e360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4007b60 tx=0x7f1ab40094c0 comp rx=0 tx=0).stop 2024-05-08T21:15:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.072+0000 7f1ac1010700 1 -- 172.21.15.71:0/872388380 shutdown_connections 2024-05-08T21:15:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.072+0000 7f1ac1010700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1aa40a1120 0x7f1aa40a35e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.072+0000 7f1ac1010700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1abc102ba0 0x7f1abc19e360 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.072+0000 7f1ac1010700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc101000 0x7f1abc196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.072+0000 7f1ac1010700 1 --2- 172.21.15.71:0/872388380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1abc102200 0x7f1abc197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.072+0000 7f1ac1010700 1 -- 172.21.15.71:0/872388380 >> 172.21.15.71:0/872388380 conn(0x7f1abc0fc5c0 msgr2=0x7f1abc1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:15:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.072+0000 7f1ac1010700 1 -- 172.21.15.71:0/872388380 shutdown_connections 2024-05-08T21:15:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:12.072+0000 7f1ac1010700 1 -- 172.21.15.71:0/872388380 wait complete. 2024-05-08T21:15:12.458 DEBUG:teuthology.orchestra.run.smithi071:osd.7> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.7.service 2024-05-08T21:15:12.460 INFO:tasks.cephadm:Deploying osd.8 on smithi082 with /dev/vg_nvme/lv_2... 2024-05-08T21:15:12.461 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T21:15:12.503 INFO:journalctl@ceph.osd.7.smithi071.stdout:-- Logs begin at Wed 2024-05-08 20:53:45 UTC. -- 2024-05-08T21:15:12.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:12.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:12.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:12.678 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.678 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.678 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:12.678 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:12.678 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.678 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.678 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.678 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:12.708 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:15:12.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:12.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:12.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:12.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:13.552 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:13 smithi082 ceph-mon[47512]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:13.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:13 smithi071 ceph-mon[47074]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:13.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:13 smithi033 ceph-mon[42528]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:14.071 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:15:13 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:15:13.780+0000 7f968bfb47c0 -1 osd.7 0 log_to_monitors true 2024-05-08T21:15:14.145 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T21:15:14.145 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T21:15:14.145 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-05-08T21:15:14.145 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-05-08T21:15:14.145 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115013 s, 912 MB/s 2024-05-08T21:15:14.145 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-05-08T21:15:14.422 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:14 smithi071 ceph-mon[47074]: from='osd.7 [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T21:15:14.423 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:14 smithi071 ceph-mon[47074]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T21:15:14.472 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:14 smithi082 ceph-mon[47512]: from='osd.7 [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T21:15:14.472 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:14 smithi082 ceph-mon[47512]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T21:15:14.493 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_2 2024-05-08T21:15:14.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:14 smithi033 ceph-mon[42528]: from='osd.7 [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T21:15:14.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:14 smithi033 ceph-mon[42528]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T21:15:14.711 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:15:15.528 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:15.528 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T21:15:15.528 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T21:15:15.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:15.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='osd.7 [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:15:15.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:15:15.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:15.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:15.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:15.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:15.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:15.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:15.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:15.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:15.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:15.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.695+0000 7f715ec5e700 1 -- 172.21.15.82:0/3009853548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7158100fe0 msgr2=0x7f7158101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:15.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.695+0000 7f715ec5e700 1 --2- 172.21.15.82:0/3009853548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7158100fe0 0x7f7158101400 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f714c009990 tx=0x7f714c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:15:15.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.695+0000 7f715ec5e700 1 -- 172.21.15.82:0/3009853548 shutdown_connections 2024-05-08T21:15:15.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.695+0000 7f715ec5e700 1 --2- 172.21.15.82:0/3009853548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7158102b80 0x7f7158109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:15.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.695+0000 7f715ec5e700 1 --2- 172.21.15.82:0/3009853548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7158100fe0 0x7f7158101400 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:15.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.695+0000 7f715ec5e700 1 --2- 172.21.15.82:0/3009853548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71581021e0 0x7f7158102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:15.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.695+0000 7f715ec5e700 1 -- 172.21.15.82:0/3009853548 >> 172.21.15.82:0/3009853548 conn(0x7f71580fc580 msgr2=0x7f71580fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:15:15.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.695+0000 7f715ec5e700 1 -- 172.21.15.82:0/3009853548 shutdown_connections 2024-05-08T21:15:15.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.695+0000 7f715ec5e700 1 -- 172.21.15.82:0/3009853548 wait complete. 2024-05-08T21:15:15.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715ec5e700 1 Processor -- start 2024-05-08T21:15:15.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715ec5e700 1 -- start start 2024-05-08T21:15:15.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715ec5e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7158100fe0 0x7f7158196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:15:15.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715ec5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71581021e0 0x7f71581973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:15:15.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715ec5e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158102b80 0x7f715819e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:15:15.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715ec5e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715819e980 con 0x7f71581021e0 2024-05-08T21:15:15.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715ec5e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715819eac0 con 0x7f7158102b80 2024-05-08T21:15:15.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715ec5e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715819ec50 con 0x7f7158100fe0 2024-05-08T21:15:15.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715c9fa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7158100fe0 0x7f7158196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:15:15.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715c9fa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7158100fe0 0x7f7158196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58292/0 (socket says 172.21.15.82:58292) 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715c9fa700 1 -- 172.21.15.82:0/682748181 learned_addr learned my addr 172.21.15.82:0/682748181 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f7157fff700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71581021e0 0x7f71581973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715d1fb700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158102b80 0x7f715819e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715c9fa700 1 -- 172.21.15.82:0/682748181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158102b80 msgr2=0x7f715819e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715c9fa700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158102b80 0x7f715819e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715c9fa700 1 -- 172.21.15.82:0/682748181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71581021e0 msgr2=0x7f71581973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715c9fa700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71581021e0 0x7f71581973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715c9fa700 1 -- 172.21.15.82:0/682748181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c0095f0 con 0x7f7158100fe0 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.696+0000 7f715c9fa700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7158100fe0 0x7f7158196e90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f714c000c00 tx=0x7f714c00bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.697+0000 7f7155ffb700 1 -- 172.21.15.82:0/682748181 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c01d070 con 0x7f7158100fe0 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.697+0000 7f7155ffb700 1 -- 172.21.15.82:0/682748181 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f714c003d20 con 0x7f7158100fe0 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.697+0000 7f715ec5e700 1 -- 172.21.15.82:0/682748181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715819eed0 con 0x7f7158100fe0 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.697+0000 7f7155ffb700 1 -- 172.21.15.82:0/682748181 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c00f460 con 0x7f7158100fe0 2024-05-08T21:15:15.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.697+0000 7f715ec5e700 1 -- 172.21.15.82:0/682748181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715819f230 con 0x7f7158100fe0 2024-05-08T21:15:15.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.698+0000 7f715ec5e700 1 -- 172.21.15.82:0/682748181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71581085b0 con 0x7f7158100fe0 2024-05-08T21:15:15.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.700+0000 7f7155ffb700 1 -- 172.21.15.82:0/682748181 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f714c022670 con 0x7f7158100fe0 2024-05-08T21:15:15.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.700+0000 7f7155ffb700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71480a1120 0x7f71480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:15:15.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.700+0000 7f7155ffb700 1 -- 172.21.15.82:0/682748181 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f714c0cd570 con 0x7f7158100fe0 2024-05-08T21:15:15.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.700+0000 7f7157fff700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71480a1120 0x7f71480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:15:15.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.701+0000 7f7157fff700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71480a1120 0x7f71480a35e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7144007c60 tx=0x7f71440073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:15:15.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.704+0000 7f7155ffb700 1 -- 172.21.15.82:0/682748181 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f714c097a10 con 0x7f7158100fe0 2024-05-08T21:15:15.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:15.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T21:15:15.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T21:15:15.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:15.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='osd.7 [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:15:15.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:15:15.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:15.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:15.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:15.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:15.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:15.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:15.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:15.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:15.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:15.821 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:15:15 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:15:15.336+0000 7f9681618700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T21:15:15.821 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:15:15 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:15:15.349+0000 7f967d40d700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:15:15.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: osdmap e47: 8 total, 7 up, 8 in 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='osd.7 [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:15.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:15.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:15.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:15.904+0000 7f715ec5e700 1 -- 172.21.15.82:0/682748181 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f715802ceb0 con 0x7f71480a1120 2024-05-08T21:15:16.592 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:16 smithi082 ceph-mon[47512]: Detected new or changed devices on smithi071 2024-05-08T21:15:16.592 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:16 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi071 to 5704M 2024-05-08T21:15:16.593 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:16 smithi082 ceph-mon[47512]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-08T21:15:16.593 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:16 smithi082 ceph-mon[47512]: osdmap e48: 8 total, 7 up, 8 in 2024-05-08T21:15:16.593 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:16.593 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:16.593 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:15:16.593 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:15:16.593 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:16.593 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:16 smithi082 ceph-mon[47512]: osd.7 [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909] boot 2024-05-08T21:15:16.593 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:16 smithi082 ceph-mon[47512]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T21:15:16.593 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:16.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:16 smithi033 ceph-mon[42528]: Detected new or changed devices on smithi071 2024-05-08T21:15:16.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:16 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi071 to 5704M 2024-05-08T21:15:16.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:16 smithi033 ceph-mon[42528]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-08T21:15:16.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:16 smithi033 ceph-mon[42528]: osdmap e48: 8 total, 7 up, 8 in 2024-05-08T21:15:16.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:16.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:16.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:15:16.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:15:16.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:16.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:16 smithi033 ceph-mon[42528]: osd.7 [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909] boot 2024-05-08T21:15:16.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:16 smithi033 ceph-mon[42528]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T21:15:16.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:16.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:16 smithi071 ceph-mon[47074]: Detected new or changed devices on smithi071 2024-05-08T21:15:16.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:16 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi071 to 5704M 2024-05-08T21:15:16.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:16 smithi071 ceph-mon[47074]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-08T21:15:16.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:16 smithi071 ceph-mon[47074]: osdmap e48: 8 total, 7 up, 8 in 2024-05-08T21:15:16.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:16.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:16.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:15:16.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:15:16.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:16.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:16 smithi071 ceph-mon[47074]: osd.7 [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909] boot 2024-05-08T21:15:16.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:16 smithi071 ceph-mon[47074]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T21:15:16.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T21:15:17.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:17 smithi082 ceph-mon[47512]: purged_snaps scrub starts 2024-05-08T21:15:17.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:17 smithi082 ceph-mon[47512]: purged_snaps scrub ok 2024-05-08T21:15:17.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:17 smithi082 ceph-mon[47512]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:15:17.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:17 smithi082 ceph-mon[47512]: pgmap v138: 1 pgs: 1 unknown; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:17.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:17 smithi082 ceph-mon[47512]: osdmap e50: 8 total, 8 up, 8 in 2024-05-08T21:15:17.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:17 smithi033 ceph-mon[42528]: purged_snaps scrub starts 2024-05-08T21:15:17.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:17 smithi033 ceph-mon[42528]: purged_snaps scrub ok 2024-05-08T21:15:17.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:17 smithi033 ceph-mon[42528]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:15:17.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:17 smithi033 ceph-mon[42528]: pgmap v138: 1 pgs: 1 unknown; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:17.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:17 smithi033 ceph-mon[42528]: osdmap e50: 8 total, 8 up, 8 in 2024-05-08T21:15:17.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:17 smithi071 ceph-mon[47074]: purged_snaps scrub starts 2024-05-08T21:15:17.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:17 smithi071 ceph-mon[47074]: purged_snaps scrub ok 2024-05-08T21:15:17.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:17 smithi071 ceph-mon[47074]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:15:17.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:17 smithi071 ceph-mon[47074]: pgmap v138: 1 pgs: 1 unknown; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-08T21:15:17.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:17 smithi071 ceph-mon[47074]: osdmap e50: 8 total, 8 up, 8 in 2024-05-08T21:15:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:18 smithi082 ceph-mon[47512]: from='client.? 172.21.15.82:0/149967559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a7620db-cbad-473d-a5fd-ab506474d661"}]: dispatch 2024-05-08T21:15:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:18 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a7620db-cbad-473d-a5fd-ab506474d661"}]: dispatch 2024-05-08T21:15:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:18 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a7620db-cbad-473d-a5fd-ab506474d661"}]': finished 2024-05-08T21:15:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:18 smithi082 ceph-mon[47512]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T21:15:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:18 smithi033 ceph-mon[42528]: from='client.? 172.21.15.82:0/149967559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a7620db-cbad-473d-a5fd-ab506474d661"}]: dispatch 2024-05-08T21:15:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:18 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a7620db-cbad-473d-a5fd-ab506474d661"}]: dispatch 2024-05-08T21:15:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:18 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a7620db-cbad-473d-a5fd-ab506474d661"}]': finished 2024-05-08T21:15:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:18 smithi033 ceph-mon[42528]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T21:15:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:18.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:18 smithi071 ceph-mon[47074]: from='client.? 172.21.15.82:0/149967559' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a7620db-cbad-473d-a5fd-ab506474d661"}]: dispatch 2024-05-08T21:15:18.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:18 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a7620db-cbad-473d-a5fd-ab506474d661"}]: dispatch 2024-05-08T21:15:18.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:18 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a7620db-cbad-473d-a5fd-ab506474d661"}]': finished 2024-05-08T21:15:18.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:18 smithi071 ceph-mon[47074]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T21:15:18.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:19.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:19 smithi082 ceph-mon[47512]: pgmap v142: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:19.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:19 smithi082 ceph-mon[47512]: from='client.? 172.21.15.82:0/1824538117' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:15:19.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:19 smithi082 ceph-mon[47512]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T21:15:19.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:19.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:19 smithi033 ceph-mon[42528]: pgmap v142: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:19.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:19 smithi033 ceph-mon[42528]: from='client.? 172.21.15.82:0/1824538117' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:15:19.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:19 smithi033 ceph-mon[42528]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T21:15:19.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:19 smithi071 ceph-mon[47074]: pgmap v142: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:19 smithi071 ceph-mon[47074]: from='client.? 172.21.15.82:0/1824538117' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:15:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:19 smithi071 ceph-mon[47074]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T21:15:19.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:21.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:20 smithi071 ceph-mon[47074]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T21:15:21.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:20 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:21.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:20 smithi071 ceph-mon[47074]: pgmap v145: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:21.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:20 smithi082 ceph-mon[47512]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T21:15:21.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:20 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:21.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:20 smithi082 ceph-mon[47512]: pgmap v145: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:21.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:20 smithi033 ceph-mon[42528]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T21:15:21.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:20 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:21.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:20 smithi033 ceph-mon[42528]: pgmap v145: 1 pgs: 1 unknown; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:23.320 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:23 smithi082 ceph-mon[47512]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:23.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:23 smithi033 ceph-mon[42528]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:23.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:23 smithi071 ceph-mon[47074]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:24.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T21:15:24.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:24.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T21:15:24.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:24.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T21:15:24.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:25.452 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:25 smithi082 ceph-mon[47512]: Deploying daemon osd.8 on smithi082 2024-05-08T21:15:25.452 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:25 smithi082 ceph-mon[47512]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:25.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:25 smithi033 ceph-mon[42528]: Deploying daemon osd.8 on smithi082 2024-05-08T21:15:25.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:25 smithi033 ceph-mon[42528]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:25.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:25 smithi071 ceph-mon[47074]: Deploying daemon osd.8 on smithi082 2024-05-08T21:15:25.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:25 smithi071 ceph-mon[47074]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:27.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:27 smithi033 ceph-mon[42528]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:27.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:27 smithi071 ceph-mon[47074]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:27.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:27 smithi082 ceph-mon[47512]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:28.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:28 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:28 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:28 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:28.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:28 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:28.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:28 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:28.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:28 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:28.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:28 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:28.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:28 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:28.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:28 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:29.585 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:29 smithi082 ceph-mon[47512]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:29.585 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:29 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:29.585 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:29 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:29.585 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:29 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:29.585 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:29 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:29.586 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:29 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:29 smithi033 ceph-mon[42528]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:29 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:29 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:29 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:29 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:29 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:29.717 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 8 on host 'smithi082' 2024-05-08T21:15:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.714+0000 7f7155ffb700 1 -- 172.21.15.82:0/682748181 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f715802ceb0 con 0x7f71480a1120 2024-05-08T21:15:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.716+0000 7f715ec5e700 1 -- 172.21.15.82:0/682748181 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71480a1120 msgr2=0x7f71480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.716+0000 7f715ec5e700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71480a1120 0x7f71480a35e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7144007c60 tx=0x7f71440073d0 comp rx=0 tx=0).stop 2024-05-08T21:15:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.717+0000 7f715ec5e700 1 -- 172.21.15.82:0/682748181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7158100fe0 msgr2=0x7f7158196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.717+0000 7f715ec5e700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7158100fe0 0x7f7158196e90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f714c000c00 tx=0x7f714c00bb20 comp rx=0 tx=0).stop 2024-05-08T21:15:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.717+0000 7f715ec5e700 1 -- 172.21.15.82:0/682748181 shutdown_connections 2024-05-08T21:15:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.717+0000 7f715ec5e700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71480a1120 0x7f71480a35e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.717+0000 7f715ec5e700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7158102b80 0x7f715819e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.717+0000 7f715ec5e700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7158100fe0 0x7f7158196e90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.717+0000 7f715ec5e700 1 --2- 172.21.15.82:0/682748181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71581021e0 0x7f71581973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.717+0000 7f715ec5e700 1 -- 172.21.15.82:0/682748181 >> 172.21.15.82:0/682748181 conn(0x7f71580fc580 msgr2=0x7f71581056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:15:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.717+0000 7f715ec5e700 1 -- 172.21.15.82:0/682748181 shutdown_connections 2024-05-08T21:15:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:15:29.717+0000 7f715ec5e700 1 -- 172.21.15.82:0/682748181 wait complete. 2024-05-08T21:15:29.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:29 smithi071 ceph-mon[47074]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:29.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:29 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:29.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:29 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:29.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:29 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:29.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:29 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:29.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:29 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:30.099 DEBUG:teuthology.orchestra.run.smithi082:osd.8> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.8.service 2024-05-08T21:15:30.101 INFO:tasks.cephadm:Deploying osd.9 on smithi033 with /dev/vg_nvme/lv_1... 2024-05-08T21:15:30.101 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T21:15:30.132 INFO:journalctl@ceph.osd.8.smithi082.stdout:-- Logs begin at Wed 2024-05-08 20:53:53 UTC. -- 2024-05-08T21:15:30.372 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:15:30.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:30 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:30.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:30 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:30.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:30 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:30.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:30 smithi033 ceph-mon[42528]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:30.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:30 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:30.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:30 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:30.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:30 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:30.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:30 smithi082 ceph-mon[47512]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:31.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:30 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:31.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:30 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:31.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:30 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:31.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:30 smithi071 ceph-mon[47074]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:31.604 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:15:31 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:15:31.308+0000 7f486506c7c0 -1 osd.8 0 log_to_monitors true 2024-05-08T21:15:31.763 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T21:15:31.763 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T21:15:31.763 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-05-08T21:15:31.764 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-05-08T21:15:31.764 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121988 s, 860 MB/s 2024-05-08T21:15:31.764 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-05-08T21:15:32.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:31 smithi071 ceph-mon[47074]: from='osd.8 [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T21:15:32.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:31 smithi071 ceph-mon[47074]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T21:15:32.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:31 smithi082 ceph-mon[47512]: from='osd.8 [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T21:15:32.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:31 smithi082 ceph-mon[47512]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T21:15:32.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:31 smithi033 ceph-mon[42528]: from='osd.8 [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T21:15:32.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:31 smithi033 ceph-mon[42528]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T21:15:32.289 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_1 2024-05-08T21:15:32.558 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:15:33.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='osd.8 [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: Detected new or changed devices on smithi082 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi082 to 5704M 2024-05-08T21:15:33.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:33.073 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:33.073 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:33.073 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:32 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:33.083 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T21:15:33.083 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T21:15:33.083 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:33.083 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='osd.8 [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:15:33.083 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:15:33.083 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:33.083 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: Detected new or changed devices on smithi082 2024-05-08T21:15:33.084 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:33.084 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:33.084 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:33.084 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:33.084 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:33.084 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi082 to 5704M 2024-05-08T21:15:33.084 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:33.084 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:33.084 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:33.084 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:32 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:33.104 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:15:32 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:15:32.725+0000 7f485a6d0700 -1 osd.8 0 waiting for initial osdmap 2024-05-08T21:15:33.104 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:15:32 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:15:32.736+0000 7f48564c5700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:15:33.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T21:15:33.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: osdmap e54: 9 total, 8 up, 9 in 2024-05-08T21:15:33.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:33.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='osd.8 [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:15:33.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:15:33.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:33.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: Detected new or changed devices on smithi082 2024-05-08T21:15:33.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:33.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:33.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:33.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:33.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:33.105 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi082 to 5704M 2024-05-08T21:15:33.106 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:33.106 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:33.106 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:33.106 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:32 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:33.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.711+0000 7ff68b609700 1 -- 172.21.15.33:0/3470874939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff684101060 msgr2=0x7ff6841014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:33.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.711+0000 7ff68b609700 1 --2- 172.21.15.33:0/3470874939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff684101060 0x7ff6841014e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff674009990 tx=0x7ff674009ca0 comp rx=0 tx=0).stop 2024-05-08T21:15:33.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.711+0000 7ff68b609700 1 -- 172.21.15.33:0/3470874939 shutdown_connections 2024-05-08T21:15:33.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.711+0000 7ff68b609700 1 --2- 172.21.15.33:0/3470874939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff684101a20 0x7ff684108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:33.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.711+0000 7ff68b609700 1 --2- 172.21.15.33:0/3470874939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6840fff00 0x7ff684100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:33.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.711+0000 7ff68b609700 1 --2- 172.21.15.33:0/3470874939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff684101060 0x7ff6841014e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:33.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.711+0000 7ff68b609700 1 -- 172.21.15.33:0/3470874939 >> 172.21.15.33:0/3470874939 conn(0x7ff6840fb4a0 msgr2=0x7ff6840fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:15:33.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.711+0000 7ff68b609700 1 -- 172.21.15.33:0/3470874939 shutdown_connections 2024-05-08T21:15:33.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.711+0000 7ff68b609700 1 -- 172.21.15.33:0/3470874939 wait complete. 2024-05-08T21:15:33.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff68b609700 1 Processor -- start 2024-05-08T21:15:33.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff68b609700 1 -- start start 2024-05-08T21:15:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff68b609700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6840fff00 0x7ff684069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:15:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff68b609700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff684101060 0x7ff68419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:15:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff68b609700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff684101a20 0x7ff68419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:15:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff68b609700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68419e8d0 con 0x7ff684101a20 2024-05-08T21:15:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff68b609700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68419ea40 con 0x7ff684101060 2024-05-08T21:15:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff68b609700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68419ec00 con 0x7ff6840fff00 2024-05-08T21:15:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff689ba6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff684101a20 0x7ff68419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:15:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff689ba6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff684101a20 0x7ff68419e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41386/0 (socket says 172.21.15.33:41386) 2024-05-08T21:15:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff689ba6700 1 -- 172.21.15.33:0/3348657178 learned_addr learned my addr 172.21.15.33:0/3348657178 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:15:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff6893a5700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6840fff00 0x7ff684069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:15:33.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.712+0000 7ff688ba4700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff684101060 0x7ff68419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:15:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.713+0000 7ff689ba6700 1 -- 172.21.15.33:0/3348657178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6840fff00 msgr2=0x7ff684069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.713+0000 7ff689ba6700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6840fff00 0x7ff684069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.713+0000 7ff689ba6700 1 -- 172.21.15.33:0/3348657178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff684101060 msgr2=0x7ff68419bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.713+0000 7ff689ba6700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff684101060 0x7ff68419bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.713+0000 7ff689ba6700 1 -- 172.21.15.33:0/3348657178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740095f0 con 0x7ff684101a20 2024-05-08T21:15:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.713+0000 7ff689ba6700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff684101a20 0x7ff68419e130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00ef30 tx=0x7ff67c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:15:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.713+0000 7ff67a7fc700 1 -- 172.21.15.33:0/3348657178 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c018070 con 0x7ff684101a20 2024-05-08T21:15:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.713+0000 7ff67a7fc700 1 -- 172.21.15.33:0/3348657178 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff67c00f040 con 0x7ff684101a20 2024-05-08T21:15:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.713+0000 7ff68b609700 1 -- 172.21.15.33:0/3348657178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68419ee80 con 0x7ff684101a20 2024-05-08T21:15:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.713+0000 7ff67a7fc700 1 -- 172.21.15.33:0/3348657178 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c00cc70 con 0x7ff684101a20 2024-05-08T21:15:33.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.713+0000 7ff68b609700 1 -- 172.21.15.33:0/3348657178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68419f2f0 con 0x7ff684101a20 2024-05-08T21:15:33.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.714+0000 7ff68b609700 1 -- 172.21.15.33:0/3348657178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6841073a0 con 0x7ff684101a20 2024-05-08T21:15:33.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.715+0000 7ff67a7fc700 1 -- 172.21.15.33:0/3348657178 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff67c003a00 con 0x7ff684101a20 2024-05-08T21:15:33.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.716+0000 7ff67a7fc700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff6700a11a0 0x7ff6700a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:15:33.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.716+0000 7ff6893a5700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff6700a11a0 0x7ff6700a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:15:33.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.717+0000 7ff67a7fc700 1 -- 172.21.15.33:0/3348657178 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5549+0+0 (secure 0 0 0) 0x7ff67c0cbf60 con 0x7ff684101a20 2024-05-08T21:15:33.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.717+0000 7ff6893a5700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff6700a11a0 0x7ff6700a3660 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff680005950 tx=0x7ff68000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:15:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.721+0000 7ff67a7fc700 1 -- 172.21.15.33:0/3348657178 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff67c0961b0 con 0x7ff684101a20 2024-05-08T21:15:33.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:33.947+0000 7ff68b609700 1 -- 172.21.15.33:0/3348657178 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff68402ce40 con 0x7ff6700a11a0 2024-05-08T21:15:33.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:33 smithi033 ceph-mon[42528]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-08T21:15:33.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:33 smithi033 ceph-mon[42528]: osdmap e55: 9 total, 8 up, 9 in 2024-05-08T21:15:33.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:33 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:33.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:33 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:34.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:33 smithi071 ceph-mon[47074]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-08T21:15:34.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:33 smithi071 ceph-mon[47074]: osdmap e55: 9 total, 8 up, 9 in 2024-05-08T21:15:34.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:33 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:34.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:33 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:34.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:33 smithi082 ceph-mon[47512]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-08T21:15:34.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:33 smithi082 ceph-mon[47512]: osdmap e55: 9 total, 8 up, 9 in 2024-05-08T21:15:34.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:33 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:34.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:33 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:34.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:34 smithi033 ceph-mon[42528]: purged_snaps scrub starts 2024-05-08T21:15:34.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:34 smithi033 ceph-mon[42528]: purged_snaps scrub ok 2024-05-08T21:15:34.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:34 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:34.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:34 smithi033 ceph-mon[42528]: osd.8 [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395] boot 2024-05-08T21:15:34.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:34 smithi033 ceph-mon[42528]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T21:15:34.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:34 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:34.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:34 smithi033 ceph-mon[42528]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:15:34.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:34 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:15:34.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:34 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:15:34.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:34 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:34.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:34 smithi033 ceph-mon[42528]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:35.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:34 smithi071 ceph-mon[47074]: purged_snaps scrub starts 2024-05-08T21:15:35.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:34 smithi071 ceph-mon[47074]: purged_snaps scrub ok 2024-05-08T21:15:35.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:35.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:34 smithi071 ceph-mon[47074]: osd.8 [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395] boot 2024-05-08T21:15:35.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:34 smithi071 ceph-mon[47074]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T21:15:35.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:35.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:34 smithi071 ceph-mon[47074]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:15:35.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:15:35.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:15:35.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:34 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:35.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:34 smithi071 ceph-mon[47074]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:35.107 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:34 smithi082 ceph-mon[47512]: purged_snaps scrub starts 2024-05-08T21:15:35.107 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:34 smithi082 ceph-mon[47512]: purged_snaps scrub ok 2024-05-08T21:15:35.107 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:34 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:35.107 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:34 smithi082 ceph-mon[47512]: osd.8 [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395] boot 2024-05-08T21:15:35.107 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:34 smithi082 ceph-mon[47512]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T21:15:35.107 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:34 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T21:15:35.107 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:34 smithi082 ceph-mon[47512]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:15:35.107 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:34 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:15:35.107 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:34 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:15:35.107 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:34 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:35.108 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:34 smithi082 ceph-mon[47512]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-08T21:15:36.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:35 smithi071 ceph-mon[47074]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T21:15:36.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:35 smithi082 ceph-mon[47512]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T21:15:36.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:35 smithi033 ceph-mon[42528]: osdmap e57: 9 total, 9 up, 9 in 2024-05-08T21:15:37.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:36 smithi071 ceph-mon[47074]: osdmap e58: 9 total, 9 up, 9 in 2024-05-08T21:15:37.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:36 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/770163407' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "227000e2-dc41-45c2-84f6-4a4345b83328"}]: dispatch 2024-05-08T21:15:37.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:36 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/770163407' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "227000e2-dc41-45c2-84f6-4a4345b83328"}]': finished 2024-05-08T21:15:37.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:36 smithi071 ceph-mon[47074]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T21:15:37.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:36 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:37.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:36 smithi071 ceph-mon[47074]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:15:37.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:36 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1557033181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:15:37.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:36 smithi082 ceph-mon[47512]: osdmap e58: 9 total, 9 up, 9 in 2024-05-08T21:15:37.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:36 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/770163407' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "227000e2-dc41-45c2-84f6-4a4345b83328"}]: dispatch 2024-05-08T21:15:37.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:36 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/770163407' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "227000e2-dc41-45c2-84f6-4a4345b83328"}]': finished 2024-05-08T21:15:37.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:36 smithi082 ceph-mon[47512]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T21:15:37.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:36 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:37.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:36 smithi082 ceph-mon[47512]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:15:37.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:36 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1557033181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:15:37.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:36 smithi033 ceph-mon[42528]: osdmap e58: 9 total, 9 up, 9 in 2024-05-08T21:15:37.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:36 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/770163407' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "227000e2-dc41-45c2-84f6-4a4345b83328"}]: dispatch 2024-05-08T21:15:37.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:36 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/770163407' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "227000e2-dc41-45c2-84f6-4a4345b83328"}]': finished 2024-05-08T21:15:37.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:36 smithi033 ceph-mon[42528]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T21:15:37.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:36 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:37.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:36 smithi033 ceph-mon[42528]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:15:37.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:36 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1557033181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:15:38.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:37 smithi071 ceph-mon[47074]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T21:15:38.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:37 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:38.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:37 smithi082 ceph-mon[47512]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T21:15:38.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:37 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:38.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:37 smithi033 ceph-mon[42528]: osdmap e60: 10 total, 9 up, 10 in 2024-05-08T21:15:38.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:37 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:39.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:38 smithi071 ceph-mon[47074]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T21:15:39.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:39.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:38 smithi071 ceph-mon[47074]: pgmap v162: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T21:15:39.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:38 smithi082 ceph-mon[47512]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T21:15:39.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:39.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:38 smithi082 ceph-mon[47512]: pgmap v162: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T21:15:39.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:38 smithi033 ceph-mon[42528]: osdmap e61: 10 total, 9 up, 10 in 2024-05-08T21:15:39.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:39.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:38 smithi033 ceph-mon[42528]: pgmap v162: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T21:15:40.069 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:39 smithi033 ceph-mon[42528]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-08T21:15:40.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:39 smithi071 ceph-mon[47074]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-08T21:15:40.353 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:39 smithi082 ceph-mon[47512]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-08T21:15:41.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:40 smithi033 ceph-mon[42528]: pgmap v163: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T21:15:41.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:40 smithi071 ceph-mon[47074]: pgmap v163: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T21:15:41.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:40 smithi082 ceph-mon[47512]: pgmap v163: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T21:15:42.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T21:15:42.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:42.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T21:15:42.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:42.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T21:15:42.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:43.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:43 smithi033 ceph-mon[42528]: Deploying daemon osd.9 on smithi033 2024-05-08T21:15:43.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:43 smithi033 ceph-mon[42528]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-08T21:15:43.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:43 smithi033 ceph-mon[42528]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-08T21:15:43.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:43 smithi033 ceph-mon[42528]: Cluster is now healthy 2024-05-08T21:15:43.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:43 smithi071 ceph-mon[47074]: Deploying daemon osd.9 on smithi033 2024-05-08T21:15:43.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:43 smithi071 ceph-mon[47074]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-08T21:15:43.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:43 smithi071 ceph-mon[47074]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-08T21:15:43.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:43 smithi071 ceph-mon[47074]: Cluster is now healthy 2024-05-08T21:15:43.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:43 smithi082 ceph-mon[47512]: Deploying daemon osd.9 on smithi033 2024-05-08T21:15:43.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:43 smithi082 ceph-mon[47512]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-05-08T21:15:43.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:43 smithi082 ceph-mon[47512]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-08T21:15:43.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:43 smithi082 ceph-mon[47512]: Cluster is now healthy 2024-05-08T21:15:45.161 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:45 smithi033 ceph-mon[42528]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:15:45.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:45 smithi071 ceph-mon[47074]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:15:45.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:45 smithi082 ceph-mon[47512]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:15:47.395 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:47 smithi033 ceph-mon[42528]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-08T21:15:47.395 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:47 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:47.395 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:47 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:47.396 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:47 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:47 smithi071 ceph-mon[47074]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-08T21:15:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:47 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:47 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:47.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:47 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:47.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:47 smithi082 ceph-mon[47512]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-08T21:15:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:47 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:47 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:47.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:47 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:49.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:49 smithi033 ceph-mon[42528]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-08T21:15:49.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:49.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:49.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:49.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:49.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.458+0000 7ff67a7fc700 1 -- 172.21.15.33:0/3348657178 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff68402ce40 con 0x7ff6700a11a0 2024-05-08T21:15:49.461 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 9 on host 'smithi033' 2024-05-08T21:15:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.461+0000 7ff68b609700 1 -- 172.21.15.33:0/3348657178 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff6700a11a0 msgr2=0x7ff6700a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.461+0000 7ff68b609700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff6700a11a0 0x7ff6700a3660 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff680005950 tx=0x7ff68000b410 comp rx=0 tx=0).stop 2024-05-08T21:15:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.461+0000 7ff68b609700 1 -- 172.21.15.33:0/3348657178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff684101a20 msgr2=0x7ff68419e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.461+0000 7ff68b609700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff684101a20 0x7ff68419e130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00ef30 tx=0x7ff67c00c3a0 comp rx=0 tx=0).stop 2024-05-08T21:15:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.461+0000 7ff68b609700 1 -- 172.21.15.33:0/3348657178 shutdown_connections 2024-05-08T21:15:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.461+0000 7ff68b609700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff684101a20 0x7ff68419e130 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.461+0000 7ff68b609700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6840fff00 0x7ff684069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.461+0000 7ff68b609700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff6700a11a0 0x7ff6700a3660 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.461+0000 7ff68b609700 1 --2- 172.21.15.33:0/3348657178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff684101060 0x7ff68419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.461+0000 7ff68b609700 1 -- 172.21.15.33:0/3348657178 >> 172.21.15.33:0/3348657178 conn(0x7ff6840fb4a0 msgr2=0x7ff6841044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:15:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.461+0000 7ff68b609700 1 -- 172.21.15.33:0/3348657178 shutdown_connections 2024-05-08T21:15:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:15:49.461+0000 7ff68b609700 1 -- 172.21.15.33:0/3348657178 wait complete. 2024-05-08T21:15:49.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:49 smithi071 ceph-mon[47074]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-08T21:15:49.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:49.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:49.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:49.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:49.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:49 smithi082 ceph-mon[47512]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-08T21:15:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:49.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:49.877 DEBUG:teuthology.orchestra.run.smithi033:osd.9> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.9.service 2024-05-08T21:15:49.879 INFO:tasks.cephadm:Deploying osd.10 on smithi071 with /dev/vg_nvme/lv_1... 2024-05-08T21:15:49.879 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T21:15:49.921 INFO:journalctl@ceph.osd.9.smithi033.stdout:-- Logs begin at Wed 2024-05-08 20:53:21 UTC. -- 2024-05-08T21:15:50.111 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:15:50.519 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:50.520 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:50.520 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:50.688 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:50.688 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:50.688 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:50.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:50.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:50.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:15:50.958 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:15:50 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:15:50.683+0000 7fcf3ba4f7c0 -1 osd.9 0 log_to_monitors true 2024-05-08T21:15:51.401 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T21:15:51.401 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T21:15:51.401 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-05-08T21:15:51.401 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-05-08T21:15:51.401 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111984 s, 936 MB/s 2024-05-08T21:15:51.402 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-05-08T21:15:51.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:51 smithi033 ceph-mon[42528]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T21:15:51.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:51 smithi033 ceph-mon[42528]: from='osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T21:15:51.773 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:51 smithi071 ceph-mon[47074]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T21:15:51.773 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:51 smithi071 ceph-mon[47074]: from='osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T21:15:51.817 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_1 2024-05-08T21:15:51.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:51 smithi082 ceph-mon[47512]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T21:15:51.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:51 smithi082 ceph-mon[47512]: from='osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T21:15:52.035 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:15:52.428 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:15:52 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:15:52.419+0000 7fcf310b3700 -1 osd.9 0 waiting for initial osdmap 2024-05-08T21:15:52.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T21:15:52.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: osdmap e62: 10 total, 9 up, 10 in 2024-05-08T21:15:52.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:52.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:15:52.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:52.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:52.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:52.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:52.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:52.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:52.710 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:52.710 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:52.710 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:52.710 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:52.710 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-08T21:15:52.710 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: osdmap e63: 10 total, 9 up, 10 in 2024-05-08T21:15:52.710 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:52.710 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:15:52 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:15:52.441+0000 7fcf2cea8700 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:15:52.780 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T21:15:52.780 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: osdmap e62: 10 total, 9 up, 10 in 2024-05-08T21:15:52.780 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:52.780 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:15:52.780 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:52.781 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:52.781 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:52.781 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:52.781 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:52.781 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:52.781 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:52.781 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:52.781 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:52.781 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:52.781 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-08T21:15:52.781 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: osdmap e63: 10 total, 9 up, 10 in 2024-05-08T21:15:52.781 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:52.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T21:15:52.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: osdmap e62: 10 total, 9 up, 10 in 2024-05-08T21:15:52.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:52.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-05-08T21:15:52.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:52.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:52.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:52.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:52.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:52.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:15:52.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:52.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:52.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:15:52.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:15:52.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-05-08T21:15:52.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: osdmap e63: 10 total, 9 up, 10 in 2024-05-08T21:15:52.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.945+0000 7f4e439bf700 1 -- 172.21.15.71:0/920526350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c1043c0 msgr2=0x7f4e3c1067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.945+0000 7f4e439bf700 1 --2- 172.21.15.71:0/920526350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c1043c0 0x7f4e3c1067b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e380098f0 tx=0x7f4e38009c00 comp rx=0 tx=0).stop 2024-05-08T21:15:52.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.945+0000 7f4e439bf700 1 -- 172.21.15.71:0/920526350 shutdown_connections 2024-05-08T21:15:52.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.945+0000 7f4e439bf700 1 --2- 172.21.15.71:0/920526350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c106cf0 0x7f4e3c109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:52.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.945+0000 7f4e439bf700 1 --2- 172.21.15.71:0/920526350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c101a90 0x7f4e3c103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:52.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.945+0000 7f4e439bf700 1 --2- 172.21.15.71:0/920526350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c1043c0 0x7f4e3c1067b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:52.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.945+0000 7f4e439bf700 1 -- 172.21.15.71:0/920526350 >> 172.21.15.71:0/920526350 conn(0x7f4e3c0fb4a0 msgr2=0x7f4e3c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:15:52.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.945+0000 7f4e439bf700 1 -- 172.21.15.71:0/920526350 shutdown_connections 2024-05-08T21:15:52.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.945+0000 7f4e439bf700 1 -- 172.21.15.71:0/920526350 wait complete. 2024-05-08T21:15:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.946+0000 7f4e439bf700 1 Processor -- start 2024-05-08T21:15:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.946+0000 7f4e439bf700 1 -- start start 2024-05-08T21:15:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.946+0000 7f4e439bf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c101a90 0x7f4e3c196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:15:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.946+0000 7f4e439bf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c1043c0 0x7f4e3c197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:15:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.946+0000 7f4e439bf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c106cf0 0x7f4e3c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:15:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.946+0000 7f4e439bf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c19ea30 con 0x7f4e3c101a90 2024-05-08T21:15:52.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.946+0000 7f4e439bf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c19eb70 con 0x7f4e3c106cf0 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.946+0000 7f4e439bf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c19ed00 con 0x7f4e3c1043c0 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.946+0000 7f4e41f5c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c106cf0 0x7f4e3c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.946+0000 7f4e41f5c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c106cf0 0x7f4e3c19e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36262/0 (socket says 172.21.15.71:36262) 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.946+0000 7f4e41f5c700 1 -- 172.21.15.71:0/1031479405 learned_addr learned my addr 172.21.15.71:0/1031479405 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e4175b700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c101a90 0x7f4e3c196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e40f5a700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c1043c0 0x7f4e3c197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e41f5c700 1 -- 172.21.15.71:0/1031479405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c1043c0 msgr2=0x7f4e3c197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e41f5c700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c1043c0 0x7f4e3c197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e41f5c700 1 -- 172.21.15.71:0/1031479405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c101a90 msgr2=0x7f4e3c196ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e41f5c700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c101a90 0x7f4e3c196ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e41f5c700 1 -- 172.21.15.71:0/1031479405 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38009520 con 0x7f4e3c106cf0 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e41f5c700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c106cf0 0x7f4e3c19e460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e340094f0 tx=0x7f4e3400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:15:52.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e327fc700 1 -- 172.21.15.71:0/1031479405 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3400ddd0 con 0x7f4e3c106cf0 2024-05-08T21:15:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e327fc700 1 -- 172.21.15.71:0/1031479405 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e3400f040 con 0x7f4e3c106cf0 2024-05-08T21:15:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e439bf700 1 -- 172.21.15.71:0/1031479405 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3c19efe0 con 0x7f4e3c106cf0 2024-05-08T21:15:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e327fc700 1 -- 172.21.15.71:0/1031479405 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e340058d0 con 0x7f4e3c106cf0 2024-05-08T21:15:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.947+0000 7f4e439bf700 1 -- 172.21.15.71:0/1031479405 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c19f450 con 0x7f4e3c106cf0 2024-05-08T21:15:52.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.948+0000 7f4e439bf700 1 -- 172.21.15.71:0/1031479405 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3c0fd020 con 0x7f4e3c106cf0 2024-05-08T21:15:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.949+0000 7f4e327fc700 1 -- 172.21.15.71:0/1031479405 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4e34004fa0 con 0x7f4e3c106cf0 2024-05-08T21:15:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.950+0000 7f4e327fc700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e280a11a0 0x7f4e280a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:15:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.950+0000 7f4e327fc700 1 -- 172.21.15.71:0/1031479405 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5981+0+0 (secure 0 0 0) 0x7f4e340cbf10 con 0x7f4e3c106cf0 2024-05-08T21:15:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.950+0000 7f4e4175b700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e280a11a0 0x7f4e280a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:15:52.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.950+0000 7f4e4175b700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e280a11a0 0x7f4e280a3660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c005fd0 tx=0x7f4e2c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:15:52.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:52.953+0000 7f4e327fc700 1 -- 172.21.15.71:0/1031479405 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4e34095fc0 con 0x7f4e3c106cf0 2024-05-08T21:15:53.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:15:53.151+0000 7f4e439bf700 1 -- 172.21.15.71:0/1031479405 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e3c02ce40 con 0x7f4e280a11a0 2024-05-08T21:15:53.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:53 smithi033 ceph-mon[42528]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:15:53.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:53 smithi033 ceph-mon[42528]: Detected new or changed devices on smithi033 2024-05-08T21:15:53.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:53 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi033 to 4279M 2024-05-08T21:15:53.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:53.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:15:53.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:15:53.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:53.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:53.794 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:53 smithi071 ceph-mon[47074]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:15:53.794 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:53 smithi071 ceph-mon[47074]: Detected new or changed devices on smithi033 2024-05-08T21:15:53.794 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:53 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi033 to 4279M 2024-05-08T21:15:53.794 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:53.794 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:15:53.794 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:15:53.795 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:53.795 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:53.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:53 smithi082 ceph-mon[47512]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:15:53.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:53 smithi082 ceph-mon[47512]: Detected new or changed devices on smithi033 2024-05-08T21:15:53.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:53 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi033 to 4279M 2024-05-08T21:15:53.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:53.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:15:53.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:15:53.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:15:53.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:54.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:54 smithi033 ceph-mon[42528]: purged_snaps scrub starts 2024-05-08T21:15:54.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:54 smithi033 ceph-mon[42528]: purged_snaps scrub ok 2024-05-08T21:15:54.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:54 smithi033 ceph-mon[42528]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:15:54.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:54 smithi033 ceph-mon[42528]: osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180] boot 2024-05-08T21:15:54.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:54 smithi033 ceph-mon[42528]: osdmap e64: 10 total, 10 up, 10 in 2024-05-08T21:15:54.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:54 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:54.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:54 smithi071 ceph-mon[47074]: purged_snaps scrub starts 2024-05-08T21:15:54.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:54 smithi071 ceph-mon[47074]: purged_snaps scrub ok 2024-05-08T21:15:54.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:54 smithi071 ceph-mon[47074]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:15:54.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:54 smithi071 ceph-mon[47074]: osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180] boot 2024-05-08T21:15:54.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:54 smithi071 ceph-mon[47074]: osdmap e64: 10 total, 10 up, 10 in 2024-05-08T21:15:54.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:54 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:54.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:54 smithi082 ceph-mon[47512]: purged_snaps scrub starts 2024-05-08T21:15:54.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:54 smithi082 ceph-mon[47512]: purged_snaps scrub ok 2024-05-08T21:15:54.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:54 smithi082 ceph-mon[47512]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:15:54.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:54 smithi082 ceph-mon[47512]: osd.9 [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180] boot 2024-05-08T21:15:54.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:54 smithi082 ceph-mon[47512]: osdmap e64: 10 total, 10 up, 10 in 2024-05-08T21:15:54.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:54 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T21:15:55.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:55 smithi033 ceph-mon[42528]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:15:55.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:55 smithi033 ceph-mon[42528]: osdmap e65: 10 total, 10 up, 10 in 2024-05-08T21:15:55.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:55 smithi033 ceph-mon[42528]: from='client.? 172.21.15.71:0/759489499' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "768308bc-54b7-43ff-af95-87a481d38bb0"}]: dispatch 2024-05-08T21:15:55.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:55 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "768308bc-54b7-43ff-af95-87a481d38bb0"}]: dispatch 2024-05-08T21:15:55.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:55 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "768308bc-54b7-43ff-af95-87a481d38bb0"}]': finished 2024-05-08T21:15:55.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:55 smithi033 ceph-mon[42528]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T21:15:55.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:55 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:15:55.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:55 smithi071 ceph-mon[47074]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:15:55.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:55 smithi071 ceph-mon[47074]: osdmap e65: 10 total, 10 up, 10 in 2024-05-08T21:15:55.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:55 smithi071 ceph-mon[47074]: from='client.? 172.21.15.71:0/759489499' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "768308bc-54b7-43ff-af95-87a481d38bb0"}]: dispatch 2024-05-08T21:15:55.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:55 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "768308bc-54b7-43ff-af95-87a481d38bb0"}]: dispatch 2024-05-08T21:15:55.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:55 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "768308bc-54b7-43ff-af95-87a481d38bb0"}]': finished 2024-05-08T21:15:55.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:55 smithi071 ceph-mon[47074]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T21:15:55.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:55 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:15:55.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:55 smithi082 ceph-mon[47512]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-08T21:15:55.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:55 smithi082 ceph-mon[47512]: osdmap e65: 10 total, 10 up, 10 in 2024-05-08T21:15:55.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:55 smithi082 ceph-mon[47512]: from='client.? 172.21.15.71:0/759489499' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "768308bc-54b7-43ff-af95-87a481d38bb0"}]: dispatch 2024-05-08T21:15:55.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:55 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "768308bc-54b7-43ff-af95-87a481d38bb0"}]: dispatch 2024-05-08T21:15:55.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:55 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "768308bc-54b7-43ff-af95-87a481d38bb0"}]': finished 2024-05-08T21:15:55.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:55 smithi082 ceph-mon[47512]: osdmap e66: 11 total, 10 up, 11 in 2024-05-08T21:15:55.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:55 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:15:56.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:56 smithi033 ceph-mon[42528]: from='client.? 172.21.15.71:0/1584610050' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:15:56.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:56 smithi033 ceph-mon[42528]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:15:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:56 smithi071 ceph-mon[47074]: from='client.? 172.21.15.71:0/1584610050' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:15:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:56 smithi071 ceph-mon[47074]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:15:56.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:56 smithi082 ceph-mon[47512]: from='client.? 172.21.15.71:0/1584610050' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:15:56.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:56 smithi082 ceph-mon[47512]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:15:59.444 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:15:59 smithi071 ceph-mon[47074]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:15:59.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:15:59 smithi033 ceph-mon[42528]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:15:59.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:15:59 smithi082 ceph-mon[47512]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:01.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:01 smithi033 ceph-mon[42528]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:01.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T21:16:01.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:01.484 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:01 smithi071 ceph-mon[47074]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:01.484 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T21:16:01.484 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:01.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:01 smithi082 ceph-mon[47512]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T21:16:01.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:02.454 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:02 smithi071 ceph-mon[47074]: Deploying daemon osd.10 on smithi071 2024-05-08T21:16:02.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:02 smithi033 ceph-mon[42528]: Deploying daemon osd.10 on smithi071 2024-05-08T21:16:02.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:02 smithi082 ceph-mon[47512]: Deploying daemon osd.10 on smithi071 2024-05-08T21:16:03.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:03 smithi033 ceph-mon[42528]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:03.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:03 smithi071 ceph-mon[47074]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:03.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:03 smithi082 ceph-mon[47512]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:05.194 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:05 smithi071 ceph-mon[47074]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:05.194 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:05.195 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:05.195 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:05.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:05 smithi033 ceph-mon[42528]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:05.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:05.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:05.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:05 smithi082 ceph-mon[47512]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:06.709 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 10 on host 'smithi071' 2024-05-08T21:16:06.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.705+0000 7f4e327fc700 1 -- 172.21.15.71:0/1031479405 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f4e3c02ce40 con 0x7f4e280a11a0 2024-05-08T21:16:06.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.707+0000 7f4e439bf700 1 -- 172.21.15.71:0/1031479405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e280a11a0 msgr2=0x7f4e280a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:06.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.707+0000 7f4e439bf700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e280a11a0 0x7f4e280a3660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c005fd0 tx=0x7f4e2c009310 comp rx=0 tx=0).stop 2024-05-08T21:16:06.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.707+0000 7f4e439bf700 1 -- 172.21.15.71:0/1031479405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c106cf0 msgr2=0x7f4e3c19e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:06.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.707+0000 7f4e439bf700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c106cf0 0x7f4e3c19e460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e340094f0 tx=0x7f4e3400d3b0 comp rx=0 tx=0).stop 2024-05-08T21:16:06.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.707+0000 7f4e439bf700 1 -- 172.21.15.71:0/1031479405 shutdown_connections 2024-05-08T21:16:06.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.707+0000 7f4e439bf700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e280a11a0 0x7f4e280a3660 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:06.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.707+0000 7f4e439bf700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c106cf0 0x7f4e3c19e460 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:06.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.707+0000 7f4e439bf700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c101a90 0x7f4e3c196ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:06.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.707+0000 7f4e439bf700 1 --2- 172.21.15.71:0/1031479405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c1043c0 0x7f4e3c197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:06.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.707+0000 7f4e439bf700 1 -- 172.21.15.71:0/1031479405 >> 172.21.15.71:0/1031479405 conn(0x7f4e3c0fb4a0 msgr2=0x7f4e3c102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:06.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.707+0000 7f4e439bf700 1 -- 172.21.15.71:0/1031479405 shutdown_connections 2024-05-08T21:16:06.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-05-08T21:16:06.707+0000 7f4e439bf700 1 -- 172.21.15.71:0/1031479405 wait complete. 2024-05-08T21:16:07.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:06 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:06 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:06 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:07.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:06 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:07.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:06 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:06 smithi071 ceph-mon[47074]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:07.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:06 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:06 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:06 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:07.088 DEBUG:teuthology.orchestra.run.smithi071:osd.10> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.10.service 2024-05-08T21:16:07.090 INFO:tasks.cephadm:Deploying osd.11 on smithi082 with /dev/vg_nvme/lv_1... 2024-05-08T21:16:07.090 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T21:16:07.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:06 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:06 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:06 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:07.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:06 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:07.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:06 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:06 smithi082 ceph-mon[47512]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:07.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:06 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:06 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:06 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:07.119 INFO:journalctl@ceph.osd.10.smithi071.stdout:-- Logs begin at Wed 2024-05-08 20:53:45 UTC. -- 2024-05-08T21:16:07.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:06 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:06 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:06 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:07.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:06 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:07.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:06 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:06 smithi033 ceph-mon[42528]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:07.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:06 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:06 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:07.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:06 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:07.323 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:16:08.761 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T21:16:08.761 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T21:16:08.761 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2024-05-08T21:16:08.761 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2024-05-08T21:16:08.761 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0152565 s, 687 MB/s 2024-05-08T21:16:08.762 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 2024-05-08T21:16:08.762 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2024-05-08T21:16:08.821 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:16:08 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:16:08.460+0000 7fb4420697c0 -1 osd.10 0 log_to_monitors true 2024-05-08T21:16:09.124 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi082:vg_nvme/lv_1 2024-05-08T21:16:09.384 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:16:09.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:09 smithi033 ceph-mon[42528]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:09.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:09 smithi033 ceph-mon[42528]: from='osd.10 [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T21:16:09.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:09 smithi033 ceph-mon[42528]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T21:16:09.496 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:09 smithi082 ceph-mon[47512]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:09.496 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:09 smithi082 ceph-mon[47512]: from='osd.10 [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T21:16:09.496 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:09 smithi082 ceph-mon[47512]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T21:16:09.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:09 smithi071 ceph-mon[47074]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:09.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:09 smithi071 ceph-mon[47074]: from='osd.10 [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T21:16:09.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:09 smithi071 ceph-mon[47074]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T21:16:10.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.406+0000 7fc1bc1de700 1 -- 172.21.15.82:0/3994598081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1b4100fe0 msgr2=0x7fc1b4101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:10.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.406+0000 7fc1bc1de700 1 --2- 172.21.15.82:0/3994598081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1b4100fe0 0x7fc1b4101400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009990 tx=0x7fc1a4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:16:10.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.406+0000 7fc1bc1de700 1 -- 172.21.15.82:0/3994598081 shutdown_connections 2024-05-08T21:16:10.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.406+0000 7fc1bc1de700 1 --2- 172.21.15.82:0/3994598081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4102b80 0x7fc1b4109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:10.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.406+0000 7fc1bc1de700 1 --2- 172.21.15.82:0/3994598081 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1b4100fe0 0x7fc1b4101400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:10.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.406+0000 7fc1bc1de700 1 --2- 172.21.15.82:0/3994598081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1b41021e0 0x7fc1b4102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:10.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.406+0000 7fc1bc1de700 1 -- 172.21.15.82:0/3994598081 >> 172.21.15.82:0/3994598081 conn(0x7fc1b40fc580 msgr2=0x7fc1b40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:10.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.406+0000 7fc1bc1de700 1 -- 172.21.15.82:0/3994598081 shutdown_connections 2024-05-08T21:16:10.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.406+0000 7fc1bc1de700 1 -- 172.21.15.82:0/3994598081 wait complete. 2024-05-08T21:16:10.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.407+0000 7fc1bc1de700 1 Processor -- start 2024-05-08T21:16:10.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.407+0000 7fc1bc1de700 1 -- start start 2024-05-08T21:16:10.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.407+0000 7fc1bc1de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1b4100fe0 0x7fc1b4196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:10.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.407+0000 7fc1bc1de700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1b41021e0 0x7fc1b41972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:10.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.407+0000 7fc1bc1de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4102b80 0x7fc1b419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:10.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.407+0000 7fc1bc1de700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b419e8f0 con 0x7fc1b4102b80 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.407+0000 7fc1bc1de700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b419ea30 con 0x7fc1b41021e0 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.407+0000 7fc1bc1de700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b419ebc0 con 0x7fc1b4100fe0 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.407+0000 7fc1b9f7a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1b4100fe0 0x7fc1b4196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.407+0000 7fc1b9f7a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1b4100fe0 0x7fc1b4196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40216/0 (socket says 172.21.15.82:40216) 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.407+0000 7fc1b9f7a700 1 -- 172.21.15.82:0/2171632430 learned_addr learned my addr 172.21.15.82:0/2171632430 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.407+0000 7fc1b9779700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1b41021e0 0x7fc1b41972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1ba77b700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4102b80 0x7fc1b419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1b9f7a700 1 -- 172.21.15.82:0/2171632430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1b41021e0 msgr2=0x7fc1b41972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1b9f7a700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1b41021e0 0x7fc1b41972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1b9f7a700 1 -- 172.21.15.82:0/2171632430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4102b80 msgr2=0x7fc1b419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1b9f7a700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4102b80 0x7fc1b419e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1b9f7a700 1 -- 172.21.15.82:0/2171632430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a40095f0 con 0x7fc1b4100fe0 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1ba77b700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4102b80 0x7fc1b419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:16:10.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1b9f7a700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1b4100fe0 0x7fc1b4196db0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40095c0 tx=0x7fc1a4005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:10.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1aaffd700 1 -- 172.21.15.82:0/2171632430 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a401d070 con 0x7fc1b4100fe0 2024-05-08T21:16:10.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1aaffd700 1 -- 172.21.15.82:0/2171632430 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1a4004000 con 0x7fc1b4100fe0 2024-05-08T21:16:10.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1bc1de700 1 -- 172.21.15.82:0/2171632430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b419ee40 con 0x7fc1b4100fe0 2024-05-08T21:16:10.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1aaffd700 1 -- 172.21.15.82:0/2171632430 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4022cd0 con 0x7fc1b4100fe0 2024-05-08T21:16:10.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.408+0000 7fc1bc1de700 1 -- 172.21.15.82:0/2171632430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b419f1a0 con 0x7fc1b4100fe0 2024-05-08T21:16:10.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.409+0000 7fc1bc1de700 1 -- 172.21.15.82:0/2171632430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b41084c0 con 0x7fc1b4100fe0 2024-05-08T21:16:10.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.410+0000 7fc1aaffd700 1 -- 172.21.15.82:0/2171632430 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc1a4022470 con 0x7fc1b4100fe0 2024-05-08T21:16:10.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.410+0000 7fc1aaffd700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1a00a1120 0x7fc1a00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:10.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.411+0000 7fc1aaffd700 1 -- 172.21.15.82:0/2171632430 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc1a40cd6e0 con 0x7fc1b4100fe0 2024-05-08T21:16:10.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.411+0000 7fc1b9779700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1a00a1120 0x7fc1a00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:10.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.411+0000 7fc1b9779700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1a00a1120 0x7fc1a00a35e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000ac50 tx=0x7fc1b000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:10.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.415+0000 7fc1aaffd700 1 -- 172.21.15.82:0/2171632430 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc1a40975d0 con 0x7fc1b4100fe0 2024-05-08T21:16:10.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: Detected new or changed devices on smithi071 2024-05-08T21:16:10.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:10.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T21:16:10.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: osdmap e67: 11 total, 10 up, 11 in 2024-05-08T21:16:10.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:16:10.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='osd.10 [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:16:10.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:10.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:16:10.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:10.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:10.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:10.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:10.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi071 to 4278M 2024-05-08T21:16:10.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:10.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:10.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:10.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:10 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:10.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: Detected new or changed devices on smithi071 2024-05-08T21:16:10.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:10.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T21:16:10.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: osdmap e67: 11 total, 10 up, 11 in 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='osd.10 [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi071 to 4278M 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:10.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:10 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:10.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: Detected new or changed devices on smithi071 2024-05-08T21:16:10.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:10.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T21:16:10.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: osdmap e67: 11 total, 10 up, 11 in 2024-05-08T21:16:10.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:16:10.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='osd.10 [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:16:10.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:10.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-05-08T21:16:10.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:10.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:10.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:10.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:10.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi071 to 4278M 2024-05-08T21:16:10.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:10.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:10.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:10.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:10 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:10.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:10.614+0000 7fc1bc1de700 1 -- 172.21.15.82:0/2171632430 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1b402cf00 con 0x7fc1a00a1120 2024-05-08T21:16:11.178 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:16:10 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:16:10.840+0000 7fb4376cd700 -1 osd.10 0 waiting for initial osdmap 2024-05-08T21:16:11.178 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:16:10 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:16:10.853+0000 7fb42f4ba700 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:16:11.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:11 smithi082 ceph-mon[47512]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:11.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:11 smithi082 ceph-mon[47512]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-08T21:16:11.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:11 smithi082 ceph-mon[47512]: osdmap e68: 11 total, 10 up, 11 in 2024-05-08T21:16:11.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:16:11.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:16:11.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:16:11.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:16:11.355 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:11.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:11 smithi082 ceph-mon[47512]: from='osd.10 ' entity='osd.10' 2024-05-08T21:16:11.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:11 smithi082 ceph-mon[47512]: osd.10 [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576] boot 2024-05-08T21:16:11.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:11 smithi082 ceph-mon[47512]: osdmap e69: 11 total, 11 up, 11 in 2024-05-08T21:16:11.356 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:16:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:11 smithi033 ceph-mon[42528]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:11 smithi033 ceph-mon[42528]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-08T21:16:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:11 smithi033 ceph-mon[42528]: osdmap e68: 11 total, 10 up, 11 in 2024-05-08T21:16:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:16:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:16:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:16:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:16:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:11 smithi033 ceph-mon[42528]: from='osd.10 ' entity='osd.10' 2024-05-08T21:16:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:11 smithi033 ceph-mon[42528]: osd.10 [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576] boot 2024-05-08T21:16:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:11 smithi033 ceph-mon[42528]: osdmap e69: 11 total, 11 up, 11 in 2024-05-08T21:16:11.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:16:11.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:11 smithi071 ceph-mon[47074]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-05-08T21:16:11.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:11 smithi071 ceph-mon[47074]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-05-08T21:16:11.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:11 smithi071 ceph-mon[47074]: osdmap e68: 11 total, 10 up, 11 in 2024-05-08T21:16:11.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:16:11.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:16:11.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T21:16:11.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T21:16:11.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:11.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:11 smithi071 ceph-mon[47074]: from='osd.10 ' entity='osd.10' 2024-05-08T21:16:11.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:11 smithi071 ceph-mon[47074]: osd.10 [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576] boot 2024-05-08T21:16:11.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:11 smithi071 ceph-mon[47074]: osdmap e69: 11 total, 11 up, 11 in 2024-05-08T21:16:11.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T21:16:12.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:12 smithi033 ceph-mon[42528]: purged_snaps scrub starts 2024-05-08T21:16:12.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:12 smithi033 ceph-mon[42528]: purged_snaps scrub ok 2024-05-08T21:16:12.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:12 smithi033 ceph-mon[42528]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:16:12.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:12 smithi033 ceph-mon[42528]: osdmap e70: 11 total, 11 up, 11 in 2024-05-08T21:16:12.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:12 smithi071 ceph-mon[47074]: purged_snaps scrub starts 2024-05-08T21:16:12.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:12 smithi071 ceph-mon[47074]: purged_snaps scrub ok 2024-05-08T21:16:12.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:12 smithi071 ceph-mon[47074]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:16:12.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:12 smithi071 ceph-mon[47074]: osdmap e70: 11 total, 11 up, 11 in 2024-05-08T21:16:12.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:12 smithi082 ceph-mon[47512]: purged_snaps scrub starts 2024-05-08T21:16:12.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:12 smithi082 ceph-mon[47512]: purged_snaps scrub ok 2024-05-08T21:16:12.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:12 smithi082 ceph-mon[47512]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:16:12.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:12 smithi082 ceph-mon[47512]: osdmap e70: 11 total, 11 up, 11 in 2024-05-08T21:16:13.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:13 smithi033 ceph-mon[42528]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 647 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:16:13.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:13 smithi033 ceph-mon[42528]: from='client.? 172.21.15.82:0/494015753' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c"}]: dispatch 2024-05-08T21:16:13.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:13 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c"}]: dispatch 2024-05-08T21:16:13.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:13 smithi033 ceph-mon[42528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c"}]': finished 2024-05-08T21:16:13.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:13 smithi033 ceph-mon[42528]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T21:16:13.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:13.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:13 smithi033 ceph-mon[42528]: from='client.? 172.21.15.82:0/1896661019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:16:13.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:13 smithi071 ceph-mon[47074]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 647 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:16:13.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:13 smithi071 ceph-mon[47074]: from='client.? 172.21.15.82:0/494015753' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c"}]: dispatch 2024-05-08T21:16:13.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:13 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c"}]: dispatch 2024-05-08T21:16:13.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:13 smithi071 ceph-mon[47074]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c"}]': finished 2024-05-08T21:16:13.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:13 smithi071 ceph-mon[47074]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T21:16:13.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:13.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:13 smithi071 ceph-mon[47074]: from='client.? 172.21.15.82:0/1896661019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:16:13.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:13 smithi082 ceph-mon[47512]: pgmap v187: 1 pgs: 1 remapped+peering; 577 KiB data, 647 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:16:13.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:13 smithi082 ceph-mon[47512]: from='client.? 172.21.15.82:0/494015753' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c"}]: dispatch 2024-05-08T21:16:13.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:13 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c"}]: dispatch 2024-05-08T21:16:13.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:13 smithi082 ceph-mon[47512]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c"}]': finished 2024-05-08T21:16:13.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:13 smithi082 ceph-mon[47512]: osdmap e71: 12 total, 11 up, 12 in 2024-05-08T21:16:13.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:13.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:13 smithi082 ceph-mon[47512]: from='client.? 172.21.15.82:0/1896661019' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T21:16:15.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:15 smithi033 ceph-mon[42528]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 647 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:16:15.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:15 smithi033 ceph-mon[42528]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T21:16:15.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:15.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:15 smithi071 ceph-mon[47074]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 647 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:16:15.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:15 smithi071 ceph-mon[47074]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T21:16:15.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:15.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:15 smithi082 ceph-mon[47512]: pgmap v190: 1 pgs: 1 remapped+peering; 577 KiB data, 647 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:16:15.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:15 smithi082 ceph-mon[47512]: osdmap e72: 12 total, 11 up, 12 in 2024-05-08T21:16:15.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:16.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:16 smithi033 ceph-mon[42528]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T21:16:16.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:16.532 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:16 smithi082 ceph-mon[47512]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T21:16:16.532 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:16.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:16 smithi071 ceph-mon[47074]: osdmap e73: 12 total, 11 up, 12 in 2024-05-08T21:16:16.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:17.291 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:17 smithi082 ceph-mon[47512]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 647 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:16:17.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:17 smithi033 ceph-mon[42528]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 647 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:16:17.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:17 smithi071 ceph-mon[47074]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 647 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:16:19.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:19 smithi033 ceph-mon[42528]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-08T21:16:19.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T21:16:19.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:19.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:19 smithi033 ceph-mon[42528]: Deploying daemon osd.11 on smithi082 2024-05-08T21:16:19.553 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:19 smithi082 ceph-mon[47512]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-08T21:16:19.554 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T21:16:19.554 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:19.554 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:19 smithi082 ceph-mon[47512]: Deploying daemon osd.11 on smithi082 2024-05-08T21:16:19.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:19 smithi071 ceph-mon[47074]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-08T21:16:19.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T21:16:19.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:19.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:19 smithi071 ceph-mon[47074]: Deploying daemon osd.11 on smithi082 2024-05-08T21:16:21.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:21 smithi082 ceph-mon[47512]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T21:16:21.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:21 smithi071 ceph-mon[47074]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T21:16:21.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:21 smithi033 ceph-mon[42528]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T21:16:23.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:23 smithi033 ceph-mon[42528]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:16:23.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:23 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:23.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:23 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:23.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:23 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:23.486 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:23 smithi082 ceph-mon[47512]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:16:23.486 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:23 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:23.486 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:23 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:23.486 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:23 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:23.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:23 smithi071 ceph-mon[47074]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T21:16:23.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:23 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:23.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:23 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:23.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:23 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:24.371 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 11 on host 'smithi082' 2024-05-08T21:16:24.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.368+0000 7fc1aaffd700 1 -- 172.21.15.82:0/2171632430 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc1b402cf00 con 0x7fc1a00a1120 2024-05-08T21:16:24.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.370+0000 7fc1bc1de700 1 -- 172.21.15.82:0/2171632430 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1a00a1120 msgr2=0x7fc1a00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:24.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.370+0000 7fc1bc1de700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1a00a1120 0x7fc1a00a35e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000ac50 tx=0x7fc1b000a380 comp rx=0 tx=0).stop 2024-05-08T21:16:24.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.370+0000 7fc1bc1de700 1 -- 172.21.15.82:0/2171632430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1b4100fe0 msgr2=0x7fc1b4196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:24.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.370+0000 7fc1bc1de700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1b4100fe0 0x7fc1b4196db0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc1a40095c0 tx=0x7fc1a4005740 comp rx=0 tx=0).stop 2024-05-08T21:16:24.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.370+0000 7fc1bc1de700 1 -- 172.21.15.82:0/2171632430 shutdown_connections 2024-05-08T21:16:24.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.370+0000 7fc1bc1de700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4102b80 0x7fc1b419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:24.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.370+0000 7fc1bc1de700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1b4100fe0 0x7fc1b4196db0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:24.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.370+0000 7fc1bc1de700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1a00a1120 0x7fc1a00a35e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:24.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.370+0000 7fc1bc1de700 1 --2- 172.21.15.82:0/2171632430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1b41021e0 0x7fc1b41972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:24.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.370+0000 7fc1bc1de700 1 -- 172.21.15.82:0/2171632430 >> 172.21.15.82:0/2171632430 conn(0x7fc1b40fc580 msgr2=0x7fc1b4105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:24.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.370+0000 7fc1bc1de700 1 -- 172.21.15.82:0/2171632430 shutdown_connections 2024-05-08T21:16:24.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:24.370+0000 7fc1bc1de700 1 -- 172.21.15.82:0/2171632430 wait complete. 2024-05-08T21:16:24.805 DEBUG:teuthology.orchestra.run.smithi082:osd.11> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.11.service 2024-05-08T21:16:24.807 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-08T21:16:24.808 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T21:16:24.831 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:24.831 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:24.831 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:24.831 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:24.831 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:24.831 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:24 smithi082 ceph-mon[47512]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T21:16:24.831 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:24.831 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:24.832 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:24 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:24.836 INFO:journalctl@ceph.osd.11.smithi082.stdout:-- Logs begin at Wed 2024-05-08 20:53:53 UTC. -- 2024-05-08T21:16:24.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:24.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:24.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:24.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:24.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:24.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:24 smithi033 ceph-mon[42528]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T21:16:24.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:24.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:24.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:24 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:25.068 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:16:25.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:25.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:25.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:25.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:25.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:25.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:24 smithi071 ceph-mon[47074]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T21:16:25.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:25.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:25.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:24 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:26.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.398+0000 7f07640fc700 1 -- 172.21.15.33:0/3815886018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f075c101060 msgr2=0x7f075c1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:26.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.398+0000 7f07640fc700 1 --2- 172.21.15.33:0/3815886018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f075c101060 0x7f075c1014e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f074c009990 tx=0x7f074c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:16:26.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.399+0000 7f07640fc700 1 -- 172.21.15.33:0/3815886018 shutdown_connections 2024-05-08T21:16:26.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.399+0000 7f07640fc700 1 --2- 172.21.15.33:0/3815886018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f075c101a20 0x7f075c108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:26.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.399+0000 7f07640fc700 1 --2- 172.21.15.33:0/3815886018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f075c0fff00 0x7f075c100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:26.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.399+0000 7f07640fc700 1 --2- 172.21.15.33:0/3815886018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f075c101060 0x7f075c1014e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:26.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.399+0000 7f07640fc700 1 -- 172.21.15.33:0/3815886018 >> 172.21.15.33:0/3815886018 conn(0x7f075c0fb4a0 msgr2=0x7f075c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:26.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.399+0000 7f07640fc700 1 -- 172.21.15.33:0/3815886018 shutdown_connections 2024-05-08T21:16:26.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.399+0000 7f07640fc700 1 -- 172.21.15.33:0/3815886018 wait complete. 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.399+0000 7f07640fc700 1 Processor -- start 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.399+0000 7f07640fc700 1 -- start start 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f07640fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f075c0fff00 0x7f075c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f07640fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f075c101060 0x7f075c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f07640fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f075c101a20 0x7f075c1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f07640fc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c1a0b60 con 0x7f075c101060 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f07640fc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c1a0ca0 con 0x7f075c0fff00 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f07640fc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c1a0e30 con 0x7f075c101a20 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0761697700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f075c101060 0x7f075c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0761697700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f075c101060 0x7f075c199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58620/0 (socket says 172.21.15.33:58620) 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0761697700 1 -- 172.21.15.33:0/3361471694 learned_addr learned my addr 172.21.15.33:0/3361471694 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0762699700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f075c101a20 0x7f075c1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0761e98700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f075c0fff00 0x7f075c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0761697700 1 -- 172.21.15.33:0/3361471694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f075c101a20 msgr2=0x7f075c1a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0761697700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f075c101a20 0x7f075c1a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0761697700 1 -- 172.21.15.33:0/3361471694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f075c0fff00 msgr2=0x7f075c199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0761697700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f075c0fff00 0x7f075c199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0761697700 1 -- 172.21.15.33:0/3361471694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c0095f0 con 0x7f075c101060 2024-05-08T21:16:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0762699700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f075c101a20 0x7f075c1a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:16:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0761697700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f075c101060 0x7f075c199560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f074c00b610 tx=0x7f074c004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.400+0000 7f0752ffd700 1 -- 172.21.15.33:0/3361471694 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c01d070 con 0x7f075c101060 2024-05-08T21:16:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.401+0000 7f0752ffd700 1 -- 172.21.15.33:0/3361471694 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f074c00bd50 con 0x7f075c101060 2024-05-08T21:16:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.401+0000 7f07640fc700 1 -- 172.21.15.33:0/3361471694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075c1a10b0 con 0x7f075c101060 2024-05-08T21:16:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.401+0000 7f0752ffd700 1 -- 172.21.15.33:0/3361471694 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c022d20 con 0x7f075c101060 2024-05-08T21:16:26.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.401+0000 7f07640fc700 1 -- 172.21.15.33:0/3361471694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075c1a1520 con 0x7f075c101060 2024-05-08T21:16:26.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.402+0000 7f07640fc700 1 -- 172.21.15.33:0/3361471694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f075c04ec60 con 0x7f075c101060 2024-05-08T21:16:26.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.403+0000 7f0752ffd700 1 -- 172.21.15.33:0/3361471694 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f074c022470 con 0x7f075c101060 2024-05-08T21:16:26.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.403+0000 7f0752ffd700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f07480a11a0 0x7f07480a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:26.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.404+0000 7f0761e98700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f07480a11a0 0x7f07480a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:26.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.404+0000 7f0752ffd700 1 -- 172.21.15.33:0/3361471694 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f074c0cdae0 con 0x7f075c101060 2024-05-08T21:16:26.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.404+0000 7f0761e98700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f07480a11a0 0x7f07480a3660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0758009c60 tx=0x7f0758009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:26.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.409+0000 7f0752ffd700 1 -- 172.21.15.33:0/3361471694 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f074c097830 con 0x7f075c101060 2024-05-08T21:16:26.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:26 smithi033 ceph-mon[42528]: from='osd.11 [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T21:16:26.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:26 smithi033 ceph-mon[42528]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T21:16:26.554 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:26 smithi082 ceph-mon[47512]: from='osd.11 [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T21:16:26.554 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:26 smithi082 ceph-mon[47512]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T21:16:26.555 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:16:26 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:16:26.135+0000 7fe5b96887c0 -1 osd.11 0 log_to_monitors true 2024-05-08T21:16:26.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:26 smithi071 ceph-mon[47074]: from='osd.11 [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T21:16:26.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:26 smithi071 ceph-mon[47074]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T21:16:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.622+0000 7f07640fc700 1 -- 172.21.15.33:0/3361471694 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f075c10b5c0 con 0x7f075c101060 2024-05-08T21:16:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.622+0000 7f0752ffd700 1 -- 172.21.15.33:0/3361471694 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) v1 ==== 74+0+133 (secure 0 0 0) 0x7f074c09ae50 con 0x7f075c101060 2024-05-08T21:16:26.626 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.625+0000 7f07640fc700 1 -- 172.21.15.33:0/3361471694 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f07480a11a0 msgr2=0x7f07480a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.625+0000 7f07640fc700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f07480a11a0 0x7f07480a3660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0758009c60 tx=0x7f0758009260 comp rx=0 tx=0).stop 2024-05-08T21:16:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.625+0000 7f07640fc700 1 -- 172.21.15.33:0/3361471694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f075c101060 msgr2=0x7f075c199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.625+0000 7f07640fc700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f075c101060 0x7f075c199560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f074c00b610 tx=0x7f074c004710 comp rx=0 tx=0).stop 2024-05-08T21:16:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.625+0000 7f07640fc700 1 -- 172.21.15.33:0/3361471694 shutdown_connections 2024-05-08T21:16:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.625+0000 7f07640fc700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f075c101a20 0x7f075c1a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.625+0000 7f07640fc700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f07480a11a0 0x7f07480a3660 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.625+0000 7f07640fc700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f075c0fff00 0x7f075c199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.625+0000 7f07640fc700 1 --2- 172.21.15.33:0/3361471694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f075c101060 0x7f075c199560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.625+0000 7f07640fc700 1 -- 172.21.15.33:0/3361471694 >> 172.21.15.33:0/3361471694 conn(0x7f075c0fb4a0 msgr2=0x7f075c068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.625+0000 7f07640fc700 1 -- 172.21.15.33:0/3361471694 shutdown_connections 2024-05-08T21:16:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:26.625+0000 7f07640fc700 1 -- 172.21.15.33:0/3361471694 wait complete. 2024-05-08T21:16:27.000 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1715202971,"num_in_osds":12,"osd_in_since":1715202972,"num_remapped_pgs":0} 2024-05-08T21:16:27.207 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-08T21:16:27.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T21:16:27.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: osdmap e74: 12 total, 11 up, 12 in 2024-05-08T21:16:27.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:27.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='osd.11 [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:16:27.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:16:27.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3361471694' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:16:27.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:27.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:27 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:27.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-08T21:16:27.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T21:16:27.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: osdmap e74: 12 total, 11 up, 12 in 2024-05-08T21:16:27.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:27.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='osd.11 [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:16:27.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:16:27.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3361471694' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:16:27.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:27.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:27.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:27.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:27.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:27.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:27.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:27.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:27.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:27.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:27 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:27.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:16:27 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:16:27.208+0000 7fe5aecec700 -1 osd.11 0 waiting for initial osdmap 2024-05-08T21:16:27.604 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:16:27 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:16:27.221+0000 7fe5aaae1700 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T21:16:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-08T21:16:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T21:16:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: osdmap e74: 12 total, 11 up, 12 in 2024-05-08T21:16:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='osd.11 [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:16:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2024-05-08T21:16:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3361471694' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:16:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:16:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:27.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:27 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:28.001 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T21:16:28.262 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:16:28.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:28 smithi033 ceph-mon[42528]: Detected new or changed devices on smithi082 2024-05-08T21:16:28.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:28 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi082 to 4278M 2024-05-08T21:16:28.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:28 smithi033 ceph-mon[42528]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-08T21:16:28.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:28 smithi033 ceph-mon[42528]: osdmap e75: 12 total, 11 up, 12 in 2024-05-08T21:16:28.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:28 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:28.408 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:28 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:28.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:28 smithi071 ceph-mon[47074]: Detected new or changed devices on smithi082 2024-05-08T21:16:28.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:28 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi082 to 4278M 2024-05-08T21:16:28.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:28 smithi071 ceph-mon[47074]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-08T21:16:28.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:28 smithi071 ceph-mon[47074]: osdmap e75: 12 total, 11 up, 12 in 2024-05-08T21:16:28.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:28 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:28.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:28 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:28.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:28 smithi082 ceph-mon[47512]: Detected new or changed devices on smithi082 2024-05-08T21:16:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:28 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi082 to 4278M 2024-05-08T21:16:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:28 smithi082 ceph-mon[47512]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2024-05-08T21:16:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:28 smithi082 ceph-mon[47512]: osdmap e75: 12 total, 11 up, 12 in 2024-05-08T21:16:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:28 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:28.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:28 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:29.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.361+0000 7f536881d700 1 -- 172.21.15.33:0/2139944393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5364101a20 msgr2=0x7f5364108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:29.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.361+0000 7f536881d700 1 --2- 172.21.15.33:0/2139944393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5364101a20 0x7f5364108340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f535c00b0a0 tx=0x7f535c00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:16:29.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.361+0000 7f536881d700 1 -- 172.21.15.33:0/2139944393 shutdown_connections 2024-05-08T21:16:29.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.361+0000 7f536881d700 1 --2- 172.21.15.33:0/2139944393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5364101a20 0x7f5364108340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:29.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.361+0000 7f536881d700 1 --2- 172.21.15.33:0/2139944393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53640fff00 0x7f5364100320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:29.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.361+0000 7f536881d700 1 --2- 172.21.15.33:0/2139944393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5364101060 0x7f53641014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:29.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.361+0000 7f536881d700 1 -- 172.21.15.33:0/2139944393 >> 172.21.15.33:0/2139944393 conn(0x7f53640fb4c0 msgr2=0x7f53640fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:29.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.362+0000 7f536881d700 1 -- 172.21.15.33:0/2139944393 shutdown_connections 2024-05-08T21:16:29.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.362+0000 7f536881d700 1 -- 172.21.15.33:0/2139944393 wait complete. 2024-05-08T21:16:29.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.362+0000 7f536881d700 1 Processor -- start 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536881d700 1 -- start start 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536881d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53640fff00 0x7f5364195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536881d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5364101060 0x7f5364196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536881d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5364101a20 0x7f536419d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536881d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536419d760 con 0x7f53640fff00 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536881d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536419d8a0 con 0x7f5364101060 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536881d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536419da30 con 0x7f5364101a20 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53640fff00 0x7f5364195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53640fff00 0x7f5364195c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58632/0 (socket says 172.21.15.33:58632) 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536259c700 1 -- 172.21.15.33:0/2091102107 learned_addr learned my addr 172.21.15.33:0/2091102107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f5359d9b700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5364101060 0x7f5364196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f5362d9d700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5364101a20 0x7f536419d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:29.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536259c700 1 -- 172.21.15.33:0/2091102107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5364101a20 msgr2=0x7f536419d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536259c700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5364101a20 0x7f536419d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536259c700 1 -- 172.21.15.33:0/2091102107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5364101060 msgr2=0x7f5364196160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536259c700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5364101060 0x7f5364196160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f536259c700 1 -- 172.21.15.33:0/2091102107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5354009520 con 0x7f53640fff00 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f5359d9b700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5364101060 0x7f5364196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.363+0000 7f5362d9d700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5364101a20 0x7f536419d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.364+0000 7f536259c700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53640fff00 0x7f5364195c20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f535400ec70 tx=0x7f535400b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.364+0000 7f535bfff700 1 -- 172.21.15.33:0/2091102107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354013070 con 0x7f53640fff00 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.364+0000 7f535bfff700 1 -- 172.21.15.33:0/2091102107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53540044b0 con 0x7f53640fff00 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.364+0000 7f536881d700 1 -- 172.21.15.33:0/2091102107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c009d00 con 0x7f53640fff00 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.364+0000 7f535bfff700 1 -- 172.21.15.33:0/2091102107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354003e30 con 0x7f53640fff00 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.364+0000 7f536881d700 1 -- 172.21.15.33:0/2091102107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53640695d0 con 0x7f53640fff00 2024-05-08T21:16:29.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.365+0000 7f536881d700 1 -- 172.21.15.33:0/2091102107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536404ec60 con 0x7f53640fff00 2024-05-08T21:16:29.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.366+0000 7f535bfff700 1 -- 172.21.15.33:0/2091102107 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5354003f90 con 0x7f53640fff00 2024-05-08T21:16:29.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.367+0000 7f535bfff700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53500a1120 0x7f53500a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:29.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.367+0000 7f5359d9b700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53500a1120 0x7f53500a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:29.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.367+0000 7f535bfff700 1 -- 172.21.15.33:0/2091102107 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f53540cdff0 con 0x7f53640fff00 2024-05-08T21:16:29.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.367+0000 7f5359d9b700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53500a1120 0x7f53500a35e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f534c009cb0 tx=0x7f534c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:29.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.371+0000 7f535bfff700 1 -- 172.21.15.33:0/2091102107 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5354097be0 con 0x7f53640fff00 2024-05-08T21:16:29.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:29 smithi033 ceph-mon[42528]: purged_snaps scrub starts 2024-05-08T21:16:29.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:29 smithi033 ceph-mon[42528]: purged_snaps scrub ok 2024-05-08T21:16:29.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:29 smithi033 ceph-mon[42528]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:16:29.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:29 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:29.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:29 smithi033 ceph-mon[42528]: osd.11 [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386] boot 2024-05-08T21:16:29.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:29 smithi033 ceph-mon[42528]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T21:16:29.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:29 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:29 smithi071 ceph-mon[47074]: purged_snaps scrub starts 2024-05-08T21:16:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:29 smithi071 ceph-mon[47074]: purged_snaps scrub ok 2024-05-08T21:16:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:29 smithi071 ceph-mon[47074]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:16:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:29 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:29 smithi071 ceph-mon[47074]: osd.11 [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386] boot 2024-05-08T21:16:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:29 smithi071 ceph-mon[47074]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T21:16:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:29 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.581+0000 7f536881d700 1 -- 172.21.15.33:0/2091102107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5364069fc0 con 0x7f53640fff00 2024-05-08T21:16:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.581+0000 7f535bfff700 1 -- 172.21.15.33:0/2091102107 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) v1 ==== 74+0+133 (secure 0 0 0) 0x7f53540250a0 con 0x7f53640fff00 2024-05-08T21:16:29.584 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:29.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.584+0000 7f536881d700 1 -- 172.21.15.33:0/2091102107 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53500a1120 msgr2=0x7f53500a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.584+0000 7f536881d700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53500a1120 0x7f53500a35e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f534c009cb0 tx=0x7f534c009260 comp rx=0 tx=0).stop 2024-05-08T21:16:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.584+0000 7f536881d700 1 -- 172.21.15.33:0/2091102107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53640fff00 msgr2=0x7f5364195c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.584+0000 7f536881d700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53640fff00 0x7f5364195c20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f535400ec70 tx=0x7f535400b5a0 comp rx=0 tx=0).stop 2024-05-08T21:16:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.584+0000 7f536881d700 1 -- 172.21.15.33:0/2091102107 shutdown_connections 2024-05-08T21:16:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.584+0000 7f536881d700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5364101a20 0x7f536419d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.584+0000 7f536881d700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53640fff00 0x7f5364195c20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.584+0000 7f536881d700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53500a1120 0x7f53500a35e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.584+0000 7f536881d700 1 --2- 172.21.15.33:0/2091102107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5364101060 0x7f5364196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.584+0000 7f536881d700 1 -- 172.21.15.33:0/2091102107 >> 172.21.15.33:0/2091102107 conn(0x7f53640fb4c0 msgr2=0x7f53641044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.584+0000 7f536881d700 1 -- 172.21.15.33:0/2091102107 shutdown_connections 2024-05-08T21:16:29.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:29.584+0000 7f536881d700 1 -- 172.21.15.33:0/2091102107 wait complete. 2024-05-08T21:16:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:29 smithi082 ceph-mon[47512]: purged_snaps scrub starts 2024-05-08T21:16:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:29 smithi082 ceph-mon[47512]: purged_snaps scrub ok 2024-05-08T21:16:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:29 smithi082 ceph-mon[47512]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-08T21:16:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:29 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:29 smithi082 ceph-mon[47512]: osd.11 [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386] boot 2024-05-08T21:16:29.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:29 smithi082 ceph-mon[47512]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T21:16:29.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:29 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T21:16:30.013 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":77,"num_osds":12,"num_up_osds":12,"osd_up_since":1715202988,"num_in_osds":12,"osd_in_since":1715202972,"num_remapped_pgs":0} 2024-05-08T21:16:30.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T21:16:30.270 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:16:30.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:30 smithi033 ceph-mon[42528]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T21:16:30.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:30 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2091102107' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:16:30.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:30 smithi071 ceph-mon[47074]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T21:16:30.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:30 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2091102107' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:16:30.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:30 smithi082 ceph-mon[47512]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T21:16:30.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:30 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2091102107' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T21:16:31.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.278+0000 7f4859ece700 1 -- 172.21.15.33:0/987172474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4854100890 msgr2=0x7f485410cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.278+0000 7f4859ece700 1 --2- 172.21.15.33:0/987172474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4854100890 0x7f485410cd40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b0a0 tx=0x7f484c00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:16:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.278+0000 7f4859ece700 1 -- 172.21.15.33:0/987172474 shutdown_connections 2024-05-08T21:16:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.278+0000 7f4859ece700 1 --2- 172.21.15.33:0/987172474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4854100890 0x7f485410cd40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.278+0000 7f4859ece700 1 --2- 172.21.15.33:0/987172474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4854105f00 0x7f48541062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.278+0000 7f4859ece700 1 --2- 172.21.15.33:0/987172474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48540ffed0 0x7f4854100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.278+0000 7f4859ece700 1 -- 172.21.15.33:0/987172474 >> 172.21.15.33:0/987172474 conn(0x7f4854075f80 msgr2=0x7f4854076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.279+0000 7f4859ece700 1 -- 172.21.15.33:0/987172474 shutdown_connections 2024-05-08T21:16:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.279+0000 7f4859ece700 1 -- 172.21.15.33:0/987172474 wait complete. 2024-05-08T21:16:31.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.279+0000 7f4859ece700 1 Processor -- start 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.279+0000 7f4859ece700 1 -- start start 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4859ece700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48540ffed0 0x7f485410d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4859ece700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4854100890 0x7f485410dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4859ece700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4854105f00 0x7f485410e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4859ece700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48541a5760 con 0x7f4854105f00 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4859ece700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48541a58d0 con 0x7f48540ffed0 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4859ece700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48541a5a90 con 0x7f4854100890 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4853fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4854105f00 0x7f485410e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4853fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4854105f00 0x7f485410e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36330/0 (socket says 172.21.15.33:36330) 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4853fff700 1 -- 172.21.15.33:0/1213875392 learned_addr learned my addr 172.21.15.33:0/1213875392 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4852ffd700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4854100890 0x7f485410dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f48537fe700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48540ffed0 0x7f485410d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4853fff700 1 -- 172.21.15.33:0/1213875392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4854100890 msgr2=0x7f485410dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:31.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4853fff700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4854100890 0x7f485410dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4853fff700 1 -- 172.21.15.33:0/1213875392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48540ffed0 msgr2=0x7f485410d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4853fff700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48540ffed0 0x7f485410d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4853fff700 1 -- 172.21.15.33:0/1213875392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c009d00 con 0x7f4854105f00 2024-05-08T21:16:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.280+0000 7f4853fff700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4854105f00 0x7f485410e190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b070 tx=0x7f484c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.281+0000 7f4850ff9700 1 -- 172.21.15.33:0/1213875392 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c00e040 con 0x7f4854105f00 2024-05-08T21:16:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.281+0000 7f4850ff9700 1 -- 172.21.15.33:0/1213875392 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f484c009460 con 0x7f4854105f00 2024-05-08T21:16:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.281+0000 7f4859ece700 1 -- 172.21.15.33:0/1213875392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48541a5d10 con 0x7f4854105f00 2024-05-08T21:16:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.281+0000 7f4850ff9700 1 -- 172.21.15.33:0/1213875392 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c01b450 con 0x7f4854105f00 2024-05-08T21:16:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.281+0000 7f4859ece700 1 -- 172.21.15.33:0/1213875392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48541a6180 con 0x7f4854105f00 2024-05-08T21:16:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.282+0000 7f4859ece700 1 -- 172.21.15.33:0/1213875392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48541083a0 con 0x7f4854105f00 2024-05-08T21:16:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.283+0000 7f4850ff9700 1 -- 172.21.15.33:0/1213875392 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f484c01d480 con 0x7f4854105f00 2024-05-08T21:16:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.283+0000 7f4850ff9700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f483c0a1120 0x7f483c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.284+0000 7f48537fe700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f483c0a1120 0x7f483c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.284+0000 7f4850ff9700 1 -- 172.21.15.33:0/1213875392 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f484c0cdf50 con 0x7f4854105f00 2024-05-08T21:16:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.284+0000 7f48537fe700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f483c0a1120 0x7f483c0a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4844005950 tx=0x7f484400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.288+0000 7f4850ff9700 1 -- 172.21.15.33:0/1213875392 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f484c097c10 con 0x7f4854105f00 2024-05-08T21:16:31.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:31 smithi033 ceph-mon[42528]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:31.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.500+0000 7f4859ece700 1 -- 172.21.15.33:0/1213875392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48541015b0 con 0x7f4854105f00 2024-05-08T21:16:31.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.501+0000 7f4850ff9700 1 -- 172.21.15.33:0/1213875392 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18364 (secure 0 0 0) 0x7f484c025e90 con 0x7f4854105f00 2024-05-08T21:16:31.504 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:31.504 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":77,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:16:29.210489+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:16:31.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.503+0000 7f4859ece700 1 -- 172.21.15.33:0/1213875392 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f483c0a1120 msgr2=0x7f483c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:31.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.503+0000 7f4859ece700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f483c0a1120 0x7f483c0a35e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4844005950 tx=0x7f484400b410 comp rx=0 tx=0).stop 2024-05-08T21:16:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.503+0000 7f4859ece700 1 -- 172.21.15.33:0/1213875392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4854105f00 msgr2=0x7f485410e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.503+0000 7f4859ece700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4854105f00 0x7f485410e190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b070 tx=0x7f484c00cd70 comp rx=0 tx=0).stop 2024-05-08T21:16:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.503+0000 7f4859ece700 1 -- 172.21.15.33:0/1213875392 shutdown_connections 2024-05-08T21:16:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.503+0000 7f4859ece700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4854105f00 0x7f485410e190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.503+0000 7f4859ece700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f483c0a1120 0x7f483c0a35e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.503+0000 7f4859ece700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48540ffed0 0x7f485410d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.503+0000 7f4859ece700 1 --2- 172.21.15.33:0/1213875392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4854100890 0x7f485410dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.503+0000 7f4859ece700 1 -- 172.21.15.33:0/1213875392 >> 172.21.15.33:0/1213875392 conn(0x7f4854075f80 msgr2=0x7f48540fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.503+0000 7f4859ece700 1 -- 172.21.15.33:0/1213875392 shutdown_connections 2024-05-08T21:16:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:31.503+0000 7f4859ece700 1 -- 172.21.15.33:0/1213875392 wait complete. 2024-05-08T21:16:31.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:31 smithi071 ceph-mon[47074]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:31.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:31 smithi082 ceph-mon[47512]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:31.884 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T21:16:31.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-08T21:16:32.137 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:16:32.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:32 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1213875392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:16:32.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:32 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1213875392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:16:32.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:32 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1213875392' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:16:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.107+0000 7f016881d700 1 -- 172.21.15.33:0/3522560246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0164101990 msgr2=0x7f016410de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.107+0000 7f016881d700 1 --2- 172.21.15.33:0/3522560246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0164101990 0x7f016410de70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f015c00b0a0 tx=0x7f015c00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:16:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.107+0000 7f016881d700 1 -- 172.21.15.33:0/3522560246 shutdown_connections 2024-05-08T21:16:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.107+0000 7f016881d700 1 --2- 172.21.15.33:0/3522560246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0164101990 0x7f016410de70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.107+0000 7f016881d700 1 --2- 172.21.15.33:0/3522560246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164107030 0x7f0164107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.107+0000 7f016881d700 1 --2- 172.21.15.33:0/3522560246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0164100ff0 0x7f0164101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.107+0000 7f016881d700 1 -- 172.21.15.33:0/3522560246 >> 172.21.15.33:0/3522560246 conn(0x7f0164078190 msgr2=0x7f01640785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.107+0000 7f016881d700 1 -- 172.21.15.33:0/3522560246 shutdown_connections 2024-05-08T21:16:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.107+0000 7f016881d700 1 -- 172.21.15.33:0/3522560246 wait complete. 2024-05-08T21:16:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.108+0000 7f016881d700 1 Processor -- start 2024-05-08T21:16:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.108+0000 7f016881d700 1 -- start start 2024-05-08T21:16:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.108+0000 7f016881d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0164100ff0 0x7f0164197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.108+0000 7f016881d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164101990 0x7f01641979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.108+0000 7f016881d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0164107030 0x7f016419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.108+0000 7f016881d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016419c370 con 0x7f0164100ff0 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.108+0000 7f016881d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016419c4e0 con 0x7f0164107030 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.108+0000 7f016881d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016419c6a0 con 0x7f0164101990 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.108+0000 7f016259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0164100ff0 0x7f0164197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.108+0000 7f016259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0164100ff0 0x7f0164197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36356/0 (socket says 172.21.15.33:36356) 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f016259c700 1 -- 172.21.15.33:0/159326720 learned_addr learned my addr 172.21.15.33:0/159326720 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f0161d9b700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164101990 0x7f01641979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f0162d9d700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0164107030 0x7f016419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f016259c700 1 -- 172.21.15.33:0/159326720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164101990 msgr2=0x7f01641979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f016259c700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164101990 0x7f01641979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f016259c700 1 -- 172.21.15.33:0/159326720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0164107030 msgr2=0x7f016419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f016259c700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0164107030 0x7f016419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f016259c700 1 -- 172.21.15.33:0/159326720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c009d00 con 0x7f0164100ff0 2024-05-08T21:16:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f016259c700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0164100ff0 0x7f0164197470 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f015400d980 tx=0x7f015400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f01537fe700 1 -- 172.21.15.33:0/159326720 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015400bb60 con 0x7f0164100ff0 2024-05-08T21:16:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f01537fe700 1 -- 172.21.15.33:0/159326720 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0154007640 con 0x7f0164100ff0 2024-05-08T21:16:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f016881d700 1 -- 172.21.15.33:0/159326720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016419c980 con 0x7f0164100ff0 2024-05-08T21:16:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f01537fe700 1 -- 172.21.15.33:0/159326720 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154004d60 con 0x7f0164100ff0 2024-05-08T21:16:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.109+0000 7f016881d700 1 -- 172.21.15.33:0/159326720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01641a41e0 con 0x7f0164100ff0 2024-05-08T21:16:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.110+0000 7f016881d700 1 -- 172.21.15.33:0/159326720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0164109550 con 0x7f0164100ff0 2024-05-08T21:16:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.111+0000 7f01537fe700 1 -- 172.21.15.33:0/159326720 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0154004020 con 0x7f0164100ff0 2024-05-08T21:16:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.112+0000 7f01537fe700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f014c0a11a0 0x7f014c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.112+0000 7f0161d9b700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f014c0a11a0 0x7f014c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.113+0000 7f01537fe700 1 -- 172.21.15.33:0/159326720 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f01540cc2b0 con 0x7f0164100ff0 2024-05-08T21:16:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.113+0000 7f0161d9b700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f014c0a11a0 0x7f014c0a3660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0158005950 tx=0x7f01580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.118+0000 7f01537fe700 1 -- 172.21.15.33:0/159326720 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0154095e40 con 0x7f0164100ff0 2024-05-08T21:16:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.328+0000 7f016881d700 1 -- 172.21.15.33:0/159326720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0164198780 con 0x7f0164100ff0 2024-05-08T21:16:33.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.329+0000 7f01537fe700 1 -- 172.21.15.33:0/159326720 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f015400de60 con 0x7f0164100ff0 2024-05-08T21:16:33.332 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:16:33.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.331+0000 7f016881d700 1 -- 172.21.15.33:0/159326720 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f014c0a11a0 msgr2=0x7f014c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:33.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.331+0000 7f016881d700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f014c0a11a0 0x7f014c0a3660 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0158005950 tx=0x7f01580058e0 comp rx=0 tx=0).stop 2024-05-08T21:16:33.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.331+0000 7f016881d700 1 -- 172.21.15.33:0/159326720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0164100ff0 msgr2=0x7f0164197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:33.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.331+0000 7f016881d700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0164100ff0 0x7f0164197470 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f015400d980 tx=0x7f015400dd40 comp rx=0 tx=0).stop 2024-05-08T21:16:33.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.331+0000 7f016881d700 1 -- 172.21.15.33:0/159326720 shutdown_connections 2024-05-08T21:16:33.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.331+0000 7f016881d700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f014c0a11a0 0x7f014c0a3660 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:33.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.331+0000 7f016881d700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0164107030 0x7f016419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:33.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.331+0000 7f016881d700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0164100ff0 0x7f0164197470 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:33.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.331+0000 7f016881d700 1 --2- 172.21.15.33:0/159326720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0164101990 0x7f01641979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:33.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.331+0000 7f016881d700 1 -- 172.21.15.33:0/159326720 >> 172.21.15.33:0/159326720 conn(0x7f0164078190 msgr2=0x7f01640fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:33.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.331+0000 7f016881d700 1 -- 172.21.15.33:0/159326720 shutdown_connections 2024-05-08T21:16:33.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:33.331+0000 7f016881d700 1 -- 172.21.15.33:0/159326720 wait complete. 2024-05-08T21:16:33.514 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:33 smithi033 ceph-mon[42528]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:33.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:33 smithi071 ceph-mon[47074]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:33.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:33 smithi082 ceph-mon[47512]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:33.736 INFO:tasks.cephadm:Adding mds.a on smithi033 2024-05-08T21:16:33.737 INFO:tasks.cephadm:Adding mds.d on smithi033 2024-05-08T21:16:33.737 INFO:tasks.cephadm:Adding mds.g on smithi033 2024-05-08T21:16:33.737 INFO:tasks.cephadm:Adding mds.j on smithi033 2024-05-08T21:16:33.737 INFO:tasks.cephadm:Adding mds.b on smithi071 2024-05-08T21:16:33.737 INFO:tasks.cephadm:Adding mds.e on smithi071 2024-05-08T21:16:33.737 INFO:tasks.cephadm:Adding mds.h on smithi071 2024-05-08T21:16:33.737 INFO:tasks.cephadm:Adding mds.k on smithi071 2024-05-08T21:16:33.737 INFO:tasks.cephadm:Adding mds.c on smithi082 2024-05-08T21:16:33.737 INFO:tasks.cephadm:Adding mds.f on smithi082 2024-05-08T21:16:33.737 INFO:tasks.cephadm:Adding mds.i on smithi082 2024-05-08T21:16:33.737 INFO:tasks.cephadm:Adding mds.l on smithi082 2024-05-08T21:16:33.737 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l' 2024-05-08T21:16:33.973 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:16:34.491 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:34 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/159326720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:16:34.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:34 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/159326720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:16:34.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:34 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/159326720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:16:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.009+0000 7f2fc18fd700 1 -- 172.21.15.82:0/4029840229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fbc102b80 msgr2=0x7f2fbc109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.009+0000 7f2fc18fd700 1 --2- 172.21.15.82:0/4029840229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fbc102b80 0x7f2fbc109430 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400b0a0 tx=0x7f2fb400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.009+0000 7f2fc18fd700 1 -- 172.21.15.82:0/4029840229 shutdown_connections 2024-05-08T21:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.009+0000 7f2fc18fd700 1 --2- 172.21.15.82:0/4029840229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fbc102b80 0x7f2fbc109430 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.009+0000 7f2fc18fd700 1 --2- 172.21.15.82:0/4029840229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fbc100fe0 0x7f2fbc101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.009+0000 7f2fc18fd700 1 --2- 172.21.15.82:0/4029840229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fbc1021e0 0x7f2fbc102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.009+0000 7f2fc18fd700 1 -- 172.21.15.82:0/4029840229 >> 172.21.15.82:0/4029840229 conn(0x7f2fbc0fc580 msgr2=0x7f2fbc0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.009+0000 7f2fc18fd700 1 -- 172.21.15.82:0/4029840229 shutdown_connections 2024-05-08T21:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.009+0000 7f2fc18fd700 1 -- 172.21.15.82:0/4029840229 wait complete. 2024-05-08T21:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.010+0000 7f2fc18fd700 1 Processor -- start 2024-05-08T21:16:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.010+0000 7f2fc18fd700 1 -- start start 2024-05-08T21:16:35.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.010+0000 7f2fc18fd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fbc100fe0 0x7f2fbc196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:35.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.010+0000 7f2fc18fd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fbc1021e0 0x7f2fbc1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:35.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.010+0000 7f2fc18fd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fbc102b80 0x7f2fbc19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:35.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.010+0000 7f2fc18fd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc19e9a0 con 0x7f2fbc1021e0 2024-05-08T21:16:35.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.010+0000 7f2fc18fd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc19eae0 con 0x7f2fbc102b80 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.010+0000 7f2fc18fd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc19ec70 con 0x7f2fbc100fe0 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.010+0000 7f2fbaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fbc100fe0 0x7f2fbc196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.010+0000 7f2fbaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fbc100fe0 0x7f2fbc196e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59780/0 (socket says 172.21.15.82:59780) 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.010+0000 7f2fbaffd700 1 -- 172.21.15.82:0/202764441 learned_addr learned my addr 172.21.15.82:0/202764441 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fba7fc700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fbc1021e0 0x7f2fbc1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fbb7fe700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fbc102b80 0x7f2fbc19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fbaffd700 1 -- 172.21.15.82:0/202764441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fbc102b80 msgr2=0x7f2fbc19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fbaffd700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fbc102b80 0x7f2fbc19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fbaffd700 1 -- 172.21.15.82:0/202764441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fbc1021e0 msgr2=0x7f2fbc1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fbaffd700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fbc1021e0 0x7f2fbc1973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fbaffd700 1 -- 172.21.15.82:0/202764441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb4009d00 con 0x7f2fbc100fe0 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fbaffd700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fbc100fe0 0x7f2fbc196e60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00c9a0 tx=0x7f2fac00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fc08fb700 1 -- 172.21.15.82:0/202764441 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac004220 con 0x7f2fbc100fe0 2024-05-08T21:16:35.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fc08fb700 1 -- 172.21.15.82:0/202764441 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fac007cf0 con 0x7f2fbc100fe0 2024-05-08T21:16:35.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fc18fd700 1 -- 172.21.15.82:0/202764441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc19ef50 con 0x7f2fbc100fe0 2024-05-08T21:16:35.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fc08fb700 1 -- 172.21.15.82:0/202764441 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac003b30 con 0x7f2fbc100fe0 2024-05-08T21:16:35.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.011+0000 7f2fc18fd700 1 -- 172.21.15.82:0/202764441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc19f360 con 0x7f2fbc100fe0 2024-05-08T21:16:35.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.012+0000 7f2fc18fd700 1 -- 172.21.15.82:0/202764441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc108420 con 0x7f2fbc100fe0 2024-05-08T21:16:35.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.013+0000 7f2fc08fb700 1 -- 172.21.15.82:0/202764441 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2fac003cd0 con 0x7f2fbc100fe0 2024-05-08T21:16:35.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.014+0000 7f2fc08fb700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2fa80a1120 0x7f2fa80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:35.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.014+0000 7f2fc08fb700 1 -- 172.21.15.82:0/202764441 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2fac0cc050 con 0x7f2fbc100fe0 2024-05-08T21:16:35.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.014+0000 7f2fba7fc700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2fa80a1120 0x7f2fa80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:35.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.014+0000 7f2fba7fc700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2fa80a1120 0x7f2fa80a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400ac50 tx=0x7f2fa400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:35.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.018+0000 7f2fc08fb700 1 -- 172.21.15.82:0/202764441 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2fac095c80 con 0x7f2fbc100fe0 2024-05-08T21:16:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.221+0000 7f2fc18fd700 1 -- 172.21.15.82:0/202764441 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fbc02cf60 con 0x7f2fa80a1120 2024-05-08T21:16:35.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.230+0000 7f2fc08fb700 1 -- 172.21.15.82:0/202764441 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f2fbc02cf60 con 0x7f2fa80a1120 2024-05-08T21:16:35.233 INFO:teuthology.orchestra.run.smithi082.stdout:Scheduled mds.all update... 2024-05-08T21:16:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.232+0000 7f2fc18fd700 1 -- 172.21.15.82:0/202764441 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2fa80a1120 msgr2=0x7f2fa80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.232+0000 7f2fc18fd700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2fa80a1120 0x7f2fa80a35e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400ac50 tx=0x7f2fa400a380 comp rx=0 tx=0).stop 2024-05-08T21:16:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.232+0000 7f2fc18fd700 1 -- 172.21.15.82:0/202764441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fbc100fe0 msgr2=0x7f2fbc196e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.232+0000 7f2fc18fd700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fbc100fe0 0x7f2fbc196e60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00c9a0 tx=0x7f2fac00ccb0 comp rx=0 tx=0).stop 2024-05-08T21:16:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.232+0000 7f2fc18fd700 1 -- 172.21.15.82:0/202764441 shutdown_connections 2024-05-08T21:16:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.232+0000 7f2fc18fd700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2fa80a1120 0x7f2fa80a35e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.232+0000 7f2fc18fd700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fbc102b80 0x7f2fbc19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.232+0000 7f2fc18fd700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fbc100fe0 0x7f2fbc196e60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.232+0000 7f2fc18fd700 1 --2- 172.21.15.82:0/202764441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fbc1021e0 0x7f2fbc1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.232+0000 7f2fc18fd700 1 -- 172.21.15.82:0/202764441 >> 172.21.15.82:0/202764441 conn(0x7f2fbc0fc580 msgr2=0x7f2fbc105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:35.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.232+0000 7f2fc18fd700 1 -- 172.21.15.82:0/202764441 shutdown_connections 2024-05-08T21:16:35.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-05-08T21:16:35.232+0000 7f2fc18fd700 1 -- 172.21.15.82:0/202764441 wait complete. 2024-05-08T21:16:35.511 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:35 smithi082 ceph-mon[47512]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:35.511 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:35 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:35.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:35 smithi071 ceph-mon[47074]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:35.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:35 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:35.603 DEBUG:teuthology.orchestra.run.smithi033:mds.a> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.a.service 2024-05-08T21:16:35.606 DEBUG:teuthology.orchestra.run.smithi033:mds.d> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.d.service 2024-05-08T21:16:35.633 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:35 smithi033 ceph-mon[42528]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:35.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:35 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:35.635 DEBUG:teuthology.orchestra.run.smithi033:mds.g> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.g.service 2024-05-08T21:16:35.642 INFO:journalctl@ceph.mds.a.smithi033.stdout:-- Logs begin at Wed 2024-05-08 20:53:21 UTC. -- 2024-05-08T21:16:35.644 DEBUG:teuthology.orchestra.run.smithi033:mds.j> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.j.service 2024-05-08T21:16:35.667 INFO:journalctl@ceph.mds.d.smithi033.stdout:-- Logs begin at Wed 2024-05-08 20:53:21 UTC. -- 2024-05-08T21:16:35.668 DEBUG:teuthology.orchestra.run.smithi071:mds.b> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.b.service 2024-05-08T21:16:35.670 DEBUG:teuthology.orchestra.run.smithi071:mds.e> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.e.service 2024-05-08T21:16:35.673 INFO:journalctl@ceph.mds.g.smithi033.stdout:-- Logs begin at Wed 2024-05-08 20:53:21 UTC. -- 2024-05-08T21:16:35.700 INFO:journalctl@ceph.mds.j.smithi033.stdout:-- Logs begin at Wed 2024-05-08 20:53:21 UTC. -- 2024-05-08T21:16:35.709 INFO:journalctl@ceph.mds.b.smithi071.stdout:-- Logs begin at Wed 2024-05-08 20:53:45 UTC. -- 2024-05-08T21:16:35.710 DEBUG:teuthology.orchestra.run.smithi071:mds.h> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.h.service 2024-05-08T21:16:35.740 INFO:journalctl@ceph.mds.e.smithi071.stdout:-- Logs begin at Wed 2024-05-08 20:53:45 UTC. -- 2024-05-08T21:16:35.741 DEBUG:teuthology.orchestra.run.smithi071:mds.k> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.k.service 2024-05-08T21:16:35.769 INFO:journalctl@ceph.mds.h.smithi071.stdout:-- Logs begin at Wed 2024-05-08 20:53:45 UTC. -- 2024-05-08T21:16:35.770 DEBUG:teuthology.orchestra.run.smithi082:mds.c> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.c.service 2024-05-08T21:16:35.772 DEBUG:teuthology.orchestra.run.smithi082:mds.f> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.f.service 2024-05-08T21:16:35.796 INFO:journalctl@ceph.mds.k.smithi071.stdout:-- Logs begin at Wed 2024-05-08 20:53:45 UTC. -- 2024-05-08T21:16:35.802 INFO:journalctl@ceph.mds.c.smithi082.stdout:-- Logs begin at Wed 2024-05-08 20:53:53 UTC. -- 2024-05-08T21:16:35.803 DEBUG:teuthology.orchestra.run.smithi082:mds.i> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.i.service 2024-05-08T21:16:35.830 INFO:journalctl@ceph.mds.f.smithi082.stdout:-- Logs begin at Wed 2024-05-08 20:53:53 UTC. -- 2024-05-08T21:16:35.831 DEBUG:teuthology.orchestra.run.smithi082:mds.l> sudo journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.l.service 2024-05-08T21:16:35.857 INFO:journalctl@ceph.mds.i.smithi082.stdout:-- Logs begin at Wed 2024-05-08 20:53:53 UTC. -- 2024-05-08T21:16:35.858 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-08T21:16:35.858 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:16:35.884 INFO:journalctl@ceph.mds.l.smithi082.stdout:-- Logs begin at Wed 2024-05-08 20:53:53 UTC. -- 2024-05-08T21:16:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.012+0000 7f8459c5b700 1 -- 172.21.15.33:0/4200057961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454100e80 msgr2=0x7f84541012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.012+0000 7f8459c5b700 1 --2- 172.21.15.33:0/4200057961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454100e80 0x7f84541012a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8444009990 tx=0x7f8444009ca0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.012+0000 7f8459c5b700 1 -- 172.21.15.33:0/4200057961 shutdown_connections 2024-05-08T21:16:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.012+0000 7f8459c5b700 1 --2- 172.21.15.33:0/4200057961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8454102990 0x7f84541092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.012+0000 7f8459c5b700 1 --2- 172.21.15.33:0/4200057961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454100e80 0x7f84541012a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.012+0000 7f8459c5b700 1 --2- 172.21.15.33:0/4200057961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8454101fd0 0x7f8454102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.012+0000 7f8459c5b700 1 -- 172.21.15.33:0/4200057961 >> 172.21.15.33:0/4200057961 conn(0x7f84540fc410 msgr2=0x7f84540fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.012+0000 7f8459c5b700 1 -- 172.21.15.33:0/4200057961 shutdown_connections 2024-05-08T21:16:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.012+0000 7f8459c5b700 1 -- 172.21.15.33:0/4200057961 wait complete. 2024-05-08T21:16:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.013+0000 7f8459c5b700 1 Processor -- start 2024-05-08T21:16:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.013+0000 7f8459c5b700 1 -- start start 2024-05-08T21:16:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.013+0000 7f8459c5b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454100e80 0x7f8454196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.013+0000 7f8459c5b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8454101fd0 0x7f84541972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.013+0000 7f8459c5b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8454102990 0x7f845419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.013+0000 7f8459c5b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845419e850 con 0x7f8454100e80 2024-05-08T21:16:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.013+0000 7f8459c5b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845419e9c0 con 0x7f8454101fd0 2024-05-08T21:16:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.013+0000 7f8459c5b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845419eb80 con 0x7f8454102990 2024-05-08T21:16:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f84537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454100e80 0x7f8454196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f84537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454100e80 0x7f8454196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36366/0 (socket says 172.21.15.33:36366) 2024-05-08T21:16:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f84537fe700 1 -- 172.21.15.33:0/3419660131 learned_addr learned my addr 172.21.15.33:0/3419660131 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f8452ffd700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8454101fd0 0x7f84541972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f8453fff700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8454102990 0x7f845419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f84537fe700 1 -- 172.21.15.33:0/3419660131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8454102990 msgr2=0x7f845419e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f84537fe700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8454102990 0x7f845419e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f84537fe700 1 -- 172.21.15.33:0/3419660131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8454101fd0 msgr2=0x7f84541972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f84537fe700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8454101fd0 0x7f84541972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f84537fe700 1 -- 172.21.15.33:0/3419660131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84440095f0 con 0x7f8454100e80 2024-05-08T21:16:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f8453fff700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8454102990 0x7f845419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:16:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f8452ffd700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8454101fd0 0x7f84541972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:16:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f84537fe700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454100e80 0x7f8454196db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f84440059d0 tx=0x7f8444004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f8450ff9700 1 -- 172.21.15.33:0/3419660131 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844401d070 con 0x7f8454100e80 2024-05-08T21:16:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f8459c5b700 1 -- 172.21.15.33:0/3419660131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845419ee00 con 0x7f8454100e80 2024-05-08T21:16:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f8450ff9700 1 -- 172.21.15.33:0/3419660131 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f844400bdd0 con 0x7f8454100e80 2024-05-08T21:16:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f8450ff9700 1 -- 172.21.15.33:0/3419660131 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444022da0 con 0x7f8454100e80 2024-05-08T21:16:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.014+0000 7f8459c5b700 1 -- 172.21.15.33:0/3419660131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845419f1d0 con 0x7f8454100e80 2024-05-08T21:16:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.015+0000 7f8459c5b700 1 -- 172.21.15.33:0/3419660131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84541083e0 con 0x7f8454100e80 2024-05-08T21:16:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.016+0000 7f8450ff9700 1 -- 172.21.15.33:0/3419660131 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8444022570 con 0x7f8454100e80 2024-05-08T21:16:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.017+0000 7f8450ff9700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f84400a1120 0x7f84400a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.017+0000 7f8450ff9700 1 -- 172.21.15.33:0/3419660131 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f84440cdb40 con 0x7f8454100e80 2024-05-08T21:16:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.017+0000 7f8452ffd700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f84400a1120 0x7f84400a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.018+0000 7f8452ffd700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f84400a1120 0x7f84400a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f843c011490 tx=0x7f843c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.022+0000 7f8450ff9700 1 -- 172.21.15.33:0/3419660131 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8444097920 con 0x7f8454100e80 2024-05-08T21:16:36.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.239+0000 7f8459c5b700 1 -- 172.21.15.33:0/3419660131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f845410c530 con 0x7f8454100e80 2024-05-08T21:16:36.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.240+0000 7f8450ff9700 1 -- 172.21.15.33:0/3419660131 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18364 (secure 0 0 0) 0x7f8444027090 con 0x7f8454100e80 2024-05-08T21:16:36.243 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:36.243 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":77,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:16:29.210489+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:16:36.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.243+0000 7f8459c5b700 1 -- 172.21.15.33:0/3419660131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f84400a1120 msgr2=0x7f84400a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.243+0000 7f8459c5b700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f84400a1120 0x7f84400a35e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f843c011490 tx=0x7f843c009260 comp rx=0 tx=0).stop 2024-05-08T21:16:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.243+0000 7f8459c5b700 1 -- 172.21.15.33:0/3419660131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454100e80 msgr2=0x7f8454196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.243+0000 7f8459c5b700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454100e80 0x7f8454196db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f84440059d0 tx=0x7f8444004730 comp rx=0 tx=0).stop 2024-05-08T21:16:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.243+0000 7f8459c5b700 1 -- 172.21.15.33:0/3419660131 shutdown_connections 2024-05-08T21:16:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.243+0000 7f8459c5b700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8454102990 0x7f845419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.243+0000 7f8459c5b700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8454100e80 0x7f8454196db0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.243+0000 7f8459c5b700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f84400a1120 0x7f84400a35e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.243+0000 7f8459c5b700 1 --2- 172.21.15.33:0/3419660131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8454101fd0 0x7f84541972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.243+0000 7f8459c5b700 1 -- 172.21.15.33:0/3419660131 >> 172.21.15.33:0/3419660131 conn(0x7f84540fc410 msgr2=0x7f8454105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.243+0000 7f8459c5b700 1 -- 172.21.15.33:0/3419660131 shutdown_connections 2024-05-08T21:16:36.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.243+0000 7f8459c5b700 1 -- 172.21.15.33:0/3419660131 wait complete. 2024-05-08T21:16:36.260 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T21:16:36.261 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:16:36.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:36 smithi082 ceph-mon[47512]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:16:36.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:36 smithi082 ceph-mon[47512]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l;count:12 2024-05-08T21:16:36.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:36 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:36.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:36 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:36.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:36 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:36.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:36 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:36.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:36 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:36.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:36 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:36.354 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:36 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:36.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.398+0000 7f9e4e059700 1 -- 172.21.15.33:0/2676129262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e48100f50 msgr2=0x7f9e481013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.398+0000 7f9e4e059700 1 --2- 172.21.15.33:0/2676129262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e48100f50 0x7f9e481013b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e380099c0 tx=0x7f9e38009cd0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.398+0000 7f9e4e059700 1 -- 172.21.15.33:0/2676129262 shutdown_connections 2024-05-08T21:16:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.398+0000 7f9e4e059700 1 --2- 172.21.15.33:0/2676129262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e481018f0 0x7f9e481081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.398+0000 7f9e4e059700 1 --2- 172.21.15.33:0/2676129262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e480ffd60 0x7f9e48100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.398+0000 7f9e4e059700 1 --2- 172.21.15.33:0/2676129262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e48100f50 0x7f9e481013b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.398+0000 7f9e4e059700 1 -- 172.21.15.33:0/2676129262 >> 172.21.15.33:0/2676129262 conn(0x7f9e480fb290 msgr2=0x7f9e480fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.398+0000 7f9e4e059700 1 -- 172.21.15.33:0/2676129262 shutdown_connections 2024-05-08T21:16:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.398+0000 7f9e4e059700 1 -- 172.21.15.33:0/2676129262 wait complete. 2024-05-08T21:16:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.399+0000 7f9e4e059700 1 Processor -- start 2024-05-08T21:16:36.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.399+0000 7f9e4e059700 1 -- start start 2024-05-08T21:16:36.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.399+0000 7f9e4e059700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480ffd60 0x7f9e48196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:36.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.399+0000 7f9e4e059700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e48100f50 0x7f9e48197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:36.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.399+0000 7f9e4e059700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e481018f0 0x7f9e4819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.399+0000 7f9e4e059700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4819e870 con 0x7f9e480ffd60 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.399+0000 7f9e4e059700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4819e9e0 con 0x7f9e48100f50 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.399+0000 7f9e4e059700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4819eba0 con 0x7f9e481018f0 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.399+0000 7f9e477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480ffd60 0x7f9e48196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.399+0000 7f9e477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480ffd60 0x7f9e48196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36390/0 (socket says 172.21.15.33:36390) 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.399+0000 7f9e477fe700 1 -- 172.21.15.33:0/1964918898 learned_addr learned my addr 172.21.15.33:0/1964918898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e46ffd700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e48100f50 0x7f9e48197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e47fff700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e481018f0 0x7f9e4819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e477fe700 1 -- 172.21.15.33:0/1964918898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e481018f0 msgr2=0x7f9e4819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e477fe700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e481018f0 0x7f9e4819e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e477fe700 1 -- 172.21.15.33:0/1964918898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e48100f50 msgr2=0x7f9e48197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e477fe700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e48100f50 0x7f9e48197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e477fe700 1 -- 172.21.15.33:0/1964918898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e380095f0 con 0x7f9e480ffd60 2024-05-08T21:16:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e477fe700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480ffd60 0x7f9e48196dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000d980 tx=0x7f9e3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:36.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e44ff9700 1 -- 172.21.15.33:0/1964918898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30004020 con 0x7f9e480ffd60 2024-05-08T21:16:36.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e4e059700 1 -- 172.21.15.33:0/1964918898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4819ee80 con 0x7f9e480ffd60 2024-05-08T21:16:36.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e44ff9700 1 -- 172.21.15.33:0/1964918898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e3000f690 con 0x7f9e480ffd60 2024-05-08T21:16:36.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e44ff9700 1 -- 172.21.15.33:0/1964918898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e300039e0 con 0x7f9e480ffd60 2024-05-08T21:16:36.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.400+0000 7f9e4e059700 1 -- 172.21.15.33:0/1964918898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4819f2b0 con 0x7f9e480ffd60 2024-05-08T21:16:36.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.401+0000 7f9e4e059700 1 -- 172.21.15.33:0/1964918898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e48067d10 con 0x7f9e480ffd60 2024-05-08T21:16:36.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.402+0000 7f9e44ff9700 1 -- 172.21.15.33:0/1964918898 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9e3000f800 con 0x7f9e480ffd60 2024-05-08T21:16:36.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.403+0000 7f9e44ff9700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9e340a1120 0x7f9e340a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:36.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.403+0000 7f9e44ff9700 1 -- 172.21.15.33:0/1964918898 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9e300cbfc0 con 0x7f9e480ffd60 2024-05-08T21:16:36.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.403+0000 7f9e46ffd700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9e340a1120 0x7f9e340a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:36.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.404+0000 7f9e46ffd700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9e340a1120 0x7f9e340a35e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800b580 tx=0x7f9e38005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:36.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.408+0000 7f9e44ff9700 1 -- 172.21.15.33:0/1964918898 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9e300ffd60 con 0x7f9e480ffd60 2024-05-08T21:16:36.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:36 smithi071 ceph-mon[47074]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:16:36.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:36 smithi071 ceph-mon[47074]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l;count:12 2024-05-08T21:16:36.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:36 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:36.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:36 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:36.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:36 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:36.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:36 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:36.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:36 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:36.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:36 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:36.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:36 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:36.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.627+0000 7f9e4e059700 1 -- 172.21.15.33:0/1964918898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e4810b700 con 0x7f9e480ffd60 2024-05-08T21:16:36.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.628+0000 7f9e44ff9700 1 -- 172.21.15.33:0/1964918898 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e30095bf0 con 0x7f9e480ffd60 2024-05-08T21:16:36.631 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:16:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.630+0000 7f9e4e059700 1 -- 172.21.15.33:0/1964918898 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9e340a1120 msgr2=0x7f9e340a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.630+0000 7f9e4e059700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9e340a1120 0x7f9e340a35e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800b580 tx=0x7f9e38005e80 comp rx=0 tx=0).stop 2024-05-08T21:16:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.630+0000 7f9e4e059700 1 -- 172.21.15.33:0/1964918898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480ffd60 msgr2=0x7f9e48196dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.630+0000 7f9e4e059700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480ffd60 0x7f9e48196dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000d980 tx=0x7f9e3000dc90 comp rx=0 tx=0).stop 2024-05-08T21:16:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.630+0000 7f9e4e059700 1 -- 172.21.15.33:0/1964918898 shutdown_connections 2024-05-08T21:16:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.630+0000 7f9e4e059700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e481018f0 0x7f9e4819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.630+0000 7f9e4e059700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480ffd60 0x7f9e48196dd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.630+0000 7f9e4e059700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9e340a1120 0x7f9e340a35e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.630+0000 7f9e4e059700 1 --2- 172.21.15.33:0/1964918898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e48100f50 0x7f9e48197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.630+0000 7f9e4e059700 1 -- 172.21.15.33:0/1964918898 >> 172.21.15.33:0/1964918898 conn(0x7f9e480fb290 msgr2=0x7f9e480681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:36.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.630+0000 7f9e4e059700 1 -- 172.21.15.33:0/1964918898 shutdown_connections 2024-05-08T21:16:36.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.630+0000 7f9e4e059700 1 -- 172.21.15.33:0/1964918898 wait complete. 2024-05-08T21:16:36.645 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:36 smithi033 ceph-mon[42528]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:16:36.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:36 smithi033 ceph-mon[42528]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l;count:12 2024-05-08T21:16:36.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:36 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:36.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:36 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:36.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:36 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:36.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:36 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:36.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:36 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:36.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:36 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:36.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:36 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:36.647 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:16:36.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.781+0000 7fd6761c0700 1 -- 172.21.15.33:0/3218796413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd670102130 msgr2=0x7fd670102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.781+0000 7fd6761c0700 1 --2- 172.21.15.33:0/3218796413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd670102130 0x7fd670102590 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd664009990 tx=0x7fd664009ca0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.781+0000 7fd6761c0700 1 -- 172.21.15.33:0/3218796413 shutdown_connections 2024-05-08T21:16:36.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.781+0000 7fd6761c0700 1 --2- 172.21.15.33:0/3218796413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd670102ad0 0x7fd670109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.781+0000 7fd6761c0700 1 --2- 172.21.15.33:0/3218796413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6700ff160 0x7fd6700ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.781+0000 7fd6761c0700 1 --2- 172.21.15.33:0/3218796413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd670102130 0x7fd670102590 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.781+0000 7fd6761c0700 1 -- 172.21.15.33:0/3218796413 >> 172.21.15.33:0/3218796413 conn(0x7fd6700faa90 msgr2=0x7fd6700fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:36.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.781+0000 7fd6761c0700 1 -- 172.21.15.33:0/3218796413 shutdown_connections 2024-05-08T21:16:36.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.782+0000 7fd6761c0700 1 -- 172.21.15.33:0/3218796413 wait complete. 2024-05-08T21:16:36.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.782+0000 7fd6761c0700 1 Processor -- start 2024-05-08T21:16:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.782+0000 7fd6761c0700 1 -- start start 2024-05-08T21:16:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd6761c0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6700ff160 0x7fd670101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd6761c0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd670102130 0x7fd6701000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd6761c0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd670102ad0 0x7fd670100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd6761c0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd670100dc0 con 0x7fd6700ff160 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd6761c0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd670100f30 con 0x7fd670102ad0 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd6761c0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6701a0e30 con 0x7fd670102130 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6700ff160 0x7fd670101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6700ff160 0x7fd670101a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36408/0 (socket says 172.21.15.33:36408) 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66f7fe700 1 -- 172.21.15.33:0/3131602982 learned_addr learned my addr 172.21.15.33:0/3131602982 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66ffff700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd670102ad0 0x7fd670100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66f7fe700 1 -- 172.21.15.33:0/3131602982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd670102130 msgr2=0x7fd6701000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66f7fe700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd670102130 0x7fd6701000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66f7fe700 1 -- 172.21.15.33:0/3131602982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd670102ad0 msgr2=0x7fd670100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66f7fe700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd670102ad0 0x7fd670100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66f7fe700 1 -- 172.21.15.33:0/3131602982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6640095f0 con 0x7fd6700ff160 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66f7fe700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6700ff160 0x7fd670101a50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd66000b500 tx=0x7fd66000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66cff9700 1 -- 172.21.15.33:0/3131602982 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660011830 con 0x7fd6700ff160 2024-05-08T21:16:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd6761c0700 1 -- 172.21.15.33:0/3131602982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6701a10c0 con 0x7fd6700ff160 2024-05-08T21:16:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66cff9700 1 -- 172.21.15.33:0/3131602982 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd660011e20 con 0x7fd6700ff160 2024-05-08T21:16:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd66cff9700 1 -- 172.21.15.33:0/3131602982 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660010d20 con 0x7fd6700ff160 2024-05-08T21:16:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.783+0000 7fd6761c0700 1 -- 172.21.15.33:0/3131602982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6701a1650 con 0x7fd6700ff160 2024-05-08T21:16:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.784+0000 7fd6761c0700 1 -- 172.21.15.33:0/3131602982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd670108400 con 0x7fd6700ff160 2024-05-08T21:16:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.786+0000 7fd66cff9700 1 -- 172.21.15.33:0/3131602982 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd660011990 con 0x7fd6700ff160 2024-05-08T21:16:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.786+0000 7fd66cff9700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd6580a1120 0x7fd6580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.786+0000 7fd66cff9700 1 -- 172.21.15.33:0/3131602982 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd6600cc380 con 0x7fd6700ff160 2024-05-08T21:16:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.786+0000 7fd66effd700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd6580a1120 0x7fd6580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.787+0000 7fd66effd700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd6580a1120 0x7fd6580a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd6640098e0 tx=0x7fd664000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:36.790+0000 7fd66cff9700 1 -- 172.21.15.33:0/3131602982 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd660095fb0 con 0x7fd6700ff160 2024-05-08T21:16:37.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.054+0000 7fd6761c0700 1 -- 172.21.15.33:0/3131602982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd670067480 con 0x7fd6700ff160 2024-05-08T21:16:37.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.055+0000 7fd66cff9700 1 -- 172.21.15.33:0/3131602982 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fd6600995d0 con 0x7fd6700ff160 2024-05-08T21:16:37.058 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:37.058 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-08T21:16:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.057+0000 7fd6761c0700 1 -- 172.21.15.33:0/3131602982 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd6580a1120 msgr2=0x7fd6580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.057+0000 7fd6761c0700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd6580a1120 0x7fd6580a35e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd6640098e0 tx=0x7fd664000bc0 comp rx=0 tx=0).stop 2024-05-08T21:16:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.057+0000 7fd6761c0700 1 -- 172.21.15.33:0/3131602982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6700ff160 msgr2=0x7fd670101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.057+0000 7fd6761c0700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6700ff160 0x7fd670101a50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd66000b500 tx=0x7fd66000b8c0 comp rx=0 tx=0).stop 2024-05-08T21:16:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.057+0000 7fd6761c0700 1 -- 172.21.15.33:0/3131602982 shutdown_connections 2024-05-08T21:16:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.057+0000 7fd6761c0700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd6580a1120 0x7fd6580a35e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.057+0000 7fd6761c0700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd670102ad0 0x7fd670100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.057+0000 7fd6761c0700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6700ff160 0x7fd670101a50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:37.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.057+0000 7fd6761c0700 1 --2- 172.21.15.33:0/3131602982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd670102130 0x7fd6701000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:37.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.057+0000 7fd6761c0700 1 -- 172.21.15.33:0/3131602982 >> 172.21.15.33:0/3131602982 conn(0x7fd6700faa90 msgr2=0x7fd670105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:37.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.057+0000 7fd6761c0700 1 -- 172.21.15.33:0/3131602982 shutdown_connections 2024-05-08T21:16:37.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:37.057+0000 7fd6761c0700 1 -- 172.21.15.33:0/3131602982 wait complete. 2024-05-08T21:16:37.061 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 1 2024-05-08T21:16:37.103 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:16:37 smithi082 systemd[1]: Starting Ceph mds.l for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:16:37.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:37 smithi033 ceph-mon[42528]: Deploying daemon mds.l on smithi082 2024-05-08T21:16:37.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:37 smithi033 ceph-mon[42528]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:37.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:37 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3419660131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:16:37.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:37 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1964918898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:16:37.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:37 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3131602982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:37.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:37 smithi071 ceph-mon[47074]: Deploying daemon mds.l on smithi082 2024-05-08T21:16:37.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:37 smithi071 ceph-mon[47074]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:37.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:37 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3419660131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:16:37.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:37 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1964918898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:16:37.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:37 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3131602982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:37.604 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:16:37 smithi082 ceph-mds[83998]: -- 172.21.15.82:0/4019219379 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b6ad7221a0 con 0x55b6ad6f9000 2024-05-08T21:16:37.604 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:16:37 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-l[83975]: starting mds.l at 2024-05-08T21:16:37.604 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:16:37 smithi082 bash[83960]: ab47aa1857f24174afa33a35b7843ffedde33d7e7d872ae2ca21db4fe9634ea2 2024-05-08T21:16:37.604 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:16:37 smithi082 systemd[1]: Started Ceph mds.l for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:16:37.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:37 smithi082 ceph-mon[47512]: Deploying daemon mds.l on smithi082 2024-05-08T21:16:37.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:37 smithi082 ceph-mon[47512]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:37.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:37 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3419660131' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:16:37.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:37 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1964918898' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:16:37.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:37 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3131602982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:38.492 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:38 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] up:boot 2024-05-08T21:16:38.492 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:38 smithi071 ceph-mon[47074]: fsmap 1 up:standby 2024-05-08T21:16:38.492 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T21:16:38.492 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:38.492 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:38.492 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:38.492 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:38.493 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:38.493 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:38 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:38.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:38 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] up:boot 2024-05-08T21:16:38.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:38 smithi033 ceph-mon[42528]: fsmap 1 up:standby 2024-05-08T21:16:38.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T21:16:38.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:38.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:38.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:38.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:38.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:38.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:38 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:38.821 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:16:38 smithi071 systemd[1]: Starting Ceph mds.k for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:16:38.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:38 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] up:boot 2024-05-08T21:16:38.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:38 smithi082 ceph-mon[47512]: fsmap 1 up:standby 2024-05-08T21:16:38.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T21:16:38.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:38.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:38.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:38.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:38.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:38.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:38 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:39.074 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:16:39.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.227+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/3220500623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffd80 msgr2=0x7f5c341001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.227+0000 7f5c3b5b7700 1 --2- 172.21.15.33:0/3220500623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffd80 0x7f5c341001a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c300099c0 tx=0x7f5c30009cd0 comp rx=0 tx=0).stop 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.227+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/3220500623 shutdown_connections 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.227+0000 7f5c3b5b7700 1 --2- 172.21.15.33:0/3220500623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c34101910 0x7f5c341081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.227+0000 7f5c3b5b7700 1 --2- 172.21.15.33:0/3220500623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffd80 0x7f5c341001a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.227+0000 7f5c3b5b7700 1 --2- 172.21.15.33:0/3220500623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34100f70 0x7f5c341013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.227+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/3220500623 >> 172.21.15.33:0/3220500623 conn(0x7f5c340fb310 msgr2=0x7f5c340fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.227+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/3220500623 shutdown_connections 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.227+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/3220500623 wait complete. 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.228+0000 7f5c3b5b7700 1 Processor -- start 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.228+0000 7f5c3b5b7700 1 -- start start 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.228+0000 7f5c3b5b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffd80 0x7f5c34196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.228+0000 7f5c3b5b7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34100f70 0x7f5c34197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:39.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.228+0000 7f5c3b5b7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c34101910 0x7f5c3419e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.228+0000 7f5c3b5b7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3419e890 con 0x7f5c340ffd80 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.228+0000 7f5c3b5b7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3419ea00 con 0x7f5c34101910 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.228+0000 7f5c3b5b7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3419ebc0 con 0x7f5c34100f70 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.228+0000 7f5c39353700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffd80 0x7f5c34196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.228+0000 7f5c39353700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffd80 0x7f5c34196df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36424/0 (socket says 172.21.15.33:36424) 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c39353700 1 -- 172.21.15.33:0/1726483165 learned_addr learned my addr 172.21.15.33:0/1726483165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c39b54700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c34101910 0x7f5c3419e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c38b52700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34100f70 0x7f5c34197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c39353700 1 -- 172.21.15.33:0/1726483165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34100f70 msgr2=0x7f5c34197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c39353700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34100f70 0x7f5c34197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c39353700 1 -- 172.21.15.33:0/1726483165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c34101910 msgr2=0x7f5c3419e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c39353700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c34101910 0x7f5c3419e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c39353700 1 -- 172.21.15.33:0/1726483165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c300095f0 con 0x7f5c340ffd80 2024-05-08T21:16:39.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c38b52700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34100f70 0x7f5c34197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:16:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c39353700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffd80 0x7f5c34196df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5c30000c00 tx=0x7f5c3000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1726483165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3001d070 con 0x7f5c340ffd80 2024-05-08T21:16:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1726483165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c300054c0 con 0x7f5c340ffd80 2024-05-08T21:16:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/1726483165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3419ee40 con 0x7f5c340ffd80 2024-05-08T21:16:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1726483165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30004ba0 con 0x7f5c340ffd80 2024-05-08T21:16:39.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.229+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/1726483165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3419f200 con 0x7f5c340ffd80 2024-05-08T21:16:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.230+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/1726483165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c34067d10 con 0x7f5c340ffd80 2024-05-08T21:16:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.232+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1726483165 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5c30004d00 con 0x7f5c340ffd80 2024-05-08T21:16:39.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.232+0000 7f5c2a7fc700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5c200a11a0 0x7f5c200a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:39.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.233+0000 7f5c38b52700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5c200a11a0 0x7f5c200a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:39.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.233+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1726483165 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5c300cd520 con 0x7f5c340ffd80 2024-05-08T21:16:39.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.233+0000 7f5c38b52700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5c200a11a0 0x7f5c200a3660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009be0 tx=0x7f5c24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:39.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.237+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1726483165 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5c30097140 con 0x7f5c340ffd80 2024-05-08T21:16:39.415 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:16:39 smithi071 ceph-mds[82721]: -- 172.21.15.71:0/4110357884 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c4d21f41a0 con 0x55c4d21cb000 2024-05-08T21:16:39.415 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:16:39 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-k[82698]: starting mds.k at 2024-05-08T21:16:39.416 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:16:39 smithi071 bash[82683]: 72c661c2ce5f8047007a4f81b472cf66f7373cfff1fcb364b526466c9a7a5633 2024-05-08T21:16:39.416 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:16:39 smithi071 systemd[1]: Started Ceph mds.k for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:16:39.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.531+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/1726483165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c3410b490 con 0x7f5c340ffd80 2024-05-08T21:16:39.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.532+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1726483165 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2011 (secure 0 0 0) 0x7f5c3009a760 con 0x7f5c340ffd80 2024-05-08T21:16:39.534 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:39.535 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-08T21:16:39.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.534+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/1726483165 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5c200a11a0 msgr2=0x7f5c200a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:39.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.534+0000 7f5c3b5b7700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5c200a11a0 0x7f5c200a3660 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009be0 tx=0x7f5c24009260 comp rx=0 tx=0).stop 2024-05-08T21:16:39.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.534+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/1726483165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffd80 msgr2=0x7f5c34196df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:39.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.534+0000 7f5c3b5b7700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffd80 0x7f5c34196df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5c30000c00 tx=0x7f5c3000f690 comp rx=0 tx=0).stop 2024-05-08T21:16:39.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.534+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/1726483165 shutdown_connections 2024-05-08T21:16:39.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.534+0000 7f5c3b5b7700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5c200a11a0 0x7f5c200a3660 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:39.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.534+0000 7f5c3b5b7700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c34101910 0x7f5c3419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:39.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.534+0000 7f5c3b5b7700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffd80 0x7f5c34196df0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:39.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.534+0000 7f5c3b5b7700 1 --2- 172.21.15.33:0/1726483165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34100f70 0x7f5c34197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:39.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.534+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/1726483165 >> 172.21.15.33:0/1726483165 conn(0x7f5c340fb310 msgr2=0x7f5c34068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:39.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.534+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/1726483165 shutdown_connections 2024-05-08T21:16:39.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:39.534+0000 7f5c3b5b7700 1 -- 172.21.15.33:0/1726483165 wait complete. 2024-05-08T21:16:39.538 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-05-08T21:16:39.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:39 smithi033 ceph-mon[42528]: Deploying daemon mds.k on smithi071 2024-05-08T21:16:39.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:39 smithi033 ceph-mon[42528]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:39.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:39.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:39.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:39.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:39.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:39.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:39 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:39 smithi071 ceph-mon[47074]: Deploying daemon mds.k on smithi071 2024-05-08T21:16:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:39 smithi071 ceph-mon[47074]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:39 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:39 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:39 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:39 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:39 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:39 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:39.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:39 smithi082 ceph-mon[47512]: Deploying daemon mds.k on smithi071 2024-05-08T21:16:39.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:39 smithi082 ceph-mon[47512]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:39.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:39.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:39.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:39.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:39.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:39.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:39 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:40.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:40 smithi033 ceph-mon[42528]: Deploying daemon mds.j on smithi033 2024-05-08T21:16:40.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:40 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.71:6832/497942273,v1:172.21.15.71:6833/497942273] up:boot 2024-05-08T21:16:40.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:40 smithi033 ceph-mon[42528]: fsmap 2 up:standby 2024-05-08T21:16:40.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:40 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T21:16:40.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:40 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1726483165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:40.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:40 smithi071 ceph-mon[47074]: Deploying daemon mds.j on smithi033 2024-05-08T21:16:40.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:40 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.71:6832/497942273,v1:172.21.15.71:6833/497942273] up:boot 2024-05-08T21:16:40.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:40 smithi071 ceph-mon[47074]: fsmap 2 up:standby 2024-05-08T21:16:40.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:40 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T21:16:40.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:40 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1726483165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:40.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:40 smithi082 ceph-mon[47512]: Deploying daemon mds.j on smithi033 2024-05-08T21:16:40.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:40 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.71:6832/497942273,v1:172.21.15.71:6833/497942273] up:boot 2024-05-08T21:16:40.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:40 smithi082 ceph-mon[47512]: fsmap 2 up:standby 2024-05-08T21:16:40.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:40 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T21:16:40.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:40 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1726483165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:40.890 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:16:40 smithi033 systemd[1]: Starting Ceph mds.j for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:16:41.419 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:16:41 smithi033 bash[92126]: 177d48c7fc705fba86e78fb2540d7f74dcaad3f2b86be2504c97a61eb3d23934 2024-05-08T21:16:41.419 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:16:41 smithi033 systemd[1]: Started Ceph mds.j for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:16:41.419 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:16:41 smithi033 ceph-mds[92163]: -- 172.21.15.33:0/2905053491 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f8eb3561a0 con 0x55f8eb32c800 2024-05-08T21:16:41.419 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:16:41 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-j[92140]: starting mds.j at 2024-05-08T21:16:41.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:41 smithi033 ceph-mon[42528]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:41.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:41 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:41.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:41 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:41.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:41 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:41.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:41 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:41.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:41 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:41.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:41 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:41.553 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:16:41.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.696+0000 7f8309033700 1 -- 172.21.15.33:0/4067895467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8304102130 msgr2=0x7f8304102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.696+0000 7f8309033700 1 --2- 172.21.15.33:0/4067895467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8304102130 0x7f8304102590 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009990 tx=0x7f82f8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:16:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.696+0000 7f8309033700 1 -- 172.21.15.33:0/4067895467 shutdown_connections 2024-05-08T21:16:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.696+0000 7f8309033700 1 --2- 172.21.15.33:0/4067895467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8304102ad0 0x7f8304109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.696+0000 7f8309033700 1 --2- 172.21.15.33:0/4067895467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83040ff160 0x7f83040ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.696+0000 7f8309033700 1 --2- 172.21.15.33:0/4067895467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8304102130 0x7f8304102590 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.696+0000 7f8309033700 1 -- 172.21.15.33:0/4067895467 >> 172.21.15.33:0/4067895467 conn(0x7f83040faa90 msgr2=0x7f83040fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.696+0000 7f8309033700 1 -- 172.21.15.33:0/4067895467 shutdown_connections 2024-05-08T21:16:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.697+0000 7f8309033700 1 -- 172.21.15.33:0/4067895467 wait complete. 2024-05-08T21:16:41.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.697+0000 7f8309033700 1 Processor -- start 2024-05-08T21:16:41.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8309033700 1 -- start start 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8309033700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83040ff160 0x7f8304101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8309033700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8304102130 0x7f83041000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8309033700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8304102ad0 0x7f8304100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8309033700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8304100dc0 con 0x7f83040ff160 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8309033700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8304100f30 con 0x7f8304102ad0 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8309033700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83041a0df0 con 0x7f8304102130 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83040ff160 0x7f8304101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83040ff160 0x7f8304101a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55252/0 (socket says 172.21.15.33:55252) 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8302d9d700 1 -- 172.21.15.33:0/1951128486 learned_addr learned my addr 172.21.15.33:0/1951128486 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f830359e700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8304102ad0 0x7f8304100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f830259c700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8304102130 0x7f83041000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8302d9d700 1 -- 172.21.15.33:0/1951128486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8304102130 msgr2=0x7f83041000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:41.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8302d9d700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8304102130 0x7f83041000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8302d9d700 1 -- 172.21.15.33:0/1951128486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8304102ad0 msgr2=0x7f8304100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8302d9d700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8304102ad0 0x7f8304100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8302d9d700 1 -- 172.21.15.33:0/1951128486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80095f0 con 0x7f83040ff160 2024-05-08T21:16:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.698+0000 7f8302d9d700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83040ff160 0x7f8304101a50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f82f400e970 tx=0x7f82f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.699+0000 7f82ebfff700 1 -- 172.21.15.33:0/1951128486 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4009790 con 0x7f83040ff160 2024-05-08T21:16:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.699+0000 7f8309033700 1 -- 172.21.15.33:0/1951128486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83041a1080 con 0x7f83040ff160 2024-05-08T21:16:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.699+0000 7f82ebfff700 1 -- 172.21.15.33:0/1951128486 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f82f400eec0 con 0x7f83040ff160 2024-05-08T21:16:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.699+0000 7f82ebfff700 1 -- 172.21.15.33:0/1951128486 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4003c10 con 0x7f83040ff160 2024-05-08T21:16:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.699+0000 7f8309033700 1 -- 172.21.15.33:0/1951128486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83041a15e0 con 0x7f83040ff160 2024-05-08T21:16:41.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.700+0000 7f8309033700 1 -- 172.21.15.33:0/1951128486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8304108410 con 0x7f83040ff160 2024-05-08T21:16:41.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.701+0000 7f82ebfff700 1 -- 172.21.15.33:0/1951128486 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f82f400cb70 con 0x7f83040ff160 2024-05-08T21:16:41.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.702+0000 7f82ebfff700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f82ec0a1120 0x7f82ec0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:41.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.702+0000 7f82ebfff700 1 -- 172.21.15.33:0/1951128486 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f82f4014070 con 0x7f83040ff160 2024-05-08T21:16:41.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.702+0000 7f830259c700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f82ec0a1120 0x7f82ec0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:41.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.702+0000 7f830259c700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f82ec0a1120 0x7f82ec0a35e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82f800b670 tx=0x7f82f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:41.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.706+0000 7f82ebfff700 1 -- 172.21.15.33:0/1951128486 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f82f4051d60 con 0x7f83040ff160 2024-05-08T21:16:41.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:41 smithi071 ceph-mon[47074]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:41.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:41.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:41.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:41.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:41.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:41.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:41 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:41.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:41 smithi082 ceph-mon[47512]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:41.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:41 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:41.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:41 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:41.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:41 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:41.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:41 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:41.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:41 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:41.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:41 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.992+0000 7f8309033700 1 -- 172.21.15.33:0/1951128486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f830410c7e0 con 0x7f83040ff160 2024-05-08T21:16:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.992+0000 7f82ebfff700 1 -- 172.21.15.33:0/1951128486 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2762 (secure 0 0 0) 0x7f82f4099eb0 con 0x7f83040ff160 2024-05-08T21:16:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:41.995 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-05-08T21:16:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.995+0000 7f8309033700 1 -- 172.21.15.33:0/1951128486 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f82ec0a1120 msgr2=0x7f82ec0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.995+0000 7f8309033700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f82ec0a1120 0x7f82ec0a35e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82f800b670 tx=0x7f82f8005fb0 comp rx=0 tx=0).stop 2024-05-08T21:16:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.995+0000 7f8309033700 1 -- 172.21.15.33:0/1951128486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83040ff160 msgr2=0x7f8304101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.995+0000 7f8309033700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83040ff160 0x7f8304101a50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f82f400e970 tx=0x7f82f400ec80 comp rx=0 tx=0).stop 2024-05-08T21:16:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.995+0000 7f8309033700 1 -- 172.21.15.33:0/1951128486 shutdown_connections 2024-05-08T21:16:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.995+0000 7f8309033700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f82ec0a1120 0x7f82ec0a35e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.995+0000 7f8309033700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8304102ad0 0x7f8304100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.995+0000 7f8309033700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83040ff160 0x7f8304101a50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.995+0000 7f8309033700 1 --2- 172.21.15.33:0/1951128486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8304102130 0x7f83041000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.995+0000 7f8309033700 1 -- 172.21.15.33:0/1951128486 >> 172.21.15.33:0/1951128486 conn(0x7f83040faa90 msgr2=0x7f8304105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.995+0000 7f8309033700 1 -- 172.21.15.33:0/1951128486 shutdown_connections 2024-05-08T21:16:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:41.995+0000 7f8309033700 1 -- 172.21.15.33:0/1951128486 wait complete. 2024-05-08T21:16:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 4 2024-05-08T21:16:42.708 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:16:42 smithi033 systemd[1]: Starting Ceph mds.d for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:16:42.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:42 smithi033 ceph-mon[42528]: Deploying daemon mds.d on smithi033 2024-05-08T21:16:42.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:42 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.33:6834/82548780,v1:172.21.15.33:6835/82548780] up:boot 2024-05-08T21:16:42.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:42 smithi033 ceph-mon[42528]: fsmap 3 up:standby 2024-05-08T21:16:42.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:42 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T21:16:42.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:42 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1951128486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:42.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:42 smithi071 ceph-mon[47074]: Deploying daemon mds.d on smithi033 2024-05-08T21:16:42.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:42 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.33:6834/82548780,v1:172.21.15.33:6835/82548780] up:boot 2024-05-08T21:16:42.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:42 smithi071 ceph-mon[47074]: fsmap 3 up:standby 2024-05-08T21:16:42.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:42 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T21:16:42.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:42 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1951128486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:42.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:42 smithi082 ceph-mon[47512]: Deploying daemon mds.d on smithi033 2024-05-08T21:16:42.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:42 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.33:6834/82548780,v1:172.21.15.33:6835/82548780] up:boot 2024-05-08T21:16:42.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:42 smithi082 ceph-mon[47512]: fsmap 3 up:standby 2024-05-08T21:16:42.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:42 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T21:16:42.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:42 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1951128486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:43.208 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:16:42 smithi033 bash[92514]: 9af8667f927d91e0e7a015fbc14501424215483f73f94ad84daa8408cb07e92b 2024-05-08T21:16:43.208 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:16:42 smithi033 systemd[1]: Started Ceph mds.d for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:16:43.208 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:16:42 smithi033 ceph-mds[92552]: -- 172.21.15.33:0/3119681814 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a755aac1a0 con 0x55a755a82c00 2024-05-08T21:16:43.208 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:16:42 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-d[92529]: starting mds.d at 2024-05-08T21:16:43.700 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:43 smithi071 ceph-mon[47074]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:43.701 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:43.701 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:43.701 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:43.701 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:43.701 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:43.701 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:43 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:43.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:43 smithi033 ceph-mon[42528]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:43.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:43.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:43.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:43.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:43.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:43.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:43 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:43.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:43 smithi082 ceph-mon[47512]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:43.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:43.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:43.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:43.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:43.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:43.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:43 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:44.013 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:16:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.164+0000 7f6b6841b700 1 -- 172.21.15.33:0/1464072274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60100e80 msgr2=0x7f6b601012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.164+0000 7f6b6841b700 1 --2- 172.21.15.33:0/1464072274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60100e80 0x7f6b601012a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009990 tx=0x7f6b5c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:16:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.164+0000 7f6b6841b700 1 -- 172.21.15.33:0/1464072274 shutdown_connections 2024-05-08T21:16:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.164+0000 7f6b6841b700 1 --2- 172.21.15.33:0/1464072274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b60102990 0x7f6b601092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.164+0000 7f6b6841b700 1 --2- 172.21.15.33:0/1464072274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60100e80 0x7f6b601012a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.164+0000 7f6b6841b700 1 --2- 172.21.15.33:0/1464072274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b60101fd0 0x7f6b60102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:44.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.164+0000 7f6b6841b700 1 -- 172.21.15.33:0/1464072274 >> 172.21.15.33:0/1464072274 conn(0x7f6b600fc410 msgr2=0x7f6b600fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:44.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.164+0000 7f6b6841b700 1 -- 172.21.15.33:0/1464072274 shutdown_connections 2024-05-08T21:16:44.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.164+0000 7f6b6841b700 1 -- 172.21.15.33:0/1464072274 wait complete. 2024-05-08T21:16:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.165+0000 7f6b6841b700 1 Processor -- start 2024-05-08T21:16:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.165+0000 7f6b6841b700 1 -- start start 2024-05-08T21:16:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b6841b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b60100e80 0x7f6b60196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b6841b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60101fd0 0x7f6b601972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b6841b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b60102990 0x7f6b6019e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b6841b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6019e860 con 0x7f6b60101fd0 2024-05-08T21:16:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b6841b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6019e9d0 con 0x7f6b60102990 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b6841b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6019eb90 con 0x7f6b60100e80 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b659b6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60101fd0 0x7f6b601972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b659b6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60101fd0 0x7f6b601972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55274/0 (socket says 172.21.15.33:55274) 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b659b6700 1 -- 172.21.15.33:0/2245771840 learned_addr learned my addr 172.21.15.33:0/2245771840 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b661b7700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b60100e80 0x7f6b60196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b669b8700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b60102990 0x7f6b6019e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b659b6700 1 -- 172.21.15.33:0/2245771840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b60100e80 msgr2=0x7f6b60196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b659b6700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b60100e80 0x7f6b60196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b659b6700 1 -- 172.21.15.33:0/2245771840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b60102990 msgr2=0x7f6b6019e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b659b6700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b60102990 0x7f6b6019e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b659b6700 1 -- 172.21.15.33:0/2245771840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0095f0 con 0x7f6b60101fd0 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b659b6700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60101fd0 0x7f6b601972b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000e9a0 tx=0x7f6b5000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b577fe700 1 -- 172.21.15.33:0/2245771840 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50009790 con 0x7f6b60101fd0 2024-05-08T21:16:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b577fe700 1 -- 172.21.15.33:0/2245771840 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b50010430 con 0x7f6b60101fd0 2024-05-08T21:16:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b6841b700 1 -- 172.21.15.33:0/2245771840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6019ee70 con 0x7f6b60101fd0 2024-05-08T21:16:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.166+0000 7f6b577fe700 1 -- 172.21.15.33:0/2245771840 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5000cea0 con 0x7f6b60101fd0 2024-05-08T21:16:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.167+0000 7f6b6841b700 1 -- 172.21.15.33:0/2245771840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6019f2a0 con 0x7f6b60101fd0 2024-05-08T21:16:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.168+0000 7f6b6841b700 1 -- 172.21.15.33:0/2245771840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b60108390 con 0x7f6b60101fd0 2024-05-08T21:16:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.169+0000 7f6b577fe700 1 -- 172.21.15.33:0/2245771840 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6b500105a0 con 0x7f6b60101fd0 2024-05-08T21:16:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.170+0000 7f6b577fe700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b4c0a11a0 0x7f6b4c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.170+0000 7f6b661b7700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b4c0a11a0 0x7f6b4c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.170+0000 7f6b577fe700 1 -- 172.21.15.33:0/2245771840 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6b50014070 con 0x7f6b60101fd0 2024-05-08T21:16:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.170+0000 7f6b661b7700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b4c0a11a0 0x7f6b4c0a3660 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c006010 tx=0x7f6b5c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:44.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.175+0000 7f6b577fe700 1 -- 172.21.15.33:0/2245771840 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6b500961d0 con 0x7f6b60101fd0 2024-05-08T21:16:44.245 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:16:44 smithi071 systemd[1]: Starting Ceph mds.e for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:16:44.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.446+0000 7f6b6841b700 1 -- 172.21.15.33:0/2245771840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b60067480 con 0x7f6b60101fd0 2024-05-08T21:16:44.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.446+0000 7f6b577fe700 1 -- 172.21.15.33:0/2245771840 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3516 (secure 0 0 0) 0x7f6b500997f0 con 0x7f6b60101fd0 2024-05-08T21:16:44.449 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:44.449 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[]} 2024-05-08T21:16:44.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.449+0000 7f6b6841b700 1 -- 172.21.15.33:0/2245771840 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b4c0a11a0 msgr2=0x7f6b4c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:44.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.449+0000 7f6b6841b700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b4c0a11a0 0x7f6b4c0a3660 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c006010 tx=0x7f6b5c00b5b0 comp rx=0 tx=0).stop 2024-05-08T21:16:44.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.449+0000 7f6b6841b700 1 -- 172.21.15.33:0/2245771840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60101fd0 msgr2=0x7f6b601972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:44.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.449+0000 7f6b6841b700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60101fd0 0x7f6b601972b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000e9a0 tx=0x7f6b5000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:16:44.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.449+0000 7f6b6841b700 1 -- 172.21.15.33:0/2245771840 shutdown_connections 2024-05-08T21:16:44.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.449+0000 7f6b6841b700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b4c0a11a0 0x7f6b4c0a3660 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:44.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.449+0000 7f6b6841b700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b60102990 0x7f6b6019e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:44.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.449+0000 7f6b6841b700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b60100e80 0x7f6b60196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:44.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.449+0000 7f6b6841b700 1 --2- 172.21.15.33:0/2245771840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b60101fd0 0x7f6b601972b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:44.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.449+0000 7f6b6841b700 1 -- 172.21.15.33:0/2245771840 >> 172.21.15.33:0/2245771840 conn(0x7f6b600fc410 msgr2=0x7f6b601054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:44.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.449+0000 7f6b6841b700 1 -- 172.21.15.33:0/2245771840 shutdown_connections 2024-05-08T21:16:44.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:44.449+0000 7f6b6841b700 1 -- 172.21.15.33:0/2245771840 wait complete. 2024-05-08T21:16:44.453 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 5 2024-05-08T21:16:44.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:44 smithi033 ceph-mon[42528]: Deploying daemon mds.e on smithi071 2024-05-08T21:16:44.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:44 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.33:6836/362093971,v1:172.21.15.33:6837/362093971] up:boot 2024-05-08T21:16:44.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:44 smithi033 ceph-mon[42528]: fsmap 4 up:standby 2024-05-08T21:16:44.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:44 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T21:16:44.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:44 smithi071 ceph-mon[47074]: Deploying daemon mds.e on smithi071 2024-05-08T21:16:44.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:44 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.33:6836/362093971,v1:172.21.15.33:6837/362093971] up:boot 2024-05-08T21:16:44.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:44 smithi071 ceph-mon[47074]: fsmap 4 up:standby 2024-05-08T21:16:44.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:44 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T21:16:44.821 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:16:44 smithi071 ceph-mds[83063]: -- 172.21.15.71:0/2883250631 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555b5fb001a0 con 0x555b5fad6c00 2024-05-08T21:16:44.821 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:16:44 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-e[83040]: starting mds.e at 2024-05-08T21:16:44.821 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:16:44 smithi071 bash[83025]: 8256d60fba74bd0e50e9a5a06949627917f164c31c47ef1ed9fd8e83d5a006b5 2024-05-08T21:16:44.822 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:16:44 smithi071 systemd[1]: Started Ceph mds.e for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:16:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:44 smithi082 ceph-mon[47512]: Deploying daemon mds.e on smithi071 2024-05-08T21:16:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:44 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.33:6836/362093971,v1:172.21.15.33:6837/362093971] up:boot 2024-05-08T21:16:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:44 smithi082 ceph-mon[47512]: fsmap 4 up:standby 2024-05-08T21:16:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:44 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T21:16:45.683 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:45 smithi082 ceph-mon[47512]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:45.684 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:45 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2245771840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:45.684 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:45.685 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:45.685 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:45.685 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:45.685 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:45.685 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:45 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:45.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:45 smithi033 ceph-mon[42528]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:45.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:45 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2245771840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:45.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:45.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:45.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:45.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:45.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:45.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:45 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:45.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:45 smithi071 ceph-mon[47074]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:45.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:45 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2245771840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:45.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:45.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:45.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:45.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:45.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:45.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:45 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:46.104 INFO:journalctl@ceph.mds.f.smithi082.stdout:May 08 21:16:45 smithi082 systemd[1]: Starting Ceph mds.f for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:16:46.466 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:16:46.603 INFO:journalctl@ceph.mds.f.smithi082.stdout:May 08 21:16:46 smithi082 bash[84310]: 3f16dded9fcf6750107f22ce3f38728a934197817342a5532f8579a6193fe48d 2024-05-08T21:16:46.604 INFO:journalctl@ceph.mds.f.smithi082.stdout:May 08 21:16:46 smithi082 systemd[1]: Started Ceph mds.f for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:16:46.604 INFO:journalctl@ceph.mds.f.smithi082.stdout:May 08 21:16:46 smithi082 ceph-mds[84348]: -- 172.21.15.82:0/921794363 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55705bd141a0 con 0x55705bcea800 2024-05-08T21:16:46.604 INFO:journalctl@ceph.mds.f.smithi082.stdout:May 08 21:16:46 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-f[84325]: starting mds.f at 2024-05-08T21:16:46.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:46 smithi082 ceph-mon[47512]: Deploying daemon mds.f on smithi082 2024-05-08T21:16:46.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:46 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] up:boot 2024-05-08T21:16:46.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:46 smithi082 ceph-mon[47512]: fsmap 5 up:standby 2024-05-08T21:16:46.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T21:16:46.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:46.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:46.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:46.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:46.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:46.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:46 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:46.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.612+0000 7f0094590700 1 -- 172.21.15.33:0/990404684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f008c101fd0 msgr2=0x7f008c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.612+0000 7f0094590700 1 --2- 172.21.15.33:0/990404684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f008c101fd0 0x7f008c102450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0080009990 tx=0x7f0080009ca0 comp rx=0 tx=0).stop 2024-05-08T21:16:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.612+0000 7f0094590700 1 -- 172.21.15.33:0/990404684 shutdown_connections 2024-05-08T21:16:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.612+0000 7f0094590700 1 --2- 172.21.15.33:0/990404684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c102990 0x7f008c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.612+0000 7f0094590700 1 --2- 172.21.15.33:0/990404684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c100e80 0x7f008c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.612+0000 7f0094590700 1 --2- 172.21.15.33:0/990404684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f008c101fd0 0x7f008c102450 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.612+0000 7f0094590700 1 -- 172.21.15.33:0/990404684 >> 172.21.15.33:0/990404684 conn(0x7f008c0fc410 msgr2=0x7f008c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.612+0000 7f0094590700 1 -- 172.21.15.33:0/990404684 shutdown_connections 2024-05-08T21:16:46.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.612+0000 7f0094590700 1 -- 172.21.15.33:0/990404684 wait complete. 2024-05-08T21:16:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.612+0000 7f0094590700 1 Processor -- start 2024-05-08T21:16:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.612+0000 7f0094590700 1 -- start start 2024-05-08T21:16:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f0094590700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f008c100e80 0x7f008c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f0094590700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c101fd0 0x7f008c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f0094590700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c102990 0x7f008c1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f0094590700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c1a09a0 con 0x7f008c100e80 2024-05-08T21:16:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f0094590700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c1a0ae0 con 0x7f008c102990 2024-05-08T21:16:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f0094590700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c1a0c70 con 0x7f008c101fd0 2024-05-08T21:16:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f009232c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f008c100e80 0x7f008c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f009232c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f008c100e80 0x7f008c198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55290/0 (socket says 172.21.15.33:55290) 2024-05-08T21:16:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f009232c700 1 -- 172.21.15.33:0/3868126690 learned_addr learned my addr 172.21.15.33:0/3868126690 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f0091b2b700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c101fd0 0x7f008c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f0092b2d700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c102990 0x7f008c1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f009232c700 1 -- 172.21.15.33:0/3868126690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c101fd0 msgr2=0x7f008c1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f009232c700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c101fd0 0x7f008c1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f009232c700 1 -- 172.21.15.33:0/3868126690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c102990 msgr2=0x7f008c1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f009232c700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c102990 0x7f008c1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f009232c700 1 -- 172.21.15.33:0/3868126690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00800095f0 con 0x7f008c100e80 2024-05-08T21:16:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f0092b2d700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c102990 0x7f008c1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:16:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f009232c700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f008c100e80 0x7f008c198e70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f008800c9b0 tx=0x7f008800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f007f7fe700 1 -- 172.21.15.33:0/3868126690 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088007b00 con 0x7f008c100e80 2024-05-08T21:16:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f007f7fe700 1 -- 172.21.15.33:0/3868126690 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0088007c60 con 0x7f008c100e80 2024-05-08T21:16:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f0094590700 1 -- 172.21.15.33:0/3868126690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c1a0f50 con 0x7f008c100e80 2024-05-08T21:16:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.613+0000 7f007f7fe700 1 -- 172.21.15.33:0/3868126690 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088018660 con 0x7f008c100e80 2024-05-08T21:16:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.614+0000 7f0094590700 1 -- 172.21.15.33:0/3868126690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c1a1370 con 0x7f008c100e80 2024-05-08T21:16:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.615+0000 7f0094590700 1 -- 172.21.15.33:0/3868126690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c067480 con 0x7f008c100e80 2024-05-08T21:16:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.616+0000 7f007f7fe700 1 -- 172.21.15.33:0/3868126690 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f008800f450 con 0x7f008c100e80 2024-05-08T21:16:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.616+0000 7f007f7fe700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f00780a1120 0x7f00780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.617+0000 7f0091b2b700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f00780a1120 0x7f00780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.617+0000 7f007f7fe700 1 -- 172.21.15.33:0/3868126690 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f00880cc750 con 0x7f008c100e80 2024-05-08T21:16:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.617+0000 7f0091b2b700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f00780a1120 0x7f00780a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f008000b5c0 tx=0x7f0080003720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.621+0000 7f007f7fe700 1 -- 172.21.15.33:0/3868126690 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0088096370 con 0x7f008c100e80 2024-05-08T21:16:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:46 smithi033 ceph-mon[42528]: Deploying daemon mds.f on smithi082 2024-05-08T21:16:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:46 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] up:boot 2024-05-08T21:16:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:46 smithi033 ceph-mon[42528]: fsmap 5 up:standby 2024-05-08T21:16:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T21:16:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:46 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:46.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:46 smithi071 ceph-mon[47074]: Deploying daemon mds.f on smithi082 2024-05-08T21:16:46.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:46 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] up:boot 2024-05-08T21:16:46.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:46 smithi071 ceph-mon[47074]: fsmap 5 up:standby 2024-05-08T21:16:46.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T21:16:46.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:46.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:46.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:46.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:46.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:46.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:46 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:46.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.888+0000 7f0094590700 1 -- 172.21.15.33:0/3868126690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f008c1a15c0 con 0x7f008c100e80 2024-05-08T21:16:46.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.888+0000 7f007f7fe700 1 -- 172.21.15.33:0/3868126690 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5030 (secure 0 0 0) 0x7f008800fe60 con 0x7f008c100e80 2024-05-08T21:16:46.891 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:46.891 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-05-08T21:16:46.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.890+0000 7f0094590700 1 -- 172.21.15.33:0/3868126690 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f00780a1120 msgr2=0x7f00780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:46.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.890+0000 7f0094590700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f00780a1120 0x7f00780a35e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f008000b5c0 tx=0x7f0080003720 comp rx=0 tx=0).stop 2024-05-08T21:16:46.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.890+0000 7f0094590700 1 -- 172.21.15.33:0/3868126690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f008c100e80 msgr2=0x7f008c198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:46.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.890+0000 7f0094590700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f008c100e80 0x7f008c198e70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f008800c9b0 tx=0x7f008800ccc0 comp rx=0 tx=0).stop 2024-05-08T21:16:46.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.891+0000 7f0094590700 1 -- 172.21.15.33:0/3868126690 shutdown_connections 2024-05-08T21:16:46.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.891+0000 7f0094590700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f00780a1120 0x7f00780a35e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:46.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.891+0000 7f0094590700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f008c102990 0x7f008c1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:46.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.891+0000 7f0094590700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f008c100e80 0x7f008c198e70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:46.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.891+0000 7f0094590700 1 --2- 172.21.15.33:0/3868126690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f008c101fd0 0x7f008c1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:46.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.891+0000 7f0094590700 1 -- 172.21.15.33:0/3868126690 >> 172.21.15.33:0/3868126690 conn(0x7f008c0fc410 msgr2=0x7f008c0731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:46.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.891+0000 7f0094590700 1 -- 172.21.15.33:0/3868126690 shutdown_connections 2024-05-08T21:16:46.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:46.891+0000 7f0094590700 1 -- 172.21.15.33:0/3868126690 wait complete. 2024-05-08T21:16:46.895 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2024-05-08T21:16:47.639 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:47 smithi082 ceph-mon[47512]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:47.639 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:47 smithi082 ceph-mon[47512]: Deploying daemon mds.c on smithi082 2024-05-08T21:16:47.639 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:47 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.82:6834/1877231014,v1:172.21.15.82:6835/1877231014] up:boot 2024-05-08T21:16:47.639 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:47 smithi082 ceph-mon[47512]: fsmap 6 up:standby 2024-05-08T21:16:47.639 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:47 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T21:16:47.639 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:47 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3868126690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:47.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:47 smithi033 ceph-mon[42528]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:47.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:47 smithi033 ceph-mon[42528]: Deploying daemon mds.c on smithi082 2024-05-08T21:16:47.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:47 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.82:6834/1877231014,v1:172.21.15.82:6835/1877231014] up:boot 2024-05-08T21:16:47.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:47 smithi033 ceph-mon[42528]: fsmap 6 up:standby 2024-05-08T21:16:47.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:47 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T21:16:47.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:47 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3868126690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:47.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:47 smithi071 ceph-mon[47074]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:47.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:47 smithi071 ceph-mon[47074]: Deploying daemon mds.c on smithi082 2024-05-08T21:16:47.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:47 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.82:6834/1877231014,v1:172.21.15.82:6835/1877231014] up:boot 2024-05-08T21:16:47.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:47 smithi071 ceph-mon[47074]: fsmap 6 up:standby 2024-05-08T21:16:47.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:47 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T21:16:47.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:47 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3868126690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:47.891 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:16:47 smithi082 systemd[1]: Starting Ceph mds.c for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:16:48.353 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:16:48 smithi082 ceph-mds[84694]: -- 172.21.15.82:0/342863109 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561bc73261a0 con 0x561bc72fc800 2024-05-08T21:16:48.353 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:16:48 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-c[84671]: starting mds.c at 2024-05-08T21:16:48.353 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:16:48 smithi082 bash[84656]: 49995cf0a2b71fd89cea49570388e9d1a2bf9c0d838a8fa95fca34194d81fef0 2024-05-08T21:16:48.354 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:16:48 smithi082 systemd[1]: Started Ceph mds.c for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:16:48.907 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:16:49.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.051+0000 7f4e4212c700 1 -- 172.21.15.33:0/1304617415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c0ffd80 msgr2=0x7f4e3c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:49.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.051+0000 7f4e4212c700 1 --2- 172.21.15.33:0/1304617415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c0ffd80 0x7f4e3c1001a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4e240099c0 tx=0x7f4e24009cd0 comp rx=0 tx=0).stop 2024-05-08T21:16:49.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.051+0000 7f4e4212c700 1 -- 172.21.15.33:0/1304617415 shutdown_connections 2024-05-08T21:16:49.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.051+0000 7f4e4212c700 1 --2- 172.21.15.33:0/1304617415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c101910 0x7f4e3c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.051+0000 7f4e4212c700 1 --2- 172.21.15.33:0/1304617415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c0ffd80 0x7f4e3c1001a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.051+0000 7f4e4212c700 1 --2- 172.21.15.33:0/1304617415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c100f70 0x7f4e3c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.051+0000 7f4e4212c700 1 -- 172.21.15.33:0/1304617415 >> 172.21.15.33:0/1304617415 conn(0x7f4e3c0fb310 msgr2=0x7f4e3c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.051+0000 7f4e4212c700 1 -- 172.21.15.33:0/1304617415 shutdown_connections 2024-05-08T21:16:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.051+0000 7f4e4212c700 1 -- 172.21.15.33:0/1304617415 wait complete. 2024-05-08T21:16:49.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e4212c700 1 Processor -- start 2024-05-08T21:16:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e4212c700 1 -- start start 2024-05-08T21:16:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e4212c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c0ffd80 0x7f4e3c198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e4212c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c100f70 0x7f4e3c1994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e4212c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c101910 0x7f4e3c1a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e4212c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c1a0a90 con 0x7f4e3c100f70 2024-05-08T21:16:49.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e4212c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c1a0bd0 con 0x7f4e3c101910 2024-05-08T21:16:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e4212c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c1a0d60 con 0x7f4e3c0ffd80 2024-05-08T21:16:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e3affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c100f70 0x7f4e3c1994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e3affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c100f70 0x7f4e3c1994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55316/0 (socket says 172.21.15.33:55316) 2024-05-08T21:16:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e3affd700 1 -- 172.21.15.33:0/1436010385 learned_addr learned my addr 172.21.15.33:0/1436010385 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e3b7fe700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c0ffd80 0x7f4e3c198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e3bfff700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c101910 0x7f4e3c1a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e3affd700 1 -- 172.21.15.33:0/1436010385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c0ffd80 msgr2=0x7f4e3c198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e3affd700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c0ffd80 0x7f4e3c198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e3affd700 1 -- 172.21.15.33:0/1436010385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c101910 msgr2=0x7f4e3c1a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:49.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e3affd700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c101910 0x7f4e3c1a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e3affd700 1 -- 172.21.15.33:0/1436010385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e240095f0 con 0x7f4e3c100f70 2024-05-08T21:16:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.052+0000 7f4e3b7fe700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c0ffd80 0x7f4e3c198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:16:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.053+0000 7f4e3affd700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c100f70 0x7f4e3c1994f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00d950 tx=0x7f4e2c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.053+0000 7f4e38ff9700 1 -- 172.21.15.33:0/1436010385 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c009b40 con 0x7f4e3c100f70 2024-05-08T21:16:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.053+0000 7f4e38ff9700 1 -- 172.21.15.33:0/1436010385 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4e2c009ca0 con 0x7f4e3c100f70 2024-05-08T21:16:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.053+0000 7f4e4212c700 1 -- 172.21.15.33:0/1436010385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3c1a0fe0 con 0x7f4e3c100f70 2024-05-08T21:16:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.053+0000 7f4e38ff9700 1 -- 172.21.15.33:0/1436010385 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c00f620 con 0x7f4e3c100f70 2024-05-08T21:16:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.053+0000 7f4e4212c700 1 -- 172.21.15.33:0/1436010385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c1a14c0 con 0x7f4e3c100f70 2024-05-08T21:16:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.054+0000 7f4e4212c700 1 -- 172.21.15.33:0/1436010385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3c068160 con 0x7f4e3c100f70 2024-05-08T21:16:49.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.055+0000 7f4e38ff9700 1 -- 172.21.15.33:0/1436010385 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4e2c010460 con 0x7f4e3c100f70 2024-05-08T21:16:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.056+0000 7f4e38ff9700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e280a1120 0x7f4e280a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.056+0000 7f4e38ff9700 1 -- 172.21.15.33:0/1436010385 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4e2c0cc7f0 con 0x7f4e3c100f70 2024-05-08T21:16:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.056+0000 7f4e3b7fe700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e280a1120 0x7f4e280a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:49.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.056+0000 7f4e3b7fe700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e280a1120 0x7f4e280a35e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009fd0 tx=0x7f4e2401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:49.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.060+0000 7f4e38ff9700 1 -- 172.21.15.33:0/1436010385 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4e2c096380 con 0x7f4e3c100f70 2024-05-08T21:16:49.205 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:49 smithi071 ceph-mon[47074]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:49.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:49.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:49.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:49.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:49.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:49.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:49.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:49 smithi071 ceph-mon[47074]: Deploying daemon mds.h on smithi071 2024-05-08T21:16:49.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:49 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] up:boot 2024-05-08T21:16:49.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:49 smithi071 ceph-mon[47074]: fsmap 7 up:standby 2024-05-08T21:16:49.206 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:49 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T21:16:49.207 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:49 smithi033 ceph-mon[42528]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:49.207 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:49.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:49.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:49.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:49.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:49.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:49.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:49 smithi033 ceph-mon[42528]: Deploying daemon mds.h on smithi071 2024-05-08T21:16:49.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:49 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] up:boot 2024-05-08T21:16:49.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:49 smithi033 ceph-mon[42528]: fsmap 7 up:standby 2024-05-08T21:16:49.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:49 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T21:16:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.329+0000 7f4e4212c700 1 -- 172.21.15.33:0/1436010385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e3c10b690 con 0x7f4e3c100f70 2024-05-08T21:16:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.330+0000 7f4e38ff9700 1 -- 172.21.15.33:0/1436010385 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5787 (secure 0 0 0) 0x7f4e2c0999a0 con 0x7f4e3c100f70 2024-05-08T21:16:49.332 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:49.333 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2024-05-08T21:16:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.332+0000 7f4e4212c700 1 -- 172.21.15.33:0/1436010385 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e280a1120 msgr2=0x7f4e280a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.332+0000 7f4e4212c700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e280a1120 0x7f4e280a35e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009fd0 tx=0x7f4e2401a040 comp rx=0 tx=0).stop 2024-05-08T21:16:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.332+0000 7f4e4212c700 1 -- 172.21.15.33:0/1436010385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c100f70 msgr2=0x7f4e3c1994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.332+0000 7f4e4212c700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c100f70 0x7f4e3c1994f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00d950 tx=0x7f4e2c00dd10 comp rx=0 tx=0).stop 2024-05-08T21:16:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.332+0000 7f4e4212c700 1 -- 172.21.15.33:0/1436010385 shutdown_connections 2024-05-08T21:16:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.332+0000 7f4e4212c700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4e280a1120 0x7f4e280a35e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.332+0000 7f4e4212c700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e3c101910 0x7f4e3c1a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.332+0000 7f4e4212c700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e3c0ffd80 0x7f4e3c198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.332+0000 7f4e4212c700 1 --2- 172.21.15.33:0/1436010385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e3c100f70 0x7f4e3c1994f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.332+0000 7f4e4212c700 1 -- 172.21.15.33:0/1436010385 >> 172.21.15.33:0/1436010385 conn(0x7f4e3c0fb310 msgr2=0x7f4e3c0694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.332+0000 7f4e4212c700 1 -- 172.21.15.33:0/1436010385 shutdown_connections 2024-05-08T21:16:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:49.332+0000 7f4e4212c700 1 -- 172.21.15.33:0/1436010385 wait complete. 2024-05-08T21:16:49.336 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-05-08T21:16:49.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:49 smithi082 ceph-mon[47512]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:49 smithi082 ceph-mon[47512]: Deploying daemon mds.h on smithi071 2024-05-08T21:16:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:49 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] up:boot 2024-05-08T21:16:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:49 smithi082 ceph-mon[47512]: fsmap 7 up:standby 2024-05-08T21:16:49.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:49 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T21:16:49.753 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:16:49 smithi071 systemd[1]: Starting Ceph mds.h for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:16:50.138 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:16:49 smithi071 ceph-mds[83405]: -- 172.21.15.71:0/3254686734 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5636c14161a0 con 0x5636c13ed000 2024-05-08T21:16:50.138 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:16:49 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-h[83382]: starting mds.h at 2024-05-08T21:16:50.138 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:16:49 smithi071 bash[83367]: edc8e8e0bd38241c9428ed4d5e9ce9ce1b43d66cb29ab0d9751cc758bc6f0427 2024-05-08T21:16:50.138 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:16:49 smithi071 systemd[1]: Started Ceph mds.h for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:16:50.391 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:50 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1436010385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:50.391 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:50.391 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:50.391 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:50.391 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:50.391 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:50.391 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:50 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:50.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:50 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1436010385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:50.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:50.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:50.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:50.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:50.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:50.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:50 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:50.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:50 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1436010385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:50.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:50.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:50.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:50.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:50.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:50.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:50 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:51.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:16:51.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:51 smithi082 ceph-mon[47512]: Deploying daemon mds.i on smithi082 2024-05-08T21:16:51.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:51 smithi082 ceph-mon[47512]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:51.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:51 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] up:boot 2024-05-08T21:16:51.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:51 smithi082 ceph-mon[47512]: fsmap 8 up:standby 2024-05-08T21:16:51.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:51 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T21:16:51.352 INFO:journalctl@ceph.mds.i.smithi082.stdout:May 08 21:16:51 smithi082 systemd[1]: Starting Ceph mds.i for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:16:51.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:51 smithi033 ceph-mon[42528]: Deploying daemon mds.i on smithi082 2024-05-08T21:16:51.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:51 smithi033 ceph-mon[42528]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:51.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:51 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] up:boot 2024-05-08T21:16:51.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:51 smithi033 ceph-mon[42528]: fsmap 8 up:standby 2024-05-08T21:16:51.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:51 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T21:16:51.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.493+0000 7ff4418c6700 1 -- 172.21.15.33:0/3509523321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c102990 msgr2=0x7ff43c1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:51.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.493+0000 7ff4418c6700 1 --2- 172.21.15.33:0/3509523321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c102990 0x7ff43c1092b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff43400b0a0 tx=0x7ff43400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:16:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.493+0000 7ff4418c6700 1 -- 172.21.15.33:0/3509523321 shutdown_connections 2024-05-08T21:16:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.493+0000 7ff4418c6700 1 --2- 172.21.15.33:0/3509523321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c102990 0x7ff43c1092b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.493+0000 7ff4418c6700 1 --2- 172.21.15.33:0/3509523321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100e80 0x7ff43c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.493+0000 7ff4418c6700 1 --2- 172.21.15.33:0/3509523321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff43c101fd0 0x7ff43c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.493+0000 7ff4418c6700 1 -- 172.21.15.33:0/3509523321 >> 172.21.15.33:0/3509523321 conn(0x7ff43c0fc410 msgr2=0x7ff43c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:51.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.493+0000 7ff4418c6700 1 -- 172.21.15.33:0/3509523321 shutdown_connections 2024-05-08T21:16:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.493+0000 7ff4418c6700 1 -- 172.21.15.33:0/3509523321 wait complete. 2024-05-08T21:16:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.493+0000 7ff4418c6700 1 Processor -- start 2024-05-08T21:16:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff4418c6700 1 -- start start 2024-05-08T21:16:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff4418c6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100e80 0x7ff43c196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff4418c6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c101fd0 0x7ff43c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff4418c6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff43c102990 0x7ff43c19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff4418c6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c19e8d0 con 0x7ff43c101fd0 2024-05-08T21:16:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff4418c6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c19ea10 con 0x7ff43c100e80 2024-05-08T21:16:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff4418c6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c19eba0 con 0x7ff43c102990 2024-05-08T21:16:51.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff43a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c101fd0 0x7ff43c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff43a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c101fd0 0x7ff43c1972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34758/0 (socket says 172.21.15.33:34758) 2024-05-08T21:16:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff43a7fc700 1 -- 172.21.15.33:0/818423616 learned_addr learned my addr 172.21.15.33:0/818423616 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff43affd700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100e80 0x7ff43c196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff43b7fe700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff43c102990 0x7ff43c19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff43a7fc700 1 -- 172.21.15.33:0/818423616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff43c102990 msgr2=0x7ff43c19e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff43a7fc700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff43c102990 0x7ff43c19e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff43a7fc700 1 -- 172.21.15.33:0/818423616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100e80 msgr2=0x7ff43c196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff43a7fc700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100e80 0x7ff43c196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff43a7fc700 1 -- 172.21.15.33:0/818423616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff434009d00 con 0x7ff43c101fd0 2024-05-08T21:16:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.494+0000 7ff43affd700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100e80 0x7ff43c196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:16:51.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.495+0000 7ff43a7fc700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c101fd0 0x7ff43c1972e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00e9a0 tx=0x7ff42c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.495+0000 7ff4408c4700 1 -- 172.21.15.33:0/818423616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c009790 con 0x7ff43c101fd0 2024-05-08T21:16:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.495+0000 7ff4418c6700 1 -- 172.21.15.33:0/818423616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c19ee80 con 0x7ff43c101fd0 2024-05-08T21:16:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.495+0000 7ff4408c4700 1 -- 172.21.15.33:0/818423616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff42c00cbd0 con 0x7ff43c101fd0 2024-05-08T21:16:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.495+0000 7ff4408c4700 1 -- 172.21.15.33:0/818423616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c018680 con 0x7ff43c101fd0 2024-05-08T21:16:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.495+0000 7ff4418c6700 1 -- 172.21.15.33:0/818423616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c19f350 con 0x7ff43c101fd0 2024-05-08T21:16:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.496+0000 7ff4418c6700 1 -- 172.21.15.33:0/818423616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c1082a0 con 0x7ff43c101fd0 2024-05-08T21:16:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.497+0000 7ff4408c4700 1 -- 172.21.15.33:0/818423616 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff42c00cd40 con 0x7ff43c101fd0 2024-05-08T21:16:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.498+0000 7ff4408c4700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4280a11a0 0x7ff4280a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.498+0000 7ff43affd700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4280a11a0 0x7ff4280a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.498+0000 7ff4408c4700 1 -- 172.21.15.33:0/818423616 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff42c014070 con 0x7ff43c101fd0 2024-05-08T21:16:51.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.498+0000 7ff43affd700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4280a11a0 0x7ff4280a3660 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff424009cb0 tx=0x7ff424009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:51.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.503+0000 7ff4408c4700 1 -- 172.21.15.33:0/818423616 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff42c052d60 con 0x7ff43c101fd0 2024-05-08T21:16:51.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:51 smithi071 ceph-mon[47074]: Deploying daemon mds.i on smithi082 2024-05-08T21:16:51.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:51 smithi071 ceph-mon[47074]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:51.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:51 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] up:boot 2024-05-08T21:16:51.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:51 smithi071 ceph-mon[47074]: fsmap 8 up:standby 2024-05-08T21:16:51.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:51 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T21:16:51.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.768+0000 7ff4418c6700 1 -- 172.21.15.33:0/818423616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff43c068160 con 0x7ff43c101fd0 2024-05-08T21:16:51.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.768+0000 7ff4408c4700 1 -- 172.21.15.33:0/818423616 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6544 (secure 0 0 0) 0x7ff42c096230 con 0x7ff43c101fd0 2024-05-08T21:16:51.771 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:51.771 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-05-08T21:16:51.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.771+0000 7ff4418c6700 1 -- 172.21.15.33:0/818423616 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4280a11a0 msgr2=0x7ff4280a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:51.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.771+0000 7ff4418c6700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4280a11a0 0x7ff4280a3660 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff424009cb0 tx=0x7ff424009260 comp rx=0 tx=0).stop 2024-05-08T21:16:51.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.771+0000 7ff4418c6700 1 -- 172.21.15.33:0/818423616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c101fd0 msgr2=0x7ff43c1972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:51.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.771+0000 7ff4418c6700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c101fd0 0x7ff43c1972e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00e9a0 tx=0x7ff42c00ecb0 comp rx=0 tx=0).stop 2024-05-08T21:16:51.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.771+0000 7ff4418c6700 1 -- 172.21.15.33:0/818423616 shutdown_connections 2024-05-08T21:16:51.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.771+0000 7ff4418c6700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff43c102990 0x7ff43c19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:51.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.771+0000 7ff4418c6700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4280a11a0 0x7ff4280a3660 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:51.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.771+0000 7ff4418c6700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100e80 0x7ff43c196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:51.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.771+0000 7ff4418c6700 1 --2- 172.21.15.33:0/818423616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff43c101fd0 0x7ff43c1972e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:51.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.771+0000 7ff4418c6700 1 -- 172.21.15.33:0/818423616 >> 172.21.15.33:0/818423616 conn(0x7ff43c0fc410 msgr2=0x7ff43c1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:51.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.771+0000 7ff4418c6700 1 -- 172.21.15.33:0/818423616 shutdown_connections 2024-05-08T21:16:51.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:51.771+0000 7ff4418c6700 1 -- 172.21.15.33:0/818423616 wait complete. 2024-05-08T21:16:51.775 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-05-08T21:16:52.103 INFO:journalctl@ceph.mds.i.smithi082.stdout:May 08 21:16:51 smithi082 ceph-mds[85038]: -- 172.21.15.82:0/29892057 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5593cb8821a0 con 0x5593cb859000 2024-05-08T21:16:52.103 INFO:journalctl@ceph.mds.i.smithi082.stdout:May 08 21:16:51 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-i[85015]: starting mds.i at 2024-05-08T21:16:52.103 INFO:journalctl@ceph.mds.i.smithi082.stdout:May 08 21:16:51 smithi082 bash[85000]: c9f046ccb079f1ff16b7ffc5b5164df1930149de3098380a1d3d1f5232236112 2024-05-08T21:16:52.104 INFO:journalctl@ceph.mds.i.smithi082.stdout:May 08 21:16:51 smithi082 systemd[1]: Started Ceph mds.i for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:16:52.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:52 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/818423616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:52.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:52.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:52.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:52.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:52.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:52.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:52 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:52.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:52 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/818423616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:52.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:52.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:52.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:52.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:52.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:52.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:52 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:52.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:52 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/818423616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:52.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:52.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:52.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:52.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:52.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:52.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:52 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:53.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:53 smithi033 ceph-mon[42528]: Deploying daemon mds.g on smithi033 2024-05-08T21:16:53.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:53 smithi033 ceph-mon[42528]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:53.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:53 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.82:6838/1299124477,v1:172.21.15.82:6839/1299124477] up:boot 2024-05-08T21:16:53.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:53 smithi033 ceph-mon[42528]: fsmap 9 up:standby 2024-05-08T21:16:53.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:53 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T21:16:53.459 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 08 21:16:53 smithi033 systemd[1]: Starting Ceph mds.g for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:16:53.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:53 smithi071 ceph-mon[47074]: Deploying daemon mds.g on smithi033 2024-05-08T21:16:53.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:53 smithi071 ceph-mon[47074]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:53.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:53 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.82:6838/1299124477,v1:172.21.15.82:6839/1299124477] up:boot 2024-05-08T21:16:53.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:53 smithi071 ceph-mon[47074]: fsmap 9 up:standby 2024-05-08T21:16:53.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:53 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T21:16:53.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:53 smithi082 ceph-mon[47512]: Deploying daemon mds.g on smithi033 2024-05-08T21:16:53.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:53 smithi082 ceph-mon[47512]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:53.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:53 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.82:6838/1299124477,v1:172.21.15.82:6839/1299124477] up:boot 2024-05-08T21:16:53.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:53 smithi082 ceph-mon[47512]: fsmap 9 up:standby 2024-05-08T21:16:53.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:53 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T21:16:53.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:16:53.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.934+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2211302616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34106c20 msgr2=0x7f9c34109010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.934+0000 7f9c3aab2700 1 --2- 172.21.15.33:0/2211302616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34106c20 0x7f9c34109010 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c00b0a0 tx=0x7f9c2c00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:16:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.934+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2211302616 shutdown_connections 2024-05-08T21:16:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.934+0000 7f9c3aab2700 1 --2- 172.21.15.33:0/2211302616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c34106c20 0x7f9c34109010 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.934+0000 7f9c3aab2700 1 --2- 172.21.15.33:0/2211302616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c341019c0 0x7f9c34103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.934+0000 7f9c3aab2700 1 --2- 172.21.15.33:0/2211302616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c341042f0 0x7f9c341066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.934+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2211302616 >> 172.21.15.33:0/2211302616 conn(0x7f9c340fb310 msgr2=0x7f9c340fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.934+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2211302616 shutdown_connections 2024-05-08T21:16:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.935+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2211302616 wait complete. 2024-05-08T21:16:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.935+0000 7f9c3aab2700 1 Processor -- start 2024-05-08T21:16:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.935+0000 7f9c3aab2700 1 -- start start 2024-05-08T21:16:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.935+0000 7f9c3aab2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c341019c0 0x7f9c341012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.935+0000 7f9c3aab2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c341042f0 0x7f9c340ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.935+0000 7f9c3aab2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c34106c20 0x7f9c340ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.935+0000 7f9c3aab2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c34100650 con 0x7f9c341042f0 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.935+0000 7f9c3aab2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c341007c0 con 0x7f9c341019c0 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.935+0000 7f9c3aab2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c341a0e80 con 0x7f9c34106c20 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.935+0000 7f9c33fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c341042f0 0x7f9c340ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.935+0000 7f9c33fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c341042f0 0x7f9c340ff970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34776/0 (socket says 172.21.15.33:34776) 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.935+0000 7f9c33fff700 1 -- 172.21.15.33:0/2313584245 learned_addr learned my addr 172.21.15.33:0/2313584245 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c3884e700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c341019c0 0x7f9c341012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c3904f700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c34106c20 0x7f9c340ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c33fff700 1 -- 172.21.15.33:0/2313584245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c34106c20 msgr2=0x7f9c340ffeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c33fff700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c34106c20 0x7f9c340ffeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c33fff700 1 -- 172.21.15.33:0/2313584245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c341019c0 msgr2=0x7f9c341012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c33fff700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c341019c0 0x7f9c341012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c33fff700 1 -- 172.21.15.33:0/2313584245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c009d00 con 0x7f9c341042f0 2024-05-08T21:16:53.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c3904f700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c34106c20 0x7f9c340ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:16:53.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c33fff700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c341042f0 0x7f9c340ff970 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000d950 tx=0x7f9c2000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:53.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c31ffb700 1 -- 172.21.15.33:0/2313584245 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c200098e0 con 0x7f9c341042f0 2024-05-08T21:16:53.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2313584245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c341a1110 con 0x7f9c341042f0 2024-05-08T21:16:53.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c31ffb700 1 -- 172.21.15.33:0/2313584245 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c20010460 con 0x7f9c341042f0 2024-05-08T21:16:53.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.936+0000 7f9c31ffb700 1 -- 172.21.15.33:0/2313584245 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2000f620 con 0x7f9c341042f0 2024-05-08T21:16:53.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.937+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2313584245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c341a1670 con 0x7f9c341042f0 2024-05-08T21:16:53.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.938+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2313584245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c340fce90 con 0x7f9c341042f0 2024-05-08T21:16:53.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.939+0000 7f9c31ffb700 1 -- 172.21.15.33:0/2313584245 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9c200105d0 con 0x7f9c341042f0 2024-05-08T21:16:53.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.939+0000 7f9c31ffb700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9c240a1120 0x7f9c240a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:53.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.939+0000 7f9c31ffb700 1 -- 172.21.15.33:0/2313584245 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9c200cc2a0 con 0x7f9c341042f0 2024-05-08T21:16:53.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.940+0000 7f9c3884e700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9c240a1120 0x7f9c240a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:53.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.940+0000 7f9c3884e700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9c240a1120 0x7f9c240a35e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9c28005950 tx=0x7f9c2800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:53.945+0000 7f9c31ffb700 1 -- 172.21.15.33:0/2313584245 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9c20095ed0 con 0x7f9c341042f0 2024-05-08T21:16:53.958 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 08 21:16:53 smithi033 ceph-mds[93072]: -- 172.21.15.33:0/3099459504 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560256c161a0 con 0x560256bed000 2024-05-08T21:16:53.958 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 08 21:16:53 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-g[93049]: starting mds.g at 2024-05-08T21:16:53.958 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 08 21:16:53 smithi033 bash[93034]: 05246cfe86e68be8a9aa7849c0e56368e6b8f431439921ab8e3304b6b380521b 2024-05-08T21:16:53.958 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 08 21:16:53 smithi033 systemd[1]: Started Ceph mds.g for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:16:54.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.223+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2313584245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c34067d10 con 0x7f9c341042f0 2024-05-08T21:16:54.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.223+0000 7f9c31ffb700 1 -- 172.21.15.33:0/2313584245 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8061 (secure 0 0 0) 0x7f9c200994f0 con 0x7f9c341042f0 2024-05-08T21:16:54.226 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:54.226 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-05-08T21:16:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.226+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2313584245 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9c240a1120 msgr2=0x7f9c240a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.226+0000 7f9c3aab2700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9c240a1120 0x7f9c240a35e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9c28005950 tx=0x7f9c2800b410 comp rx=0 tx=0).stop 2024-05-08T21:16:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.226+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2313584245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c341042f0 msgr2=0x7f9c340ff970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.226+0000 7f9c3aab2700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c341042f0 0x7f9c340ff970 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9c2000d950 tx=0x7f9c2000dc60 comp rx=0 tx=0).stop 2024-05-08T21:16:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.226+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2313584245 shutdown_connections 2024-05-08T21:16:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.226+0000 7f9c3aab2700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c34106c20 0x7f9c340ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.226+0000 7f9c3aab2700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9c240a1120 0x7f9c240a35e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.226+0000 7f9c3aab2700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c341019c0 0x7f9c341012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.226+0000 7f9c3aab2700 1 --2- 172.21.15.33:0/2313584245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c341042f0 0x7f9c340ff970 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.226+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2313584245 >> 172.21.15.33:0/2313584245 conn(0x7f9c340fb310 msgr2=0x7f9c34102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.226+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2313584245 shutdown_connections 2024-05-08T21:16:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:54.226+0000 7f9c3aab2700 1 -- 172.21.15.33:0/2313584245 wait complete. 2024-05-08T21:16:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 11 2024-05-08T21:16:54.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:54 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:54.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:54 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:54.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:54 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:54.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:54 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:54.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:54 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:54.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:54 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:54.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:54 smithi033 ceph-mon[42528]: Deploying daemon mds.a on smithi033 2024-05-08T21:16:54.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:54 smithi033 ceph-mon[42528]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:54.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:54 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.33:6838/2620722111,v1:172.21.15.33:6839/2620722111] up:boot 2024-05-08T21:16:54.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:54 smithi033 ceph-mon[42528]: fsmap 10 up:standby 2024-05-08T21:16:54.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:54 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T21:16:54.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:54 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2313584245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:55.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:54 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:55.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:54 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:55.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:54 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:55.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:54 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:55.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:54 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:55.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:54 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:55.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:54 smithi071 ceph-mon[47074]: Deploying daemon mds.a on smithi033 2024-05-08T21:16:55.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:54 smithi071 ceph-mon[47074]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:55.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:54 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.33:6838/2620722111,v1:172.21.15.33:6839/2620722111] up:boot 2024-05-08T21:16:55.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:54 smithi071 ceph-mon[47074]: fsmap 10 up:standby 2024-05-08T21:16:55.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:54 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T21:16:55.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:54 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2313584245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:55.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:54 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:55.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:54 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:55.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:54 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:55.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:54 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:55.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:54 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:55.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:54 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:55.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:54 smithi082 ceph-mon[47512]: Deploying daemon mds.a on smithi033 2024-05-08T21:16:55.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:54 smithi082 ceph-mon[47512]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:55.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:54 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.33:6838/2620722111,v1:172.21.15.33:6839/2620722111] up:boot 2024-05-08T21:16:55.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:54 smithi082 ceph-mon[47512]: fsmap 10 up:standby 2024-05-08T21:16:55.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:54 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T21:16:55.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:54 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2313584245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:55.169 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 08 21:16:55 smithi033 systemd[1]: Starting Ceph mds.a for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:16:55.958 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 08 21:16:55 smithi033 ceph-mds[93457]: -- 172.21.15.33:0/3771227205 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b54943e1a0 con 0x55b549415000 2024-05-08T21:16:55.958 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 08 21:16:55 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-a[93434]: starting mds.a at 2024-05-08T21:16:55.958 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 08 21:16:55 smithi033 bash[93419]: 04dd9f55aa25f832a6ef6f8c10e716371394b89f13088d99f16c9aaa95b51bd2 2024-05-08T21:16:55.958 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 08 21:16:55 smithi033 systemd[1]: Started Ceph mds.a for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:16:56.245 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:16:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.396+0000 7f67735a3700 1 -- 172.21.15.33:0/1633941250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f676c0ffd80 msgr2=0x7f676c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.396+0000 7f67735a3700 1 --2- 172.21.15.33:0/1633941250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f676c0ffd80 0x7f676c1001a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f67680099c0 tx=0x7f6768009cd0 comp rx=0 tx=0).stop 2024-05-08T21:16:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.397+0000 7f67735a3700 1 -- 172.21.15.33:0/1633941250 shutdown_connections 2024-05-08T21:16:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.397+0000 7f67735a3700 1 --2- 172.21.15.33:0/1633941250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c101910 0x7f676c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.397+0000 7f67735a3700 1 --2- 172.21.15.33:0/1633941250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f676c0ffd80 0x7f676c1001a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.397+0000 7f67735a3700 1 --2- 172.21.15.33:0/1633941250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c100f70 0x7f676c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.397+0000 7f67735a3700 1 -- 172.21.15.33:0/1633941250 >> 172.21.15.33:0/1633941250 conn(0x7f676c0fb310 msgr2=0x7f676c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.397+0000 7f67735a3700 1 -- 172.21.15.33:0/1633941250 shutdown_connections 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.397+0000 7f67735a3700 1 -- 172.21.15.33:0/1633941250 wait complete. 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.397+0000 7f67735a3700 1 Processor -- start 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.397+0000 7f67735a3700 1 -- start start 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f67735a3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f676c0ffd80 0x7f676c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f67735a3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c100f70 0x7f676c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f67735a3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c101910 0x7f676c1a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f67735a3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c1a0a60 con 0x7f676c0ffd80 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f67735a3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c1a0ba0 con 0x7f676c101910 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f67735a3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c1a0d30 con 0x7f676c100f70 2024-05-08T21:16:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f677133f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f676c0ffd80 0x7f676c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f677133f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f676c0ffd80 0x7f676c198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34808/0 (socket says 172.21.15.33:34808) 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f677133f700 1 -- 172.21.15.33:0/306649273 learned_addr learned my addr 172.21.15.33:0/306649273 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f6771b40700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c101910 0x7f676c1a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f6770b3e700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c100f70 0x7f676c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f677133f700 1 -- 172.21.15.33:0/306649273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c100f70 msgr2=0x7f676c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f677133f700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c100f70 0x7f676c199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f677133f700 1 -- 172.21.15.33:0/306649273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c101910 msgr2=0x7f676c1a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f677133f700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c101910 0x7f676c1a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f677133f700 1 -- 172.21.15.33:0/306649273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67680095f0 con 0x7f676c0ffd80 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f677133f700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f676c0ffd80 0x7f676c198f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6768009990 tx=0x7f676800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f675e7fc700 1 -- 172.21.15.33:0/306649273 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676801d070 con 0x7f676c0ffd80 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f675e7fc700 1 -- 172.21.15.33:0/306649273 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f676800bd10 con 0x7f676c0ffd80 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f67735a3700 1 -- 172.21.15.33:0/306649273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c1a0fb0 con 0x7f676c0ffd80 2024-05-08T21:16:56.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f675e7fc700 1 -- 172.21.15.33:0/306649273 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768017650 con 0x7f676c0ffd80 2024-05-08T21:16:56.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.398+0000 7f67735a3700 1 -- 172.21.15.33:0/306649273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c1a1320 con 0x7f676c0ffd80 2024-05-08T21:16:56.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.399+0000 7f67735a3700 1 -- 172.21.15.33:0/306649273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c067d80 con 0x7f676c0ffd80 2024-05-08T21:16:56.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.401+0000 7f675e7fc700 1 -- 172.21.15.33:0/306649273 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6768022470 con 0x7f676c0ffd80 2024-05-08T21:16:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.401+0000 7f675e7fc700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f67580a1120 0x7f67580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.401+0000 7f6770b3e700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f67580a1120 0x7f67580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.401+0000 7f675e7fc700 1 -- 172.21.15.33:0/306649273 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f67680cd990 con 0x7f676c0ffd80 2024-05-08T21:16:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.402+0000 7f6770b3e700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f67580a1120 0x7f67580a35e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6760009bf0 tx=0x7f6760009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:56.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.405+0000 7f675e7fc700 1 -- 172.21.15.33:0/306649273 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6768097650 con 0x7f676c0ffd80 2024-05-08T21:16:56.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.670+0000 7f67735a3700 1 -- 172.21.15.33:0/306649273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f676c067d10 con 0x7f676c0ffd80 2024-05-08T21:16:56.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.671+0000 7f675e7fc700 1 -- 172.21.15.33:0/306649273 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8819 (secure 0 0 0) 0x7f676809ac70 con 0x7f676c0ffd80 2024-05-08T21:16:56.673 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:56.673 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-05-08T21:16:56.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.673+0000 7f67735a3700 1 -- 172.21.15.33:0/306649273 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f67580a1120 msgr2=0x7f67580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:56.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.673+0000 7f67735a3700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f67580a1120 0x7f67580a35e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6760009bf0 tx=0x7f6760009260 comp rx=0 tx=0).stop 2024-05-08T21:16:56.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.673+0000 7f67735a3700 1 -- 172.21.15.33:0/306649273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f676c0ffd80 msgr2=0x7f676c198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:56.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.673+0000 7f67735a3700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f676c0ffd80 0x7f676c198f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6768009990 tx=0x7f676800fa00 comp rx=0 tx=0).stop 2024-05-08T21:16:56.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.673+0000 7f67735a3700 1 -- 172.21.15.33:0/306649273 shutdown_connections 2024-05-08T21:16:56.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.673+0000 7f67735a3700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f67580a1120 0x7f67580a35e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:56.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.673+0000 7f67735a3700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c101910 0x7f676c1a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:56.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.673+0000 7f67735a3700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f676c0ffd80 0x7f676c198f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:56.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.673+0000 7f67735a3700 1 --2- 172.21.15.33:0/306649273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c100f70 0x7f676c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:56.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.673+0000 7f67735a3700 1 -- 172.21.15.33:0/306649273 >> 172.21.15.33:0/306649273 conn(0x7f676c0fb310 msgr2=0x7f676c106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:56.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.673+0000 7f67735a3700 1 -- 172.21.15.33:0/306649273 shutdown_connections 2024-05-08T21:16:56.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:56.673+0000 7f67735a3700 1 -- 172.21.15.33:0/306649273 wait complete. 2024-05-08T21:16:56.677 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 12 2024-05-08T21:16:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:56.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:56.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:56 smithi033 ceph-mon[42528]: Deploying daemon mds.b on smithi071 2024-05-08T21:16:56.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:56 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.33:6840/2132539478,v1:172.21.15.33:6841/2132539478] up:boot 2024-05-08T21:16:56.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:56 smithi033 ceph-mon[42528]: fsmap 11 up:standby 2024-05-08T21:16:56.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T21:16:56.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:56 smithi033 ceph-mon[42528]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:56.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:56 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/306649273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:57.010 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:16:57 smithi071 systemd[1]: Starting Ceph mds.b for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:16:57.010 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:57.010 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:57.010 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:57.010 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:57.010 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:57.010 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:57.011 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:56 smithi071 ceph-mon[47074]: Deploying daemon mds.b on smithi071 2024-05-08T21:16:57.011 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:56 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.33:6840/2132539478,v1:172.21.15.33:6841/2132539478] up:boot 2024-05-08T21:16:57.011 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:56 smithi071 ceph-mon[47074]: fsmap 11 up:standby 2024-05-08T21:16:57.011 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T21:16:57.011 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:56 smithi071 ceph-mon[47074]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:57.011 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:56 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/306649273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:57.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:57.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:57.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:57.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T21:16:57.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T21:16:57.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:57.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:56 smithi082 ceph-mon[47512]: Deploying daemon mds.b on smithi071 2024-05-08T21:16:57.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:56 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.33:6840/2132539478,v1:172.21.15.33:6841/2132539478] up:boot 2024-05-08T21:16:57.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:56 smithi082 ceph-mon[47512]: fsmap 11 up:standby 2024-05-08T21:16:57.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T21:16:57.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:56 smithi082 ceph-mon[47512]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:57.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:56 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/306649273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:57.821 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:16:57 smithi071 ceph-mds[83747]: -- 172.21.15.71:0/4277869614 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561f6d9dc1a0 con 0x561f6d9b3000 2024-05-08T21:16:57.821 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:16:57 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-b[83724]: starting mds.b at 2024-05-08T21:16:57.821 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:16:57 smithi071 bash[83709]: d00127dbd5c056b8a16c27c951ff89300d8af7262b0cbfd724ab22d5df85d026 2024-05-08T21:16:57.821 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:16:57 smithi071 systemd[1]: Started Ceph mds.b for 53ae3a5a-0d7f-11ef-bc97-c7b262605968. 2024-05-08T21:16:58.690 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:16:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:58 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] up:boot 2024-05-08T21:16:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:58 smithi033 ceph-mon[42528]: fsmap 12 up:standby 2024-05-08T21:16:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T21:16:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:58 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] up:boot 2024-05-08T21:16:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:58 smithi071 ceph-mon[47074]: fsmap 12 up:standby 2024-05-08T21:16:58.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T21:16:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.849+0000 7ff7c8af9700 1 -- 172.21.15.33:0/241916751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7c4106c00 msgr2=0x7ff7c4108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.849+0000 7ff7c8af9700 1 --2- 172.21.15.33:0/241916751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7c4106c00 0x7ff7c4108ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00b0a0 tx=0x7ff7bc00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:16:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.849+0000 7ff7c8af9700 1 -- 172.21.15.33:0/241916751 shutdown_connections 2024-05-08T21:16:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.849+0000 7ff7c8af9700 1 --2- 172.21.15.33:0/241916751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7c4106c00 0x7ff7c4108ff0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.849+0000 7ff7c8af9700 1 --2- 172.21.15.33:0/241916751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c41019a0 0x7ff7c4103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.849+0000 7ff7c8af9700 1 --2- 172.21.15.33:0/241916751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c41042d0 0x7ff7c41066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.849+0000 7ff7c8af9700 1 -- 172.21.15.33:0/241916751 >> 172.21.15.33:0/241916751 conn(0x7ff7c40fb310 msgr2=0x7ff7c40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.849+0000 7ff7c8af9700 1 -- 172.21.15.33:0/241916751 shutdown_connections 2024-05-08T21:16:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.849+0000 7ff7c8af9700 1 -- 172.21.15.33:0/241916751 wait complete. 2024-05-08T21:16:58.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:58.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:16:58.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:58 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] up:boot 2024-05-08T21:16:58.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:58 smithi082 ceph-mon[47512]: fsmap 12 up:standby 2024-05-08T21:16:58.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T21:16:58.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.850+0000 7ff7c8af9700 1 Processor -- start 2024-05-08T21:16:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c8af9700 1 -- start start 2024-05-08T21:16:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c8af9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c41019a0 0x7ff7c41012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c8af9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c41042d0 0x7ff7c40ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c8af9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7c4106c00 0x7ff7c40ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c8af9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c41a6fb0 con 0x7ff7c4106c00 2024-05-08T21:16:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c8af9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c41003d0 con 0x7ff7c41042d0 2024-05-08T21:16:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c8af9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c4100590 con 0x7ff7c41019a0 2024-05-08T21:16:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c41019a0 0x7ff7c41012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7c4106c00 0x7ff7c40ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c1d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c41042d0 0x7ff7c40ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c41019a0 0x7ff7c41012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:58264/0 (socket says 172.21.15.33:58264) 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7c4106c00 0x7ff7c40ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34814/0 (socket says 172.21.15.33:34814) 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c259c700 1 -- 172.21.15.33:0/1730236208 learned_addr learned my addr 172.21.15.33:0/1730236208 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c259c700 1 -- 172.21.15.33:0/1730236208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c41042d0 msgr2=0x7ff7c40ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c259c700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c41042d0 0x7ff7c40ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c259c700 1 -- 172.21.15.33:0/1730236208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7c4106c00 msgr2=0x7ff7c40ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.851+0000 7ff7c259c700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7c4106c00 0x7ff7c40ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.852+0000 7ff7c259c700 1 -- 172.21.15.33:0/1730236208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b8009520 con 0x7ff7c41019a0 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.852+0000 7ff7c2d9d700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7c4106c00 0x7ff7c40ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.852+0000 7ff7c1d9b700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c41042d0 0x7ff7c40ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.852+0000 7ff7c259c700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c41019a0 0x7ff7c41012c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400e9a0 tx=0x7ff7b400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.852+0000 7ff7b37fe700 1 -- 172.21.15.33:0/1730236208 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4009790 con 0x7ff7c41019a0 2024-05-08T21:16:58.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.852+0000 7ff7b37fe700 1 -- 172.21.15.33:0/1730236208 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff7b4010430 con 0x7ff7c41019a0 2024-05-08T21:16:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.852+0000 7ff7c8af9700 1 -- 172.21.15.33:0/1730236208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc009d00 con 0x7ff7c41019a0 2024-05-08T21:16:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.852+0000 7ff7b37fe700 1 -- 172.21.15.33:0/1730236208 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b400cea0 con 0x7ff7c41019a0 2024-05-08T21:16:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.852+0000 7ff7c8af9700 1 -- 172.21.15.33:0/1730236208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c4071910 con 0x7ff7c41019a0 2024-05-08T21:16:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.854+0000 7ff7c8af9700 1 -- 172.21.15.33:0/1730236208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c410ce00 con 0x7ff7c41019a0 2024-05-08T21:16:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.855+0000 7ff7b37fe700 1 -- 172.21.15.33:0/1730236208 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff7b40105a0 con 0x7ff7c41019a0 2024-05-08T21:16:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.856+0000 7ff7b37fe700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff7ac0a11a0 0x7ff7ac0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.856+0000 7ff7c1d9b700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff7ac0a11a0 0x7ff7ac0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.856+0000 7ff7b37fe700 1 -- 172.21.15.33:0/1730236208 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff7b4014070 con 0x7ff7c41019a0 2024-05-08T21:16:58.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.857+0000 7ff7c1d9b700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff7ac0a11a0 0x7ff7ac0a3660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009fd0 tx=0x7ff7b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:58.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:58.864+0000 7ff7b37fe700 1 -- 172.21.15.33:0/1730236208 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff7b4052d60 con 0x7ff7c41019a0 2024-05-08T21:16:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.146+0000 7ff7c8af9700 1 -- 172.21.15.33:0/1730236208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7c4072f00 con 0x7ff7c41019a0 2024-05-08T21:16:59.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.147+0000 7ff7b37fe700 1 -- 172.21.15.33:0/1730236208 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9577 (secure 0 0 0) 0x7ff7b40961d0 con 0x7ff7c41019a0 2024-05-08T21:16:59.150 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:59.150 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24563,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-05-08T21:16:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.150+0000 7ff7c8af9700 1 -- 172.21.15.33:0/1730236208 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff7ac0a11a0 msgr2=0x7ff7ac0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.150+0000 7ff7c8af9700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff7ac0a11a0 0x7ff7ac0a3660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009fd0 tx=0x7ff7b8009260 comp rx=0 tx=0).stop 2024-05-08T21:16:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.150+0000 7ff7c8af9700 1 -- 172.21.15.33:0/1730236208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c41019a0 msgr2=0x7ff7c41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.150+0000 7ff7c8af9700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c41019a0 0x7ff7c41012c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400e9a0 tx=0x7ff7b400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.150+0000 7ff7c8af9700 1 -- 172.21.15.33:0/1730236208 shutdown_connections 2024-05-08T21:16:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.150+0000 7ff7c8af9700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7c4106c00 0x7ff7c40ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.150+0000 7ff7c8af9700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7c41019a0 0x7ff7c41012c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.150+0000 7ff7c8af9700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff7ac0a11a0 0x7ff7ac0a3660 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.150+0000 7ff7c8af9700 1 --2- 172.21.15.33:0/1730236208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c41042d0 0x7ff7c40ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.150+0000 7ff7c8af9700 1 -- 172.21.15.33:0/1730236208 >> 172.21.15.33:0/1730236208 conn(0x7ff7c40fb310 msgr2=0x7ff7c41023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.150+0000 7ff7c8af9700 1 -- 172.21.15.33:0/1730236208 shutdown_connections 2024-05-08T21:16:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.150+0000 7ff7c8af9700 1 -- 172.21.15.33:0/1730236208 wait complete. 2024-05-08T21:16:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 13 2024-05-08T21:16:59.167 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:16:59.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.345+0000 7fa3fe150700 1 -- 172.21.15.33:0/1297257346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f8106b80 msgr2=0x7fa3f8108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:59.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.345+0000 7fa3fe150700 1 --2- 172.21.15.33:0/1297257346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f8106b80 0x7fa3f8108ff0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000b0a0 tx=0x7fa3f000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.346+0000 7fa3fe150700 1 -- 172.21.15.33:0/1297257346 shutdown_connections 2024-05-08T21:16:59.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.346+0000 7fa3fe150700 1 --2- 172.21.15.33:0/1297257346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f8106b80 0x7fa3f8108ff0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.346+0000 7fa3fe150700 1 --2- 172.21.15.33:0/1297257346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f8101920 0x7fa3f8103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.346+0000 7fa3fe150700 1 --2- 172.21.15.33:0/1297257346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f8104250 0x7fa3f8106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.346+0000 7fa3fe150700 1 -- 172.21.15.33:0/1297257346 >> 172.21.15.33:0/1297257346 conn(0x7fa3f80fb310 msgr2=0x7fa3f80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:59.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.346+0000 7fa3fe150700 1 -- 172.21.15.33:0/1297257346 shutdown_connections 2024-05-08T21:16:59.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.346+0000 7fa3fe150700 1 -- 172.21.15.33:0/1297257346 wait complete. 2024-05-08T21:16:59.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.346+0000 7fa3fe150700 1 Processor -- start 2024-05-08T21:16:59.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3fe150700 1 -- start start 2024-05-08T21:16:59.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3fe150700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f8101920 0x7fa3f8196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:59.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3fe150700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f8104250 0x7fa3f8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:59.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3fe150700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f8106b80 0x7fa3f819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:59.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3fe150700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f819e8f0 con 0x7fa3f8106b80 2024-05-08T21:16:59.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3fe150700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f819ea60 con 0x7fa3f8104250 2024-05-08T21:16:59.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3fe150700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f819ec20 con 0x7fa3f8101920 2024-05-08T21:16:59.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f8106b80 0x7fa3f819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:59.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f8106b80 0x7fa3f819e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34830/0 (socket says 172.21.15.33:34830) 2024-05-08T21:16:59.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3f7fff700 1 -- 172.21.15.33:0/153189695 learned_addr learned my addr 172.21.15.33:0/153189695 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:59.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3f6ffd700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f8104250 0x7fa3f8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:59.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3f77fe700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f8101920 0x7fa3f8196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:59.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3f7fff700 1 -- 172.21.15.33:0/153189695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f8101920 msgr2=0x7fa3f8196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:59.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3f7fff700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f8101920 0x7fa3f8196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3f7fff700 1 -- 172.21.15.33:0/153189695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f8104250 msgr2=0x7fa3f8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:59.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3f7fff700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f8104250 0x7fa3f8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3f7fff700 1 -- 172.21.15.33:0/153189695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0009d00 con 0x7fa3f8106b80 2024-05-08T21:16:59.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.347+0000 7fa3f77fe700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f8101920 0x7fa3f8196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:16:59.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.348+0000 7fa3f7fff700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f8106b80 0x7fa3f819e320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0007870 tx=0x7fa3f00078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:59.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.348+0000 7fa3f4ff9700 1 -- 172.21.15.33:0/153189695 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000e040 con 0x7fa3f8106b80 2024-05-08T21:16:59.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.348+0000 7fa3fe150700 1 -- 172.21.15.33:0/153189695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f819eea0 con 0x7fa3f8106b80 2024-05-08T21:16:59.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.348+0000 7fa3f4ff9700 1 -- 172.21.15.33:0/153189695 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3f0004c50 con 0x7fa3f8106b80 2024-05-08T21:16:59.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.348+0000 7fa3f4ff9700 1 -- 172.21.15.33:0/153189695 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00052a0 con 0x7fa3f8106b80 2024-05-08T21:16:59.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.348+0000 7fa3fe150700 1 -- 172.21.15.33:0/153189695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f819f2c0 con 0x7fa3f8106b80 2024-05-08T21:16:59.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.349+0000 7fa3fe150700 1 -- 172.21.15.33:0/153189695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f80fce90 con 0x7fa3f8106b80 2024-05-08T21:16:59.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.350+0000 7fa3f4ff9700 1 -- 172.21.15.33:0/153189695 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa3f001d470 con 0x7fa3f8106b80 2024-05-08T21:16:59.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.351+0000 7fa3f4ff9700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa3e00a1120 0x7fa3e00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:59.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.351+0000 7fa3f4ff9700 1 -- 172.21.15.33:0/153189695 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa3f00cde30 con 0x7fa3f8106b80 2024-05-08T21:16:59.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.351+0000 7fa3f77fe700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa3e00a1120 0x7fa3e00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:59.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.352+0000 7fa3f77fe700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa3e00a1120 0x7fa3e00a35e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8198280 tx=0x7fa3e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:59.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.355+0000 7fa3f4ff9700 1 -- 172.21.15.33:0/153189695 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa3f0097af0 con 0x7fa3f8106b80 2024-05-08T21:16:59.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.580+0000 7fa3fe150700 1 -- 172.21.15.33:0/153189695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3f8067480 con 0x7fa3f8106b80 2024-05-08T21:16:59.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.581+0000 7fa3f4ff9700 1 -- 172.21.15.33:0/153189695 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+18364 (secure 0 0 0) 0x7fa3f009b110 con 0x7fa3f8106b80 2024-05-08T21:16:59.585 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:16:59.585 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":77,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:16:29.210489+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:16:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.584+0000 7fa3fe150700 1 -- 172.21.15.33:0/153189695 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa3e00a1120 msgr2=0x7fa3e00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.584+0000 7fa3fe150700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa3e00a1120 0x7fa3e00a35e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8198280 tx=0x7fa3e8009260 comp rx=0 tx=0).stop 2024-05-08T21:16:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.584+0000 7fa3fe150700 1 -- 172.21.15.33:0/153189695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f8106b80 msgr2=0x7fa3f819e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.584+0000 7fa3fe150700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f8106b80 0x7fa3f819e320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0007870 tx=0x7fa3f00078a0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.584+0000 7fa3fe150700 1 -- 172.21.15.33:0/153189695 shutdown_connections 2024-05-08T21:16:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.584+0000 7fa3fe150700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3f8106b80 0x7fa3f819e320 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.584+0000 7fa3fe150700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3f8101920 0x7fa3f8196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.584+0000 7fa3fe150700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa3e00a1120 0x7fa3e00a35e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.584+0000 7fa3fe150700 1 --2- 172.21.15.33:0/153189695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f8104250 0x7fa3f8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.584+0000 7fa3fe150700 1 -- 172.21.15.33:0/153189695 >> 172.21.15.33:0/153189695 conn(0x7fa3f80fb310 msgr2=0x7fa3f8102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.584+0000 7fa3fe150700 1 -- 172.21.15.33:0/153189695 shutdown_connections 2024-05-08T21:16:59.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.584+0000 7fa3fe150700 1 -- 172.21.15.33:0/153189695 wait complete. 2024-05-08T21:16:59.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T21:16:59.602 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:16:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:59 smithi033 ceph-mon[42528]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:59 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1730236208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:16:59 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.748+0000 7fb792e8c700 1 -- 172.21.15.33:0/1410321881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78c100e70 msgr2=0x7fb78c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:59.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.748+0000 7fb792e8c700 1 --2- 172.21.15.33:0/1410321881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78c100e70 0x7fb78c101290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb780009990 tx=0x7fb780009ca0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.748+0000 7fb792e8c700 1 -- 172.21.15.33:0/1410321881 shutdown_connections 2024-05-08T21:16:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.748+0000 7fb792e8c700 1 --2- 172.21.15.33:0/1410321881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb78c102980 0x7fb78c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.748+0000 7fb792e8c700 1 --2- 172.21.15.33:0/1410321881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78c100e70 0x7fb78c101290 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.748+0000 7fb792e8c700 1 --2- 172.21.15.33:0/1410321881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb78c101fc0 0x7fb78c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.748+0000 7fb792e8c700 1 -- 172.21.15.33:0/1410321881 >> 172.21.15.33:0/1410321881 conn(0x7fb78c0fc400 msgr2=0x7fb78c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.748+0000 7fb792e8c700 1 -- 172.21.15.33:0/1410321881 shutdown_connections 2024-05-08T21:16:59.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.748+0000 7fb792e8c700 1 -- 172.21.15.33:0/1410321881 wait complete. 2024-05-08T21:16:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.749+0000 7fb792e8c700 1 Processor -- start 2024-05-08T21:16:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.749+0000 7fb792e8c700 1 -- start start 2024-05-08T21:16:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb792e8c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb78c100e70 0x7fb78c198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb792e8c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb78c101fc0 0x7fb78c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb792e8c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78c102980 0x7fb78c1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb792e8c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1a0a20 con 0x7fb78c102980 2024-05-08T21:16:59.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb792e8c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1a0b60 con 0x7fb78c101fc0 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb792e8c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1a0cf0 con 0x7fb78c100e70 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb791429700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78c102980 0x7fb78c1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb790c28700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb78c100e70 0x7fb78c198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb791429700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78c102980 0x7fb78c1a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34848/0 (socket says 172.21.15.33:34848) 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb791429700 1 -- 172.21.15.33:0/2104645147 learned_addr learned my addr 172.21.15.33:0/2104645147 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb78bfff700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb78c101fc0 0x7fb78c199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb790c28700 1 -- 172.21.15.33:0/2104645147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb78c101fc0 msgr2=0x7fb78c199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb790c28700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb78c101fc0 0x7fb78c199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb790c28700 1 -- 172.21.15.33:0/2104645147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78c102980 msgr2=0x7fb78c1a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb790c28700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78c102980 0x7fb78c1a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb790c28700 1 -- 172.21.15.33:0/2104645147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7800095f0 con 0x7fb78c100e70 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb791429700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78c102980 0x7fb78c1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.750+0000 7fb790c28700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb78c100e70 0x7fb78c198ef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb780003f60 tx=0x7fb780004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:59.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.751+0000 7fb789ffb700 1 -- 172.21.15.33:0/2104645147 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78001d070 con 0x7fb78c100e70 2024-05-08T21:16:59.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.751+0000 7fb792e8c700 1 -- 172.21.15.33:0/2104645147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78c1a0f70 con 0x7fb78c100e70 2024-05-08T21:16:59.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.751+0000 7fb789ffb700 1 -- 172.21.15.33:0/2104645147 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb78000bdb0 con 0x7fb78c100e70 2024-05-08T21:16:59.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.751+0000 7fb789ffb700 1 -- 172.21.15.33:0/2104645147 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780022ca0 con 0x7fb78c100e70 2024-05-08T21:16:59.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.751+0000 7fb792e8c700 1 -- 172.21.15.33:0/2104645147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78c1a12e0 con 0x7fb78c100e70 2024-05-08T21:16:59.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.752+0000 7fb792e8c700 1 -- 172.21.15.33:0/2104645147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78c108320 con 0x7fb78c100e70 2024-05-08T21:16:59.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.753+0000 7fb789ffb700 1 -- 172.21.15.33:0/2104645147 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb780022470 con 0x7fb78c100e70 2024-05-08T21:16:59.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.754+0000 7fb789ffb700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb77c0a1120 0x7fb77c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:16:59.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.754+0000 7fb78bfff700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb77c0a1120 0x7fb77c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:16:59.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.754+0000 7fb789ffb700 1 -- 172.21.15.33:0/2104645147 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb7800cdb50 con 0x7fb78c100e70 2024-05-08T21:16:59.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.755+0000 7fb78bfff700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb77c0a1120 0x7fb77c0a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb778009c60 tx=0x7fb778009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:16:59.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.762+0000 7fb789ffb700 1 -- 172.21.15.33:0/2104645147 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb780097890 con 0x7fb78c100e70 2024-05-08T21:16:59.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:59 smithi071 ceph-mon[47074]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:59.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:59 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1730236208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:59.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:59.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:59.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:16:59 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:59 smithi082 ceph-mon[47512]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:16:59.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:59 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1730236208' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:16:59.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:16:59.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:16:59.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:16:59 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:16:59.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.989+0000 7fb792e8c700 1 -- 172.21.15.33:0/2104645147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb78c10ea00 con 0x7fb78c100e70 2024-05-08T21:16:59.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.989+0000 7fb789ffb700 1 -- 172.21.15.33:0/2104645147 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7800270a0 con 0x7fb78c100e70 2024-05-08T21:16:59.993 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:16:59.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.991+0000 7fb792e8c700 1 -- 172.21.15.33:0/2104645147 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb77c0a1120 msgr2=0x7fb77c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:59.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.991+0000 7fb792e8c700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb77c0a1120 0x7fb77c0a35e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb778009c60 tx=0x7fb778009260 comp rx=0 tx=0).stop 2024-05-08T21:16:59.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.991+0000 7fb792e8c700 1 -- 172.21.15.33:0/2104645147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb78c100e70 msgr2=0x7fb78c198ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:16:59.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.992+0000 7fb792e8c700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb78c100e70 0x7fb78c198ef0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb780003f60 tx=0x7fb780004730 comp rx=0 tx=0).stop 2024-05-08T21:16:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.992+0000 7fb792e8c700 1 -- 172.21.15.33:0/2104645147 shutdown_connections 2024-05-08T21:16:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.992+0000 7fb792e8c700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb78c102980 0x7fb78c1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.992+0000 7fb792e8c700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb78c100e70 0x7fb78c198ef0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.992+0000 7fb792e8c700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb77c0a1120 0x7fb77c0a35e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.992+0000 7fb792e8c700 1 --2- 172.21.15.33:0/2104645147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb78c101fc0 0x7fb78c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:16:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.992+0000 7fb792e8c700 1 -- 172.21.15.33:0/2104645147 >> 172.21.15.33:0/2104645147 conn(0x7fb78c0fc400 msgr2=0x7fb78c105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:16:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.992+0000 7fb792e8c700 1 -- 172.21.15.33:0/2104645147 shutdown_connections 2024-05-08T21:16:59.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:16:59.992+0000 7fb792e8c700 1 -- 172.21.15.33:0/2104645147 wait complete. 2024-05-08T21:17:00.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-08T21:17:00.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.153+0000 7f4b15655700 1 -- 172.21.15.33:0/1606033057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b10100e90 msgr2=0x7f4b101012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:00.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.153+0000 7f4b15655700 1 --2- 172.21.15.33:0/1606033057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b10100e90 0x7f4b101012b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000b0a0 tx=0x7f4b0000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.153+0000 7f4b15655700 1 -- 172.21.15.33:0/1606033057 shutdown_connections 2024-05-08T21:17:00.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.153+0000 7f4b15655700 1 --2- 172.21.15.33:0/1606033057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b101029a0 0x7f4b101092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.153+0000 7f4b15655700 1 --2- 172.21.15.33:0/1606033057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b10100e90 0x7f4b101012b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.153+0000 7f4b15655700 1 --2- 172.21.15.33:0/1606033057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b10101fe0 0x7f4b10102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.153+0000 7f4b15655700 1 -- 172.21.15.33:0/1606033057 >> 172.21.15.33:0/1606033057 conn(0x7f4b100fc3e0 msgr2=0x7f4b100fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:00.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.153+0000 7f4b15655700 1 -- 172.21.15.33:0/1606033057 shutdown_connections 2024-05-08T21:17:00.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.153+0000 7f4b15655700 1 -- 172.21.15.33:0/1606033057 wait complete. 2024-05-08T21:17:00.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b15655700 1 Processor -- start 2024-05-08T21:17:00.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b15655700 1 -- start start 2024-05-08T21:17:00.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b15655700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b10100e90 0x7f4b10196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:00.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b15655700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b10101fe0 0x7f4b10197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:00.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b15655700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b101029a0 0x7f4b1019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:00.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b15655700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1019e9f0 con 0x7f4b101029a0 2024-05-08T21:17:00.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b15655700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1019eb30 con 0x7f4b10100e90 2024-05-08T21:17:00.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b15655700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1019ecc0 con 0x7f4b10101fe0 2024-05-08T21:17:00.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b0f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b101029a0 0x7f4b1019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:00.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b0effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b10100e90 0x7f4b10196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b0f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b101029a0 0x7f4b1019e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34866/0 (socket says 172.21.15.33:34866) 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b0f7fe700 1 -- 172.21.15.33:0/431906024 learned_addr learned my addr 172.21.15.33:0/431906024 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b0e7fc700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b10101fe0 0x7f4b10197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.154+0000 7f4b0effd700 1 -- 172.21.15.33:0/431906024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b10101fe0 msgr2=0x7f4b10197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.155+0000 7f4b0effd700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b10101fe0 0x7f4b10197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.155+0000 7f4b0effd700 1 -- 172.21.15.33:0/431906024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b101029a0 msgr2=0x7f4b1019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.155+0000 7f4b0effd700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b101029a0 0x7f4b1019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.155+0000 7f4b0effd700 1 -- 172.21.15.33:0/431906024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b00009d00 con 0x7f4b10100e90 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.155+0000 7f4b0f7fe700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b101029a0 0x7f4b1019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.155+0000 7f4b0e7fc700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b10101fe0 0x7f4b10197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.155+0000 7f4b0effd700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b10100e90 0x7f4b10196f10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000b070 tx=0x7f4b000094f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.155+0000 7f4af7fff700 1 -- 172.21.15.33:0/431906024 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000e040 con 0x7f4b10100e90 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.155+0000 7f4b15655700 1 -- 172.21.15.33:0/431906024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1019ef40 con 0x7f4b10100e90 2024-05-08T21:17:00.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.155+0000 7f4af7fff700 1 -- 172.21.15.33:0/431906024 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b0000be20 con 0x7f4b10100e90 2024-05-08T21:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.155+0000 7f4af7fff700 1 -- 172.21.15.33:0/431906024 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0001b460 con 0x7f4b10100e90 2024-05-08T21:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.155+0000 7f4b15655700 1 -- 172.21.15.33:0/431906024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1019f2b0 con 0x7f4b10100e90 2024-05-08T21:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.156+0000 7f4b15655700 1 -- 172.21.15.33:0/431906024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b100681d0 con 0x7f4b10100e90 2024-05-08T21:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.157+0000 7f4af7fff700 1 -- 172.21.15.33:0/431906024 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4b0001d470 con 0x7f4b10100e90 2024-05-08T21:17:00.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.158+0000 7f4af7fff700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4af80a1120 0x7f4af80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:00.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.158+0000 7f4b0e7fc700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4af80a1120 0x7f4af80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:00.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.158+0000 7f4af7fff700 1 -- 172.21.15.33:0/431906024 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4b000cd420 con 0x7f4b10100e90 2024-05-08T21:17:00.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.158+0000 7f4b0e7fc700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4af80a1120 0x7f4af80a35e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009cb0 tx=0x7f4b04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:00.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.163+0000 7f4af7fff700 1 -- 172.21.15.33:0/431906024 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4b00097040 con 0x7f4b10100e90 2024-05-08T21:17:00.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.417+0000 7f4b15655700 1 -- 172.21.15.33:0/431906024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4b1010c590 con 0x7f4b10100e90 2024-05-08T21:17:00.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.418+0000 7f4af7fff700 1 -- 172.21.15.33:0/431906024 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4b0009a660 con 0x7f4b10100e90 2024-05-08T21:17:00.421 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:00.422 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2024-05-08T21:17:00.422 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:17:00.422 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2024-05-08T21:17:00.422 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2024-05-08T21:17:00.422 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:17:00.422 INFO:teuthology.orchestra.run.smithi033.stdout:] 2024-05-08T21:17:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.420+0000 7f4b15655700 1 -- 172.21.15.33:0/431906024 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4af80a1120 msgr2=0x7f4af80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.420+0000 7f4b15655700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4af80a1120 0x7f4af80a35e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009cb0 tx=0x7f4b04009260 comp rx=0 tx=0).stop 2024-05-08T21:17:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.420+0000 7f4b15655700 1 -- 172.21.15.33:0/431906024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b10100e90 msgr2=0x7f4b10196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.420+0000 7f4b15655700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b10100e90 0x7f4b10196f10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000b070 tx=0x7f4b000094f0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.420+0000 7f4b15655700 1 -- 172.21.15.33:0/431906024 shutdown_connections 2024-05-08T21:17:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.420+0000 7f4b15655700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b101029a0 0x7f4b1019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.420+0000 7f4b15655700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4af80a1120 0x7f4af80a35e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.420+0000 7f4b15655700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b10100e90 0x7f4b10196f10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.420+0000 7f4b15655700 1 --2- 172.21.15.33:0/431906024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b10101fe0 0x7f4b10197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.420+0000 7f4b15655700 1 -- 172.21.15.33:0/431906024 >> 172.21.15.33:0/431906024 conn(0x7f4b100fc3e0 msgr2=0x7f4b10105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.420+0000 7f4b15655700 1 -- 172.21.15.33:0/431906024 shutdown_connections 2024-05-08T21:17:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.420+0000 7f4b15655700 1 -- 172.21.15.33:0/431906024 wait complete. 2024-05-08T21:17:00.439 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T21:17:00.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T21:17:00.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.593+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3653706968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b80101900 msgr2=0x7f3b80103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:00.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.593+0000 7f3b85ceb700 1 --2- 172.21.15.33:0/3653706968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b80101900 0x7f3b80103cf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b700098f0 tx=0x7f3b70009c00 comp rx=0 tx=0).stop 2024-05-08T21:17:00.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.593+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3653706968 shutdown_connections 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.593+0000 7f3b85ceb700 1 --2- 172.21.15.33:0/3653706968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b80106b60 0x7f3b80108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.593+0000 7f3b85ceb700 1 --2- 172.21.15.33:0/3653706968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b80101900 0x7f3b80103cf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.593+0000 7f3b85ceb700 1 --2- 172.21.15.33:0/3653706968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80104230 0x7f3b80106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.593+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3653706968 >> 172.21.15.33:0/3653706968 conn(0x7f3b800fb310 msgr2=0x7f3b800fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.593+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3653706968 shutdown_connections 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.593+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3653706968 wait complete. 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.593+0000 7f3b85ceb700 1 Processor -- start 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b85ceb700 1 -- start start 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b85ceb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b80101900 0x7f3b80101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b85ceb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80104230 0x7f3b800ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b85ceb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b80106b60 0x7f3b800ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b85ceb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b80100590 con 0x7f3b80106b60 2024-05-08T21:17:00.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b85ceb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b80100700 con 0x7f3b80104230 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b85ceb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b801a0d50 con 0x7f3b80101900 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b80106b60 0x7f3b800ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b7f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b80101900 0x7f3b80101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b7f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b80101900 0x7f3b80101220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43104/0 (socket says 172.21.15.33:43104) 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b80106b60 0x7f3b800ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37604/0 (socket says 172.21.15.33:37604) 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b7f7fe700 1 -- 172.21.15.33:0/3687305414 learned_addr learned my addr 172.21.15.33:0/3687305414 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b7effd700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80104230 0x7f3b800ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.594+0000 7f3b7f7fe700 1 -- 172.21.15.33:0/3687305414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80104230 msgr2=0x7f3b800ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.595+0000 7f3b7f7fe700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80104230 0x7f3b800ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.595+0000 7f3b7f7fe700 1 -- 172.21.15.33:0/3687305414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b80106b60 msgr2=0x7f3b800ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.595+0000 7f3b7f7fe700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b80106b60 0x7f3b800ffdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.595+0000 7f3b7f7fe700 1 -- 172.21.15.33:0/3687305414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b70009520 con 0x7f3b80101900 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.595+0000 7f3b7effd700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80104230 0x7f3b800ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.595+0000 7f3b7f7fe700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b80101900 0x7f3b80101220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009810 tx=0x7f3b7000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.595+0000 7f3b7cff9700 1 -- 172.21.15.33:0/3687305414 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7001c070 con 0x7f3b80101900 2024-05-08T21:17:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.595+0000 7f3b7cff9700 1 -- 172.21.15.33:0/3687305414 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b7000f7d0 con 0x7f3b80101900 2024-05-08T21:17:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.595+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3687305414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b801a0f80 con 0x7f3b80101900 2024-05-08T21:17:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.595+0000 7f3b7cff9700 1 -- 172.21.15.33:0/3687305414 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70003a00 con 0x7f3b80101900 2024-05-08T21:17:00.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.595+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3687305414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b801a12f0 con 0x7f3b80101900 2024-05-08T21:17:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.596+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3687305414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b800681d0 con 0x7f3b80101900 2024-05-08T21:17:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.597+0000 7f3b7cff9700 1 -- 172.21.15.33:0/3687305414 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3b70020030 con 0x7f3b80101900 2024-05-08T21:17:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.598+0000 7f3b7cff9700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3b680a11a0 0x7f3b680a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.598+0000 7f3b7cff9700 1 -- 172.21.15.33:0/3687305414 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3b700cd390 con 0x7f3b80101900 2024-05-08T21:17:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.598+0000 7f3b7effd700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3b680a11a0 0x7f3b680a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:00.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.598+0000 7f3b7effd700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3b680a11a0 0x7f3b680a3660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009cb0 tx=0x7f3b74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:00.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.603+0000 7f3b7cff9700 1 -- 172.21.15.33:0/3687305414 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3b800681d0 con 0x7f3b80101900 2024-05-08T21:17:00.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:00 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:17:00.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:00 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/153189695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:00.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:00 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2104645147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:17:00.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:00 smithi033 ceph-mon[42528]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:17:00.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:00 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/431906024' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T21:17:00.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:00 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:17:00.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:00 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/153189695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:00.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:00 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2104645147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:17:00.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:00 smithi071 ceph-mon[47074]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:17:00.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:00 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/431906024' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T21:17:00.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:00.830+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3687305414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3b80068160 con 0x7f3b80101900 2024-05-08T21:17:00.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:00 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:17:00.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:00 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/153189695' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:00.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:00 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2104645147' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:17:00.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:00 smithi082 ceph-mon[47512]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:17:00.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:00 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/431906024' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T21:17:01.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.703+0000 7f3b7cff9700 1 -- 172.21.15.33:0/3687305414 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3b70097050 con 0x7f3b80101900 2024-05-08T21:17:01.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.709+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3687305414 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3b680a11a0 msgr2=0x7f3b680a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:01.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.709+0000 7f3b85ceb700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3b680a11a0 0x7f3b680a3660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009cb0 tx=0x7f3b74009260 comp rx=0 tx=0).stop 2024-05-08T21:17:01.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.709+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3687305414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b80101900 msgr2=0x7f3b80101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:01.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.709+0000 7f3b85ceb700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b80101900 0x7f3b80101220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009810 tx=0x7f3b7000b790 comp rx=0 tx=0).stop 2024-05-08T21:17:01.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.709+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3687305414 shutdown_connections 2024-05-08T21:17:01.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.709+0000 7f3b85ceb700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b80106b60 0x7f3b800ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:01.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.709+0000 7f3b85ceb700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b80101900 0x7f3b80101220 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:01.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.709+0000 7f3b85ceb700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3b680a11a0 0x7f3b680a3660 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:01.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.709+0000 7f3b85ceb700 1 --2- 172.21.15.33:0/3687305414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b80104230 0x7f3b800ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:01.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.709+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3687305414 >> 172.21.15.33:0/3687305414 conn(0x7f3b800fb310 msgr2=0x7f3b80103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:01.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.709+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3687305414 shutdown_connections 2024-05-08T21:17:01.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.709+0000 7f3b85ceb700 1 -- 172.21.15.33:0/3687305414 wait complete. 2024-05-08T21:17:01.715 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2024-05-08T21:17:01.729 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-08T21:17:01.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:17:01.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:17:01.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:01 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3687305414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T21:17:01.759 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:01 smithi033 ceph-mon[42528]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T21:17:01.759 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:17:01.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:17:01.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:01 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3687305414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T21:17:01.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:01 smithi071 ceph-mon[47074]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T21:17:01.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:17:01.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:17:01.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:01 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3687305414' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T21:17:01.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:01 smithi082 ceph-mon[47512]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T21:17:01.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:17:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.875+0000 7f5158ba7700 1 -- 172.21.15.33:0/3255315356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5154101920 msgr2=0x7f51541081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.875+0000 7f5158ba7700 1 --2- 172.21.15.33:0/3255315356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5154101920 0x7f51541081d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f514c00b0d0 tx=0x7f514c00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:17:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.875+0000 7f5158ba7700 1 -- 172.21.15.33:0/3255315356 shutdown_connections 2024-05-08T21:17:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.875+0000 7f5158ba7700 1 --2- 172.21.15.33:0/3255315356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5154101920 0x7f51541081d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.875+0000 7f5158ba7700 1 --2- 172.21.15.33:0/3255315356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51540ffd90 0x7f51541001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.875+0000 7f5158ba7700 1 --2- 172.21.15.33:0/3255315356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5154100f80 0x7f51541013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.875+0000 7f5158ba7700 1 -- 172.21.15.33:0/3255315356 >> 172.21.15.33:0/3255315356 conn(0x7f51540fb300 msgr2=0x7f51540fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.875+0000 7f5158ba7700 1 -- 172.21.15.33:0/3255315356 shutdown_connections 2024-05-08T21:17:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.875+0000 7f5158ba7700 1 -- 172.21.15.33:0/3255315356 wait complete. 2024-05-08T21:17:01.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f5158ba7700 1 Processor -- start 2024-05-08T21:17:01.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f5158ba7700 1 -- start start 2024-05-08T21:17:01.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f5158ba7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51540ffd90 0x7f5154103cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:01.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f5158ba7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5154100f80 0x7f5154102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:01.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f5158ba7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5154101920 0x7f51541028a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:01.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f5158ba7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5154103040 con 0x7f51540ffd90 2024-05-08T21:17:01.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f5158ba7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51541031b0 con 0x7f5154100f80 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f5158ba7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51541a0e90 con 0x7f5154101920 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f515259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51540ffd90 0x7f5154103cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f515259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51540ffd90 0x7f5154103cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37622/0 (socket says 172.21.15.33:37622) 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f515259c700 1 -- 172.21.15.33:0/2404651708 learned_addr learned my addr 172.21.15.33:0/2404651708 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f5151d9b700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5154100f80 0x7f5154102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.876+0000 7f5152d9d700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5154101920 0x7f51541028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.877+0000 7f515259c700 1 -- 172.21.15.33:0/2404651708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5154101920 msgr2=0x7f51541028a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.877+0000 7f515259c700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5154101920 0x7f51541028a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.877+0000 7f515259c700 1 -- 172.21.15.33:0/2404651708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5154100f80 msgr2=0x7f5154102360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.877+0000 7f515259c700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5154100f80 0x7f5154102360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.877+0000 7f515259c700 1 -- 172.21.15.33:0/2404651708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c009d00 con 0x7f51540ffd90 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.877+0000 7f5152d9d700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5154101920 0x7f51541028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.877+0000 7f515259c700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51540ffd90 0x7f5154103cd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f514400e970 tx=0x7f514400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:01.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.877+0000 7f51437fe700 1 -- 172.21.15.33:0/2404651708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144009790 con 0x7f51540ffd90 2024-05-08T21:17:01.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.877+0000 7f5158ba7700 1 -- 172.21.15.33:0/2404651708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51541a1120 con 0x7f51540ffd90 2024-05-08T21:17:01.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.877+0000 7f51437fe700 1 -- 172.21.15.33:0/2404651708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f514400eec0 con 0x7f51540ffd90 2024-05-08T21:17:01.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.877+0000 7f51437fe700 1 -- 172.21.15.33:0/2404651708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144003c10 con 0x7f51540ffd90 2024-05-08T21:17:01.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.877+0000 7f5158ba7700 1 -- 172.21.15.33:0/2404651708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51541a1680 con 0x7f51540ffd90 2024-05-08T21:17:01.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.878+0000 7f5158ba7700 1 -- 172.21.15.33:0/2404651708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5154067480 con 0x7f51540ffd90 2024-05-08T21:17:01.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.879+0000 7f51437fe700 1 -- 172.21.15.33:0/2404651708 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f514400cb70 con 0x7f51540ffd90 2024-05-08T21:17:01.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.880+0000 7f51437fe700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f513c0a1120 0x7f513c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:01.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.880+0000 7f5151d9b700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f513c0a1120 0x7f513c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:01.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.880+0000 7f51437fe700 1 -- 172.21.15.33:0/2404651708 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f5144014070 con 0x7f51540ffd90 2024-05-08T21:17:01.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.881+0000 7f5151d9b700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f513c0a1120 0x7f513c0a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5148007be0 tx=0x7f51480073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:01.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:01.884+0000 7f51437fe700 1 -- 172.21.15.33:0/2404651708 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5144096410 con 0x7f51540ffd90 2024-05-08T21:17:02.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.104+0000 7f5158ba7700 1 -- 172.21.15.33:0/2404651708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f515410b410 con 0x7f51540ffd90 2024-05-08T21:17:02.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.715+0000 7f51437fe700 1 -- 172.21.15.33:0/2404651708 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v79) v1 ==== 171+0+0 (secure 0 0 0) 0x7f5144099a30 con 0x7f51540ffd90 2024-05-08T21:17:02.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.725+0000 7f5158ba7700 1 -- 172.21.15.33:0/2404651708 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f513c0a1120 msgr2=0x7f513c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:02.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.725+0000 7f5158ba7700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f513c0a1120 0x7f513c0a35e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5148007be0 tx=0x7f51480073d0 comp rx=0 tx=0).stop 2024-05-08T21:17:02.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.725+0000 7f5158ba7700 1 -- 172.21.15.33:0/2404651708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51540ffd90 msgr2=0x7f5154103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:02.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.725+0000 7f5158ba7700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51540ffd90 0x7f5154103cd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f514400e970 tx=0x7f514400ec80 comp rx=0 tx=0).stop 2024-05-08T21:17:02.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.730+0000 7f5158ba7700 1 -- 172.21.15.33:0/2404651708 shutdown_connections 2024-05-08T21:17:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.730+0000 7f5158ba7700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5154101920 0x7f51541028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.730+0000 7f5158ba7700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51540ffd90 0x7f5154103cd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.730+0000 7f5158ba7700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f513c0a1120 0x7f513c0a35e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.730+0000 7f5158ba7700 1 --2- 172.21.15.33:0/2404651708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5154100f80 0x7f5154102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.730+0000 7f5158ba7700 1 -- 172.21.15.33:0/2404651708 >> 172.21.15.33:0/2404651708 conn(0x7f51540fb300 msgr2=0x7f51540690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.731+0000 7f5158ba7700 1 -- 172.21.15.33:0/2404651708 shutdown_connections 2024-05-08T21:17:02.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.733+0000 7f5158ba7700 1 -- 172.21.15.33:0/2404651708 wait complete. 2024-05-08T21:17:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2024-05-08T21:17:02.754 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T21:17:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.938+0000 7f3d46ec0700 1 -- 172.21.15.33:0/2116257739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d40106b50 msgr2=0x7f3d40108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.938+0000 7f3d46ec0700 1 --2- 172.21.15.33:0/2116257739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d40106b50 0x7f3d40108fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800b0a0 tx=0x7f3d3800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.938+0000 7f3d46ec0700 1 -- 172.21.15.33:0/2116257739 shutdown_connections 2024-05-08T21:17:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.938+0000 7f3d46ec0700 1 --2- 172.21.15.33:0/2116257739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d40106b50 0x7f3d40108fc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.938+0000 7f3d46ec0700 1 --2- 172.21.15.33:0/2116257739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d401018f0 0x7f3d40103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.938+0000 7f3d46ec0700 1 --2- 172.21.15.33:0/2116257739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d40104220 0x7f3d40106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.938+0000 7f3d46ec0700 1 -- 172.21.15.33:0/2116257739 >> 172.21.15.33:0/2116257739 conn(0x7f3d400fb2a0 msgr2=0x7f3d400fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.938+0000 7f3d46ec0700 1 -- 172.21.15.33:0/2116257739 shutdown_connections 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.938+0000 7f3d46ec0700 1 -- 172.21.15.33:0/2116257739 wait complete. 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d46ec0700 1 Processor -- start 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d46ec0700 1 -- start start 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d46ec0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d401018f0 0x7f3d40101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d46ec0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d40104220 0x7f3d400ff8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d46ec0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d40106b50 0x7f3d400ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d46ec0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d40100580 con 0x7f3d40106b50 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d46ec0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d401006f0 con 0x7f3d40104220 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d46ec0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d401a0df0 con 0x7f3d401018f0 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d4545d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d40106b50 0x7f3d400ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d44c5c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d401018f0 0x7f3d40101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d4545d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d40106b50 0x7f3d400ffde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37644/0 (socket says 172.21.15.33:37644) 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d4545d700 1 -- 172.21.15.33:0/410589433 learned_addr learned my addr 172.21.15.33:0/410589433 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d3ffff700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d40104220 0x7f3d400ff8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d4545d700 1 -- 172.21.15.33:0/410589433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d401018f0 msgr2=0x7f3d40101210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d4545d700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d401018f0 0x7f3d40101210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d4545d700 1 -- 172.21.15.33:0/410589433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d40104220 msgr2=0x7f3d400ff8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d4545d700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d40104220 0x7f3d400ff8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d4545d700 1 -- 172.21.15.33:0/410589433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d38009d00 con 0x7f3d40106b50 2024-05-08T21:17:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.939+0000 7f3d4545d700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d40106b50 0x7f3d400ffde0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3d38004d00 tx=0x7f3d38004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.940+0000 7f3d3dffb700 1 -- 172.21.15.33:0/410589433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3800e040 con 0x7f3d40106b50 2024-05-08T21:17:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.940+0000 7f3d3dffb700 1 -- 172.21.15.33:0/410589433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3d3801db20 con 0x7f3d40106b50 2024-05-08T21:17:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.940+0000 7f3d46ec0700 1 -- 172.21.15.33:0/410589433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d401a1020 con 0x7f3d40106b50 2024-05-08T21:17:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.940+0000 7f3d3dffb700 1 -- 172.21.15.33:0/410589433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3801bad0 con 0x7f3d40106b50 2024-05-08T21:17:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.940+0000 7f3d46ec0700 1 -- 172.21.15.33:0/410589433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d401a14a0 con 0x7f3d40106b50 2024-05-08T21:17:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.941+0000 7f3d46ec0700 1 -- 172.21.15.33:0/410589433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d400fce20 con 0x7f3d40106b50 2024-05-08T21:17:02.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.943+0000 7f3d3dffb700 1 -- 172.21.15.33:0/410589433 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3d3801dc90 con 0x7f3d40106b50 2024-05-08T21:17:02.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.943+0000 7f3d3dffb700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3d300a11a0 0x7f3d300a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:02.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.944+0000 7f3d44c5c700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3d300a11a0 0x7f3d300a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:02.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.944+0000 7f3d3dffb700 1 -- 172.21.15.33:0/410589433 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f3d380cec10 con 0x7f3d40106b50 2024-05-08T21:17:02.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.944+0000 7f3d44c5c700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3d300a11a0 0x7f3d300a3660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3d34005950 tx=0x7f3d3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:02.948+0000 7f3d3dffb700 1 -- 172.21.15.33:0/410589433 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3d380985e0 con 0x7f3d40106b50 2024-05-08T21:17:03.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:02 smithi071 ceph-mon[47074]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T21:17:03.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:02 smithi071 ceph-mon[47074]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T21:17:03.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:02 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2404651708' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T21:17:03.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:02 smithi071 ceph-mon[47074]: pgmap v221: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:17:03.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:02 smithi082 ceph-mon[47512]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T21:17:03.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:02 smithi082 ceph-mon[47512]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T21:17:03.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:02 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2404651708' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T21:17:03.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:02 smithi082 ceph-mon[47512]: pgmap v221: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:17:03.207 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:02 smithi033 ceph-mon[42528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T21:17:03.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:02 smithi033 ceph-mon[42528]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T21:17:03.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:02 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2404651708' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T21:17:03.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:02 smithi033 ceph-mon[42528]: pgmap v221: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T21:17:03.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.216+0000 7f3d46ec0700 1 -- 172.21.15.33:0/410589433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3d4010c500 con 0x7f3d40106b50 2024-05-08T21:17:03.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.223+0000 7f3d3dffb700 1 -- 172.21.15.33:0/410589433 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3d3809bc00 con 0x7f3d40106b50 2024-05-08T21:17:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.229+0000 7f3d46ec0700 1 -- 172.21.15.33:0/410589433 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3d300a11a0 msgr2=0x7f3d300a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.229+0000 7f3d46ec0700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3d300a11a0 0x7f3d300a3660 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3d34005950 tx=0x7f3d3400b410 comp rx=0 tx=0).stop 2024-05-08T21:17:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.229+0000 7f3d46ec0700 1 -- 172.21.15.33:0/410589433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d40106b50 msgr2=0x7f3d400ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.229+0000 7f3d46ec0700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d40106b50 0x7f3d400ffde0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3d38004d00 tx=0x7f3d38004d30 comp rx=0 tx=0).stop 2024-05-08T21:17:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.232+0000 7f3d46ec0700 1 -- 172.21.15.33:0/410589433 shutdown_connections 2024-05-08T21:17:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.232+0000 7f3d46ec0700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d40106b50 0x7f3d400ffde0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.232+0000 7f3d46ec0700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d401018f0 0x7f3d40101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.232+0000 7f3d46ec0700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3d300a11a0 0x7f3d300a3660 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.232+0000 7f3d46ec0700 1 --2- 172.21.15.33:0/410589433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d40104220 0x7f3d400ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.232+0000 7f3d46ec0700 1 -- 172.21.15.33:0/410589433 >> 172.21.15.33:0/410589433 conn(0x7f3d400fb2a0 msgr2=0x7f3d401021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:03.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.241+0000 7f3d46ec0700 1 -- 172.21.15.33:0/410589433 shutdown_connections 2024-05-08T21:17:03.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.245+0000 7f3d46ec0700 1 -- 172.21.15.33:0/410589433 wait complete. 2024-05-08T21:17:03.255 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T21:17:03.268 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-08T21:17:03.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-08T21:17:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.408+0000 7f31a5af6700 1 -- 172.21.15.33:0/1002061916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31a00ff770 msgr2=0x7f31a00ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.408+0000 7f31a5af6700 1 --2- 172.21.15.33:0/1002061916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31a00ff770 0x7f31a00ffb90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3194009990 tx=0x7f3194009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.408+0000 7f31a5af6700 1 -- 172.21.15.33:0/1002061916 shutdown_connections 2024-05-08T21:17:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.408+0000 7f31a5af6700 1 --2- 172.21.15.33:0/1002061916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a0100a90 0x7f31a0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.408+0000 7f31a5af6700 1 --2- 172.21.15.33:0/1002061916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31a00ff770 0x7f31a00ffb90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:03.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.408+0000 7f31a5af6700 1 --2- 172.21.15.33:0/1002061916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31a01000d0 0x7f31a0100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:03.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.408+0000 7f31a5af6700 1 -- 172.21.15.33:0/1002061916 >> 172.21.15.33:0/1002061916 conn(0x7f31a00fb310 msgr2=0x7f31a00fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:03.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.408+0000 7f31a5af6700 1 -- 172.21.15.33:0/1002061916 shutdown_connections 2024-05-08T21:17:03.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.408+0000 7f31a5af6700 1 -- 172.21.15.33:0/1002061916 wait complete. 2024-05-08T21:17:03.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.409+0000 7f31a5af6700 1 Processor -- start 2024-05-08T21:17:03.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.409+0000 7f31a5af6700 1 -- start start 2024-05-08T21:17:03.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.409+0000 7f31a5af6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31a00ff770 0x7f31a0198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:03.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.409+0000 7f31a5af6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a01000d0 0x7f31a01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:03.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.409+0000 7f31a5af6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31a0100a90 0x7f31a01a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:03.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.409+0000 7f31a5af6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a01a09c0 con 0x7f31a00ff770 2024-05-08T21:17:03.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.409+0000 7f31a5af6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a01a0b00 con 0x7f31a01000d0 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.409+0000 7f31a5af6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a01a0c90 con 0x7f31a0100a90 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.409+0000 7f319f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31a00ff770 0x7f31a0198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.409+0000 7f319f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31a00ff770 0x7f31a0198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37664/0 (socket says 172.21.15.33:37664) 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.409+0000 7f319f7fe700 1 -- 172.21.15.33:0/831809930 learned_addr learned my addr 172.21.15.33:0/831809930 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f319effd700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a01000d0 0x7f31a01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f319ffff700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31a0100a90 0x7f31a01a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f319f7fe700 1 -- 172.21.15.33:0/831809930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31a0100a90 msgr2=0x7f31a01a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f319f7fe700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31a0100a90 0x7f31a01a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f319f7fe700 1 -- 172.21.15.33:0/831809930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a01000d0 msgr2=0x7f31a01993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f319f7fe700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a01000d0 0x7f31a01993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f319f7fe700 1 -- 172.21.15.33:0/831809930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31940095f0 con 0x7f31a00ff770 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f319ffff700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31a0100a90 0x7f31a01a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f319f7fe700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31a00ff770 0x7f31a0198e90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f319400b5c0 tx=0x7f3194003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f319cff9700 1 -- 172.21.15.33:0/831809930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319401d070 con 0x7f31a00ff770 2024-05-08T21:17:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f319cff9700 1 -- 172.21.15.33:0/831809930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f319400bd10 con 0x7f31a00ff770 2024-05-08T21:17:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f31a5af6700 1 -- 172.21.15.33:0/831809930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a01a0f10 con 0x7f31a00ff770 2024-05-08T21:17:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f319cff9700 1 -- 172.21.15.33:0/831809930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194022d20 con 0x7f31a00ff770 2024-05-08T21:17:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.410+0000 7f31a5af6700 1 -- 172.21.15.33:0/831809930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a01a1280 con 0x7f31a00ff770 2024-05-08T21:17:03.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.411+0000 7f31a5af6700 1 -- 172.21.15.33:0/831809930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a00681d0 con 0x7f31a00ff770 2024-05-08T21:17:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.412+0000 7f319cff9700 1 -- 172.21.15.33:0/831809930 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3194003f40 con 0x7f31a00ff770 2024-05-08T21:17:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.413+0000 7f319cff9700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f31880a1120 0x7f31880a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.413+0000 7f319effd700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f31880a1120 0x7f31880a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.413+0000 7f319cff9700 1 -- 172.21.15.33:0/831809930 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f31940cd800 con 0x7f31a00ff770 2024-05-08T21:17:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.414+0000 7f319effd700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f31880a1120 0x7f31880a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3190005950 tx=0x7f319000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:03.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.418+0000 7f319cff9700 1 -- 172.21.15.33:0/831809930 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f31a00681d0 con 0x7f31a00ff770 2024-05-08T21:17:03.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:03.647+0000 7f31a5af6700 1 -- 172.21.15.33:0/831809930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f31a0068160 con 0x7f31a00ff770 2024-05-08T21:17:04.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:03 smithi071 ceph-mon[47074]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T21:17:04.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:03 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2404651708' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T21:17:04.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:03 smithi071 ceph-mon[47074]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T21:17:04.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:03 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/410589433' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T21:17:04.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:03 smithi071 ceph-mon[47074]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T21:17:04.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:03 smithi071 ceph-mon[47074]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T21:17:04.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:03 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/410589433' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T21:17:04.071 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:03 smithi071 ceph-mon[47074]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T21:17:04.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:03 smithi071 ceph-mon[47074]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T21:17:04.072 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:03 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/831809930' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T21:17:04.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:03 smithi082 ceph-mon[47512]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T21:17:04.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:03 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2404651708' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T21:17:04.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:03 smithi082 ceph-mon[47512]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T21:17:04.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:03 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/410589433' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T21:17:04.103 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:03 smithi082 ceph-mon[47512]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T21:17:04.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:03 smithi082 ceph-mon[47512]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T21:17:04.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:03 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/410589433' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T21:17:04.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:03 smithi082 ceph-mon[47512]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T21:17:04.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:03 smithi082 ceph-mon[47512]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T21:17:04.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:03 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/831809930' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T21:17:04.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:03 smithi033 ceph-mon[42528]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T21:17:04.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:03 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2404651708' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T21:17:04.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:03 smithi033 ceph-mon[42528]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T21:17:04.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:03 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/410589433' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T21:17:04.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:03 smithi033 ceph-mon[42528]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T21:17:04.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:03 smithi033 ceph-mon[42528]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T21:17:04.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:03 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/410589433' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T21:17:04.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:03 smithi033 ceph-mon[42528]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T21:17:04.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:03 smithi033 ceph-mon[42528]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T21:17:04.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:03 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/831809930' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T21:17:04.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.229+0000 7f319cff9700 1 -- 172.21.15.33:0/831809930 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) v1 ==== 157+0+0 (secure 0 0 0) 0x7f319409ac00 con 0x7f31a00ff770 2024-05-08T21:17:04.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.233+0000 7f31a5af6700 1 -- 172.21.15.33:0/831809930 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f31880a1120 msgr2=0x7f31880a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:04.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.233+0000 7f31a5af6700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f31880a1120 0x7f31880a35e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3190005950 tx=0x7f319000b410 comp rx=0 tx=0).stop 2024-05-08T21:17:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.233+0000 7f31a5af6700 1 -- 172.21.15.33:0/831809930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31a00ff770 msgr2=0x7f31a0198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.233+0000 7f31a5af6700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31a00ff770 0x7f31a0198e90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f319400b5c0 tx=0x7f3194003a50 comp rx=0 tx=0).stop 2024-05-08T21:17:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.233+0000 7f31a5af6700 1 -- 172.21.15.33:0/831809930 shutdown_connections 2024-05-08T21:17:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.233+0000 7f31a5af6700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31a0100a90 0x7f31a01a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.233+0000 7f31a5af6700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31a00ff770 0x7f31a0198e90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.233+0000 7f31a5af6700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f31880a1120 0x7f31880a35e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.233+0000 7f31a5af6700 1 --2- 172.21.15.33:0/831809930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31a01000d0 0x7f31a01993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.233+0000 7f31a5af6700 1 -- 172.21.15.33:0/831809930 >> 172.21.15.33:0/831809930 conn(0x7f31a00fb310 msgr2=0x7f31a00fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.233+0000 7f31a5af6700 1 -- 172.21.15.33:0/831809930 shutdown_connections 2024-05-08T21:17:04.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.233+0000 7f31a5af6700 1 -- 172.21.15.33:0/831809930 wait complete. 2024-05-08T21:17:04.253 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T21:17:04.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.400+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/489233039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0ff8a0 msgr2=0x7ff9ac109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:04.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.400+0000 7ff9b2bf5700 1 --2- 172.21.15.33:0/489233039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0ff8a0 0x7ff9ac109470 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400b0d0 tx=0x7ff9a400b3e0 comp rx=0 tx=0).stop 2024-05-08T21:17:04.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.400+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/489233039 shutdown_connections 2024-05-08T21:17:04.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.400+0000 7ff9b2bf5700 1 --2- 172.21.15.33:0/489233039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0ff8a0 0x7ff9ac109470 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:04.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.400+0000 7ff9b2bf5700 1 --2- 172.21.15.33:0/489233039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9ac102490 0x7ff9ac1028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:04.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.400+0000 7ff9b2bf5700 1 --2- 172.21.15.33:0/489233039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9ac0fef00 0x7ff9ac0ff360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:04.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.400+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/489233039 >> 172.21.15.33:0/489233039 conn(0x7ff9ac0faaa0 msgr2=0x7ff9ac0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:04.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.400+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/489233039 shutdown_connections 2024-05-08T21:17:04.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.400+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/489233039 wait complete. 2024-05-08T21:17:04.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.401+0000 7ff9b2bf5700 1 Processor -- start 2024-05-08T21:17:04.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.401+0000 7ff9b2bf5700 1 -- start start 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.401+0000 7ff9b2bf5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9ac0fef00 0x7ff9ac199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.401+0000 7ff9b2bf5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9ac0ff8a0 0x7ff9ac1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.401+0000 7ff9b2bf5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac102490 0x7ff9ac1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.401+0000 7ff9b2bf5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac1a0b50 con 0x7ff9ac102490 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.401+0000 7ff9b2bf5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac1a0c90 con 0x7ff9ac0fef00 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.401+0000 7ff9b2bf5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac1a0e20 con 0x7ff9ac0ff8a0 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.401+0000 7ff9b1192700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac102490 0x7ff9ac1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.401+0000 7ff9b1192700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac102490 0x7ff9ac1a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37688/0 (socket says 172.21.15.33:37688) 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.401+0000 7ff9b1192700 1 -- 172.21.15.33:0/1134769717 learned_addr learned my addr 172.21.15.33:0/1134769717 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9b0991700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9ac0fef00 0x7ff9ac199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9a3fff700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9ac0ff8a0 0x7ff9ac1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9b1192700 1 -- 172.21.15.33:0/1134769717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9ac0ff8a0 msgr2=0x7ff9ac1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9b1192700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9ac0ff8a0 0x7ff9ac1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:04.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9b1192700 1 -- 172.21.15.33:0/1134769717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9ac0fef00 msgr2=0x7ff9ac199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:04.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9b1192700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9ac0fef00 0x7ff9ac199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:04.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9b1192700 1 -- 172.21.15.33:0/1134769717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a4009d00 con 0x7ff9ac102490 2024-05-08T21:17:04.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9a3fff700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9ac0ff8a0 0x7ff9ac1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:04.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9b1192700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac102490 0x7ff9ac1a0580 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4007af0 tx=0x7ff9a400baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:04.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9a1ffb700 1 -- 172.21.15.33:0/1134769717 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400e040 con 0x7ff9ac102490 2024-05-08T21:17:04.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9a1ffb700 1 -- 172.21.15.33:0/1134769717 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9a401db00 con 0x7ff9ac102490 2024-05-08T21:17:04.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/1134769717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac1a10a0 con 0x7ff9ac102490 2024-05-08T21:17:04.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9a1ffb700 1 -- 172.21.15.33:0/1134769717 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a401baf0 con 0x7ff9ac102490 2024-05-08T21:17:04.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.402+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/1134769717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac1a1520 con 0x7ff9ac102490 2024-05-08T21:17:04.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.403+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/1134769717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac068160 con 0x7ff9ac102490 2024-05-08T21:17:04.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.405+0000 7ff9a1ffb700 1 -- 172.21.15.33:0/1134769717 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff9a401dc70 con 0x7ff9ac102490 2024-05-08T21:17:04.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.405+0000 7ff9a1ffb700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff9940a1120 0x7ff9940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:04.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.406+0000 7ff9a1ffb700 1 -- 172.21.15.33:0/1134769717 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8205+0+0 (secure 0 0 0) 0x7ff9a40ce650 con 0x7ff9ac102490 2024-05-08T21:17:04.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.406+0000 7ff9b0991700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff9940a1120 0x7ff9940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:04.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.406+0000 7ff9b0991700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff9940a1120 0x7ff9940a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac069250 tx=0x7ff9a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:04.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.410+0000 7ff9a1ffb700 1 -- 172.21.15.33:0/1134769717 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff9a4097e90 con 0x7ff9ac102490 2024-05-08T21:17:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:04.631+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/1134769717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff9ac10c6c0 con 0x7ff9ac102490 2024-05-08T21:17:05.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:05 smithi071 ceph-mon[47074]: pgmap v224: 97 pgs: 3 creating+peering, 15 active+clean, 79 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T21:17:05.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:05 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] up:active 2024-05-08T21:17:05.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:05 smithi071 ceph-mon[47074]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:05.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:05 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/831809930' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T21:17:05.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:05 smithi071 ceph-mon[47074]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T21:17:05.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:05 smithi071 ceph-mon[47074]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:05.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:05 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1134769717' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T21:17:05.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:05 smithi071 ceph-mon[47074]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T21:17:05.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:05 smithi071 ceph-mon[47074]: Cluster is now healthy 2024-05-08T21:17:05.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:05 smithi082 ceph-mon[47512]: pgmap v224: 97 pgs: 3 creating+peering, 15 active+clean, 79 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T21:17:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:05 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] up:active 2024-05-08T21:17:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:05 smithi082 ceph-mon[47512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:05 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/831809930' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T21:17:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:05 smithi082 ceph-mon[47512]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T21:17:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:05 smithi082 ceph-mon[47512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:05 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1134769717' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T21:17:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:05 smithi082 ceph-mon[47512]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T21:17:05.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:05 smithi082 ceph-mon[47512]: Cluster is now healthy 2024-05-08T21:17:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:05 smithi033 ceph-mon[42528]: pgmap v224: 97 pgs: 3 creating+peering, 15 active+clean, 79 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T21:17:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:05 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] up:active 2024-05-08T21:17:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:05 smithi033 ceph-mon[42528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:05 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/831809930' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T21:17:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:05 smithi033 ceph-mon[42528]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T21:17:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:05 smithi033 ceph-mon[42528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:05 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1134769717' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T21:17:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:05 smithi033 ceph-mon[42528]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T21:17:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:05 smithi033 ceph-mon[42528]: Cluster is now healthy 2024-05-08T21:17:06.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.238+0000 7ff9a1ffb700 1 -- 172.21.15.33:0/1134769717 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) v1 ==== 228+0+0 (secure 0 0 0) 0x7ff9a409b4b0 con 0x7ff9ac102490 2024-05-08T21:17:06.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.244+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/1134769717 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff9940a1120 msgr2=0x7ff9940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:06.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.244+0000 7ff9b2bf5700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff9940a1120 0x7ff9940a35e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac069250 tx=0x7ff9a8009260 comp rx=0 tx=0).stop 2024-05-08T21:17:06.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.244+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/1134769717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac102490 msgr2=0x7ff9ac1a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:06.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.244+0000 7ff9b2bf5700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac102490 0x7ff9ac1a0580 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4007af0 tx=0x7ff9a400baa0 comp rx=0 tx=0).stop 2024-05-08T21:17:06.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.245+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/1134769717 shutdown_connections 2024-05-08T21:17:06.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.245+0000 7ff9b2bf5700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac102490 0x7ff9ac1a0580 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:06.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.245+0000 7ff9b2bf5700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff9940a1120 0x7ff9940a35e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:06.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.245+0000 7ff9b2bf5700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9ac0fef00 0x7ff9ac199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:06.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.245+0000 7ff9b2bf5700 1 --2- 172.21.15.33:0/1134769717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9ac0ff8a0 0x7ff9ac1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:06.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.245+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/1134769717 >> 172.21.15.33:0/1134769717 conn(0x7ff9ac0faaa0 msgr2=0x7ff9ac100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:06.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.248+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/1134769717 shutdown_connections 2024-05-08T21:17:06.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.248+0000 7ff9b2bf5700 1 -- 172.21.15.33:0/1134769717 wait complete. 2024-05-08T21:17:06.252 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data_ec' created 2024-05-08T21:17:06.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-08T21:17:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.410+0000 7fdbb18c2700 1 -- 172.21.15.33:0/4250518561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac100e80 msgr2=0x7fdbac1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.410+0000 7fdbb18c2700 1 --2- 172.21.15.33:0/4250518561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac100e80 0x7fdbac1012a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009990 tx=0x7fdb9c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.410+0000 7fdbb18c2700 1 -- 172.21.15.33:0/4250518561 shutdown_connections 2024-05-08T21:17:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.410+0000 7fdbb18c2700 1 --2- 172.21.15.33:0/4250518561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbac102990 0x7fdbac1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.410+0000 7fdbb18c2700 1 --2- 172.21.15.33:0/4250518561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac100e80 0x7fdbac1012a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.410+0000 7fdbb18c2700 1 --2- 172.21.15.33:0/4250518561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbac101fd0 0x7fdbac102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.410+0000 7fdbb18c2700 1 -- 172.21.15.33:0/4250518561 >> 172.21.15.33:0/4250518561 conn(0x7fdbac0fc3f0 msgr2=0x7fdbac0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.410+0000 7fdbb18c2700 1 -- 172.21.15.33:0/4250518561 shutdown_connections 2024-05-08T21:17:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.410+0000 7fdbb18c2700 1 -- 172.21.15.33:0/4250518561 wait complete. 2024-05-08T21:17:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbb18c2700 1 Processor -- start 2024-05-08T21:17:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbb18c2700 1 -- start start 2024-05-08T21:17:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbb18c2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac100e80 0x7fdbac198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbb18c2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbac101fd0 0x7fdbac199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbb18c2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbac102990 0x7fdbac1a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbb18c2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac1a0a40 con 0x7fdbac100e80 2024-05-08T21:17:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbb18c2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac1a0b80 con 0x7fdbac101fd0 2024-05-08T21:17:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbb18c2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac1a0d10 con 0x7fdbac102990 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbaaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac100e80 0x7fdbac198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbaaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac100e80 0x7fdbac198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37700/0 (socket says 172.21.15.33:37700) 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbaaffd700 1 -- 172.21.15.33:0/1054708122 learned_addr learned my addr 172.21.15.33:0/1054708122 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbaa7fc700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbac101fd0 0x7fdbac199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbab7fe700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbac102990 0x7fdbac1a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbaaffd700 1 -- 172.21.15.33:0/1054708122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbac102990 msgr2=0x7fdbac1a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbaaffd700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbac102990 0x7fdbac1a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbaaffd700 1 -- 172.21.15.33:0/1054708122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbac101fd0 msgr2=0x7fdbac199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbaaffd700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbac101fd0 0x7fdbac199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.411+0000 7fdbaaffd700 1 -- 172.21.15.33:0/1054708122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c0095f0 con 0x7fdbac100e80 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.412+0000 7fdbaaffd700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac100e80 0x7fdbac198f10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0059d0 tx=0x7fdb9c004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.412+0000 7fdbb08c0700 1 -- 172.21.15.33:0/1054708122 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c01d070 con 0x7fdbac100e80 2024-05-08T21:17:06.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.412+0000 7fdbb08c0700 1 -- 172.21.15.33:0/1054708122 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb9c00be50 con 0x7fdbac100e80 2024-05-08T21:17:06.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.412+0000 7fdbb18c2700 1 -- 172.21.15.33:0/1054708122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac1a0f90 con 0x7fdbac100e80 2024-05-08T21:17:06.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.412+0000 7fdbb08c0700 1 -- 172.21.15.33:0/1054708122 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c022e10 con 0x7fdbac100e80 2024-05-08T21:17:06.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.412+0000 7fdbb18c2700 1 -- 172.21.15.33:0/1054708122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac1a1300 con 0x7fdbac100e80 2024-05-08T21:17:06.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.413+0000 7fdbb18c2700 1 -- 172.21.15.33:0/1054708122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbac193530 con 0x7fdbac100e80 2024-05-08T21:17:06.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.414+0000 7fdbb08c0700 1 -- 172.21.15.33:0/1054708122 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdb9c022570 con 0x7fdbac100e80 2024-05-08T21:17:06.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.415+0000 7fdbb08c0700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdb980a1120 0x7fdb980a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:06.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.415+0000 7fdbb08c0700 1 -- 172.21.15.33:0/1054708122 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fdb9c0ce650 con 0x7fdbac100e80 2024-05-08T21:17:06.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.415+0000 7fdbaa7fc700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdb980a1120 0x7fdb980a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:06.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.415+0000 7fdbaa7fc700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdb980a1120 0x7fdb980a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdb94007c60 tx=0x7fdb940073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.419+0000 7fdbb08c0700 1 -- 172.21.15.33:0/1054708122 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdb9c097c40 con 0x7fdbac100e80 2024-05-08T21:17:06.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:06 smithi071 ceph-mon[47074]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T21:17:06.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:06 smithi082 ceph-mon[47512]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T21:17:06.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:06.635+0000 7fdbb18c2700 1 -- 172.21.15.33:0/1054708122 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fdbac068160 con 0x7fdbac100e80 2024-05-08T21:17:06.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:06 smithi033 ceph-mon[42528]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T21:17:07.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.249+0000 7fdbb08c0700 1 -- 172.21.15.33:0/1054708122 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdb9c0b9e60 con 0x7fdbac100e80 2024-05-08T21:17:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.258+0000 7fdbb18c2700 1 -- 172.21.15.33:0/1054708122 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdb980a1120 msgr2=0x7fdb980a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.258+0000 7fdbb18c2700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdb980a1120 0x7fdb980a35e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdb94007c60 tx=0x7fdb940073d0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.258+0000 7fdbb18c2700 1 -- 172.21.15.33:0/1054708122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac100e80 msgr2=0x7fdbac198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.258+0000 7fdbb18c2700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac100e80 0x7fdbac198f10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0059d0 tx=0x7fdb9c004710 comp rx=0 tx=0).stop 2024-05-08T21:17:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.258+0000 7fdbb18c2700 1 -- 172.21.15.33:0/1054708122 shutdown_connections 2024-05-08T21:17:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.258+0000 7fdbb18c2700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbac102990 0x7fdbac1a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.258+0000 7fdbb18c2700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac100e80 0x7fdbac198f10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.258+0000 7fdbb18c2700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdb980a1120 0x7fdb980a35e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.258+0000 7fdbb18c2700 1 --2- 172.21.15.33:0/1054708122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbac101fd0 0x7fdbac199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.258+0000 7fdbb18c2700 1 -- 172.21.15.33:0/1054708122 >> 172.21.15.33:0/1054708122 conn(0x7fdbac0fc3f0 msgr2=0x7fdbac0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.259+0000 7fdbb18c2700 1 -- 172.21.15.33:0/1054708122 shutdown_connections 2024-05-08T21:17:07.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.259+0000 7fdbb18c2700 1 -- 172.21.15.33:0/1054708122 wait complete. 2024-05-08T21:17:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:set pool 4 allow_ec_overwrites to true 2024-05-08T21:17:07.287 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-08T21:17:07.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.426+0000 7fde6a37e700 1 -- 172.21.15.33:0/4152833330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde64101900 msgr2=0x7fde641081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:07.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.426+0000 7fde6a37e700 1 --2- 172.21.15.33:0/4152833330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde64101900 0x7fde641081b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00b0d0 tx=0x7fde5c00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.426+0000 7fde6a37e700 1 -- 172.21.15.33:0/4152833330 shutdown_connections 2024-05-08T21:17:07.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.426+0000 7fde6a37e700 1 --2- 172.21.15.33:0/4152833330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde64101900 0x7fde641081b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.426+0000 7fde6a37e700 1 --2- 172.21.15.33:0/4152833330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde640ffd70 0x7fde64100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.426+0000 7fde6a37e700 1 --2- 172.21.15.33:0/4152833330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde64100f60 0x7fde641013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.426+0000 7fde6a37e700 1 -- 172.21.15.33:0/4152833330 >> 172.21.15.33:0/4152833330 conn(0x7fde640fb2a0 msgr2=0x7fde640fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.426+0000 7fde6a37e700 1 -- 172.21.15.33:0/4152833330 shutdown_connections 2024-05-08T21:17:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.426+0000 7fde6a37e700 1 -- 172.21.15.33:0/4152833330 wait complete. 2024-05-08T21:17:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde6a37e700 1 Processor -- start 2024-05-08T21:17:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde6a37e700 1 -- start start 2024-05-08T21:17:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde6a37e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde640ffd70 0x7fde64195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde6a37e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde64100f60 0x7fde64196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde6a37e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde64101900 0x7fde6419d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde6a37e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6419d840 con 0x7fde64101900 2024-05-08T21:17:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde6a37e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6419d9b0 con 0x7fde64100f60 2024-05-08T21:17:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde6a37e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6419db70 con 0x7fde640ffd70 2024-05-08T21:17:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde6891b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde64101900 0x7fde6419d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde6891b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde64101900 0x7fde6419d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37714/0 (socket says 172.21.15.33:37714) 2024-05-08T21:17:07.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde6891b700 1 -- 172.21.15.33:0/2774721039 learned_addr learned my addr 172.21.15.33:0/2774721039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde63fff700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde640ffd70 0x7fde64195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.427+0000 7fde637fe700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde64100f60 0x7fde64196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde6891b700 1 -- 172.21.15.33:0/2774721039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde640ffd70 msgr2=0x7fde64195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde6891b700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde640ffd70 0x7fde64195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde6891b700 1 -- 172.21.15.33:0/2774721039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde64100f60 msgr2=0x7fde64196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde6891b700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde64100f60 0x7fde64196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde6891b700 1 -- 172.21.15.33:0/2774721039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c009d00 con 0x7fde64101900 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde637fe700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde64100f60 0x7fde64196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde63fff700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde640ffd70 0x7fde64195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde6891b700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde64101900 0x7fde6419d270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fde5c007af0 tx=0x7fde5c00baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde617fa700 1 -- 172.21.15.33:0/2774721039 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c00e040 con 0x7fde64101900 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde6a37e700 1 -- 172.21.15.33:0/2774721039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6419ddf0 con 0x7fde64101900 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde617fa700 1 -- 172.21.15.33:0/2774721039 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fde5c01db00 con 0x7fde64101900 2024-05-08T21:17:07.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde617fa700 1 -- 172.21.15.33:0/2774721039 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c01baf0 con 0x7fde64101900 2024-05-08T21:17:07.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.428+0000 7fde6a37e700 1 -- 172.21.15.33:0/2774721039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6419e1c0 con 0x7fde64101900 2024-05-08T21:17:07.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.429+0000 7fde6a37e700 1 -- 172.21.15.33:0/2774721039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde641071e0 con 0x7fde64101900 2024-05-08T21:17:07.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.430+0000 7fde617fa700 1 -- 172.21.15.33:0/2774721039 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fde5c01dc70 con 0x7fde64101900 2024-05-08T21:17:07.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.431+0000 7fde617fa700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fde500a1120 0x7fde500a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:07.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.431+0000 7fde617fa700 1 -- 172.21.15.33:0/2774721039 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fde5c0ce100 con 0x7fde64101900 2024-05-08T21:17:07.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.431+0000 7fde63fff700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fde500a1120 0x7fde500a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:07.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.431+0000 7fde63fff700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fde500a1120 0x7fde500a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde641971d0 tx=0x7fde54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:07.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.436+0000 7fde617fa700 1 -- 172.21.15.33:0/2774721039 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fde5c097750 con 0x7fde64101900 2024-05-08T21:17:07.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:07 smithi071 ceph-mon[47074]: pgmap v227: 97 pgs: 3 creating+peering, 54 active+clean, 40 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 8 op/s 2024-05-08T21:17:07.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:07 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1134769717' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T21:17:07.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:07 smithi071 ceph-mon[47074]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T21:17:07.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:07 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1054708122' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T21:17:07.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:07 smithi082 ceph-mon[47512]: pgmap v227: 97 pgs: 3 creating+peering, 54 active+clean, 40 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 8 op/s 2024-05-08T21:17:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:07 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1134769717' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T21:17:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:07 smithi082 ceph-mon[47512]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T21:17:07.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:07 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1054708122' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T21:17:07.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:07 smithi033 ceph-mon[42528]: pgmap v227: 97 pgs: 3 creating+peering, 54 active+clean, 40 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 8 op/s 2024-05-08T21:17:07.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:07 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1134769717' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T21:17:07.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:07 smithi033 ceph-mon[42528]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T21:17:07.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:07 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1054708122' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T21:17:07.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.708+0000 7fde6a37e700 1 -- 172.21.15.33:0/2774721039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fde6402ce40 con 0x7fde64101900 2024-05-08T21:17:07.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.713+0000 7fde617fa700 1 -- 172.21.15.33:0/2774721039 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fde5c09ad70 con 0x7fde64101900 2024-05-08T21:17:07.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.716+0000 7fde6a37e700 1 -- 172.21.15.33:0/2774721039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fde500a1120 msgr2=0x7fde500a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:07.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.716+0000 7fde6a37e700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fde500a1120 0x7fde500a35e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fde641971d0 tx=0x7fde54009260 comp rx=0 tx=0).stop 2024-05-08T21:17:07.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.716+0000 7fde6a37e700 1 -- 172.21.15.33:0/2774721039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde64101900 msgr2=0x7fde6419d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:07.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.716+0000 7fde6a37e700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde64101900 0x7fde6419d270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fde5c007af0 tx=0x7fde5c00baa0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.716+0000 7fde6a37e700 1 -- 172.21.15.33:0/2774721039 shutdown_connections 2024-05-08T21:17:07.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.716+0000 7fde6a37e700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde64101900 0x7fde6419d270 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.716+0000 7fde6a37e700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde640ffd70 0x7fde64195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.716+0000 7fde6a37e700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fde500a1120 0x7fde500a35e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.716+0000 7fde6a37e700 1 --2- 172.21.15.33:0/2774721039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde64100f60 0x7fde64196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.716+0000 7fde6a37e700 1 -- 172.21.15.33:0/2774721039 >> 172.21.15.33:0/2774721039 conn(0x7fde640fb2a0 msgr2=0x7fde64104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:07.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.716+0000 7fde6a37e700 1 -- 172.21.15.33:0/2774721039 shutdown_connections 2024-05-08T21:17:07.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.716+0000 7fde6a37e700 1 -- 172.21.15.33:0/2774721039 wait complete. 2024-05-08T21:17:07.720 INFO:teuthology.orchestra.run.smithi033.stderr:added data pool 4 to fsmap 2024-05-08T21:17:07.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:17:07.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.868+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/402375690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d38102980 msgr2=0x7f4d381092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:07.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.868+0000 7f4d3d7fb700 1 --2- 172.21.15.33:0/402375690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d38102980 0x7f4d381092a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000b0a0 tx=0x7f4d3000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.868+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/402375690 shutdown_connections 2024-05-08T21:17:07.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.868+0000 7f4d3d7fb700 1 --2- 172.21.15.33:0/402375690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d38102980 0x7f4d381092a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.868+0000 7f4d3d7fb700 1 --2- 172.21.15.33:0/402375690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d38100e70 0x7f4d38101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.868+0000 7f4d3d7fb700 1 --2- 172.21.15.33:0/402375690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d38101fc0 0x7f4d38102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.868+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/402375690 >> 172.21.15.33:0/402375690 conn(0x7f4d380fc3e0 msgr2=0x7f4d380fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:07.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.868+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/402375690 shutdown_connections 2024-05-08T21:17:07.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.868+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/402375690 wait complete. 2024-05-08T21:17:07.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.869+0000 7f4d3d7fb700 1 Processor -- start 2024-05-08T21:17:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.869+0000 7f4d3d7fb700 1 -- start start 2024-05-08T21:17:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.869+0000 7f4d3d7fb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d38100e70 0x7f4d38196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.869+0000 7f4d3d7fb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d38101fc0 0x7f4d38197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.869+0000 7f4d3d7fb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d38102980 0x7f4d3819e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.869+0000 7f4d3d7fb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3819e930 con 0x7f4d38100e70 2024-05-08T21:17:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.869+0000 7f4d3d7fb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3819ea70 con 0x7f4d38101fc0 2024-05-08T21:17:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.869+0000 7f4d3d7fb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3819ec00 con 0x7f4d38102980 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.869+0000 7f4d36ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d38100e70 0x7f4d38196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.869+0000 7f4d36ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d38100e70 0x7f4d38196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37726/0 (socket says 172.21.15.33:37726) 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.869+0000 7f4d36ffd700 1 -- 172.21.15.33:0/2379137444 learned_addr learned my addr 172.21.15.33:0/2379137444 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d367fc700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d38101fc0 0x7f4d38197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d377fe700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d38102980 0x7f4d3819e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d36ffd700 1 -- 172.21.15.33:0/2379137444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d38102980 msgr2=0x7f4d3819e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d36ffd700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d38102980 0x7f4d3819e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d36ffd700 1 -- 172.21.15.33:0/2379137444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d38101fc0 msgr2=0x7f4d38197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d36ffd700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d38101fc0 0x7f4d38197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d36ffd700 1 -- 172.21.15.33:0/2379137444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d30009d00 con 0x7f4d38100e70 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d377fe700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d38102980 0x7f4d3819e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d367fc700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d38101fc0 0x7f4d38197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d36ffd700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d38100e70 0x7f4d38196e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800e9a0 tx=0x7f4d2800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d1ffff700 1 -- 172.21.15.33:0/2379137444 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28009790 con 0x7f4d38100e70 2024-05-08T21:17:07.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/2379137444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3819eee0 con 0x7f4d38100e70 2024-05-08T21:17:07.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d1ffff700 1 -- 172.21.15.33:0/2379137444 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d2800cbd0 con 0x7f4d38100e70 2024-05-08T21:17:07.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d1ffff700 1 -- 172.21.15.33:0/2379137444 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28018680 con 0x7f4d38100e70 2024-05-08T21:17:07.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.870+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/2379137444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3819f300 con 0x7f4d38100e70 2024-05-08T21:17:07.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.871+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/2379137444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d381082e0 con 0x7f4d38100e70 2024-05-08T21:17:07.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.872+0000 7f4d1ffff700 1 -- 172.21.15.33:0/2379137444 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4d2800cd40 con 0x7f4d38100e70 2024-05-08T21:17:07.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.873+0000 7f4d1ffff700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4d200a1120 0x7f4d200a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:07.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.873+0000 7f4d367fc700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4d200a1120 0x7f4d200a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:07.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.873+0000 7f4d1ffff700 1 -- 172.21.15.33:0/2379137444 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4d28014070 con 0x7f4d38100e70 2024-05-08T21:17:07.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.874+0000 7f4d367fc700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4d200a1120 0x7f4d200a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00bc10 tx=0x7f4d2c00b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:07.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:07.877+0000 7f4d1ffff700 1 -- 172.21.15.33:0/2379137444 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f4d28096540 con 0x7f4d38100e70 2024-05-08T21:17:08.071 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:17:07 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-b[83724]: 2024-05-08T21:17:07.793+0000 7f9a07df2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T21:17:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.149+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/2379137444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d38068160 con 0x7f4d38100e70 2024-05-08T21:17:08.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.150+0000 7f4d1ffff700 1 -- 172.21.15.33:0/2379137444 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10786 (secure 0 0 0) 0x7f4d28026020 con 0x7f4d38100e70 2024-05-08T21:17:08.153 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:08.153 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:17:07.709790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-08T21:17:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.152+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/2379137444 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4d200a1120 msgr2=0x7f4d200a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.152+0000 7f4d3d7fb700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4d200a1120 0x7f4d200a35e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c00bc10 tx=0x7f4d2c00b5f0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.152+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/2379137444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d38100e70 msgr2=0x7f4d38196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.152+0000 7f4d3d7fb700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d38100e70 0x7f4d38196e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800e9a0 tx=0x7f4d2800ed60 comp rx=0 tx=0).stop 2024-05-08T21:17:08.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.152+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/2379137444 shutdown_connections 2024-05-08T21:17:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.152+0000 7f4d3d7fb700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d38102980 0x7f4d3819e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.152+0000 7f4d3d7fb700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d38100e70 0x7f4d38196e00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.152+0000 7f4d3d7fb700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4d200a1120 0x7f4d200a35e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.152+0000 7f4d3d7fb700 1 --2- 172.21.15.33:0/2379137444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d38101fc0 0x7f4d38197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.152+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/2379137444 >> 172.21.15.33:0/2379137444 conn(0x7f4d380fc3e0 msgr2=0x7f4d38105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.153+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/2379137444 shutdown_connections 2024-05-08T21:17:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.153+0000 7f4d3d7fb700 1 -- 172.21.15.33:0/2379137444 wait complete. 2024-05-08T21:17:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-05-08T21:17:08.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:17:08.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.305+0000 7f23d3433700 1 -- 172.21.15.33:0/1518627306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23cc101910 msgr2=0x7f23cc1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.305+0000 7f23d3433700 1 --2- 172.21.15.33:0/1518627306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23cc101910 0x7f23cc1081c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f23c400b0d0 tx=0x7f23c400b3e0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.305+0000 7f23d3433700 1 -- 172.21.15.33:0/1518627306 shutdown_connections 2024-05-08T21:17:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.305+0000 7f23d3433700 1 --2- 172.21.15.33:0/1518627306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23cc101910 0x7f23cc1081c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.305+0000 7f23d3433700 1 --2- 172.21.15.33:0/1518627306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23cc0ffd80 0x7f23cc1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.305+0000 7f23d3433700 1 --2- 172.21.15.33:0/1518627306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23cc100f70 0x7f23cc1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.305+0000 7f23d3433700 1 -- 172.21.15.33:0/1518627306 >> 172.21.15.33:0/1518627306 conn(0x7f23cc0fb310 msgr2=0x7f23cc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.306+0000 7f23d3433700 1 -- 172.21.15.33:0/1518627306 shutdown_connections 2024-05-08T21:17:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.306+0000 7f23d3433700 1 -- 172.21.15.33:0/1518627306 wait complete. 2024-05-08T21:17:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.306+0000 7f23d3433700 1 Processor -- start 2024-05-08T21:17:08.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.306+0000 7f23d3433700 1 -- start start 2024-05-08T21:17:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.306+0000 7f23d3433700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23cc0ffd80 0x7f23cc103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.306+0000 7f23d3433700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23cc100f70 0x7f23cc102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:08.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.306+0000 7f23d3433700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23cc101910 0x7f23cc102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.306+0000 7f23d3433700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc102f90 con 0x7f23cc0ffd80 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d3433700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc103100 con 0x7f23cc101910 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d3433700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc1a0ec0 con 0x7f23cc100f70 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d11cf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23cc0ffd80 0x7f23cc103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d11cf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23cc0ffd80 0x7f23cc103c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37752/0 (socket says 172.21.15.33:37752) 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d11cf700 1 -- 172.21.15.33:0/1095700884 learned_addr learned my addr 172.21.15.33:0/1095700884 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d19d0700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23cc101910 0x7f23cc102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d09ce700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23cc100f70 0x7f23cc102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d11cf700 1 -- 172.21.15.33:0/1095700884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23cc100f70 msgr2=0x7f23cc102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d11cf700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23cc100f70 0x7f23cc102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d11cf700 1 -- 172.21.15.33:0/1095700884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23cc101910 msgr2=0x7f23cc102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d11cf700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23cc101910 0x7f23cc102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d11cf700 1 -- 172.21.15.33:0/1095700884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c4009d00 con 0x7f23cc0ffd80 2024-05-08T21:17:08.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d11cf700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23cc0ffd80 0x7f23cc103c70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f23c800e970 tx=0x7f23c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23c27fc700 1 -- 172.21.15.33:0/1095700884 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8009790 con 0x7f23cc0ffd80 2024-05-08T21:17:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d3433700 1 -- 172.21.15.33:0/1095700884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc1a1120 con 0x7f23cc0ffd80 2024-05-08T21:17:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23c27fc700 1 -- 172.21.15.33:0/1095700884 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23c800eec0 con 0x7f23cc0ffd80 2024-05-08T21:17:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23c27fc700 1 -- 172.21.15.33:0/1095700884 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8003c10 con 0x7f23cc0ffd80 2024-05-08T21:17:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.307+0000 7f23d3433700 1 -- 172.21.15.33:0/1095700884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc1a1630 con 0x7f23cc0ffd80 2024-05-08T21:17:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.308+0000 7f23d3433700 1 -- 172.21.15.33:0/1095700884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc067d10 con 0x7f23cc0ffd80 2024-05-08T21:17:08.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.310+0000 7f23c27fc700 1 -- 172.21.15.33:0/1095700884 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f23c8003d90 con 0x7f23cc0ffd80 2024-05-08T21:17:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.310+0000 7f23c27fc700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f23b80a1120 0x7f23b80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.310+0000 7f23d09ce700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f23b80a1120 0x7f23b80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.310+0000 7f23c27fc700 1 -- 172.21.15.33:0/1095700884 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f23c8014070 con 0x7f23cc0ffd80 2024-05-08T21:17:08.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.311+0000 7f23d09ce700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f23b80a1120 0x7f23b80a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009bf0 tx=0x7f23bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:08.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.314+0000 7f23c27fc700 1 -- 172.21.15.33:0/1095700884 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f23c8096440 con 0x7f23cc0ffd80 2024-05-08T21:17:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:08 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1054708122' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T21:17:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:08 smithi033 ceph-mon[42528]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T21:17:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:08 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2774721039' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T21:17:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:08 smithi033 ceph-mon[42528]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T21:17:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:08 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2774721039' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T21:17:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:08 smithi033 ceph-mon[42528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:08.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:08 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2379137444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:17:08.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.535+0000 7f23d3433700 1 -- 172.21.15.33:0/1095700884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23cc10b400 con 0x7f23cc0ffd80 2024-05-08T21:17:08.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.537+0000 7f23c27fc700 1 -- 172.21.15.33:0/1095700884 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23902 (secure 0 0 0) 0x7f23c8099a60 con 0x7f23cc0ffd80 2024-05-08T21:17:08.540 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:08.540 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:07.709538+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:17:08.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.539+0000 7f23d3433700 1 -- 172.21.15.33:0/1095700884 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f23b80a1120 msgr2=0x7f23b80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:08.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.539+0000 7f23d3433700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f23b80a1120 0x7f23b80a35e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009bf0 tx=0x7f23bc009260 comp rx=0 tx=0).stop 2024-05-08T21:17:08.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.539+0000 7f23d3433700 1 -- 172.21.15.33:0/1095700884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23cc0ffd80 msgr2=0x7f23cc103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:08.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.539+0000 7f23d3433700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23cc0ffd80 0x7f23cc103c70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f23c800e970 tx=0x7f23c800ec80 comp rx=0 tx=0).stop 2024-05-08T21:17:08.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.539+0000 7f23d3433700 1 -- 172.21.15.33:0/1095700884 shutdown_connections 2024-05-08T21:17:08.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.539+0000 7f23d3433700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f23b80a1120 0x7f23b80a35e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.539+0000 7f23d3433700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23cc101910 0x7f23cc102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.539+0000 7f23d3433700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23cc0ffd80 0x7f23cc103c70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.539+0000 7f23d3433700 1 --2- 172.21.15.33:0/1095700884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23cc100f70 0x7f23cc102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.539+0000 7f23d3433700 1 -- 172.21.15.33:0/1095700884 >> 172.21.15.33:0/1095700884 conn(0x7f23cc0fb310 msgr2=0x7f23cc068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:08.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.540+0000 7f23d3433700 1 -- 172.21.15.33:0/1095700884 shutdown_connections 2024-05-08T21:17:08.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.540+0000 7f23d3433700 1 -- 172.21.15.33:0/1095700884 wait complete. 2024-05-08T21:17:08.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:17:08.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:08 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1054708122' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T21:17:08.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:08 smithi071 ceph-mon[47074]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T21:17:08.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:08 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2774721039' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T21:17:08.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:08 smithi071 ceph-mon[47074]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T21:17:08.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:08 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2774721039' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T21:17:08.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:08 smithi071 ceph-mon[47074]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:08.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:08 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2379137444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:17:08.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:08 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1054708122' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T21:17:08.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:08 smithi082 ceph-mon[47512]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T21:17:08.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:08 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2774721039' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T21:17:08.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:08 smithi082 ceph-mon[47512]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T21:17:08.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:08 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2774721039' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T21:17:08.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:08 smithi082 ceph-mon[47512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:08.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:08 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2379137444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:17:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.691+0000 7f64d245a700 1 -- 172.21.15.33:0/1681476173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64cc100e80 msgr2=0x7f64cc1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.691+0000 7f64d245a700 1 --2- 172.21.15.33:0/1681476173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64cc100e80 0x7f64cc1012a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009990 tx=0x7f64bc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.691+0000 7f64d245a700 1 -- 172.21.15.33:0/1681476173 shutdown_connections 2024-05-08T21:17:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.691+0000 7f64d245a700 1 --2- 172.21.15.33:0/1681476173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64cc102990 0x7f64cc1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.691+0000 7f64d245a700 1 --2- 172.21.15.33:0/1681476173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64cc100e80 0x7f64cc1012a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.691+0000 7f64d245a700 1 --2- 172.21.15.33:0/1681476173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc101fd0 0x7f64cc102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.691+0000 7f64d245a700 1 -- 172.21.15.33:0/1681476173 >> 172.21.15.33:0/1681476173 conn(0x7f64cc0fc410 msgr2=0x7f64cc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.691+0000 7f64d245a700 1 -- 172.21.15.33:0/1681476173 shutdown_connections 2024-05-08T21:17:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.691+0000 7f64d245a700 1 -- 172.21.15.33:0/1681476173 wait complete. 2024-05-08T21:17:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.692+0000 7f64d245a700 1 Processor -- start 2024-05-08T21:17:08.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.692+0000 7f64d245a700 1 -- start start 2024-05-08T21:17:08.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.692+0000 7f64d245a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64cc100e80 0x7f64cc074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:08.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.692+0000 7f64d245a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc101fd0 0x7f64cc073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:08.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.692+0000 7f64d245a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64cc102990 0x7f64cc0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:08.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.692+0000 7f64d245a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc073fd0 con 0x7f64cc101fd0 2024-05-08T21:17:08.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.692+0000 7f64d245a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc074140 con 0x7f64cc102990 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.692+0000 7f64d245a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc1a0d70 con 0x7f64cc100e80 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.692+0000 7f64cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc101fd0 0x7f64cc073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.692+0000 7f64cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc101fd0 0x7f64cc073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37774/0 (socket says 172.21.15.33:37774) 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.692+0000 7f64cb7fe700 1 -- 172.21.15.33:0/4095000522 learned_addr learned my addr 172.21.15.33:0/4095000522 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64cbfff700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64cc100e80 0x7f64cc074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64d09f7700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64cc102990 0x7f64cc0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64cb7fe700 1 -- 172.21.15.33:0/4095000522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64cc100e80 msgr2=0x7f64cc074cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64cb7fe700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64cc100e80 0x7f64cc074cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64cb7fe700 1 -- 172.21.15.33:0/4095000522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64cc102990 msgr2=0x7f64cc0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64cb7fe700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64cc102990 0x7f64cc0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64cb7fe700 1 -- 172.21.15.33:0/4095000522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc0095f0 con 0x7f64cc101fd0 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64cbfff700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64cc100e80 0x7f64cc074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64cb7fe700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc101fd0 0x7f64cc073390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f64c000e9a0 tx=0x7f64c000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:08.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64c97fa700 1 -- 172.21.15.33:0/4095000522 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0009790 con 0x7f64cc101fd0 2024-05-08T21:17:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64c97fa700 1 -- 172.21.15.33:0/4095000522 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64c000cbd0 con 0x7f64cc101fd0 2024-05-08T21:17:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64d245a700 1 -- 172.21.15.33:0/4095000522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc1a0fd0 con 0x7f64cc101fd0 2024-05-08T21:17:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64c97fa700 1 -- 172.21.15.33:0/4095000522 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0018680 con 0x7f64cc101fd0 2024-05-08T21:17:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.693+0000 7f64d245a700 1 -- 172.21.15.33:0/4095000522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc1a1450 con 0x7f64cc101fd0 2024-05-08T21:17:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.694+0000 7f64d245a700 1 -- 172.21.15.33:0/4095000522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64cc108330 con 0x7f64cc101fd0 2024-05-08T21:17:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.695+0000 7f64c97fa700 1 -- 172.21.15.33:0/4095000522 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f64c000cd40 con 0x7f64cc101fd0 2024-05-08T21:17:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.696+0000 7f64c97fa700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f64b40a1120 0x7f64b40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.696+0000 7f64cbfff700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f64b40a1120 0x7f64b40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.696+0000 7f64c97fa700 1 -- 172.21.15.33:0/4095000522 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f64c0014070 con 0x7f64cc101fd0 2024-05-08T21:17:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.697+0000 7f64cbfff700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f64b40a1120 0x7f64b40a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0059d0 tx=0x7f64bc005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.701+0000 7f64c97fa700 1 -- 172.21.15.33:0/4095000522 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f64c0052d60 con 0x7f64cc101fd0 2024-05-08T21:17:08.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.922+0000 7f64d245a700 1 -- 172.21.15.33:0/4095000522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64cc067480 con 0x7f64cc101fd0 2024-05-08T21:17:08.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.923+0000 7f64c97fa700 1 -- 172.21.15.33:0/4095000522 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23902 (secure 0 0 0) 0x7f64c0099e00 con 0x7f64cc101fd0 2024-05-08T21:17:08.926 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:08.927 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:07.709538+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:17:08.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.926+0000 7f64d245a700 1 -- 172.21.15.33:0/4095000522 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f64b40a1120 msgr2=0x7f64b40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:08.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.926+0000 7f64d245a700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f64b40a1120 0x7f64b40a35e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0059d0 tx=0x7f64bc005940 comp rx=0 tx=0).stop 2024-05-08T21:17:08.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.926+0000 7f64d245a700 1 -- 172.21.15.33:0/4095000522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc101fd0 msgr2=0x7f64cc073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:08.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.926+0000 7f64d245a700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc101fd0 0x7f64cc073390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f64c000e9a0 tx=0x7f64c000ed60 comp rx=0 tx=0).stop 2024-05-08T21:17:08.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.926+0000 7f64d245a700 1 -- 172.21.15.33:0/4095000522 shutdown_connections 2024-05-08T21:17:08.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.926+0000 7f64d245a700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f64b40a1120 0x7f64b40a35e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.926+0000 7f64d245a700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64cc102990 0x7f64cc0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.926+0000 7f64d245a700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64cc100e80 0x7f64cc074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.926+0000 7f64d245a700 1 --2- 172.21.15.33:0/4095000522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64cc101fd0 0x7f64cc073390 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:08.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.926+0000 7f64d245a700 1 -- 172.21.15.33:0/4095000522 >> 172.21.15.33:0/4095000522 conn(0x7f64cc0fc410 msgr2=0x7f64cc105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:08.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.926+0000 7f64d245a700 1 -- 172.21.15.33:0/4095000522 shutdown_connections 2024-05-08T21:17:08.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:08.926+0000 7f64d245a700 1 -- 172.21.15.33:0/4095000522 wait complete. 2024-05-08T21:17:08.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:17:09.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.100+0000 7f1db0d50700 1 -- 172.21.15.33:0/759272971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dac1019a0 msgr2=0x7f1dac103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.100+0000 7f1db0d50700 1 --2- 172.21.15.33:0/759272971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dac1019a0 0x7f1dac103d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009990 tx=0x7f1d9c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.100+0000 7f1db0d50700 1 -- 172.21.15.33:0/759272971 shutdown_connections 2024-05-08T21:17:09.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.100+0000 7f1db0d50700 1 --2- 172.21.15.33:0/759272971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dac106c00 0x7f1dac108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.100+0000 7f1db0d50700 1 --2- 172.21.15.33:0/759272971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dac1019a0 0x7f1dac103d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.100+0000 7f1db0d50700 1 --2- 172.21.15.33:0/759272971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac1042d0 0x7f1dac1066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.100+0000 7f1db0d50700 1 -- 172.21.15.33:0/759272971 >> 172.21.15.33:0/759272971 conn(0x7f1dac0fb2f0 msgr2=0x7f1dac0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.100+0000 7f1db0d50700 1 -- 172.21.15.33:0/759272971 shutdown_connections 2024-05-08T21:17:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.100+0000 7f1db0d50700 1 -- 172.21.15.33:0/759272971 wait complete. 2024-05-08T21:17:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.101+0000 7f1db0d50700 1 Processor -- start 2024-05-08T21:17:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.101+0000 7f1db0d50700 1 -- start start 2024-05-08T21:17:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.101+0000 7f1db0d50700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac1019a0 0x7f1dac1012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.101+0000 7f1db0d50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dac1042d0 0x7f1dac0ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.101+0000 7f1db0d50700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dac106c00 0x7f1dac0ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.101+0000 7f1db0d50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac100630 con 0x7f1dac1042d0 2024-05-08T21:17:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.101+0000 7f1db0d50700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac1007a0 con 0x7f1dac1019a0 2024-05-08T21:17:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.101+0000 7f1db0d50700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac1a0d70 con 0x7f1dac106c00 2024-05-08T21:17:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.101+0000 7f1da9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dac1042d0 0x7f1dac0ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.101+0000 7f1da9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dac1042d0 0x7f1dac0ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37784/0 (socket says 172.21.15.33:37784) 2024-05-08T21:17:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.101+0000 7f1da9d9b700 1 -- 172.21.15.33:0/2856973384 learned_addr learned my addr 172.21.15.33:0/2856973384 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.101+0000 7f1daa59c700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac1019a0 0x7f1dac1012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1daad9d700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dac106c00 0x7f1dac0ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1da9d9b700 1 -- 172.21.15.33:0/2856973384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dac106c00 msgr2=0x7f1dac0ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1da9d9b700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dac106c00 0x7f1dac0ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1da9d9b700 1 -- 172.21.15.33:0/2856973384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac1019a0 msgr2=0x7f1dac1012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1da9d9b700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac1019a0 0x7f1dac1012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1da9d9b700 1 -- 172.21.15.33:0/2856973384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da0009520 con 0x7f1dac1042d0 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1daa59c700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac1019a0 0x7f1dac1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1da9d9b700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dac1042d0 0x7f1dac0ff950 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1da000eb30 tx=0x7f1da000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1d9b7fe700 1 -- 172.21.15.33:0/2856973384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0009880 con 0x7f1dac1042d0 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1d9b7fe700 1 -- 172.21.15.33:0/2856973384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1da000c970 con 0x7f1dac1042d0 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1db0d50700 1 -- 172.21.15.33:0/2856973384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c0095f0 con 0x7f1dac1042d0 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1d9b7fe700 1 -- 172.21.15.33:0/2856973384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0003c70 con 0x7f1dac1042d0 2024-05-08T21:17:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.102+0000 7f1db0d50700 1 -- 172.21.15.33:0/2856973384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac1a1340 con 0x7f1dac1042d0 2024-05-08T21:17:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.103+0000 7f1db0d50700 1 -- 172.21.15.33:0/2856973384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac0fce70 con 0x7f1dac1042d0 2024-05-08T21:17:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.105+0000 7f1d9b7fe700 1 -- 172.21.15.33:0/2856973384 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1da000cae0 con 0x7f1dac1042d0 2024-05-08T21:17:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.105+0000 7f1d9b7fe700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1d940a11a0 0x7f1d940a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.105+0000 7f1daa59c700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1d940a11a0 0x7f1d940a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.105+0000 7f1d9b7fe700 1 -- 172.21.15.33:0/2856973384 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1da0014070 con 0x7f1dac1042d0 2024-05-08T21:17:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.106+0000 7f1daa59c700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1d940a11a0 0x7f1d940a3660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c006010 tx=0x7f1d9c00b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:09.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.109+0000 7f1d9b7fe700 1 -- 172.21.15.33:0/2856973384 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1da0096530 con 0x7f1dac1042d0 2024-05-08T21:17:09.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.376+0000 7f1db0d50700 1 -- 172.21.15.33:0/2856973384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dac067480 con 0x7f1dac1042d0 2024-05-08T21:17:09.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.376+0000 7f1d9b7fe700 1 -- 172.21.15.33:0/2856973384 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10786 (secure 0 0 0) 0x7f1da0003dd0 con 0x7f1dac1042d0 2024-05-08T21:17:09.379 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:09.379 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24460,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24472,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24487,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24499,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:17:07.709790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24563},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-05-08T21:17:09.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.379+0000 7f1db0d50700 1 -- 172.21.15.33:0/2856973384 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1d940a11a0 msgr2=0x7f1d940a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.379+0000 7f1db0d50700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1d940a11a0 0x7f1d940a3660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c006010 tx=0x7f1d9c00b610 comp rx=0 tx=0).stop 2024-05-08T21:17:09.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.379+0000 7f1db0d50700 1 -- 172.21.15.33:0/2856973384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dac1042d0 msgr2=0x7f1dac0ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.379+0000 7f1db0d50700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dac1042d0 0x7f1dac0ff950 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1da000eb30 tx=0x7f1da000ee40 comp rx=0 tx=0).stop 2024-05-08T21:17:09.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.379+0000 7f1db0d50700 1 -- 172.21.15.33:0/2856973384 shutdown_connections 2024-05-08T21:17:09.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.379+0000 7f1db0d50700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dac106c00 0x7f1dac0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.379+0000 7f1db0d50700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1d940a11a0 0x7f1d940a3660 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.379+0000 7f1db0d50700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac1019a0 0x7f1dac1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.379+0000 7f1db0d50700 1 --2- 172.21.15.33:0/2856973384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dac1042d0 0x7f1dac0ff950 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.379+0000 7f1db0d50700 1 -- 172.21.15.33:0/2856973384 >> 172.21.15.33:0/2856973384 conn(0x7f1dac0fb2f0 msgr2=0x7f1dac103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:09.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.379+0000 7f1db0d50700 1 -- 172.21.15.33:0/2856973384 shutdown_connections 2024-05-08T21:17:09.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.379+0000 7f1db0d50700 1 -- 172.21.15.33:0/2856973384 wait complete. 2024-05-08T21:17:09.383 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-05-08T21:17:09.396 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:17:09.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.530+0000 7fb801cab700 1 -- 172.21.15.33:0/3848964287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc0ffd80 msgr2=0x7fb7fc1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.530+0000 7fb801cab700 1 --2- 172.21.15.33:0/3848964287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc0ffd80 0x7fb7fc1001a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00b0d0 tx=0x7fb7ec00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.531+0000 7fb801cab700 1 -- 172.21.15.33:0/3848964287 shutdown_connections 2024-05-08T21:17:09.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.531+0000 7fb801cab700 1 --2- 172.21.15.33:0/3848964287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7fc101910 0x7fb7fc1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.531+0000 7fb801cab700 1 --2- 172.21.15.33:0/3848964287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc0ffd80 0x7fb7fc1001a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.531+0000 7fb801cab700 1 --2- 172.21.15.33:0/3848964287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc100f70 0x7fb7fc1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.531+0000 7fb801cab700 1 -- 172.21.15.33:0/3848964287 >> 172.21.15.33:0/3848964287 conn(0x7fb7fc0fb2f0 msgr2=0x7fb7fc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.531+0000 7fb801cab700 1 -- 172.21.15.33:0/3848964287 shutdown_connections 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.531+0000 7fb801cab700 1 -- 172.21.15.33:0/3848964287 wait complete. 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.531+0000 7fb801cab700 1 Processor -- start 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.531+0000 7fb801cab700 1 -- start start 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb801cab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7fc0ffd80 0x7fb7fc069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb801cab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc100f70 0x7fb7fc19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb801cab700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc101910 0x7fb7fc19e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb801cab700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc19e9b0 con 0x7fb7fc100f70 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb801cab700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc19eb20 con 0x7fb7fc101910 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb801cab700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc19ece0 con 0x7fb7fc0ffd80 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb7faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc100f70 0x7fb7fc19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb7faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc100f70 0x7fb7fc19bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37812/0 (socket says 172.21.15.33:37812) 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb7faffd700 1 -- 172.21.15.33:0/3521353755 learned_addr learned my addr 172.21.15.33:0/3521353755 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:09.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb7fb7fe700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7fc0ffd80 0x7fb7fc069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb7fbfff700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc101910 0x7fb7fc19e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb7faffd700 1 -- 172.21.15.33:0/3521353755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7fc0ffd80 msgr2=0x7fb7fc069ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb7faffd700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7fc0ffd80 0x7fb7fc069ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb7faffd700 1 -- 172.21.15.33:0/3521353755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc101910 msgr2=0x7fb7fc19e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb7faffd700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc101910 0x7fb7fc19e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb7faffd700 1 -- 172.21.15.33:0/3521353755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec009d00 con 0x7fb7fc100f70 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb7faffd700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc100f70 0x7fb7fc19bc60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000e9a0 tx=0x7fb7f000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.532+0000 7fb7f8ff9700 1 -- 172.21.15.33:0/3521353755 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0009790 con 0x7fb7fc100f70 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.533+0000 7fb7f8ff9700 1 -- 172.21.15.33:0/3521353755 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7f0010430 con 0x7fb7fc100f70 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.533+0000 7fb801cab700 1 -- 172.21.15.33:0/3521353755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc19efc0 con 0x7fb7fc100f70 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.533+0000 7fb7f8ff9700 1 -- 172.21.15.33:0/3521353755 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f000cea0 con 0x7fb7fc100f70 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.533+0000 7fb801cab700 1 -- 172.21.15.33:0/3521353755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc19f490 con 0x7fb7fc100f70 2024-05-08T21:17:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.534+0000 7fb801cab700 1 -- 172.21.15.33:0/3521353755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc107340 con 0x7fb7fc100f70 2024-05-08T21:17:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.535+0000 7fb7f8ff9700 1 -- 172.21.15.33:0/3521353755 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb7f00105a0 con 0x7fb7fc100f70 2024-05-08T21:17:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.535+0000 7fb7f8ff9700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb7e40a11a0 0x7fb7e40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.536+0000 7fb7f8ff9700 1 -- 172.21.15.33:0/3521353755 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb7f0014070 con 0x7fb7fc100f70 2024-05-08T21:17:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.536+0000 7fb7fb7fe700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb7e40a11a0 0x7fb7e40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.536+0000 7fb7fb7fe700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb7e40a11a0 0x7fb7e40a3660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec006070 tx=0x7fb7ec00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:09.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.539+0000 7fb7f8ff9700 1 -- 172.21.15.33:0/3521353755 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb7f0096350 con 0x7fb7fc100f70 2024-05-08T21:17:09.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:09 smithi071 ceph-mon[47074]: pgmap v231: 129 pgs: 5 creating+peering, 99 active+clean, 25 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s wr, 9 op/s 2024-05-08T21:17:09.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:09 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1095700884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:09.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:09 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/4095000522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:09.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:09 smithi082 ceph-mon[47512]: pgmap v231: 129 pgs: 5 creating+peering, 99 active+clean, 25 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s wr, 9 op/s 2024-05-08T21:17:09.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:09 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1095700884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:09.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:09 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/4095000522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:09.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:09 smithi033 ceph-mon[42528]: pgmap v231: 129 pgs: 5 creating+peering, 99 active+clean, 25 unknown; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s wr, 9 op/s 2024-05-08T21:17:09.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:09 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1095700884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:09.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:09 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4095000522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:09.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.758+0000 7fb801cab700 1 -- 172.21.15.33:0/3521353755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7fc10b440 con 0x7fb7fc100f70 2024-05-08T21:17:09.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.759+0000 7fb7f8ff9700 1 -- 172.21.15.33:0/3521353755 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23902 (secure 0 0 0) 0x7fb7f0026090 con 0x7fb7fc100f70 2024-05-08T21:17:09.762 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:09.762 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:07.709538+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:17:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.762+0000 7fb801cab700 1 -- 172.21.15.33:0/3521353755 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb7e40a11a0 msgr2=0x7fb7e40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.762+0000 7fb801cab700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb7e40a11a0 0x7fb7e40a3660 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec006070 tx=0x7fb7ec00b040 comp rx=0 tx=0).stop 2024-05-08T21:17:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.762+0000 7fb801cab700 1 -- 172.21.15.33:0/3521353755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc100f70 msgr2=0x7fb7fc19bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.762+0000 7fb801cab700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc100f70 0x7fb7fc19bc60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000e9a0 tx=0x7fb7f000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.762+0000 7fb801cab700 1 -- 172.21.15.33:0/3521353755 shutdown_connections 2024-05-08T21:17:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.762+0000 7fb801cab700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb7e40a11a0 0x7fb7e40a3660 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.762+0000 7fb801cab700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc101910 0x7fb7fc19e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.762+0000 7fb801cab700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7fc0ffd80 0x7fb7fc069ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.762+0000 7fb801cab700 1 --2- 172.21.15.33:0/3521353755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7fc100f70 0x7fb7fc19bc60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.762+0000 7fb801cab700 1 -- 172.21.15.33:0/3521353755 >> 172.21.15.33:0/3521353755 conn(0x7fb7fc0fb2f0 msgr2=0x7fb7fc104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.762+0000 7fb801cab700 1 -- 172.21.15.33:0/3521353755 shutdown_connections 2024-05-08T21:17:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.762+0000 7fb801cab700 1 -- 172.21.15.33:0/3521353755 wait complete. 2024-05-08T21:17:09.779 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T21:17:03.219607+0000', 'modified': '2024-05-08T21:17:07.709790+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24563}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6839/1515211151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1515211151}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1515211151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-05-08T21:17:09.779 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T21:17:09.779 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T21:17:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.914+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/3917364191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dc4074f60 msgr2=0x7f7dc40733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.915+0000 7f7dcbb4e700 1 --2- 172.21.15.33:0/3917364191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dc4074f60 0x7f7dc40733b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009990 tx=0x7f7dc0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.915+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/3917364191 shutdown_connections 2024-05-08T21:17:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.915+0000 7f7dcbb4e700 1 --2- 172.21.15.33:0/3917364191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc4108d60 0x7f7dc410b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.915+0000 7f7dcbb4e700 1 --2- 172.21.15.33:0/3917364191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dc4074f60 0x7f7dc40733b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.915+0000 7f7dcbb4e700 1 --2- 172.21.15.33:0/3917364191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7dc4073980 0x7f7dc4073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.915+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/3917364191 >> 172.21.15.33:0/3917364191 conn(0x7f7dc40fc400 msgr2=0x7f7dc40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.915+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/3917364191 shutdown_connections 2024-05-08T21:17:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.915+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/3917364191 wait complete. 2024-05-08T21:17:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.915+0000 7f7dcbb4e700 1 Processor -- start 2024-05-08T21:17:09.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.915+0000 7f7dcbb4e700 1 -- start start 2024-05-08T21:17:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dcbb4e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc4073980 0x7f7dc4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dcbb4e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7dc4074f60 0x7f7dc4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dcbb4e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dc4108d60 0x7f7dc41a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dcbb4e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc41a0a00 con 0x7f7dc4108d60 2024-05-08T21:17:09.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dcbb4e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc41a0b40 con 0x7f7dc4073980 2024-05-08T21:17:09.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dcbb4e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc41a0cd0 con 0x7f7dc4074f60 2024-05-08T21:17:09.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dca0eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dc4108d60 0x7f7dc41a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dc90e9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7dc4074f60 0x7f7dc4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dca0eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dc4108d60 0x7f7dc41a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37836/0 (socket says 172.21.15.33:37836) 2024-05-08T21:17:09.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dca0eb700 1 -- 172.21.15.33:0/873174212 learned_addr learned my addr 172.21.15.33:0/873174212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:09.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dc98ea700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc4073980 0x7f7dc4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dc90e9700 1 -- 172.21.15.33:0/873174212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc4073980 msgr2=0x7f7dc4198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc4073980 0x7f7dc4198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dc90e9700 1 -- 172.21.15.33:0/873174212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dc4108d60 msgr2=0x7f7dc41a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dc4108d60 0x7f7dc41a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dc90e9700 1 -- 172.21.15.33:0/873174212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc00095f0 con 0x7f7dc4074f60 2024-05-08T21:17:09.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7dc4074f60 0x7f7dc4199410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7db400e9f0 tx=0x7f7db400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:09.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.916+0000 7f7dbaffd700 1 -- 172.21.15.33:0/873174212 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4004220 con 0x7f7dc4074f60 2024-05-08T21:17:09.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.917+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/873174212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc41a0fb0 con 0x7f7dc4074f60 2024-05-08T21:17:09.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.917+0000 7f7dbaffd700 1 -- 172.21.15.33:0/873174212 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7db4004cc0 con 0x7f7dc4074f60 2024-05-08T21:17:09.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.917+0000 7f7dbaffd700 1 -- 172.21.15.33:0/873174212 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db400ce00 con 0x7f7dc4074f60 2024-05-08T21:17:09.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.917+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/873174212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc41a13e0 con 0x7f7dc4074f60 2024-05-08T21:17:09.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.919+0000 7f7dbaffd700 1 -- 172.21.15.33:0/873174212 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7db4004380 con 0x7f7dc4074f60 2024-05-08T21:17:09.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.920+0000 7f7dbaffd700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7db00a1120 0x7f7db00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.920+0000 7f7dc98ea700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7db00a1120 0x7f7db00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.920+0000 7f7dbaffd700 1 -- 172.21.15.33:0/873174212 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7db4014070 con 0x7f7dc4074f60 2024-05-08T21:17:09.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.920+0000 7f7dc98ea700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7db00a1120 0x7f7db00a35e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0006010 tx=0x7f7dc000b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:09.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.920+0000 7f7dd7d08b80 1 Processor -- start 2024-05-08T21:17:09.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dd7d08b80 1 -- start start 2024-05-08T21:17:09.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dd7d08b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90ba4f0 0x55c6a90ba910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dd7d08b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bae50 0x55c6a90bb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dd7d08b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bb810 0x55c6a90bbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dd7d08b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a9011100 con 0x55c6a90bb810 2024-05-08T21:17:09.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dd7d08b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a900eac0 con 0x55c6a90ba4f0 2024-05-08T21:17:09.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dd7d08b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a900a3b0 con 0x55c6a90bae50 2024-05-08T21:17:09.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dca0eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bb810 0x55c6a90bbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dca0eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bb810 0x55c6a90bbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37846/0 (socket says 172.21.15.33:37846) 2024-05-08T21:17:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dca0eb700 1 -- 172.21.15.33:0/3701736129 learned_addr learned my addr 172.21.15.33:0/3701736129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dc98ea700 1 --2- 172.21.15.33:0/3701736129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90ba4f0 0x55c6a90ba910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/3701736129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bae50 0x55c6a90bb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dca0eb700 1 -- 172.21.15.33:0/3701736129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bae50 msgr2=0x55c6a90bb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:17:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dca0eb700 1 --2- 172.21.15.33:0/3701736129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bae50 0x55c6a90bb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dca0eb700 1 -- 172.21.15.33:0/3701736129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90ba4f0 msgr2=0x55c6a90ba910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:17:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dca0eb700 1 --2- 172.21.15.33:0/3701736129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90ba4f0 0x55c6a90ba910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dca0eb700 1 -- 172.21.15.33:0/3701736129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a8e8b360 con 0x55c6a90bb810 2024-05-08T21:17:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.921+0000 7f7dc98ea700 1 --2- 172.21.15.33:0/3701736129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90ba4f0 0x55c6a90ba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.922+0000 7f7dca0eb700 1 --2- 172.21.15.33:0/3701736129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bb810 0x55c6a90bbc90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009f60 tx=0x7f7dbc009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e7bfa42012116b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:09.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.922+0000 7f7daffff700 1 -- 172.21.15.33:0/3701736129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0147f0 con 0x55c6a90bb810 2024-05-08T21:17:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.922+0000 7f7daffff700 1 -- 172.21.15.33:0/3701736129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7dbc00f040 con 0x55c6a90bb810 2024-05-08T21:17:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.922+0000 7f7daffff700 1 -- 172.21.15.33:0/3701736129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0159f0 con 0x55c6a90bb810 2024-05-08T21:17:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.923+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/3701736129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bb810 msgr2=0x55c6a90bbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.923+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/3701736129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bb810 0x55c6a90bbc90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009f60 tx=0x7f7dbc009230 comp rx=0 tx=0).stop 2024-05-08T21:17:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.923+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/3701736129 shutdown_connections 2024-05-08T21:17:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.923+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/3701736129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bb810 0x55c6a90bbc90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.923+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/3701736129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90ba4f0 0x55c6a90ba910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.923+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/3701736129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bae50 0x55c6a90bb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.923+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/3701736129 >> 172.21.15.33:0/3701736129 conn(0x55c6a90181c0 msgr2=0x55c6a90b4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.923+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/3701736129 shutdown_connections 2024-05-08T21:17:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.923+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/3701736129 wait complete. 2024-05-08T21:17:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.923+0000 7f7dd7d08b80 1 -- start start 2024-05-08T21:17:09.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.923+0000 7f7dd7d08b80 1 Processor -- start 2024-05-08T21:17:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dd7d08b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dd7d08b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bae50 0x55c6a90c4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dd7d08b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bb810 0x55c6a90cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dd7d08b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8fa6510 con 0x55c6a90ba4f0 2024-05-08T21:17:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dd7d08b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8fa61b0 con 0x55c6a90bb810 2024-05-08T21:17:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dd7d08b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8e8d650 con 0x55c6a90bae50 2024-05-08T21:17:09.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dd7d08b80 10 client.0 _ll_register_callbacks cb 0x55c6a8fafae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dc98ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dc98ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37848/0 (socket says 172.21.15.33:37848) 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dc98ea700 1 -- 172.21.15.33:0/1888806035 learned_addr learned my addr 172.21.15.33:0/1888806035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bae50 0x55c6a90c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dca0eb700 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bb810 0x55c6a90cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dc98ea700 1 -- 172.21.15.33:0/1888806035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bae50 msgr2=0x55c6a90c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dc98ea700 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bae50 0x55c6a90c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dc98ea700 1 -- 172.21.15.33:0/1888806035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bb810 msgr2=0x55c6a90cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.924+0000 7f7dc98ea700 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bb810 0x55c6a90cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dc98ea700 1 -- 172.21.15.33:0/1888806035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dbc009b90 con 0x55c6a90ba4f0 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dc98ea700 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c3d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0003f60 tx=0x7f7dc001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4aadbf567f5a019 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dad7fa700 1 -- 172.21.15.33:0/1888806035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0003d70 con 0x55c6a90ba4f0 2024-05-08T21:17:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dad7fa700 1 -- 172.21.15.33:0/1888806035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7dc000bd80 con 0x55c6a90ba4f0 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1888806035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6a8f63220 con 0x55c6a90ba4f0 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dad7fa700 1 -- 172.21.15.33:0/1888806035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc0004500 con 0x55c6a90ba4f0 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dd7d08b80 10 client.14745 fetch_fsmap learned FSMap version 17 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1888806035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55c6a8e37b10 con 0x55c6a90ba4f0 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dad7fa700 1 -- 172.21.15.33:0/1888806035 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c6a8f63220 con 0x55c6a90ba4f0 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dad7fa700 1 -- 172.21.15.33:0/1888806035 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7dc000f580 con 0x55c6a90ba4f0 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dd7d08b80 10 client.14745 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.925+0000 7f7dd7d08b80 10 client.14745 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.926+0000 7f7dd7d08b80 20 client.14745 populate_metadata read hostname 'smithi033' 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.926+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c6a90cef10 0x55c6a90d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.926+0000 7f7dd7d08b80 4 client.14745 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.926+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1888806035 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6a90897c0 con 0x55c6a90cef10 2024-05-08T21:17:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.926+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c6a90cef10 0x55c6a90d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.926+0000 7f7dad7fa700 10 client.14745 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:17:09.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.926+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c6a90cef10 0x55c6a90d4230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56454316615b4718 server_cookie=81f8ed7af37ca5c5 in_seq=0 out_seq=0 2024-05-08T21:17:09.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dad7fa700 1 -- 172.21.15.33:0/1888806035 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c6a90897c0 con 0x55c6a90cef10 2024-05-08T21:17:09.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dad7fa700 10 client.14745 handle_command_reply: tid=0 2024-05-08T21:17:09.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dd7d08b80 1 client.14745 shutdown 2024-05-08T21:17:09.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1888806035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 msgr2=0x55c6a90c3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c3d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0003f60 tx=0x7f7dc001a050 comp rx=0 tx=0).stop 2024-05-08T21:17:09.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1888806035 shutdown_connections 2024-05-08T21:17:09.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c6a90cef10 0x55c6a90d4230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bb810 0x55c6a90cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c3d50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dad7fa700 0 client.14745 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:17:09.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/1888806035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bae50 0x55c6a90c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1888806035 >> 172.21.15.33:0/1888806035 conn(0x55c6a90181c0 msgr2=0x55c6a90b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:09.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1888806035 shutdown_connections 2024-05-08T21:17:09.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1888806035 wait complete. 2024-05-08T21:17:09.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.927+0000 7f7dd7d08b80 20 client.14745 trim_cache size 0 max 16384 2024-05-08T21:17:09.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.947+0000 7f7dd7d08b80 1 Processor -- start 2024-05-08T21:17:09.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.947+0000 7f7dd7d08b80 1 -- start start 2024-05-08T21:17:09.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.947+0000 7f7dd7d08b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.947+0000 7f7dd7d08b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bae50 0x55c6a90cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.947+0000 7f7dd7d08b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bb810 0x55c6a90b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.947+0000 7f7dd7d08b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8f63220 con 0x55c6a90bae50 2024-05-08T21:17:09.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.947+0000 7f7dd7d08b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a8fc82b0 con 0x55c6a90bb810 2024-05-08T21:17:09.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.947+0000 7f7dd7d08b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a9003150 con 0x55c6a90ba4f0 2024-05-08T21:17:09.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.947+0000 7f7dc90e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bae50 0x55c6a90cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.947+0000 7f7dc90e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bae50 0x55c6a90cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37860/0 (socket says 172.21.15.33:37860) 2024-05-08T21:17:09.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.947+0000 7f7dc90e9700 1 -- 172.21.15.33:0/2279819533 learned_addr learned my addr 172.21.15.33:0/2279819533 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:09.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.948+0000 7f7dc98ea700 1 --2- 172.21.15.33:0/2279819533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.948+0000 7f7dca0eb700 1 --2- 172.21.15.33:0/2279819533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bb810 0x55c6a90b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.948+0000 7f7dc90e9700 1 -- 172.21.15.33:0/2279819533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90ba4f0 msgr2=0x55c6a90c3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:17:09.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.948+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/2279819533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.948+0000 7f7dc90e9700 1 -- 172.21.15.33:0/2279819533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bb810 msgr2=0x55c6a90b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:17:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.948+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/2279819533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bb810 0x55c6a90b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.948+0000 7f7dc90e9700 1 -- 172.21.15.33:0/2279819533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6a8fd0510 con 0x55c6a90bae50 2024-05-08T21:17:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.948+0000 7f7dca0eb700 1 --2- 172.21.15.33:0/2279819533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bb810 0x55c6a90b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:17:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.948+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/2279819533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bae50 0x55c6a90cb290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7db40ca2e0 tx=0x7f7db40ca1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b60d7fe58b2d8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.948+0000 7f7db8da4700 1 -- 172.21.15.33:0/2279819533 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db409da60 con 0x55c6a90bae50 2024-05-08T21:17:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.948+0000 7f7db8da4700 1 -- 172.21.15.33:0/2279819533 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7db40a20b0 con 0x55c6a90bae50 2024-05-08T21:17:09.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.948+0000 7f7db8da4700 1 -- 172.21.15.33:0/2279819533 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db40a1c20 con 0x55c6a90bae50 2024-05-08T21:17:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.949+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/2279819533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bae50 msgr2=0x55c6a90cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.949+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/2279819533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bae50 0x55c6a90cb290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7db40ca2e0 tx=0x7f7db40ca1b0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.949+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/2279819533 shutdown_connections 2024-05-08T21:17:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.949+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/2279819533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bb810 0x55c6a90b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.949+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/2279819533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.949+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/2279819533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90bae50 0x55c6a90cb290 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.949+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/2279819533 >> 172.21.15.33:0/2279819533 conn(0x55c6a90181c0 msgr2=0x55c6a90c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.949+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/2279819533 shutdown_connections 2024-05-08T21:17:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.949+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/2279819533 wait complete. 2024-05-08T21:17:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.949+0000 7f7dd7d08b80 1 -- start start 2024-05-08T21:17:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.949+0000 7f7dd7d08b80 1 Processor -- start 2024-05-08T21:17:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dd7d08b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dd7d08b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bae50 0x55c6a90c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dd7d08b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bb810 0x55c6a9125ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dd7d08b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a90cb3b0 con 0x55c6a90ba4f0 2024-05-08T21:17:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dd7d08b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a90cb520 con 0x55c6a90bae50 2024-05-08T21:17:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dd7d08b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6a9126430 con 0x55c6a90bb810 2024-05-08T21:17:09.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dd7d08b80 10 client.0 _ll_register_callbacks cb 0x55c6a8fafae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:17:09.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dc98ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dc98ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37864/0 (socket says 172.21.15.33:37864) 2024-05-08T21:17:09.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dc98ea700 1 -- 172.21.15.33:0/1051747155 learned_addr learned my addr 172.21.15.33:0/1051747155 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:09.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bae50 0x55c6a90c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dca0eb700 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bb810 0x55c6a9125ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dc98ea700 1 -- 172.21.15.33:0/1051747155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bb810 msgr2=0x55c6a9125ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:17:09.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dc98ea700 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bb810 0x55c6a9125ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dc98ea700 1 -- 172.21.15.33:0/1051747155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bae50 msgr2=0x55c6a90c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:17:09.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dc98ea700 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bae50 0x55c6a90c6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dc98ea700 1 -- 172.21.15.33:0/1051747155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc41a13e0 con 0x55c6a90ba4f0 2024-05-08T21:17:09.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dc98ea700 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c6500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0003f60 tx=0x7f7dc000fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b5e71993da310f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:09.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:17:09.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dad7fa700 1 -- 172.21.15.33:0/1051747155 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000fe10 con 0x55c6a90ba4f0 2024-05-08T21:17:09.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dad7fa700 1 -- 172.21.15.33:0/1051747155 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7dc000fbd0 con 0x55c6a90ba4f0 2024-05-08T21:17:09.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.950+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1051747155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dbc00f040 con 0x55c6a90ba4f0 2024-05-08T21:17:09.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.951+0000 7f7dad7fa700 1 -- 172.21.15.33:0/1051747155 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dc0003d70 con 0x55c6a90ba4f0 2024-05-08T21:17:09.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.951+0000 7f7dd7d08b80 10 client.14751 fetch_fsmap learned FSMap version 17 2024-05-08T21:17:09.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.951+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1051747155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55c6a8e8fc80 con 0x55c6a90ba4f0 2024-05-08T21:17:09.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.951+0000 7f7dad7fa700 1 -- 172.21.15.33:0/1051747155 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7dbc00f040 con 0x55c6a90ba4f0 2024-05-08T21:17:09.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.951+0000 7f7dad7fa700 1 -- 172.21.15.33:0/1051747155 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f7dc00039d0 con 0x55c6a90ba4f0 2024-05-08T21:17:09.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.951+0000 7f7dd7d08b80 10 client.14751 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T21:17:09.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.951+0000 7f7dd7d08b80 10 client.14751 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T21:17:09.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.951+0000 7f7dd7d08b80 20 client.14751 populate_metadata read hostname 'smithi033' 2024-05-08T21:17:09.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.951+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c6a912ab50 0x55c6a9127590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:09.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.951+0000 7f7dd7d08b80 4 client.14751 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:17:09.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.951+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1051747155 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c6a9002e70 con 0x55c6a912ab50 2024-05-08T21:17:09.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.951+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c6a912ab50 0x55c6a9127590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:09.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dad7fa700 10 client.14751 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:17:09.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dc90e9700 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c6a912ab50 0x55c6a9127590 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de66777738c423a9 server_cookie=e48a65c16216e959 in_seq=0 out_seq=0 2024-05-08T21:17:09.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dad7fa700 1 -- 172.21.15.33:0/1051747155 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55c6a9002e70 con 0x55c6a912ab50 2024-05-08T21:17:09.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dad7fa700 10 client.14751 handle_command_reply: tid=0 2024-05-08T21:17:09.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dd7d08b80 1 client.14751 shutdown 2024-05-08T21:17:09.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1051747155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 msgr2=0x55c6a90c6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c6500 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0003f60 tx=0x7f7dc000fa80 comp rx=0 tx=0).stop 2024-05-08T21:17:09.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1051747155 shutdown_connections 2024-05-08T21:17:09.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c6a912ab50 0x55c6a9127590 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c6a90bb810 0x55c6a9125ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dad7fa700 0 client.14751 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:17:09.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a90ba4f0 0x55c6a90c6500 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dd7d08b80 1 --2- 172.21.15.33:0/1051747155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6a90bae50 0x55c6a90c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1051747155 >> 172.21.15.33:0/1051747155 conn(0x55c6a90181c0 msgr2=0x55c6a90c3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:09.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1051747155 shutdown_connections 2024-05-08T21:17:09.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dd7d08b80 1 -- 172.21.15.33:0/1051747155 wait complete. 2024-05-08T21:17:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.952+0000 7f7dd7d08b80 20 client.14751 trim_cache size 0 max 16384 2024-05-08T21:17:09.970 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:17:09.970 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:17:09.970 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-05-08T21:17:09.970 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24563, 2024-05-08T21:17:09.970 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-08T21:17:09.970 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-08T21:17:09.970 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-08T21:17:09.970 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 6.7299312259999997, 2024-05-08T21:17:09.970 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 17, 2024-05-08T21:17:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 85, 2024-05-08T21:17:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 84, 2024-05-08T21:17:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.552760183 2024-05-08T21:17:09.971 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:17:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.953+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/873174212 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7db00a1120 msgr2=0x7f7db00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.953+0000 7f7dcbb4e700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7db00a1120 0x7f7db00a35e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0006010 tx=0x7f7dc000b610 comp rx=0 tx=0).stop 2024-05-08T21:17:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.953+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/873174212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7dc4074f60 msgr2=0x7f7dc4199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.953+0000 7f7dcbb4e700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7dc4074f60 0x7f7dc4199410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7db400e9f0 tx=0x7f7db400ed00 comp rx=0 tx=0).stop 2024-05-08T21:17:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.953+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/873174212 shutdown_connections 2024-05-08T21:17:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.953+0000 7f7dcbb4e700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dc4108d60 0x7f7dc41a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.953+0000 7f7dcbb4e700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7db00a1120 0x7f7db00a35e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.953+0000 7f7dcbb4e700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc4073980 0x7f7dc4198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.953+0000 7f7dcbb4e700 1 --2- 172.21.15.33:0/873174212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7dc4074f60 0x7f7dc4199410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.953+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/873174212 >> 172.21.15.33:0/873174212 conn(0x7f7dc40fc400 msgr2=0x7f7dc4104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.953+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/873174212 shutdown_connections 2024-05-08T21:17:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:09.953+0000 7f7dcbb4e700 1 -- 172.21.15.33:0/873174212 wait complete. 2024-05-08T21:17:09.974 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T21:17:09.974 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T21:17:09.974 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T21:17:10.220 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:10.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:10 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2856973384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:17:10.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:10 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3521353755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:10.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:10 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2856973384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:17:10.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:10 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3521353755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:10.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:10 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2856973384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:17:10.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:10 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3521353755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.407+0000 7f2148174700 1 -- 172.21.15.33:0/2809062819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2140106fe0 msgr2=0x7f21401073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.407+0000 7f2148174700 1 --2- 172.21.15.33:0/2809062819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2140106fe0 0x7f21401073c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f213c0099c0 tx=0x7f213c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:17:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.407+0000 7f2148174700 1 -- 172.21.15.33:0/2809062819 shutdown_connections 2024-05-08T21:17:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.407+0000 7f2148174700 1 --2- 172.21.15.33:0/2809062819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2140069bd0 0x7f214010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.407+0000 7f2148174700 1 --2- 172.21.15.33:0/2809062819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2140106fe0 0x7f21401073c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:11.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.407+0000 7f2148174700 1 --2- 172.21.15.33:0/2809062819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2140069210 0x7f2140069690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.407+0000 7f2148174700 1 -- 172.21.15.33:0/2809062819 >> 172.21.15.33:0/2809062819 conn(0x7f2140076e70 msgr2=0x7f2140077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.407+0000 7f2148174700 1 -- 172.21.15.33:0/2809062819 shutdown_connections 2024-05-08T21:17:11.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.407+0000 7f2148174700 1 -- 172.21.15.33:0/2809062819 wait complete. 2024-05-08T21:17:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.408+0000 7f2148174700 1 Processor -- start 2024-05-08T21:17:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.408+0000 7f2148174700 1 -- start start 2024-05-08T21:17:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2148174700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2140069210 0x7f2140197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2148174700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2140069bd0 0x7f2140197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2148174700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2140106fe0 0x7f214019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2148174700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214019c120 con 0x7f2140106fe0 2024-05-08T21:17:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2148174700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214019c290 con 0x7f2140069210 2024-05-08T21:17:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2148174700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214019c450 con 0x7f2140069bd0 2024-05-08T21:17:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2146711700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2140106fe0 0x7f214019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:11.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2145f10700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2140069210 0x7f2140197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2146711700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2140106fe0 0x7f214019bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38996/0 (socket says 172.21.15.33:38996) 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2146711700 1 -- 172.21.15.33:0/1799824937 learned_addr learned my addr 172.21.15.33:0/1799824937 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f214570f700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2140069bd0 0x7f2140197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2146711700 1 -- 172.21.15.33:0/1799824937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2140069bd0 msgr2=0x7f2140197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2146711700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2140069bd0 0x7f2140197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2146711700 1 -- 172.21.15.33:0/1799824937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2140069210 msgr2=0x7f2140197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2146711700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2140069210 0x7f2140197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2146711700 1 -- 172.21.15.33:0/1799824937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c0095f0 con 0x7f2140106fe0 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.409+0000 7f2145f10700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2140069210 0x7f2140197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.410+0000 7f2146711700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2140106fe0 0x7f214019bb10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2138007ee0 tx=0x7f213800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.410+0000 7f2136ffd700 1 -- 172.21.15.33:0/1799824937 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2138017070 con 0x7f2140106fe0 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.410+0000 7f2136ffd700 1 -- 172.21.15.33:0/1799824937 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f213800f040 con 0x7f2140106fe0 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.410+0000 7f2148174700 1 -- 172.21.15.33:0/1799824937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214019c730 con 0x7f2140106fe0 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.410+0000 7f2136ffd700 1 -- 172.21.15.33:0/1799824937 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2138005570 con 0x7f2140106fe0 2024-05-08T21:17:11.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.410+0000 7f2148174700 1 -- 172.21.15.33:0/1799824937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21401a3f90 con 0x7f2140106fe0 2024-05-08T21:17:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.411+0000 7f2148174700 1 -- 172.21.15.33:0/1799824937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21401094e0 con 0x7f2140106fe0 2024-05-08T21:17:11.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.413+0000 7f2136ffd700 1 -- 172.21.15.33:0/1799824937 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f213800db20 con 0x7f2140106fe0 2024-05-08T21:17:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.413+0000 7f2136ffd700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f212c0a11a0 0x7f212c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.414+0000 7f2145f10700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f212c0a11a0 0x7f212c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.414+0000 7f2136ffd700 1 -- 172.21.15.33:0/1799824937 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f21380cc8d0 con 0x7f2140106fe0 2024-05-08T21:17:11.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.414+0000 7f2145f10700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f212c0a11a0 0x7f212c0a3660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f213c000c00 tx=0x7f213c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:11.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.417+0000 7f2136ffd700 1 -- 172.21.15.33:0/1799824937 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2138095e10 con 0x7f2140106fe0 2024-05-08T21:17:11.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:11 smithi071 ceph-mon[47074]: pgmap v232: 129 pgs: 5 creating+peering, 118 active+clean, 6 unknown; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-05-08T21:17:11.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:11 smithi082 ceph-mon[47512]: pgmap v232: 129 pgs: 5 creating+peering, 118 active+clean, 6 unknown; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-05-08T21:17:11.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.675+0000 7f2148174700 1 -- 172.21.15.33:0/1799824937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2140102690 con 0x7f2140106fe0 2024-05-08T21:17:11.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.676+0000 7f2136ffd700 1 -- 172.21.15.33:0/1799824937 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2138099430 con 0x7f2140106fe0 2024-05-08T21:17:11.679 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.678+0000 7f2148174700 1 -- 172.21.15.33:0/1799824937 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f212c0a11a0 msgr2=0x7f212c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.678+0000 7f2148174700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f212c0a11a0 0x7f212c0a3660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f213c000c00 tx=0x7f213c009f10 comp rx=0 tx=0).stop 2024-05-08T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.678+0000 7f2148174700 1 -- 172.21.15.33:0/1799824937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2140106fe0 msgr2=0x7f214019bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.678+0000 7f2148174700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2140106fe0 0x7f214019bb10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2138007ee0 tx=0x7f213800d3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.678+0000 7f2148174700 1 -- 172.21.15.33:0/1799824937 shutdown_connections 2024-05-08T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.678+0000 7f2148174700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2140106fe0 0x7f214019bb10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.678+0000 7f2148174700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f212c0a11a0 0x7f212c0a3660 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.678+0000 7f2148174700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2140069210 0x7f2140197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.678+0000 7f2148174700 1 --2- 172.21.15.33:0/1799824937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2140069bd0 0x7f2140197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.678+0000 7f2148174700 1 -- 172.21.15.33:0/1799824937 >> 172.21.15.33:0/1799824937 conn(0x7f2140076e70 msgr2=0x7f21400fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.678+0000 7f2148174700 1 -- 172.21.15.33:0/1799824937 shutdown_connections 2024-05-08T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:11.678+0000 7f2148174700 1 -- 172.21.15.33:0/1799824937 wait complete. 2024-05-08T21:17:11.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:11 smithi033 ceph-mon[42528]: pgmap v232: 129 pgs: 5 creating+peering, 118 active+clean, 6 unknown; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-05-08T21:17:12.079 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T21:17:12.079 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T21:17:12.079 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-05-08T21:17:12.079 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-08T21:17:12.079 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T21:17:12.079 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T21:17:12.079 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T21:17:12.079 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T21:17:12.080 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T21:17:12.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:12.080 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-05-08T21:17:12.103 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T21:17:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T21:17:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T21:17:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-08T21:17:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-05-08T21:17:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-08T21:17:12.103 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41789sec preferred_lft 41789sec 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-08T21:17:12.104 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T21:17:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-05-08T21:17:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-05-08T21:17:12.105 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T21:17:12.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:12.105 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T21:17:12.106 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T21:17:12.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-05-08T21:17:12.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-05-08T21:17:12.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-05-08T21:17:12.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T21:17:12.106 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T21:17:12.279 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:12 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1799824937' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T21:17:12.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:12.311 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T21:17:12.345 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-05-08T21:17:12.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:12.347 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-05-08T21:17:12.510 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-05-08T21:17:12.510 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T21:17:12.510 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T21:17:12.510 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T21:17:12.510 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T21:17:12.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:12.511 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T21:17:12.511 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T21:17:12.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T21:17:12.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T21:17:12.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T21:17:12.511 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T21:17:12.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:12 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1799824937' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T21:17:12.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:12 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1799824937' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T21:17:12.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:12.729 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-08T21:17:12.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:12.750 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-05-08T21:17:12.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:12.812 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T21:17:12.812 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T21:17:12.812 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T21:17:12.812 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-08T21:17:12.812 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T21:17:12.926 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-08T21:17:12.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:12.926 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T21:17:12.926 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T21:17:12.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-08T21:17:12.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T21:17:12.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-08T21:17:12.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-08T21:17:12.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-08T21:17:12.927 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T21:17:13.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:13.180 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T21:17:13.180 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T21:17:13.180 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-05-08T21:17:13.180 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T21:17:13.180 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T21:17:13.290 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-08T21:17:13.291 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T21:17:13.291 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-08T21:17:13.310 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T21:17:13.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T21:17:13.310 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-08T21:17:13.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-05-08T21:17:13.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:13 smithi033 ceph-mon[42528]: pgmap v233: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-05-08T21:17:13.447 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T21:17:13.447 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T21:17:13.522 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T21:17:13.524 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T21:17:13.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:13.524 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-05-08T21:17:13.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:13 smithi071 ceph-mon[47074]: pgmap v233: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-05-08T21:17:13.586 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T21:17:13.587 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-08T21:17:13.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:13 smithi082 ceph-mon[47512]: pgmap v233: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-05-08T21:17:13.629 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-05-08T21:17:13.664 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T21:17:13.665 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T21:17:13.736 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T21:17:13.737 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T21:17:13.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:13.738 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-05-08T21:17:13.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:2024-05-08T21:17:13.735+0000 7f46967295c0 -1 init, newargv = 0x55e3cd66b8b0 newargc=15 2024-05-08T21:17:13.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[95857]: starting ceph client 2024-05-08T21:17:13.745 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[95857]: starting fuse 2024-05-08T21:17:13.758 INFO:teuthology.orchestra.run.smithi033.stdout:124 2024-05-08T21:17:13.758 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2024-05-08T21:17:13.758 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T21:17:13.758 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-05-08T21:17:13.758 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-05-08T21:17:13.759 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-05-08T21:17:13.759 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-05-08T21:17:13.759 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-08T21:17:13.759 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-05-08T21:17:13.759 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-08T21:17:13.759 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-05-08T21:17:13.759 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-08T21:17:13.759 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-08T21:17:13.759 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-05-08T21:17:13.759 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-05-08T21:17:13.759 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.admin")) 2024-05-08T21:17:13.760 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-05-08T21:17:13.895 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.admin.95857.asok 2024-05-08T21:17:14.076 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.95857.asok 2024-05-08T21:17:14.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:14.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.95857.asok status 2024-05-08T21:17:14.257 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:17:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-05-08T21:17:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T21:17:14.257 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "admin", 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "95857", 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 14763, 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-05-08T21:17:14.258 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 14763 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 4061459783 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 4061459783 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.14763 192.168.0.1:0/4061459783", 2024-05-08T21:17:14.259 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/4061459783", 2024-05-08T21:17:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2024-05-08T21:17:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 17, 2024-05-08T21:17:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 85, 2024-05-08T21:17:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-05-08T21:17:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-05-08T21:17:14.260 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-05-08T21:17:14.260 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:17:14.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:14.269 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-05-08T21:17:14.269 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-08T21:17:14.288 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-05-08T21:17:14.288 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-08T21:17:14.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:14.288 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-08T21:17:14.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:14.362 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-08T21:17:14.490 INFO:teuthology.orchestra.run.smithi033.stdout:# file: . 2024-05-08T21:17:14.490 INFO:teuthology.orchestra.run.smithi033.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-08T21:17:14.490 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:14.491 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:17:14.491 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T21:17:14.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:17:14.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.698+0000 7f6363152700 1 -- 172.21.15.33:0/2739022663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f635c0ffd80 msgr2=0x7f635c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.698+0000 7f6363152700 1 --2- 172.21.15.33:0/2739022663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f635c0ffd80 0x7f635c1001a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f634c0099c0 tx=0x7f634c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.698+0000 7f6363152700 1 -- 172.21.15.33:0/2739022663 shutdown_connections 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.698+0000 7f6363152700 1 --2- 172.21.15.33:0/2739022663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f635c101910 0x7f635c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.698+0000 7f6363152700 1 --2- 172.21.15.33:0/2739022663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f635c0ffd80 0x7f635c1001a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.698+0000 7f6363152700 1 --2- 172.21.15.33:0/2739022663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f635c100f70 0x7f635c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.698+0000 7f6363152700 1 -- 172.21.15.33:0/2739022663 >> 172.21.15.33:0/2739022663 conn(0x7f635c0fb310 msgr2=0x7f635c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.698+0000 7f6363152700 1 -- 172.21.15.33:0/2739022663 shutdown_connections 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.698+0000 7f6363152700 1 -- 172.21.15.33:0/2739022663 wait complete. 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.699+0000 7f6363152700 1 Processor -- start 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.699+0000 7f6363152700 1 -- start start 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.699+0000 7f6363152700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f635c0ffd80 0x7f635c196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.699+0000 7f6363152700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f635c100f70 0x7f635c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:14.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.699+0000 7f6363152700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f635c101910 0x7f635c19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.699+0000 7f6363152700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c19e850 con 0x7f635c0ffd80 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.699+0000 7f6363152700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c19e9c0 con 0x7f635c100f70 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.699+0000 7f6363152700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c19eb80 con 0x7f635c101910 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.699+0000 7f6360eee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f635c0ffd80 0x7f635c196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.699+0000 7f6360eee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f635c0ffd80 0x7f635c196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39016/0 (socket says 172.21.15.33:39016) 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.699+0000 7f6360eee700 1 -- 172.21.15.33:0/1000993652 learned_addr learned my addr 172.21.15.33:0/1000993652 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f635bfff700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f635c100f70 0x7f635c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f63616ef700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f635c101910 0x7f635c19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f6360eee700 1 -- 172.21.15.33:0/1000993652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f635c101910 msgr2=0x7f635c19e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f6360eee700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f635c101910 0x7f635c19e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f6360eee700 1 -- 172.21.15.33:0/1000993652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f635c100f70 msgr2=0x7f635c1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f6360eee700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f635c100f70 0x7f635c1972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f6360eee700 1 -- 172.21.15.33:0/1000993652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f634c0095f0 con 0x7f635c0ffd80 2024-05-08T21:17:14.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f63616ef700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f635c101910 0x7f635c19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f6360eee700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f635c0ffd80 0x7f635c196d60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f634c000c00 tx=0x7f634c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f6359ffb700 1 -- 172.21.15.33:0/1000993652 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c01d070 con 0x7f635c0ffd80 2024-05-08T21:17:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f6359ffb700 1 -- 172.21.15.33:0/1000993652 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f634c0054c0 con 0x7f635c0ffd80 2024-05-08T21:17:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f6363152700 1 -- 172.21.15.33:0/1000993652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635c19ee00 con 0x7f635c0ffd80 2024-05-08T21:17:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f6359ffb700 1 -- 172.21.15.33:0/1000993652 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c004ba0 con 0x7f635c0ffd80 2024-05-08T21:17:14.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.700+0000 7f6363152700 1 -- 172.21.15.33:0/1000993652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635c19f1d0 con 0x7f635c0ffd80 2024-05-08T21:17:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.701+0000 7f6363152700 1 -- 172.21.15.33:0/1000993652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635c1071f0 con 0x7f635c0ffd80 2024-05-08T21:17:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.704+0000 7f6359ffb700 1 -- 172.21.15.33:0/1000993652 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f634c004d00 con 0x7f635c0ffd80 2024-05-08T21:17:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.704+0000 7f6359ffb700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f63440a11a0 0x7f63440a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.704+0000 7f6359ffb700 1 -- 172.21.15.33:0/1000993652 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f634c0cdb10 con 0x7f635c0ffd80 2024-05-08T21:17:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.704+0000 7f635bfff700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f63440a11a0 0x7f63440a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:14.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.705+0000 7f635bfff700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f63440a11a0 0x7f63440a3660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6350009c60 tx=0x7f6350009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:14.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.709+0000 7f6359ffb700 1 -- 172.21.15.33:0/1000993652 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f634c097140 con 0x7f635c0ffd80 2024-05-08T21:17:14.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.929+0000 7f6363152700 1 -- 172.21.15.33:0/1000993652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f635c067d10 con 0x7f635c0ffd80 2024-05-08T21:17:14.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.931+0000 7f6359ffb700 1 -- 172.21.15.33:0/1000993652 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23902 (secure 0 0 0) 0x7f634c09a760 con 0x7f635c0ffd80 2024-05-08T21:17:14.934 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:14.934 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:07.709538+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:17:14.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.933+0000 7f6363152700 1 -- 172.21.15.33:0/1000993652 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f63440a11a0 msgr2=0x7f63440a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:14.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.933+0000 7f6363152700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f63440a11a0 0x7f63440a3660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6350009c60 tx=0x7f6350009260 comp rx=0 tx=0).stop 2024-05-08T21:17:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.933+0000 7f6363152700 1 -- 172.21.15.33:0/1000993652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f635c0ffd80 msgr2=0x7f635c196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.933+0000 7f6363152700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f635c0ffd80 0x7f635c196d60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f634c000c00 tx=0x7f634c00f690 comp rx=0 tx=0).stop 2024-05-08T21:17:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.933+0000 7f6363152700 1 -- 172.21.15.33:0/1000993652 shutdown_connections 2024-05-08T21:17:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.933+0000 7f6363152700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f635c101910 0x7f635c19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.933+0000 7f6363152700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f635c0ffd80 0x7f635c196d60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.933+0000 7f6363152700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f63440a11a0 0x7f63440a3660 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.933+0000 7f6363152700 1 --2- 172.21.15.33:0/1000993652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f635c100f70 0x7f635c1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.933+0000 7f6363152700 1 -- 172.21.15.33:0/1000993652 >> 172.21.15.33:0/1000993652 conn(0x7f635c0fb310 msgr2=0x7f635c104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.934+0000 7f6363152700 1 -- 172.21.15.33:0/1000993652 shutdown_connections 2024-05-08T21:17:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:14.934+0000 7f6363152700 1 -- 172.21.15.33:0/1000993652 wait complete. 2024-05-08T21:17:14.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:17:01.282803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:17:02.555552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T21:17:05.686277+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T21:17:14.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:17:15.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.088+0000 7fd4b2485700 1 -- 172.21.15.33:0/3899868569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac100f50 msgr2=0x7fd4ac1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.088+0000 7fd4b2485700 1 --2- 172.21.15.33:0/3899868569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac100f50 0x7fd4ac1013b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd494009990 tx=0x7fd494009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.088+0000 7fd4b2485700 1 -- 172.21.15.33:0/3899868569 shutdown_connections 2024-05-08T21:17:15.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.088+0000 7fd4b2485700 1 --2- 172.21.15.33:0/3899868569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4ac1018f0 0x7fd4ac1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.088+0000 7fd4b2485700 1 --2- 172.21.15.33:0/3899868569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4ac0ffd60 0x7fd4ac100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.088+0000 7fd4b2485700 1 --2- 172.21.15.33:0/3899868569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac100f50 0x7fd4ac1013b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.088+0000 7fd4b2485700 1 -- 172.21.15.33:0/3899868569 >> 172.21.15.33:0/3899868569 conn(0x7fd4ac0fb290 msgr2=0x7fd4ac0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:15.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.088+0000 7fd4b2485700 1 -- 172.21.15.33:0/3899868569 shutdown_connections 2024-05-08T21:17:15.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.088+0000 7fd4b2485700 1 -- 172.21.15.33:0/3899868569 wait complete. 2024-05-08T21:17:15.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.089+0000 7fd4b2485700 1 Processor -- start 2024-05-08T21:17:15.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.089+0000 7fd4b2485700 1 -- start start 2024-05-08T21:17:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.089+0000 7fd4b2485700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4ac0ffd60 0x7fd4ac068ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.089+0000 7fd4b2485700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac100f50 0x7fd4ac19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.089+0000 7fd4b2485700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4ac1018f0 0x7fd4ac19e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.089+0000 7fd4b2485700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac19e7b0 con 0x7fd4ac1018f0 2024-05-08T21:17:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.089+0000 7fd4b2485700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac19e920 con 0x7fd4ac0ffd60 2024-05-08T21:17:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.089+0000 7fd4b2485700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac19eae0 con 0x7fd4ac100f50 2024-05-08T21:17:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.089+0000 7fd4b0a22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4ac1018f0 0x7fd4ac19e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.089+0000 7fd4b0a22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4ac1018f0 0x7fd4ac19e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39036/0 (socket says 172.21.15.33:39036) 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.089+0000 7fd4b0a22700 1 -- 172.21.15.33:0/4115094139 learned_addr learned my addr 172.21.15.33:0/4115094139 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4abfff700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4ac0ffd60 0x7fd4ac068ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4ab7fe700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac100f50 0x7fd4ac19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4b0a22700 1 -- 172.21.15.33:0/4115094139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac100f50 msgr2=0x7fd4ac19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4b0a22700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac100f50 0x7fd4ac19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4b0a22700 1 -- 172.21.15.33:0/4115094139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4ac0ffd60 msgr2=0x7fd4ac068ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4b0a22700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4ac0ffd60 0x7fd4ac068ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4b0a22700 1 -- 172.21.15.33:0/4115094139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4940095f0 con 0x7fd4ac1018f0 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4abfff700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4ac0ffd60 0x7fd4ac068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4b0a22700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4ac1018f0 0x7fd4ac19e010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400ef30 tx=0x7fd4a400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4a97fa700 1 -- 172.21.15.33:0/4115094139 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4018070 con 0x7fd4ac1018f0 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4a97fa700 1 -- 172.21.15.33:0/4115094139 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4a400f040 con 0x7fd4ac1018f0 2024-05-08T21:17:15.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4b2485700 1 -- 172.21.15.33:0/4115094139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac19ed60 con 0x7fd4ac1018f0 2024-05-08T21:17:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4a97fa700 1 -- 172.21.15.33:0/4115094139 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4003d00 con 0x7fd4ac1018f0 2024-05-08T21:17:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.090+0000 7fd4b2485700 1 -- 172.21.15.33:0/4115094139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac19f1c0 con 0x7fd4ac1018f0 2024-05-08T21:17:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.091+0000 7fd4b2485700 1 -- 172.21.15.33:0/4115094139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ac107280 con 0x7fd4ac1018f0 2024-05-08T21:17:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.092+0000 7fd4a97fa700 1 -- 172.21.15.33:0/4115094139 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd4a400caa0 con 0x7fd4ac1018f0 2024-05-08T21:17:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.093+0000 7fd4a97fa700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4980a1120 0x7fd4980a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.093+0000 7fd4abfff700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4980a1120 0x7fd4980a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.093+0000 7fd4a97fa700 1 -- 172.21.15.33:0/4115094139 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd4a40cc860 con 0x7fd4ac1018f0 2024-05-08T21:17:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.094+0000 7fd4abfff700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4980a1120 0x7fd4980a35e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac100db0 tx=0x7fd49c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:15.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.098+0000 7fd4a97fa700 1 -- 172.21.15.33:0/4115094139 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd4a4095e30 con 0x7fd4ac1018f0 2024-05-08T21:17:15.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.318+0000 7fd4b2485700 1 -- 172.21.15.33:0/4115094139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4ac02ce40 con 0x7fd4ac1018f0 2024-05-08T21:17:15.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.319+0000 7fd4a97fa700 1 -- 172.21.15.33:0/4115094139 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4a4099450 con 0x7fd4ac1018f0 2024-05-08T21:17:15.321 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:17:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.321+0000 7fd4b2485700 1 -- 172.21.15.33:0/4115094139 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4980a1120 msgr2=0x7fd4980a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.321+0000 7fd4b2485700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4980a1120 0x7fd4980a35e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac100db0 tx=0x7fd49c009260 comp rx=0 tx=0).stop 2024-05-08T21:17:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.321+0000 7fd4b2485700 1 -- 172.21.15.33:0/4115094139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4ac1018f0 msgr2=0x7fd4ac19e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.321+0000 7fd4b2485700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4ac1018f0 0x7fd4ac19e010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400ef30 tx=0x7fd4a400c450 comp rx=0 tx=0).stop 2024-05-08T21:17:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.321+0000 7fd4b2485700 1 -- 172.21.15.33:0/4115094139 shutdown_connections 2024-05-08T21:17:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.321+0000 7fd4b2485700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4ac1018f0 0x7fd4ac19e010 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.321+0000 7fd4b2485700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4980a1120 0x7fd4980a35e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.321+0000 7fd4b2485700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4ac0ffd60 0x7fd4ac068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.321+0000 7fd4b2485700 1 --2- 172.21.15.33:0/4115094139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4ac100f50 0x7fd4ac19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.321+0000 7fd4b2485700 1 -- 172.21.15.33:0/4115094139 >> 172.21.15.33:0/4115094139 conn(0x7fd4ac0fb290 msgr2=0x7fd4ac1043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.321+0000 7fd4b2485700 1 -- 172.21.15.33:0/4115094139 shutdown_connections 2024-05-08T21:17:15.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.321+0000 7fd4b2485700 1 -- 172.21.15.33:0/4115094139 wait complete. 2024-05-08T21:17:15.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:17:15.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.476+0000 7fd5c81ac700 1 -- 172.21.15.33:0/962102950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c0100f50 msgr2=0x7fd5c01013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.476+0000 7fd5c81ac700 1 --2- 172.21.15.33:0/962102950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c0100f50 0x7fd5c01013b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00099c0 tx=0x7fd5b0009cd0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.476+0000 7fd5c81ac700 1 -- 172.21.15.33:0/962102950 shutdown_connections 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.476+0000 7fd5c81ac700 1 --2- 172.21.15.33:0/962102950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5c01018f0 0x7fd5c01081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.476+0000 7fd5c81ac700 1 --2- 172.21.15.33:0/962102950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5c00ffd60 0x7fd5c0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.476+0000 7fd5c81ac700 1 --2- 172.21.15.33:0/962102950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c0100f50 0x7fd5c01013b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.476+0000 7fd5c81ac700 1 -- 172.21.15.33:0/962102950 >> 172.21.15.33:0/962102950 conn(0x7fd5c00fb290 msgr2=0x7fd5c00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.476+0000 7fd5c81ac700 1 -- 172.21.15.33:0/962102950 shutdown_connections 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.476+0000 7fd5c81ac700 1 -- 172.21.15.33:0/962102950 wait complete. 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.476+0000 7fd5c81ac700 1 Processor -- start 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.476+0000 7fd5c81ac700 1 -- start start 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c81ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c00ffd60 0x7fd5c0103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c81ac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5c0100f50 0x7fd5c0102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:15.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c81ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5c01018f0 0x7fd5c0102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c81ac700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0103010 con 0x7fd5c00ffd60 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c81ac700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0103180 con 0x7fd5c0100f50 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c81ac700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c01a0d60 con 0x7fd5c01018f0 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c5f48700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c00ffd60 0x7fd5c0103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c5f48700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c00ffd60 0x7fd5c0103ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39050/0 (socket says 172.21.15.33:39050) 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c5f48700 1 -- 172.21.15.33:0/2593909853 learned_addr learned my addr 172.21.15.33:0/2593909853 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c6749700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5c01018f0 0x7fd5c0102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c5747700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5c0100f50 0x7fd5c0102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c5f48700 1 -- 172.21.15.33:0/2593909853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5c01018f0 msgr2=0x7fd5c0102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c5f48700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5c01018f0 0x7fd5c0102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c5f48700 1 -- 172.21.15.33:0/2593909853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5c0100f50 msgr2=0x7fd5c0102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c5f48700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5c0100f50 0x7fd5c0102330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c5f48700 1 -- 172.21.15.33:0/2593909853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b00095f0 con 0x7fd5c00ffd60 2024-05-08T21:17:15.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.477+0000 7fd5c5f48700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c00ffd60 0x7fd5c0103ca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00e970 tx=0x7fd5bc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:15.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.478+0000 7fd5b6ffd700 1 -- 172.21.15.33:0/2593909853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc009790 con 0x7fd5c00ffd60 2024-05-08T21:17:15.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.478+0000 7fd5b6ffd700 1 -- 172.21.15.33:0/2593909853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd5bc00c780 con 0x7fd5c00ffd60 2024-05-08T21:17:15.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.478+0000 7fd5c81ac700 1 -- 172.21.15.33:0/2593909853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c01a0ff0 con 0x7fd5c00ffd60 2024-05-08T21:17:15.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.478+0000 7fd5b6ffd700 1 -- 172.21.15.33:0/2593909853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc003b40 con 0x7fd5c00ffd60 2024-05-08T21:17:15.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.478+0000 7fd5c81ac700 1 -- 172.21.15.33:0/2593909853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c01a1410 con 0x7fd5c00ffd60 2024-05-08T21:17:15.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.479+0000 7fd5c81ac700 1 -- 172.21.15.33:0/2593909853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c0067d10 con 0x7fd5c00ffd60 2024-05-08T21:17:15.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.480+0000 7fd5b6ffd700 1 -- 172.21.15.33:0/2593909853 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd5bc00cbf0 con 0x7fd5c00ffd60 2024-05-08T21:17:15.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.481+0000 7fd5b6ffd700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd5ac0a1120 0x7fd5ac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:15.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.481+0000 7fd5c5747700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd5ac0a1120 0x7fd5ac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:15.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.481+0000 7fd5b6ffd700 1 -- 172.21.15.33:0/2593909853 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd5bc014070 con 0x7fd5c00ffd60 2024-05-08T21:17:15.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.481+0000 7fd5c5747700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd5ac0a1120 0x7fd5ac0a35e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009990 tx=0x7fd5b0003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:15.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.485+0000 7fd5b6ffd700 1 -- 172.21.15.33:0/2593909853 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd5bc0962e0 con 0x7fd5c00ffd60 2024-05-08T21:17:15.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:15 smithi071 ceph-mon[47074]: pgmap v234: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 909 B/s wr, 0 op/s 2024-05-08T21:17:15.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:15 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1000993652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:15.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:15 smithi082 ceph-mon[47512]: pgmap v234: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 909 B/s wr, 0 op/s 2024-05-08T21:17:15.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:15 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1000993652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.703+0000 7fd5c81ac700 1 -- 172.21.15.33:0/2593909853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5c01a1660 con 0x7fd5c00ffd60 2024-05-08T21:17:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.704+0000 7fd5b6ffd700 1 -- 172.21.15.33:0/2593909853 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5bc099900 con 0x7fd5c00ffd60 2024-05-08T21:17:15.706 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:17:15.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:15 smithi033 ceph-mon[42528]: pgmap v234: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 909 B/s wr, 0 op/s 2024-05-08T21:17:15.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:15 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1000993652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:15.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.706+0000 7fd5c81ac700 1 -- 172.21.15.33:0/2593909853 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd5ac0a1120 msgr2=0x7fd5ac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.706+0000 7fd5c81ac700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd5ac0a1120 0x7fd5ac0a35e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009990 tx=0x7fd5b0003680 comp rx=0 tx=0).stop 2024-05-08T21:17:15.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.706+0000 7fd5c81ac700 1 -- 172.21.15.33:0/2593909853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c00ffd60 msgr2=0x7fd5c0103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.706+0000 7fd5c81ac700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c00ffd60 0x7fd5c0103ca0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00e970 tx=0x7fd5bc00ec80 comp rx=0 tx=0).stop 2024-05-08T21:17:15.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.706+0000 7fd5c81ac700 1 -- 172.21.15.33:0/2593909853 shutdown_connections 2024-05-08T21:17:15.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.706+0000 7fd5c81ac700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5c01018f0 0x7fd5c0102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.706+0000 7fd5c81ac700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c00ffd60 0x7fd5c0103ca0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.706+0000 7fd5c81ac700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd5ac0a1120 0x7fd5ac0a35e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.706+0000 7fd5c81ac700 1 --2- 172.21.15.33:0/2593909853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5c0100f50 0x7fd5c0102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.706+0000 7fd5c81ac700 1 -- 172.21.15.33:0/2593909853 >> 172.21.15.33:0/2593909853 conn(0x7fd5c00fb290 msgr2=0x7fd5c0068190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.706+0000 7fd5c81ac700 1 -- 172.21.15.33:0/2593909853 shutdown_connections 2024-05-08T21:17:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.706+0000 7fd5c81ac700 1 -- 172.21.15.33:0/2593909853 wait complete. 2024-05-08T21:17:15.722 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:17:15.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.860+0000 7f34d4913700 1 -- 172.21.15.33:0/2773422877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0104270 msgr2=0x7f34d0106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.860+0000 7f34d4913700 1 --2- 172.21.15.33:0/2773422877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0104270 0x7f34d0106660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f34b80098f0 tx=0x7f34b8009c00 comp rx=0 tx=0).stop 2024-05-08T21:17:15.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.861+0000 7f34d4913700 1 -- 172.21.15.33:0/2773422877 shutdown_connections 2024-05-08T21:17:15.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.861+0000 7f34d4913700 1 --2- 172.21.15.33:0/2773422877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34d0106ba0 0x7f34d0108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.861+0000 7f34d4913700 1 --2- 172.21.15.33:0/2773422877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d0101940 0x7f34d0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.861+0000 7f34d4913700 1 --2- 172.21.15.33:0/2773422877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0104270 0x7f34d0106660 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.861+0000 7f34d4913700 1 -- 172.21.15.33:0/2773422877 >> 172.21.15.33:0/2773422877 conn(0x7f34d00fb290 msgr2=0x7f34d00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:15.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.861+0000 7f34d4913700 1 -- 172.21.15.33:0/2773422877 shutdown_connections 2024-05-08T21:17:15.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.861+0000 7f34d4913700 1 -- 172.21.15.33:0/2773422877 wait complete. 2024-05-08T21:17:15.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.861+0000 7f34d4913700 1 Processor -- start 2024-05-08T21:17:15.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34d4913700 1 -- start start 2024-05-08T21:17:15.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34d4913700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0101940 0x7f34d0196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:15.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34d4913700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d0104270 0x7f34d0197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:15.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34d4913700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34d0106ba0 0x7f34d019e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:15.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34d4913700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d019e890 con 0x7f34d0106ba0 2024-05-08T21:17:15.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34d4913700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d019ea00 con 0x7f34d0104270 2024-05-08T21:17:15.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34d4913700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d019ebc0 con 0x7f34d0101940 2024-05-08T21:17:15.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34ced9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34d0106ba0 0x7f34d019e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:15.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34ced9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34d0106ba0 0x7f34d019e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39068/0 (socket says 172.21.15.33:39068) 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34ced9d700 1 -- 172.21.15.33:0/2774216112 learned_addr learned my addr 172.21.15.33:0/2774216112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34ce59c700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0101940 0x7f34d0196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34c5d9b700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d0104270 0x7f34d0197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34ced9d700 1 -- 172.21.15.33:0/2774216112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0101940 msgr2=0x7f34d0196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34ced9d700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0101940 0x7f34d0196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34ced9d700 1 -- 172.21.15.33:0/2774216112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d0104270 msgr2=0x7f34d0197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34ced9d700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d0104270 0x7f34d0197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34ced9d700 1 -- 172.21.15.33:0/2774216112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b8009520 con 0x7f34d0106ba0 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.862+0000 7f34ce59c700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0101940 0x7f34d0196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.863+0000 7f34ced9d700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34d0106ba0 0x7f34d019e2c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f34c800c110 tx=0x7f34c800c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.863+0000 7f34c7fff700 1 -- 172.21.15.33:0/2774216112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8017070 con 0x7f34d0106ba0 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.863+0000 7f34c7fff700 1 -- 172.21.15.33:0/2774216112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34c800e050 con 0x7f34d0106ba0 2024-05-08T21:17:15.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.863+0000 7f34d4913700 1 -- 172.21.15.33:0/2774216112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d019ee40 con 0x7f34d0106ba0 2024-05-08T21:17:15.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.863+0000 7f34c7fff700 1 -- 172.21.15.33:0/2774216112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8007820 con 0x7f34d0106ba0 2024-05-08T21:17:15.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.863+0000 7f34d4913700 1 -- 172.21.15.33:0/2774216112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d019f210 con 0x7f34d0106ba0 2024-05-08T21:17:15.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.864+0000 7f34d4913700 1 -- 172.21.15.33:0/2774216112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d00fce10 con 0x7f34d0106ba0 2024-05-08T21:17:15.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.865+0000 7f34c7fff700 1 -- 172.21.15.33:0/2774216112 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f34c800ca20 con 0x7f34d0106ba0 2024-05-08T21:17:15.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.866+0000 7f34c7fff700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34bc0a1120 0x7f34bc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:15.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.866+0000 7f34c7fff700 1 -- 172.21.15.33:0/2774216112 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f34c80ccf70 con 0x7f34d0106ba0 2024-05-08T21:17:15.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.866+0000 7f34ce59c700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34bc0a1120 0x7f34bc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:15.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.866+0000 7f34ce59c700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34bc0a1120 0x7f34bc0a35e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f34d0198220 tx=0x7f34c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:15.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:15.870+0000 7f34c7fff700 1 -- 172.21.15.33:0/2774216112 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f34c80964b0 con 0x7f34d0106ba0 2024-05-08T21:17:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.087+0000 7f34d4913700 1 -- 172.21.15.33:0/2774216112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34d010c7b0 con 0x7f34d0106ba0 2024-05-08T21:17:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.087+0000 7f34c7fff700 1 -- 172.21.15.33:0/2774216112 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34c800ccd0 con 0x7f34d0106ba0 2024-05-08T21:17:16.090 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T21:17:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.090+0000 7f34d4913700 1 -- 172.21.15.33:0/2774216112 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34bc0a1120 msgr2=0x7f34bc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.090+0000 7f34d4913700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34bc0a1120 0x7f34bc0a35e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f34d0198220 tx=0x7f34c0009260 comp rx=0 tx=0).stop 2024-05-08T21:17:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.090+0000 7f34d4913700 1 -- 172.21.15.33:0/2774216112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34d0106ba0 msgr2=0x7f34d019e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.090+0000 7f34d4913700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34d0106ba0 0x7f34d019e2c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f34c800c110 tx=0x7f34c800c420 comp rx=0 tx=0).stop 2024-05-08T21:17:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.090+0000 7f34d4913700 1 -- 172.21.15.33:0/2774216112 shutdown_connections 2024-05-08T21:17:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.090+0000 7f34d4913700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34d0106ba0 0x7f34d019e2c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.090+0000 7f34d4913700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34d0101940 0x7f34d0196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.090+0000 7f34d4913700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34bc0a1120 0x7f34bc0a35e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.090+0000 7f34d4913700 1 --2- 172.21.15.33:0/2774216112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34d0104270 0x7f34d0197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.090+0000 7f34d4913700 1 -- 172.21.15.33:0/2774216112 >> 172.21.15.33:0/2774216112 conn(0x7f34d00fb290 msgr2=0x7f34d0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.090+0000 7f34d4913700 1 -- 172.21.15.33:0/2774216112 shutdown_connections 2024-05-08T21:17:16.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.090+0000 7f34d4913700 1 -- 172.21.15.33:0/2774216112 wait complete. 2024-05-08T21:17:16.107 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T21:17:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.245+0000 7f48ed06f700 1 -- 172.21.15.33:0/1543624438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e8101fc0 msgr2=0x7f48e8102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.245+0000 7f48ed06f700 1 --2- 172.21.15.33:0/1543624438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e8101fc0 0x7f48e8102440 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f48dc009990 tx=0x7f48dc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.245+0000 7f48ed06f700 1 -- 172.21.15.33:0/1543624438 shutdown_connections 2024-05-08T21:17:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.245+0000 7f48ed06f700 1 --2- 172.21.15.33:0/1543624438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48e8102980 0x7f48e81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.245+0000 7f48ed06f700 1 --2- 172.21.15.33:0/1543624438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e8100e70 0x7f48e8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.245+0000 7f48ed06f700 1 --2- 172.21.15.33:0/1543624438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e8101fc0 0x7f48e8102440 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.245+0000 7f48ed06f700 1 -- 172.21.15.33:0/1543624438 >> 172.21.15.33:0/1543624438 conn(0x7f48e80fc400 msgr2=0x7f48e80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:16.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.245+0000 7f48ed06f700 1 -- 172.21.15.33:0/1543624438 shutdown_connections 2024-05-08T21:17:16.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.245+0000 7f48ed06f700 1 -- 172.21.15.33:0/1543624438 wait complete. 2024-05-08T21:17:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.246+0000 7f48ed06f700 1 Processor -- start 2024-05-08T21:17:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.246+0000 7f48ed06f700 1 -- start start 2024-05-08T21:17:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.246+0000 7f48ed06f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e8100e70 0x7f48e8196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.246+0000 7f48ed06f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48e8101fc0 0x7f48e8197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.246+0000 7f48ed06f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e8102980 0x7f48e819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.246+0000 7f48ed06f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e819e7f0 con 0x7f48e8100e70 2024-05-08T21:17:16.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.246+0000 7f48ed06f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e819e960 con 0x7f48e8101fc0 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.246+0000 7f48ed06f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e819eb20 con 0x7f48e8102980 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.246+0000 7f48e6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e8100e70 0x7f48e8196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.246+0000 7f48e6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e8100e70 0x7f48e8196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39098/0 (socket says 172.21.15.33:39098) 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.246+0000 7f48e6d9d700 1 -- 172.21.15.33:0/3342131223 learned_addr learned my addr 172.21.15.33:0/3342131223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48e659c700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48e8101fc0 0x7f48e8197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48e759e700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e8102980 0x7f48e819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48e6d9d700 1 -- 172.21.15.33:0/3342131223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e8102980 msgr2=0x7f48e819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48e6d9d700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e8102980 0x7f48e819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48e6d9d700 1 -- 172.21.15.33:0/3342131223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48e8101fc0 msgr2=0x7f48e8197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48e6d9d700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48e8101fc0 0x7f48e8197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48e6d9d700 1 -- 172.21.15.33:0/3342131223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc0095f0 con 0x7f48e8100e70 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48e6d9d700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e8100e70 0x7f48e8196cd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f48d800d950 tx=0x7f48d800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48cffff700 1 -- 172.21.15.33:0/3342131223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8009940 con 0x7f48e8100e70 2024-05-08T21:17:16.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48cffff700 1 -- 172.21.15.33:0/3342131223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48d8010460 con 0x7f48e8100e70 2024-05-08T21:17:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48ed06f700 1 -- 172.21.15.33:0/3342131223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e819ee00 con 0x7f48e8100e70 2024-05-08T21:17:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48cffff700 1 -- 172.21.15.33:0/3342131223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d800f620 con 0x7f48e8100e70 2024-05-08T21:17:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.247+0000 7f48ed06f700 1 -- 172.21.15.33:0/3342131223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e819f280 con 0x7f48e8100e70 2024-05-08T21:17:16.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.248+0000 7f48ed06f700 1 -- 172.21.15.33:0/3342131223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e8108380 con 0x7f48e8100e70 2024-05-08T21:17:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.250+0000 7f48cffff700 1 -- 172.21.15.33:0/3342131223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f48d80105d0 con 0x7f48e8100e70 2024-05-08T21:17:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.251+0000 7f48cffff700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48d00a11a0 0x7f48d00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.251+0000 7f48e659c700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48d00a11a0 0x7f48d00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.251+0000 7f48cffff700 1 -- 172.21.15.33:0/3342131223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f48d80cc8d0 con 0x7f48e8100e70 2024-05-08T21:17:16.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.251+0000 7f48e659c700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48d00a11a0 0x7f48d00a3660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f48dc006010 tx=0x7f48dc00b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:16.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.255+0000 7f48cffff700 1 -- 172.21.15.33:0/3342131223 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f48d8095e10 con 0x7f48e8100e70 2024-05-08T21:17:16.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:16 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4115094139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:17:16.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:16 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2593909853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:17:16.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:16 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2774216112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:17:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.474+0000 7f48ed06f700 1 -- 172.21.15.33:0/3342131223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f48e810c6b0 con 0x7f48e8100e70 2024-05-08T21:17:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.474+0000 7f48cffff700 1 -- 172.21.15.33:0/3342131223 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7f48d80200a0 con 0x7f48e8100e70 2024-05-08T21:17:16.477 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:17:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.477+0000 7f48ed06f700 1 -- 172.21.15.33:0/3342131223 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48d00a11a0 msgr2=0x7f48d00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.477+0000 7f48ed06f700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48d00a11a0 0x7f48d00a3660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f48dc006010 tx=0x7f48dc00b5d0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.477+0000 7f48ed06f700 1 -- 172.21.15.33:0/3342131223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e8100e70 msgr2=0x7f48e8196cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.477+0000 7f48ed06f700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e8100e70 0x7f48e8196cd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f48d800d950 tx=0x7f48d800dc60 comp rx=0 tx=0).stop 2024-05-08T21:17:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.477+0000 7f48ed06f700 1 -- 172.21.15.33:0/3342131223 shutdown_connections 2024-05-08T21:17:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.477+0000 7f48ed06f700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48e8102980 0x7f48e819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.477+0000 7f48ed06f700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48e8100e70 0x7f48e8196cd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.477+0000 7f48ed06f700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48d00a11a0 0x7f48d00a3660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.477+0000 7f48ed06f700 1 --2- 172.21.15.33:0/3342131223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48e8101fc0 0x7f48e8197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.477+0000 7f48ed06f700 1 -- 172.21.15.33:0/3342131223 >> 172.21.15.33:0/3342131223 conn(0x7f48e80fc400 msgr2=0x7f48e81054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.477+0000 7f48ed06f700 1 -- 172.21.15.33:0/3342131223 shutdown_connections 2024-05-08T21:17:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.477+0000 7f48ed06f700 1 -- 172.21.15.33:0/3342131223 wait complete. 2024-05-08T21:17:16.493 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T21:17:16.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:16 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/4115094139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:17:16.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:16 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2593909853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:17:16.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:16 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2774216112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:17:16.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:16 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/4115094139' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:17:16.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:16 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2593909853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:17:16.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:16 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2774216112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:17:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.628+0000 7f34f2934700 1 -- 172.21.15.33:0/2389322802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ec1018e0 msgr2=0x7f34ec103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.628+0000 7f34f2934700 1 --2- 172.21.15.33:0/2389322802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ec1018e0 0x7f34ec103cd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009990 tx=0x7f34dc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.628+0000 7f34f2934700 1 -- 172.21.15.33:0/2389322802 shutdown_connections 2024-05-08T21:17:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.628+0000 7f34f2934700 1 --2- 172.21.15.33:0/2389322802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ec106b40 0x7f34ec108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.628+0000 7f34f2934700 1 --2- 172.21.15.33:0/2389322802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ec1018e0 0x7f34ec103cd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.628+0000 7f34f2934700 1 --2- 172.21.15.33:0/2389322802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34ec104210 0x7f34ec106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.628+0000 7f34f2934700 1 -- 172.21.15.33:0/2389322802 >> 172.21.15.33:0/2389322802 conn(0x7f34ec0fb290 msgr2=0x7f34ec0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.628+0000 7f34f2934700 1 -- 172.21.15.33:0/2389322802 shutdown_connections 2024-05-08T21:17:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.628+0000 7f34f2934700 1 -- 172.21.15.33:0/2389322802 wait complete. 2024-05-08T21:17:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.629+0000 7f34f2934700 1 Processor -- start 2024-05-08T21:17:16.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.629+0000 7f34f2934700 1 -- start start 2024-05-08T21:17:16.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34f2934700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34ec1018e0 0x7f34ec196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34f2934700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ec104210 0x7f34ec197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34f2934700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ec106b40 0x7f34ec19e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34f2934700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec19e7e0 con 0x7f34ec104210 2024-05-08T21:17:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34f2934700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec19e950 con 0x7f34ec1018e0 2024-05-08T21:17:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34f2934700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec19eb10 con 0x7f34ec106b40 2024-05-08T21:17:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34eb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ec104210 0x7f34ec197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34eb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ec104210 0x7f34ec197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39114/0 (socket says 172.21.15.33:39114) 2024-05-08T21:17:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34eb7fe700 1 -- 172.21.15.33:0/2361196174 learned_addr learned my addr 172.21.15.33:0/2361196174 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34ebfff700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34ec1018e0 0x7f34ec196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:16.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34f0ed1700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ec106b40 0x7f34ec19e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34eb7fe700 1 -- 172.21.15.33:0/2361196174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ec106b40 msgr2=0x7f34ec19e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34eb7fe700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ec106b40 0x7f34ec19e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34eb7fe700 1 -- 172.21.15.33:0/2361196174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34ec1018e0 msgr2=0x7f34ec196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34eb7fe700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34ec1018e0 0x7f34ec196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34eb7fe700 1 -- 172.21.15.33:0/2361196174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e0009520 con 0x7f34ec104210 2024-05-08T21:17:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34eb7fe700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ec104210 0x7f34ec197200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f34e000eb30 tx=0x7f34e000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34e97fa700 1 -- 172.21.15.33:0/2361196174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e0009880 con 0x7f34ec104210 2024-05-08T21:17:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34f2934700 1 -- 172.21.15.33:0/2361196174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34dc0095f0 con 0x7f34ec104210 2024-05-08T21:17:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34e97fa700 1 -- 172.21.15.33:0/2361196174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34e00044b0 con 0x7f34ec104210 2024-05-08T21:17:16.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.630+0000 7f34e97fa700 1 -- 172.21.15.33:0/2361196174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e0003df0 con 0x7f34ec104210 2024-05-08T21:17:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.631+0000 7f34f2934700 1 -- 172.21.15.33:0/2361196174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec19f170 con 0x7f34ec104210 2024-05-08T21:17:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.631+0000 7f34f2934700 1 -- 172.21.15.33:0/2361196174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec0fce10 con 0x7f34ec104210 2024-05-08T21:17:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.633+0000 7f34e97fa700 1 -- 172.21.15.33:0/2361196174 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f34e0003f50 con 0x7f34ec104210 2024-05-08T21:17:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.633+0000 7f34e97fa700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34d40a1120 0x7f34d40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:16.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.633+0000 7f34e97fa700 1 -- 172.21.15.33:0/2361196174 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f34e0014070 con 0x7f34ec104210 2024-05-08T21:17:16.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.633+0000 7f34ebfff700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34d40a1120 0x7f34d40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:16.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.634+0000 7f34ebfff700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34d40a1120 0x7f34d40a35e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f34ec103f90 tx=0x7f34dc00b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:16.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.638+0000 7f34e97fa700 1 -- 172.21.15.33:0/2361196174 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f34e00965c0 con 0x7f34ec104210 2024-05-08T21:17:16.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.871+0000 7f34f2934700 1 -- 172.21.15.33:0/2361196174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f34ec067d10 con 0x7f34ec104210 2024-05-08T21:17:16.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.871+0000 7f34e97fa700 1 -- 172.21.15.33:0/2361196174 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) v1 ==== 81+0+592 (secure 0 0 0) 0x7f34e0099be0 con 0x7f34ec104210 2024-05-08T21:17:16.874 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3156563197 2024-05-09T21:11:56.047045+0000 2024-05-08T21:17:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3413344235 2024-05-09T21:11:56.047045+0000 2024-05-08T21:17:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3413344235 2024-05-09T21:11:56.047045+0000 2024-05-08T21:17:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1762147139 2024-05-09T21:11:56.047045+0000 2024-05-08T21:17:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2047278949 2024-05-09T21:11:32.156884+0000 2024-05-08T21:17:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1002276836 2024-05-09T21:11:32.156884+0000 2024-05-08T21:17:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3475366293 2024-05-09T21:11:32.156884+0000 2024-05-08T21:17:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3475366293 2024-05-09T21:11:32.156884+0000 2024-05-08T21:17:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1708721255 2024-05-09T21:11:32.156884+0000 2024-05-08T21:17:16.875 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3864964868 2024-05-09T21:11:56.047045+0000 2024-05-08T21:17:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.874+0000 7f34f2934700 1 -- 172.21.15.33:0/2361196174 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34d40a1120 msgr2=0x7f34d40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:16.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.874+0000 7f34f2934700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34d40a1120 0x7f34d40a35e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f34ec103f90 tx=0x7f34dc00b610 comp rx=0 tx=0).stop 2024-05-08T21:17:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.874+0000 7f34f2934700 1 -- 172.21.15.33:0/2361196174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ec104210 msgr2=0x7f34ec197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.874+0000 7f34f2934700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ec104210 0x7f34ec197200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f34e000eb30 tx=0x7f34e000ee40 comp rx=0 tx=0).stop 2024-05-08T21:17:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.874+0000 7f34f2934700 1 -- 172.21.15.33:0/2361196174 shutdown_connections 2024-05-08T21:17:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.874+0000 7f34f2934700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34ec106b40 0x7f34ec19e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.874+0000 7f34f2934700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34d40a1120 0x7f34d40a35e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.874+0000 7f34f2934700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34ec1018e0 0x7f34ec196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.874+0000 7f34f2934700 1 --2- 172.21.15.33:0/2361196174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ec104210 0x7f34ec197200 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.874+0000 7f34f2934700 1 -- 172.21.15.33:0/2361196174 >> 172.21.15.33:0/2361196174 conn(0x7f34ec0fb290 msgr2=0x7f34ec102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.874+0000 7f34f2934700 1 -- 172.21.15.33:0/2361196174 shutdown_connections 2024-05-08T21:17:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:16.874+0000 7f34f2934700 1 -- 172.21.15.33:0/2361196174 wait complete. 2024-05-08T21:17:16.877 INFO:teuthology.orchestra.run.smithi033.stderr:listed 10 entries 2024-05-08T21:17:16.890 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:17:16.890 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T21:17:16.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:17:17.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.088+0000 7f066fa71700 1 -- 172.21.15.33:0/2528673912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f066810d450 msgr2=0x7f066810f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.088+0000 7f066fa71700 1 --2- 172.21.15.33:0/2528673912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f066810d450 0x7f066810f840 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f066000b0a0 tx=0x7f066000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.088+0000 7f066fa71700 1 -- 172.21.15.33:0/2528673912 shutdown_connections 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.088+0000 7f066fa71700 1 --2- 172.21.15.33:0/2528673912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f066810d450 0x7f066810f840 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.088+0000 7f066fa71700 1 --2- 172.21.15.33:0/2528673912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0668074fe0 0x7f0668073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.088+0000 7f066fa71700 1 --2- 172.21.15.33:0/2528673912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0668073a00 0x7f0668073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.088+0000 7f066fa71700 1 -- 172.21.15.33:0/2528673912 >> 172.21.15.33:0/2528673912 conn(0x7f06680fc410 msgr2=0x7f06680fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.088+0000 7f066fa71700 1 -- 172.21.15.33:0/2528673912 shutdown_connections 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.088+0000 7f066fa71700 1 -- 172.21.15.33:0/2528673912 wait complete. 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066fa71700 1 Processor -- start 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066fa71700 1 -- start start 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066fa71700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0668073a00 0x7f066819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066fa71700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0668074fe0 0x7f066819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066fa71700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f066810d450 0x7f06681a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066fa71700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06681a4f30 con 0x7f0668074fe0 2024-05-08T21:17:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066fa71700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06681a5070 con 0x7f066810d450 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066fa71700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06681a5200 con 0x7f0668073a00 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066d00c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0668074fe0 0x7f066819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066d00c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0668074fe0 0x7f066819d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39120/0 (socket says 172.21.15.33:39120) 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066d00c700 1 -- 172.21.15.33:0/1055970236 learned_addr learned my addr 172.21.15.33:0/1055970236 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066e00e700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f066810d450 0x7f06681a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066d80d700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0668073a00 0x7f066819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066d00c700 1 -- 172.21.15.33:0/1055970236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0668073a00 msgr2=0x7f066819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066d00c700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0668073a00 0x7f066819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066d00c700 1 -- 172.21.15.33:0/1055970236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f066810d450 msgr2=0x7f06681a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066d00c700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f066810d450 0x7f06681a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066d00c700 1 -- 172.21.15.33:0/1055970236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0660009d00 con 0x7f0668074fe0 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.089+0000 7f066d00c700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0668074fe0 0x7f066819d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f065800d980 tx=0x7f065800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.090+0000 7f065effd700 1 -- 172.21.15.33:0/1055970236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658004020 con 0x7f0668074fe0 2024-05-08T21:17:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.090+0000 7f065effd700 1 -- 172.21.15.33:0/1055970236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f065800f690 con 0x7f0668074fe0 2024-05-08T21:17:17.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.090+0000 7f066fa71700 1 -- 172.21.15.33:0/1055970236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06681a54e0 con 0x7f0668074fe0 2024-05-08T21:17:17.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.090+0000 7f065effd700 1 -- 172.21.15.33:0/1055970236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06580039e0 con 0x7f0668074fe0 2024-05-08T21:17:17.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.090+0000 7f066fa71700 1 -- 172.21.15.33:0/1055970236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06681a5a40 con 0x7f0668074fe0 2024-05-08T21:17:17.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.091+0000 7f066fa71700 1 -- 172.21.15.33:0/1055970236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06681036b0 con 0x7f0668074fe0 2024-05-08T21:17:17.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.092+0000 7f065effd700 1 -- 172.21.15.33:0/1055970236 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f065800f800 con 0x7f0668074fe0 2024-05-08T21:17:17.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.093+0000 7f065effd700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f06540a11a0 0x7f06540a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:17.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.093+0000 7f066d80d700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f06540a11a0 0x7f06540a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:17.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.093+0000 7f065effd700 1 -- 172.21.15.33:0/1055970236 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f06580cc660 con 0x7f0668074fe0 2024-05-08T21:17:17.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.093+0000 7f066d80d700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f06540a11a0 0x7f06540a3660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0664009c60 tx=0x7f0664009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:17.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.098+0000 7f065effd700 1 -- 172.21.15.33:0/1055970236 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0658095ba0 con 0x7f0668074fe0 2024-05-08T21:17:17.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.322+0000 7f066fa71700 1 -- 172.21.15.33:0/1055970236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0668068160 con 0x7f0668074fe0 2024-05-08T21:17:17.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.323+0000 7f065effd700 1 -- 172.21.15.33:0/1055970236 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23902 (secure 0 0 0) 0x7f06580991c0 con 0x7f0668074fe0 2024-05-08T21:17:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:17.326 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:07.709538+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:17:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.326+0000 7f066fa71700 1 -- 172.21.15.33:0/1055970236 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f06540a11a0 msgr2=0x7f06540a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.326+0000 7f066fa71700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f06540a11a0 0x7f06540a3660 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0664009c60 tx=0x7f0664009260 comp rx=0 tx=0).stop 2024-05-08T21:17:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.326+0000 7f066fa71700 1 -- 172.21.15.33:0/1055970236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0668074fe0 msgr2=0x7f066819d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.326+0000 7f066fa71700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0668074fe0 0x7f066819d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f065800d980 tx=0x7f065800dc90 comp rx=0 tx=0).stop 2024-05-08T21:17:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.326+0000 7f066fa71700 1 -- 172.21.15.33:0/1055970236 shutdown_connections 2024-05-08T21:17:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.326+0000 7f066fa71700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f06540a11a0 0x7f06540a3660 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.326+0000 7f066fa71700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f066810d450 0x7f06681a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.326+0000 7f066fa71700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0668073a00 0x7f066819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.326+0000 7f066fa71700 1 --2- 172.21.15.33:0/1055970236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0668074fe0 0x7f066819d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.326+0000 7f066fa71700 1 -- 172.21.15.33:0/1055970236 >> 172.21.15.33:0/1055970236 conn(0x7f06680fc410 msgr2=0x7f06681007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.326+0000 7f066fa71700 1 -- 172.21.15.33:0/1055970236 shutdown_connections 2024-05-08T21:17:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.326+0000 7f066fa71700 1 -- 172.21.15.33:0/1055970236 wait complete. 2024-05-08T21:17:17.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:17:01.282803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:17:02.555552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T21:17:05.686277+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T21:17:17.342 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:17:17.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.480+0000 7f54fc684700 1 -- 172.21.15.33:0/2346160910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f4073980 msgr2=0x7f54f4073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.480+0000 7f54fc684700 1 --2- 172.21.15.33:0/2346160910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f4073980 0x7f54f4073e00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009990 tx=0x7f54e4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.480+0000 7f54fc684700 1 -- 172.21.15.33:0/2346160910 shutdown_connections 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.480+0000 7f54fc684700 1 --2- 172.21.15.33:0/2346160910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4108d60 0x7f54f410b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.480+0000 7f54fc684700 1 --2- 172.21.15.33:0/2346160910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f4074f60 0x7f54f40733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.480+0000 7f54fc684700 1 --2- 172.21.15.33:0/2346160910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f4073980 0x7f54f4073e00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.480+0000 7f54fc684700 1 -- 172.21.15.33:0/2346160910 >> 172.21.15.33:0/2346160910 conn(0x7f54f40fc400 msgr2=0x7f54f40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.481+0000 7f54fc684700 1 -- 172.21.15.33:0/2346160910 shutdown_connections 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.481+0000 7f54fc684700 1 -- 172.21.15.33:0/2346160910 wait complete. 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.481+0000 7f54fc684700 1 Processor -- start 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.481+0000 7f54fc684700 1 -- start start 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.481+0000 7f54fc684700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f4073980 0x7f54f4198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.481+0000 7f54fc684700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4074f60 0x7f54f4199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.481+0000 7f54fc684700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f4108d60 0x7f54f41a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.481+0000 7f54fc684700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f41a0a10 con 0x7f54f4108d60 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fc684700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f41a0b50 con 0x7f54f4074f60 2024-05-08T21:17:17.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fc684700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f41a0ce0 con 0x7f54f4073980 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fac21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f4108d60 0x7f54f41a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fac21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f4108d60 0x7f54f41a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39128/0 (socket says 172.21.15.33:39128) 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fac21700 1 -- 172.21.15.33:0/22084315 learned_addr learned my addr 172.21.15.33:0/22084315 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54f9c1f700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4074f60 0x7f54f4199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fa420700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f4073980 0x7f54f4198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fac21700 1 -- 172.21.15.33:0/22084315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f4073980 msgr2=0x7f54f4198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fac21700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f4073980 0x7f54f4198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fac21700 1 -- 172.21.15.33:0/22084315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4074f60 msgr2=0x7f54f4199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fac21700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4074f60 0x7f54f4199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fac21700 1 -- 172.21.15.33:0/22084315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40095f0 con 0x7f54f4108d60 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54f9c1f700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4074f60 0x7f54f4199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fac21700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f4108d60 0x7f54f41a0440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00ef30 tx=0x7f54ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54eb7fe700 1 -- 172.21.15.33:0/22084315 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec018070 con 0x7f54f4108d60 2024-05-08T21:17:17.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54eb7fe700 1 -- 172.21.15.33:0/22084315 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54ec00f040 con 0x7f54f4108d60 2024-05-08T21:17:17.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fc684700 1 -- 172.21.15.33:0/22084315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f41a0f60 con 0x7f54f4108d60 2024-05-08T21:17:17.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54eb7fe700 1 -- 172.21.15.33:0/22084315 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00cbc0 con 0x7f54f4108d60 2024-05-08T21:17:17.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.482+0000 7f54fc684700 1 -- 172.21.15.33:0/22084315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f41a13e0 con 0x7f54f4108d60 2024-05-08T21:17:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.483+0000 7f54fc684700 1 -- 172.21.15.33:0/22084315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f4107b70 con 0x7f54f4108d60 2024-05-08T21:17:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.485+0000 7f54eb7fe700 1 -- 172.21.15.33:0/22084315 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f54ec00cd80 con 0x7f54f4108d60 2024-05-08T21:17:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.485+0000 7f54eb7fe700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f54e00a11a0 0x7f54e00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.485+0000 7f54fa420700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f54e00a11a0 0x7f54e00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.486+0000 7f54eb7fe700 1 -- 172.21.15.33:0/22084315 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f54ec0ccac0 con 0x7f54f4108d60 2024-05-08T21:17:17.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.486+0000 7f54fa420700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f54e00a11a0 0x7f54e00a3660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f54f0005950 tx=0x7f54f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.490+0000 7f54eb7fe700 1 -- 172.21.15.33:0/22084315 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f54ec0960f0 con 0x7f54f4108d60 2024-05-08T21:17:17.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:17 smithi071 ceph-mon[47074]: pgmap v235: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T21:17:17.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:17 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3342131223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:17:17.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:17 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2361196174' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T21:17:17.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:17 smithi082 ceph-mon[47512]: pgmap v235: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T21:17:17.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:17 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3342131223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:17:17.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:17 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2361196174' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T21:17:17.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:17 smithi033 ceph-mon[42528]: pgmap v235: 129 pgs: 129 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T21:17:17.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:17 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3342131223' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:17:17.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:17 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2361196174' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T21:17:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.710+0000 7f54fc684700 1 -- 172.21.15.33:0/22084315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54f410e710 con 0x7f54f4108d60 2024-05-08T21:17:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.710+0000 7f54eb7fe700 1 -- 172.21.15.33:0/22084315 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54ec099710 con 0x7f54f4108d60 2024-05-08T21:17:17.713 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:17:17.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.713+0000 7f54fc684700 1 -- 172.21.15.33:0/22084315 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f54e00a11a0 msgr2=0x7f54e00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.713+0000 7f54fc684700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f54e00a11a0 0x7f54e00a3660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f54f0005950 tx=0x7f54f000b410 comp rx=0 tx=0).stop 2024-05-08T21:17:17.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.713+0000 7f54fc684700 1 -- 172.21.15.33:0/22084315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f4108d60 msgr2=0x7f54f41a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.713+0000 7f54fc684700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f4108d60 0x7f54f41a0440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00ef30 tx=0x7f54ec00c3a0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.713+0000 7f54fc684700 1 -- 172.21.15.33:0/22084315 shutdown_connections 2024-05-08T21:17:17.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.713+0000 7f54fc684700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54f4108d60 0x7f54f41a0440 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.713+0000 7f54fc684700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54f4073980 0x7f54f4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.713+0000 7f54fc684700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f54e00a11a0 0x7f54e00a3660 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.713+0000 7f54fc684700 1 --2- 172.21.15.33:0/22084315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4074f60 0x7f54f4199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.713+0000 7f54fc684700 1 -- 172.21.15.33:0/22084315 >> 172.21.15.33:0/22084315 conn(0x7f54f40fc400 msgr2=0x7f54f4104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:17.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.713+0000 7f54fc684700 1 -- 172.21.15.33:0/22084315 shutdown_connections 2024-05-08T21:17:17.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.713+0000 7f54fc684700 1 -- 172.21.15.33:0/22084315 wait complete. 2024-05-08T21:17:17.729 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:17:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.866+0000 7ff903bbb700 1 -- 172.21.15.33:0/1243109397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8fc101fc0 msgr2=0x7ff8fc102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.866+0000 7ff903bbb700 1 --2- 172.21.15.33:0/1243109397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8fc101fc0 0x7ff8fc102440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009990 tx=0x7ff8f8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.866+0000 7ff903bbb700 1 -- 172.21.15.33:0/1243109397 shutdown_connections 2024-05-08T21:17:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.866+0000 7ff903bbb700 1 --2- 172.21.15.33:0/1243109397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc102980 0x7ff8fc1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.866+0000 7ff903bbb700 1 --2- 172.21.15.33:0/1243109397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8fc100e70 0x7ff8fc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.866+0000 7ff903bbb700 1 --2- 172.21.15.33:0/1243109397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8fc101fc0 0x7ff8fc102440 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.866+0000 7ff903bbb700 1 -- 172.21.15.33:0/1243109397 >> 172.21.15.33:0/1243109397 conn(0x7ff8fc0fc400 msgr2=0x7ff8fc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:17.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.866+0000 7ff903bbb700 1 -- 172.21.15.33:0/1243109397 shutdown_connections 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.866+0000 7ff903bbb700 1 -- 172.21.15.33:0/1243109397 wait complete. 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff903bbb700 1 Processor -- start 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff903bbb700 1 -- start start 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff903bbb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8fc100e70 0x7ff8fc198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff903bbb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8fc101fc0 0x7ff8fc199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff903bbb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc102980 0x7ff8fc1a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff903bbb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc1a0a70 con 0x7ff8fc101fc0 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff903bbb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc1a0bb0 con 0x7ff8fc100e70 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff903bbb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc1a0d40 con 0x7ff8fc102980 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff901156700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8fc101fc0 0x7ff8fc199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff901156700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8fc101fc0 0x7ff8fc199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39148/0 (socket says 172.21.15.33:39148) 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff901156700 1 -- 172.21.15.33:0/3481047984 learned_addr learned my addr 172.21.15.33:0/3481047984 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff902158700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc102980 0x7ff8fc1a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.867+0000 7ff901957700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8fc100e70 0x7ff8fc198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:17.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.868+0000 7ff901156700 1 -- 172.21.15.33:0/3481047984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc102980 msgr2=0x7ff8fc1a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.868+0000 7ff901156700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc102980 0x7ff8fc1a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.868+0000 7ff901156700 1 -- 172.21.15.33:0/3481047984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8fc100e70 msgr2=0x7ff8fc198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.868+0000 7ff901156700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8fc100e70 0x7ff8fc198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.868+0000 7ff901156700 1 -- 172.21.15.33:0/3481047984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f80095f0 con 0x7ff8fc101fc0 2024-05-08T21:17:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.868+0000 7ff901156700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8fc101fc0 0x7ff8fc199480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800b5c0 tx=0x7ff8f8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.868+0000 7ff8eeffd700 1 -- 172.21.15.33:0/3481047984 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f801d070 con 0x7ff8fc101fc0 2024-05-08T21:17:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.868+0000 7ff8eeffd700 1 -- 172.21.15.33:0/3481047984 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8f8003ec0 con 0x7ff8fc101fc0 2024-05-08T21:17:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.868+0000 7ff903bbb700 1 -- 172.21.15.33:0/3481047984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc1a0fc0 con 0x7ff8fc101fc0 2024-05-08T21:17:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.868+0000 7ff8eeffd700 1 -- 172.21.15.33:0/3481047984 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8022b10 con 0x7ff8fc101fc0 2024-05-08T21:17:17.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.868+0000 7ff903bbb700 1 -- 172.21.15.33:0/3481047984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc1a13e0 con 0x7ff8fc101fc0 2024-05-08T21:17:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.869+0000 7ff903bbb700 1 -- 172.21.15.33:0/3481047984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc067480 con 0x7ff8fc101fc0 2024-05-08T21:17:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.870+0000 7ff8eeffd700 1 -- 172.21.15.33:0/3481047984 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff8f8004030 con 0x7ff8fc101fc0 2024-05-08T21:17:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.871+0000 7ff8eeffd700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff8e80a11a0 0x7ff8e80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.871+0000 7ff901957700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff8e80a11a0 0x7ff8e80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.871+0000 7ff8eeffd700 1 -- 172.21.15.33:0/3481047984 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff8f80ce710 con 0x7ff8fc101fc0 2024-05-08T21:17:17.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.872+0000 7ff901957700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff8e80a11a0 0x7ff8e80a3660 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0007c60 tx=0x7ff8f00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:17.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:17.875+0000 7ff8eeffd700 1 -- 172.21.15.33:0/3481047984 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff8f8097ce0 con 0x7ff8fc101fc0 2024-05-08T21:17:18.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.093+0000 7ff903bbb700 1 -- 172.21.15.33:0/3481047984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8fc10c570 con 0x7ff8fc101fc0 2024-05-08T21:17:18.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.094+0000 7ff8eeffd700 1 -- 172.21.15.33:0/3481047984 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff8f809b300 con 0x7ff8fc101fc0 2024-05-08T21:17:18.096 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:17:18.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.096+0000 7ff903bbb700 1 -- 172.21.15.33:0/3481047984 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff8e80a11a0 msgr2=0x7ff8e80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:18.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.096+0000 7ff903bbb700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff8e80a11a0 0x7ff8e80a3660 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0007c60 tx=0x7ff8f00073d0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.096+0000 7ff903bbb700 1 -- 172.21.15.33:0/3481047984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8fc101fc0 msgr2=0x7ff8fc199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:18.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.096+0000 7ff903bbb700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8fc101fc0 0x7ff8fc199480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800b5c0 tx=0x7ff8f8005740 comp rx=0 tx=0).stop 2024-05-08T21:17:18.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.096+0000 7ff903bbb700 1 -- 172.21.15.33:0/3481047984 shutdown_connections 2024-05-08T21:17:18.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.096+0000 7ff903bbb700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc102980 0x7ff8fc1a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.096+0000 7ff903bbb700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff8e80a11a0 0x7ff8e80a3660 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.096+0000 7ff903bbb700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8fc100e70 0x7ff8fc198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.096+0000 7ff903bbb700 1 --2- 172.21.15.33:0/3481047984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8fc101fc0 0x7ff8fc199480 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.096+0000 7ff903bbb700 1 -- 172.21.15.33:0/3481047984 >> 172.21.15.33:0/3481047984 conn(0x7ff8fc0fc400 msgr2=0x7ff8fc107820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:18.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.096+0000 7ff903bbb700 1 -- 172.21.15.33:0/3481047984 shutdown_connections 2024-05-08T21:17:18.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.096+0000 7ff903bbb700 1 -- 172.21.15.33:0/3481047984 wait complete. 2024-05-08T21:17:18.112 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:17:18.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.249+0000 7f7666166700 1 -- 172.21.15.33:0/2650732283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660102990 msgr2=0x7f76601092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:18.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.249+0000 7f7666166700 1 --2- 172.21.15.33:0/2650732283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660102990 0x7f76601092b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f765800b0a0 tx=0x7f765800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.249+0000 7f7666166700 1 -- 172.21.15.33:0/2650732283 shutdown_connections 2024-05-08T21:17:18.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.249+0000 7f7666166700 1 --2- 172.21.15.33:0/2650732283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660102990 0x7f76601092b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.249+0000 7f7666166700 1 --2- 172.21.15.33:0/2650732283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7660100e80 0x7f76601012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.249+0000 7f7666166700 1 --2- 172.21.15.33:0/2650732283 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7660101fd0 0x7f7660102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.249+0000 7f7666166700 1 -- 172.21.15.33:0/2650732283 >> 172.21.15.33:0/2650732283 conn(0x7f76600fc3f0 msgr2=0x7f76600fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.249+0000 7f7666166700 1 -- 172.21.15.33:0/2650732283 shutdown_connections 2024-05-08T21:17:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.249+0000 7f7666166700 1 -- 172.21.15.33:0/2650732283 wait complete. 2024-05-08T21:17:18.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.250+0000 7f7666166700 1 Processor -- start 2024-05-08T21:17:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.250+0000 7f7666166700 1 -- start start 2024-05-08T21:17:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.250+0000 7f7666166700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660100e80 0x7f7660196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.250+0000 7f7666166700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7660101fd0 0x7f7660197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.250+0000 7f7666166700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7660102990 0x7f766019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:18.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.250+0000 7f7666166700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766019e900 con 0x7f7660100e80 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.250+0000 7f7666166700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766019ea70 con 0x7f7660101fd0 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.250+0000 7f7666166700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766019ec30 con 0x7f7660102990 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.250+0000 7f765f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660100e80 0x7f7660196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.250+0000 7f765f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660100e80 0x7f7660196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39170/0 (socket says 172.21.15.33:39170) 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.250+0000 7f765f7fe700 1 -- 172.21.15.33:0/3182946648 learned_addr learned my addr 172.21.15.33:0/3182946648 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f765effd700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7660101fd0 0x7f7660197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f765ffff700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7660102990 0x7f766019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f765f7fe700 1 -- 172.21.15.33:0/3182946648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7660102990 msgr2=0x7f766019e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f765f7fe700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7660102990 0x7f766019e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f765f7fe700 1 -- 172.21.15.33:0/3182946648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7660101fd0 msgr2=0x7f7660197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f765f7fe700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7660101fd0 0x7f7660197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f765f7fe700 1 -- 172.21.15.33:0/3182946648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7658009d00 con 0x7f7660100e80 2024-05-08T21:17:18.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f765ffff700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7660102990 0x7f766019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:17:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f765f7fe700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660100e80 0x7f7660196e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f764800e9a0 tx=0x7f764800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f765cff9700 1 -- 172.21.15.33:0/3182946648 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648009790 con 0x7f7660100e80 2024-05-08T21:17:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f765cff9700 1 -- 172.21.15.33:0/3182946648 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f764800cbd0 con 0x7f7660100e80 2024-05-08T21:17:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f7666166700 1 -- 172.21.15.33:0/3182946648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766019ef10 con 0x7f7660100e80 2024-05-08T21:17:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f765cff9700 1 -- 172.21.15.33:0/3182946648 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648018680 con 0x7f7660100e80 2024-05-08T21:17:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.251+0000 7f7666166700 1 -- 172.21.15.33:0/3182946648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766019f340 con 0x7f7660100e80 2024-05-08T21:17:18.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.252+0000 7f7666166700 1 -- 172.21.15.33:0/3182946648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76601082f0 con 0x7f7660100e80 2024-05-08T21:17:18.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.253+0000 7f765cff9700 1 -- 172.21.15.33:0/3182946648 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f764800cd40 con 0x7f7660100e80 2024-05-08T21:17:18.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.254+0000 7f765cff9700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f764c0a11a0 0x7f764c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:18.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.254+0000 7f765effd700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f764c0a11a0 0x7f764c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:18.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.254+0000 7f765cff9700 1 -- 172.21.15.33:0/3182946648 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7648014070 con 0x7f7660100e80 2024-05-08T21:17:18.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.255+0000 7f765effd700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f764c0a11a0 0x7f764c0a3660 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7650009c60 tx=0x7f7650009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:18.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.259+0000 7f765cff9700 1 -- 172.21.15.33:0/3182946648 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7648052d60 con 0x7f7660100e80 2024-05-08T21:17:18.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:18 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1055970236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:18.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:18 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/22084315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:17:18.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:18 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3481047984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:17:18.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.481+0000 7f7666166700 1 -- 172.21.15.33:0/3182946648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7660068160 con 0x7f7660100e80 2024-05-08T21:17:18.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.482+0000 7f765cff9700 1 -- 172.21.15.33:0/3182946648 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7648096230 con 0x7f7660100e80 2024-05-08T21:17:18.484 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T21:17:18.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.484+0000 7f7666166700 1 -- 172.21.15.33:0/3182946648 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f764c0a11a0 msgr2=0x7f764c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:18.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.484+0000 7f7666166700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f764c0a11a0 0x7f764c0a3660 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7650009c60 tx=0x7f7650009260 comp rx=0 tx=0).stop 2024-05-08T21:17:18.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.484+0000 7f7666166700 1 -- 172.21.15.33:0/3182946648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660100e80 msgr2=0x7f7660196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:18.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.484+0000 7f7666166700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660100e80 0x7f7660196e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f764800e9a0 tx=0x7f764800ecb0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.484+0000 7f7666166700 1 -- 172.21.15.33:0/3182946648 shutdown_connections 2024-05-08T21:17:18.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.484+0000 7f7666166700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7660102990 0x7f766019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.484+0000 7f7666166700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7660100e80 0x7f7660196e10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.484+0000 7f7666166700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f764c0a11a0 0x7f764c0a3660 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.484+0000 7f7666166700 1 --2- 172.21.15.33:0/3182946648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7660101fd0 0x7f7660197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.484+0000 7f7666166700 1 -- 172.21.15.33:0/3182946648 >> 172.21.15.33:0/3182946648 conn(0x7f76600fc3f0 msgr2=0x7f7660105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:18.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.484+0000 7f7666166700 1 -- 172.21.15.33:0/3182946648 shutdown_connections 2024-05-08T21:17:18.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.484+0000 7f7666166700 1 -- 172.21.15.33:0/3182946648 wait complete. 2024-05-08T21:17:18.501 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T21:17:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:18 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1055970236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:18 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/22084315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:17:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:18 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3481047984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:17:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:18 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1055970236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:18 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/22084315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:17:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:18 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3481047984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:17:18.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.639+0000 7f39da2d1700 1 -- 172.21.15.33:0/3670713592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d40ffd60 msgr2=0x7f39d4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.639+0000 7f39da2d1700 1 --2- 172.21.15.33:0/3670713592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d40ffd60 0x7f39d4100180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f39c40099c0 tx=0x7f39c4009cd0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.639+0000 7f39da2d1700 1 -- 172.21.15.33:0/3670713592 shutdown_connections 2024-05-08T21:17:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.639+0000 7f39da2d1700 1 --2- 172.21.15.33:0/3670713592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d41018f0 0x7f39d41081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.639+0000 7f39da2d1700 1 --2- 172.21.15.33:0/3670713592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d40ffd60 0x7f39d4100180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.639+0000 7f39da2d1700 1 --2- 172.21.15.33:0/3670713592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d4100f50 0x7f39d41013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.639+0000 7f39da2d1700 1 -- 172.21.15.33:0/3670713592 >> 172.21.15.33:0/3670713592 conn(0x7f39d40fb290 msgr2=0x7f39d40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.639+0000 7f39da2d1700 1 -- 172.21.15.33:0/3670713592 shutdown_connections 2024-05-08T21:17:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.639+0000 7f39da2d1700 1 -- 172.21.15.33:0/3670713592 wait complete. 2024-05-08T21:17:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.640+0000 7f39da2d1700 1 Processor -- start 2024-05-08T21:17:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.640+0000 7f39da2d1700 1 -- start start 2024-05-08T21:17:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39da2d1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d40ffd60 0x7f39d4196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39da2d1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d4100f50 0x7f39d4197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39da2d1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d41018f0 0x7f39d419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39da2d1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d419e830 con 0x7f39d40ffd60 2024-05-08T21:17:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39da2d1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d419e9a0 con 0x7f39d41018f0 2024-05-08T21:17:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39da2d1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d419eb60 con 0x7f39d4100f50 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d40ffd60 0x7f39d4196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d40ffd60 0x7f39d4196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39186/0 (socket says 172.21.15.33:39186) 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d3fff700 1 -- 172.21.15.33:0/2891428792 learned_addr learned my addr 172.21.15.33:0/2891428792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d37fe700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d4100f50 0x7f39d4197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d886e700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d41018f0 0x7f39d419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d3fff700 1 -- 172.21.15.33:0/2891428792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d4100f50 msgr2=0x7f39d4197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d3fff700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d4100f50 0x7f39d4197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d3fff700 1 -- 172.21.15.33:0/2891428792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d41018f0 msgr2=0x7f39d419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d3fff700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d41018f0 0x7f39d419e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d3fff700 1 -- 172.21.15.33:0/2891428792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c40095f0 con 0x7f39d40ffd60 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d3fff700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d40ffd60 0x7f39d4196d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f39c40098e0 tx=0x7f39c4003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d17fa700 1 -- 172.21.15.33:0/2891428792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c401d070 con 0x7f39d40ffd60 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39da2d1700 1 -- 172.21.15.33:0/2891428792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d419ede0 con 0x7f39d40ffd60 2024-05-08T21:17:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d17fa700 1 -- 172.21.15.33:0/2891428792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39c400bd50 con 0x7f39d40ffd60 2024-05-08T21:17:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39d17fa700 1 -- 172.21.15.33:0/2891428792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4022c80 con 0x7f39d40ffd60 2024-05-08T21:17:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.641+0000 7f39da2d1700 1 -- 172.21.15.33:0/2891428792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d419f220 con 0x7f39d40ffd60 2024-05-08T21:17:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.643+0000 7f39da2d1700 1 -- 172.21.15.33:0/2891428792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d4107250 con 0x7f39d40ffd60 2024-05-08T21:17:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.644+0000 7f39d17fa700 1 -- 172.21.15.33:0/2891428792 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f39c4022470 con 0x7f39d40ffd60 2024-05-08T21:17:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.644+0000 7f39d17fa700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39bc0a11a0 0x7f39bc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.644+0000 7f39d37fe700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39bc0a11a0 0x7f39bc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.644+0000 7f39d17fa700 1 -- 172.21.15.33:0/2891428792 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f39c40ce5a0 con 0x7f39d40ffd60 2024-05-08T21:17:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.645+0000 7f39d37fe700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39bc0a11a0 0x7f39bc0a3660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005950 tx=0x7f39c800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:18.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.649+0000 7f39d17fa700 1 -- 172.21.15.33:0/2891428792 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f39c4097bd0 con 0x7f39d40ffd60 2024-05-08T21:17:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.866+0000 7f39da2d1700 1 -- 172.21.15.33:0/2891428792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f39d410c6d0 con 0x7f39d40ffd60 2024-05-08T21:17:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.866+0000 7f39d17fa700 1 -- 172.21.15.33:0/2891428792 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v85) v1 ==== 103+0+11 (secure 0 0 0) 0x7f39c40270a0 con 0x7f39d40ffd60 2024-05-08T21:17:18.869 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:17:18.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.868+0000 7f39da2d1700 1 -- 172.21.15.33:0/2891428792 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39bc0a11a0 msgr2=0x7f39bc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:18.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.868+0000 7f39da2d1700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39bc0a11a0 0x7f39bc0a3660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005950 tx=0x7f39c800b410 comp rx=0 tx=0).stop 2024-05-08T21:17:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.868+0000 7f39da2d1700 1 -- 172.21.15.33:0/2891428792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d40ffd60 msgr2=0x7f39d4196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.868+0000 7f39da2d1700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d40ffd60 0x7f39d4196d40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f39c40098e0 tx=0x7f39c4003ad0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.868+0000 7f39da2d1700 1 -- 172.21.15.33:0/2891428792 shutdown_connections 2024-05-08T21:17:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.868+0000 7f39da2d1700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39bc0a11a0 0x7f39bc0a3660 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.868+0000 7f39da2d1700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39d41018f0 0x7f39d419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.869+0000 7f39da2d1700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d40ffd60 0x7f39d4196d40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.869+0000 7f39da2d1700 1 --2- 172.21.15.33:0/2891428792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39d4100f50 0x7f39d4197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.869+0000 7f39da2d1700 1 -- 172.21.15.33:0/2891428792 >> 172.21.15.33:0/2891428792 conn(0x7f39d40fb290 msgr2=0x7f39d4104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.869+0000 7f39da2d1700 1 -- 172.21.15.33:0/2891428792 shutdown_connections 2024-05-08T21:17:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:18.869+0000 7f39da2d1700 1 -- 172.21.15.33:0/2891428792 wait complete. 2024-05-08T21:17:18.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T21:17:19.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.019+0000 7fa62c8ac700 1 -- 172.21.15.33:0/4153860676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa628100fb0 msgr2=0x7fa628101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:19.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.019+0000 7fa62c8ac700 1 --2- 172.21.15.33:0/4153860676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa628100fb0 0x7fa628101410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa6100099c0 tx=0x7fa610009cd0 comp rx=0 tx=0).stop 2024-05-08T21:17:19.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.019+0000 7fa62c8ac700 1 -- 172.21.15.33:0/4153860676 shutdown_connections 2024-05-08T21:17:19.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.019+0000 7fa62c8ac700 1 --2- 172.21.15.33:0/4153860676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa628101950 0x7fa628108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:19.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.019+0000 7fa62c8ac700 1 --2- 172.21.15.33:0/4153860676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6280ffdc0 0x7fa6281001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.019+0000 7fa62c8ac700 1 --2- 172.21.15.33:0/4153860676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa628100fb0 0x7fa628101410 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.019+0000 7fa62c8ac700 1 -- 172.21.15.33:0/4153860676 >> 172.21.15.33:0/4153860676 conn(0x7fa6280fb330 msgr2=0x7fa6280fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.019+0000 7fa62c8ac700 1 -- 172.21.15.33:0/4153860676 shutdown_connections 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.019+0000 7fa62c8ac700 1 -- 172.21.15.33:0/4153860676 wait complete. 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.020+0000 7fa62c8ac700 1 Processor -- start 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.020+0000 7fa62c8ac700 1 -- start start 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.020+0000 7fa62c8ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6280ffdc0 0x7fa628069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.020+0000 7fa62c8ac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa628100fb0 0x7fa62819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.020+0000 7fa62c8ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa628101950 0x7fa62819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.020+0000 7fa62c8ac700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62819e840 con 0x7fa628101950 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.020+0000 7fa62c8ac700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62819e9b0 con 0x7fa628100fb0 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.020+0000 7fa62c8ac700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62819eb70 con 0x7fa6280ffdc0 2024-05-08T21:17:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa626d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa628101950 0x7fa62819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa62659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6280ffdc0 0x7fa628069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa626d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa628101950 0x7fa62819e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39204/0 (socket says 172.21.15.33:39204) 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa626d9d700 1 -- 172.21.15.33:0/3337960293 learned_addr learned my addr 172.21.15.33:0/3337960293 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa61ffff700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa628100fb0 0x7fa62819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa626d9d700 1 -- 172.21.15.33:0/3337960293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6280ffdc0 msgr2=0x7fa628069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa626d9d700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6280ffdc0 0x7fa628069b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa626d9d700 1 -- 172.21.15.33:0/3337960293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa628100fb0 msgr2=0x7fa62819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa626d9d700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa628100fb0 0x7fa62819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa626d9d700 1 -- 172.21.15.33:0/3337960293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100095f0 con 0x7fa628101950 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa61ffff700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa628100fb0 0x7fa62819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa626d9d700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa628101950 0x7fa62819e050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa620007ee0 tx=0x7fa62000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa61f7fe700 1 -- 172.21.15.33:0/3337960293 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620017070 con 0x7fa628101950 2024-05-08T21:17:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa61f7fe700 1 -- 172.21.15.33:0/3337960293 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa62000f040 con 0x7fa628101950 2024-05-08T21:17:19.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa62c8ac700 1 -- 172.21.15.33:0/3337960293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62819edf0 con 0x7fa628101950 2024-05-08T21:17:19.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa61f7fe700 1 -- 172.21.15.33:0/3337960293 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620005710 con 0x7fa628101950 2024-05-08T21:17:19.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.021+0000 7fa62c8ac700 1 -- 172.21.15.33:0/3337960293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62819f2d0 con 0x7fa628101950 2024-05-08T21:17:19.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.022+0000 7fa62c8ac700 1 -- 172.21.15.33:0/3337960293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa628107290 con 0x7fa628101950 2024-05-08T21:17:19.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.024+0000 7fa61f7fe700 1 -- 172.21.15.33:0/3337960293 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa620005870 con 0x7fa628101950 2024-05-08T21:17:19.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.024+0000 7fa61f7fe700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa6140a11a0 0x7fa6140a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:19.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.024+0000 7fa61f7fe700 1 -- 172.21.15.33:0/3337960293 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa6200cc890 con 0x7fa628101950 2024-05-08T21:17:19.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.024+0000 7fa62659c700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa6140a11a0 0x7fa6140a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:19.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.025+0000 7fa62659c700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa6140a11a0 0x7fa6140a3660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa628100e10 tx=0x7fa61800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:19.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.028+0000 7fa61f7fe700 1 -- 172.21.15.33:0/3337960293 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa620095e30 con 0x7fa628101950 2024-05-08T21:17:19.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.269+0000 7fa62c8ac700 1 -- 172.21.15.33:0/3337960293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa628067480 con 0x7fa628101950 2024-05-08T21:17:19.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.269+0000 7fa61f7fe700 1 -- 172.21.15.33:0/3337960293 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) v1 ==== 81+0+592 (secure 0 0 0) 0x7fa620099450 con 0x7fa628101950 2024-05-08T21:17:19.272 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3156563197 2024-05-09T21:11:56.047045+0000 2024-05-08T21:17:19.272 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3413344235 2024-05-09T21:11:56.047045+0000 2024-05-08T21:17:19.272 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3413344235 2024-05-09T21:11:56.047045+0000 2024-05-08T21:17:19.272 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1762147139 2024-05-09T21:11:56.047045+0000 2024-05-08T21:17:19.272 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2047278949 2024-05-09T21:11:32.156884+0000 2024-05-08T21:17:19.272 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1002276836 2024-05-09T21:11:32.156884+0000 2024-05-08T21:17:19.272 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3475366293 2024-05-09T21:11:32.156884+0000 2024-05-08T21:17:19.273 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3475366293 2024-05-09T21:11:32.156884+0000 2024-05-08T21:17:19.273 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1708721255 2024-05-09T21:11:32.156884+0000 2024-05-08T21:17:19.273 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3864964868 2024-05-09T21:11:56.047045+0000 2024-05-08T21:17:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.271+0000 7fa62c8ac700 1 -- 172.21.15.33:0/3337960293 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa6140a11a0 msgr2=0x7fa6140a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:19.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.271+0000 7fa62c8ac700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa6140a11a0 0x7fa6140a3660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa628100e10 tx=0x7fa61800b410 comp rx=0 tx=0).stop 2024-05-08T21:17:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.272+0000 7fa62c8ac700 1 -- 172.21.15.33:0/3337960293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa628101950 msgr2=0x7fa62819e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.272+0000 7fa62c8ac700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa628101950 0x7fa62819e050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa620007ee0 tx=0x7fa62000d3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.272+0000 7fa62c8ac700 1 -- 172.21.15.33:0/3337960293 shutdown_connections 2024-05-08T21:17:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.272+0000 7fa62c8ac700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa628101950 0x7fa62819e050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.272+0000 7fa62c8ac700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6280ffdc0 0x7fa628069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.272+0000 7fa62c8ac700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa6140a11a0 0x7fa6140a3660 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.272+0000 7fa62c8ac700 1 --2- 172.21.15.33:0/3337960293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa628100fb0 0x7fa62819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.272+0000 7fa62c8ac700 1 -- 172.21.15.33:0/3337960293 >> 172.21.15.33:0/3337960293 conn(0x7fa6280fb330 msgr2=0x7fa6281043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.272+0000 7fa62c8ac700 1 -- 172.21.15.33:0/3337960293 shutdown_connections 2024-05-08T21:17:19.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:19.272+0000 7fa62c8ac700 1 -- 172.21.15.33:0/3337960293 wait complete. 2024-05-08T21:17:19.276 INFO:teuthology.orchestra.run.smithi033.stderr:listed 10 entries 2024-05-08T21:17:19.289 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-05-08T21:17:19.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:19.290 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-08T21:17:19.335 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[95857]: fuse finished with error 0 and tester_r 0 2024-05-08T21:17:19.337 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T21:17:19.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:19 smithi071 ceph-mon[47074]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 295 B/s wr, 0 op/s 2024-05-08T21:17:19.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:19 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3182946648' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:17:19.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:19 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2891428792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:17:19.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:19 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3337960293' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T21:17:19.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:19 smithi082 ceph-mon[47512]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 295 B/s wr, 0 op/s 2024-05-08T21:17:19.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:19 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3182946648' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:17:19.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:19 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2891428792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:17:19.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:19 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3337960293' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T21:17:19.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:19 smithi033 ceph-mon[42528]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 295 B/s wr, 0 op/s 2024-05-08T21:17:19.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:19 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3182946648' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:17:19.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:19 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2891428792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:17:19.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:19 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3337960293' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T21:17:21.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:21 smithi071 ceph-mon[47074]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T21:17:21.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:21 smithi082 ceph-mon[47512]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T21:17:21.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:21 smithi033 ceph-mon[42528]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-08T21:17:23.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:23 smithi071 ceph-mon[47074]: pgmap v238: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T21:17:23.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:23 smithi082 ceph-mon[47512]: pgmap v238: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T21:17:23.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:23 smithi033 ceph-mon[42528]: pgmap v238: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T21:17:25.439 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T21:17:25.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:17:25.439 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-08T21:17:25.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:17:25.550 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:25 smithi033 ceph-mon[42528]: pgmap v239: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T21:17:25.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:25 smithi071 ceph-mon[47074]: pgmap v239: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T21:17:25.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:25 smithi082 ceph-mon[47512]: pgmap v239: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T21:17:25.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.662+0000 7f145ccd1700 1 -- 172.21.15.33:0/3234501977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1458073980 msgr2=0x7f1458073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.662+0000 7f145ccd1700 1 --2- 172.21.15.33:0/3234501977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1458073980 0x7f1458073e00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1440009990 tx=0x7f1440009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.662+0000 7f145ccd1700 1 -- 172.21.15.33:0/3234501977 shutdown_connections 2024-05-08T21:17:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.662+0000 7f145ccd1700 1 --2- 172.21.15.33:0/3234501977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1458108d70 0x7f145810b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.662+0000 7f145ccd1700 1 --2- 172.21.15.33:0/3234501977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1458074f60 0x7f14580733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.662+0000 7f145ccd1700 1 --2- 172.21.15.33:0/3234501977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1458073980 0x7f1458073e00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.662+0000 7f145ccd1700 1 -- 172.21.15.33:0/3234501977 >> 172.21.15.33:0/3234501977 conn(0x7f14580fc410 msgr2=0x7f14580fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.662+0000 7f145ccd1700 1 -- 172.21.15.33:0/3234501977 shutdown_connections 2024-05-08T21:17:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.662+0000 7f145ccd1700 1 -- 172.21.15.33:0/3234501977 wait complete. 2024-05-08T21:17:25.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.663+0000 7f145ccd1700 1 Processor -- start 2024-05-08T21:17:25.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f145ccd1700 1 -- start start 2024-05-08T21:17:25.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f145ccd1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1458073980 0x7f1458198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:25.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f145ccd1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1458074f60 0x7f1458199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:25.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f145ccd1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1458108d70 0x7f14581a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f145ccd1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14581a0a30 con 0x7f1458074f60 2024-05-08T21:17:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f145ccd1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14581a0b70 con 0x7f1458108d70 2024-05-08T21:17:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f145ccd1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14581a0d00 con 0x7f1458073980 2024-05-08T21:17:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f144dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1458074f60 0x7f1458199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f144dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1458074f60 0x7f1458199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49154/0 (socket says 172.21.15.33:49154) 2024-05-08T21:17:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f144dd9b700 1 -- 172.21.15.33:0/2587161573 learned_addr learned my addr 172.21.15.33:0/2587161573 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f145659c700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1458073980 0x7f1458198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f1456d9d700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1458108d70 0x7f14581a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f144dd9b700 1 -- 172.21.15.33:0/2587161573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1458073980 msgr2=0x7f1458198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:25.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f144dd9b700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1458073980 0x7f1458198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f144dd9b700 1 -- 172.21.15.33:0/2587161573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1458108d70 msgr2=0x7f14581a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f144dd9b700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1458108d70 0x7f14581a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.664+0000 7f144dd9b700 1 -- 172.21.15.33:0/2587161573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14400095f0 con 0x7f1458074f60 2024-05-08T21:17:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.665+0000 7f144dd9b700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1458074f60 0x7f1458199440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1440006010 tx=0x7f1440004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.665+0000 7f144ffff700 1 -- 172.21.15.33:0/2587161573 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144001d070 con 0x7f1458074f60 2024-05-08T21:17:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.665+0000 7f144ffff700 1 -- 172.21.15.33:0/2587161573 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f144000bdb0 con 0x7f1458074f60 2024-05-08T21:17:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.665+0000 7f145ccd1700 1 -- 172.21.15.33:0/2587161573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14581a0f80 con 0x7f1458074f60 2024-05-08T21:17:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.665+0000 7f144ffff700 1 -- 172.21.15.33:0/2587161573 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144000fda0 con 0x7f1458074f60 2024-05-08T21:17:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.665+0000 7f145ccd1700 1 -- 172.21.15.33:0/2587161573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14581a1340 con 0x7f1458074f60 2024-05-08T21:17:25.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.666+0000 7f145ccd1700 1 -- 172.21.15.33:0/2587161573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1458107b80 con 0x7f1458074f60 2024-05-08T21:17:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.667+0000 7f144ffff700 1 -- 172.21.15.33:0/2587161573 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f144000f460 con 0x7f1458074f60 2024-05-08T21:17:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.668+0000 7f144ffff700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f14440a1120 0x7f14440a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.668+0000 7f145659c700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f14440a1120 0x7f14440a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.668+0000 7f144ffff700 1 -- 172.21.15.33:0/2587161573 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f14400ce1f0 con 0x7f1458074f60 2024-05-08T21:17:25.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.668+0000 7f145659c700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f14440a1120 0x7f14440a35e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1448009c60 tx=0x7f1448009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:25.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.672+0000 7f144ffff700 1 -- 172.21.15.33:0/2587161573 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f14400977c0 con 0x7f1458074f60 2024-05-08T21:17:25.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.890+0000 7f145ccd1700 1 -- 172.21.15.33:0/2587161573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f145810e780 con 0x7f1458074f60 2024-05-08T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.891+0000 7f144ffff700 1 -- 172.21.15.33:0/2587161573 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23902 (secure 0 0 0) 0x7f144009ade0 con 0x7f1458074f60 2024-05-08T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:25.894 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:07.709538+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:17:25.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.894+0000 7f145ccd1700 1 -- 172.21.15.33:0/2587161573 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f14440a1120 msgr2=0x7f14440a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.894+0000 7f145ccd1700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f14440a1120 0x7f14440a35e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1448009c60 tx=0x7f1448009260 comp rx=0 tx=0).stop 2024-05-08T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.894+0000 7f145ccd1700 1 -- 172.21.15.33:0/2587161573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1458074f60 msgr2=0x7f1458199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.894+0000 7f145ccd1700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1458074f60 0x7f1458199440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1440006010 tx=0x7f1440004730 comp rx=0 tx=0).stop 2024-05-08T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.894+0000 7f145ccd1700 1 -- 172.21.15.33:0/2587161573 shutdown_connections 2024-05-08T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.894+0000 7f145ccd1700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f14440a1120 0x7f14440a35e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.894+0000 7f145ccd1700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1458108d70 0x7f14581a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.894+0000 7f145ccd1700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1458073980 0x7f1458198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.894+0000 7f145ccd1700 1 --2- 172.21.15.33:0/2587161573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1458074f60 0x7f1458199440 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.894+0000 7f145ccd1700 1 -- 172.21.15.33:0/2587161573 >> 172.21.15.33:0/2587161573 conn(0x7f14580fc410 msgr2=0x7f1458104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.894+0000 7f145ccd1700 1 -- 172.21.15.33:0/2587161573 shutdown_connections 2024-05-08T21:17:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:25.894+0000 7f145ccd1700 1 -- 172.21.15.33:0/2587161573 wait complete. 2024-05-08T21:17:25.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:17:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.090+0000 7f853b8cb700 1 -- 172.21.15.33:0/3135635936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8534104a30 msgr2=0x7f8534104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.090+0000 7f853b8cb700 1 --2- 172.21.15.33:0/3135635936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8534104a30 0x7f8534104eb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f852c00b0a0 tx=0x7f852c00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.090+0000 7f853b8cb700 1 -- 172.21.15.33:0/3135635936 shutdown_connections 2024-05-08T21:17:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.090+0000 7f853b8cb700 1 --2- 172.21.15.33:0/3135635936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8534104a30 0x7f8534104eb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.090+0000 7f853b8cb700 1 --2- 172.21.15.33:0/3135635936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85340ff6f0 0x7f8534101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.090+0000 7f853b8cb700 1 --2- 172.21.15.33:0/3135635936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8534102060 0x7f85341044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.090+0000 7f853b8cb700 1 -- 172.21.15.33:0/3135635936 >> 172.21.15.33:0/3135635936 conn(0x7f85340fb2f0 msgr2=0x7f85340fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.090+0000 7f853b8cb700 1 -- 172.21.15.33:0/3135635936 shutdown_connections 2024-05-08T21:17:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.090+0000 7f853b8cb700 1 -- 172.21.15.33:0/3135635936 wait complete. 2024-05-08T21:17:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f853b8cb700 1 Processor -- start 2024-05-08T21:17:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f853b8cb700 1 -- start start 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f853b8cb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85340ff6f0 0x7f853419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f853b8cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8534102060 0x7f853419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f853b8cb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8534104a30 0x7f85341a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f853b8cb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85341a4f20 con 0x7f8534102060 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f853b8cb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85341a5060 con 0x7f8534104a30 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f853b8cb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85341a51f0 con 0x7f85340ff6f0 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f8538e66700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8534102060 0x7f853419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f8538e66700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8534102060 0x7f853419d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49178/0 (socket says 172.21.15.33:49178) 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f8538e66700 1 -- 172.21.15.33:0/2053935034 learned_addr learned my addr 172.21.15.33:0/2053935034 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f8539667700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85340ff6f0 0x7f853419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f8539e68700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8534104a30 0x7f85341a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f8538e66700 1 -- 172.21.15.33:0/2053935034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85340ff6f0 msgr2=0x7f853419d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f8538e66700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85340ff6f0 0x7f853419d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f8538e66700 1 -- 172.21.15.33:0/2053935034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8534104a30 msgr2=0x7f85341a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f8538e66700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8534104a30 0x7f85341a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.091+0000 7f8538e66700 1 -- 172.21.15.33:0/2053935034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c009d00 con 0x7f8534102060 2024-05-08T21:17:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.092+0000 7f8538e66700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8534102060 0x7f853419d930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f852400e9a0 tx=0x7f852400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.092+0000 7f852a7fc700 1 -- 172.21.15.33:0/2053935034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524009790 con 0x7f8534102060 2024-05-08T21:17:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.092+0000 7f852a7fc700 1 -- 172.21.15.33:0/2053935034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8524010430 con 0x7f8534102060 2024-05-08T21:17:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.092+0000 7f853b8cb700 1 -- 172.21.15.33:0/2053935034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85341a54d0 con 0x7f8534102060 2024-05-08T21:17:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.092+0000 7f852a7fc700 1 -- 172.21.15.33:0/2053935034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852400cea0 con 0x7f8534102060 2024-05-08T21:17:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.092+0000 7f853b8cb700 1 -- 172.21.15.33:0/2053935034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85341a5a30 con 0x7f8534102060 2024-05-08T21:17:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.093+0000 7f853b8cb700 1 -- 172.21.15.33:0/2053935034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85340fce70 con 0x7f8534102060 2024-05-08T21:17:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.094+0000 7f852a7fc700 1 -- 172.21.15.33:0/2053935034 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f85240105a0 con 0x7f8534102060 2024-05-08T21:17:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.095+0000 7f852a7fc700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f85200a11a0 0x7f85200a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.095+0000 7f852a7fc700 1 -- 172.21.15.33:0/2053935034 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8524014070 con 0x7f8534102060 2024-05-08T21:17:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.095+0000 7f8539667700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f85200a11a0 0x7f85200a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.095+0000 7f8539667700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f85200a11a0 0x7f85200a3660 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8530009c60 tx=0x7f8530009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:26.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.099+0000 7f852a7fc700 1 -- 172.21.15.33:0/2053935034 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f85240963c0 con 0x7f8534102060 2024-05-08T21:17:26.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.319+0000 7f853b8cb700 1 -- 172.21.15.33:0/2053935034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8534067480 con 0x7f8534102060 2024-05-08T21:17:26.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.321+0000 7f852a7fc700 1 -- 172.21.15.33:0/2053935034 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23902 (secure 0 0 0) 0x7f8524026090 con 0x7f8534102060 2024-05-08T21:17:26.324 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:26.324 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:07.709538+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:17:26.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.323+0000 7f853b8cb700 1 -- 172.21.15.33:0/2053935034 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f85200a11a0 msgr2=0x7f85200a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:26.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.323+0000 7f853b8cb700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f85200a11a0 0x7f85200a3660 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8530009c60 tx=0x7f8530009260 comp rx=0 tx=0).stop 2024-05-08T21:17:26.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.323+0000 7f853b8cb700 1 -- 172.21.15.33:0/2053935034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8534102060 msgr2=0x7f853419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:26.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.323+0000 7f853b8cb700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8534102060 0x7f853419d930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f852400e9a0 tx=0x7f852400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.323+0000 7f853b8cb700 1 -- 172.21.15.33:0/2053935034 shutdown_connections 2024-05-08T21:17:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.323+0000 7f853b8cb700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f85200a11a0 0x7f85200a3660 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.323+0000 7f853b8cb700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8534104a30 0x7f85341a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.323+0000 7f853b8cb700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85340ff6f0 0x7f853419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.323+0000 7f853b8cb700 1 --2- 172.21.15.33:0/2053935034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8534102060 0x7f853419d930 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.323+0000 7f853b8cb700 1 -- 172.21.15.33:0/2053935034 >> 172.21.15.33:0/2053935034 conn(0x7f85340fb2f0 msgr2=0x7f85341001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.324+0000 7f853b8cb700 1 -- 172.21.15.33:0/2053935034 shutdown_connections 2024-05-08T21:17:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.324+0000 7f853b8cb700 1 -- 172.21.15.33:0/2053935034 wait complete. 2024-05-08T21:17:26.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-08T21:17:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.480+0000 7f97a4247700 1 -- 172.21.15.33:0/157798383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c0ffd60 msgr2=0x7f979c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.480+0000 7f97a4247700 1 --2- 172.21.15.33:0/157798383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c0ffd60 0x7f979c100180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f97900099c0 tx=0x7f9790009cd0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.480+0000 7f97a4247700 1 -- 172.21.15.33:0/157798383 shutdown_connections 2024-05-08T21:17:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.480+0000 7f97a4247700 1 --2- 172.21.15.33:0/157798383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f979c1018f0 0x7f979c1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.480+0000 7f97a4247700 1 --2- 172.21.15.33:0/157798383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c0ffd60 0x7f979c100180 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.480+0000 7f97a4247700 1 --2- 172.21.15.33:0/157798383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f979c100f50 0x7f979c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.480+0000 7f97a4247700 1 -- 172.21.15.33:0/157798383 >> 172.21.15.33:0/157798383 conn(0x7f979c0fb290 msgr2=0x7f979c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.480+0000 7f97a4247700 1 -- 172.21.15.33:0/157798383 shutdown_connections 2024-05-08T21:17:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.480+0000 7f97a4247700 1 -- 172.21.15.33:0/157798383 wait complete. 2024-05-08T21:17:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.481+0000 7f97a4247700 1 Processor -- start 2024-05-08T21:17:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.481+0000 7f97a4247700 1 -- start start 2024-05-08T21:17:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.481+0000 7f97a4247700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f979c0ffd60 0x7f979c103ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.481+0000 7f97a4247700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c100f50 0x7f979c102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.481+0000 7f97a4247700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f979c1018f0 0x7f979c102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.481+0000 7f97a4247700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c103010 con 0x7f979c100f50 2024-05-08T21:17:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.481+0000 7f97a4247700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c103180 con 0x7f979c0ffd60 2024-05-08T21:17:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.481+0000 7f97a4247700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979c1a0e50 con 0x7f979c1018f0 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.481+0000 7f97a17e2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c100f50 0x7f979c102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.481+0000 7f97a17e2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c100f50 0x7f979c102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49202/0 (socket says 172.21.15.33:49202) 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.481+0000 7f97a17e2700 1 -- 172.21.15.33:0/1111021308 learned_addr learned my addr 172.21.15.33:0/1111021308 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f97a1fe3700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f979c0ffd60 0x7f979c103ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f97a27e4700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f979c1018f0 0x7f979c102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f97a17e2700 1 -- 172.21.15.33:0/1111021308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f979c1018f0 msgr2=0x7f979c102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f97a17e2700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f979c1018f0 0x7f979c102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f97a17e2700 1 -- 172.21.15.33:0/1111021308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f979c0ffd60 msgr2=0x7f979c103ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f97a17e2700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f979c0ffd60 0x7f979c103ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f97a17e2700 1 -- 172.21.15.33:0/1111021308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97900095f0 con 0x7f979c100f50 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f97a17e2700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c100f50 0x7f979c102330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f979800e9a0 tx=0x7f979800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f978effd700 1 -- 172.21.15.33:0/1111021308 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798009790 con 0x7f979c100f50 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f978effd700 1 -- 172.21.15.33:0/1111021308 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9798010430 con 0x7f979c100f50 2024-05-08T21:17:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f978effd700 1 -- 172.21.15.33:0/1111021308 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798018680 con 0x7f979c100f50 2024-05-08T21:17:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f97a4247700 1 -- 172.21.15.33:0/1111021308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c1a10e0 con 0x7f979c100f50 2024-05-08T21:17:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.482+0000 7f97a4247700 1 -- 172.21.15.33:0/1111021308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979c1a15a0 con 0x7f979c100f50 2024-05-08T21:17:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.483+0000 7f97a4247700 1 -- 172.21.15.33:0/1111021308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979c067d10 con 0x7f979c100f50 2024-05-08T21:17:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.485+0000 7f978effd700 1 -- 172.21.15.33:0/1111021308 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f97980105a0 con 0x7f979c100f50 2024-05-08T21:17:26.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.485+0000 7f978effd700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f97880a1120 0x7f97880a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:26.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.485+0000 7f97a1fe3700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f97880a1120 0x7f97880a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:26.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.485+0000 7f978effd700 1 -- 172.21.15.33:0/1111021308 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9798014070 con 0x7f979c100f50 2024-05-08T21:17:26.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.486+0000 7f97a1fe3700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f97880a1120 0x7f97880a35e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9790006010 tx=0x7f9790003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:26.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.489+0000 7f978effd700 1 -- 172.21.15.33:0/1111021308 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9798096520 con 0x7f979c100f50 2024-05-08T21:17:26.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:26 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2587161573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:26.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:26 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2587161573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:26.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:26 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2587161573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:26.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:26.762+0000 7f97a4247700 1 -- 172.21.15.33:0/1111021308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f979c10b670 con 0x7f979c100f50 2024-05-08T21:17:27.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.309+0000 7f978effd700 1 -- 172.21.15.33:0/1111021308 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f97980260b0 con 0x7f979c100f50 2024-05-08T21:17:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.312+0000 7f97a4247700 1 -- 172.21.15.33:0/1111021308 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f97880a1120 msgr2=0x7f97880a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.312+0000 7f97a4247700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f97880a1120 0x7f97880a35e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9790006010 tx=0x7f9790003680 comp rx=0 tx=0).stop 2024-05-08T21:17:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.312+0000 7f97a4247700 1 -- 172.21.15.33:0/1111021308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c100f50 msgr2=0x7f979c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.312+0000 7f97a4247700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c100f50 0x7f979c102330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f979800e9a0 tx=0x7f979800ecb0 comp rx=0 tx=0).stop 2024-05-08T21:17:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.312+0000 7f97a4247700 1 -- 172.21.15.33:0/1111021308 shutdown_connections 2024-05-08T21:17:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.312+0000 7f97a4247700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f979c1018f0 0x7f979c102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.312+0000 7f97a4247700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f97880a1120 0x7f97880a35e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.312+0000 7f97a4247700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f979c0ffd60 0x7f979c103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.312+0000 7f97a4247700 1 --2- 172.21.15.33:0/1111021308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c100f50 0x7f979c102330 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:27.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.312+0000 7f97a4247700 1 -- 172.21.15.33:0/1111021308 >> 172.21.15.33:0/1111021308 conn(0x7f979c0fb290 msgr2=0x7f979c0681f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:27.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.312+0000 7f97a4247700 1 -- 172.21.15.33:0/1111021308 shutdown_connections 2024-05-08T21:17:27.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.312+0000 7f97a4247700 1 -- 172.21.15.33:0/1111021308 wait complete. 2024-05-08T21:17:27.331 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-08T21:17:27.331 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-05-08T21:17:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.473+0000 7fa7a09fe700 1 -- 172.21.15.33:0/793441527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa79c104110 msgr2=0x7fa79c106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.473+0000 7fa7a09fe700 1 --2- 172.21.15.33:0/793441527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa79c104110 0x7fa79c106500 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa784009990 tx=0x7fa784009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.473+0000 7fa7a09fe700 1 -- 172.21.15.33:0/793441527 shutdown_connections 2024-05-08T21:17:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.473+0000 7fa7a09fe700 1 --2- 172.21.15.33:0/793441527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa79c106a40 0x7fa79c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.473+0000 7fa7a09fe700 1 --2- 172.21.15.33:0/793441527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa79c1017e0 0x7fa79c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.473+0000 7fa7a09fe700 1 --2- 172.21.15.33:0/793441527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa79c104110 0x7fa79c106500 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.473+0000 7fa7a09fe700 1 -- 172.21.15.33:0/793441527 >> 172.21.15.33:0/793441527 conn(0x7fa79c0fb130 msgr2=0x7fa79c0fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.473+0000 7fa7a09fe700 1 -- 172.21.15.33:0/793441527 shutdown_connections 2024-05-08T21:17:27.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.473+0000 7fa7a09fe700 1 -- 172.21.15.33:0/793441527 wait complete. 2024-05-08T21:17:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.474+0000 7fa7a09fe700 1 Processor -- start 2024-05-08T21:17:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.474+0000 7fa7a09fe700 1 -- start start 2024-05-08T21:17:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.474+0000 7fa7a09fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa79c1017e0 0x7fa79c196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.474+0000 7fa7a09fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa79c104110 0x7fa79c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.474+0000 7fa7a09fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa79c106a40 0x7fa79c19e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.474+0000 7fa7a09fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c19e7c0 con 0x7fa79c104110 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.474+0000 7fa7a09fe700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c19e930 con 0x7fa79c106a40 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.474+0000 7fa7a09fe700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c19eaf0 con 0x7fa79c1017e0 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa79259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa79c104110 0x7fa79c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa79259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa79c104110 0x7fa79c197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49206/0 (socket says 172.21.15.33:49206) 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa79259c700 1 -- 172.21.15.33:0/778499117 learned_addr learned my addr 172.21.15.33:0/778499117 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa79ad9d700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa79c1017e0 0x7fa79c196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa79b59e700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa79c106a40 0x7fa79c19e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa79259c700 1 -- 172.21.15.33:0/778499117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa79c1017e0 msgr2=0x7fa79c196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa79259c700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa79c1017e0 0x7fa79c196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa79259c700 1 -- 172.21.15.33:0/778499117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa79c106a40 msgr2=0x7fa79c19e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa79259c700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa79c106a40 0x7fa79c19e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa79259c700 1 -- 172.21.15.33:0/778499117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7840095f0 con 0x7fa79c104110 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa79b59e700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa79c106a40 0x7fa79c19e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:27.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa79259c700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa79c104110 0x7fa79c197210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa784004710 tx=0x7fa784009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa798d99700 1 -- 172.21.15.33:0/778499117 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78401d070 con 0x7fa79c104110 2024-05-08T21:17:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa7a09fe700 1 -- 172.21.15.33:0/778499117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79c19ed70 con 0x7fa79c104110 2024-05-08T21:17:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa798d99700 1 -- 172.21.15.33:0/778499117 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa78400be70 con 0x7fa79c104110 2024-05-08T21:17:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa7a09fe700 1 -- 172.21.15.33:0/778499117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79c19f1a0 con 0x7fa79c104110 2024-05-08T21:17:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.475+0000 7fa798d99700 1 -- 172.21.15.33:0/778499117 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784021410 con 0x7fa79c104110 2024-05-08T21:17:27.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.476+0000 7fa7a09fe700 1 -- 172.21.15.33:0/778499117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79c0fccb0 con 0x7fa79c104110 2024-05-08T21:17:27.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.478+0000 7fa798d99700 1 -- 172.21.15.33:0/778499117 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa78400f6f0 con 0x7fa79c104110 2024-05-08T21:17:27.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.479+0000 7fa798d99700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7880a11c0 0x7fa7880a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:27.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.479+0000 7fa79ad9d700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7880a11c0 0x7fa7880a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:27.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.479+0000 7fa798d99700 1 -- 172.21.15.33:0/778499117 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa7840cdf20 con 0x7fa79c104110 2024-05-08T21:17:27.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.479+0000 7fa79ad9d700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7880a11c0 0x7fa7880a3680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa79c198150 tx=0x7fa78c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:27.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.483+0000 7fa798d99700 1 -- 172.21.15.33:0/778499117 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa784097550 con 0x7fa79c104110 2024-05-08T21:17:27.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:27 smithi071 ceph-mon[47074]: pgmap v240: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T21:17:27.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:27 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2053935034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:27.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:27 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1111021308' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T21:17:27.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:27 smithi082 ceph-mon[47512]: pgmap v240: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T21:17:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:27 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2053935034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:27.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:27 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1111021308' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T21:17:27.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:27 smithi033 ceph-mon[42528]: pgmap v240: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T21:17:27.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:27 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2053935034' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:27.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:27 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1111021308' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T21:17:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:27.750+0000 7fa7a09fe700 1 -- 172.21.15.33:0/778499117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fa79c0008d0 con 0x7fa79c104110 2024-05-08T21:17:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.319+0000 7fa798d99700 1 -- 172.21.15.33:0/778499117 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fa78409ab70 con 0x7fa79c104110 2024-05-08T21:17:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.322+0000 7fa7a09fe700 1 -- 172.21.15.33:0/778499117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7880a11c0 msgr2=0x7fa7880a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.322+0000 7fa7a09fe700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7880a11c0 0x7fa7880a3680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa79c198150 tx=0x7fa78c009260 comp rx=0 tx=0).stop 2024-05-08T21:17:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.322+0000 7fa7a09fe700 1 -- 172.21.15.33:0/778499117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa79c104110 msgr2=0x7fa79c197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.322+0000 7fa7a09fe700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa79c104110 0x7fa79c197210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa784004710 tx=0x7fa784009580 comp rx=0 tx=0).stop 2024-05-08T21:17:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.322+0000 7fa7a09fe700 1 -- 172.21.15.33:0/778499117 shutdown_connections 2024-05-08T21:17:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.322+0000 7fa7a09fe700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7880a11c0 0x7fa7880a3680 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.322+0000 7fa7a09fe700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa79c106a40 0x7fa79c19e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.322+0000 7fa7a09fe700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa79c1017e0 0x7fa79c196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.322+0000 7fa7a09fe700 1 --2- 172.21.15.33:0/778499117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa79c104110 0x7fa79c197210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.322+0000 7fa7a09fe700 1 -- 172.21.15.33:0/778499117 >> 172.21.15.33:0/778499117 conn(0x7fa79c0fb130 msgr2=0x7fa79c102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:28.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.322+0000 7fa7a09fe700 1 -- 172.21.15.33:0/778499117 shutdown_connections 2024-05-08T21:17:28.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.322+0000 7fa7a09fe700 1 -- 172.21.15.33:0/778499117 wait complete. 2024-05-08T21:17:28.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-05-08T21:17:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.485+0000 7efc7c144700 1 -- 172.21.15.33:0/4253788084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc740ffd80 msgr2=0x7efc741001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.485+0000 7efc7c144700 1 --2- 172.21.15.33:0/4253788084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc740ffd80 0x7efc741001a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efc700099c0 tx=0x7efc70009cd0 comp rx=0 tx=0).stop 2024-05-08T21:17:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.485+0000 7efc7c144700 1 -- 172.21.15.33:0/4253788084 shutdown_connections 2024-05-08T21:17:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.485+0000 7efc7c144700 1 --2- 172.21.15.33:0/4253788084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc74101910 0x7efc741081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.485+0000 7efc7c144700 1 --2- 172.21.15.33:0/4253788084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc740ffd80 0x7efc741001a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.485+0000 7efc7c144700 1 --2- 172.21.15.33:0/4253788084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc74100f70 0x7efc741013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.485+0000 7efc7c144700 1 -- 172.21.15.33:0/4253788084 >> 172.21.15.33:0/4253788084 conn(0x7efc740fb310 msgr2=0x7efc740fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.486+0000 7efc7c144700 1 -- 172.21.15.33:0/4253788084 shutdown_connections 2024-05-08T21:17:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.486+0000 7efc7c144700 1 -- 172.21.15.33:0/4253788084 wait complete. 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.486+0000 7efc7c144700 1 Processor -- start 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.486+0000 7efc7c144700 1 -- start start 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.486+0000 7efc7c144700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc740ffd80 0x7efc7410a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.486+0000 7efc7c144700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc74100f70 0x7efc74108cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.486+0000 7efc7c144700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc74101910 0x7efc74109210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.486+0000 7efc7c144700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc741099b0 con 0x7efc74100f70 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.486+0000 7efc7c144700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc74109b20 con 0x7efc74101910 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.486+0000 7efc7c144700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc741a0d40 con 0x7efc740ffd80 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.486+0000 7efc796df700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc74100f70 0x7efc74108cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc796df700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc74100f70 0x7efc74108cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49226/0 (socket says 172.21.15.33:49226) 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc796df700 1 -- 172.21.15.33:0/3587431382 learned_addr learned my addr 172.21.15.33:0/3587431382 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc79ee0700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc740ffd80 0x7efc7410a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc7a6e1700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc74101910 0x7efc74109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc796df700 1 -- 172.21.15.33:0/3587431382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc740ffd80 msgr2=0x7efc7410a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc796df700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc740ffd80 0x7efc7410a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc796df700 1 -- 172.21.15.33:0/3587431382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc74101910 msgr2=0x7efc74109210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc796df700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc74101910 0x7efc74109210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc796df700 1 -- 172.21.15.33:0/3587431382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc700095f0 con 0x7efc74100f70 2024-05-08T21:17:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc796df700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc74100f70 0x7efc74108cd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efc6400c9b0 tx=0x7efc6400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc6affd700 1 -- 172.21.15.33:0/3587431382 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc64007aa0 con 0x7efc74100f70 2024-05-08T21:17:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc7c144700 1 -- 172.21.15.33:0/3587431382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc741a0fd0 con 0x7efc74100f70 2024-05-08T21:17:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc6affd700 1 -- 172.21.15.33:0/3587431382 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc64007c00 con 0x7efc74100f70 2024-05-08T21:17:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc6affd700 1 -- 172.21.15.33:0/3587431382 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc64018660 con 0x7efc74100f70 2024-05-08T21:17:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.487+0000 7efc7c144700 1 -- 172.21.15.33:0/3587431382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc741a1450 con 0x7efc74100f70 2024-05-08T21:17:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.488+0000 7efc7c144700 1 -- 172.21.15.33:0/3587431382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc74107220 con 0x7efc74100f70 2024-05-08T21:17:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.490+0000 7efc6affd700 1 -- 172.21.15.33:0/3587431382 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7efc6400f450 con 0x7efc74100f70 2024-05-08T21:17:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.490+0000 7efc6affd700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7efc600a1120 0x7efc600a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.490+0000 7efc79ee0700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7efc600a1120 0x7efc600a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.490+0000 7efc6affd700 1 -- 172.21.15.33:0/3587431382 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efc640ccdc0 con 0x7efc74100f70 2024-05-08T21:17:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.491+0000 7efc79ee0700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7efc600a1120 0x7efc600a35e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efc70000c00 tx=0x7efc70005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.494+0000 7efc6affd700 1 -- 172.21.15.33:0/3587431382 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7efc640963f0 con 0x7efc74100f70 2024-05-08T21:17:28.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:28 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1111021308' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T21:17:28.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:28 smithi071 ceph-mon[47074]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:28.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:28 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/778499117' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T21:17:28.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:28 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1111021308' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T21:17:28.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:28 smithi082 ceph-mon[47512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:28.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:28 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/778499117' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T21:17:28.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:28 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1111021308' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T21:17:28.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:28 smithi033 ceph-mon[42528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:28.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:28 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/778499117' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T21:17:28.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:28.763+0000 7efc7c144700 1 -- 172.21.15.33:0/3587431382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7efc74068160 con 0x7efc74100f70 2024-05-08T21:17:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.326+0000 7efc6affd700 1 -- 172.21.15.33:0/3587431382 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7efc64099a10 con 0x7efc74100f70 2024-05-08T21:17:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.330+0000 7efc7c144700 1 -- 172.21.15.33:0/3587431382 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7efc600a1120 msgr2=0x7efc600a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.330+0000 7efc7c144700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7efc600a1120 0x7efc600a35e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efc70000c00 tx=0x7efc70005fd0 comp rx=0 tx=0).stop 2024-05-08T21:17:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.330+0000 7efc7c144700 1 -- 172.21.15.33:0/3587431382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc74100f70 msgr2=0x7efc74108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.330+0000 7efc7c144700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc74100f70 0x7efc74108cd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efc6400c9b0 tx=0x7efc6400cd70 comp rx=0 tx=0).stop 2024-05-08T21:17:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.330+0000 7efc7c144700 1 -- 172.21.15.33:0/3587431382 shutdown_connections 2024-05-08T21:17:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.330+0000 7efc7c144700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7efc600a1120 0x7efc600a35e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.330+0000 7efc7c144700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc74101910 0x7efc74109210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.330+0000 7efc7c144700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc740ffd80 0x7efc7410a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.330+0000 7efc7c144700 1 --2- 172.21.15.33:0/3587431382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc74100f70 0x7efc74108cd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.330+0000 7efc7c144700 1 -- 172.21.15.33:0/3587431382 >> 172.21.15.33:0/3587431382 conn(0x7efc740fb310 msgr2=0x7efc74104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.330+0000 7efc7c144700 1 -- 172.21.15.33:0/3587431382 shutdown_connections 2024-05-08T21:17:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.330+0000 7efc7c144700 1 -- 172.21.15.33:0/3587431382 wait complete. 2024-05-08T21:17:29.352 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-05-08T21:17:29.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.493+0000 7fd8df0a4700 1 -- 172.21.15.33:0/3384891724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80ff750 msgr2=0x7fd8d80ffb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:29.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.493+0000 7fd8df0a4700 1 --2- 172.21.15.33:0/3384891724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80ff750 0x7fd8d80ffb70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80099c0 tx=0x7fd8c8009cd0 comp rx=0 tx=0).stop 2024-05-08T21:17:29.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.493+0000 7fd8df0a4700 1 -- 172.21.15.33:0/3384891724 shutdown_connections 2024-05-08T21:17:29.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.493+0000 7fd8df0a4700 1 --2- 172.21.15.33:0/3384891724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8d8100a70 0x7fd8d8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:29.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.493+0000 7fd8df0a4700 1 --2- 172.21.15.33:0/3384891724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80ff750 0x7fd8d80ffb70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:29.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.493+0000 7fd8df0a4700 1 --2- 172.21.15.33:0/3384891724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8d81000b0 0x7fd8d8100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:29.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.493+0000 7fd8df0a4700 1 -- 172.21.15.33:0/3384891724 >> 172.21.15.33:0/3384891724 conn(0x7fd8d80fb310 msgr2=0x7fd8d80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:29.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.493+0000 7fd8df0a4700 1 -- 172.21.15.33:0/3384891724 shutdown_connections 2024-05-08T21:17:29.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.493+0000 7fd8df0a4700 1 -- 172.21.15.33:0/3384891724 wait complete. 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8df0a4700 1 Processor -- start 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8df0a4700 1 -- start start 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8df0a4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8d80ff750 0x7fd8d8198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8df0a4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d81000b0 0x7fd8d8199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8df0a4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8d8100a70 0x7fd8d81a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8df0a4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d81a0a50 con 0x7fd8d81000b0 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8df0a4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d81a0b90 con 0x7fd8d8100a70 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8df0a4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d81a0d20 con 0x7fd8d80ff750 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d81000b0 0x7fd8d8199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d81000b0 0x7fd8d8199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49244/0 (socket says 172.21.15.33:49244) 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8d7fff700 1 -- 172.21.15.33:0/293116622 learned_addr learned my addr 172.21.15.33:0/293116622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8dce40700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8d80ff750 0x7fd8d8198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8dd641700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8d8100a70 0x7fd8d81a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8d7fff700 1 -- 172.21.15.33:0/293116622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8d80ff750 msgr2=0x7fd8d8198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8d7fff700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8d80ff750 0x7fd8d8198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8d7fff700 1 -- 172.21.15.33:0/293116622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8d8100a70 msgr2=0x7fd8d81a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8d7fff700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8d8100a70 0x7fd8d81a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8d7fff700 1 -- 172.21.15.33:0/293116622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c80095f0 con 0x7fd8d81000b0 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.494+0000 7fd8dd641700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8d8100a70 0x7fd8d81a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.495+0000 7fd8d7fff700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d81000b0 0x7fd8d8199460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00e9a0 tx=0x7fd8cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.495+0000 7fd8d5ffb700 1 -- 172.21.15.33:0/293116622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc009790 con 0x7fd8d81000b0 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.495+0000 7fd8d5ffb700 1 -- 172.21.15.33:0/293116622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8cc00cbd0 con 0x7fd8d81000b0 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.495+0000 7fd8df0a4700 1 -- 172.21.15.33:0/293116622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d81a0fa0 con 0x7fd8d81000b0 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.495+0000 7fd8d5ffb700 1 -- 172.21.15.33:0/293116622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc018680 con 0x7fd8d81000b0 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.495+0000 7fd8df0a4700 1 -- 172.21.15.33:0/293116622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d81a14d0 con 0x7fd8d81000b0 2024-05-08T21:17:29.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.496+0000 7fd8df0a4700 1 -- 172.21.15.33:0/293116622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d810a810 con 0x7fd8d81000b0 2024-05-08T21:17:29.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.497+0000 7fd8d5ffb700 1 -- 172.21.15.33:0/293116622 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd8cc00cd40 con 0x7fd8d81000b0 2024-05-08T21:17:29.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.498+0000 7fd8d5ffb700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd8c00a11a0 0x7fd8c00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:29.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.499+0000 7fd8dce40700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd8c00a11a0 0x7fd8c00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:29.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.499+0000 7fd8d5ffb700 1 -- 172.21.15.33:0/293116622 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd8cc014070 con 0x7fd8d81000b0 2024-05-08T21:17:29.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.499+0000 7fd8dce40700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd8c00a11a0 0x7fd8c00a3660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8000c00 tx=0x7fd8c8003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:29.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.503+0000 7fd8d5ffb700 1 -- 172.21.15.33:0/293116622 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd8cc0965c0 con 0x7fd8d81000b0 2024-05-08T21:17:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:29 smithi071 ceph-mon[47074]: pgmap v241: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T21:17:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:29 smithi071 ceph-mon[47074]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T21:17:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:29 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/778499117' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T21:17:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:29 smithi071 ceph-mon[47074]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T21:17:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:29 smithi071 ceph-mon[47074]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:29 smithi071 ceph-mon[47074]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-08T21:17:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:29 smithi071 ceph-mon[47074]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-08T21:17:29.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:29 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3587431382' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T21:17:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:29 smithi082 ceph-mon[47512]: pgmap v241: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T21:17:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:29 smithi082 ceph-mon[47512]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T21:17:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:29 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/778499117' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T21:17:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:29 smithi082 ceph-mon[47512]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T21:17:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:29 smithi082 ceph-mon[47512]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:29 smithi082 ceph-mon[47512]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-08T21:17:29.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:29 smithi082 ceph-mon[47512]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-08T21:17:29.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:29 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3587431382' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T21:17:29.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:29 smithi033 ceph-mon[42528]: pgmap v241: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T21:17:29.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:29 smithi033 ceph-mon[42528]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T21:17:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:29 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/778499117' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T21:17:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:29 smithi033 ceph-mon[42528]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T21:17:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:29 smithi033 ceph-mon[42528]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T21:17:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:29 smithi033 ceph-mon[42528]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-08T21:17:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:29 smithi033 ceph-mon[42528]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-08T21:17:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:29 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3587431382' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T21:17:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:29.778+0000 7fd8df0a4700 1 -- 172.21.15.33:0/293116622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fd8d80681c0 con 0x7fd8d81000b0 2024-05-08T21:17:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.340+0000 7fd8d5ffb700 1 -- 172.21.15.33:0/293116622 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd8cc026020 con 0x7fd8d81000b0 2024-05-08T21:17:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.343+0000 7fd8df0a4700 1 -- 172.21.15.33:0/293116622 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd8c00a11a0 msgr2=0x7fd8c00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.343+0000 7fd8df0a4700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd8c00a11a0 0x7fd8c00a3660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8000c00 tx=0x7fd8c8003680 comp rx=0 tx=0).stop 2024-05-08T21:17:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.343+0000 7fd8df0a4700 1 -- 172.21.15.33:0/293116622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d81000b0 msgr2=0x7fd8d8199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.343+0000 7fd8df0a4700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d81000b0 0x7fd8d8199460 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00e9a0 tx=0x7fd8cc00ecb0 comp rx=0 tx=0).stop 2024-05-08T21:17:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.343+0000 7fd8df0a4700 1 -- 172.21.15.33:0/293116622 shutdown_connections 2024-05-08T21:17:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.343+0000 7fd8df0a4700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd8c00a11a0 0x7fd8c00a3660 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.343+0000 7fd8df0a4700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8d8100a70 0x7fd8d81a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.343+0000 7fd8df0a4700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8d80ff750 0x7fd8d8198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.343+0000 7fd8df0a4700 1 --2- 172.21.15.33:0/293116622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d81000b0 0x7fd8d8199460 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:30.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.343+0000 7fd8df0a4700 1 -- 172.21.15.33:0/293116622 >> 172.21.15.33:0/293116622 conn(0x7fd8d80fb310 msgr2=0x7fd8d80fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:30.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.343+0000 7fd8df0a4700 1 -- 172.21.15.33:0/293116622 shutdown_connections 2024-05-08T21:17:30.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.343+0000 7fd8df0a4700 1 -- 172.21.15.33:0/293116622 wait complete. 2024-05-08T21:17:30.365 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-05-08T21:17:30.365 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-05-08T21:17:30.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.508+0000 7f6db469a700 1 -- 172.21.15.33:0/1221620723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dac1018f0 msgr2=0x7f6dac1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.508+0000 7f6db469a700 1 --2- 172.21.15.33:0/1221620723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dac1018f0 0x7f6dac1081a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6da400b0d0 tx=0x7f6da400b3e0 comp rx=0 tx=0).stop 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.508+0000 7f6db469a700 1 -- 172.21.15.33:0/1221620723 shutdown_connections 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.508+0000 7f6db469a700 1 --2- 172.21.15.33:0/1221620723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dac1018f0 0x7f6dac1081a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.508+0000 7f6db469a700 1 --2- 172.21.15.33:0/1221620723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dac0ffd60 0x7f6dac100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.508+0000 7f6db469a700 1 --2- 172.21.15.33:0/1221620723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dac100f50 0x7f6dac1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.508+0000 7f6db469a700 1 -- 172.21.15.33:0/1221620723 >> 172.21.15.33:0/1221620723 conn(0x7f6dac0fb290 msgr2=0x7f6dac0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.508+0000 7f6db469a700 1 -- 172.21.15.33:0/1221620723 shutdown_connections 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.508+0000 7f6db469a700 1 -- 172.21.15.33:0/1221620723 wait complete. 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.508+0000 7f6db469a700 1 Processor -- start 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.508+0000 7f6db469a700 1 -- start start 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db469a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dac0ffd60 0x7f6dac196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db469a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dac100f50 0x7f6dac197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:30.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db469a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dac1018f0 0x7f6dac19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db469a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac19e9f0 con 0x7f6dac0ffd60 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db469a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac19eb30 con 0x7f6dac1018f0 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db469a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dac19ecc0 con 0x7f6dac100f50 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db2436700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dac0ffd60 0x7f6dac196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db2436700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dac0ffd60 0x7f6dac196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46232/0 (socket says 172.21.15.33:46232) 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db2436700 1 -- 172.21.15.33:0/1531815521 learned_addr learned my addr 172.21.15.33:0/1531815521 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db1c35700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dac100f50 0x7f6dac197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db2c37700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dac1018f0 0x7f6dac19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db2436700 1 -- 172.21.15.33:0/1531815521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dac100f50 msgr2=0x7f6dac197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db2436700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dac100f50 0x7f6dac197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db2436700 1 -- 172.21.15.33:0/1531815521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dac1018f0 msgr2=0x7f6dac19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db2436700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dac1018f0 0x7f6dac19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db2436700 1 -- 172.21.15.33:0/1531815521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da4009d00 con 0x7f6dac0ffd60 2024-05-08T21:17:30.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db2c37700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dac1018f0 0x7f6dac19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6db2436700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dac0ffd60 0x7f6dac196f10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6da800e970 tx=0x7f6da800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.509+0000 7f6da37fe700 1 -- 172.21.15.33:0/1531815521 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8009790 con 0x7f6dac0ffd60 2024-05-08T21:17:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.510+0000 7f6da37fe700 1 -- 172.21.15.33:0/1531815521 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6da800eec0 con 0x7f6dac0ffd60 2024-05-08T21:17:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.510+0000 7f6db469a700 1 -- 172.21.15.33:0/1531815521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dac19efa0 con 0x7f6dac0ffd60 2024-05-08T21:17:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.510+0000 7f6da37fe700 1 -- 172.21.15.33:0/1531815521 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8003c10 con 0x7f6dac0ffd60 2024-05-08T21:17:30.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.510+0000 7f6db469a700 1 -- 172.21.15.33:0/1531815521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dac19f3c0 con 0x7f6dac0ffd60 2024-05-08T21:17:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.511+0000 7f6db469a700 1 -- 172.21.15.33:0/1531815521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dac067d10 con 0x7f6dac0ffd60 2024-05-08T21:17:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.512+0000 7f6da37fe700 1 -- 172.21.15.33:0/1531815521 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6da8004870 con 0x7f6dac0ffd60 2024-05-08T21:17:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.513+0000 7f6da37fe700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6d980a1120 0x7f6d980a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.513+0000 7f6db1c35700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6d980a1120 0x7f6d980a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.513+0000 7f6da37fe700 1 -- 172.21.15.33:0/1531815521 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6da8014070 con 0x7f6dac0ffd60 2024-05-08T21:17:30.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.513+0000 7f6db1c35700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6d980a1120 0x7f6d980a35e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c007be0 tx=0x7f6d9c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:30.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.517+0000 7f6da37fe700 1 -- 172.21.15.33:0/1531815521 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6da80963e0 con 0x7f6dac0ffd60 2024-05-08T21:17:30.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:30 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] up:active 2024-05-08T21:17:30.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:30 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3587431382' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T21:17:30.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:30 smithi082 ceph-mon[47512]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T21:17:30.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:30 smithi082 ceph-mon[47512]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-08T21:17:30.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:30 smithi082 ceph-mon[47512]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:creating} 9 up:standby 2024-05-08T21:17:30.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:30 smithi082 ceph-mon[47512]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-08T21:17:30.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:30 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/293116622' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T21:17:30.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:30 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] up:active 2024-05-08T21:17:30.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:30 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3587431382' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T21:17:30.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:30 smithi033 ceph-mon[42528]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T21:17:30.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:30 smithi033 ceph-mon[42528]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-08T21:17:30.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:30 smithi033 ceph-mon[42528]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:creating} 9 up:standby 2024-05-08T21:17:30.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:30 smithi033 ceph-mon[42528]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-08T21:17:30.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:30 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/293116622' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T21:17:30.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:30.772+0000 7f6db469a700 1 -- 172.21.15.33:0/1531815521 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f6dac068900 con 0x7f6d980a1120 2024-05-08T21:17:30.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:30 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] up:active 2024-05-08T21:17:30.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:30 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3587431382' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T21:17:30.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:30 smithi071 ceph-mon[47074]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T21:17:30.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:30 smithi071 ceph-mon[47074]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-08T21:17:30.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:30 smithi071 ceph-mon[47074]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:creating} 9 up:standby 2024-05-08T21:17:30.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:30 smithi071 ceph-mon[47074]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-08T21:17:30.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:30 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/293116622' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T21:17:31.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:17:30 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:17:30.778+0000 7ff7184ab700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T21:17:31.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:17:30 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:17:30.778+0000 7ff7184ab700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T21:17:31.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:17:30 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:17:30.778+0000 7ff7184ab700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T21:17:31.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:17:30 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:17:30.778+0000 7ff7184ab700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T21:17:31.208 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:17:30 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:17:30.778+0000 7ff7184ab700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T21:17:31.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.318+0000 7f6da37fe700 1 -- 172.21.15.33:0/1531815521 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6dac068900 con 0x7f6d980a1120 2024-05-08T21:17:31.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.321+0000 7f6db469a700 1 -- 172.21.15.33:0/1531815521 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6d980a1120 msgr2=0x7f6d980a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.321+0000 7f6db469a700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6d980a1120 0x7f6d980a35e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c007be0 tx=0x7f6d9c0073d0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.321+0000 7f6db469a700 1 -- 172.21.15.33:0/1531815521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dac0ffd60 msgr2=0x7f6dac196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.321+0000 7f6db469a700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dac0ffd60 0x7f6dac196f10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6da800e970 tx=0x7f6da800ec80 comp rx=0 tx=0).stop 2024-05-08T21:17:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.321+0000 7f6db469a700 1 -- 172.21.15.33:0/1531815521 shutdown_connections 2024-05-08T21:17:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.321+0000 7f6db469a700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6d980a1120 0x7f6d980a35e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.321+0000 7f6db469a700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dac1018f0 0x7f6dac19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.321+0000 7f6db469a700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dac0ffd60 0x7f6dac196f10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.321+0000 7f6db469a700 1 --2- 172.21.15.33:0/1531815521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dac100f50 0x7f6dac197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.321+0000 7f6db469a700 1 -- 172.21.15.33:0/1531815521 >> 172.21.15.33:0/1531815521 conn(0x7f6dac0fb290 msgr2=0x7f6dac0681f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.321+0000 7f6db469a700 1 -- 172.21.15.33:0/1531815521 shutdown_connections 2024-05-08T21:17:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.321+0000 7f6db469a700 1 -- 172.21.15.33:0/1531815521 wait complete. 2024-05-08T21:17:31.339 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-08T21:17:31.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.484+0000 7fd576c79700 1 -- 172.21.15.33:0/552826974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd570102990 msgr2=0x7fd5701092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:31.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.484+0000 7fd576c79700 1 --2- 172.21.15.33:0/552826974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd570102990 0x7fd5701092b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd56800b0a0 tx=0x7fd56800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.484+0000 7fd576c79700 1 -- 172.21.15.33:0/552826974 shutdown_connections 2024-05-08T21:17:31.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.484+0000 7fd576c79700 1 --2- 172.21.15.33:0/552826974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd570102990 0x7fd5701092b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.484+0000 7fd576c79700 1 --2- 172.21.15.33:0/552826974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd570100e80 0x7fd5701012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.484+0000 7fd576c79700 1 --2- 172.21.15.33:0/552826974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd570101fd0 0x7fd570102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.484+0000 7fd576c79700 1 -- 172.21.15.33:0/552826974 >> 172.21.15.33:0/552826974 conn(0x7fd5700fc3d0 msgr2=0x7fd5700fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:31.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.484+0000 7fd576c79700 1 -- 172.21.15.33:0/552826974 shutdown_connections 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.484+0000 7fd576c79700 1 -- 172.21.15.33:0/552826974 wait complete. 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd576c79700 1 Processor -- start 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd576c79700 1 -- start start 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd576c79700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd570100e80 0x7fd5701990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd576c79700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd570101fd0 0x7fd5701995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd576c79700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd570102990 0x7fd5701a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd576c79700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701a0b30 con 0x7fd570102990 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd576c79700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701a0c70 con 0x7fd570101fd0 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd576c79700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701a0e00 con 0x7fd570100e80 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd575216700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd570102990 0x7fd5701a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd575216700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd570102990 0x7fd5701a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46266/0 (socket says 172.21.15.33:46266) 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd575216700 1 -- 172.21.15.33:0/1266204939 learned_addr learned my addr 172.21.15.33:0/1266204939 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd56ffff700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd570101fd0 0x7fd5701995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:31.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.485+0000 7fd574a15700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd570100e80 0x7fd5701990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:31.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.486+0000 7fd575216700 1 -- 172.21.15.33:0/1266204939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd570100e80 msgr2=0x7fd5701990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:31.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.486+0000 7fd575216700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd570100e80 0x7fd5701990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.486+0000 7fd575216700 1 -- 172.21.15.33:0/1266204939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd570101fd0 msgr2=0x7fd5701995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:31.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.486+0000 7fd575216700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd570101fd0 0x7fd5701995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.486+0000 7fd575216700 1 -- 172.21.15.33:0/1266204939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd568009d00 con 0x7fd570102990 2024-05-08T21:17:31.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.486+0000 7fd575216700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd570102990 0x7fd5701a0560 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd568009570 tx=0x7fd5680095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:31.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.486+0000 7fd56dffb700 1 -- 172.21.15.33:0/1266204939 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56800e040 con 0x7fd570102990 2024-05-08T21:17:31.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.486+0000 7fd56dffb700 1 -- 172.21.15.33:0/1266204939 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd56801d470 con 0x7fd570102990 2024-05-08T21:17:31.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.486+0000 7fd576c79700 1 -- 172.21.15.33:0/1266204939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5701a1080 con 0x7fd570102990 2024-05-08T21:17:31.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.486+0000 7fd56dffb700 1 -- 172.21.15.33:0/1266204939 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5680052a0 con 0x7fd570102990 2024-05-08T21:17:31.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.486+0000 7fd576c79700 1 -- 172.21.15.33:0/1266204939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5701a1550 con 0x7fd570102990 2024-05-08T21:17:31.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.487+0000 7fd576c79700 1 -- 172.21.15.33:0/1266204939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd570067d10 con 0x7fd570102990 2024-05-08T21:17:31.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.489+0000 7fd56dffb700 1 -- 172.21.15.33:0/1266204939 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd56801d5e0 con 0x7fd570102990 2024-05-08T21:17:31.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.489+0000 7fd56dffb700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd5600a11a0 0x7fd5600a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:31.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.490+0000 7fd574a15700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd5600a11a0 0x7fd5600a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:31.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.490+0000 7fd56dffb700 1 -- 172.21.15.33:0/1266204939 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd5680ce4b0 con 0x7fd570102990 2024-05-08T21:17:31.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.490+0000 7fd574a15700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd5600a11a0 0x7fd5600a3660 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd564009c60 tx=0x7fd564009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:31.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.494+0000 7fd56dffb700 1 -- 172.21.15.33:0/1266204939 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd568097b60 con 0x7fd570102990 2024-05-08T21:17:31.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: pgmap v242: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 2 op/s 2024-05-08T21:17:31.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] up:active 2024-05-08T21:17:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/293116622' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T21:17:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T21:17:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 9 up:standby 2024-05-08T21:17:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=c=up:active,3=e=up:creating} 8 up:standby 2024-05-08T21:17:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-05-08T21:17:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] up:active 2024-05-08T21:17:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T21:17:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T21:17:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: Cluster is now healthy 2024-05-08T21:17:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=c=up:active,3=e=up:active} 4 up:standby-replay 4 up:standby 2024-05-08T21:17:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-05-08T21:17:31.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:31 smithi082 ceph-mon[47512]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-05-08T21:17:31.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: pgmap v242: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 2 op/s 2024-05-08T21:17:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] up:active 2024-05-08T21:17:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/293116622' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T21:17:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T21:17:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 9 up:standby 2024-05-08T21:17:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=c=up:active,3=e=up:creating} 8 up:standby 2024-05-08T21:17:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-05-08T21:17:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] up:active 2024-05-08T21:17:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T21:17:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T21:17:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: Cluster is now healthy 2024-05-08T21:17:31.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=c=up:active,3=e=up:active} 4 up:standby-replay 4 up:standby 2024-05-08T21:17:31.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-05-08T21:17:31.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:31 smithi033 ceph-mon[42528]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-05-08T21:17:31.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.746+0000 7fd576c79700 1 -- 172.21.15.33:0/1266204939 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fd570108fa0 con 0x7fd5600a11a0 2024-05-08T21:17:31.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.770+0000 7fd56dffb700 1 -- 172.21.15.33:0/1266204939 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd570108fa0 con 0x7fd5600a11a0 2024-05-08T21:17:31.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.772+0000 7fd576c79700 1 -- 172.21.15.33:0/1266204939 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd5600a11a0 msgr2=0x7fd5600a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:31.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.772+0000 7fd576c79700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd5600a11a0 0x7fd5600a3660 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd564009c60 tx=0x7fd564009260 comp rx=0 tx=0).stop 2024-05-08T21:17:31.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.772+0000 7fd576c79700 1 -- 172.21.15.33:0/1266204939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd570102990 msgr2=0x7fd5701a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:31.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.772+0000 7fd576c79700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd570102990 0x7fd5701a0560 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd568009570 tx=0x7fd5680095a0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.772+0000 7fd576c79700 1 -- 172.21.15.33:0/1266204939 shutdown_connections 2024-05-08T21:17:31.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.772+0000 7fd576c79700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd570102990 0x7fd5701a0560 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.772+0000 7fd576c79700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd570100e80 0x7fd5701990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.772+0000 7fd576c79700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd5600a11a0 0x7fd5600a3660 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.772+0000 7fd576c79700 1 --2- 172.21.15.33:0/1266204939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd570101fd0 0x7fd5701995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.772+0000 7fd576c79700 1 -- 172.21.15.33:0/1266204939 >> 172.21.15.33:0/1266204939 conn(0x7fd5700fc3d0 msgr2=0x7fd570107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:31.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.773+0000 7fd576c79700 1 -- 172.21.15.33:0/1266204939 shutdown_connections 2024-05-08T21:17:31.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.773+0000 7fd576c79700 1 -- 172.21.15.33:0/1266204939 wait complete. 2024-05-08T21:17:31.791 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-08T21:17:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: pgmap v242: 129 pgs: 129 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 2 op/s 2024-05-08T21:17:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] up:active 2024-05-08T21:17:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/293116622' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T21:17:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T21:17:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 9 up:standby 2024-05-08T21:17:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=c=up:active,3=e=up:creating} 8 up:standby 2024-05-08T21:17:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-05-08T21:17:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] up:active 2024-05-08T21:17:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T21:17:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T21:17:31.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: Cluster is now healthy 2024-05-08T21:17:31.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=c=up:active,3=e=up:active} 4 up:standby-replay 4 up:standby 2024-05-08T21:17:31.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-05-08T21:17:31.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:31 smithi071 ceph-mon[47074]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-05-08T21:17:31.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.933+0000 7f1138b75700 1 -- 172.21.15.33:0/1820199211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11341018f0 msgr2=0x7f11341081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:31.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.933+0000 7f1138b75700 1 --2- 172.21.15.33:0/1820199211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11341018f0 0x7f11341081a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f112c00b0d0 tx=0x7f112c00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.933+0000 7f1138b75700 1 -- 172.21.15.33:0/1820199211 shutdown_connections 2024-05-08T21:17:31.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.933+0000 7f1138b75700 1 --2- 172.21.15.33:0/1820199211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11341018f0 0x7f11341081a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.933+0000 7f1138b75700 1 --2- 172.21.15.33:0/1820199211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11340ffd60 0x7f1134100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.933+0000 7f1138b75700 1 --2- 172.21.15.33:0/1820199211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1134100f50 0x7f11341013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.933+0000 7f1138b75700 1 -- 172.21.15.33:0/1820199211 >> 172.21.15.33:0/1820199211 conn(0x7f11340fb290 msgr2=0x7f11340fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:31.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.933+0000 7f1138b75700 1 -- 172.21.15.33:0/1820199211 shutdown_connections 2024-05-08T21:17:31.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.933+0000 7f1138b75700 1 -- 172.21.15.33:0/1820199211 wait complete. 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.934+0000 7f1138b75700 1 Processor -- start 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.934+0000 7f1138b75700 1 -- start start 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.934+0000 7f1138b75700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11340ffd60 0x7f1134196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.934+0000 7f1138b75700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1134100f50 0x7f1134197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.934+0000 7f1138b75700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11341018f0 0x7f113419e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.934+0000 7f1138b75700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113419e890 con 0x7f1134100f50 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.934+0000 7f1138b75700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113419ea00 con 0x7f11341018f0 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.934+0000 7f1138b75700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113419ebc0 con 0x7f11340ffd60 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f1129d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1134100f50 0x7f1134197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f1129d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1134100f50 0x7f1134197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46288/0 (socket says 172.21.15.33:46288) 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f1129d9b700 1 -- 172.21.15.33:0/2094100920 learned_addr learned my addr 172.21.15.33:0/2094100920 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f113259c700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11340ffd60 0x7f1134196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:31.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f1132d9d700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11341018f0 0x7f113419e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:31.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f1129d9b700 1 -- 172.21.15.33:0/2094100920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11340ffd60 msgr2=0x7f1134196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:31.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f1129d9b700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11340ffd60 0x7f1134196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f1129d9b700 1 -- 172.21.15.33:0/2094100920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11341018f0 msgr2=0x7f113419e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:31.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f1129d9b700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11341018f0 0x7f113419e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:31.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f1129d9b700 1 -- 172.21.15.33:0/2094100920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c009d00 con 0x7f1134100f50 2024-05-08T21:17:31.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f1129d9b700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1134100f50 0x7f1134197330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f111c00c9b0 tx=0x7f111c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:31.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f112bfff700 1 -- 172.21.15.33:0/2094100920 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c007b00 con 0x7f1134100f50 2024-05-08T21:17:31.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f1138b75700 1 -- 172.21.15.33:0/2094100920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113419eea0 con 0x7f1134100f50 2024-05-08T21:17:31.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f112bfff700 1 -- 172.21.15.33:0/2094100920 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f111c007c60 con 0x7f1134100f50 2024-05-08T21:17:31.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f112bfff700 1 -- 172.21.15.33:0/2094100920 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c018660 con 0x7f1134100f50 2024-05-08T21:17:31.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.935+0000 7f1138b75700 1 -- 172.21.15.33:0/2094100920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113419f2c0 con 0x7f1134100f50 2024-05-08T21:17:31.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.936+0000 7f1138b75700 1 -- 172.21.15.33:0/2094100920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1134107110 con 0x7f1134100f50 2024-05-08T21:17:31.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.938+0000 7f112bfff700 1 -- 172.21.15.33:0/2094100920 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f111c00f450 con 0x7f1134100f50 2024-05-08T21:17:31.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.938+0000 7f112bfff700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f11200a1120 0x7f11200a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:31.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.938+0000 7f113259c700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f11200a1120 0x7f11200a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:31.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.938+0000 7f112bfff700 1 -- 172.21.15.33:0/2094100920 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f111c0ccda0 con 0x7f1134100f50 2024-05-08T21:17:31.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.939+0000 7f113259c700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f11200a1120 0x7f11200a35e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1134198220 tx=0x7f112400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:31.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:31.942+0000 7f112bfff700 1 -- 172.21.15.33:0/2094100920 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f111c0963d0 con 0x7f1134100f50 2024-05-08T21:17:32.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.201+0000 7f1138b75700 1 -- 172.21.15.33:0/2094100920 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f113402ceb0 con 0x7f11200a1120 2024-05-08T21:17:32.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.205+0000 7f112bfff700 1 -- 172.21.15.33:0/2094100920 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f113402ceb0 con 0x7f11200a1120 2024-05-08T21:17:32.208 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_0/72e45633-b2f0-4430-9cea-2dd00820c0b7 2024-05-08T21:17:32.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.208+0000 7f1138b75700 1 -- 172.21.15.33:0/2094100920 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f11200a1120 msgr2=0x7f11200a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:32.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.208+0000 7f1138b75700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f11200a1120 0x7f11200a35e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1134198220 tx=0x7f112400b410 comp rx=0 tx=0).stop 2024-05-08T21:17:32.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.208+0000 7f1138b75700 1 -- 172.21.15.33:0/2094100920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1134100f50 msgr2=0x7f1134197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:32.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.208+0000 7f1138b75700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1134100f50 0x7f1134197330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f111c00c9b0 tx=0x7f111c00ccc0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.208+0000 7f1138b75700 1 -- 172.21.15.33:0/2094100920 shutdown_connections 2024-05-08T21:17:32.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.208+0000 7f1138b75700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f11200a1120 0x7f11200a35e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.208+0000 7f1138b75700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11341018f0 0x7f113419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.208+0000 7f1138b75700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11340ffd60 0x7f1134196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.208+0000 7f1138b75700 1 --2- 172.21.15.33:0/2094100920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1134100f50 0x7f1134197330 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.208+0000 7f1138b75700 1 -- 172.21.15.33:0/2094100920 >> 172.21.15.33:0/2094100920 conn(0x7f11340fb290 msgr2=0x7f1134104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:32.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.208+0000 7f1138b75700 1 -- 172.21.15.33:0/2094100920 shutdown_connections 2024-05-08T21:17:32.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.208+0000 7f1138b75700 1 -- 172.21.15.33:0/2094100920 wait complete. 2024-05-08T21:17:32.224 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-08T21:17:32.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.373+0000 7f52c8c3f700 1 -- 172.21.15.33:0/3359579665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52c40ffd80 msgr2=0x7f52c41001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.373+0000 7f52c8c3f700 1 --2- 172.21.15.33:0/3359579665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52c40ffd80 0x7f52c41001a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f52b40099c0 tx=0x7f52b4009cd0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.374+0000 7f52c8c3f700 1 -- 172.21.15.33:0/3359579665 shutdown_connections 2024-05-08T21:17:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.374+0000 7f52c8c3f700 1 --2- 172.21.15.33:0/3359579665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c4101910 0x7f52c41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.374+0000 7f52c8c3f700 1 --2- 172.21.15.33:0/3359579665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52c40ffd80 0x7f52c41001a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.374+0000 7f52c8c3f700 1 --2- 172.21.15.33:0/3359579665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c4100f70 0x7f52c41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.374+0000 7f52c8c3f700 1 -- 172.21.15.33:0/3359579665 >> 172.21.15.33:0/3359579665 conn(0x7f52c40fb310 msgr2=0x7f52c40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.374+0000 7f52c8c3f700 1 -- 172.21.15.33:0/3359579665 shutdown_connections 2024-05-08T21:17:32.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.374+0000 7f52c8c3f700 1 -- 172.21.15.33:0/3359579665 wait complete. 2024-05-08T21:17:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52c8c3f700 1 Processor -- start 2024-05-08T21:17:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52c8c3f700 1 -- start start 2024-05-08T21:17:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52c8c3f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ffd80 0x7f52c4198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:32.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52c8c3f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52c4100f70 0x7f52c4199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52c8c3f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c4101910 0x7f52c41a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52c8c3f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c41a0a70 con 0x7f52c4100f70 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52c8c3f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c41a0bb0 con 0x7f52c40ffd80 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52c8c3f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c41a0d40 con 0x7f52c4101910 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52c4100f70 0x7f52c4199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52c4100f70 0x7f52c4199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46302/0 (socket says 172.21.15.33:46302) 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52bbfff700 1 -- 172.21.15.33:0/1982062001 learned_addr learned my addr 172.21.15.33:0/1982062001 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52c2d9d700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c4101910 0x7f52c41a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.375+0000 7f52c259c700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ffd80 0x7f52c4198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.376+0000 7f52bbfff700 1 -- 172.21.15.33:0/1982062001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c4101910 msgr2=0x7f52c41a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.376+0000 7f52bbfff700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c4101910 0x7f52c41a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.376+0000 7f52bbfff700 1 -- 172.21.15.33:0/1982062001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ffd80 msgr2=0x7f52c4198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.376+0000 7f52bbfff700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ffd80 0x7f52c4198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.376+0000 7f52bbfff700 1 -- 172.21.15.33:0/1982062001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b40095f0 con 0x7f52c4100f70 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.376+0000 7f52c2d9d700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c4101910 0x7f52c41a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.376+0000 7f52bbfff700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52c4100f70 0x7f52c4199480 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00d950 tx=0x7f52ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.376+0000 7f52bb7fe700 1 -- 172.21.15.33:0/1982062001 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00f890 con 0x7f52c4100f70 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.376+0000 7f52bb7fe700 1 -- 172.21.15.33:0/1982062001 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52ac00fe80 con 0x7f52c4100f70 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.376+0000 7f52c8c3f700 1 -- 172.21.15.33:0/1982062001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c41a0fc0 con 0x7f52c4100f70 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.376+0000 7f52bb7fe700 1 -- 172.21.15.33:0/1982062001 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00e3b0 con 0x7f52c4100f70 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.376+0000 7f52c8c3f700 1 -- 172.21.15.33:0/1982062001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c41a14f0 con 0x7f52c4100f70 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.377+0000 7f52c8c3f700 1 -- 172.21.15.33:0/1982062001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c4068160 con 0x7f52c4100f70 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.379+0000 7f52bb7fe700 1 -- 172.21.15.33:0/1982062001 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f52ac010460 con 0x7f52c4100f70 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.379+0000 7f52bb7fe700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f52b00a1230 0x7f52b00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.380+0000 7f52c259c700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f52b00a1230 0x7f52b00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.380+0000 7f52bb7fe700 1 -- 172.21.15.33:0/1982062001 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f52ac0cc9a0 con 0x7f52c4100f70 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.380+0000 7f52c259c700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f52b00a1230 0x7f52b00a36f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000c00 tx=0x7f52b4003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:32.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.384+0000 7f52bb7fe700 1 -- 172.21.15.33:0/1982062001 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f52ac095d80 con 0x7f52c4100f70 2024-05-08T21:17:32.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:32 smithi082 ceph-mon[47512]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:32.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:32 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] up:active 2024-05-08T21:17:32.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:32 smithi082 ceph-mon[47512]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-08T21:17:32.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:32 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:17:32.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.639+0000 7f52c8c3f700 1 -- 172.21.15.33:0/1982062001 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f52c4069c90 con 0x7f52b00a1230 2024-05-08T21:17:32.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.663+0000 7f52bb7fe700 1 -- 172.21.15.33:0/1982062001 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f52c4069c90 con 0x7f52b00a1230 2024-05-08T21:17:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.666+0000 7f52c8c3f700 1 -- 172.21.15.33:0/1982062001 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f52b00a1230 msgr2=0x7f52b00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.666+0000 7f52c8c3f700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f52b00a1230 0x7f52b00a36f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000c00 tx=0x7f52b4003680 comp rx=0 tx=0).stop 2024-05-08T21:17:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.666+0000 7f52c8c3f700 1 -- 172.21.15.33:0/1982062001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52c4100f70 msgr2=0x7f52c4199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.666+0000 7f52c8c3f700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52c4100f70 0x7f52c4199480 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00d950 tx=0x7f52ac00dc60 comp rx=0 tx=0).stop 2024-05-08T21:17:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.666+0000 7f52c8c3f700 1 -- 172.21.15.33:0/1982062001 shutdown_connections 2024-05-08T21:17:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.666+0000 7f52c8c3f700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c4101910 0x7f52c41a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.666+0000 7f52c8c3f700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f52b00a1230 0x7f52b00a36f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.666+0000 7f52c8c3f700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52c40ffd80 0x7f52c4198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.666+0000 7f52c8c3f700 1 --2- 172.21.15.33:0/1982062001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52c4100f70 0x7f52c4199480 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.666+0000 7f52c8c3f700 1 -- 172.21.15.33:0/1982062001 >> 172.21.15.33:0/1982062001 conn(0x7f52c40fb310 msgr2=0x7f52c4069580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.666+0000 7f52c8c3f700 1 -- 172.21.15.33:0/1982062001 shutdown_connections 2024-05-08T21:17:32.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.666+0000 7f52c8c3f700 1 -- 172.21.15.33:0/1982062001 wait complete. 2024-05-08T21:17:32.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:32 smithi033 ceph-mon[42528]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:32.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:32 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] up:active 2024-05-08T21:17:32.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:32 smithi033 ceph-mon[42528]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-08T21:17:32.682 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:32 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:17:32.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-08T21:17:32.797 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:32 smithi071 ceph-mon[47074]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:32.798 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:32 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] up:active 2024-05-08T21:17:32.798 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:32 smithi071 ceph-mon[47074]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-08T21:17:32.798 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:32 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:17:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.824+0000 7f89e130d700 1 -- 172.21.15.33:0/3765533057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89dc1019c0 msgr2=0x7f89dc103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.824+0000 7f89e130d700 1 --2- 172.21.15.33:0/3765533057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89dc1019c0 0x7f89dc103db0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009990 tx=0x7f89c4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.824+0000 7f89e130d700 1 -- 172.21.15.33:0/3765533057 shutdown_connections 2024-05-08T21:17:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.824+0000 7f89e130d700 1 --2- 172.21.15.33:0/3765533057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89dc106c20 0x7f89dc109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.824+0000 7f89e130d700 1 --2- 172.21.15.33:0/3765533057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89dc1019c0 0x7f89dc103db0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.824+0000 7f89e130d700 1 --2- 172.21.15.33:0/3765533057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89dc1042f0 0x7f89dc1066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.824+0000 7f89e130d700 1 -- 172.21.15.33:0/3765533057 >> 172.21.15.33:0/3765533057 conn(0x7f89dc0fb310 msgr2=0x7f89dc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.824+0000 7f89e130d700 1 -- 172.21.15.33:0/3765533057 shutdown_connections 2024-05-08T21:17:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.824+0000 7f89e130d700 1 -- 172.21.15.33:0/3765533057 wait complete. 2024-05-08T21:17:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89e130d700 1 Processor -- start 2024-05-08T21:17:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89e130d700 1 -- start start 2024-05-08T21:17:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89e130d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89dc1019c0 0x7f89dc1012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89e130d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89dc1042f0 0x7f89dc0ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89e130d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89dc106c20 0x7f89dc0ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89e130d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc100650 con 0x7f89dc106c20 2024-05-08T21:17:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89e130d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc1007c0 con 0x7f89dc1019c0 2024-05-08T21:17:32.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89e130d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc1a0d80 con 0x7f89dc1042f0 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89db7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89dc106c20 0x7f89dc0ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89db7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89dc106c20 0x7f89dc0ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46320/0 (socket says 172.21.15.33:46320) 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89db7fe700 1 -- 172.21.15.33:0/3181059297 learned_addr learned my addr 172.21.15.33:0/3181059297 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89da7fc700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89dc1042f0 0x7f89dc0ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.825+0000 7f89daffd700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89dc1019c0 0x7f89dc1012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.826+0000 7f89db7fe700 1 -- 172.21.15.33:0/3181059297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89dc1042f0 msgr2=0x7f89dc0ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.826+0000 7f89db7fe700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89dc1042f0 0x7f89dc0ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.826+0000 7f89db7fe700 1 -- 172.21.15.33:0/3181059297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89dc1019c0 msgr2=0x7f89dc1012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.826+0000 7f89db7fe700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89dc1019c0 0x7f89dc1012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.826+0000 7f89db7fe700 1 -- 172.21.15.33:0/3181059297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c40095f0 con 0x7f89dc106c20 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.826+0000 7f89da7fc700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89dc1042f0 0x7f89dc0ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.826+0000 7f89db7fe700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89dc106c20 0x7f89dc0ffeb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f89d400ef30 tx=0x7f89d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.826+0000 7f89d3fff700 1 -- 172.21.15.33:0/3181059297 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4018070 con 0x7f89dc106c20 2024-05-08T21:17:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.826+0000 7f89e130d700 1 -- 172.21.15.33:0/3181059297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89dc1a0fb0 con 0x7f89dc106c20 2024-05-08T21:17:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.826+0000 7f89d3fff700 1 -- 172.21.15.33:0/3181059297 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89d400f040 con 0x7f89dc106c20 2024-05-08T21:17:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.826+0000 7f89d3fff700 1 -- 172.21.15.33:0/3181059297 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d400cb20 con 0x7f89dc106c20 2024-05-08T21:17:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.826+0000 7f89e130d700 1 -- 172.21.15.33:0/3181059297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89dc1a1430 con 0x7f89dc106c20 2024-05-08T21:17:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.827+0000 7f89e130d700 1 -- 172.21.15.33:0/3181059297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89dc0fce90 con 0x7f89dc106c20 2024-05-08T21:17:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.828+0000 7f89d3fff700 1 -- 172.21.15.33:0/3181059297 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f89d400cd90 con 0x7f89dc106c20 2024-05-08T21:17:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.829+0000 7f89d3fff700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f89c80a11b0 0x7f89c80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.829+0000 7f89d3fff700 1 -- 172.21.15.33:0/3181059297 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f89d40cc780 con 0x7f89dc106c20 2024-05-08T21:17:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.829+0000 7f89daffd700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f89c80a11b0 0x7f89c80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.829+0000 7f89daffd700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f89c80a11b0 0x7f89c80a3670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f89c4006010 tx=0x7f89c400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:32.834+0000 7f89d3fff700 1 -- 172.21.15.33:0/3181059297 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f89d4095bf0 con 0x7f89dc106c20 2024-05-08T21:17:33.071 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:17:32 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-b[83724]: 2024-05-08T21:17:32.795+0000 7f9a07df2700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T21:17:33.071 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:17:32 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-b[83724]: 2024-05-08T21:17:32.795+0000 7f9a07df2700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-08T21:17:33.071 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:17:32 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-b[83724]: 2024-05-08T21:17:32.795+0000 7f9a07df2700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-08T21:17:33.071 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:17:32 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-b[83724]: 2024-05-08T21:17:32.795+0000 7f9a07df2700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-08T21:17:33.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.091+0000 7f89e130d700 1 -- 172.21.15.33:0/3181059297 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f89dc02ceb0 con 0x7f89c80a11b0 2024-05-08T21:17:33.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.095+0000 7f89d3fff700 1 -- 172.21.15.33:0/3181059297 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f89dc02ceb0 con 0x7f89c80a11b0 2024-05-08T21:17:33.098 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_1/09238a57-40bb-4ed6-b694-e592c8d9be89 2024-05-08T21:17:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.098+0000 7f89e130d700 1 -- 172.21.15.33:0/3181059297 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f89c80a11b0 msgr2=0x7f89c80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.098+0000 7f89e130d700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f89c80a11b0 0x7f89c80a3670 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f89c4006010 tx=0x7f89c400b610 comp rx=0 tx=0).stop 2024-05-08T21:17:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.098+0000 7f89e130d700 1 -- 172.21.15.33:0/3181059297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89dc106c20 msgr2=0x7f89dc0ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.098+0000 7f89e130d700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89dc106c20 0x7f89dc0ffeb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f89d400ef30 tx=0x7f89d400c3a0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.098+0000 7f89e130d700 1 -- 172.21.15.33:0/3181059297 shutdown_connections 2024-05-08T21:17:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.098+0000 7f89e130d700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89dc106c20 0x7f89dc0ffeb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.098+0000 7f89e130d700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f89c80a11b0 0x7f89c80a3670 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.098+0000 7f89e130d700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89dc1019c0 0x7f89dc1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.098+0000 7f89e130d700 1 --2- 172.21.15.33:0/3181059297 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89dc1042f0 0x7f89dc0ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.098+0000 7f89e130d700 1 -- 172.21.15.33:0/3181059297 >> 172.21.15.33:0/3181059297 conn(0x7f89dc0fb310 msgr2=0x7f89dc103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.098+0000 7f89e130d700 1 -- 172.21.15.33:0/3181059297 shutdown_connections 2024-05-08T21:17:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.098+0000 7f89e130d700 1 -- 172.21.15.33:0/3181059297 wait complete. 2024-05-08T21:17:33.114 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:17:33.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.248+0000 7f13ec93c700 1 -- 172.21.15.33:0/23815920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13e8106b60 msgr2=0x7f13e8108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:33.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.248+0000 7f13ec93c700 1 --2- 172.21.15.33:0/23815920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13e8106b60 0x7f13e8108fd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f13e000b0a0 tx=0x7f13e000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.248+0000 7f13ec93c700 1 -- 172.21.15.33:0/23815920 shutdown_connections 2024-05-08T21:17:33.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.248+0000 7f13ec93c700 1 --2- 172.21.15.33:0/23815920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13e8106b60 0x7f13e8108fd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.248+0000 7f13ec93c700 1 --2- 172.21.15.33:0/23815920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e8101900 0x7f13e8103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.248+0000 7f13ec93c700 1 --2- 172.21.15.33:0/23815920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8104230 0x7f13e8106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.248+0000 7f13ec93c700 1 -- 172.21.15.33:0/23815920 >> 172.21.15.33:0/23815920 conn(0x7f13e80fb2f0 msgr2=0x7f13e80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:33.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.248+0000 7f13ec93c700 1 -- 172.21.15.33:0/23815920 shutdown_connections 2024-05-08T21:17:33.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.248+0000 7f13ec93c700 1 -- 172.21.15.33:0/23815920 wait complete. 2024-05-08T21:17:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.249+0000 7f13ec93c700 1 Processor -- start 2024-05-08T21:17:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.249+0000 7f13ec93c700 1 -- start start 2024-05-08T21:17:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.249+0000 7f13ec93c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8101900 0x7f13e8101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.249+0000 7f13ec93c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e8104230 0x7f13e80ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.249+0000 7f13ec93c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13e8106b60 0x7f13e80ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.249+0000 7f13ec93c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e8100590 con 0x7f13e8106b60 2024-05-08T21:17:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.249+0000 7f13ec93c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e8100700 con 0x7f13e8101900 2024-05-08T21:17:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.249+0000 7f13ec93c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e81a0e10 con 0x7f13e8104230 2024-05-08T21:17:33.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13e6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13e8106b60 0x7f13e80ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13e659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8101900 0x7f13e8101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13e6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13e8106b60 0x7f13e80ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46344/0 (socket says 172.21.15.33:46344) 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13e6d9d700 1 -- 172.21.15.33:0/1550710131 learned_addr learned my addr 172.21.15.33:0/1550710131 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13e5d9b700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e8104230 0x7f13e80ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13e6d9d700 1 -- 172.21.15.33:0/1550710131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e8104230 msgr2=0x7f13e80ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13e6d9d700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e8104230 0x7f13e80ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13e6d9d700 1 -- 172.21.15.33:0/1550710131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8101900 msgr2=0x7f13e8101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13e6d9d700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8101900 0x7f13e8101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13e6d9d700 1 -- 172.21.15.33:0/1550710131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e0009d00 con 0x7f13e8106b60 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13e6d9d700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13e8106b60 0x7f13e80ffdf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f13e000b780 tx=0x7f13e000bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13d77fe700 1 -- 172.21.15.33:0/1550710131 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e000e040 con 0x7f13e8106b60 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13d77fe700 1 -- 172.21.15.33:0/1550710131 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f13e0003e10 con 0x7f13e8106b60 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13ec93c700 1 -- 172.21.15.33:0/1550710131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e81a1040 con 0x7f13e8106b60 2024-05-08T21:17:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13d77fe700 1 -- 172.21.15.33:0/1550710131 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e00052c0 con 0x7f13e8106b60 2024-05-08T21:17:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.250+0000 7f13ec93c700 1 -- 172.21.15.33:0/1550710131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e81a1410 con 0x7f13e8106b60 2024-05-08T21:17:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.252+0000 7f13ec93c700 1 -- 172.21.15.33:0/1550710131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e80fce70 con 0x7f13e8106b60 2024-05-08T21:17:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.253+0000 7f13d77fe700 1 -- 172.21.15.33:0/1550710131 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f13e0005420 con 0x7f13e8106b60 2024-05-08T21:17:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.253+0000 7f13d77fe700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13d00a11b0 0x7f13d00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.253+0000 7f13e659c700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13d00a11b0 0x7f13d00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.253+0000 7f13d77fe700 1 -- 172.21.15.33:0/1550710131 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f13e00ce000 con 0x7f13e8106b60 2024-05-08T21:17:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.254+0000 7f13e659c700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13d00a11b0 0x7f13d00a3670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f13d8005950 tx=0x7f13d800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:33.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.258+0000 7f13d77fe700 1 -- 172.21.15.33:0/1550710131 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f13e00974d0 con 0x7f13e8106b60 2024-05-08T21:17:33.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:33 smithi033 ceph-mon[42528]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:33.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:33 smithi033 ceph-mon[42528]: pgmap v243: 129 pgs: 129 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 6 op/s 2024-05-08T21:17:33.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:33 smithi033 ceph-mon[42528]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:33.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:33 smithi033 ceph-mon[42528]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-08T21:17:33.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.528+0000 7f13ec93c700 1 -- 172.21.15.33:0/1550710131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13e810c620 con 0x7f13e8106b60 2024-05-08T21:17:33.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.529+0000 7f13d77fe700 1 -- 172.21.15.33:0/1550710131 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10899 (secure 0 0 0) 0x7f13e009aaf0 con 0x7f13e8106b60 2024-05-08T21:17:33.531 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:33.531 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:17:32.303648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:17:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.531+0000 7f13ec93c700 1 -- 172.21.15.33:0/1550710131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13d00a11b0 msgr2=0x7f13d00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.531+0000 7f13ec93c700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13d00a11b0 0x7f13d00a3670 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f13d8005950 tx=0x7f13d800b410 comp rx=0 tx=0).stop 2024-05-08T21:17:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.531+0000 7f13ec93c700 1 -- 172.21.15.33:0/1550710131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13e8106b60 msgr2=0x7f13e80ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.531+0000 7f13ec93c700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13e8106b60 0x7f13e80ffdf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f13e000b780 tx=0x7f13e000bcc0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.531+0000 7f13ec93c700 1 -- 172.21.15.33:0/1550710131 shutdown_connections 2024-05-08T21:17:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.531+0000 7f13ec93c700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13e8106b60 0x7f13e80ffdf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.531+0000 7f13ec93c700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13d00a11b0 0x7f13d00a3670 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.531+0000 7f13ec93c700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13e8101900 0x7f13e8101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.531+0000 7f13ec93c700 1 --2- 172.21.15.33:0/1550710131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e8104230 0x7f13e80ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.531+0000 7f13ec93c700 1 -- 172.21.15.33:0/1550710131 >> 172.21.15.33:0/1550710131 conn(0x7f13e80fb2f0 msgr2=0x7f13e8102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.531+0000 7f13ec93c700 1 -- 172.21.15.33:0/1550710131 shutdown_connections 2024-05-08T21:17:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.531+0000 7f13ec93c700 1 -- 172.21.15.33:0/1550710131 wait complete. 2024-05-08T21:17:33.536 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-05-08T21:17:33.548 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:17:33.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:33 smithi082 ceph-mon[47512]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:33.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:33 smithi082 ceph-mon[47512]: pgmap v243: 129 pgs: 129 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 6 op/s 2024-05-08T21:17:33.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:33 smithi082 ceph-mon[47512]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:33.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:33 smithi082 ceph-mon[47512]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-08T21:17:33.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.683+0000 7f02ffc27700 1 -- 172.21.15.33:0/2282527076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f8101910 msgr2=0x7f02f81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:33.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.683+0000 7f02ffc27700 1 --2- 172.21.15.33:0/2282527076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f8101910 0x7f02f81081c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f02f000b0d0 tx=0x7f02f000b3e0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.683+0000 7f02ffc27700 1 -- 172.21.15.33:0/2282527076 shutdown_connections 2024-05-08T21:17:33.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.683+0000 7f02ffc27700 1 --2- 172.21.15.33:0/2282527076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f8101910 0x7f02f81081c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.683+0000 7f02ffc27700 1 --2- 172.21.15.33:0/2282527076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02f80ffd80 0x7f02f81001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.683+0000 7f02ffc27700 1 --2- 172.21.15.33:0/2282527076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02f8100f70 0x7f02f81013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.683+0000 7f02ffc27700 1 -- 172.21.15.33:0/2282527076 >> 172.21.15.33:0/2282527076 conn(0x7f02f80fb2f0 msgr2=0x7f02f80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.683+0000 7f02ffc27700 1 -- 172.21.15.33:0/2282527076 shutdown_connections 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.683+0000 7f02ffc27700 1 -- 172.21.15.33:0/2282527076 wait complete. 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.684+0000 7f02ffc27700 1 Processor -- start 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.684+0000 7f02ffc27700 1 -- start start 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.684+0000 7f02ffc27700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f80ffd80 0x7f02f8103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.684+0000 7f02ffc27700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02f8100f70 0x7f02f8102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.684+0000 7f02ffc27700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02f8101910 0x7f02f8102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.684+0000 7f02ffc27700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f8103030 con 0x7f02f80ffd80 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.684+0000 7f02ffc27700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f81031a0 con 0x7f02f8100f70 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.684+0000 7f02ffc27700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f81a0e00 con 0x7f02f8101910 2024-05-08T21:17:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.684+0000 7f02fd9c3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f80ffd80 0x7f02f8103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.684+0000 7f02fd9c3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f80ffd80 0x7f02f8103cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46358/0 (socket says 172.21.15.33:46358) 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.684+0000 7f02fd9c3700 1 -- 172.21.15.33:0/1152939620 learned_addr learned my addr 172.21.15.33:0/1152939620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02fd1c2700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02f8100f70 0x7f02f8102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02fe1c4700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02f8101910 0x7f02f8102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02fd9c3700 1 -- 172.21.15.33:0/1152939620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02f8101910 msgr2=0x7f02f8102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02fd9c3700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02f8101910 0x7f02f8102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02fd9c3700 1 -- 172.21.15.33:0/1152939620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02f8100f70 msgr2=0x7f02f8102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02fd9c3700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02f8100f70 0x7f02f8102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02fd9c3700 1 -- 172.21.15.33:0/1152939620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f0009d00 con 0x7f02f80ffd80 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02fd9c3700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f80ffd80 0x7f02f8103cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f02f400c990 tx=0x7f02f400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02eeffd700 1 -- 172.21.15.33:0/1152939620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f40078e0 con 0x7f02f80ffd80 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02eeffd700 1 -- 172.21.15.33:0/1152939620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02f400f450 con 0x7f02f80ffd80 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02ffc27700 1 -- 172.21.15.33:0/1152939620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f81a1090 con 0x7f02f80ffd80 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02eeffd700 1 -- 172.21.15.33:0/1152939620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4018660 con 0x7f02f80ffd80 2024-05-08T21:17:33.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.685+0000 7f02ffc27700 1 -- 172.21.15.33:0/1152939620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f81a15f0 con 0x7f02f80ffd80 2024-05-08T21:17:33.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.686+0000 7f02ffc27700 1 -- 172.21.15.33:0/1152939620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f8067480 con 0x7f02f80ffd80 2024-05-08T21:17:33.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.687+0000 7f02eeffd700 1 -- 172.21.15.33:0/1152939620 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f02f400f5c0 con 0x7f02f80ffd80 2024-05-08T21:17:33.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.688+0000 7f02eeffd700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02e40a1230 0x7f02e40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:33.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.688+0000 7f02eeffd700 1 -- 172.21.15.33:0/1152939620 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f02f40ccac0 con 0x7f02f80ffd80 2024-05-08T21:17:33.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.688+0000 7f02fd1c2700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02e40a1230 0x7f02e40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:33.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.689+0000 7f02fd1c2700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02e40a1230 0x7f02e40a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f02e8005950 tx=0x7f02e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:33.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.693+0000 7f02eeffd700 1 -- 172.21.15.33:0/1152939620 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f02f4095ea0 con 0x7f02f80ffd80 2024-05-08T21:17:33.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:33 smithi071 ceph-mon[47074]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:33.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:33 smithi071 ceph-mon[47074]: pgmap v243: 129 pgs: 129 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 6 op/s 2024-05-08T21:17:33.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:33 smithi071 ceph-mon[47074]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:33.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:33 smithi071 ceph-mon[47074]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-05-08T21:17:33.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.912+0000 7f02ffc27700 1 -- 172.21.15.33:0/1152939620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02f810b690 con 0x7f02f80ffd80 2024-05-08T21:17:33.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.913+0000 7f02eeffd700 1 -- 172.21.15.33:0/1152939620 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23902 (secure 0 0 0) 0x7f02f40994c0 con 0x7f02f80ffd80 2024-05-08T21:17:33.917 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:33.917 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:07.709538+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:17:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.916+0000 7f02ffc27700 1 -- 172.21.15.33:0/1152939620 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02e40a1230 msgr2=0x7f02e40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.916+0000 7f02ffc27700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02e40a1230 0x7f02e40a36f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f02e8005950 tx=0x7f02e800b410 comp rx=0 tx=0).stop 2024-05-08T21:17:33.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.916+0000 7f02ffc27700 1 -- 172.21.15.33:0/1152939620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f80ffd80 msgr2=0x7f02f8103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.916+0000 7f02ffc27700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f80ffd80 0x7f02f8103cc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f02f400c990 tx=0x7f02f400cca0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.916+0000 7f02ffc27700 1 -- 172.21.15.33:0/1152939620 shutdown_connections 2024-05-08T21:17:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.916+0000 7f02ffc27700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02f8101910 0x7f02f8102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.916+0000 7f02ffc27700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02f80ffd80 0x7f02f8103cc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.916+0000 7f02ffc27700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02e40a1230 0x7f02e40a36f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.916+0000 7f02ffc27700 1 --2- 172.21.15.33:0/1152939620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02f8100f70 0x7f02f8102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.916+0000 7f02ffc27700 1 -- 172.21.15.33:0/1152939620 >> 172.21.15.33:0/1152939620 conn(0x7f02f80fb2f0 msgr2=0x7f02f80fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.916+0000 7f02ffc27700 1 -- 172.21.15.33:0/1152939620 shutdown_connections 2024-05-08T21:17:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:33.916+0000 7f02ffc27700 1 -- 172.21.15.33:0/1152939620 wait complete. 2024-05-08T21:17:33.934 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-08T21:17:33.934 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T21:17:34.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.070+0000 7f33d7067700 1 -- 172.21.15.33:0/3925372109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33d0104290 msgr2=0x7f33d0106680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:34.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.070+0000 7f33d7067700 1 --2- 172.21.15.33:0/3925372109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33d0104290 0x7f33d0106680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009990 tx=0x7f33bc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.070+0000 7f33d7067700 1 -- 172.21.15.33:0/3925372109 shutdown_connections 2024-05-08T21:17:34.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.070+0000 7f33d7067700 1 --2- 172.21.15.33:0/3925372109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33d0106bc0 0x7f33d0108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.070+0000 7f33d7067700 1 --2- 172.21.15.33:0/3925372109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33d0101960 0x7f33d0103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.070+0000 7f33d7067700 1 --2- 172.21.15.33:0/3925372109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33d0104290 0x7f33d0106680 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.070+0000 7f33d7067700 1 -- 172.21.15.33:0/3925372109 >> 172.21.15.33:0/3925372109 conn(0x7f33d00fb2f0 msgr2=0x7f33d00fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:34.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.070+0000 7f33d7067700 1 -- 172.21.15.33:0/3925372109 shutdown_connections 2024-05-08T21:17:34.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.070+0000 7f33d7067700 1 -- 172.21.15.33:0/3925372109 wait complete. 2024-05-08T21:17:34.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33d7067700 1 Processor -- start 2024-05-08T21:17:34.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33d7067700 1 -- start start 2024-05-08T21:17:34.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33d7067700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33d0101960 0x7f33d0101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:34.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33d7067700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33d0104290 0x7f33d00ff910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:34.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33d7067700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33d0106bc0 0x7f33d00ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:34.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33d7067700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d01005f0 con 0x7f33d0104290 2024-05-08T21:17:34.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33d7067700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d0100760 con 0x7f33d0106bc0 2024-05-08T21:17:34.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33d7067700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d01a0d60 con 0x7f33d0101960 2024-05-08T21:17:34.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33d0104290 0x7f33d00ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:34.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33cffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33d0104290 0x7f33d00ff910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46374/0 (socket says 172.21.15.33:46374) 2024-05-08T21:17:34.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33cffff700 1 -- 172.21.15.33:0/4009577111 learned_addr learned my addr 172.21.15.33:0/4009577111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:34.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33d4e03700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33d0101960 0x7f33d0101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:34.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33d5604700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33d0106bc0 0x7f33d00ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:34.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33cffff700 1 -- 172.21.15.33:0/4009577111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33d0101960 msgr2=0x7f33d0101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:34.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33cffff700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33d0101960 0x7f33d0101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33cffff700 1 -- 172.21.15.33:0/4009577111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33d0106bc0 msgr2=0x7f33d00ffe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:34.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33cffff700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33d0106bc0 0x7f33d00ffe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.071+0000 7f33cffff700 1 -- 172.21.15.33:0/4009577111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc0095f0 con 0x7f33d0104290 2024-05-08T21:17:34.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.072+0000 7f33cffff700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33d0104290 0x7f33d00ff910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f33bc003b00 tx=0x7f33bc003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:34.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.072+0000 7f33cdffb700 1 -- 172.21.15.33:0/4009577111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc01d070 con 0x7f33d0104290 2024-05-08T21:17:34.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.072+0000 7f33cdffb700 1 -- 172.21.15.33:0/4009577111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f33bc00bd50 con 0x7f33d0104290 2024-05-08T21:17:34.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.072+0000 7f33cdffb700 1 -- 172.21.15.33:0/4009577111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc005280 con 0x7f33d0104290 2024-05-08T21:17:34.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.072+0000 7f33d7067700 1 -- 172.21.15.33:0/4009577111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d01a0f90 con 0x7f33d0104290 2024-05-08T21:17:34.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.072+0000 7f33d7067700 1 -- 172.21.15.33:0/4009577111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d01a1300 con 0x7f33d0104290 2024-05-08T21:17:34.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.073+0000 7f33d7067700 1 -- 172.21.15.33:0/4009577111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d00fce70 con 0x7f33d0104290 2024-05-08T21:17:34.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.075+0000 7f33cdffb700 1 -- 172.21.15.33:0/4009577111 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f33bc004930 con 0x7f33d0104290 2024-05-08T21:17:34.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.076+0000 7f33cdffb700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f33c00a11b0 0x7f33c00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:34.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.076+0000 7f33d4e03700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f33c00a11b0 0x7f33c00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:34.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.076+0000 7f33cdffb700 1 -- 172.21.15.33:0/4009577111 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f33bc0ce2e0 con 0x7f33d0104290 2024-05-08T21:17:34.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.076+0000 7f33d4e03700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f33c00a11b0 0x7f33c00a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f33c4005950 tx=0x7f33c40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:34.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.080+0000 7f33cdffb700 1 -- 172.21.15.33:0/4009577111 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f33bc0977b0 con 0x7f33d0104290 2024-05-08T21:17:34.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.296+0000 7f33d7067700 1 -- 172.21.15.33:0/4009577111 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33d002cf10 con 0x7f33c00a11b0 2024-05-08T21:17:34.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.310+0000 7f33cdffb700 1 -- 172.21.15.33:0/4009577111 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418015 (secure 0 0 0) 0x7f33d002cf10 con 0x7f33c00a11b0 2024-05-08T21:17:34.319 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.314+0000 7f33d7067700 1 -- 172.21.15.33:0/4009577111 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f33c00a11b0 msgr2=0x7f33c00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.314+0000 7f33d7067700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f33c00a11b0 0x7f33c00a3670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f33c4005950 tx=0x7f33c40058e0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.314+0000 7f33d7067700 1 -- 172.21.15.33:0/4009577111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33d0104290 msgr2=0x7f33d00ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.314+0000 7f33d7067700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33d0104290 0x7f33d00ff910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f33bc003b00 tx=0x7f33bc003be0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.314+0000 7f33d7067700 1 -- 172.21.15.33:0/4009577111 shutdown_connections 2024-05-08T21:17:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.314+0000 7f33d7067700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f33c00a11b0 0x7f33c00a3670 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.314+0000 7f33d7067700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33d0106bc0 0x7f33d00ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.314+0000 7f33d7067700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33d0101960 0x7f33d0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.314+0000 7f33d7067700 1 --2- 172.21.15.33:0/4009577111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33d0104290 0x7f33d00ff910 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.314+0000 7f33d7067700 1 -- 172.21.15.33:0/4009577111 >> 172.21.15.33:0/4009577111 conn(0x7f33d00fb2f0 msgr2=0x7f33d0103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.315+0000 7f33d7067700 1 -- 172.21.15.33:0/4009577111 shutdown_connections 2024-05-08T21:17:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.315+0000 7f33d7067700 1 -- 172.21.15.33:0/4009577111 wait complete. 2024-05-08T21:17:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T21:17:34.331 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-05-08T21:17:34.119712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600142,"num_objects":83,"num_object_clones":0,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":146,"num_read_kb":125,"num_write":354,"num_write_kb":3004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":288,"ondisk_log_size":288,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369996,"kb_used_data":13036,"kb_used_omap":18,"kb_used_meta":356845,"kb_avail":1124571828,"statfs":{"total":1151940427776,"available":1151561551872,"internally_reserved":0,"allocated":13348864,"data_stored":7329577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":365409606},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5180,"num_objects":59,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":124,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003659"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.260649+0000","last_change":"2024-05-08T21:17:03.243903+0000","last_active":"2024-05-08T21:17:07.260649+0000","last_peered":"2024-05-08T21:17:07.260649+0000","last_clean":"2024-05-08T21:17:07.260649+0000","last_became_active":"2024-05-08T21:17:03.240821+0000","last_became_peered":"2024-05-08T21:17:03.240821+0000","last_unstale":"2024-05-08T21:17:07.260649+0000","last_undegraded":"2024-05-08T21:17:07.260649+0000","last_fullsized":"2024-05-08T21:17:07.260649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:01.878614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489502+0000","last_change":"2024-05-08T21:17:03.243119+0000","last_active":"2024-05-08T21:17:07.489502+0000","last_peered":"2024-05-08T21:17:07.489502+0000","last_clean":"2024-05-08T21:17:07.489502+0000","last_became_active":"2024-05-08T21:17:03.242772+0000","last_became_peered":"2024-05-08T21:17:03.242772+0000","last_unstale":"2024-05-08T21:17:07.489502+0000","last_undegraded":"2024-05-08T21:17:07.489502+0000","last_fullsized":"2024-05-08T21:17:07.489502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:20.550646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489471+0000","last_change":"2024-05-08T21:17:03.237843+0000","last_active":"2024-05-08T21:17:07.489471+0000","last_peered":"2024-05-08T21:17:07.489471+0000","last_clean":"2024-05-08T21:17:07.489471+0000","last_became_active":"2024-05-08T21:17:03.237567+0000","last_became_peered":"2024-05-08T21:17:03.237567+0000","last_unstale":"2024-05-08T21:17:07.489471+0000","last_undegraded":"2024-05-08T21:17:07.489471+0000","last_fullsized":"2024-05-08T21:17:07.489471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:23.856461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.253998+0000","last_change":"2024-05-08T21:17:03.245083+0000","last_active":"2024-05-08T21:17:07.253998+0000","last_peered":"2024-05-08T21:17:07.253998+0000","last_clean":"2024-05-08T21:17:07.253998+0000","last_became_active":"2024-05-08T21:17:03.244756+0000","last_became_peered":"2024-05-08T21:17:03.244756+0000","last_unstale":"2024-05-08T21:17:07.253998+0000","last_undegraded":"2024-05-08T21:17:07.253998+0000","last_fullsized":"2024-05-08T21:17:07.253998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:57.884086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.252739+0000","last_change":"2024-05-08T21:17:03.252246+0000","last_active":"2024-05-08T21:17:07.252739+0000","last_peered":"2024-05-08T21:17:07.252739+0000","last_clean":"2024-05-08T21:17:07.252739+0000","last_became_active":"2024-05-08T21:17:03.251898+0000","last_became_peered":"2024-05-08T21:17:03.251898+0000","last_unstale":"2024-05-08T21:17:07.252739+0000","last_undegraded":"2024-05-08T21:17:07.252739+0000","last_fullsized":"2024-05-08T21:17:07.252739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:08.996033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.437630+0000","last_change":"2024-05-08T21:17:03.238422+0000","last_active":"2024-05-08T21:17:07.437630+0000","last_peered":"2024-05-08T21:17:07.437630+0000","last_clean":"2024-05-08T21:17:07.437630+0000","last_became_active":"2024-05-08T21:17:03.238096+0000","last_became_peered":"2024-05-08T21:17:03.238096+0000","last_unstale":"2024-05-08T21:17:07.437630+0000","last_undegraded":"2024-05-08T21:17:07.437630+0000","last_fullsized":"2024-05-08T21:17:07.437630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:54.635089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.255071+0000","last_change":"2024-05-08T21:17:03.244315+0000","last_active":"2024-05-08T21:17:07.255071+0000","last_peered":"2024-05-08T21:17:07.255071+0000","last_clean":"2024-05-08T21:17:07.255071+0000","last_became_active":"2024-05-08T21:17:03.243939+0000","last_became_peered":"2024-05-08T21:17:03.243939+0000","last_unstale":"2024-05-08T21:17:07.255071+0000","last_undegraded":"2024-05-08T21:17:07.255071+0000","last_fullsized":"2024-05-08T21:17:07.255071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:24.371495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.262968+0000","last_change":"2024-05-08T21:17:03.240144+0000","last_active":"2024-05-08T21:17:07.262968+0000","last_peered":"2024-05-08T21:17:07.262968+0000","last_clean":"2024-05-08T21:17:07.262968+0000","last_became_active":"2024-05-08T21:17:03.239493+0000","last_became_peered":"2024-05-08T21:17:03.239493+0000","last_unstale":"2024-05-08T21:17:07.262968+0000","last_undegraded":"2024-05-08T21:17:07.262968+0000","last_fullsized":"2024-05-08T21:17:07.262968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:21.374357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.249258+0000","last_change":"2024-05-08T21:17:03.256706+0000","last_active":"2024-05-08T21:17:07.249258+0000","last_peered":"2024-05-08T21:17:07.249258+0000","last_clean":"2024-05-08T21:17:07.249258+0000","last_became_active":"2024-05-08T21:17:03.255345+0000","last_became_peered":"2024-05-08T21:17:03.255345+0000","last_unstale":"2024-05-08T21:17:07.249258+0000","last_undegraded":"2024-05-08T21:17:07.249258+0000","last_fullsized":"2024-05-08T21:17:07.249258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:09:50.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.438066+0000","last_change":"2024-05-08T21:17:03.240062+0000","last_active":"2024-05-08T21:17:07.438066+0000","last_peered":"2024-05-08T21:17:07.438066+0000","last_clean":"2024-05-08T21:17:07.438066+0000","last_became_active":"2024-05-08T21:17:03.239740+0000","last_became_peered":"2024-05-08T21:17:03.239740+0000","last_unstale":"2024-05-08T21:17:07.438066+0000","last_undegraded":"2024-05-08T21:17:07.438066+0000","last_fullsized":"2024-05-08T21:17:07.438066+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:06:21.937721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.256041+0000","last_change":"2024-05-08T21:17:03.253996+0000","last_active":"2024-05-08T21:17:07.256041+0000","last_peered":"2024-05-08T21:17:07.256041+0000","last_clean":"2024-05-08T21:17:07.256041+0000","last_became_active":"2024-05-08T21:17:03.253686+0000","last_became_peered":"2024-05-08T21:17:03.253686+0000","last_unstale":"2024-05-08T21:17:07.256041+0000","last_undegraded":"2024-05-08T21:17:07.256041+0000","last_fullsized":"2024-05-08T21:17:07.256041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:17.995692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.253962+0000","last_change":"2024-05-08T21:17:03.254357+0000","last_active":"2024-05-08T21:17:07.253962+0000","last_peered":"2024-05-08T21:17:07.253962+0000","last_clean":"2024-05-08T21:17:07.253962+0000","last_became_active":"2024-05-08T21:17:03.254013+0000","last_became_peered":"2024-05-08T21:17:03.254013+0000","last_unstale":"2024-05-08T21:17:07.253962+0000","last_undegraded":"2024-05-08T21:17:07.253962+0000","last_fullsized":"2024-05-08T21:17:07.253962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:21.480612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.255067+0000","last_change":"2024-05-08T21:17:03.244190+0000","last_active":"2024-05-08T21:17:07.255067+0000","last_peered":"2024-05-08T21:17:07.255067+0000","last_clean":"2024-05-08T21:17:07.255067+0000","last_became_active":"2024-05-08T21:17:03.243989+0000","last_became_peered":"2024-05-08T21:17:03.243989+0000","last_unstale":"2024-05-08T21:17:07.255067+0000","last_undegraded":"2024-05-08T21:17:07.255067+0000","last_fullsized":"2024-05-08T21:17:07.255067+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:45.059343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489412+0000","last_change":"2024-05-08T21:17:03.242996+0000","last_active":"2024-05-08T21:17:07.489412+0000","last_peered":"2024-05-08T21:17:07.489412+0000","last_clean":"2024-05-08T21:17:07.489412+0000","last_became_active":"2024-05-08T21:17:03.242648+0000","last_became_peered":"2024-05-08T21:17:03.242648+0000","last_unstale":"2024-05-08T21:17:07.489412+0000","last_undegraded":"2024-05-08T21:17:07.489412+0000","last_fullsized":"2024-05-08T21:17:07.489412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:47:19.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489384+0000","last_change":"2024-05-08T21:17:03.244970+0000","last_active":"2024-05-08T21:17:07.489384+0000","last_peered":"2024-05-08T21:17:07.489384+0000","last_clean":"2024-05-08T21:17:07.489384+0000","last_became_active":"2024-05-08T21:17:03.244604+0000","last_became_peered":"2024-05-08T21:17:03.244604+0000","last_unstale":"2024-05-08T21:17:07.489384+0000","last_undegraded":"2024-05-08T21:17:07.489384+0000","last_fullsized":"2024-05-08T21:17:07.489384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:32.484155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.248910+0000","last_change":"2024-05-08T21:17:03.254628+0000","last_active":"2024-05-08T21:17:07.248910+0000","last_peered":"2024-05-08T21:17:07.248910+0000","last_clean":"2024-05-08T21:17:07.248910+0000","last_became_active":"2024-05-08T21:17:03.254305+0000","last_became_peered":"2024-05-08T21:17:03.254305+0000","last_unstale":"2024-05-08T21:17:07.248910+0000","last_undegraded":"2024-05-08T21:17:07.248910+0000","last_fullsized":"2024-05-08T21:17:07.248910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:52.727821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.260275+0000","last_change":"2024-05-08T21:17:03.243406+0000","last_active":"2024-05-08T21:17:07.260275+0000","last_peered":"2024-05-08T21:17:07.260275+0000","last_clean":"2024-05-08T21:17:07.260275+0000","last_became_active":"2024-05-08T21:17:03.240379+0000","last_became_peered":"2024-05-08T21:17:03.240379+0000","last_unstale":"2024-05-08T21:17:07.260275+0000","last_undegraded":"2024-05-08T21:17:07.260275+0000","last_fullsized":"2024-05-08T21:17:07.260275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:49.897459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.248571+0000","last_change":"2024-05-08T21:17:03.239664+0000","last_active":"2024-05-08T21:17:07.248571+0000","last_peered":"2024-05-08T21:17:07.248571+0000","last_clean":"2024-05-08T21:17:07.248571+0000","last_became_active":"2024-05-08T21:17:03.239347+0000","last_became_peered":"2024-05-08T21:17:03.239347+0000","last_unstale":"2024-05-08T21:17:07.248571+0000","last_undegraded":"2024-05-08T21:17:07.248571+0000","last_fullsized":"2024-05-08T21:17:07.248571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:06:37.557835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.250023+0000","last_change":"2024-05-08T21:17:03.249424+0000","last_active":"2024-05-08T21:17:07.250023+0000","last_peered":"2024-05-08T21:17:07.250023+0000","last_clean":"2024-05-08T21:17:07.250023+0000","last_became_active":"2024-05-08T21:17:03.248971+0000","last_became_peered":"2024-05-08T21:17:03.248971+0000","last_unstale":"2024-05-08T21:17:07.250023+0000","last_undegraded":"2024-05-08T21:17:07.250023+0000","last_fullsized":"2024-05-08T21:17:07.250023+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:08.096762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.273118+0000","last_change":"2024-05-08T21:17:07.272414+0000","last_active":"2024-05-08T21:17:07.273118+0000","last_peered":"2024-05-08T21:17:07.273118+0000","last_clean":"2024-05-08T21:17:07.273118+0000","last_became_active":"2024-05-08T21:17:07.271660+0000","last_became_peered":"2024-05-08T21:17:07.271660+0000","last_unstale":"2024-05-08T21:17:07.273118+0000","last_undegraded":"2024-05-08T21:17:07.273118+0000","last_fullsized":"2024-05-08T21:17:07.273118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:15.697910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.341532+0000","last_change":"2024-05-08T21:17:02.732779+0000","last_active":"2024-05-08T21:17:29.341532+0000","last_peered":"2024-05-08T21:17:29.341532+0000","last_clean":"2024-05-08T21:17:29.341532+0000","last_became_active":"2024-05-08T21:17:02.732439+0000","last_became_peered":"2024-05-08T21:17:02.732439+0000","last_unstale":"2024-05-08T21:17:29.341532+0000","last_undegraded":"2024-05-08T21:17:29.341532+0000","last_fullsized":"2024-05-08T21:17:29.341532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:44:42.064364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.267792+0000","last_change":"2024-05-08T21:17:03.240780+0000","last_active":"2024-05-08T21:17:07.267792+0000","last_peered":"2024-05-08T21:17:07.267792+0000","last_clean":"2024-05-08T21:17:07.267792+0000","last_became_active":"2024-05-08T21:17:03.240472+0000","last_became_peered":"2024-05-08T21:17:03.240472+0000","last_unstale":"2024-05-08T21:17:07.267792+0000","last_undegraded":"2024-05-08T21:17:07.267792+0000","last_fullsized":"2024-05-08T21:17:07.267792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.089760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713994+0000","last_change":"2024-05-08T21:17:07.272308+0000","last_active":"2024-05-08T21:17:07.713994+0000","last_peered":"2024-05-08T21:17:07.713994+0000","last_clean":"2024-05-08T21:17:07.713994+0000","last_became_active":"2024-05-08T21:17:07.271869+0000","last_became_peered":"2024-05-08T21:17:07.271869+0000","last_unstale":"2024-05-08T21:17:07.713994+0000","last_undegraded":"2024-05-08T21:17:07.713994+0000","last_fullsized":"2024-05-08T21:17:07.713994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:31.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.971656+0000","last_change":"2024-05-08T21:17:02.736446+0000","last_active":"2024-05-08T21:17:07.971656+0000","last_peered":"2024-05-08T21:17:07.971656+0000","last_clean":"2024-05-08T21:17:07.971656+0000","last_became_active":"2024-05-08T21:17:02.736187+0000","last_became_peered":"2024-05-08T21:17:02.736187+0000","last_unstale":"2024-05-08T21:17:07.971656+0000","last_undegraded":"2024-05-08T21:17:07.971656+0000","last_fullsized":"2024-05-08T21:17:07.971656+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:29.302558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489541+0000","last_change":"2024-05-08T21:17:03.251896+0000","last_active":"2024-05-08T21:17:07.489541+0000","last_peered":"2024-05-08T21:17:07.489541+0000","last_clean":"2024-05-08T21:17:07.489541+0000","last_became_active":"2024-05-08T21:17:03.251495+0000","last_became_peered":"2024-05-08T21:17:03.251495+0000","last_unstale":"2024-05-08T21:17:07.489541+0000","last_undegraded":"2024-05-08T21:17:07.489541+0000","last_fullsized":"2024-05-08T21:17:07.489541+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:50.774848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714159+0000","last_change":"2024-05-08T21:17:07.272150+0000","last_active":"2024-05-08T21:17:07.714159+0000","last_peered":"2024-05-08T21:17:07.714159+0000","last_clean":"2024-05-08T21:17:07.714159+0000","last_became_active":"2024-05-08T21:17:07.271727+0000","last_became_peered":"2024-05-08T21:17:07.271727+0000","last_unstale":"2024-05-08T21:17:07.714159+0000","last_undegraded":"2024-05-08T21:17:07.714159+0000","last_fullsized":"2024-05-08T21:17:07.714159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.058277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.339824+0000","last_change":"2024-05-08T21:17:02.733611+0000","last_active":"2024-05-08T21:17:29.339824+0000","last_peered":"2024-05-08T21:17:29.339824+0000","last_clean":"2024-05-08T21:17:29.339824+0000","last_became_active":"2024-05-08T21:17:02.733306+0000","last_became_peered":"2024-05-08T21:17:02.733306+0000","last_unstale":"2024-05-08T21:17:29.339824+0000","last_undegraded":"2024-05-08T21:17:29.339824+0000","last_fullsized":"2024-05-08T21:17:29.339824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:33.304753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.249070+0000","last_change":"2024-05-08T21:17:03.248480+0000","last_active":"2024-05-08T21:17:07.249070+0000","last_peered":"2024-05-08T21:17:07.249070+0000","last_clean":"2024-05-08T21:17:07.249070+0000","last_became_active":"2024-05-08T21:17:03.248126+0000","last_became_peered":"2024-05-08T21:17:03.248126+0000","last_unstale":"2024-05-08T21:17:07.249070+0000","last_undegraded":"2024-05-08T21:17:07.249070+0000","last_fullsized":"2024-05-08T21:17:07.249070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:08.785761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714035+0000","last_change":"2024-05-08T21:17:07.272344+0000","last_active":"2024-05-08T21:17:07.714035+0000","last_peered":"2024-05-08T21:17:07.714035+0000","last_clean":"2024-05-08T21:17:07.714035+0000","last_became_active":"2024-05-08T21:17:07.271629+0000","last_became_peered":"2024-05-08T21:17:07.271629+0000","last_unstale":"2024-05-08T21:17:07.714035+0000","last_undegraded":"2024-05-08T21:17:07.714035+0000","last_fullsized":"2024-05-08T21:17:07.714035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:15.052748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.252992+0000","last_change":"2024-05-08T21:17:02.730364+0000","last_active":"2024-05-08T21:17:07.252992+0000","last_peered":"2024-05-08T21:17:07.252992+0000","last_clean":"2024-05-08T21:17:07.252992+0000","last_became_active":"2024-05-08T21:17:02.728854+0000","last_became_peered":"2024-05-08T21:17:02.728854+0000","last_unstale":"2024-05-08T21:17:07.252992+0000","last_undegraded":"2024-05-08T21:17:07.252992+0000","last_fullsized":"2024-05-08T21:17:07.252992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:09.552016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.258497+0000","last_change":"2024-05-08T21:17:03.252713+0000","last_active":"2024-05-08T21:17:07.258497+0000","last_peered":"2024-05-08T21:17:07.258497+0000","last_clean":"2024-05-08T21:17:07.258497+0000","last_became_active":"2024-05-08T21:17:03.252388+0000","last_became_peered":"2024-05-08T21:17:03.252388+0000","last_unstale":"2024-05-08T21:17:07.258497+0000","last_undegraded":"2024-05-08T21:17:07.258497+0000","last_fullsized":"2024-05-08T21:17:07.258497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:09.114064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.261781+0000","last_change":"2024-05-08T21:17:07.261112+0000","last_active":"2024-05-08T21:17:07.261781+0000","last_peered":"2024-05-08T21:17:07.261781+0000","last_clean":"2024-05-08T21:17:07.261781+0000","last_became_active":"2024-05-08T21:17:07.260713+0000","last_became_peered":"2024-05-08T21:17:07.260713+0000","last_unstale":"2024-05-08T21:17:07.261781+0000","last_undegraded":"2024-05-08T21:17:07.261781+0000","last_fullsized":"2024-05-08T21:17:07.261781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:37.262113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.335526+0000","last_change":"2024-05-08T21:17:02.728480+0000","last_active":"2024-05-08T21:17:28.335526+0000","last_peered":"2024-05-08T21:17:28.335526+0000","last_clean":"2024-05-08T21:17:28.335526+0000","last_became_active":"2024-05-08T21:17:02.728170+0000","last_became_peered":"2024-05-08T21:17:02.728170+0000","last_unstale":"2024-05-08T21:17:28.335526+0000","last_undegraded":"2024-05-08T21:17:28.335526+0000","last_fullsized":"2024-05-08T21:17:28.335526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:00.540077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.253712+0000","last_change":"2024-05-08T21:17:03.251245+0000","last_active":"2024-05-08T21:17:07.253712+0000","last_peered":"2024-05-08T21:17:07.253712+0000","last_clean":"2024-05-08T21:17:07.253712+0000","last_became_active":"2024-05-08T21:17:03.250296+0000","last_became_peered":"2024-05-08T21:17:03.250296+0000","last_unstale":"2024-05-08T21:17:07.253712+0000","last_undegraded":"2024-05-08T21:17:07.253712+0000","last_fullsized":"2024-05-08T21:17:07.253712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:28.391927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.267323+0000","last_change":"2024-05-08T21:17:07.266752+0000","last_active":"2024-05-08T21:17:07.267323+0000","last_peered":"2024-05-08T21:17:07.267323+0000","last_clean":"2024-05-08T21:17:07.267323+0000","last_became_active":"2024-05-08T21:17:07.266367+0000","last_became_peered":"2024-05-08T21:17:07.266367+0000","last_unstale":"2024-05-08T21:17:07.267323+0000","last_undegraded":"2024-05-08T21:17:07.267323+0000","last_fullsized":"2024-05-08T21:17:07.267323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:49.874073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:30.355827+0000","last_change":"2024-05-08T21:17:02.728113+0000","last_active":"2024-05-08T21:17:30.355827+0000","last_peered":"2024-05-08T21:17:30.355827+0000","last_clean":"2024-05-08T21:17:30.355827+0000","last_became_active":"2024-05-08T21:17:02.727754+0000","last_became_peered":"2024-05-08T21:17:02.727754+0000","last_unstale":"2024-05-08T21:17:30.355827+0000","last_undegraded":"2024-05-08T21:17:30.355827+0000","last_fullsized":"2024-05-08T21:17:30.355827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:53.789218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.258170+0000","last_change":"2024-05-08T21:17:03.245482+0000","last_active":"2024-05-08T21:17:07.258170+0000","last_peered":"2024-05-08T21:17:07.258170+0000","last_clean":"2024-05-08T21:17:07.258170+0000","last_became_active":"2024-05-08T21:17:03.245144+0000","last_became_peered":"2024-05-08T21:17:03.245144+0000","last_unstale":"2024-05-08T21:17:07.258170+0000","last_undegraded":"2024-05-08T21:17:07.258170+0000","last_fullsized":"2024-05-08T21:17:07.258170+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:04.051234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.001109+0000","last_change":"2024-05-08T21:17:07.268505+0000","last_active":"2024-05-08T21:17:08.001109+0000","last_peered":"2024-05-08T21:17:08.001109+0000","last_clean":"2024-05-08T21:17:08.001109+0000","last_became_active":"2024-05-08T21:17:07.268157+0000","last_became_peered":"2024-05-08T21:17:07.268157+0000","last_unstale":"2024-05-08T21:17:08.001109+0000","last_undegraded":"2024-05-08T21:17:08.001109+0000","last_fullsized":"2024-05-08T21:17:08.001109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:55.220521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.343196+0000","last_change":"2024-05-08T21:17:02.727264+0000","last_active":"2024-05-08T21:17:29.343196+0000","last_peered":"2024-05-08T21:17:29.343196+0000","last_clean":"2024-05-08T21:17:29.343196+0000","last_became_active":"2024-05-08T21:17:02.726922+0000","last_became_peered":"2024-05-08T21:17:02.726922+0000","last_unstale":"2024-05-08T21:17:29.343196+0000","last_undegraded":"2024-05-08T21:17:29.343196+0000","last_fullsized":"2024-05-08T21:17:29.343196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:27.410231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.263093+0000","last_change":"2024-05-08T21:17:03.233910+0000","last_active":"2024-05-08T21:17:07.263093+0000","last_peered":"2024-05-08T21:17:07.263093+0000","last_clean":"2024-05-08T21:17:07.263093+0000","last_became_active":"2024-05-08T21:17:03.233562+0000","last_became_peered":"2024-05-08T21:17:03.233562+0000","last_unstale":"2024-05-08T21:17:07.263093+0000","last_undegraded":"2024-05-08T21:17:07.263093+0000","last_fullsized":"2024-05-08T21:17:07.263093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:40.309400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489487+0000","last_change":"2024-05-08T21:17:07.264992+0000","last_active":"2024-05-08T21:17:07.489487+0000","last_peered":"2024-05-08T21:17:07.489487+0000","last_clean":"2024-05-08T21:17:07.489487+0000","last_became_active":"2024-05-08T21:17:07.264498+0000","last_became_peered":"2024-05-08T21:17:07.264498+0000","last_unstale":"2024-05-08T21:17:07.489487+0000","last_undegraded":"2024-05-08T21:17:07.489487+0000","last_fullsized":"2024-05-08T21:17:07.489487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:23:26.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.334783+0000","last_change":"2024-05-08T21:17:02.725531+0000","last_active":"2024-05-08T21:17:28.334783+0000","last_peered":"2024-05-08T21:17:28.334783+0000","last_clean":"2024-05-08T21:17:28.334783+0000","last_became_active":"2024-05-08T21:17:02.725235+0000","last_became_peered":"2024-05-08T21:17:02.725235+0000","last_unstale":"2024-05-08T21:17:28.334783+0000","last_undegraded":"2024-05-08T21:17:28.334783+0000","last_fullsized":"2024-05-08T21:17:28.334783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:27.265590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.248011+0000","last_change":"2024-05-08T21:17:03.236393+0000","last_active":"2024-05-08T21:17:07.248011+0000","last_peered":"2024-05-08T21:17:07.248011+0000","last_clean":"2024-05-08T21:17:07.248011+0000","last_became_active":"2024-05-08T21:17:03.236092+0000","last_became_peered":"2024-05-08T21:17:03.236092+0000","last_unstale":"2024-05-08T21:17:07.248011+0000","last_undegraded":"2024-05-08T21:17:07.248011+0000","last_fullsized":"2024-05-08T21:17:07.248011+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:58.232669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.971004+0000","last_change":"2024-05-08T21:17:07.271374+0000","last_active":"2024-05-08T21:17:07.971004+0000","last_peered":"2024-05-08T21:17:07.971004+0000","last_clean":"2024-05-08T21:17:07.971004+0000","last_became_active":"2024-05-08T21:17:07.270778+0000","last_became_peered":"2024-05-08T21:17:07.270778+0000","last_unstale":"2024-05-08T21:17:07.971004+0000","last_undegraded":"2024-05-08T21:17:07.971004+0000","last_fullsized":"2024-05-08T21:17:07.971004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:50.212213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.335923+0000","last_change":"2024-05-08T21:17:02.725624+0000","last_active":"2024-05-08T21:17:28.335923+0000","last_peered":"2024-05-08T21:17:28.335923+0000","last_clean":"2024-05-08T21:17:28.335923+0000","last_became_active":"2024-05-08T21:17:02.725367+0000","last_became_peered":"2024-05-08T21:17:02.725367+0000","last_unstale":"2024-05-08T21:17:28.335923+0000","last_undegraded":"2024-05-08T21:17:28.335923+0000","last_fullsized":"2024-05-08T21:17:28.335923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:19.573931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.249341+0000","last_change":"2024-05-08T21:17:03.248729+0000","last_active":"2024-05-08T21:17:07.249341+0000","last_peered":"2024-05-08T21:17:07.249341+0000","last_clean":"2024-05-08T21:17:07.249341+0000","last_became_active":"2024-05-08T21:17:03.247899+0000","last_became_peered":"2024-05-08T21:17:03.247899+0000","last_unstale":"2024-05-08T21:17:07.249341+0000","last_undegraded":"2024-05-08T21:17:07.249341+0000","last_fullsized":"2024-05-08T21:17:07.249341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:27:57.012249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713082+0000","last_change":"2024-05-08T21:17:07.261257+0000","last_active":"2024-05-08T21:17:07.713082+0000","last_peered":"2024-05-08T21:17:07.713082+0000","last_clean":"2024-05-08T21:17:07.713082+0000","last_became_active":"2024-05-08T21:17:07.260785+0000","last_became_peered":"2024-05-08T21:17:07.260785+0000","last_unstale":"2024-05-08T21:17:07.713082+0000","last_undegraded":"2024-05-08T21:17:07.713082+0000","last_fullsized":"2024-05-08T21:17:07.713082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:58.469134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.311324+0000","last_change":"2024-05-08T21:17:02.729569+0000","last_active":"2024-05-08T21:17:31.311324+0000","last_peered":"2024-05-08T21:17:31.311324+0000","last_clean":"2024-05-08T21:17:31.311324+0000","last_became_active":"2024-05-08T21:17:02.729096+0000","last_became_peered":"2024-05-08T21:17:02.729096+0000","last_unstale":"2024-05-08T21:17:31.311324+0000","last_undegraded":"2024-05-08T21:17:31.311324+0000","last_fullsized":"2024-05-08T21:17:31.311324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:30.668169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489564+0000","last_change":"2024-05-08T21:17:03.246387+0000","last_active":"2024-05-08T21:17:07.489564+0000","last_peered":"2024-05-08T21:17:07.489564+0000","last_clean":"2024-05-08T21:17:07.489564+0000","last_became_active":"2024-05-08T21:17:03.244658+0000","last_became_peered":"2024-05-08T21:17:03.244658+0000","last_unstale":"2024-05-08T21:17:07.489564+0000","last_undegraded":"2024-05-08T21:17:07.489564+0000","last_fullsized":"2024-05-08T21:17:07.489564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:02.602080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714291+0000","last_change":"2024-05-08T21:17:07.261256+0000","last_active":"2024-05-08T21:17:07.714291+0000","last_peered":"2024-05-08T21:17:07.714291+0000","last_clean":"2024-05-08T21:17:07.714291+0000","last_became_active":"2024-05-08T21:17:07.260762+0000","last_became_peered":"2024-05-08T21:17:07.260762+0000","last_unstale":"2024-05-08T21:17:07.714291+0000","last_undegraded":"2024-05-08T21:17:07.714291+0000","last_fullsized":"2024-05-08T21:17:07.714291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:26.329820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.333081+0000","last_change":"2024-05-08T21:17:02.732738+0000","last_active":"2024-05-08T21:17:28.333081+0000","last_peered":"2024-05-08T21:17:28.333081+0000","last_clean":"2024-05-08T21:17:28.333081+0000","last_became_active":"2024-05-08T21:17:02.731733+0000","last_became_peered":"2024-05-08T21:17:02.731733+0000","last_unstale":"2024-05-08T21:17:28.333081+0000","last_undegraded":"2024-05-08T21:17:28.333081+0000","last_fullsized":"2024-05-08T21:17:28.333081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:24.201171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.249185+0000","last_change":"2024-05-08T21:17:03.254405+0000","last_active":"2024-05-08T21:17:07.249185+0000","last_peered":"2024-05-08T21:17:07.249185+0000","last_clean":"2024-05-08T21:17:07.249185+0000","last_became_active":"2024-05-08T21:17:03.252137+0000","last_became_peered":"2024-05-08T21:17:03.252137+0000","last_unstale":"2024-05-08T21:17:07.249185+0000","last_undegraded":"2024-05-08T21:17:07.249185+0000","last_fullsized":"2024-05-08T21:17:07.249185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:15.472225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713049+0000","last_change":"2024-05-08T21:17:07.266325+0000","last_active":"2024-05-08T21:17:07.713049+0000","last_peered":"2024-05-08T21:17:07.713049+0000","last_clean":"2024-05-08T21:17:07.713049+0000","last_became_active":"2024-05-08T21:17:07.265933+0000","last_became_peered":"2024-05-08T21:17:07.265933+0000","last_unstale":"2024-05-08T21:17:07.713049+0000","last_undegraded":"2024-05-08T21:17:07.713049+0000","last_fullsized":"2024-05-08T21:17:07.713049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:46.507275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'12","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.334385+0000","last_change":"2024-05-08T21:17:02.723975+0000","last_active":"2024-05-08T21:17:28.334385+0000","last_peered":"2024-05-08T21:17:28.334385+0000","last_clean":"2024-05-08T21:17:28.334385+0000","last_became_active":"2024-05-08T21:17:02.723631+0000","last_became_peered":"2024-05-08T21:17:02.723631+0000","last_unstale":"2024-05-08T21:17:28.334385+0000","last_undegraded":"2024-05-08T21:17:28.334385+0000","last_fullsized":"2024-05-08T21:17:28.334385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:17.676007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.256353+0000","last_change":"2024-05-08T21:17:03.251570+0000","last_active":"2024-05-08T21:17:07.256353+0000","last_peered":"2024-05-08T21:17:07.256353+0000","last_clean":"2024-05-08T21:17:07.256353+0000","last_became_active":"2024-05-08T21:17:03.251134+0000","last_became_peered":"2024-05-08T21:17:03.251134+0000","last_unstale":"2024-05-08T21:17:07.256353+0000","last_undegraded":"2024-05-08T21:17:07.256353+0000","last_fullsized":"2024-05-08T21:17:07.256353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:28:09.339958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714619+0000","last_change":"2024-05-08T21:17:07.272051+0000","last_active":"2024-05-08T21:17:07.714619+0000","last_peered":"2024-05-08T21:17:07.714619+0000","last_clean":"2024-05-08T21:17:07.714619+0000","last_became_active":"2024-05-08T21:17:07.271616+0000","last_became_peered":"2024-05-08T21:17:07.271616+0000","last_unstale":"2024-05-08T21:17:07.714619+0000","last_undegraded":"2024-05-08T21:17:07.714619+0000","last_fullsized":"2024-05-08T21:17:07.714619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.364731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.312177+0000","last_change":"2024-05-08T21:17:02.736000+0000","last_active":"2024-05-08T21:17:31.312177+0000","last_peered":"2024-05-08T21:17:31.312177+0000","last_clean":"2024-05-08T21:17:31.312177+0000","last_became_active":"2024-05-08T21:17:02.735629+0000","last_became_peered":"2024-05-08T21:17:02.735629+0000","last_unstale":"2024-05-08T21:17:31.312177+0000","last_undegraded":"2024-05-08T21:17:31.312177+0000","last_fullsized":"2024-05-08T21:17:31.312177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:21.842135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.255526+0000","last_change":"2024-05-08T21:17:03.237144+0000","last_active":"2024-05-08T21:17:07.255526+0000","last_peered":"2024-05-08T21:17:07.255526+0000","last_clean":"2024-05-08T21:17:07.255526+0000","last_became_active":"2024-05-08T21:17:03.236792+0000","last_became_peered":"2024-05-08T21:17:03.236792+0000","last_unstale":"2024-05-08T21:17:07.255526+0000","last_undegraded":"2024-05-08T21:17:07.255526+0000","last_fullsized":"2024-05-08T21:17:07.255526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:04.520986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.001388+0000","last_change":"2024-05-08T21:17:07.268311+0000","last_active":"2024-05-08T21:17:08.001388+0000","last_peered":"2024-05-08T21:17:08.001388+0000","last_clean":"2024-05-08T21:17:08.001388+0000","last_became_active":"2024-05-08T21:17:07.267876+0000","last_became_peered":"2024-05-08T21:17:07.267876+0000","last_unstale":"2024-05-08T21:17:08.001388+0000","last_undegraded":"2024-05-08T21:17:08.001388+0000","last_fullsized":"2024-05-08T21:17:08.001388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:19.041321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.335383+0000","last_change":"2024-05-08T21:17:02.727748+0000","last_active":"2024-05-08T21:17:28.335383+0000","last_peered":"2024-05-08T21:17:28.335383+0000","last_clean":"2024-05-08T21:17:28.335383+0000","last_became_active":"2024-05-08T21:17:02.727444+0000","last_became_peered":"2024-05-08T21:17:02.727444+0000","last_unstale":"2024-05-08T21:17:28.335383+0000","last_undegraded":"2024-05-08T21:17:28.335383+0000","last_fullsized":"2024-05-08T21:17:28.335383+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:47.939235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.252351+0000","last_change":"2024-05-08T21:17:03.247185+0000","last_active":"2024-05-08T21:17:07.252351+0000","last_peered":"2024-05-08T21:17:07.252351+0000","last_clean":"2024-05-08T21:17:07.252351+0000","last_became_active":"2024-05-08T21:17:03.246880+0000","last_became_peered":"2024-05-08T21:17:03.246880+0000","last_unstale":"2024-05-08T21:17:07.252351+0000","last_undegraded":"2024-05-08T21:17:07.252351+0000","last_fullsized":"2024-05-08T21:17:07.252351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:38.978353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713572+0000","last_change":"2024-05-08T21:17:07.273710+0000","last_active":"2024-05-08T21:17:07.713572+0000","last_peered":"2024-05-08T21:17:07.713572+0000","last_clean":"2024-05-08T21:17:07.713572+0000","last_became_active":"2024-05-08T21:17:07.273119+0000","last_became_peered":"2024-05-08T21:17:07.273119+0000","last_unstale":"2024-05-08T21:17:07.713572+0000","last_undegraded":"2024-05-08T21:17:07.713572+0000","last_fullsized":"2024-05-08T21:17:07.713572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:30.589440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:32.306086+0000","last_change":"2024-05-08T21:17:02.725508+0000","last_active":"2024-05-08T21:17:32.306086+0000","last_peered":"2024-05-08T21:17:32.306086+0000","last_clean":"2024-05-08T21:17:32.306086+0000","last_became_active":"2024-05-08T21:17:02.725107+0000","last_became_peered":"2024-05-08T21:17:02.725107+0000","last_unstale":"2024-05-08T21:17:32.306086+0000","last_undegraded":"2024-05-08T21:17:32.306086+0000","last_fullsized":"2024-05-08T21:17:32.306086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:13.503702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.254342+0000","last_change":"2024-05-08T21:17:03.245372+0000","last_active":"2024-05-08T21:17:07.254342+0000","last_peered":"2024-05-08T21:17:07.254342+0000","last_clean":"2024-05-08T21:17:07.254342+0000","last_became_active":"2024-05-08T21:17:03.245054+0000","last_became_peered":"2024-05-08T21:17:03.245054+0000","last_unstale":"2024-05-08T21:17:07.254342+0000","last_undegraded":"2024-05-08T21:17:07.254342+0000","last_fullsized":"2024-05-08T21:17:07.254342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:30:25.900846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.044639+0000","last_change":"2024-05-08T21:17:07.272081+0000","last_active":"2024-05-08T21:17:08.044639+0000","last_peered":"2024-05-08T21:17:08.044639+0000","last_clean":"2024-05-08T21:17:08.044639+0000","last_became_active":"2024-05-08T21:17:07.271560+0000","last_became_peered":"2024-05-08T21:17:07.271560+0000","last_unstale":"2024-05-08T21:17:08.044639+0000","last_undegraded":"2024-05-08T21:17:08.044639+0000","last_fullsized":"2024-05-08T21:17:08.044639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:00.490798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.341116+0000","last_change":"2024-05-08T21:17:02.734420+0000","last_active":"2024-05-08T21:17:29.341116+0000","last_peered":"2024-05-08T21:17:29.341116+0000","last_clean":"2024-05-08T21:17:29.341116+0000","last_became_active":"2024-05-08T21:17:02.734071+0000","last_became_peered":"2024-05-08T21:17:02.734071+0000","last_unstale":"2024-05-08T21:17:29.341116+0000","last_undegraded":"2024-05-08T21:17:29.341116+0000","last_fullsized":"2024-05-08T21:17:29.341116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:28.537464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.257017+0000","last_change":"2024-05-08T21:17:03.250706+0000","last_active":"2024-05-08T21:17:07.257017+0000","last_peered":"2024-05-08T21:17:07.257017+0000","last_clean":"2024-05-08T21:17:07.257017+0000","last_became_active":"2024-05-08T21:17:03.250408+0000","last_became_peered":"2024-05-08T21:17:03.250408+0000","last_unstale":"2024-05-08T21:17:07.257017+0000","last_undegraded":"2024-05-08T21:17:07.257017+0000","last_fullsized":"2024-05-08T21:17:07.257017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:43.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713885+0000","last_change":"2024-05-08T21:17:07.272066+0000","last_active":"2024-05-08T21:17:07.713885+0000","last_peered":"2024-05-08T21:17:07.713885+0000","last_clean":"2024-05-08T21:17:07.713885+0000","last_became_active":"2024-05-08T21:17:07.271628+0000","last_became_peered":"2024-05-08T21:17:07.271628+0000","last_unstale":"2024-05-08T21:17:07.713885+0000","last_undegraded":"2024-05-08T21:17:07.713885+0000","last_fullsized":"2024-05-08T21:17:07.713885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:27.731995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.252941+0000","last_change":"2024-05-08T21:17:02.728464+0000","last_active":"2024-05-08T21:17:07.252941+0000","last_peered":"2024-05-08T21:17:07.252941+0000","last_clean":"2024-05-08T21:17:07.252941+0000","last_became_active":"2024-05-08T21:17:02.727914+0000","last_became_peered":"2024-05-08T21:17:02.727914+0000","last_unstale":"2024-05-08T21:17:07.252941+0000","last_undegraded":"2024-05-08T21:17:07.252941+0000","last_fullsized":"2024-05-08T21:17:07.252941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:55.948661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.259706+0000","last_change":"2024-05-08T21:17:03.245578+0000","last_active":"2024-05-08T21:17:07.259706+0000","last_peered":"2024-05-08T21:17:07.259706+0000","last_clean":"2024-05-08T21:17:07.259706+0000","last_became_active":"2024-05-08T21:17:03.245256+0000","last_became_peered":"2024-05-08T21:17:03.245256+0000","last_unstale":"2024-05-08T21:17:07.259706+0000","last_undegraded":"2024-05-08T21:17:07.259706+0000","last_fullsized":"2024-05-08T21:17:07.259706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.002235+0000","last_change":"2024-05-08T21:17:07.439525+0000","last_active":"2024-05-08T21:17:08.002235+0000","last_peered":"2024-05-08T21:17:08.002235+0000","last_clean":"2024-05-08T21:17:08.002235+0000","last_became_active":"2024-05-08T21:17:07.438987+0000","last_became_peered":"2024-05-08T21:17:07.438987+0000","last_unstale":"2024-05-08T21:17:08.002235+0000","last_undegraded":"2024-05-08T21:17:08.002235+0000","last_fullsized":"2024-05-08T21:17:08.002235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:57:45.373981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.248207+0000","last_change":"2024-05-08T21:17:02.725912+0000","last_active":"2024-05-08T21:17:07.248207+0000","last_peered":"2024-05-08T21:17:07.248207+0000","last_clean":"2024-05-08T21:17:07.248207+0000","last_became_active":"2024-05-08T21:17:02.725537+0000","last_became_peered":"2024-05-08T21:17:02.725537+0000","last_unstale":"2024-05-08T21:17:07.248207+0000","last_undegraded":"2024-05-08T21:17:07.248207+0000","last_fullsized":"2024-05-08T21:17:07.248207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:28.184551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.437903+0000","last_change":"2024-05-08T21:17:03.235556+0000","last_active":"2024-05-08T21:17:07.437903+0000","last_peered":"2024-05-08T21:17:07.437903+0000","last_clean":"2024-05-08T21:17:07.437903+0000","last_became_active":"2024-05-08T21:17:03.235214+0000","last_became_peered":"2024-05-08T21:17:03.235214+0000","last_unstale":"2024-05-08T21:17:07.437903+0000","last_undegraded":"2024-05-08T21:17:07.437903+0000","last_fullsized":"2024-05-08T21:17:07.437903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:45.042596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.044143+0000","last_change":"2024-05-08T21:17:07.266080+0000","last_active":"2024-05-08T21:17:08.044143+0000","last_peered":"2024-05-08T21:17:08.044143+0000","last_clean":"2024-05-08T21:17:08.044143+0000","last_became_active":"2024-05-08T21:17:07.265605+0000","last_became_peered":"2024-05-08T21:17:07.265605+0000","last_unstale":"2024-05-08T21:17:08.044143+0000","last_undegraded":"2024-05-08T21:17:08.044143+0000","last_fullsized":"2024-05-08T21:17:08.044143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:49:48.387170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.311476+0000","last_change":"2024-05-08T21:17:02.728743+0000","last_active":"2024-05-08T21:17:31.311476+0000","last_peered":"2024-05-08T21:17:31.311476+0000","last_clean":"2024-05-08T21:17:31.311476+0000","last_became_active":"2024-05-08T21:17:02.728330+0000","last_became_peered":"2024-05-08T21:17:02.728330+0000","last_unstale":"2024-05-08T21:17:31.311476+0000","last_undegraded":"2024-05-08T21:17:31.311476+0000","last_fullsized":"2024-05-08T21:17:31.311476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:43.865117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.437816+0000","last_change":"2024-05-08T21:17:03.245672+0000","last_active":"2024-05-08T21:17:07.437816+0000","last_peered":"2024-05-08T21:17:07.437816+0000","last_clean":"2024-05-08T21:17:07.437816+0000","last_became_active":"2024-05-08T21:17:03.245318+0000","last_became_peered":"2024-05-08T21:17:03.245318+0000","last_unstale":"2024-05-08T21:17:07.437816+0000","last_undegraded":"2024-05-08T21:17:07.437816+0000","last_fullsized":"2024-05-08T21:17:07.437816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:43.030011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.044374+0000","last_change":"2024-05-08T21:17:07.439289+0000","last_active":"2024-05-08T21:17:08.044374+0000","last_peered":"2024-05-08T21:17:08.044374+0000","last_clean":"2024-05-08T21:17:08.044374+0000","last_became_active":"2024-05-08T21:17:07.438874+0000","last_became_peered":"2024-05-08T21:17:07.438874+0000","last_unstale":"2024-05-08T21:17:08.044374+0000","last_undegraded":"2024-05-08T21:17:08.044374+0000","last_fullsized":"2024-05-08T21:17:08.044374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:14.820087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.257823+0000","last_change":"2024-05-08T21:17:03.247081+0000","last_active":"2024-05-08T21:17:07.257823+0000","last_peered":"2024-05-08T21:17:07.257823+0000","last_clean":"2024-05-08T21:17:07.257823+0000","last_became_active":"2024-05-08T21:17:03.246418+0000","last_became_peered":"2024-05-08T21:17:03.246418+0000","last_unstale":"2024-05-08T21:17:07.257823+0000","last_undegraded":"2024-05-08T21:17:07.257823+0000","last_fullsized":"2024-05-08T21:17:07.257823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:03.993770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:32.311386+0000","last_change":"2024-05-08T21:17:02.730764+0000","last_active":"2024-05-08T21:17:32.311386+0000","last_peered":"2024-05-08T21:17:32.311386+0000","last_clean":"2024-05-08T21:17:32.311386+0000","last_became_active":"2024-05-08T21:17:02.730494+0000","last_became_peered":"2024-05-08T21:17:02.730494+0000","last_unstale":"2024-05-08T21:17:32.311386+0000","last_undegraded":"2024-05-08T21:17:32.311386+0000","last_fullsized":"2024-05-08T21:17:32.311386+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:50.070368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.266792+0000","last_change":"2024-05-08T21:17:07.266424+0000","last_active":"2024-05-08T21:17:07.266792+0000","last_peered":"2024-05-08T21:17:07.266792+0000","last_clean":"2024-05-08T21:17:07.266792+0000","last_became_active":"2024-05-08T21:17:07.266144+0000","last_became_peered":"2024-05-08T21:17:07.266144+0000","last_unstale":"2024-05-08T21:17:07.266792+0000","last_undegraded":"2024-05-08T21:17:07.266792+0000","last_fullsized":"2024-05-08T21:17:07.266792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:22.285749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.263866+0000","last_change":"2024-05-08T21:17:03.250461+0000","last_active":"2024-05-08T21:17:07.263866+0000","last_peered":"2024-05-08T21:17:07.263866+0000","last_clean":"2024-05-08T21:17:07.263866+0000","last_became_active":"2024-05-08T21:17:03.249974+0000","last_became_peered":"2024-05-08T21:17:03.249974+0000","last_unstale":"2024-05-08T21:17:07.263866+0000","last_undegraded":"2024-05-08T21:17:07.263866+0000","last_fullsized":"2024-05-08T21:17:07.263866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:25:58.759668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.311413+0000","last_change":"2024-05-08T21:17:02.725648+0000","last_active":"2024-05-08T21:17:31.311413+0000","last_peered":"2024-05-08T21:17:31.311413+0000","last_clean":"2024-05-08T21:17:31.311413+0000","last_became_active":"2024-05-08T21:17:02.725297+0000","last_became_peered":"2024-05-08T21:17:02.725297+0000","last_unstale":"2024-05-08T21:17:31.311413+0000","last_undegraded":"2024-05-08T21:17:31.311413+0000","last_fullsized":"2024-05-08T21:17:31.311413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:41.462830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":326,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.001480+0000","last_change":"2024-05-08T21:16:15.194478+0000","last_active":"2024-05-08T21:17:08.001480+0000","last_peered":"2024-05-08T21:17:08.001480+0000","last_clean":"2024-05-08T21:17:08.001480+0000","last_became_active":"2024-05-08T21:16:15.193964+0000","last_became_peered":"2024-05-08T21:16:15.193964+0000","last_unstale":"2024-05-08T21:17:08.001480+0000","last_undegraded":"2024-05-08T21:17:08.001480+0000","last_fullsized":"2024-05-08T21:17:08.001480+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_clean_scrub_stamp":"2024-05-08T21:13:48.584026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:28.091200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714199+0000","last_change":"2024-05-08T21:17:07.439697+0000","last_active":"2024-05-08T21:17:07.714199+0000","last_peered":"2024-05-08T21:17:07.714199+0000","last_clean":"2024-05-08T21:17:07.714199+0000","last_became_active":"2024-05-08T21:17:07.439027+0000","last_became_peered":"2024-05-08T21:17:07.439027+0000","last_unstale":"2024-05-08T21:17:07.714199+0000","last_undegraded":"2024-05-08T21:17:07.714199+0000","last_fullsized":"2024-05-08T21:17:07.714199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:12.585241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.254478+0000","last_change":"2024-05-08T21:17:03.246034+0000","last_active":"2024-05-08T21:17:07.254478+0000","last_peered":"2024-05-08T21:17:07.254478+0000","last_clean":"2024-05-08T21:17:07.254478+0000","last_became_active":"2024-05-08T21:17:03.245620+0000","last_became_peered":"2024-05-08T21:17:03.245620+0000","last_unstale":"2024-05-08T21:17:07.254478+0000","last_undegraded":"2024-05-08T21:17:07.254478+0000","last_fullsized":"2024-05-08T21:17:07.254478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:02.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.340863+0000","last_change":"2024-05-08T21:17:02.731260+0000","last_active":"2024-05-08T21:17:29.340863+0000","last_peered":"2024-05-08T21:17:29.340863+0000","last_clean":"2024-05-08T21:17:29.340863+0000","last_became_active":"2024-05-08T21:17:02.730879+0000","last_became_peered":"2024-05-08T21:17:02.730879+0000","last_unstale":"2024-05-08T21:17:29.340863+0000","last_undegraded":"2024-05-08T21:17:29.340863+0000","last_fullsized":"2024-05-08T21:17:29.340863+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:59.030674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.000869+0000","last_change":"2024-05-08T21:17:07.269791+0000","last_active":"2024-05-08T21:17:08.000869+0000","last_peered":"2024-05-08T21:17:08.000869+0000","last_clean":"2024-05-08T21:17:08.000869+0000","last_became_active":"2024-05-08T21:17:07.269345+0000","last_became_peered":"2024-05-08T21:17:07.269345+0000","last_unstale":"2024-05-08T21:17:08.000869+0000","last_undegraded":"2024-05-08T21:17:08.000869+0000","last_fullsized":"2024-05-08T21:17:08.000869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:42.866864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.001283+0000","last_change":"2024-05-08T21:17:02.735423+0000","last_active":"2024-05-08T21:17:08.001283+0000","last_peered":"2024-05-08T21:17:08.001283+0000","last_clean":"2024-05-08T21:17:08.001283+0000","last_became_active":"2024-05-08T21:17:02.735158+0000","last_became_peered":"2024-05-08T21:17:02.735158+0000","last_unstale":"2024-05-08T21:17:08.001283+0000","last_undegraded":"2024-05-08T21:17:08.001283+0000","last_fullsized":"2024-05-08T21:17:08.001283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:19.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.262817+0000","last_change":"2024-05-08T21:17:03.239316+0000","last_active":"2024-05-08T21:17:07.262817+0000","last_peered":"2024-05-08T21:17:07.262817+0000","last_clean":"2024-05-08T21:17:07.262817+0000","last_became_active":"2024-05-08T21:17:03.239018+0000","last_became_peered":"2024-05-08T21:17:03.239018+0000","last_unstale":"2024-05-08T21:17:07.262817+0000","last_undegraded":"2024-05-08T21:17:07.262817+0000","last_fullsized":"2024-05-08T21:17:07.262817+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:36.952155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.970907+0000","last_change":"2024-05-08T21:17:07.439338+0000","last_active":"2024-05-08T21:17:07.970907+0000","last_peered":"2024-05-08T21:17:07.970907+0000","last_clean":"2024-05-08T21:17:07.970907+0000","last_became_active":"2024-05-08T21:17:07.438703+0000","last_became_peered":"2024-05-08T21:17:07.438703+0000","last_unstale":"2024-05-08T21:17:07.970907+0000","last_undegraded":"2024-05-08T21:17:07.970907+0000","last_fullsized":"2024-05-08T21:17:07.970907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:50.997138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.340605+0000","last_change":"2024-05-08T21:17:02.729012+0000","last_active":"2024-05-08T21:17:29.340605+0000","last_peered":"2024-05-08T21:17:29.340605+0000","last_clean":"2024-05-08T21:17:29.340605+0000","last_became_active":"2024-05-08T21:17:02.728606+0000","last_became_peered":"2024-05-08T21:17:02.728606+0000","last_unstale":"2024-05-08T21:17:29.340605+0000","last_undegraded":"2024-05-08T21:17:29.340605+0000","last_fullsized":"2024-05-08T21:17:29.340605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.140609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.268112+0000","last_change":"2024-05-08T21:17:03.234712+0000","last_active":"2024-05-08T21:17:07.268112+0000","last_peered":"2024-05-08T21:17:07.268112+0000","last_clean":"2024-05-08T21:17:07.268112+0000","last_became_active":"2024-05-08T21:17:03.234404+0000","last_became_peered":"2024-05-08T21:17:03.234404+0000","last_unstale":"2024-05-08T21:17:07.268112+0000","last_undegraded":"2024-05-08T21:17:07.268112+0000","last_fullsized":"2024-05-08T21:17:07.268112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:41.479470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.259891+0000","last_change":"2024-05-08T21:17:07.259120+0000","last_active":"2024-05-08T21:17:07.259891+0000","last_peered":"2024-05-08T21:17:07.259891+0000","last_clean":"2024-05-08T21:17:07.259891+0000","last_became_active":"2024-05-08T21:17:07.258696+0000","last_became_peered":"2024-05-08T21:17:07.258696+0000","last_unstale":"2024-05-08T21:17:07.259891+0000","last_undegraded":"2024-05-08T21:17:07.259891+0000","last_fullsized":"2024-05-08T21:17:07.259891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:26.950767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.311398+0000","last_change":"2024-05-08T21:17:02.727795+0000","last_active":"2024-05-08T21:17:31.311398+0000","last_peered":"2024-05-08T21:17:31.311398+0000","last_clean":"2024-05-08T21:17:31.311398+0000","last_became_active":"2024-05-08T21:17:02.727553+0000","last_became_peered":"2024-05-08T21:17:02.727553+0000","last_unstale":"2024-05-08T21:17:31.311398+0000","last_undegraded":"2024-05-08T21:17:31.311398+0000","last_fullsized":"2024-05-08T21:17:31.311398+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:51.540221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489521+0000","last_change":"2024-05-08T21:17:03.247270+0000","last_active":"2024-05-08T21:17:07.489521+0000","last_peered":"2024-05-08T21:17:07.489521+0000","last_clean":"2024-05-08T21:17:07.489521+0000","last_became_active":"2024-05-08T21:17:03.246853+0000","last_became_peered":"2024-05-08T21:17:03.246853+0000","last_unstale":"2024-05-08T21:17:07.489521+0000","last_undegraded":"2024-05-08T21:17:07.489521+0000","last_fullsized":"2024-05-08T21:17:07.489521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:46.224088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714459+0000","last_change":"2024-05-08T21:17:07.439920+0000","last_active":"2024-05-08T21:17:07.714459+0000","last_peered":"2024-05-08T21:17:07.714459+0000","last_clean":"2024-05-08T21:17:07.714459+0000","last_became_active":"2024-05-08T21:17:07.439459+0000","last_became_peered":"2024-05-08T21:17:07.439459+0000","last_unstale":"2024-05-08T21:17:07.714459+0000","last_undegraded":"2024-05-08T21:17:07.714459+0000","last_fullsized":"2024-05-08T21:17:07.714459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:03.008930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.334698+0000","last_change":"2024-05-08T21:17:02.731889+0000","last_active":"2024-05-08T21:17:28.334698+0000","last_peered":"2024-05-08T21:17:28.334698+0000","last_clean":"2024-05-08T21:17:28.334698+0000","last_became_active":"2024-05-08T21:17:02.731554+0000","last_became_peered":"2024-05-08T21:17:02.731554+0000","last_unstale":"2024-05-08T21:17:28.334698+0000","last_undegraded":"2024-05-08T21:17:28.334698+0000","last_fullsized":"2024-05-08T21:17:28.334698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:25.223003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.252690+0000","last_change":"2024-05-08T21:17:03.252672+0000","last_active":"2024-05-08T21:17:07.252690+0000","last_peered":"2024-05-08T21:17:07.252690+0000","last_clean":"2024-05-08T21:17:07.252690+0000","last_became_active":"2024-05-08T21:17:03.251684+0000","last_became_peered":"2024-05-08T21:17:03.251684+0000","last_unstale":"2024-05-08T21:17:07.252690+0000","last_undegraded":"2024-05-08T21:17:07.252690+0000","last_fullsized":"2024-05-08T21:17:07.252690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:06.460419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713656+0000","last_change":"2024-05-08T21:17:07.273478+0000","last_active":"2024-05-08T21:17:07.713656+0000","last_peered":"2024-05-08T21:17:07.713656+0000","last_clean":"2024-05-08T21:17:07.713656+0000","last_became_active":"2024-05-08T21:17:07.272525+0000","last_became_peered":"2024-05-08T21:17:07.272525+0000","last_unstale":"2024-05-08T21:17:07.713656+0000","last_undegraded":"2024-05-08T21:17:07.713656+0000","last_fullsized":"2024-05-08T21:17:07.713656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:51:05.107387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:30.356496+0000","last_change":"2024-05-08T21:17:02.728858+0000","last_active":"2024-05-08T21:17:30.356496+0000","last_peered":"2024-05-08T21:17:30.356496+0000","last_clean":"2024-05-08T21:17:30.356496+0000","last_became_active":"2024-05-08T21:17:02.728516+0000","last_became_peered":"2024-05-08T21:17:02.728516+0000","last_unstale":"2024-05-08T21:17:30.356496+0000","last_undegraded":"2024-05-08T21:17:30.356496+0000","last_fullsized":"2024-05-08T21:17:30.356496+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:40.263983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.253808+0000","last_change":"2024-05-08T21:17:03.239354+0000","last_active":"2024-05-08T21:17:07.253808+0000","last_peered":"2024-05-08T21:17:07.253808+0000","last_clean":"2024-05-08T21:17:07.253808+0000","last_became_active":"2024-05-08T21:17:03.239020+0000","last_became_peered":"2024-05-08T21:17:03.239020+0000","last_unstale":"2024-05-08T21:17:07.253808+0000","last_undegraded":"2024-05-08T21:17:07.253808+0000","last_fullsized":"2024-05-08T21:17:07.253808+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:02:43.362016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.349903+0000","last_change":"2024-05-08T21:17:07.262401+0000","last_active":"2024-05-08T21:17:08.349903+0000","last_peered":"2024-05-08T21:17:08.349903+0000","last_clean":"2024-05-08T21:17:08.349903+0000","last_became_active":"2024-05-08T21:17:07.261946+0000","last_became_peered":"2024-05-08T21:17:07.261946+0000","last_unstale":"2024-05-08T21:17:08.349903+0000","last_undegraded":"2024-05-08T21:17:08.349903+0000","last_fullsized":"2024-05-08T21:17:08.349903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:56.565604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.873113+0000","last_change":"2024-05-08T21:17:02.731124+0000","last_active":"2024-05-08T21:17:29.873113+0000","last_peered":"2024-05-08T21:17:29.873113+0000","last_clean":"2024-05-08T21:17:29.873113+0000","last_became_active":"2024-05-08T21:17:02.730759+0000","last_became_peered":"2024-05-08T21:17:02.730759+0000","last_unstale":"2024-05-08T21:17:29.873113+0000","last_undegraded":"2024-05-08T21:17:29.873113+0000","last_fullsized":"2024-05-08T21:17:29.873113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:07.213439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.263623+0000","last_change":"2024-05-08T21:17:03.250382+0000","last_active":"2024-05-08T21:17:07.263623+0000","last_peered":"2024-05-08T21:17:07.263623+0000","last_clean":"2024-05-08T21:17:07.263623+0000","last_became_active":"2024-05-08T21:17:03.250031+0000","last_became_peered":"2024-05-08T21:17:03.250031+0000","last_unstale":"2024-05-08T21:17:07.263623+0000","last_undegraded":"2024-05-08T21:17:07.263623+0000","last_fullsized":"2024-05-08T21:17:07.263623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:59.961012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714358+0000","last_change":"2024-05-08T21:17:07.439341+0000","last_active":"2024-05-08T21:17:07.714358+0000","last_peered":"2024-05-08T21:17:07.714358+0000","last_clean":"2024-05-08T21:17:07.714358+0000","last_became_active":"2024-05-08T21:17:07.438896+0000","last_became_peered":"2024-05-08T21:17:07.438896+0000","last_unstale":"2024-05-08T21:17:07.714358+0000","last_undegraded":"2024-05-08T21:17:07.714358+0000","last_fullsized":"2024-05-08T21:17:07.714358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:55.180686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.334782+0000","last_change":"2024-05-08T21:17:02.732967+0000","last_active":"2024-05-08T21:17:28.334782+0000","last_peered":"2024-05-08T21:17:28.334782+0000","last_clean":"2024-05-08T21:17:28.334782+0000","last_became_active":"2024-05-08T21:17:02.732616+0000","last_became_peered":"2024-05-08T21:17:02.732616+0000","last_unstale":"2024-05-08T21:17:28.334782+0000","last_undegraded":"2024-05-08T21:17:28.334782+0000","last_fullsized":"2024-05-08T21:17:28.334782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:50:26.116841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.257264+0000","last_change":"2024-05-08T21:17:03.241974+0000","last_active":"2024-05-08T21:17:07.257264+0000","last_peered":"2024-05-08T21:17:07.257264+0000","last_clean":"2024-05-08T21:17:07.257264+0000","last_became_active":"2024-05-08T21:17:03.241627+0000","last_became_peered":"2024-05-08T21:17:03.241627+0000","last_unstale":"2024-05-08T21:17:07.257264+0000","last_undegraded":"2024-05-08T21:17:07.257264+0000","last_fullsized":"2024-05-08T21:17:07.257264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:57.351078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.002041+0000","last_change":"2024-05-08T21:17:07.259593+0000","last_active":"2024-05-08T21:17:08.002041+0000","last_peered":"2024-05-08T21:17:08.002041+0000","last_clean":"2024-05-08T21:17:08.002041+0000","last_became_active":"2024-05-08T21:17:07.258362+0000","last_became_peered":"2024-05-08T21:17:07.258362+0000","last_unstale":"2024-05-08T21:17:08.002041+0000","last_undegraded":"2024-05-08T21:17:08.002041+0000","last_fullsized":"2024-05-08T21:17:08.002041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:07.877264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.304152+0000","last_change":"2024-05-08T21:17:02.731685+0000","last_active":"2024-05-08T21:17:31.304152+0000","last_peered":"2024-05-08T21:17:31.304152+0000","last_clean":"2024-05-08T21:17:31.304152+0000","last_became_active":"2024-05-08T21:17:02.731293+0000","last_became_peered":"2024-05-08T21:17:02.731293+0000","last_unstale":"2024-05-08T21:17:31.304152+0000","last_undegraded":"2024-05-08T21:17:31.304152+0000","last_fullsized":"2024-05-08T21:17:31.304152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:36.532962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.256159+0000","last_change":"2024-05-08T21:17:03.250090+0000","last_active":"2024-05-08T21:17:07.256159+0000","last_peered":"2024-05-08T21:17:07.256159+0000","last_clean":"2024-05-08T21:17:07.256159+0000","last_became_active":"2024-05-08T21:17:03.249744+0000","last_became_peered":"2024-05-08T21:17:03.249744+0000","last_unstale":"2024-05-08T21:17:07.256159+0000","last_undegraded":"2024-05-08T21:17:07.256159+0000","last_fullsized":"2024-05-08T21:17:07.256159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:43.155040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489433+0000","last_change":"2024-05-08T21:17:07.264101+0000","last_active":"2024-05-08T21:17:07.489433+0000","last_peered":"2024-05-08T21:17:07.489433+0000","last_clean":"2024-05-08T21:17:07.489433+0000","last_became_active":"2024-05-08T21:17:07.263720+0000","last_became_peered":"2024-05-08T21:17:07.263720+0000","last_unstale":"2024-05-08T21:17:07.489433+0000","last_undegraded":"2024-05-08T21:17:07.489433+0000","last_fullsized":"2024-05-08T21:17:07.489433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:16.943044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714336+0000","last_change":"2024-05-08T21:17:02.726763+0000","last_active":"2024-05-08T21:17:07.714336+0000","last_peered":"2024-05-08T21:17:07.714336+0000","last_clean":"2024-05-08T21:17:07.714336+0000","last_became_active":"2024-05-08T21:17:02.726343+0000","last_became_peered":"2024-05-08T21:17:02.726343+0000","last_unstale":"2024-05-08T21:17:07.714336+0000","last_undegraded":"2024-05-08T21:17:07.714336+0000","last_fullsized":"2024-05-08T21:17:07.714336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:03.504667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.252549+0000","last_change":"2024-05-08T21:17:03.244468+0000","last_active":"2024-05-08T21:17:07.252549+0000","last_peered":"2024-05-08T21:17:07.252549+0000","last_clean":"2024-05-08T21:17:07.252549+0000","last_became_active":"2024-05-08T21:17:03.244132+0000","last_became_peered":"2024-05-08T21:17:03.244132+0000","last_unstale":"2024-05-08T21:17:07.252549+0000","last_undegraded":"2024-05-08T21:17:07.252549+0000","last_fullsized":"2024-05-08T21:17:07.252549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:17.423657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713671+0000","last_change":"2024-05-08T21:17:07.260981+0000","last_active":"2024-05-08T21:17:07.713671+0000","last_peered":"2024-05-08T21:17:07.713671+0000","last_clean":"2024-05-08T21:17:07.713671+0000","last_became_active":"2024-05-08T21:17:07.260582+0000","last_became_peered":"2024-05-08T21:17:07.260582+0000","last_unstale":"2024-05-08T21:17:07.713671+0000","last_undegraded":"2024-05-08T21:17:07.713671+0000","last_fullsized":"2024-05-08T21:17:07.713671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:25.319411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.310205+0000","last_change":"2024-05-08T21:17:02.732531+0000","last_active":"2024-05-08T21:17:31.310205+0000","last_peered":"2024-05-08T21:17:31.310205+0000","last_clean":"2024-05-08T21:17:31.310205+0000","last_became_active":"2024-05-08T21:17:02.732121+0000","last_became_peered":"2024-05-08T21:17:02.732121+0000","last_unstale":"2024-05-08T21:17:31.310205+0000","last_undegraded":"2024-05-08T21:17:31.310205+0000","last_fullsized":"2024-05-08T21:17:31.310205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:25.482458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.254189+0000","last_change":"2024-05-08T21:17:03.249676+0000","last_active":"2024-05-08T21:17:07.254189+0000","last_peered":"2024-05-08T21:17:07.254189+0000","last_clean":"2024-05-08T21:17:07.254189+0000","last_became_active":"2024-05-08T21:17:03.249120+0000","last_became_peered":"2024-05-08T21:17:03.249120+0000","last_unstale":"2024-05-08T21:17:07.254189+0000","last_undegraded":"2024-05-08T21:17:07.254189+0000","last_fullsized":"2024-05-08T21:17:07.254189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:58:40.678800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.253901+0000","last_change":"2024-05-08T21:17:03.245585+0000","last_active":"2024-05-08T21:17:07.253901+0000","last_peered":"2024-05-08T21:17:07.253901+0000","last_clean":"2024-05-08T21:17:07.253901+0000","last_became_active":"2024-05-08T21:17:03.245274+0000","last_became_peered":"2024-05-08T21:17:03.245274+0000","last_unstale":"2024-05-08T21:17:07.253901+0000","last_undegraded":"2024-05-08T21:17:07.253901+0000","last_fullsized":"2024-05-08T21:17:07.253901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:32.003962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.250334+0000","last_change":"2024-05-08T21:17:03.246622+0000","last_active":"2024-05-08T21:17:07.250334+0000","last_peered":"2024-05-08T21:17:07.250334+0000","last_clean":"2024-05-08T21:17:07.250334+0000","last_became_active":"2024-05-08T21:17:03.246309+0000","last_became_peered":"2024-05-08T21:17:03.246309+0000","last_unstale":"2024-05-08T21:17:07.250334+0000","last_undegraded":"2024-05-08T21:17:07.250334+0000","last_fullsized":"2024-05-08T21:17:07.250334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:32.439389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.248595+0000","last_change":"2024-05-08T21:17:03.243460+0000","last_active":"2024-05-08T21:17:07.248595+0000","last_peered":"2024-05-08T21:17:07.248595+0000","last_clean":"2024-05-08T21:17:07.248595+0000","last_became_active":"2024-05-08T21:17:03.243097+0000","last_became_peered":"2024-05-08T21:17:03.243097+0000","last_unstale":"2024-05-08T21:17:07.248595+0000","last_undegraded":"2024-05-08T21:17:07.248595+0000","last_fullsized":"2024-05-08T21:17:07.248595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:52:14.492180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489451+0000","last_change":"2024-05-08T21:17:03.245077+0000","last_active":"2024-05-08T21:17:07.489451+0000","last_peered":"2024-05-08T21:17:07.489451+0000","last_clean":"2024-05-08T21:17:07.489451+0000","last_became_active":"2024-05-08T21:17:03.244719+0000","last_became_peered":"2024-05-08T21:17:03.244719+0000","last_unstale":"2024-05-08T21:17:07.489451+0000","last_undegraded":"2024-05-08T21:17:07.489451+0000","last_fullsized":"2024-05-08T21:17:07.489451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.388318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.262726+0000","last_change":"2024-05-08T21:17:03.245261+0000","last_active":"2024-05-08T21:17:07.262726+0000","last_peered":"2024-05-08T21:17:07.262726+0000","last_clean":"2024-05-08T21:17:07.262726+0000","last_became_active":"2024-05-08T21:17:03.244892+0000","last_became_peered":"2024-05-08T21:17:03.244892+0000","last_unstale":"2024-05-08T21:17:07.262726+0000","last_undegraded":"2024-05-08T21:17:07.262726+0000","last_fullsized":"2024-05-08T21:17:07.262726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:21.075634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.259640+0000","last_change":"2024-05-08T21:17:03.246250+0000","last_active":"2024-05-08T21:17:07.259640+0000","last_peered":"2024-05-08T21:17:07.259640+0000","last_clean":"2024-05-08T21:17:07.259640+0000","last_became_active":"2024-05-08T21:17:03.245807+0000","last_became_peered":"2024-05-08T21:17:03.245807+0000","last_unstale":"2024-05-08T21:17:07.259640+0000","last_undegraded":"2024-05-08T21:17:07.259640+0000","last_fullsized":"2024-05-08T21:17:07.259640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:20.618260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.250263+0000","last_change":"2024-05-08T21:17:03.254559+0000","last_active":"2024-05-08T21:17:07.250263+0000","last_peered":"2024-05-08T21:17:07.250263+0000","last_clean":"2024-05-08T21:17:07.250263+0000","last_became_active":"2024-05-08T21:17:03.254231+0000","last_became_peered":"2024-05-08T21:17:03.254231+0000","last_unstale":"2024-05-08T21:17:07.250263+0000","last_undegraded":"2024-05-08T21:17:07.250263+0000","last_fullsized":"2024-05-08T21:17:07.250263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:47.996927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.268225+0000","last_change":"2024-05-08T21:17:03.242479+0000","last_active":"2024-05-08T21:17:07.268225+0000","last_peered":"2024-05-08T21:17:07.268225+0000","last_clean":"2024-05-08T21:17:07.268225+0000","last_became_active":"2024-05-08T21:17:03.241397+0000","last_became_peered":"2024-05-08T21:17:03.241397+0000","last_unstale":"2024-05-08T21:17:07.268225+0000","last_undegraded":"2024-05-08T21:17:07.268225+0000","last_fullsized":"2024-05-08T21:17:07.268225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:39.029139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.257663+0000","last_change":"2024-05-08T21:17:03.244208+0000","last_active":"2024-05-08T21:17:07.257663+0000","last_peered":"2024-05-08T21:17:07.257663+0000","last_clean":"2024-05-08T21:17:07.257663+0000","last_became_active":"2024-05-08T21:17:03.243828+0000","last_became_peered":"2024-05-08T21:17:03.243828+0000","last_unstale":"2024-05-08T21:17:07.257663+0000","last_undegraded":"2024-05-08T21:17:07.257663+0000","last_fullsized":"2024-05-08T21:17:07.257663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:39.936750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.263374+0000","last_change":"2024-05-08T21:17:03.240135+0000","last_active":"2024-05-08T21:17:07.263374+0000","last_peered":"2024-05-08T21:17:07.263374+0000","last_clean":"2024-05-08T21:17:07.263374+0000","last_became_active":"2024-05-08T21:17:03.239757+0000","last_became_peered":"2024-05-08T21:17:03.239757+0000","last_unstale":"2024-05-08T21:17:07.263374+0000","last_undegraded":"2024-05-08T21:17:07.263374+0000","last_fullsized":"2024-05-08T21:17:07.263374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:46.066111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.249570+0000","last_change":"2024-05-08T21:17:03.256254+0000","last_active":"2024-05-08T21:17:07.249570+0000","last_peered":"2024-05-08T21:17:07.249570+0000","last_clean":"2024-05-08T21:17:07.249570+0000","last_became_active":"2024-05-08T21:17:03.255310+0000","last_became_peered":"2024-05-08T21:17:03.255310+0000","last_unstale":"2024-05-08T21:17:07.249570+0000","last_undegraded":"2024-05-08T21:17:07.249570+0000","last_fullsized":"2024-05-08T21:17:07.249570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:59.571958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.254974+0000","last_change":"2024-05-08T21:17:03.241901+0000","last_active":"2024-05-08T21:17:07.254974+0000","last_peered":"2024-05-08T21:17:07.254974+0000","last_clean":"2024-05-08T21:17:07.254974+0000","last_became_active":"2024-05-08T21:17:03.241402+0000","last_became_peered":"2024-05-08T21:17:03.241402+0000","last_unstale":"2024-05-08T21:17:07.254974+0000","last_undegraded":"2024-05-08T21:17:07.254974+0000","last_fullsized":"2024-05-08T21:17:07.254974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:59.202987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.254310+0000","last_change":"2024-05-08T21:17:03.243050+0000","last_active":"2024-05-08T21:17:07.254310+0000","last_peered":"2024-05-08T21:17:07.254310+0000","last_clean":"2024-05-08T21:17:07.254310+0000","last_became_active":"2024-05-08T21:17:03.240676+0000","last_became_peered":"2024-05-08T21:17:03.240676+0000","last_unstale":"2024-05-08T21:17:07.254310+0000","last_undegraded":"2024-05-08T21:17:07.254310+0000","last_fullsized":"2024-05-08T21:17:07.254310+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:19:53.981559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9657,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":172,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":348160,"data_stored":45613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514511,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":459288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":10,"up_from":69,"seq":296352743442,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32656,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712496,"statfs":{"total":95995035648,"available":95961595904,"internally_reserved":0,"allocated":1581056,"data_stored":1054775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.71199999999999997}]},{"osd":3,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.627}]},{"osd":4,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.79400000000000004}]},{"osd":5,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.77600000000000002}]},{"osd":6,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.746},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.72399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":8,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.90700000000000003}]},{"osd":9,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.88100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":9,"up_from":64,"seq":274877906965,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":460072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51500000000000001}]},{"osd":1,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.81100000000000005}]},{"osd":2,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63400000000000001}]},{"osd":3,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.70899999999999996}]},{"osd":4,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.746}]},{"osd":5,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.89400000000000002}]},{"osd":6,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.875},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.47099999999999997}]},{"osd":7,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.92300000000000004}]},{"osd":8,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.94099999999999995}]},{"osd":10,"last update":"Wed May 8 21:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.99099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":8,"up_from":56,"seq":240518168602,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":459695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.628}]},{"osd":1,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.38400000000000001}]},{"osd":3,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64900000000000002}]},{"osd":4,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.60999999999999999}]},{"osd":5,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.434}]},{"osd":6,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77100000000000002}]},{"osd":9,"last update":"Wed May 8 21:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.746},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.75700000000000001}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.82299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":7,"up_from":49,"seq":210453397533,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1051420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.26300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.40400000000000003,"15min":0.39800000000000002},"min":{"1min":0.27700000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58799999999999997}]},{"osd":3,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.33600000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.745},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.316,"5min":0.315,"15min":0.315},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.38700000000000001}]},{"osd":5,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.46800000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.45900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51700000000000002}]},{"osd":6,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.50800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.503}]},{"osd":8,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.73699999999999999}]},{"osd":9,"last update":"Wed May 8 21:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.64800000000000002}]},{"osd":10,"last update":"Wed May 8 21:17:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68799999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561058,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":459962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45900000000000002,"15min":0.46600000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65300000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59899999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.432,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.63100000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.73599999999999999}]},{"osd":2,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.59699999999999998},"min":{"1min":0.438,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":1.087,"15min":1.087},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73099999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58499999999999996}]},{"osd":3,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.35699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.753,"15min":0.753},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83599999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.49099999999999999}]},{"osd":4,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60499999999999998,"15min":0.59999999999999998},"min":{"1min":0.48899999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80400000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71999999999999997}]},{"osd":5,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.51700000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88100000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.755}]},{"osd":7,"last update":"Wed May 8 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90900000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0169999999999999,"5min":1.034,"15min":1.034},"last":0.79400000000000004}]},{"osd":8,"last update":"Wed May 8 21:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.749},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78700000000000003}]},{"osd":9,"last update":"Wed May 8 21:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.82499999999999996}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.86499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]}]},{"osd":1,"up_from":12,"seq":51539607602,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":463084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.40899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.56200000000000006}]},{"osd":2,"last update":"Wed May 8 21:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.50700000000000001,"15min":0.496},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75900000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.497,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64500000000000002}]},{"osd":3,"last update":"Wed May 8 21:17:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.55300000000000005,"15min":0.51600000000000001},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96199999999999997,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.61199999999999999}]},{"osd":4,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.435,"15min":0.41099999999999998},"min":{"1min":0.315,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.70599999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.432,"15min":0.41699999999999998},"min":{"1min":0.318,"5min":0.253,"15min":0.253},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed May 8 21:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72299999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed May 8 21:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.47099999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.47399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.79900000000000004}]},{"osd":7,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92400000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.36799999999999999,"5min":0.308,"15min":0.308},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.83899999999999997}]},{"osd":9,"last update":"Wed May 8 21:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.622},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.85899999999999999}]},{"osd":10,"last update":"Wed May 8 21:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.873},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.73799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":978944,"data_stored":461811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 21:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.52600000000000002},"min":{"1min":0.39800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.70699999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.41499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.70999999999999996}]},{"osd":2,"last update":"Wed May 8 21:16:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.58899999999999997,"15min":0.57799999999999996},"min":{"1min":0.40699999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.435,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.748}]},{"osd":3,"last update":"Wed May 8 21:16:08 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47999999999999998,"15min":0.47699999999999998},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.70499999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.44900000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed May 8 21:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78300000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59999999999999998,"15min":0.58099999999999996},"min":{"1min":0.44,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.94199999999999995}]},{"osd":5,"last update":"Wed May 8 21:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.47299999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.441,"5min":0.436,"15min":0.436},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.51400000000000001}]},{"osd":6,"last update":"Wed May 8 21:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.753,"5min":0.876,"15min":0.876},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.625}]},{"osd":7,"last update":"Wed May 8 21:16:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.81599999999999995}]},{"osd":8,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56299999999999994}]},{"osd":9,"last update":"Wed May 8 21:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.92400000000000004}]},{"osd":10,"last update":"Wed May 8 21:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.877}]},{"osd":11,"last update":"Wed May 8 21:17:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.77900000000000003}]}]},{"osd":2,"up_from":17,"seq":73014444080,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":1003520,"data_stored":484654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.498,"15min":0.498},"min":{"1min":0.33200000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed May 8 21:16:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.505,"15min":0.504},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63900000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.50900000000000001,"15min":0.51700000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.72499999999999998}]},{"osd":3,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68500000000000005,"5min":0.752,"15min":0.752},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65600000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.70799999999999996}]},{"osd":4,"last update":"Wed May 8 21:16:27 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.33400000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72599999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.42099999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed May 8 21:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.27300000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.41699999999999998,"15min":0.40899999999999997},"min":{"1min":0.30299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed May 8 21:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55300000000000005,"15min":0.55200000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.748},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.38500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.76700000000000002}]},{"osd":7,"last update":"Wed May 8 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.86299999999999999}]},{"osd":8,"last update":"Wed May 8 21:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.84999999999999998}]},{"osd":9,"last update":"Wed May 8 21:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.877}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.94899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215148,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":962560,"data_stored":459696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.40500000000000003,"15min":0.38600000000000001},"min":{"1min":0.36199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.36299999999999999}]},{"osd":1,"last update":"Wed May 8 21:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.52500000000000002,"15min":0.51100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53500000000000003,"15min":0.51500000000000001},"min":{"1min":0.40999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.61499999999999999}]},{"osd":2,"last update":"Wed May 8 21:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.39700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.501},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.438}]},{"osd":4,"last update":"Wed May 8 21:17:27 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.57499999999999996,"15min":0.54100000000000004},"min":{"1min":0.47899999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.88300000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48799999999999999}]},{"osd":5,"last update":"Wed May 8 21:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.621,"15min":0.60899999999999999},"min":{"1min":0.51300000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.46800000000000003}]},{"osd":6,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67500000000000004}]},{"osd":7,"last update":"Wed May 8 21:17:23 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.69199999999999995}]},{"osd":8,"last update":"Wed May 8 21:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79800000000000004}]},{"osd":9,"last update":"Wed May 8 21:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.71599999999999997}]},{"osd":10,"last update":"Wed May 8 21:17:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.751},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.66700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986216,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":464036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.373,"15min":0.36299999999999999},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72899999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.79100000000000004}]},{"osd":3,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.52100000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed May 8 21:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.44800000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.83299999999999996}]},{"osd":6,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.74299999999999999}]},{"osd":7,"last update":"Wed May 8 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.5,"15min":0.47999999999999998},"min":{"1min":0.34300000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.53400000000000003,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.91100000000000003}]},{"osd":8,"last update":"Wed May 8 21:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.94099999999999995}]},{"osd":9,"last update":"Wed May 8 21:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.874},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.86099999999999999}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.89700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822693,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36732,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708420,"statfs":{"total":95995035648,"available":95957422080,"internally_reserved":0,"allocated":1560576,"data_stored":1051084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77600000000000002,"5min":0.877,"15min":0.877},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73099999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.42099999999999999}]},{"osd":1,"last update":"Wed May 8 21:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.39300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70199999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73899999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed May 8 21:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.308,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60999999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.373,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.435}]},{"osd":3,"last update":"Wed May 8 21:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75800000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.499},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.80200000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed May 8 21:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.749,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78700000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61699999999999999}]},{"osd":6,"last update":"Wed May 8 21:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.373,"15min":0.373},"max":{"1min":0.77300000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.47699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86899999999999999,"5min":1.01,"15min":1.01},"last":0.68000000000000005}]},{"osd":7,"last update":"Wed May 8 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81999999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71899999999999997}]},{"osd":8,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58099999999999996}]},{"osd":9,"last update":"Wed May 8 21:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.73899999999999999}]},{"osd":10,"last update":"Wed May 8 21:17:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.75700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":5450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T21:17:34.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T21:17:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.509+0000 7f693f7eb700 1 -- 172.21.15.33:0/2234379205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6938106b60 msgr2=0x7f6938108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.509+0000 7f693f7eb700 1 --2- 172.21.15.33:0/2234379205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6938106b60 0x7f6938108fd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6930009f90 tx=0x7f693000b330 comp rx=0 tx=0).stop 2024-05-08T21:17:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.509+0000 7f693f7eb700 1 -- 172.21.15.33:0/2234379205 shutdown_connections 2024-05-08T21:17:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.509+0000 7f693f7eb700 1 --2- 172.21.15.33:0/2234379205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6938106b60 0x7f6938108fd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.509+0000 7f693f7eb700 1 --2- 172.21.15.33:0/2234379205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6938101900 0x7f6938103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.509+0000 7f693f7eb700 1 --2- 172.21.15.33:0/2234379205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6938104230 0x7f6938106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.509+0000 7f693f7eb700 1 -- 172.21.15.33:0/2234379205 >> 172.21.15.33:0/2234379205 conn(0x7f69380fb2f0 msgr2=0x7f69380fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.509+0000 7f693f7eb700 1 -- 172.21.15.33:0/2234379205 shutdown_connections 2024-05-08T21:17:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.509+0000 7f693f7eb700 1 -- 172.21.15.33:0/2234379205 wait complete. 2024-05-08T21:17:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693f7eb700 1 Processor -- start 2024-05-08T21:17:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693f7eb700 1 -- start start 2024-05-08T21:17:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693f7eb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6938101900 0x7f6938196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:34.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693f7eb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6938104230 0x7f6938197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:34.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693f7eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6938106b60 0x7f693819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:34.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693f7eb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693819e8f0 con 0x7f6938106b60 2024-05-08T21:17:34.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693f7eb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693819ea60 con 0x7f6938104230 2024-05-08T21:17:34.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693f7eb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693819ec20 con 0x7f6938101900 2024-05-08T21:17:34.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693dd88700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6938106b60 0x7f693819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:34.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693dd88700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6938106b60 0x7f693819e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46392/0 (socket says 172.21.15.33:46392) 2024-05-08T21:17:34.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693dd88700 1 -- 172.21.15.33:0/3888384096 learned_addr learned my addr 172.21.15.33:0/3888384096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:34.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693d587700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6938101900 0x7f6938196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.510+0000 7f693cd86700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6938104230 0x7f6938197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.511+0000 7f693dd88700 1 -- 172.21.15.33:0/3888384096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6938101900 msgr2=0x7f6938196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.511+0000 7f693dd88700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6938101900 0x7f6938196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.511+0000 7f693dd88700 1 -- 172.21.15.33:0/3888384096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6938104230 msgr2=0x7f6938197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.511+0000 7f693dd88700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6938104230 0x7f6938197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.511+0000 7f693dd88700 1 -- 172.21.15.33:0/3888384096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6930009c40 con 0x7f6938106b60 2024-05-08T21:17:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.511+0000 7f693dd88700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6938106b60 0x7f693819e320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6930009f60 tx=0x7f6930008ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.511+0000 7f692e7fc700 1 -- 172.21.15.33:0/3888384096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693000e040 con 0x7f6938106b60 2024-05-08T21:17:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.511+0000 7f692e7fc700 1 -- 172.21.15.33:0/3888384096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f693000be00 con 0x7f6938106b60 2024-05-08T21:17:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.511+0000 7f693f7eb700 1 -- 172.21.15.33:0/3888384096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693819eea0 con 0x7f6938106b60 2024-05-08T21:17:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.511+0000 7f692e7fc700 1 -- 172.21.15.33:0/3888384096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693001dea0 con 0x7f6938106b60 2024-05-08T21:17:34.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.511+0000 7f693f7eb700 1 -- 172.21.15.33:0/3888384096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693819f260 con 0x7f6938106b60 2024-05-08T21:17:34.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.512+0000 7f693f7eb700 1 -- 172.21.15.33:0/3888384096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69380fce70 con 0x7f6938106b60 2024-05-08T21:17:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.513+0000 7f692e7fc700 1 -- 172.21.15.33:0/3888384096 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6930003a00 con 0x7f6938106b60 2024-05-08T21:17:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.514+0000 7f692e7fc700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f69240a1230 0x7f69240a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.514+0000 7f692e7fc700 1 -- 172.21.15.33:0/3888384096 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f69300ce670 con 0x7f6938106b60 2024-05-08T21:17:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.514+0000 7f693d587700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f69240a1230 0x7f69240a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:34.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.515+0000 7f693d587700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f69240a1230 0x7f69240a36f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6938198280 tx=0x7f693400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:34.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.518+0000 7f692e7fc700 1 -- 172.21.15.33:0/3888384096 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6930097ae0 con 0x7f6938106b60 2024-05-08T21:17:34.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:34 smithi033 ceph-mon[42528]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:34.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:34 smithi033 ceph-mon[42528]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:34.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:34 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1550710131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:17:34.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:34 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1152939620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.734+0000 7f693f7eb700 1 -- 172.21.15.33:0/3888384096 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f693802ceb0 con 0x7f69240a1230 2024-05-08T21:17:34.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.747+0000 7f692e7fc700 1 -- 172.21.15.33:0/3888384096 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418015 (secure 0 0 0) 0x7f693802ceb0 con 0x7f69240a1230 2024-05-08T21:17:34.755 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.751+0000 7f693f7eb700 1 -- 172.21.15.33:0/3888384096 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f69240a1230 msgr2=0x7f69240a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.751+0000 7f693f7eb700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f69240a1230 0x7f69240a36f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6938198280 tx=0x7f693400b410 comp rx=0 tx=0).stop 2024-05-08T21:17:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.751+0000 7f693f7eb700 1 -- 172.21.15.33:0/3888384096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6938106b60 msgr2=0x7f693819e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.751+0000 7f693f7eb700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6938106b60 0x7f693819e320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6930009f60 tx=0x7f6930008ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.751+0000 7f693f7eb700 1 -- 172.21.15.33:0/3888384096 shutdown_connections 2024-05-08T21:17:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.751+0000 7f693f7eb700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6938106b60 0x7f693819e320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.751+0000 7f693f7eb700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6938101900 0x7f6938196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.751+0000 7f693f7eb700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f69240a1230 0x7f69240a36f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.751+0000 7f693f7eb700 1 --2- 172.21.15.33:0/3888384096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6938104230 0x7f6938197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.751+0000 7f693f7eb700 1 -- 172.21.15.33:0/3888384096 >> 172.21.15.33:0/3888384096 conn(0x7f69380fb2f0 msgr2=0x7f6938102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.751+0000 7f693f7eb700 1 -- 172.21.15.33:0/3888384096 shutdown_connections 2024-05-08T21:17:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:34.751+0000 7f693f7eb700 1 -- 172.21.15.33:0/3888384096 wait complete. 2024-05-08T21:17:34.759 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T21:17:34.767 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-05-08T21:17:34.119712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600142,"num_objects":83,"num_object_clones":0,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":146,"num_read_kb":125,"num_write":354,"num_write_kb":3004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":288,"ondisk_log_size":288,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369996,"kb_used_data":13036,"kb_used_omap":18,"kb_used_meta":356845,"kb_avail":1124571828,"statfs":{"total":1151940427776,"available":1151561551872,"internally_reserved":0,"allocated":13348864,"data_stored":7329577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":365409606},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5180,"num_objects":59,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":124,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003659"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.260649+0000","last_change":"2024-05-08T21:17:03.243903+0000","last_active":"2024-05-08T21:17:07.260649+0000","last_peered":"2024-05-08T21:17:07.260649+0000","last_clean":"2024-05-08T21:17:07.260649+0000","last_became_active":"2024-05-08T21:17:03.240821+0000","last_became_peered":"2024-05-08T21:17:03.240821+0000","last_unstale":"2024-05-08T21:17:07.260649+0000","last_undegraded":"2024-05-08T21:17:07.260649+0000","last_fullsized":"2024-05-08T21:17:07.260649+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:01.878614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489502+0000","last_change":"2024-05-08T21:17:03.243119+0000","last_active":"2024-05-08T21:17:07.489502+0000","last_peered":"2024-05-08T21:17:07.489502+0000","last_clean":"2024-05-08T21:17:07.489502+0000","last_became_active":"2024-05-08T21:17:03.242772+0000","last_became_peered":"2024-05-08T21:17:03.242772+0000","last_unstale":"2024-05-08T21:17:07.489502+0000","last_undegraded":"2024-05-08T21:17:07.489502+0000","last_fullsized":"2024-05-08T21:17:07.489502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:20.550646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489471+0000","last_change":"2024-05-08T21:17:03.237843+0000","last_active":"2024-05-08T21:17:07.489471+0000","last_peered":"2024-05-08T21:17:07.489471+0000","last_clean":"2024-05-08T21:17:07.489471+0000","last_became_active":"2024-05-08T21:17:03.237567+0000","last_became_peered":"2024-05-08T21:17:03.237567+0000","last_unstale":"2024-05-08T21:17:07.489471+0000","last_undegraded":"2024-05-08T21:17:07.489471+0000","last_fullsized":"2024-05-08T21:17:07.489471+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:23.856461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.253998+0000","last_change":"2024-05-08T21:17:03.245083+0000","last_active":"2024-05-08T21:17:07.253998+0000","last_peered":"2024-05-08T21:17:07.253998+0000","last_clean":"2024-05-08T21:17:07.253998+0000","last_became_active":"2024-05-08T21:17:03.244756+0000","last_became_peered":"2024-05-08T21:17:03.244756+0000","last_unstale":"2024-05-08T21:17:07.253998+0000","last_undegraded":"2024-05-08T21:17:07.253998+0000","last_fullsized":"2024-05-08T21:17:07.253998+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:23:57.884086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.252739+0000","last_change":"2024-05-08T21:17:03.252246+0000","last_active":"2024-05-08T21:17:07.252739+0000","last_peered":"2024-05-08T21:17:07.252739+0000","last_clean":"2024-05-08T21:17:07.252739+0000","last_became_active":"2024-05-08T21:17:03.251898+0000","last_became_peered":"2024-05-08T21:17:03.251898+0000","last_unstale":"2024-05-08T21:17:07.252739+0000","last_undegraded":"2024-05-08T21:17:07.252739+0000","last_fullsized":"2024-05-08T21:17:07.252739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:08.996033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.437630+0000","last_change":"2024-05-08T21:17:03.238422+0000","last_active":"2024-05-08T21:17:07.437630+0000","last_peered":"2024-05-08T21:17:07.437630+0000","last_clean":"2024-05-08T21:17:07.437630+0000","last_became_active":"2024-05-08T21:17:03.238096+0000","last_became_peered":"2024-05-08T21:17:03.238096+0000","last_unstale":"2024-05-08T21:17:07.437630+0000","last_undegraded":"2024-05-08T21:17:07.437630+0000","last_fullsized":"2024-05-08T21:17:07.437630+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:54.635089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.255071+0000","last_change":"2024-05-08T21:17:03.244315+0000","last_active":"2024-05-08T21:17:07.255071+0000","last_peered":"2024-05-08T21:17:07.255071+0000","last_clean":"2024-05-08T21:17:07.255071+0000","last_became_active":"2024-05-08T21:17:03.243939+0000","last_became_peered":"2024-05-08T21:17:03.243939+0000","last_unstale":"2024-05-08T21:17:07.255071+0000","last_undegraded":"2024-05-08T21:17:07.255071+0000","last_fullsized":"2024-05-08T21:17:07.255071+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:24.371495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.262968+0000","last_change":"2024-05-08T21:17:03.240144+0000","last_active":"2024-05-08T21:17:07.262968+0000","last_peered":"2024-05-08T21:17:07.262968+0000","last_clean":"2024-05-08T21:17:07.262968+0000","last_became_active":"2024-05-08T21:17:03.239493+0000","last_became_peered":"2024-05-08T21:17:03.239493+0000","last_unstale":"2024-05-08T21:17:07.262968+0000","last_undegraded":"2024-05-08T21:17:07.262968+0000","last_fullsized":"2024-05-08T21:17:07.262968+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:21.374357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.249258+0000","last_change":"2024-05-08T21:17:03.256706+0000","last_active":"2024-05-08T21:17:07.249258+0000","last_peered":"2024-05-08T21:17:07.249258+0000","last_clean":"2024-05-08T21:17:07.249258+0000","last_became_active":"2024-05-08T21:17:03.255345+0000","last_became_peered":"2024-05-08T21:17:03.255345+0000","last_unstale":"2024-05-08T21:17:07.249258+0000","last_undegraded":"2024-05-08T21:17:07.249258+0000","last_fullsized":"2024-05-08T21:17:07.249258+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:09:50.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.438066+0000","last_change":"2024-05-08T21:17:03.240062+0000","last_active":"2024-05-08T21:17:07.438066+0000","last_peered":"2024-05-08T21:17:07.438066+0000","last_clean":"2024-05-08T21:17:07.438066+0000","last_became_active":"2024-05-08T21:17:03.239740+0000","last_became_peered":"2024-05-08T21:17:03.239740+0000","last_unstale":"2024-05-08T21:17:07.438066+0000","last_undegraded":"2024-05-08T21:17:07.438066+0000","last_fullsized":"2024-05-08T21:17:07.438066+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:06:21.937721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.256041+0000","last_change":"2024-05-08T21:17:03.253996+0000","last_active":"2024-05-08T21:17:07.256041+0000","last_peered":"2024-05-08T21:17:07.256041+0000","last_clean":"2024-05-08T21:17:07.256041+0000","last_became_active":"2024-05-08T21:17:03.253686+0000","last_became_peered":"2024-05-08T21:17:03.253686+0000","last_unstale":"2024-05-08T21:17:07.256041+0000","last_undegraded":"2024-05-08T21:17:07.256041+0000","last_fullsized":"2024-05-08T21:17:07.256041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:17.995692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.253962+0000","last_change":"2024-05-08T21:17:03.254357+0000","last_active":"2024-05-08T21:17:07.253962+0000","last_peered":"2024-05-08T21:17:07.253962+0000","last_clean":"2024-05-08T21:17:07.253962+0000","last_became_active":"2024-05-08T21:17:03.254013+0000","last_became_peered":"2024-05-08T21:17:03.254013+0000","last_unstale":"2024-05-08T21:17:07.253962+0000","last_undegraded":"2024-05-08T21:17:07.253962+0000","last_fullsized":"2024-05-08T21:17:07.253962+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:21.480612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.255067+0000","last_change":"2024-05-08T21:17:03.244190+0000","last_active":"2024-05-08T21:17:07.255067+0000","last_peered":"2024-05-08T21:17:07.255067+0000","last_clean":"2024-05-08T21:17:07.255067+0000","last_became_active":"2024-05-08T21:17:03.243989+0000","last_became_peered":"2024-05-08T21:17:03.243989+0000","last_unstale":"2024-05-08T21:17:07.255067+0000","last_undegraded":"2024-05-08T21:17:07.255067+0000","last_fullsized":"2024-05-08T21:17:07.255067+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:45.059343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489412+0000","last_change":"2024-05-08T21:17:03.242996+0000","last_active":"2024-05-08T21:17:07.489412+0000","last_peered":"2024-05-08T21:17:07.489412+0000","last_clean":"2024-05-08T21:17:07.489412+0000","last_became_active":"2024-05-08T21:17:03.242648+0000","last_became_peered":"2024-05-08T21:17:03.242648+0000","last_unstale":"2024-05-08T21:17:07.489412+0000","last_undegraded":"2024-05-08T21:17:07.489412+0000","last_fullsized":"2024-05-08T21:17:07.489412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:47:19.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489384+0000","last_change":"2024-05-08T21:17:03.244970+0000","last_active":"2024-05-08T21:17:07.489384+0000","last_peered":"2024-05-08T21:17:07.489384+0000","last_clean":"2024-05-08T21:17:07.489384+0000","last_became_active":"2024-05-08T21:17:03.244604+0000","last_became_peered":"2024-05-08T21:17:03.244604+0000","last_unstale":"2024-05-08T21:17:07.489384+0000","last_undegraded":"2024-05-08T21:17:07.489384+0000","last_fullsized":"2024-05-08T21:17:07.489384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:32.484155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.248910+0000","last_change":"2024-05-08T21:17:03.254628+0000","last_active":"2024-05-08T21:17:07.248910+0000","last_peered":"2024-05-08T21:17:07.248910+0000","last_clean":"2024-05-08T21:17:07.248910+0000","last_became_active":"2024-05-08T21:17:03.254305+0000","last_became_peered":"2024-05-08T21:17:03.254305+0000","last_unstale":"2024-05-08T21:17:07.248910+0000","last_undegraded":"2024-05-08T21:17:07.248910+0000","last_fullsized":"2024-05-08T21:17:07.248910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:52.727821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.260275+0000","last_change":"2024-05-08T21:17:03.243406+0000","last_active":"2024-05-08T21:17:07.260275+0000","last_peered":"2024-05-08T21:17:07.260275+0000","last_clean":"2024-05-08T21:17:07.260275+0000","last_became_active":"2024-05-08T21:17:03.240379+0000","last_became_peered":"2024-05-08T21:17:03.240379+0000","last_unstale":"2024-05-08T21:17:07.260275+0000","last_undegraded":"2024-05-08T21:17:07.260275+0000","last_fullsized":"2024-05-08T21:17:07.260275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:49.897459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.248571+0000","last_change":"2024-05-08T21:17:03.239664+0000","last_active":"2024-05-08T21:17:07.248571+0000","last_peered":"2024-05-08T21:17:07.248571+0000","last_clean":"2024-05-08T21:17:07.248571+0000","last_became_active":"2024-05-08T21:17:03.239347+0000","last_became_peered":"2024-05-08T21:17:03.239347+0000","last_unstale":"2024-05-08T21:17:07.248571+0000","last_undegraded":"2024-05-08T21:17:07.248571+0000","last_fullsized":"2024-05-08T21:17:07.248571+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:06:37.557835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.250023+0000","last_change":"2024-05-08T21:17:03.249424+0000","last_active":"2024-05-08T21:17:07.250023+0000","last_peered":"2024-05-08T21:17:07.250023+0000","last_clean":"2024-05-08T21:17:07.250023+0000","last_became_active":"2024-05-08T21:17:03.248971+0000","last_became_peered":"2024-05-08T21:17:03.248971+0000","last_unstale":"2024-05-08T21:17:07.250023+0000","last_undegraded":"2024-05-08T21:17:07.250023+0000","last_fullsized":"2024-05-08T21:17:07.250023+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:08.096762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.273118+0000","last_change":"2024-05-08T21:17:07.272414+0000","last_active":"2024-05-08T21:17:07.273118+0000","last_peered":"2024-05-08T21:17:07.273118+0000","last_clean":"2024-05-08T21:17:07.273118+0000","last_became_active":"2024-05-08T21:17:07.271660+0000","last_became_peered":"2024-05-08T21:17:07.271660+0000","last_unstale":"2024-05-08T21:17:07.273118+0000","last_undegraded":"2024-05-08T21:17:07.273118+0000","last_fullsized":"2024-05-08T21:17:07.273118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:15.697910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.341532+0000","last_change":"2024-05-08T21:17:02.732779+0000","last_active":"2024-05-08T21:17:29.341532+0000","last_peered":"2024-05-08T21:17:29.341532+0000","last_clean":"2024-05-08T21:17:29.341532+0000","last_became_active":"2024-05-08T21:17:02.732439+0000","last_became_peered":"2024-05-08T21:17:02.732439+0000","last_unstale":"2024-05-08T21:17:29.341532+0000","last_undegraded":"2024-05-08T21:17:29.341532+0000","last_fullsized":"2024-05-08T21:17:29.341532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:44:42.064364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.267792+0000","last_change":"2024-05-08T21:17:03.240780+0000","last_active":"2024-05-08T21:17:07.267792+0000","last_peered":"2024-05-08T21:17:07.267792+0000","last_clean":"2024-05-08T21:17:07.267792+0000","last_became_active":"2024-05-08T21:17:03.240472+0000","last_became_peered":"2024-05-08T21:17:03.240472+0000","last_unstale":"2024-05-08T21:17:07.267792+0000","last_undegraded":"2024-05-08T21:17:07.267792+0000","last_fullsized":"2024-05-08T21:17:07.267792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.089760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713994+0000","last_change":"2024-05-08T21:17:07.272308+0000","last_active":"2024-05-08T21:17:07.713994+0000","last_peered":"2024-05-08T21:17:07.713994+0000","last_clean":"2024-05-08T21:17:07.713994+0000","last_became_active":"2024-05-08T21:17:07.271869+0000","last_became_peered":"2024-05-08T21:17:07.271869+0000","last_unstale":"2024-05-08T21:17:07.713994+0000","last_undegraded":"2024-05-08T21:17:07.713994+0000","last_fullsized":"2024-05-08T21:17:07.713994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:31.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.971656+0000","last_change":"2024-05-08T21:17:02.736446+0000","last_active":"2024-05-08T21:17:07.971656+0000","last_peered":"2024-05-08T21:17:07.971656+0000","last_clean":"2024-05-08T21:17:07.971656+0000","last_became_active":"2024-05-08T21:17:02.736187+0000","last_became_peered":"2024-05-08T21:17:02.736187+0000","last_unstale":"2024-05-08T21:17:07.971656+0000","last_undegraded":"2024-05-08T21:17:07.971656+0000","last_fullsized":"2024-05-08T21:17:07.971656+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:29.302558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489541+0000","last_change":"2024-05-08T21:17:03.251896+0000","last_active":"2024-05-08T21:17:07.489541+0000","last_peered":"2024-05-08T21:17:07.489541+0000","last_clean":"2024-05-08T21:17:07.489541+0000","last_became_active":"2024-05-08T21:17:03.251495+0000","last_became_peered":"2024-05-08T21:17:03.251495+0000","last_unstale":"2024-05-08T21:17:07.489541+0000","last_undegraded":"2024-05-08T21:17:07.489541+0000","last_fullsized":"2024-05-08T21:17:07.489541+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:50.774848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714159+0000","last_change":"2024-05-08T21:17:07.272150+0000","last_active":"2024-05-08T21:17:07.714159+0000","last_peered":"2024-05-08T21:17:07.714159+0000","last_clean":"2024-05-08T21:17:07.714159+0000","last_became_active":"2024-05-08T21:17:07.271727+0000","last_became_peered":"2024-05-08T21:17:07.271727+0000","last_unstale":"2024-05-08T21:17:07.714159+0000","last_undegraded":"2024-05-08T21:17:07.714159+0000","last_fullsized":"2024-05-08T21:17:07.714159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.058277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.339824+0000","last_change":"2024-05-08T21:17:02.733611+0000","last_active":"2024-05-08T21:17:29.339824+0000","last_peered":"2024-05-08T21:17:29.339824+0000","last_clean":"2024-05-08T21:17:29.339824+0000","last_became_active":"2024-05-08T21:17:02.733306+0000","last_became_peered":"2024-05-08T21:17:02.733306+0000","last_unstale":"2024-05-08T21:17:29.339824+0000","last_undegraded":"2024-05-08T21:17:29.339824+0000","last_fullsized":"2024-05-08T21:17:29.339824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:33.304753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.249070+0000","last_change":"2024-05-08T21:17:03.248480+0000","last_active":"2024-05-08T21:17:07.249070+0000","last_peered":"2024-05-08T21:17:07.249070+0000","last_clean":"2024-05-08T21:17:07.249070+0000","last_became_active":"2024-05-08T21:17:03.248126+0000","last_became_peered":"2024-05-08T21:17:03.248126+0000","last_unstale":"2024-05-08T21:17:07.249070+0000","last_undegraded":"2024-05-08T21:17:07.249070+0000","last_fullsized":"2024-05-08T21:17:07.249070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:08.785761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714035+0000","last_change":"2024-05-08T21:17:07.272344+0000","last_active":"2024-05-08T21:17:07.714035+0000","last_peered":"2024-05-08T21:17:07.714035+0000","last_clean":"2024-05-08T21:17:07.714035+0000","last_became_active":"2024-05-08T21:17:07.271629+0000","last_became_peered":"2024-05-08T21:17:07.271629+0000","last_unstale":"2024-05-08T21:17:07.714035+0000","last_undegraded":"2024-05-08T21:17:07.714035+0000","last_fullsized":"2024-05-08T21:17:07.714035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:15.052748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.252992+0000","last_change":"2024-05-08T21:17:02.730364+0000","last_active":"2024-05-08T21:17:07.252992+0000","last_peered":"2024-05-08T21:17:07.252992+0000","last_clean":"2024-05-08T21:17:07.252992+0000","last_became_active":"2024-05-08T21:17:02.728854+0000","last_became_peered":"2024-05-08T21:17:02.728854+0000","last_unstale":"2024-05-08T21:17:07.252992+0000","last_undegraded":"2024-05-08T21:17:07.252992+0000","last_fullsized":"2024-05-08T21:17:07.252992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:09.552016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.258497+0000","last_change":"2024-05-08T21:17:03.252713+0000","last_active":"2024-05-08T21:17:07.258497+0000","last_peered":"2024-05-08T21:17:07.258497+0000","last_clean":"2024-05-08T21:17:07.258497+0000","last_became_active":"2024-05-08T21:17:03.252388+0000","last_became_peered":"2024-05-08T21:17:03.252388+0000","last_unstale":"2024-05-08T21:17:07.258497+0000","last_undegraded":"2024-05-08T21:17:07.258497+0000","last_fullsized":"2024-05-08T21:17:07.258497+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:09.114064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.261781+0000","last_change":"2024-05-08T21:17:07.261112+0000","last_active":"2024-05-08T21:17:07.261781+0000","last_peered":"2024-05-08T21:17:07.261781+0000","last_clean":"2024-05-08T21:17:07.261781+0000","last_became_active":"2024-05-08T21:17:07.260713+0000","last_became_peered":"2024-05-08T21:17:07.260713+0000","last_unstale":"2024-05-08T21:17:07.261781+0000","last_undegraded":"2024-05-08T21:17:07.261781+0000","last_fullsized":"2024-05-08T21:17:07.261781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:37.262113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.335526+0000","last_change":"2024-05-08T21:17:02.728480+0000","last_active":"2024-05-08T21:17:28.335526+0000","last_peered":"2024-05-08T21:17:28.335526+0000","last_clean":"2024-05-08T21:17:28.335526+0000","last_became_active":"2024-05-08T21:17:02.728170+0000","last_became_peered":"2024-05-08T21:17:02.728170+0000","last_unstale":"2024-05-08T21:17:28.335526+0000","last_undegraded":"2024-05-08T21:17:28.335526+0000","last_fullsized":"2024-05-08T21:17:28.335526+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:00.540077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.253712+0000","last_change":"2024-05-08T21:17:03.251245+0000","last_active":"2024-05-08T21:17:07.253712+0000","last_peered":"2024-05-08T21:17:07.253712+0000","last_clean":"2024-05-08T21:17:07.253712+0000","last_became_active":"2024-05-08T21:17:03.250296+0000","last_became_peered":"2024-05-08T21:17:03.250296+0000","last_unstale":"2024-05-08T21:17:07.253712+0000","last_undegraded":"2024-05-08T21:17:07.253712+0000","last_fullsized":"2024-05-08T21:17:07.253712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:28.391927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.267323+0000","last_change":"2024-05-08T21:17:07.266752+0000","last_active":"2024-05-08T21:17:07.267323+0000","last_peered":"2024-05-08T21:17:07.267323+0000","last_clean":"2024-05-08T21:17:07.267323+0000","last_became_active":"2024-05-08T21:17:07.266367+0000","last_became_peered":"2024-05-08T21:17:07.266367+0000","last_unstale":"2024-05-08T21:17:07.267323+0000","last_undegraded":"2024-05-08T21:17:07.267323+0000","last_fullsized":"2024-05-08T21:17:07.267323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:49.874073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:30.355827+0000","last_change":"2024-05-08T21:17:02.728113+0000","last_active":"2024-05-08T21:17:30.355827+0000","last_peered":"2024-05-08T21:17:30.355827+0000","last_clean":"2024-05-08T21:17:30.355827+0000","last_became_active":"2024-05-08T21:17:02.727754+0000","last_became_peered":"2024-05-08T21:17:02.727754+0000","last_unstale":"2024-05-08T21:17:30.355827+0000","last_undegraded":"2024-05-08T21:17:30.355827+0000","last_fullsized":"2024-05-08T21:17:30.355827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:53.789218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.258170+0000","last_change":"2024-05-08T21:17:03.245482+0000","last_active":"2024-05-08T21:17:07.258170+0000","last_peered":"2024-05-08T21:17:07.258170+0000","last_clean":"2024-05-08T21:17:07.258170+0000","last_became_active":"2024-05-08T21:17:03.245144+0000","last_became_peered":"2024-05-08T21:17:03.245144+0000","last_unstale":"2024-05-08T21:17:07.258170+0000","last_undegraded":"2024-05-08T21:17:07.258170+0000","last_fullsized":"2024-05-08T21:17:07.258170+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:04.051234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.001109+0000","last_change":"2024-05-08T21:17:07.268505+0000","last_active":"2024-05-08T21:17:08.001109+0000","last_peered":"2024-05-08T21:17:08.001109+0000","last_clean":"2024-05-08T21:17:08.001109+0000","last_became_active":"2024-05-08T21:17:07.268157+0000","last_became_peered":"2024-05-08T21:17:07.268157+0000","last_unstale":"2024-05-08T21:17:08.001109+0000","last_undegraded":"2024-05-08T21:17:08.001109+0000","last_fullsized":"2024-05-08T21:17:08.001109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:55.220521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.343196+0000","last_change":"2024-05-08T21:17:02.727264+0000","last_active":"2024-05-08T21:17:29.343196+0000","last_peered":"2024-05-08T21:17:29.343196+0000","last_clean":"2024-05-08T21:17:29.343196+0000","last_became_active":"2024-05-08T21:17:02.726922+0000","last_became_peered":"2024-05-08T21:17:02.726922+0000","last_unstale":"2024-05-08T21:17:29.343196+0000","last_undegraded":"2024-05-08T21:17:29.343196+0000","last_fullsized":"2024-05-08T21:17:29.343196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:27.410231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.263093+0000","last_change":"2024-05-08T21:17:03.233910+0000","last_active":"2024-05-08T21:17:07.263093+0000","last_peered":"2024-05-08T21:17:07.263093+0000","last_clean":"2024-05-08T21:17:07.263093+0000","last_became_active":"2024-05-08T21:17:03.233562+0000","last_became_peered":"2024-05-08T21:17:03.233562+0000","last_unstale":"2024-05-08T21:17:07.263093+0000","last_undegraded":"2024-05-08T21:17:07.263093+0000","last_fullsized":"2024-05-08T21:17:07.263093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:40.309400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489487+0000","last_change":"2024-05-08T21:17:07.264992+0000","last_active":"2024-05-08T21:17:07.489487+0000","last_peered":"2024-05-08T21:17:07.489487+0000","last_clean":"2024-05-08T21:17:07.489487+0000","last_became_active":"2024-05-08T21:17:07.264498+0000","last_became_peered":"2024-05-08T21:17:07.264498+0000","last_unstale":"2024-05-08T21:17:07.489487+0000","last_undegraded":"2024-05-08T21:17:07.489487+0000","last_fullsized":"2024-05-08T21:17:07.489487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:23:26.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.334783+0000","last_change":"2024-05-08T21:17:02.725531+0000","last_active":"2024-05-08T21:17:28.334783+0000","last_peered":"2024-05-08T21:17:28.334783+0000","last_clean":"2024-05-08T21:17:28.334783+0000","last_became_active":"2024-05-08T21:17:02.725235+0000","last_became_peered":"2024-05-08T21:17:02.725235+0000","last_unstale":"2024-05-08T21:17:28.334783+0000","last_undegraded":"2024-05-08T21:17:28.334783+0000","last_fullsized":"2024-05-08T21:17:28.334783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:27.265590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.248011+0000","last_change":"2024-05-08T21:17:03.236393+0000","last_active":"2024-05-08T21:17:07.248011+0000","last_peered":"2024-05-08T21:17:07.248011+0000","last_clean":"2024-05-08T21:17:07.248011+0000","last_became_active":"2024-05-08T21:17:03.236092+0000","last_became_peered":"2024-05-08T21:17:03.236092+0000","last_unstale":"2024-05-08T21:17:07.248011+0000","last_undegraded":"2024-05-08T21:17:07.248011+0000","last_fullsized":"2024-05-08T21:17:07.248011+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:58.232669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.971004+0000","last_change":"2024-05-08T21:17:07.271374+0000","last_active":"2024-05-08T21:17:07.971004+0000","last_peered":"2024-05-08T21:17:07.971004+0000","last_clean":"2024-05-08T21:17:07.971004+0000","last_became_active":"2024-05-08T21:17:07.270778+0000","last_became_peered":"2024-05-08T21:17:07.270778+0000","last_unstale":"2024-05-08T21:17:07.971004+0000","last_undegraded":"2024-05-08T21:17:07.971004+0000","last_fullsized":"2024-05-08T21:17:07.971004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:50.212213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.335923+0000","last_change":"2024-05-08T21:17:02.725624+0000","last_active":"2024-05-08T21:17:28.335923+0000","last_peered":"2024-05-08T21:17:28.335923+0000","last_clean":"2024-05-08T21:17:28.335923+0000","last_became_active":"2024-05-08T21:17:02.725367+0000","last_became_peered":"2024-05-08T21:17:02.725367+0000","last_unstale":"2024-05-08T21:17:28.335923+0000","last_undegraded":"2024-05-08T21:17:28.335923+0000","last_fullsized":"2024-05-08T21:17:28.335923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:19.573931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.249341+0000","last_change":"2024-05-08T21:17:03.248729+0000","last_active":"2024-05-08T21:17:07.249341+0000","last_peered":"2024-05-08T21:17:07.249341+0000","last_clean":"2024-05-08T21:17:07.249341+0000","last_became_active":"2024-05-08T21:17:03.247899+0000","last_became_peered":"2024-05-08T21:17:03.247899+0000","last_unstale":"2024-05-08T21:17:07.249341+0000","last_undegraded":"2024-05-08T21:17:07.249341+0000","last_fullsized":"2024-05-08T21:17:07.249341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:27:57.012249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713082+0000","last_change":"2024-05-08T21:17:07.261257+0000","last_active":"2024-05-08T21:17:07.713082+0000","last_peered":"2024-05-08T21:17:07.713082+0000","last_clean":"2024-05-08T21:17:07.713082+0000","last_became_active":"2024-05-08T21:17:07.260785+0000","last_became_peered":"2024-05-08T21:17:07.260785+0000","last_unstale":"2024-05-08T21:17:07.713082+0000","last_undegraded":"2024-05-08T21:17:07.713082+0000","last_fullsized":"2024-05-08T21:17:07.713082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:58.469134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.311324+0000","last_change":"2024-05-08T21:17:02.729569+0000","last_active":"2024-05-08T21:17:31.311324+0000","last_peered":"2024-05-08T21:17:31.311324+0000","last_clean":"2024-05-08T21:17:31.311324+0000","last_became_active":"2024-05-08T21:17:02.729096+0000","last_became_peered":"2024-05-08T21:17:02.729096+0000","last_unstale":"2024-05-08T21:17:31.311324+0000","last_undegraded":"2024-05-08T21:17:31.311324+0000","last_fullsized":"2024-05-08T21:17:31.311324+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:30.668169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489564+0000","last_change":"2024-05-08T21:17:03.246387+0000","last_active":"2024-05-08T21:17:07.489564+0000","last_peered":"2024-05-08T21:17:07.489564+0000","last_clean":"2024-05-08T21:17:07.489564+0000","last_became_active":"2024-05-08T21:17:03.244658+0000","last_became_peered":"2024-05-08T21:17:03.244658+0000","last_unstale":"2024-05-08T21:17:07.489564+0000","last_undegraded":"2024-05-08T21:17:07.489564+0000","last_fullsized":"2024-05-08T21:17:07.489564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:02.602080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714291+0000","last_change":"2024-05-08T21:17:07.261256+0000","last_active":"2024-05-08T21:17:07.714291+0000","last_peered":"2024-05-08T21:17:07.714291+0000","last_clean":"2024-05-08T21:17:07.714291+0000","last_became_active":"2024-05-08T21:17:07.260762+0000","last_became_peered":"2024-05-08T21:17:07.260762+0000","last_unstale":"2024-05-08T21:17:07.714291+0000","last_undegraded":"2024-05-08T21:17:07.714291+0000","last_fullsized":"2024-05-08T21:17:07.714291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:13:26.329820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.333081+0000","last_change":"2024-05-08T21:17:02.732738+0000","last_active":"2024-05-08T21:17:28.333081+0000","last_peered":"2024-05-08T21:17:28.333081+0000","last_clean":"2024-05-08T21:17:28.333081+0000","last_became_active":"2024-05-08T21:17:02.731733+0000","last_became_peered":"2024-05-08T21:17:02.731733+0000","last_unstale":"2024-05-08T21:17:28.333081+0000","last_undegraded":"2024-05-08T21:17:28.333081+0000","last_fullsized":"2024-05-08T21:17:28.333081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:24.201171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.249185+0000","last_change":"2024-05-08T21:17:03.254405+0000","last_active":"2024-05-08T21:17:07.249185+0000","last_peered":"2024-05-08T21:17:07.249185+0000","last_clean":"2024-05-08T21:17:07.249185+0000","last_became_active":"2024-05-08T21:17:03.252137+0000","last_became_peered":"2024-05-08T21:17:03.252137+0000","last_unstale":"2024-05-08T21:17:07.249185+0000","last_undegraded":"2024-05-08T21:17:07.249185+0000","last_fullsized":"2024-05-08T21:17:07.249185+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:15.472225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713049+0000","last_change":"2024-05-08T21:17:07.266325+0000","last_active":"2024-05-08T21:17:07.713049+0000","last_peered":"2024-05-08T21:17:07.713049+0000","last_clean":"2024-05-08T21:17:07.713049+0000","last_became_active":"2024-05-08T21:17:07.265933+0000","last_became_peered":"2024-05-08T21:17:07.265933+0000","last_unstale":"2024-05-08T21:17:07.713049+0000","last_undegraded":"2024-05-08T21:17:07.713049+0000","last_fullsized":"2024-05-08T21:17:07.713049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:54:46.507275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"85'12","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.334385+0000","last_change":"2024-05-08T21:17:02.723975+0000","last_active":"2024-05-08T21:17:28.334385+0000","last_peered":"2024-05-08T21:17:28.334385+0000","last_clean":"2024-05-08T21:17:28.334385+0000","last_became_active":"2024-05-08T21:17:02.723631+0000","last_became_peered":"2024-05-08T21:17:02.723631+0000","last_unstale":"2024-05-08T21:17:28.334385+0000","last_undegraded":"2024-05-08T21:17:28.334385+0000","last_fullsized":"2024-05-08T21:17:28.334385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:17.676007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.256353+0000","last_change":"2024-05-08T21:17:03.251570+0000","last_active":"2024-05-08T21:17:07.256353+0000","last_peered":"2024-05-08T21:17:07.256353+0000","last_clean":"2024-05-08T21:17:07.256353+0000","last_became_active":"2024-05-08T21:17:03.251134+0000","last_became_peered":"2024-05-08T21:17:03.251134+0000","last_unstale":"2024-05-08T21:17:07.256353+0000","last_undegraded":"2024-05-08T21:17:07.256353+0000","last_fullsized":"2024-05-08T21:17:07.256353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:28:09.339958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714619+0000","last_change":"2024-05-08T21:17:07.272051+0000","last_active":"2024-05-08T21:17:07.714619+0000","last_peered":"2024-05-08T21:17:07.714619+0000","last_clean":"2024-05-08T21:17:07.714619+0000","last_became_active":"2024-05-08T21:17:07.271616+0000","last_became_peered":"2024-05-08T21:17:07.271616+0000","last_unstale":"2024-05-08T21:17:07.714619+0000","last_undegraded":"2024-05-08T21:17:07.714619+0000","last_fullsized":"2024-05-08T21:17:07.714619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.364731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.312177+0000","last_change":"2024-05-08T21:17:02.736000+0000","last_active":"2024-05-08T21:17:31.312177+0000","last_peered":"2024-05-08T21:17:31.312177+0000","last_clean":"2024-05-08T21:17:31.312177+0000","last_became_active":"2024-05-08T21:17:02.735629+0000","last_became_peered":"2024-05-08T21:17:02.735629+0000","last_unstale":"2024-05-08T21:17:31.312177+0000","last_undegraded":"2024-05-08T21:17:31.312177+0000","last_fullsized":"2024-05-08T21:17:31.312177+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:21.842135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.255526+0000","last_change":"2024-05-08T21:17:03.237144+0000","last_active":"2024-05-08T21:17:07.255526+0000","last_peered":"2024-05-08T21:17:07.255526+0000","last_clean":"2024-05-08T21:17:07.255526+0000","last_became_active":"2024-05-08T21:17:03.236792+0000","last_became_peered":"2024-05-08T21:17:03.236792+0000","last_unstale":"2024-05-08T21:17:07.255526+0000","last_undegraded":"2024-05-08T21:17:07.255526+0000","last_fullsized":"2024-05-08T21:17:07.255526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:04.520986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.001388+0000","last_change":"2024-05-08T21:17:07.268311+0000","last_active":"2024-05-08T21:17:08.001388+0000","last_peered":"2024-05-08T21:17:08.001388+0000","last_clean":"2024-05-08T21:17:08.001388+0000","last_became_active":"2024-05-08T21:17:07.267876+0000","last_became_peered":"2024-05-08T21:17:07.267876+0000","last_unstale":"2024-05-08T21:17:08.001388+0000","last_undegraded":"2024-05-08T21:17:08.001388+0000","last_fullsized":"2024-05-08T21:17:08.001388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:27:19.041321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.335383+0000","last_change":"2024-05-08T21:17:02.727748+0000","last_active":"2024-05-08T21:17:28.335383+0000","last_peered":"2024-05-08T21:17:28.335383+0000","last_clean":"2024-05-08T21:17:28.335383+0000","last_became_active":"2024-05-08T21:17:02.727444+0000","last_became_peered":"2024-05-08T21:17:02.727444+0000","last_unstale":"2024-05-08T21:17:28.335383+0000","last_undegraded":"2024-05-08T21:17:28.335383+0000","last_fullsized":"2024-05-08T21:17:28.335383+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:47.939235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.252351+0000","last_change":"2024-05-08T21:17:03.247185+0000","last_active":"2024-05-08T21:17:07.252351+0000","last_peered":"2024-05-08T21:17:07.252351+0000","last_clean":"2024-05-08T21:17:07.252351+0000","last_became_active":"2024-05-08T21:17:03.246880+0000","last_became_peered":"2024-05-08T21:17:03.246880+0000","last_unstale":"2024-05-08T21:17:07.252351+0000","last_undegraded":"2024-05-08T21:17:07.252351+0000","last_fullsized":"2024-05-08T21:17:07.252351+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:38.978353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713572+0000","last_change":"2024-05-08T21:17:07.273710+0000","last_active":"2024-05-08T21:17:07.713572+0000","last_peered":"2024-05-08T21:17:07.713572+0000","last_clean":"2024-05-08T21:17:07.713572+0000","last_became_active":"2024-05-08T21:17:07.273119+0000","last_became_peered":"2024-05-08T21:17:07.273119+0000","last_unstale":"2024-05-08T21:17:07.713572+0000","last_undegraded":"2024-05-08T21:17:07.713572+0000","last_fullsized":"2024-05-08T21:17:07.713572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:30.589440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:32.306086+0000","last_change":"2024-05-08T21:17:02.725508+0000","last_active":"2024-05-08T21:17:32.306086+0000","last_peered":"2024-05-08T21:17:32.306086+0000","last_clean":"2024-05-08T21:17:32.306086+0000","last_became_active":"2024-05-08T21:17:02.725107+0000","last_became_peered":"2024-05-08T21:17:02.725107+0000","last_unstale":"2024-05-08T21:17:32.306086+0000","last_undegraded":"2024-05-08T21:17:32.306086+0000","last_fullsized":"2024-05-08T21:17:32.306086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:13.503702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.254342+0000","last_change":"2024-05-08T21:17:03.245372+0000","last_active":"2024-05-08T21:17:07.254342+0000","last_peered":"2024-05-08T21:17:07.254342+0000","last_clean":"2024-05-08T21:17:07.254342+0000","last_became_active":"2024-05-08T21:17:03.245054+0000","last_became_peered":"2024-05-08T21:17:03.245054+0000","last_unstale":"2024-05-08T21:17:07.254342+0000","last_undegraded":"2024-05-08T21:17:07.254342+0000","last_fullsized":"2024-05-08T21:17:07.254342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:30:25.900846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.044639+0000","last_change":"2024-05-08T21:17:07.272081+0000","last_active":"2024-05-08T21:17:08.044639+0000","last_peered":"2024-05-08T21:17:08.044639+0000","last_clean":"2024-05-08T21:17:08.044639+0000","last_became_active":"2024-05-08T21:17:07.271560+0000","last_became_peered":"2024-05-08T21:17:07.271560+0000","last_unstale":"2024-05-08T21:17:08.044639+0000","last_undegraded":"2024-05-08T21:17:08.044639+0000","last_fullsized":"2024-05-08T21:17:08.044639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:00.490798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.341116+0000","last_change":"2024-05-08T21:17:02.734420+0000","last_active":"2024-05-08T21:17:29.341116+0000","last_peered":"2024-05-08T21:17:29.341116+0000","last_clean":"2024-05-08T21:17:29.341116+0000","last_became_active":"2024-05-08T21:17:02.734071+0000","last_became_peered":"2024-05-08T21:17:02.734071+0000","last_unstale":"2024-05-08T21:17:29.341116+0000","last_undegraded":"2024-05-08T21:17:29.341116+0000","last_fullsized":"2024-05-08T21:17:29.341116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:28.537464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.257017+0000","last_change":"2024-05-08T21:17:03.250706+0000","last_active":"2024-05-08T21:17:07.257017+0000","last_peered":"2024-05-08T21:17:07.257017+0000","last_clean":"2024-05-08T21:17:07.257017+0000","last_became_active":"2024-05-08T21:17:03.250408+0000","last_became_peered":"2024-05-08T21:17:03.250408+0000","last_unstale":"2024-05-08T21:17:07.257017+0000","last_undegraded":"2024-05-08T21:17:07.257017+0000","last_fullsized":"2024-05-08T21:17:07.257017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:43.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713885+0000","last_change":"2024-05-08T21:17:07.272066+0000","last_active":"2024-05-08T21:17:07.713885+0000","last_peered":"2024-05-08T21:17:07.713885+0000","last_clean":"2024-05-08T21:17:07.713885+0000","last_became_active":"2024-05-08T21:17:07.271628+0000","last_became_peered":"2024-05-08T21:17:07.271628+0000","last_unstale":"2024-05-08T21:17:07.713885+0000","last_undegraded":"2024-05-08T21:17:07.713885+0000","last_fullsized":"2024-05-08T21:17:07.713885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:27.731995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.252941+0000","last_change":"2024-05-08T21:17:02.728464+0000","last_active":"2024-05-08T21:17:07.252941+0000","last_peered":"2024-05-08T21:17:07.252941+0000","last_clean":"2024-05-08T21:17:07.252941+0000","last_became_active":"2024-05-08T21:17:02.727914+0000","last_became_peered":"2024-05-08T21:17:02.727914+0000","last_unstale":"2024-05-08T21:17:07.252941+0000","last_undegraded":"2024-05-08T21:17:07.252941+0000","last_fullsized":"2024-05-08T21:17:07.252941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:55.948661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.259706+0000","last_change":"2024-05-08T21:17:03.245578+0000","last_active":"2024-05-08T21:17:07.259706+0000","last_peered":"2024-05-08T21:17:07.259706+0000","last_clean":"2024-05-08T21:17:07.259706+0000","last_became_active":"2024-05-08T21:17:03.245256+0000","last_became_peered":"2024-05-08T21:17:03.245256+0000","last_unstale":"2024-05-08T21:17:07.259706+0000","last_undegraded":"2024-05-08T21:17:07.259706+0000","last_fullsized":"2024-05-08T21:17:07.259706+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.002235+0000","last_change":"2024-05-08T21:17:07.439525+0000","last_active":"2024-05-08T21:17:08.002235+0000","last_peered":"2024-05-08T21:17:08.002235+0000","last_clean":"2024-05-08T21:17:08.002235+0000","last_became_active":"2024-05-08T21:17:07.438987+0000","last_became_peered":"2024-05-08T21:17:07.438987+0000","last_unstale":"2024-05-08T21:17:08.002235+0000","last_undegraded":"2024-05-08T21:17:08.002235+0000","last_fullsized":"2024-05-08T21:17:08.002235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:57:45.373981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.248207+0000","last_change":"2024-05-08T21:17:02.725912+0000","last_active":"2024-05-08T21:17:07.248207+0000","last_peered":"2024-05-08T21:17:07.248207+0000","last_clean":"2024-05-08T21:17:07.248207+0000","last_became_active":"2024-05-08T21:17:02.725537+0000","last_became_peered":"2024-05-08T21:17:02.725537+0000","last_unstale":"2024-05-08T21:17:07.248207+0000","last_undegraded":"2024-05-08T21:17:07.248207+0000","last_fullsized":"2024-05-08T21:17:07.248207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:28.184551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.437903+0000","last_change":"2024-05-08T21:17:03.235556+0000","last_active":"2024-05-08T21:17:07.437903+0000","last_peered":"2024-05-08T21:17:07.437903+0000","last_clean":"2024-05-08T21:17:07.437903+0000","last_became_active":"2024-05-08T21:17:03.235214+0000","last_became_peered":"2024-05-08T21:17:03.235214+0000","last_unstale":"2024-05-08T21:17:07.437903+0000","last_undegraded":"2024-05-08T21:17:07.437903+0000","last_fullsized":"2024-05-08T21:17:07.437903+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:45.042596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.044143+0000","last_change":"2024-05-08T21:17:07.266080+0000","last_active":"2024-05-08T21:17:08.044143+0000","last_peered":"2024-05-08T21:17:08.044143+0000","last_clean":"2024-05-08T21:17:08.044143+0000","last_became_active":"2024-05-08T21:17:07.265605+0000","last_became_peered":"2024-05-08T21:17:07.265605+0000","last_unstale":"2024-05-08T21:17:08.044143+0000","last_undegraded":"2024-05-08T21:17:08.044143+0000","last_fullsized":"2024-05-08T21:17:08.044143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:49:48.387170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.311476+0000","last_change":"2024-05-08T21:17:02.728743+0000","last_active":"2024-05-08T21:17:31.311476+0000","last_peered":"2024-05-08T21:17:31.311476+0000","last_clean":"2024-05-08T21:17:31.311476+0000","last_became_active":"2024-05-08T21:17:02.728330+0000","last_became_peered":"2024-05-08T21:17:02.728330+0000","last_unstale":"2024-05-08T21:17:31.311476+0000","last_undegraded":"2024-05-08T21:17:31.311476+0000","last_fullsized":"2024-05-08T21:17:31.311476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:43.865117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.437816+0000","last_change":"2024-05-08T21:17:03.245672+0000","last_active":"2024-05-08T21:17:07.437816+0000","last_peered":"2024-05-08T21:17:07.437816+0000","last_clean":"2024-05-08T21:17:07.437816+0000","last_became_active":"2024-05-08T21:17:03.245318+0000","last_became_peered":"2024-05-08T21:17:03.245318+0000","last_unstale":"2024-05-08T21:17:07.437816+0000","last_undegraded":"2024-05-08T21:17:07.437816+0000","last_fullsized":"2024-05-08T21:17:07.437816+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:43.030011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.044374+0000","last_change":"2024-05-08T21:17:07.439289+0000","last_active":"2024-05-08T21:17:08.044374+0000","last_peered":"2024-05-08T21:17:08.044374+0000","last_clean":"2024-05-08T21:17:08.044374+0000","last_became_active":"2024-05-08T21:17:07.438874+0000","last_became_peered":"2024-05-08T21:17:07.438874+0000","last_unstale":"2024-05-08T21:17:08.044374+0000","last_undegraded":"2024-05-08T21:17:08.044374+0000","last_fullsized":"2024-05-08T21:17:08.044374+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:14.820087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.257823+0000","last_change":"2024-05-08T21:17:03.247081+0000","last_active":"2024-05-08T21:17:07.257823+0000","last_peered":"2024-05-08T21:17:07.257823+0000","last_clean":"2024-05-08T21:17:07.257823+0000","last_became_active":"2024-05-08T21:17:03.246418+0000","last_became_peered":"2024-05-08T21:17:03.246418+0000","last_unstale":"2024-05-08T21:17:07.257823+0000","last_undegraded":"2024-05-08T21:17:07.257823+0000","last_fullsized":"2024-05-08T21:17:07.257823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:03.993770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:32.311386+0000","last_change":"2024-05-08T21:17:02.730764+0000","last_active":"2024-05-08T21:17:32.311386+0000","last_peered":"2024-05-08T21:17:32.311386+0000","last_clean":"2024-05-08T21:17:32.311386+0000","last_became_active":"2024-05-08T21:17:02.730494+0000","last_became_peered":"2024-05-08T21:17:02.730494+0000","last_unstale":"2024-05-08T21:17:32.311386+0000","last_undegraded":"2024-05-08T21:17:32.311386+0000","last_fullsized":"2024-05-08T21:17:32.311386+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:50.070368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.266792+0000","last_change":"2024-05-08T21:17:07.266424+0000","last_active":"2024-05-08T21:17:07.266792+0000","last_peered":"2024-05-08T21:17:07.266792+0000","last_clean":"2024-05-08T21:17:07.266792+0000","last_became_active":"2024-05-08T21:17:07.266144+0000","last_became_peered":"2024-05-08T21:17:07.266144+0000","last_unstale":"2024-05-08T21:17:07.266792+0000","last_undegraded":"2024-05-08T21:17:07.266792+0000","last_fullsized":"2024-05-08T21:17:07.266792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:22.285749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.263866+0000","last_change":"2024-05-08T21:17:03.250461+0000","last_active":"2024-05-08T21:17:07.263866+0000","last_peered":"2024-05-08T21:17:07.263866+0000","last_clean":"2024-05-08T21:17:07.263866+0000","last_became_active":"2024-05-08T21:17:03.249974+0000","last_became_peered":"2024-05-08T21:17:03.249974+0000","last_unstale":"2024-05-08T21:17:07.263866+0000","last_undegraded":"2024-05-08T21:17:07.263866+0000","last_fullsized":"2024-05-08T21:17:07.263866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:25:58.759668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.311413+0000","last_change":"2024-05-08T21:17:02.725648+0000","last_active":"2024-05-08T21:17:31.311413+0000","last_peered":"2024-05-08T21:17:31.311413+0000","last_clean":"2024-05-08T21:17:31.311413+0000","last_became_active":"2024-05-08T21:17:02.725297+0000","last_became_peered":"2024-05-08T21:17:02.725297+0000","last_unstale":"2024-05-08T21:17:31.311413+0000","last_undegraded":"2024-05-08T21:17:31.311413+0000","last_fullsized":"2024-05-08T21:17:31.311413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:41.462830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":326,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.001480+0000","last_change":"2024-05-08T21:16:15.194478+0000","last_active":"2024-05-08T21:17:08.001480+0000","last_peered":"2024-05-08T21:17:08.001480+0000","last_clean":"2024-05-08T21:17:08.001480+0000","last_became_active":"2024-05-08T21:16:15.193964+0000","last_became_peered":"2024-05-08T21:16:15.193964+0000","last_unstale":"2024-05-08T21:17:08.001480+0000","last_undegraded":"2024-05-08T21:17:08.001480+0000","last_fullsized":"2024-05-08T21:17:08.001480+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_clean_scrub_stamp":"2024-05-08T21:13:48.584026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:28.091200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714199+0000","last_change":"2024-05-08T21:17:07.439697+0000","last_active":"2024-05-08T21:17:07.714199+0000","last_peered":"2024-05-08T21:17:07.714199+0000","last_clean":"2024-05-08T21:17:07.714199+0000","last_became_active":"2024-05-08T21:17:07.439027+0000","last_became_peered":"2024-05-08T21:17:07.439027+0000","last_unstale":"2024-05-08T21:17:07.714199+0000","last_undegraded":"2024-05-08T21:17:07.714199+0000","last_fullsized":"2024-05-08T21:17:07.714199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:12.585241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.254478+0000","last_change":"2024-05-08T21:17:03.246034+0000","last_active":"2024-05-08T21:17:07.254478+0000","last_peered":"2024-05-08T21:17:07.254478+0000","last_clean":"2024-05-08T21:17:07.254478+0000","last_became_active":"2024-05-08T21:17:03.245620+0000","last_became_peered":"2024-05-08T21:17:03.245620+0000","last_unstale":"2024-05-08T21:17:07.254478+0000","last_undegraded":"2024-05-08T21:17:07.254478+0000","last_fullsized":"2024-05-08T21:17:07.254478+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:02.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.340863+0000","last_change":"2024-05-08T21:17:02.731260+0000","last_active":"2024-05-08T21:17:29.340863+0000","last_peered":"2024-05-08T21:17:29.340863+0000","last_clean":"2024-05-08T21:17:29.340863+0000","last_became_active":"2024-05-08T21:17:02.730879+0000","last_became_peered":"2024-05-08T21:17:02.730879+0000","last_unstale":"2024-05-08T21:17:29.340863+0000","last_undegraded":"2024-05-08T21:17:29.340863+0000","last_fullsized":"2024-05-08T21:17:29.340863+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:59.030674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.000869+0000","last_change":"2024-05-08T21:17:07.269791+0000","last_active":"2024-05-08T21:17:08.000869+0000","last_peered":"2024-05-08T21:17:08.000869+0000","last_clean":"2024-05-08T21:17:08.000869+0000","last_became_active":"2024-05-08T21:17:07.269345+0000","last_became_peered":"2024-05-08T21:17:07.269345+0000","last_unstale":"2024-05-08T21:17:08.000869+0000","last_undegraded":"2024-05-08T21:17:08.000869+0000","last_fullsized":"2024-05-08T21:17:08.000869+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:42.866864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.001283+0000","last_change":"2024-05-08T21:17:02.735423+0000","last_active":"2024-05-08T21:17:08.001283+0000","last_peered":"2024-05-08T21:17:08.001283+0000","last_clean":"2024-05-08T21:17:08.001283+0000","last_became_active":"2024-05-08T21:17:02.735158+0000","last_became_peered":"2024-05-08T21:17:02.735158+0000","last_unstale":"2024-05-08T21:17:08.001283+0000","last_undegraded":"2024-05-08T21:17:08.001283+0000","last_fullsized":"2024-05-08T21:17:08.001283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:19.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.262817+0000","last_change":"2024-05-08T21:17:03.239316+0000","last_active":"2024-05-08T21:17:07.262817+0000","last_peered":"2024-05-08T21:17:07.262817+0000","last_clean":"2024-05-08T21:17:07.262817+0000","last_became_active":"2024-05-08T21:17:03.239018+0000","last_became_peered":"2024-05-08T21:17:03.239018+0000","last_unstale":"2024-05-08T21:17:07.262817+0000","last_undegraded":"2024-05-08T21:17:07.262817+0000","last_fullsized":"2024-05-08T21:17:07.262817+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:36.952155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.970907+0000","last_change":"2024-05-08T21:17:07.439338+0000","last_active":"2024-05-08T21:17:07.970907+0000","last_peered":"2024-05-08T21:17:07.970907+0000","last_clean":"2024-05-08T21:17:07.970907+0000","last_became_active":"2024-05-08T21:17:07.438703+0000","last_became_peered":"2024-05-08T21:17:07.438703+0000","last_unstale":"2024-05-08T21:17:07.970907+0000","last_undegraded":"2024-05-08T21:17:07.970907+0000","last_fullsized":"2024-05-08T21:17:07.970907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:50.997138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.340605+0000","last_change":"2024-05-08T21:17:02.729012+0000","last_active":"2024-05-08T21:17:29.340605+0000","last_peered":"2024-05-08T21:17:29.340605+0000","last_clean":"2024-05-08T21:17:29.340605+0000","last_became_active":"2024-05-08T21:17:02.728606+0000","last_became_peered":"2024-05-08T21:17:02.728606+0000","last_unstale":"2024-05-08T21:17:29.340605+0000","last_undegraded":"2024-05-08T21:17:29.340605+0000","last_fullsized":"2024-05-08T21:17:29.340605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.140609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.268112+0000","last_change":"2024-05-08T21:17:03.234712+0000","last_active":"2024-05-08T21:17:07.268112+0000","last_peered":"2024-05-08T21:17:07.268112+0000","last_clean":"2024-05-08T21:17:07.268112+0000","last_became_active":"2024-05-08T21:17:03.234404+0000","last_became_peered":"2024-05-08T21:17:03.234404+0000","last_unstale":"2024-05-08T21:17:07.268112+0000","last_undegraded":"2024-05-08T21:17:07.268112+0000","last_fullsized":"2024-05-08T21:17:07.268112+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:41.479470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.259891+0000","last_change":"2024-05-08T21:17:07.259120+0000","last_active":"2024-05-08T21:17:07.259891+0000","last_peered":"2024-05-08T21:17:07.259891+0000","last_clean":"2024-05-08T21:17:07.259891+0000","last_became_active":"2024-05-08T21:17:07.258696+0000","last_became_peered":"2024-05-08T21:17:07.258696+0000","last_unstale":"2024-05-08T21:17:07.259891+0000","last_undegraded":"2024-05-08T21:17:07.259891+0000","last_fullsized":"2024-05-08T21:17:07.259891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:26.950767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.311398+0000","last_change":"2024-05-08T21:17:02.727795+0000","last_active":"2024-05-08T21:17:31.311398+0000","last_peered":"2024-05-08T21:17:31.311398+0000","last_clean":"2024-05-08T21:17:31.311398+0000","last_became_active":"2024-05-08T21:17:02.727553+0000","last_became_peered":"2024-05-08T21:17:02.727553+0000","last_unstale":"2024-05-08T21:17:31.311398+0000","last_undegraded":"2024-05-08T21:17:31.311398+0000","last_fullsized":"2024-05-08T21:17:31.311398+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:51.540221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489521+0000","last_change":"2024-05-08T21:17:03.247270+0000","last_active":"2024-05-08T21:17:07.489521+0000","last_peered":"2024-05-08T21:17:07.489521+0000","last_clean":"2024-05-08T21:17:07.489521+0000","last_became_active":"2024-05-08T21:17:03.246853+0000","last_became_peered":"2024-05-08T21:17:03.246853+0000","last_unstale":"2024-05-08T21:17:07.489521+0000","last_undegraded":"2024-05-08T21:17:07.489521+0000","last_fullsized":"2024-05-08T21:17:07.489521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:46.224088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714459+0000","last_change":"2024-05-08T21:17:07.439920+0000","last_active":"2024-05-08T21:17:07.714459+0000","last_peered":"2024-05-08T21:17:07.714459+0000","last_clean":"2024-05-08T21:17:07.714459+0000","last_became_active":"2024-05-08T21:17:07.439459+0000","last_became_peered":"2024-05-08T21:17:07.439459+0000","last_unstale":"2024-05-08T21:17:07.714459+0000","last_undegraded":"2024-05-08T21:17:07.714459+0000","last_fullsized":"2024-05-08T21:17:07.714459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:03.008930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.334698+0000","last_change":"2024-05-08T21:17:02.731889+0000","last_active":"2024-05-08T21:17:28.334698+0000","last_peered":"2024-05-08T21:17:28.334698+0000","last_clean":"2024-05-08T21:17:28.334698+0000","last_became_active":"2024-05-08T21:17:02.731554+0000","last_became_peered":"2024-05-08T21:17:02.731554+0000","last_unstale":"2024-05-08T21:17:28.334698+0000","last_undegraded":"2024-05-08T21:17:28.334698+0000","last_fullsized":"2024-05-08T21:17:28.334698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:25.223003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.252690+0000","last_change":"2024-05-08T21:17:03.252672+0000","last_active":"2024-05-08T21:17:07.252690+0000","last_peered":"2024-05-08T21:17:07.252690+0000","last_clean":"2024-05-08T21:17:07.252690+0000","last_became_active":"2024-05-08T21:17:03.251684+0000","last_became_peered":"2024-05-08T21:17:03.251684+0000","last_unstale":"2024-05-08T21:17:07.252690+0000","last_undegraded":"2024-05-08T21:17:07.252690+0000","last_fullsized":"2024-05-08T21:17:07.252690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:06.460419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713656+0000","last_change":"2024-05-08T21:17:07.273478+0000","last_active":"2024-05-08T21:17:07.713656+0000","last_peered":"2024-05-08T21:17:07.713656+0000","last_clean":"2024-05-08T21:17:07.713656+0000","last_became_active":"2024-05-08T21:17:07.272525+0000","last_became_peered":"2024-05-08T21:17:07.272525+0000","last_unstale":"2024-05-08T21:17:07.713656+0000","last_undegraded":"2024-05-08T21:17:07.713656+0000","last_fullsized":"2024-05-08T21:17:07.713656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:51:05.107387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:30.356496+0000","last_change":"2024-05-08T21:17:02.728858+0000","last_active":"2024-05-08T21:17:30.356496+0000","last_peered":"2024-05-08T21:17:30.356496+0000","last_clean":"2024-05-08T21:17:30.356496+0000","last_became_active":"2024-05-08T21:17:02.728516+0000","last_became_peered":"2024-05-08T21:17:02.728516+0000","last_unstale":"2024-05-08T21:17:30.356496+0000","last_undegraded":"2024-05-08T21:17:30.356496+0000","last_fullsized":"2024-05-08T21:17:30.356496+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:40.263983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.253808+0000","last_change":"2024-05-08T21:17:03.239354+0000","last_active":"2024-05-08T21:17:07.253808+0000","last_peered":"2024-05-08T21:17:07.253808+0000","last_clean":"2024-05-08T21:17:07.253808+0000","last_became_active":"2024-05-08T21:17:03.239020+0000","last_became_peered":"2024-05-08T21:17:03.239020+0000","last_unstale":"2024-05-08T21:17:07.253808+0000","last_undegraded":"2024-05-08T21:17:07.253808+0000","last_fullsized":"2024-05-08T21:17:07.253808+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:02:43.362016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.349903+0000","last_change":"2024-05-08T21:17:07.262401+0000","last_active":"2024-05-08T21:17:08.349903+0000","last_peered":"2024-05-08T21:17:08.349903+0000","last_clean":"2024-05-08T21:17:08.349903+0000","last_became_active":"2024-05-08T21:17:07.261946+0000","last_became_peered":"2024-05-08T21:17:07.261946+0000","last_unstale":"2024-05-08T21:17:08.349903+0000","last_undegraded":"2024-05-08T21:17:08.349903+0000","last_fullsized":"2024-05-08T21:17:08.349903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:56.565604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:29.873113+0000","last_change":"2024-05-08T21:17:02.731124+0000","last_active":"2024-05-08T21:17:29.873113+0000","last_peered":"2024-05-08T21:17:29.873113+0000","last_clean":"2024-05-08T21:17:29.873113+0000","last_became_active":"2024-05-08T21:17:02.730759+0000","last_became_peered":"2024-05-08T21:17:02.730759+0000","last_unstale":"2024-05-08T21:17:29.873113+0000","last_undegraded":"2024-05-08T21:17:29.873113+0000","last_fullsized":"2024-05-08T21:17:29.873113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:07.213439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.263623+0000","last_change":"2024-05-08T21:17:03.250382+0000","last_active":"2024-05-08T21:17:07.263623+0000","last_peered":"2024-05-08T21:17:07.263623+0000","last_clean":"2024-05-08T21:17:07.263623+0000","last_became_active":"2024-05-08T21:17:03.250031+0000","last_became_peered":"2024-05-08T21:17:03.250031+0000","last_unstale":"2024-05-08T21:17:07.263623+0000","last_undegraded":"2024-05-08T21:17:07.263623+0000","last_fullsized":"2024-05-08T21:17:07.263623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:59.961012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714358+0000","last_change":"2024-05-08T21:17:07.439341+0000","last_active":"2024-05-08T21:17:07.714358+0000","last_peered":"2024-05-08T21:17:07.714358+0000","last_clean":"2024-05-08T21:17:07.714358+0000","last_became_active":"2024-05-08T21:17:07.438896+0000","last_became_peered":"2024-05-08T21:17:07.438896+0000","last_unstale":"2024-05-08T21:17:07.714358+0000","last_undegraded":"2024-05-08T21:17:07.714358+0000","last_fullsized":"2024-05-08T21:17:07.714358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:55.180686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:28.334782+0000","last_change":"2024-05-08T21:17:02.732967+0000","last_active":"2024-05-08T21:17:28.334782+0000","last_peered":"2024-05-08T21:17:28.334782+0000","last_clean":"2024-05-08T21:17:28.334782+0000","last_became_active":"2024-05-08T21:17:02.732616+0000","last_became_peered":"2024-05-08T21:17:02.732616+0000","last_unstale":"2024-05-08T21:17:28.334782+0000","last_undegraded":"2024-05-08T21:17:28.334782+0000","last_fullsized":"2024-05-08T21:17:28.334782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:50:26.116841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.257264+0000","last_change":"2024-05-08T21:17:03.241974+0000","last_active":"2024-05-08T21:17:07.257264+0000","last_peered":"2024-05-08T21:17:07.257264+0000","last_clean":"2024-05-08T21:17:07.257264+0000","last_became_active":"2024-05-08T21:17:03.241627+0000","last_became_peered":"2024-05-08T21:17:03.241627+0000","last_unstale":"2024-05-08T21:17:07.257264+0000","last_undegraded":"2024-05-08T21:17:07.257264+0000","last_fullsized":"2024-05-08T21:17:07.257264+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:57.351078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:08.002041+0000","last_change":"2024-05-08T21:17:07.259593+0000","last_active":"2024-05-08T21:17:08.002041+0000","last_peered":"2024-05-08T21:17:08.002041+0000","last_clean":"2024-05-08T21:17:08.002041+0000","last_became_active":"2024-05-08T21:17:07.258362+0000","last_became_peered":"2024-05-08T21:17:07.258362+0000","last_unstale":"2024-05-08T21:17:08.002041+0000","last_undegraded":"2024-05-08T21:17:08.002041+0000","last_fullsized":"2024-05-08T21:17:08.002041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:07.877264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.304152+0000","last_change":"2024-05-08T21:17:02.731685+0000","last_active":"2024-05-08T21:17:31.304152+0000","last_peered":"2024-05-08T21:17:31.304152+0000","last_clean":"2024-05-08T21:17:31.304152+0000","last_became_active":"2024-05-08T21:17:02.731293+0000","last_became_peered":"2024-05-08T21:17:02.731293+0000","last_unstale":"2024-05-08T21:17:31.304152+0000","last_undegraded":"2024-05-08T21:17:31.304152+0000","last_fullsized":"2024-05-08T21:17:31.304152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:36.532962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.256159+0000","last_change":"2024-05-08T21:17:03.250090+0000","last_active":"2024-05-08T21:17:07.256159+0000","last_peered":"2024-05-08T21:17:07.256159+0000","last_clean":"2024-05-08T21:17:07.256159+0000","last_became_active":"2024-05-08T21:17:03.249744+0000","last_became_peered":"2024-05-08T21:17:03.249744+0000","last_unstale":"2024-05-08T21:17:07.256159+0000","last_undegraded":"2024-05-08T21:17:07.256159+0000","last_fullsized":"2024-05-08T21:17:07.256159+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:43.155040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489433+0000","last_change":"2024-05-08T21:17:07.264101+0000","last_active":"2024-05-08T21:17:07.489433+0000","last_peered":"2024-05-08T21:17:07.489433+0000","last_clean":"2024-05-08T21:17:07.489433+0000","last_became_active":"2024-05-08T21:17:07.263720+0000","last_became_peered":"2024-05-08T21:17:07.263720+0000","last_unstale":"2024-05-08T21:17:07.489433+0000","last_undegraded":"2024-05-08T21:17:07.489433+0000","last_fullsized":"2024-05-08T21:17:07.489433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:16.943044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'2","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.714336+0000","last_change":"2024-05-08T21:17:02.726763+0000","last_active":"2024-05-08T21:17:07.714336+0000","last_peered":"2024-05-08T21:17:07.714336+0000","last_clean":"2024-05-08T21:17:07.714336+0000","last_became_active":"2024-05-08T21:17:02.726343+0000","last_became_peered":"2024-05-08T21:17:02.726343+0000","last_unstale":"2024-05-08T21:17:07.714336+0000","last_undegraded":"2024-05-08T21:17:07.714336+0000","last_fullsized":"2024-05-08T21:17:07.714336+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:03.504667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.252549+0000","last_change":"2024-05-08T21:17:03.244468+0000","last_active":"2024-05-08T21:17:07.252549+0000","last_peered":"2024-05-08T21:17:07.252549+0000","last_clean":"2024-05-08T21:17:07.252549+0000","last_became_active":"2024-05-08T21:17:03.244132+0000","last_became_peered":"2024-05-08T21:17:03.244132+0000","last_unstale":"2024-05-08T21:17:07.252549+0000","last_undegraded":"2024-05-08T21:17:07.252549+0000","last_fullsized":"2024-05-08T21:17:07.252549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:17.423657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.713671+0000","last_change":"2024-05-08T21:17:07.260981+0000","last_active":"2024-05-08T21:17:07.713671+0000","last_peered":"2024-05-08T21:17:07.713671+0000","last_clean":"2024-05-08T21:17:07.713671+0000","last_became_active":"2024-05-08T21:17:07.260582+0000","last_became_peered":"2024-05-08T21:17:07.260582+0000","last_unstale":"2024-05-08T21:17:07.713671+0000","last_undegraded":"2024-05-08T21:17:07.713671+0000","last_fullsized":"2024-05-08T21:17:07.713671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:25.319411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-08T21:17:31.310205+0000","last_change":"2024-05-08T21:17:02.732531+0000","last_active":"2024-05-08T21:17:31.310205+0000","last_peered":"2024-05-08T21:17:31.310205+0000","last_clean":"2024-05-08T21:17:31.310205+0000","last_became_active":"2024-05-08T21:17:02.732121+0000","last_became_peered":"2024-05-08T21:17:02.732121+0000","last_unstale":"2024-05-08T21:17:31.310205+0000","last_undegraded":"2024-05-08T21:17:31.310205+0000","last_fullsized":"2024-05-08T21:17:31.310205+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:25.482458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.254189+0000","last_change":"2024-05-08T21:17:03.249676+0000","last_active":"2024-05-08T21:17:07.254189+0000","last_peered":"2024-05-08T21:17:07.254189+0000","last_clean":"2024-05-08T21:17:07.254189+0000","last_became_active":"2024-05-08T21:17:03.249120+0000","last_became_peered":"2024-05-08T21:17:03.249120+0000","last_unstale":"2024-05-08T21:17:07.254189+0000","last_undegraded":"2024-05-08T21:17:07.254189+0000","last_fullsized":"2024-05-08T21:17:07.254189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:58:40.678800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.253901+0000","last_change":"2024-05-08T21:17:03.245585+0000","last_active":"2024-05-08T21:17:07.253901+0000","last_peered":"2024-05-08T21:17:07.253901+0000","last_clean":"2024-05-08T21:17:07.253901+0000","last_became_active":"2024-05-08T21:17:03.245274+0000","last_became_peered":"2024-05-08T21:17:03.245274+0000","last_unstale":"2024-05-08T21:17:07.253901+0000","last_undegraded":"2024-05-08T21:17:07.253901+0000","last_fullsized":"2024-05-08T21:17:07.253901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:32.003962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.250334+0000","last_change":"2024-05-08T21:17:03.246622+0000","last_active":"2024-05-08T21:17:07.250334+0000","last_peered":"2024-05-08T21:17:07.250334+0000","last_clean":"2024-05-08T21:17:07.250334+0000","last_became_active":"2024-05-08T21:17:03.246309+0000","last_became_peered":"2024-05-08T21:17:03.246309+0000","last_unstale":"2024-05-08T21:17:07.250334+0000","last_undegraded":"2024-05-08T21:17:07.250334+0000","last_fullsized":"2024-05-08T21:17:07.250334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:32.439389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.248595+0000","last_change":"2024-05-08T21:17:03.243460+0000","last_active":"2024-05-08T21:17:07.248595+0000","last_peered":"2024-05-08T21:17:07.248595+0000","last_clean":"2024-05-08T21:17:07.248595+0000","last_became_active":"2024-05-08T21:17:03.243097+0000","last_became_peered":"2024-05-08T21:17:03.243097+0000","last_unstale":"2024-05-08T21:17:07.248595+0000","last_undegraded":"2024-05-08T21:17:07.248595+0000","last_fullsized":"2024-05-08T21:17:07.248595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:52:14.492180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.489451+0000","last_change":"2024-05-08T21:17:03.245077+0000","last_active":"2024-05-08T21:17:07.489451+0000","last_peered":"2024-05-08T21:17:07.489451+0000","last_clean":"2024-05-08T21:17:07.489451+0000","last_became_active":"2024-05-08T21:17:03.244719+0000","last_became_peered":"2024-05-08T21:17:03.244719+0000","last_unstale":"2024-05-08T21:17:07.489451+0000","last_undegraded":"2024-05-08T21:17:07.489451+0000","last_fullsized":"2024-05-08T21:17:07.489451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.388318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.262726+0000","last_change":"2024-05-08T21:17:03.245261+0000","last_active":"2024-05-08T21:17:07.262726+0000","last_peered":"2024-05-08T21:17:07.262726+0000","last_clean":"2024-05-08T21:17:07.262726+0000","last_became_active":"2024-05-08T21:17:03.244892+0000","last_became_peered":"2024-05-08T21:17:03.244892+0000","last_unstale":"2024-05-08T21:17:07.262726+0000","last_undegraded":"2024-05-08T21:17:07.262726+0000","last_fullsized":"2024-05-08T21:17:07.262726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:21.075634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.259640+0000","last_change":"2024-05-08T21:17:03.246250+0000","last_active":"2024-05-08T21:17:07.259640+0000","last_peered":"2024-05-08T21:17:07.259640+0000","last_clean":"2024-05-08T21:17:07.259640+0000","last_became_active":"2024-05-08T21:17:03.245807+0000","last_became_peered":"2024-05-08T21:17:03.245807+0000","last_unstale":"2024-05-08T21:17:07.259640+0000","last_undegraded":"2024-05-08T21:17:07.259640+0000","last_fullsized":"2024-05-08T21:17:07.259640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:20.618260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.250263+0000","last_change":"2024-05-08T21:17:03.254559+0000","last_active":"2024-05-08T21:17:07.250263+0000","last_peered":"2024-05-08T21:17:07.250263+0000","last_clean":"2024-05-08T21:17:07.250263+0000","last_became_active":"2024-05-08T21:17:03.254231+0000","last_became_peered":"2024-05-08T21:17:03.254231+0000","last_unstale":"2024-05-08T21:17:07.250263+0000","last_undegraded":"2024-05-08T21:17:07.250263+0000","last_fullsized":"2024-05-08T21:17:07.250263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:47.996927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.268225+0000","last_change":"2024-05-08T21:17:03.242479+0000","last_active":"2024-05-08T21:17:07.268225+0000","last_peered":"2024-05-08T21:17:07.268225+0000","last_clean":"2024-05-08T21:17:07.268225+0000","last_became_active":"2024-05-08T21:17:03.241397+0000","last_became_peered":"2024-05-08T21:17:03.241397+0000","last_unstale":"2024-05-08T21:17:07.268225+0000","last_undegraded":"2024-05-08T21:17:07.268225+0000","last_fullsized":"2024-05-08T21:17:07.268225+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:39.029139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.257663+0000","last_change":"2024-05-08T21:17:03.244208+0000","last_active":"2024-05-08T21:17:07.257663+0000","last_peered":"2024-05-08T21:17:07.257663+0000","last_clean":"2024-05-08T21:17:07.257663+0000","last_became_active":"2024-05-08T21:17:03.243828+0000","last_became_peered":"2024-05-08T21:17:03.243828+0000","last_unstale":"2024-05-08T21:17:07.257663+0000","last_undegraded":"2024-05-08T21:17:07.257663+0000","last_fullsized":"2024-05-08T21:17:07.257663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:39.936750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.263374+0000","last_change":"2024-05-08T21:17:03.240135+0000","last_active":"2024-05-08T21:17:07.263374+0000","last_peered":"2024-05-08T21:17:07.263374+0000","last_clean":"2024-05-08T21:17:07.263374+0000","last_became_active":"2024-05-08T21:17:03.239757+0000","last_became_peered":"2024-05-08T21:17:03.239757+0000","last_unstale":"2024-05-08T21:17:07.263374+0000","last_undegraded":"2024-05-08T21:17:07.263374+0000","last_fullsized":"2024-05-08T21:17:07.263374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:46.066111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.249570+0000","last_change":"2024-05-08T21:17:03.256254+0000","last_active":"2024-05-08T21:17:07.249570+0000","last_peered":"2024-05-08T21:17:07.249570+0000","last_clean":"2024-05-08T21:17:07.249570+0000","last_became_active":"2024-05-08T21:17:03.255310+0000","last_became_peered":"2024-05-08T21:17:03.255310+0000","last_unstale":"2024-05-08T21:17:07.249570+0000","last_undegraded":"2024-05-08T21:17:07.249570+0000","last_fullsized":"2024-05-08T21:17:07.249570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:59.571958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.254974+0000","last_change":"2024-05-08T21:17:03.241901+0000","last_active":"2024-05-08T21:17:07.254974+0000","last_peered":"2024-05-08T21:17:07.254974+0000","last_clean":"2024-05-08T21:17:07.254974+0000","last_became_active":"2024-05-08T21:17:03.241402+0000","last_became_peered":"2024-05-08T21:17:03.241402+0000","last_unstale":"2024-05-08T21:17:07.254974+0000","last_undegraded":"2024-05-08T21:17:07.254974+0000","last_fullsized":"2024-05-08T21:17:07.254974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:59.202987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-08T21:17:07.254310+0000","last_change":"2024-05-08T21:17:03.243050+0000","last_active":"2024-05-08T21:17:07.254310+0000","last_peered":"2024-05-08T21:17:07.254310+0000","last_clean":"2024-05-08T21:17:07.254310+0000","last_became_active":"2024-05-08T21:17:03.240676+0000","last_became_peered":"2024-05-08T21:17:03.240676+0000","last_unstale":"2024-05-08T21:17:07.254310+0000","last_undegraded":"2024-05-08T21:17:07.254310+0000","last_fullsized":"2024-05-08T21:17:07.254310+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:19:53.981559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9657,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":20,"num_read_kb":16,"num_write":172,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":348160,"data_stored":45613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514511,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":459288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]}]},{"osd":10,"up_from":69,"seq":296352743442,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32656,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712496,"statfs":{"total":95995035648,"available":95961595904,"internally_reserved":0,"allocated":1581056,"data_stored":1054775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31848897},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":1,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.71199999999999997}]},{"osd":3,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.627}]},{"osd":4,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.79400000000000004}]},{"osd":5,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.77600000000000002}]},{"osd":6,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.746},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.72399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":8,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.90700000000000003}]},{"osd":9,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.88100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]}]},{"osd":9,"up_from":64,"seq":274877906965,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":460072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51500000000000001}]},{"osd":1,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.81100000000000005}]},{"osd":2,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.63400000000000001}]},{"osd":3,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.70899999999999996}]},{"osd":4,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.746}]},{"osd":5,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.89400000000000002}]},{"osd":6,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.875},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.47099999999999997}]},{"osd":7,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.92300000000000004}]},{"osd":8,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.94099999999999995}]},{"osd":10,"last update":"Wed May 8 21:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.99099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]}]},{"osd":8,"up_from":56,"seq":240518168602,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":459695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.628}]},{"osd":1,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.55000000000000004}]},{"osd":2,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.38400000000000001}]},{"osd":3,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64900000000000002}]},{"osd":4,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.60999999999999999}]},{"osd":5,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.434}]},{"osd":6,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77100000000000002}]},{"osd":9,"last update":"Wed May 8 21:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.746},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.75700000000000001}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.82299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":7,"up_from":49,"seq":210453397533,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1548288,"data_stored":1051420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.26300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.441,"5min":0.40400000000000003,"15min":0.39800000000000002},"min":{"1min":0.27700000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.58799999999999997}]},{"osd":3,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.33600000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.745},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.316,"5min":0.315,"15min":0.315},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.38700000000000001}]},{"osd":5,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.46800000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.45900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.51700000000000002}]},{"osd":6,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.50800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.503}]},{"osd":8,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.73699999999999999}]},{"osd":9,"last update":"Wed May 8 21:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.64800000000000002}]},{"osd":10,"last update":"Wed May 8 21:17:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.68799999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":6,"up_from":44,"seq":188978561058,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":459962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.45900000000000002,"15min":0.46600000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65300000000000002,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.442,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59899999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.42599999999999999}]},{"osd":1,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.432,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.63100000000000001,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.73599999999999999}]},{"osd":2,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.59699999999999998},"min":{"1min":0.438,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83899999999999997,"5min":1.087,"15min":1.087},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60399999999999998,"15min":0.61099999999999999},"min":{"1min":0.41099999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73099999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58499999999999996}]},{"osd":3,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.35699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.66800000000000004,"5min":0.753,"15min":0.753},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83599999999999997,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.49099999999999999}]},{"osd":4,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60499999999999998,"15min":0.59999999999999998},"min":{"1min":0.48899999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64600000000000002,"15min":0.64800000000000002},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80400000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71999999999999997}]},{"osd":5,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.51700000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88100000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93300000000000005,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.755}]},{"osd":7,"last update":"Wed May 8 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90900000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0169999999999999,"5min":1.034,"15min":1.034},"last":0.79400000000000004}]},{"osd":8,"last update":"Wed May 8 21:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.749},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78700000000000003}]},{"osd":9,"last update":"Wed May 8 21:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.82499999999999996}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.86499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]}]},{"osd":1,"up_from":12,"seq":51539607602,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":463084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.51600000000000001,"15min":0.51300000000000001},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.52100000000000002},"min":{"1min":0.40899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.56200000000000006}]},{"osd":2,"last update":"Wed May 8 21:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.50700000000000001,"15min":0.496},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.75900000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.497,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64500000000000002}]},{"osd":3,"last update":"Wed May 8 21:17:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.55300000000000005,"15min":0.51600000000000001},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96199999999999997,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.61199999999999999}]},{"osd":4,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.435,"15min":0.41099999999999998},"min":{"1min":0.315,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.70599999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.432,"15min":0.41699999999999998},"min":{"1min":0.318,"5min":0.253,"15min":0.253},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed May 8 21:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72299999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53200000000000003,"15min":0.52500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed May 8 21:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.47099999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.47399999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.79900000000000004}]},{"osd":7,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92400000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.36799999999999999,"5min":0.308,"15min":0.308},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.58499999999999996}]},{"osd":8,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.83899999999999997}]},{"osd":9,"last update":"Wed May 8 21:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.622},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.85899999999999999}]},{"osd":10,"last update":"Wed May 8 21:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.873},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.73799999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36164,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708988,"statfs":{"total":95995035648,"available":95958003712,"internally_reserved":0,"allocated":978944,"data_stored":461811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 21:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.52600000000000002},"min":{"1min":0.39800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.70699999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.41499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.70999999999999996}]},{"osd":2,"last update":"Wed May 8 21:16:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.58899999999999997,"15min":0.57799999999999996},"min":{"1min":0.40699999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.435,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.748}]},{"osd":3,"last update":"Wed May 8 21:16:08 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47999999999999998,"15min":0.47699999999999998},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.70499999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.44900000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed May 8 21:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78300000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59999999999999998,"15min":0.58099999999999996},"min":{"1min":0.44,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.94199999999999995}]},{"osd":5,"last update":"Wed May 8 21:16:43 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61099999999999999,"15min":0.60499999999999998},"min":{"1min":0.47299999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.441,"5min":0.436,"15min":0.436},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.51400000000000001}]},{"osd":6,"last update":"Wed May 8 21:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.40500000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.753,"5min":0.876,"15min":0.876},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75700000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.625}]},{"osd":7,"last update":"Wed May 8 21:16:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.81599999999999995}]},{"osd":8,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56299999999999994}]},{"osd":9,"last update":"Wed May 8 21:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.92400000000000004}]},{"osd":10,"last update":"Wed May 8 21:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.877}]},{"osd":11,"last update":"Wed May 8 21:17:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.77900000000000003}]}]},{"osd":2,"up_from":17,"seq":73014444080,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":1003520,"data_stored":484654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.498,"15min":0.498},"min":{"1min":0.33200000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed May 8 21:16:49 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.505,"15min":0.504},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63900000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.50900000000000001,"15min":0.51700000000000002},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.72499999999999998}]},{"osd":3,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68500000000000005,"5min":0.752,"15min":0.752},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65600000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.70799999999999996}]},{"osd":4,"last update":"Wed May 8 21:16:27 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.33400000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72599999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.42099999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed May 8 21:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.27300000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.41699999999999998,"15min":0.40899999999999997},"min":{"1min":0.30299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.56200000000000006}]},{"osd":6,"last update":"Wed May 8 21:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55300000000000005,"15min":0.55200000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.748},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.58699999999999997,"15min":0.58599999999999997},"min":{"1min":0.38500000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.76700000000000002}]},{"osd":7,"last update":"Wed May 8 21:16:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.86299999999999999}]},{"osd":8,"last update":"Wed May 8 21:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.84999999999999998}]},{"osd":9,"last update":"Wed May 8 21:16:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.877}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.94899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":3,"up_from":24,"seq":103079215148,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36148,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93709004,"statfs":{"total":95995035648,"available":95958020096,"internally_reserved":0,"allocated":962560,"data_stored":459696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.40500000000000003,"15min":0.38600000000000001},"min":{"1min":0.36199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.36299999999999999}]},{"osd":1,"last update":"Wed May 8 21:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.52500000000000002,"15min":0.51100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53500000000000003,"15min":0.51500000000000001},"min":{"1min":0.40999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.61499999999999999}]},{"osd":2,"last update":"Wed May 8 21:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.39700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.501},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.438}]},{"osd":4,"last update":"Wed May 8 21:17:27 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.57499999999999996,"15min":0.54100000000000004},"min":{"1min":0.47899999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.88300000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.48799999999999999}]},{"osd":5,"last update":"Wed May 8 21:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.621,"15min":0.60899999999999999},"min":{"1min":0.51300000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.46800000000000003}]},{"osd":6,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60399999999999998,"15min":0.60099999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.67500000000000004}]},{"osd":7,"last update":"Wed May 8 21:17:23 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.69199999999999995}]},{"osd":8,"last update":"Wed May 8 21:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79800000000000004}]},{"osd":9,"last update":"Wed May 8 21:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.71599999999999997}]},{"osd":10,"last update":"Wed May 8 21:17:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.751},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.66700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986216,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":464036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.373,"15min":0.36299999999999999},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72899999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.79100000000000004}]},{"osd":3,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.52100000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed May 8 21:16:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55000000000000004,"15min":0.54800000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.54200000000000004,"15min":0.52700000000000002},"min":{"1min":0.44800000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.83299999999999996}]},{"osd":6,"last update":"Wed May 8 21:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.39700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.41199999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.74299999999999999}]},{"osd":7,"last update":"Wed May 8 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.5,"15min":0.47999999999999998},"min":{"1min":0.34300000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.53400000000000003,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.91100000000000003}]},{"osd":8,"last update":"Wed May 8 21:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.94099999999999995}]},{"osd":9,"last update":"Wed May 8 21:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.874},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.86099999999999999}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.89700000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822693,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36732,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708420,"statfs":{"total":95995035648,"available":95957422080,"internally_reserved":0,"allocated":1560576,"data_stored":1051084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52600000000000002,"15min":0.52400000000000002},"min":{"1min":0.40899999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77600000000000002,"5min":0.877,"15min":0.877},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51900000000000002,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73099999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.42099999999999999}]},{"osd":1,"last update":"Wed May 8 21:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.39300000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70199999999999996,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73899999999999999,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed May 8 21:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.308,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60999999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.373,"5min":0.40100000000000002,"15min":0.40600000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.435}]},{"osd":3,"last update":"Wed May 8 21:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.379,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.75800000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.499},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.39700000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.80200000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed May 8 21:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.749,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78700000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61699999999999999}]},{"osd":6,"last update":"Wed May 8 21:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.373,"15min":0.373},"max":{"1min":0.77300000000000002,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.61599999999999999,"15min":0.622},"min":{"1min":0.47699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86899999999999999,"5min":1.01,"15min":1.01},"last":0.68000000000000005}]},{"osd":7,"last update":"Wed May 8 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81999999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71899999999999997}]},{"osd":8,"last update":"Wed May 8 21:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58099999999999996}]},{"osd":9,"last update":"Wed May 8 21:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.73899999999999999}]},{"osd":10,"last update":"Wed May 8 21:17:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.75700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":25366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":5450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T21:17:34.774 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-08T21:17:34.778 INFO:tasks.cephadm:Setting up client nodes... 2024-05-08T21:17:34.778 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-08T21:17:34.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:34 smithi071 ceph-mon[47074]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:34.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:34 smithi071 ceph-mon[47074]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:34.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:34 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1550710131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:17:34.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:34 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1152939620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:34.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:34 smithi082 ceph-mon[47512]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:34.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:34 smithi082 ceph-mon[47512]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:17:34.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:34 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1550710131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:17:34.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:34 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1152939620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:35.058 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:35 smithi033 ceph-mon[42528]: pgmap v244: 129 pgs: 129 active+clean; 586 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-05-08T21:17:35.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:35 smithi033 ceph-mon[42528]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:17:35.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:35 smithi071 ceph-mon[47074]: pgmap v244: 129 pgs: 129 active+clean; 586 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-05-08T21:17:35.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:35 smithi071 ceph-mon[47074]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:17:35.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:35 smithi082 ceph-mon[47512]: pgmap v244: 129 pgs: 129 active+clean; 586 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-05-08T21:17:35.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:35 smithi082 ceph-mon[47512]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:17:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.269+0000 7fc4f2eef700 1 -- 172.21.15.33:0/1787612199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0ffe80 msgr2=0x7fc4ec1002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.269+0000 7fc4f2eef700 1 --2- 172.21.15.33:0/1787612199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0ffe80 0x7fc4ec1002a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009990 tx=0x7fc4e8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.269+0000 7fc4f2eef700 1 -- 172.21.15.33:0/1787612199 shutdown_connections 2024-05-08T21:17:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.269+0000 7fc4f2eef700 1 --2- 172.21.15.33:0/1787612199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4ec101a30 0x7fc4ec108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.269+0000 7fc4f2eef700 1 --2- 172.21.15.33:0/1787612199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0ffe80 0x7fc4ec1002a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.269+0000 7fc4f2eef700 1 --2- 172.21.15.33:0/1787612199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ec101070 0x7fc4ec1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.269+0000 7fc4f2eef700 1 -- 172.21.15.33:0/1787612199 >> 172.21.15.33:0/1787612199 conn(0x7fc4ec0fb470 msgr2=0x7fc4ec0fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.269+0000 7fc4f2eef700 1 -- 172.21.15.33:0/1787612199 shutdown_connections 2024-05-08T21:17:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.269+0000 7fc4f2eef700 1 -- 172.21.15.33:0/1787612199 wait complete. 2024-05-08T21:17:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.269+0000 7fc4f2eef700 1 Processor -- start 2024-05-08T21:17:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f2eef700 1 -- start start 2024-05-08T21:17:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f2eef700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0ffe80 0x7fc4ec103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f2eef700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ec101070 0x7fc4ec1024e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f2eef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4ec101a30 0x7fc4ec102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f2eef700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec1031c0 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f2eef700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec103330 con 0x7fc4ec101a30 2024-05-08T21:17:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f2eef700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec1a0f20 con 0x7fc4ec101070 2024-05-08T21:17:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f0c8b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0ffe80 0x7fc4ec103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f0c8b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0ffe80 0x7fc4ec103e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46404/0 (socket says 172.21.15.33:46404) 2024-05-08T21:17:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f0c8b700 1 -- 172.21.15.33:0/2258154061 learned_addr learned my addr 172.21.15.33:0/2258154061 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f148c700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4ec101a30 0x7fc4ec102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4e3fff700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ec101070 0x7fc4ec1024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f0c8b700 1 -- 172.21.15.33:0/2258154061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ec101070 msgr2=0x7fc4ec1024e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f0c8b700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ec101070 0x7fc4ec1024e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f0c8b700 1 -- 172.21.15.33:0/2258154061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4ec101a30 msgr2=0x7fc4ec102a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f0c8b700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4ec101a30 0x7fc4ec102a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f0c8b700 1 -- 172.21.15.33:0/2258154061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80095f0 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.270+0000 7fc4f0c8b700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0ffe80 0x7fc4ec103e40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8005910 tx=0x7fc4e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.271+0000 7fc4e1ffb700 1 -- 172.21.15.33:0/2258154061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e801d070 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.271+0000 7fc4f2eef700 1 -- 172.21.15.33:0/2258154061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ec1a1150 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.271+0000 7fc4e1ffb700 1 -- 172.21.15.33:0/2258154061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4e800bd50 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.271+0000 7fc4e1ffb700 1 -- 172.21.15.33:0/2258154061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8022cf0 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.271+0000 7fc4f2eef700 1 -- 172.21.15.33:0/2258154061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ec1a1570 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.272+0000 7fc4f2eef700 1 -- 172.21.15.33:0/2258154061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ec04ec60 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.273+0000 7fc4e1ffb700 1 -- 172.21.15.33:0/2258154061 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc4e8022470 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.274+0000 7fc4e1ffb700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc4d40a11b0 0x7fc4d40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.274+0000 7fc4e3fff700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc4d40a11b0 0x7fc4d40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:36.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.274+0000 7fc4e1ffb700 1 -- 172.21.15.33:0/2258154061 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc4e80cea80 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.275+0000 7fc4e3fff700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc4d40a11b0 0x7fc4d40a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0ffbb0 tx=0x7fc4dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:36.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.278+0000 7fc4e1ffb700 1 -- 172.21.15.33:0/2258154061 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc4e8097ef0 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.364 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:36 smithi033 ceph-mon[42528]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:17:36.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.578+0000 7fc4f2eef700 1 -- 172.21.15.33:0/2258154061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc4ec10b600 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.583+0000 7fc4e1ffb700 1 -- 172.21.15.33:0/2258154061 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc4e809b510 con 0x7fc4ec0ffe80 2024-05-08T21:17:36.586 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-05-08T21:17:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDw6ztm/K+QIhAAFNyoFrveoRBwayXeLWa2Pg== 2024-05-08T21:17:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.586+0000 7fc4f2eef700 1 -- 172.21.15.33:0/2258154061 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc4d40a11b0 msgr2=0x7fc4d40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.586+0000 7fc4f2eef700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc4d40a11b0 0x7fc4d40a3670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0ffbb0 tx=0x7fc4dc009260 comp rx=0 tx=0).stop 2024-05-08T21:17:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.586+0000 7fc4f2eef700 1 -- 172.21.15.33:0/2258154061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0ffe80 msgr2=0x7fc4ec103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:36.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.586+0000 7fc4f2eef700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0ffe80 0x7fc4ec103e40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8005910 tx=0x7fc4e8004730 comp rx=0 tx=0).stop 2024-05-08T21:17:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.586+0000 7fc4f2eef700 1 -- 172.21.15.33:0/2258154061 shutdown_connections 2024-05-08T21:17:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.586+0000 7fc4f2eef700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc4d40a11b0 0x7fc4d40a3670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.586+0000 7fc4f2eef700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4ec101a30 0x7fc4ec102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.586+0000 7fc4f2eef700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0ffe80 0x7fc4ec103e40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.586+0000 7fc4f2eef700 1 --2- 172.21.15.33:0/2258154061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ec101070 0x7fc4ec1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.586+0000 7fc4f2eef700 1 -- 172.21.15.33:0/2258154061 >> 172.21.15.33:0/2258154061 conn(0x7fc4ec0fb470 msgr2=0x7fc4ec068b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.586+0000 7fc4f2eef700 1 -- 172.21.15.33:0/2258154061 shutdown_connections 2024-05-08T21:17:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:36.586+0000 7fc4f2eef700 1 -- 172.21.15.33:0/2258154061 wait complete. 2024-05-08T21:17:36.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:36 smithi071 ceph-mon[47074]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:17:36.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:36 smithi082 ceph-mon[47512]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:17:36.999 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:17:36.999 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-08T21:17:36.999 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T21:17:37.052 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T21:17:37.052 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-08T21:17:37.053 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-08T21:17:37.349 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:37.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:37 smithi033 ceph-mon[42528]: pgmap v245: 129 pgs: 129 active+clean; 587 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.2 KiB/s wr, 16 op/s 2024-05-08T21:17:37.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:37 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2258154061' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T21:17:37.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:37 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2258154061' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T21:17:37.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:37 smithi071 ceph-mon[47074]: pgmap v245: 129 pgs: 129 active+clean; 587 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.2 KiB/s wr, 16 op/s 2024-05-08T21:17:37.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:37 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2258154061' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T21:17:37.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:37 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2258154061' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T21:17:37.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:37 smithi082 ceph-mon[47512]: pgmap v245: 129 pgs: 129 active+clean; 587 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 4.2 KiB/s wr, 16 op/s 2024-05-08T21:17:37.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:37 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2258154061' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T21:17:37.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:37 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2258154061' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T21:17:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.569+0000 7f753e24a700 1 -- 172.21.15.33:0/267363087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538105f00 msgr2=0x7f75381062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.569+0000 7f753e24a700 1 --2- 172.21.15.33:0/267363087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538105f00 0x7f75381062e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7520009990 tx=0x7f7520009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.569+0000 7f753e24a700 1 -- 172.21.15.33:0/267363087 shutdown_connections 2024-05-08T21:17:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.569+0000 7f753e24a700 1 --2- 172.21.15.33:0/267363087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7538100890 0x7f753810cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.569+0000 7f753e24a700 1 --2- 172.21.15.33:0/267363087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538105f00 0x7f75381062e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.569+0000 7f753e24a700 1 --2- 172.21.15.33:0/267363087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75380ffed0 0x7f7538100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.569+0000 7f753e24a700 1 -- 172.21.15.33:0/267363087 >> 172.21.15.33:0/267363087 conn(0x7f7538075f80 msgr2=0x7f7538076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.569+0000 7f753e24a700 1 -- 172.21.15.33:0/267363087 shutdown_connections 2024-05-08T21:17:38.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.569+0000 7f753e24a700 1 -- 172.21.15.33:0/267363087 wait complete. 2024-05-08T21:17:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.569+0000 7f753e24a700 1 Processor -- start 2024-05-08T21:17:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.570+0000 7f753e24a700 1 -- start start 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.570+0000 7f753e24a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75380ffed0 0x7f7538195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.570+0000 7f753e24a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538100890 0x7f75381964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.570+0000 7f753e24a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7538105f00 0x7f7538196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.570+0000 7f753e24a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75381a56f0 con 0x7f7538100890 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.570+0000 7f753e24a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75381a5860 con 0x7f7538105f00 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.570+0000 7f753e24a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75381a5a20 con 0x7f75380ffed0 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.570+0000 7f7536ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538100890 0x7f75381964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.570+0000 7f7536ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538100890 0x7f75381964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46424/0 (socket says 172.21.15.33:46424) 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.570+0000 7f7536ffd700 1 -- 172.21.15.33:0/3840212149 learned_addr learned my addr 172.21.15.33:0/3840212149 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.570+0000 7f7537fff700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7538105f00 0x7f7538196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.570+0000 7f75377fe700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75380ffed0 0x7f7538195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.571+0000 7f7536ffd700 1 -- 172.21.15.33:0/3840212149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75380ffed0 msgr2=0x7f7538195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:38.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.571+0000 7f7536ffd700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75380ffed0 0x7f7538195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.571+0000 7f7536ffd700 1 -- 172.21.15.33:0/3840212149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7538105f00 msgr2=0x7f7538196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.571+0000 7f7536ffd700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7538105f00 0x7f7538196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.571+0000 7f7536ffd700 1 -- 172.21.15.33:0/3840212149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75200095f0 con 0x7f7538100890 2024-05-08T21:17:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.571+0000 7f7536ffd700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538100890 0x7f75381964e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f752800d950 tx=0x7f752800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.571+0000 7f7534ff9700 1 -- 172.21.15.33:0/3840212149 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752800f890 con 0x7f7538100890 2024-05-08T21:17:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.571+0000 7f7534ff9700 1 -- 172.21.15.33:0/3840212149 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f752800fe80 con 0x7f7538100890 2024-05-08T21:17:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.571+0000 7f753e24a700 1 -- 172.21.15.33:0/3840212149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75381a5d00 con 0x7f7538100890 2024-05-08T21:17:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.571+0000 7f7534ff9700 1 -- 172.21.15.33:0/3840212149 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752800e3b0 con 0x7f7538100890 2024-05-08T21:17:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.571+0000 7f753e24a700 1 -- 172.21.15.33:0/3840212149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75381a61f0 con 0x7f7538100890 2024-05-08T21:17:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.572+0000 7f753e24a700 1 -- 172.21.15.33:0/3840212149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7538108420 con 0x7f7538100890 2024-05-08T21:17:38.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.573+0000 7f7534ff9700 1 -- 172.21.15.33:0/3840212149 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7528010460 con 0x7f7538100890 2024-05-08T21:17:38.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.574+0000 7f7534ff9700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f75240a1230 0x7f75240a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:38.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.574+0000 7f75377fe700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f75240a1230 0x7f75240a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:38.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.574+0000 7f7534ff9700 1 -- 172.21.15.33:0/3840212149 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f75280cc990 con 0x7f7538100890 2024-05-08T21:17:38.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.574+0000 7f75377fe700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f75240a1230 0x7f75240a36f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f752000b5c0 tx=0x7f7520005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.578+0000 7f7534ff9700 1 -- 172.21.15.33:0/3840212149 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7528095dd0 con 0x7f7538100890 2024-05-08T21:17:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.852+0000 7f753e24a700 1 -- 172.21.15.33:0/3840212149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7538068ef0 con 0x7f7538100890 2024-05-08T21:17:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.857+0000 7f7534ff9700 1 -- 172.21.15.33:0/3840212149 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260286 (secure 0 0 0) 0x7f7528020090 con 0x7f7538100890 2024-05-08T21:17:38.864 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.861+0000 7f753e24a700 1 -- 172.21.15.33:0/3840212149 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f75240a1230 msgr2=0x7f75240a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.861+0000 7f753e24a700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f75240a1230 0x7f75240a36f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f752000b5c0 tx=0x7f7520005fb0 comp rx=0 tx=0).stop 2024-05-08T21:17:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.861+0000 7f753e24a700 1 -- 172.21.15.33:0/3840212149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538100890 msgr2=0x7f75381964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.861+0000 7f753e24a700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538100890 0x7f75381964e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f752800d950 tx=0x7f752800dc60 comp rx=0 tx=0).stop 2024-05-08T21:17:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.861+0000 7f753e24a700 1 -- 172.21.15.33:0/3840212149 shutdown_connections 2024-05-08T21:17:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.861+0000 7f753e24a700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f75240a1230 0x7f75240a36f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:38.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.861+0000 7f753e24a700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7538105f00 0x7f7538196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:38.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.861+0000 7f753e24a700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75380ffed0 0x7f7538195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.861+0000 7f753e24a700 1 --2- 172.21.15.33:0/3840212149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538100890 0x7f75381964e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.861+0000 7f753e24a700 1 -- 172.21.15.33:0/3840212149 >> 172.21.15.33:0/3840212149 conn(0x7f7538075f80 msgr2=0x7f75380fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.861+0000 7f753e24a700 1 -- 172.21.15.33:0/3840212149 shutdown_connections 2024-05-08T21:17:38.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:38.861+0000 7f753e24a700 1 -- 172.21.15.33:0/3840212149 wait complete. 2024-05-08T21:17:39.302 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2288189532},{"type":"v1","addr":"172.21.15.33:6801","nonce":2288189532}]},"active_addr":"172.21.15.33:6801/2288189532","active_change":"2024-05-08T21:11:56.047173+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.33:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":4017296868}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1465921313}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1419901208}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2546944913}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1575584833}]}]} 2024-05-08T21:17:39.307 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-08T21:17:39.307 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-08T21:17:39.307 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T21:17:39.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:39 smithi033 ceph-mon[42528]: pgmap v246: 129 pgs: 129 active+clean; 623 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2024-05-08T21:17:39.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:39 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3840212149' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T21:17:39.567 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:39 smithi071 ceph-mon[47074]: pgmap v246: 129 pgs: 129 active+clean; 623 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2024-05-08T21:17:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:39 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3840212149' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T21:17:39.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:39 smithi082 ceph-mon[47512]: pgmap v246: 129 pgs: 129 active+clean; 623 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2024-05-08T21:17:39.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:39 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3840212149' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T21:17:40.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.802+0000 7fda44959700 1 -- 172.21.15.33:0/326792928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda401019e0 msgr2=0x7fda4010de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:40.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.802+0000 7fda44959700 1 --2- 172.21.15.33:0/326792928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda401019e0 0x7fda4010de10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fda3800b0a0 tx=0x7fda3800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:40.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.802+0000 7fda44959700 1 -- 172.21.15.33:0/326792928 shutdown_connections 2024-05-08T21:17:40.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.802+0000 7fda44959700 1 --2- 172.21.15.33:0/326792928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda401019e0 0x7fda4010de10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:40.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.802+0000 7fda44959700 1 --2- 172.21.15.33:0/326792928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda40107060 0x7fda40107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:40.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.802+0000 7fda44959700 1 --2- 172.21.15.33:0/326792928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda40101020 0x7fda401014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:40.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.802+0000 7fda44959700 1 -- 172.21.15.33:0/326792928 >> 172.21.15.33:0/326792928 conn(0x7fda40078100 msgr2=0x7fda40078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:40.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.802+0000 7fda44959700 1 -- 172.21.15.33:0/326792928 shutdown_connections 2024-05-08T21:17:40.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.802+0000 7fda44959700 1 -- 172.21.15.33:0/326792928 wait complete. 2024-05-08T21:17:40.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.803+0000 7fda44959700 1 Processor -- start 2024-05-08T21:17:40.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.803+0000 7fda44959700 1 -- start start 2024-05-08T21:17:40.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda44959700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda40101020 0x7fda40197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:40.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda44959700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda401019e0 0x7fda40197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:40.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda44959700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda40107060 0x7fda4019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:40.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda44959700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4019c220 con 0x7fda401019e0 2024-05-08T21:17:40.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda44959700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4019c390 con 0x7fda40101020 2024-05-08T21:17:40.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda44959700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4019c550 con 0x7fda40107060 2024-05-08T21:17:40.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda401019e0 0x7fda40197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:40.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda40107060 0x7fda4019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:40.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda401019e0 0x7fda40197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40862/0 (socket says 172.21.15.33:40862) 2024-05-08T21:17:40.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3dd9b700 1 -- 172.21.15.33:0/870447207 learned_addr learned my addr 172.21.15.33:0/870447207 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:40.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3e59c700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda40101020 0x7fda40197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:40.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3dd9b700 1 -- 172.21.15.33:0/870447207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda40107060 msgr2=0x7fda4019bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:40.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3dd9b700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda40107060 0x7fda4019bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:40.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3dd9b700 1 -- 172.21.15.33:0/870447207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda40101020 msgr2=0x7fda40197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:40.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3dd9b700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda40101020 0x7fda40197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:40.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3dd9b700 1 -- 172.21.15.33:0/870447207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda38009d00 con 0x7fda401019e0 2024-05-08T21:17:40.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3ed9d700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda40107060 0x7fda4019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:40.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3e59c700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda40101020 0x7fda40197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda3dd9b700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda401019e0 0x7fda40197860 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fda3400e9a0 tx=0x7fda3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda2f7fe700 1 -- 172.21.15.33:0/870447207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda34009790 con 0x7fda401019e0 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.804+0000 7fda2f7fe700 1 -- 172.21.15.33:0/870447207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fda34010430 con 0x7fda401019e0 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.805+0000 7fda44959700 1 -- 172.21.15.33:0/870447207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4019c830 con 0x7fda401019e0 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.805+0000 7fda2f7fe700 1 -- 172.21.15.33:0/870447207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3400cea0 con 0x7fda401019e0 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.805+0000 7fda44959700 1 -- 172.21.15.33:0/870447207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda401a4090 con 0x7fda401019e0 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.806+0000 7fda44959700 1 -- 172.21.15.33:0/870447207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda401094f0 con 0x7fda401019e0 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.807+0000 7fda2f7fe700 1 -- 172.21.15.33:0/870447207 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fda340105a0 con 0x7fda401019e0 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.807+0000 7fda2f7fe700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fda280a11b0 0x7fda280a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.808+0000 7fda3e59c700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fda280a11b0 0x7fda280a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.808+0000 7fda2f7fe700 1 -- 172.21.15.33:0/870447207 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fda34014070 con 0x7fda401019e0 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.808+0000 7fda3e59c700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fda280a11b0 0x7fda280a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fda30011490 tx=0x7fda30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:40.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:40.813+0000 7fda2f7fe700 1 -- 172.21.15.33:0/870447207 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fda34052d60 con 0x7fda401019e0 2024-05-08T21:17:41.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.024+0000 7fda44959700 1 -- 172.21.15.33:0/870447207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda401026a0 con 0x7fda401019e0 2024-05-08T21:17:41.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.026+0000 7fda2f7fe700 1 -- 172.21.15.33:0/870447207 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23902 (secure 0 0 0) 0x7fda34096280 con 0x7fda401019e0 2024-05-08T21:17:41.029 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:41.029 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:07.709538+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:17:41.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.028+0000 7fda44959700 1 -- 172.21.15.33:0/870447207 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fda280a11b0 msgr2=0x7fda280a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:41.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.028+0000 7fda44959700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fda280a11b0 0x7fda280a3670 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fda30011490 tx=0x7fda30009260 comp rx=0 tx=0).stop 2024-05-08T21:17:41.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.028+0000 7fda44959700 1 -- 172.21.15.33:0/870447207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda401019e0 msgr2=0x7fda40197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:41.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.028+0000 7fda44959700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda401019e0 0x7fda40197860 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fda3400e9a0 tx=0x7fda3400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:17:41.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.028+0000 7fda44959700 1 -- 172.21.15.33:0/870447207 shutdown_connections 2024-05-08T21:17:41.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.028+0000 7fda44959700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda40107060 0x7fda4019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:41.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.028+0000 7fda44959700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fda280a11b0 0x7fda280a3670 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:41.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.028+0000 7fda44959700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda40101020 0x7fda40197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:41.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.028+0000 7fda44959700 1 --2- 172.21.15.33:0/870447207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda401019e0 0x7fda40197860 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:41.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.028+0000 7fda44959700 1 -- 172.21.15.33:0/870447207 >> 172.21.15.33:0/870447207 conn(0x7fda40078100 msgr2=0x7fda400ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:41.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.028+0000 7fda44959700 1 -- 172.21.15.33:0/870447207 shutdown_connections 2024-05-08T21:17:41.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:41.028+0000 7fda44959700 1 -- 172.21.15.33:0/870447207 wait complete. 2024-05-08T21:17:41.424 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-08T21:17:41.424 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T21:17:41.681 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:41.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:41 smithi033 ceph-mon[42528]: pgmap v247: 129 pgs: 129 active+clean; 623 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.8 KiB/s wr, 22 op/s 2024-05-08T21:17:41.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:41 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/870447207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:41.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:41 smithi071 ceph-mon[47074]: pgmap v247: 129 pgs: 129 active+clean; 623 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.8 KiB/s wr, 22 op/s 2024-05-08T21:17:41.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:41 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/870447207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:41.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:41 smithi082 ceph-mon[47512]: pgmap v247: 129 pgs: 129 active+clean; 623 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.8 KiB/s wr, 22 op/s 2024-05-08T21:17:41.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:41 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/870447207' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.976+0000 7fdafda08700 1 -- 172.21.15.33:0/4264851281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8101990 msgr2=0x7fdaf810de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.976+0000 7fdafda08700 1 --2- 172.21.15.33:0/4264851281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8101990 0x7fdaf810de70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000b0a0 tx=0x7fdaf000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.976+0000 7fdafda08700 1 -- 172.21.15.33:0/4264851281 shutdown_connections 2024-05-08T21:17:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.976+0000 7fdafda08700 1 --2- 172.21.15.33:0/4264851281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8101990 0x7fdaf810de70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.976+0000 7fdafda08700 1 --2- 172.21.15.33:0/4264851281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf8107030 0x7fdaf8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.976+0000 7fdafda08700 1 --2- 172.21.15.33:0/4264851281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf8100ff0 0x7fdaf8101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.976+0000 7fdafda08700 1 -- 172.21.15.33:0/4264851281 >> 172.21.15.33:0/4264851281 conn(0x7fdaf8078190 msgr2=0x7fdaf80785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.976+0000 7fdafda08700 1 -- 172.21.15.33:0/4264851281 shutdown_connections 2024-05-08T21:17:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.976+0000 7fdafda08700 1 -- 172.21.15.33:0/4264851281 wait complete. 2024-05-08T21:17:42.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.977+0000 7fdafda08700 1 Processor -- start 2024-05-08T21:17:42.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.977+0000 7fdafda08700 1 -- start start 2024-05-08T21:17:42.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.977+0000 7fdafda08700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf8100ff0 0x7fdaf80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:42.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.977+0000 7fdafda08700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf8101990 0x7fdaf80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:42.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.977+0000 7fdafda08700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8107030 0x7fdaf8073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.977+0000 7fdafda08700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf81a57b0 con 0x7fdaf8107030 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.977+0000 7fdafda08700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf81a5920 con 0x7fdaf8101990 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.977+0000 7fdafda08700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf81a5ae0 con 0x7fdaf8100ff0 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.977+0000 7fdaf77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8107030 0x7fdaf8073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdaf77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8107030 0x7fdaf8073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40880/0 (socket says 172.21.15.33:40880) 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdaf77fe700 1 -- 172.21.15.33:0/2205657319 learned_addr learned my addr 172.21.15.33:0/2205657319 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdaf67fc700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf8101990 0x7fdaf80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdaf6ffd700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf8100ff0 0x7fdaf80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdaf77fe700 1 -- 172.21.15.33:0/2205657319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf8100ff0 msgr2=0x7fdaf80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdaf77fe700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf8100ff0 0x7fdaf80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdaf77fe700 1 -- 172.21.15.33:0/2205657319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf8101990 msgr2=0x7fdaf80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdaf77fe700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf8101990 0x7fdaf80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdaf77fe700 1 -- 172.21.15.33:0/2205657319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf0009d00 con 0x7fdaf8107030 2024-05-08T21:17:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdaf6ffd700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf8100ff0 0x7fdaf80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdaf77fe700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8107030 0x7fdaf8073d20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000b070 tx=0x7fdaf0007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdafca06700 1 -- 172.21.15.33:0/2205657319 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000e040 con 0x7fdaf8107030 2024-05-08T21:17:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdafda08700 1 -- 172.21.15.33:0/2205657319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf81a5d60 con 0x7fdaf8107030 2024-05-08T21:17:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdafca06700 1 -- 172.21.15.33:0/2205657319 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdaf001d480 con 0x7fdaf8107030 2024-05-08T21:17:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdafca06700 1 -- 172.21.15.33:0/2205657319 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf001b6b0 con 0x7fdaf8107030 2024-05-08T21:17:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.978+0000 7fdafda08700 1 -- 172.21.15.33:0/2205657319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf81a61d0 con 0x7fdaf8107030 2024-05-08T21:17:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.979+0000 7fdafda08700 1 -- 172.21.15.33:0/2205657319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf8109550 con 0x7fdaf8107030 2024-05-08T21:17:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.981+0000 7fdafca06700 1 -- 172.21.15.33:0/2205657319 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdaf001d5f0 con 0x7fdaf8107030 2024-05-08T21:17:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.981+0000 7fdafca06700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdae00a11b0 0x7fdae00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.982+0000 7fdafca06700 1 -- 172.21.15.33:0/2205657319 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdaf00ceae0 con 0x7fdaf8107030 2024-05-08T21:17:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.982+0000 7fdaf6ffd700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdae00a11b0 0x7fdae00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.982+0000 7fdaf6ffd700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdae00a11b0 0x7fdae00a3670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009c90 tx=0x7fdaec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:42.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:42.986+0000 7fdafca06700 1 -- 172.21.15.33:0/2205657319 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdaf0097f50 con 0x7fdaf8107030 2024-05-08T21:17:43.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.197+0000 7fdafda08700 1 -- 172.21.15.33:0/2205657319 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaf8074780 con 0x7fdaf8107030 2024-05-08T21:17:43.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.199+0000 7fdafca06700 1 -- 172.21.15.33:0/2205657319 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+23902 (secure 0 0 0) 0x7fdaf00bb020 con 0x7fdaf8107030 2024-05-08T21:17:43.202 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:17:43.202 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:07.709538+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:17:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.201+0000 7fdafda08700 1 -- 172.21.15.33:0/2205657319 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdae00a11b0 msgr2=0x7fdae00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.201+0000 7fdafda08700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdae00a11b0 0x7fdae00a3670 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009c90 tx=0x7fdaec009260 comp rx=0 tx=0).stop 2024-05-08T21:17:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.201+0000 7fdafda08700 1 -- 172.21.15.33:0/2205657319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8107030 msgr2=0x7fdaf8073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.201+0000 7fdafda08700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8107030 0x7fdaf8073d20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000b070 tx=0x7fdaf0007db0 comp rx=0 tx=0).stop 2024-05-08T21:17:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.201+0000 7fdafda08700 1 -- 172.21.15.33:0/2205657319 shutdown_connections 2024-05-08T21:17:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.201+0000 7fdafda08700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf8107030 0x7fdaf8073d20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.201+0000 7fdafda08700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaf8100ff0 0x7fdaf80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.201+0000 7fdafda08700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdae00a11b0 0x7fdae00a3670 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.201+0000 7fdafda08700 1 --2- 172.21.15.33:0/2205657319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaf8101990 0x7fdaf80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.201+0000 7fdafda08700 1 -- 172.21.15.33:0/2205657319 >> 172.21.15.33:0/2205657319 conn(0x7fdaf8078190 msgr2=0x7fdaf80fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.201+0000 7fdafda08700 1 -- 172.21.15.33:0/2205657319 shutdown_connections 2024-05-08T21:17:43.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:43.201+0000 7fdafda08700 1 -- 172.21.15.33:0/2205657319 wait complete. 2024-05-08T21:17:43.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:43 smithi033 ceph-mon[42528]: pgmap v248: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 23 op/s 2024-05-08T21:17:43.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:43 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2205657319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:43.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:43 smithi071 ceph-mon[47074]: pgmap v248: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 23 op/s 2024-05-08T21:17:43.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:43 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2205657319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:43.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:43 smithi082 ceph-mon[47512]: pgmap v248: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 23 op/s 2024-05-08T21:17:43.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:43 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2205657319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:17:43.917 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-08T21:17:43.917 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-08T21:17:43.918 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-08T21:17:43.918 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-08T21:17:43.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-08T21:17:43.919 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-08T21:17:43.920 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-08T21:17:43.920 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-08T21:17:43.921 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-08T21:17:43.921 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-08T21:17:43.921 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-08T21:17:43.922 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-08T21:17:44.858 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:45.287 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:45.287 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:45.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:45 smithi033 ceph-mon[42528]: pgmap v249: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 20 op/s 2024-05-08T21:17:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:45 smithi071 ceph-mon[47074]: pgmap v249: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 20 op/s 2024-05-08T21:17:45.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:45 smithi082 ceph-mon[47512]: pgmap v249: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 20 op/s 2024-05-08T21:17:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.727+0000 7f05be0ee700 1 -- 172.21.15.33:0/3964047846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05b8105f00 msgr2=0x7f05b81062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.727+0000 7f05be0ee700 1 --2- 172.21.15.33:0/3964047846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05b8105f00 0x7f05b81062e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009990 tx=0x7f05a8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.727+0000 7f05be0ee700 1 -- 172.21.15.33:0/3964047846 shutdown_connections 2024-05-08T21:17:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.727+0000 7f05be0ee700 1 --2- 172.21.15.33:0/3964047846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8100890 0x7f05b810cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.727+0000 7f05be0ee700 1 --2- 172.21.15.33:0/3964047846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05b8105f00 0x7f05b81062e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.727+0000 7f05be0ee700 1 --2- 172.21.15.33:0/3964047846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b80ffed0 0x7f05b8100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.727+0000 7f05be0ee700 1 -- 172.21.15.33:0/3964047846 >> 172.21.15.33:0/3964047846 conn(0x7f05b8075f80 msgr2=0x7f05b8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.727+0000 7f05be0ee700 1 -- 172.21.15.33:0/3964047846 shutdown_connections 2024-05-08T21:17:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.727+0000 7f05be0ee700 1 -- 172.21.15.33:0/3964047846 wait complete. 2024-05-08T21:17:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.728+0000 7f05be0ee700 1 Processor -- start 2024-05-08T21:17:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.728+0000 7f05be0ee700 1 -- start start 2024-05-08T21:17:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05be0ee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05b80ffed0 0x7f05b8197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05be0ee700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8100890 0x7f05b8197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05be0ee700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b8105f00 0x7f05b819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05be0ee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b819c1b0 con 0x7f05b80ffed0 2024-05-08T21:17:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05be0ee700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b819c320 con 0x7f05b8100890 2024-05-08T21:17:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05be0ee700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b819c4e0 con 0x7f05b8105f00 2024-05-08T21:17:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05b80ffed0 0x7f05b8197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05b80ffed0 0x7f05b8197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40894/0 (socket says 172.21.15.33:40894) 2024-05-08T21:17:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b77fe700 1 -- 172.21.15.33:0/1779222164 learned_addr learned my addr 172.21.15.33:0/1779222164 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b6ffd700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8100890 0x7f05b8197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b7fff700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b8105f00 0x7f05b819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b77fe700 1 -- 172.21.15.33:0/1779222164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b8105f00 msgr2=0x7f05b819bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b77fe700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b8105f00 0x7f05b819bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b77fe700 1 -- 172.21.15.33:0/1779222164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8100890 msgr2=0x7f05b8197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b77fe700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8100890 0x7f05b8197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b77fe700 1 -- 172.21.15.33:0/1779222164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80095f0 con 0x7f05b80ffed0 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b7fff700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05b8105f00 0x7f05b819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b6ffd700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b8100890 0x7f05b8197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.729+0000 7f05b77fe700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05b80ffed0 0x7f05b8197300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f05a8004760 tx=0x7f05a8004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.730+0000 7f05b4ff9700 1 -- 172.21.15.33:0/1779222164 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a801d070 con 0x7f05b80ffed0 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.730+0000 7f05be0ee700 1 -- 172.21.15.33:0/1779222164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b819c760 con 0x7f05b80ffed0 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.730+0000 7f05b4ff9700 1 -- 172.21.15.33:0/1779222164 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05a800bec0 con 0x7f05b80ffed0 2024-05-08T21:17:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.730+0000 7f05b4ff9700 1 -- 172.21.15.33:0/1779222164 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a8005280 con 0x7f05b80ffed0 2024-05-08T21:17:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.730+0000 7f05be0ee700 1 -- 172.21.15.33:0/1779222164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b819ca20 con 0x7f05b80ffed0 2024-05-08T21:17:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.731+0000 7f05be0ee700 1 -- 172.21.15.33:0/1779222164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f05b81015b0 con 0x7f05b80ffed0 2024-05-08T21:17:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.732+0000 7f05b4ff9700 1 -- 172.21.15.33:0/1779222164 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f05a80049c0 con 0x7f05b80ffed0 2024-05-08T21:17:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.732+0000 7f05b4ff9700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f05a00a1230 0x7f05a00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.733+0000 7f05b6ffd700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f05a00a1230 0x7f05a00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.733+0000 7f05b4ff9700 1 -- 172.21.15.33:0/1779222164 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f05a80ce330 con 0x7f05b80ffed0 2024-05-08T21:17:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.733+0000 7f05b4ff9700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540] conn(0x7f05a00abdb0 0x7f05a00ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.733+0000 7f05b4ff9700 1 -- 172.21.15.33:0/1779222164 --> [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05a00ae8c0 con 0x7f05a00abdb0 2024-05-08T21:17:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.733+0000 7f05b4ff9700 1 -- 172.21.15.33:0/1779222164 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05a80ce740 con 0x7f05b80ffed0 2024-05-08T21:17:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.733+0000 7f05b6ffd700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f05a00a1230 0x7f05a00a36f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f05b8198810 tx=0x7f05ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.733+0000 7f05b7fff700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540] conn(0x7f05a00abdb0 0x7f05a00ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.734+0000 7f05b7fff700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540] conn(0x7f05a00abdb0 0x7f05a00ae1f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.734+0000 7f05b4ff9700 1 -- 172.21.15.33:0/1779222164 <== osd.2 v2:172.21.15.82:6800/3057197540 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f05a00ae8c0 con 0x7f05a00abdb0 2024-05-08T21:17:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.751+0000 7f05be0ee700 1 -- 172.21.15.33:0/1779222164 --> [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f05b8110ff0 con 0x7f05a00abdb0 2024-05-08T21:17:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.752+0000 7f05b4ff9700 1 -- 172.21.15.33:0/1779222164 <== osd.2 v2:172.21.15.82:6800/3057197540 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f05b8110ff0 con 0x7f05a00abdb0 2024-05-08T21:17:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.752+0000 7f05be0ee700 1 -- 172.21.15.33:0/1779222164 >> [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540] conn(0x7f05a00abdb0 msgr2=0x7f05a00ae1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.752+0000 7f05be0ee700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540] conn(0x7f05a00abdb0 0x7f05a00ae1f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.752+0000 7f05be0ee700 1 -- 172.21.15.33:0/1779222164 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f05a00a1230 msgr2=0x7f05a00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.752+0000 7f05be0ee700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f05a00a1230 0x7f05a00a36f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f05b8198810 tx=0x7f05ac009260 comp rx=0 tx=0).stop 2024-05-08T21:17:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.752+0000 7f05be0ee700 1 -- 172.21.15.33:0/1779222164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05b80ffed0 msgr2=0x7f05b8197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.752+0000 7f05be0ee700 1 --2- 172.21.15.33:0/1779222164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05b80ffed0 0x7f05b8197300 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f05a8004760 tx=0x7f05a8004790 comp rx=0 tx=0).stop 2024-05-08T21:17:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.752+0000 7f05b7fff700 1 -- 172.21.15.33:0/1779222164 reap_dead start 2024-05-08T21:17:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.753+0000 7f05be0ee700 1 -- 172.21.15.33:0/1779222164 shutdown_connections 2024-05-08T21:17:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.753+0000 7f05be0ee700 1 -- 172.21.15.33:0/1779222164 >> 172.21.15.33:0/1779222164 conn(0x7f05b8075f80 msgr2=0x7f05b80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.753+0000 7f05be0ee700 1 -- 172.21.15.33:0/1779222164 shutdown_connections 2024-05-08T21:17:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:46.753+0000 7f05be0ee700 1 -- 172.21.15.33:0/1779222164 wait complete. 2024-05-08T21:17:46.873 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:47.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:47 smithi033 ceph-mon[42528]: pgmap v250: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 16 op/s 2024-05-08T21:17:47.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:47 smithi071 ceph-mon[47074]: pgmap v250: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 16 op/s 2024-05-08T21:17:47.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:47 smithi082 ceph-mon[47512]: pgmap v250: 129 pgs: 129 active+clean; 632 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.0 KiB/s wr, 16 op/s 2024-05-08T21:17:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:48.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.772+0000 7f0fcfe13700 1 -- 172.21.15.33:0/996828815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fc8111c30 msgr2=0x7f0fc8114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:48.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.772+0000 7f0fcfe13700 1 --2- 172.21.15.33:0/996828815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fc8111c30 0x7f0fc8114040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000b0a0 tx=0x7f0fc000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.772+0000 7f0fcfe13700 1 -- 172.21.15.33:0/996828815 shutdown_connections 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.772+0000 7f0fcfe13700 1 --2- 172.21.15.33:0/996828815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fc8111c30 0x7f0fc8114040 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.772+0000 7f0fcfe13700 1 --2- 172.21.15.33:0/996828815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc8075000 0x7f0fc80731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.772+0000 7f0fcfe13700 1 --2- 172.21.15.33:0/996828815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fc8073710 0x7f0fc8073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.772+0000 7f0fcfe13700 1 -- 172.21.15.33:0/996828815 >> 172.21.15.33:0/996828815 conn(0x7f0fc8078190 msgr2=0x7f0fc80785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.772+0000 7f0fcfe13700 1 -- 172.21.15.33:0/996828815 shutdown_connections 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.772+0000 7f0fcfe13700 1 -- 172.21.15.33:0/996828815 wait complete. 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.773+0000 7f0fcfe13700 1 Processor -- start 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.773+0000 7f0fcfe13700 1 -- start start 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fcfe13700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc8073710 0x7f0fc819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fcfe13700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fc8075000 0x7f0fc819dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fcfe13700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fc8111c30 0x7f0fc81a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fcfe13700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81a2950 con 0x7f0fc8111c30 2024-05-08T21:17:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fcfe13700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81a2ac0 con 0x7f0fc8075000 2024-05-08T21:17:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fcfe13700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc81a2c80 con 0x7f0fc8073710 2024-05-08T21:17:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fce3b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fc8111c30 0x7f0fc81a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fce3b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fc8111c30 0x7f0fc81a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40908/0 (socket says 172.21.15.33:40908) 2024-05-08T21:17:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fce3b0700 1 -- 172.21.15.33:0/2951912485 learned_addr learned my addr 172.21.15.33:0/2951912485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fcdbaf700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc8073710 0x7f0fc819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fcd3ae700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fc8075000 0x7f0fc819dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fce3b0700 1 -- 172.21.15.33:0/2951912485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc8073710 msgr2=0x7f0fc819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fce3b0700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc8073710 0x7f0fc819da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fce3b0700 1 -- 172.21.15.33:0/2951912485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fc8075000 msgr2=0x7f0fc819dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fce3b0700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fc8075000 0x7f0fc819dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fce3b0700 1 -- 172.21.15.33:0/2951912485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc0009d00 con 0x7f0fc8111c30 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.774+0000 7f0fce3b0700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fc8111c30 0x7f0fc81a2380 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0007fd0 tx=0x7f0fc00079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.775+0000 7f0fbeffd700 1 -- 172.21.15.33:0/2951912485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc000e040 con 0x7f0fc8111c30 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.775+0000 7f0fcfe13700 1 -- 172.21.15.33:0/2951912485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc81a2f00 con 0x7f0fc8111c30 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.775+0000 7f0fbeffd700 1 -- 172.21.15.33:0/2951912485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fc001d480 con 0x7f0fc8111c30 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.775+0000 7f0fbeffd700 1 -- 172.21.15.33:0/2951912485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc001b6b0 con 0x7f0fc8111c30 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.775+0000 7f0fcfe13700 1 -- 172.21.15.33:0/2951912485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc81aa800 con 0x7f0fc8111c30 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.776+0000 7f0fcfe13700 1 -- 172.21.15.33:0/2951912485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0fc8117650 con 0x7f0fc8111c30 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.777+0000 7f0fbeffd700 1 -- 172.21.15.33:0/2951912485 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0fc001d5f0 con 0x7f0fc8111c30 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.777+0000 7f0fbeffd700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0fb40a11b0 0x7f0fb40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.778+0000 7f0fcdbaf700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0fb40a11b0 0x7f0fb40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.778+0000 7f0fbeffd700 1 -- 172.21.15.33:0/2951912485 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0fc00ce3b0 con 0x7f0fc8111c30 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.778+0000 7f0fbeffd700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506] conn(0x7f0fb40abd30 0x7f0fb40ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.778+0000 7f0fbeffd700 1 -- 172.21.15.33:0/2951912485 --> [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fb40ae840 con 0x7f0fb40abd30 2024-05-08T21:17:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.778+0000 7f0fbeffd700 1 -- 172.21.15.33:0/2951912485 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fc00d0990 con 0x7f0fc8111c30 2024-05-08T21:17:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.778+0000 7f0fcd3ae700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506] conn(0x7f0fb40abd30 0x7f0fb40ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.778+0000 7f0fcdbaf700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0fb40a11b0 0x7f0fb40a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4005950 tx=0x7f0fc400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.778+0000 7f0fcd3ae700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506] conn(0x7f0fb40abd30 0x7f0fb40ae170 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.779+0000 7f0fbeffd700 1 -- 172.21.15.33:0/2951912485 <== osd.6 v2:172.21.15.33:6818/3543778506 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f0fb40ae840 con 0x7f0fb40abd30 2024-05-08T21:17:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.796+0000 7f0fcfe13700 1 -- 172.21.15.33:0/2951912485 --> [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0fc80748b0 con 0x7f0fb40abd30 2024-05-08T21:17:48.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.796+0000 7f0fbeffd700 1 -- 172.21.15.33:0/2951912485 <== osd.6 v2:172.21.15.33:6818/3543778506 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0fc80748b0 con 0x7f0fb40abd30 2024-05-08T21:17:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.797+0000 7f0fcfe13700 1 -- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506] conn(0x7f0fb40abd30 msgr2=0x7f0fb40ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.797+0000 7f0fcfe13700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506] conn(0x7f0fb40abd30 0x7f0fb40ae170 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.797+0000 7f0fcfe13700 1 -- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0fb40a11b0 msgr2=0x7f0fb40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.797+0000 7f0fcfe13700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0fb40a11b0 0x7f0fb40a3670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4005950 tx=0x7f0fc400b410 comp rx=0 tx=0).stop 2024-05-08T21:17:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.797+0000 7f0fcfe13700 1 -- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fc8111c30 msgr2=0x7f0fc81a2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.797+0000 7f0fcfe13700 1 --2- 172.21.15.33:0/2951912485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fc8111c30 0x7f0fc81a2380 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0007fd0 tx=0x7f0fc00079b0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.797+0000 7f0fce3b0700 1 -- 172.21.15.33:0/2951912485 reap_dead start 2024-05-08T21:17:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.797+0000 7f0fcfe13700 1 -- 172.21.15.33:0/2951912485 shutdown_connections 2024-05-08T21:17:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.797+0000 7f0fcfe13700 1 -- 172.21.15.33:0/2951912485 >> 172.21.15.33:0/2951912485 conn(0x7f0fc8078190 msgr2=0x7f0fc81104b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.798+0000 7f0fcfe13700 1 -- 172.21.15.33:0/2951912485 shutdown_connections 2024-05-08T21:17:48.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:47.798+0000 7f0fcfe13700 1 -- 172.21.15.33:0/2951912485 wait complete. 2024-05-08T21:17:48.344 INFO:teuthology.orchestra.run.smithi033.stdout:73014444083 2024-05-08T21:17:48.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-08T21:17:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.634+0000 7fce2fe7b700 1 -- 172.21.15.33:0/3822100135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce28068950 msgr2=0x7fce28068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:48.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.634+0000 7fce2fe7b700 1 --2- 172.21.15.33:0/3822100135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce28068950 0x7fce28068db0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fce18009990 tx=0x7fce18009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.634+0000 7fce2fe7b700 1 -- 172.21.15.33:0/3822100135 shutdown_connections 2024-05-08T21:17:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.634+0000 7fce2fe7b700 1 --2- 172.21.15.33:0/3822100135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce280692f0 0x7fce2810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.634+0000 7fce2fe7b700 1 --2- 172.21.15.33:0/3822100135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce28107020 0x7fce28107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.634+0000 7fce2fe7b700 1 --2- 172.21.15.33:0/3822100135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce28068950 0x7fce28068db0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.634+0000 7fce2fe7b700 1 -- 172.21.15.33:0/3822100135 >> 172.21.15.33:0/3822100135 conn(0x7fce28075f80 msgr2=0x7fce28076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.634+0000 7fce2fe7b700 1 -- 172.21.15.33:0/3822100135 shutdown_connections 2024-05-08T21:17:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.634+0000 7fce2fe7b700 1 -- 172.21.15.33:0/3822100135 wait complete. 2024-05-08T21:17:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.635+0000 7fce2fe7b700 1 Processor -- start 2024-05-08T21:17:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.635+0000 7fce2fe7b700 1 -- start start 2024-05-08T21:17:48.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.635+0000 7fce2fe7b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce28068950 0x7fce28197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.635+0000 7fce2fe7b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce280692f0 0x7fce28197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.635+0000 7fce2fe7b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce28107020 0x7fce2819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.635+0000 7fce2fe7b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2819c210 con 0x7fce28107020 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.635+0000 7fce2fe7b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2819c380 con 0x7fce28068950 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.635+0000 7fce2fe7b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2819c540 con 0x7fce280692f0 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.635+0000 7fce2e418700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce28107020 0x7fce2819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.635+0000 7fce2e418700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce28107020 0x7fce2819bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40926/0 (socket says 172.21.15.33:40926) 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.635+0000 7fce2e418700 1 -- 172.21.15.33:0/1410244892 learned_addr learned my addr 172.21.15.33:0/1410244892 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce2dc17700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce28068950 0x7fce28197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce2d416700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce280692f0 0x7fce28197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce2e418700 1 -- 172.21.15.33:0/1410244892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce280692f0 msgr2=0x7fce28197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce2e418700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce280692f0 0x7fce28197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce2e418700 1 -- 172.21.15.33:0/1410244892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce28068950 msgr2=0x7fce28197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:48.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce2e418700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce28068950 0x7fce28197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce2e418700 1 -- 172.21.15.33:0/1410244892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce180095f0 con 0x7fce28107020 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce2d416700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce280692f0 0x7fce28197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce2e418700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce28107020 0x7fce2819bc00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce20007ee0 tx=0x7fce2000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce1effd700 1 -- 172.21.15.33:0/1410244892 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20017070 con 0x7fce28107020 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce1effd700 1 -- 172.21.15.33:0/1410244892 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce2000f040 con 0x7fce28107020 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce1effd700 1 -- 172.21.15.33:0/1410244892 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20005710 con 0x7fce28107020 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce2fe7b700 1 -- 172.21.15.33:0/1410244892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2819c820 con 0x7fce28107020 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.636+0000 7fce2fe7b700 1 -- 172.21.15.33:0/1410244892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce281a4080 con 0x7fce28107020 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.637+0000 7fce2fe7b700 1 -- 172.21.15.33:0/1410244892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce281026d0 con 0x7fce28107020 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.638+0000 7fce1effd700 1 -- 172.21.15.33:0/1410244892 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fce20005870 con 0x7fce28107020 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.639+0000 7fce1effd700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fce140a1230 0x7fce140a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.639+0000 7fce1effd700 1 -- 172.21.15.33:0/1410244892 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fce200ccb70 con 0x7fce28107020 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.639+0000 7fce2dc17700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fce140a1230 0x7fce140a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:48.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.639+0000 7fce1effd700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576] conn(0x7fce140abdb0 0x7fce140ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.639+0000 7fce1effd700 1 -- 172.21.15.33:0/1410244892 --> [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce140ae8c0 con 0x7fce140abdb0 2024-05-08T21:17:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.639+0000 7fce1effd700 1 -- 172.21.15.33:0/1410244892 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce200ccf80 con 0x7fce28107020 2024-05-08T21:17:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.639+0000 7fce2dc17700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fce140a1230 0x7fce140a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fce2400bbb0 tx=0x7fce2400b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.639+0000 7fce2d416700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576] conn(0x7fce140abdb0 0x7fce140ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.640+0000 7fce2d416700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576] conn(0x7fce140abdb0 0x7fce140ae1f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.641+0000 7fce1effd700 1 -- 172.21.15.33:0/1410244892 <== osd.10 v2:172.21.15.71:6824/3245386576 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fce140ae8c0 con 0x7fce140abdb0 2024-05-08T21:17:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.657+0000 7fce2fe7b700 1 -- 172.21.15.33:0/1410244892 --> [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fce281110a0 con 0x7fce140abdb0 2024-05-08T21:17:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.658+0000 7fce1effd700 1 -- 172.21.15.33:0/1410244892 <== osd.10 v2:172.21.15.71:6824/3245386576 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fce281110a0 con 0x7fce140abdb0 2024-05-08T21:17:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.658+0000 7fce2fe7b700 1 -- 172.21.15.33:0/1410244892 >> [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576] conn(0x7fce140abdb0 msgr2=0x7fce140ae1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.658+0000 7fce2fe7b700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576] conn(0x7fce140abdb0 0x7fce140ae1f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.659+0000 7fce2fe7b700 1 -- 172.21.15.33:0/1410244892 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fce140a1230 msgr2=0x7fce140a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.659+0000 7fce2fe7b700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fce140a1230 0x7fce140a36f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fce2400bbb0 tx=0x7fce2400b590 comp rx=0 tx=0).stop 2024-05-08T21:17:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.659+0000 7fce2fe7b700 1 -- 172.21.15.33:0/1410244892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce28107020 msgr2=0x7fce2819bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.659+0000 7fce2fe7b700 1 --2- 172.21.15.33:0/1410244892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce28107020 0x7fce2819bc00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce20007ee0 tx=0x7fce2000d3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.659+0000 7fce2e418700 1 -- 172.21.15.33:0/1410244892 reap_dead start 2024-05-08T21:17:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.659+0000 7fce2fe7b700 1 -- 172.21.15.33:0/1410244892 shutdown_connections 2024-05-08T21:17:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.659+0000 7fce2fe7b700 1 -- 172.21.15.33:0/1410244892 >> 172.21.15.33:0/1410244892 conn(0x7fce28075f80 msgr2=0x7fce280fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.659+0000 7fce2fe7b700 1 -- 172.21.15.33:0/1410244892 shutdown_connections 2024-05-08T21:17:48.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:48.659+0000 7fce2fe7b700 1 -- 172.21.15.33:0/1410244892 wait complete. 2024-05-08T21:17:48.686 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:48.686 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:48.939 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:48.940 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:49.546 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:49.666 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:49 smithi033 ceph-mon[42528]: pgmap v251: 129 pgs: 129 active+clean; 640 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 14 op/s 2024-05-08T21:17:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.751+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3624784692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7ac0692f0 msgr2=0x7fc7ac10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.751+0000 7fc7b08e8700 1 --2- 172.21.15.33:0/3624784692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7ac0692f0 0x7fc7ac10de60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b0a0 tx=0x7fc7a400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.751+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3624784692 shutdown_connections 2024-05-08T21:17:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.751+0000 7fc7b08e8700 1 --2- 172.21.15.33:0/3624784692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7ac0692f0 0x7fc7ac10de60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.751+0000 7fc7b08e8700 1 --2- 172.21.15.33:0/3624784692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac107020 0x7fc7ac107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.751+0000 7fc7b08e8700 1 --2- 172.21.15.33:0/3624784692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7ac068950 0x7fc7ac068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.751+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3624784692 >> 172.21.15.33:0/3624784692 conn(0x7fc7ac075f80 msgr2=0x7fc7ac076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.751+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3624784692 shutdown_connections 2024-05-08T21:17:49.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.751+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3624784692 wait complete. 2024-05-08T21:17:49.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.752+0000 7fc7b08e8700 1 Processor -- start 2024-05-08T21:17:49.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.752+0000 7fc7b08e8700 1 -- start start 2024-05-08T21:17:49.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.752+0000 7fc7b08e8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac068950 0x7fc7ac0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.752+0000 7fc7b08e8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7ac0692f0 0x7fc7ac1004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.752+0000 7fc7b08e8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7ac107020 0x7fc7ac100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.752+0000 7fc7b08e8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac1a5780 con 0x7fc7ac107020 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.752+0000 7fc7b08e8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac1a58f0 con 0x7fc7ac068950 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.752+0000 7fc7b08e8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac1a5ab0 con 0x7fc7ac0692f0 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7aa59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac068950 0x7fc7ac0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7aad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7ac107020 0x7fc7ac100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7aa59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac068950 0x7fc7ac0fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51162/0 (socket says 172.21.15.33:51162) 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7aad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7ac107020 0x7fc7ac100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40934/0 (socket says 172.21.15.33:40934) 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7aa59c700 1 -- 172.21.15.33:0/3810979752 learned_addr learned my addr 172.21.15.33:0/3810979752 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7a9d9b700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7ac0692f0 0x7fc7ac1004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7aa59c700 1 -- 172.21.15.33:0/3810979752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7ac0692f0 msgr2=0x7fc7ac1004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7aa59c700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7ac0692f0 0x7fc7ac1004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:49.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7aa59c700 1 -- 172.21.15.33:0/3810979752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7ac107020 msgr2=0x7fc7ac100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:49.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7aa59c700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7ac107020 0x7fc7ac100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:49.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7aa59c700 1 -- 172.21.15.33:0/3810979752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4009d00 con 0x7fc7ac068950 2024-05-08T21:17:49.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7aad9d700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7ac107020 0x7fc7ac100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:49.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.753+0000 7fc7aa59c700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac068950 0x7fc7ac0fffa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00d980 tx=0x7fc79c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:49.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.754+0000 7fc79b7fe700 1 -- 172.21.15.33:0/3810979752 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c004220 con 0x7fc7ac068950 2024-05-08T21:17:49.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.754+0000 7fc79b7fe700 1 -- 172.21.15.33:0/3810979752 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc79c007640 con 0x7fc7ac068950 2024-05-08T21:17:49.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.754+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3810979752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ac1a5d90 con 0x7fc7ac068950 2024-05-08T21:17:49.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.754+0000 7fc79b7fe700 1 -- 172.21.15.33:0/3810979752 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c005660 con 0x7fc7ac068950 2024-05-08T21:17:49.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.754+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3810979752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac1a6230 con 0x7fc7ac068950 2024-05-08T21:17:49.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.754+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3810979752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc7ac110fc0 con 0x7fc7ac068950 2024-05-08T21:17:49.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.756+0000 7fc79b7fe700 1 -- 172.21.15.33:0/3810979752 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc79c004750 con 0x7fc7ac068950 2024-05-08T21:17:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.757+0000 7fc79b7fe700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc7940a11b0 0x7fc7940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:49.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.757+0000 7fc7a9d9b700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc7940a11b0 0x7fc7940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.757+0000 7fc79b7fe700 1 -- 172.21.15.33:0/3810979752 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc79c0ccfd0 con 0x7fc7ac068950 2024-05-08T21:17:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.757+0000 7fc79b7fe700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237] conn(0x7fc7940abd30 0x7fc7940ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.757+0000 7fc79b7fe700 1 -- 172.21.15.33:0/3810979752 --> [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7940ae840 con 0x7fc7940abd30 2024-05-08T21:17:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.757+0000 7fc79b7fe700 1 -- 172.21.15.33:0/3810979752 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc79c0cd340 con 0x7fc7ac068950 2024-05-08T21:17:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.758+0000 7fc7aad9d700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237] conn(0x7fc7940abd30 0x7fc7940ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.758+0000 7fc7a9d9b700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc7940a11b0 0x7fc7940a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0766d0 tx=0x7fc7a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:49.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.758+0000 7fc7aad9d700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237] conn(0x7fc7940abd30 0x7fc7940ae170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:49.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.759+0000 7fc79b7fe700 1 -- 172.21.15.33:0/3810979752 <== osd.1 v2:172.21.15.71:6800/2306412237 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc7940ae840 con 0x7fc7940abd30 2024-05-08T21:17:49.794 INFO:teuthology.orchestra.run.smithi033.stdout:188978561061 2024-05-08T21:17:49.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-08T21:17:49.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.792+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3810979752 --> [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc7ac061620 con 0x7fc7940abd30 2024-05-08T21:17:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.792+0000 7fc79b7fe700 1 -- 172.21.15.33:0/3810979752 <== osd.1 v2:172.21.15.71:6800/2306412237 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc7ac061620 con 0x7fc7940abd30 2024-05-08T21:17:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.793+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3810979752 >> [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237] conn(0x7fc7940abd30 msgr2=0x7fc7940ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.793+0000 7fc7b08e8700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237] conn(0x7fc7940abd30 0x7fc7940ae170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.793+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3810979752 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc7940a11b0 msgr2=0x7fc7940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.793+0000 7fc7b08e8700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc7940a11b0 0x7fc7940a3670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0766d0 tx=0x7fc7a000b410 comp rx=0 tx=0).stop 2024-05-08T21:17:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.793+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3810979752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac068950 msgr2=0x7fc7ac0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.793+0000 7fc7b08e8700 1 --2- 172.21.15.33:0/3810979752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ac068950 0x7fc7ac0fffa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00d980 tx=0x7fc79c00dc90 comp rx=0 tx=0).stop 2024-05-08T21:17:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.793+0000 7fc7aad9d700 1 -- 172.21.15.33:0/3810979752 reap_dead start 2024-05-08T21:17:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.793+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3810979752 shutdown_connections 2024-05-08T21:17:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.793+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3810979752 >> 172.21.15.33:0/3810979752 conn(0x7fc7ac075f80 msgr2=0x7fc7ac0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.793+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3810979752 shutdown_connections 2024-05-08T21:17:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:49.793+0000 7fc7b08e8700 1 -- 172.21.15.33:0/3810979752 wait complete. 2024-05-08T21:17:49.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:49 smithi071 ceph-mon[47074]: pgmap v251: 129 pgs: 129 active+clean; 640 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 14 op/s 2024-05-08T21:17:49.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:49 smithi082 ceph-mon[47512]: pgmap v251: 129 pgs: 129 active+clean; 640 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 14 op/s 2024-05-08T21:17:50.099 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:51.149 INFO:teuthology.orchestra.run.smithi033.stdout:51539607607 2024-05-08T21:17:51.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-08T21:17:51.434 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:51 smithi033 ceph-mon[42528]: pgmap v252: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-08T21:17:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.712+0000 7ff8cb254700 1 -- 172.21.15.33:0/522436996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c40692f0 msgr2=0x7ff8c410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.712+0000 7ff8cb254700 1 --2- 172.21.15.33:0/522436996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c40692f0 0x7ff8c410de60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00b0a0 tx=0x7ff8bc00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.712+0000 7ff8cb254700 1 -- 172.21.15.33:0/522436996 shutdown_connections 2024-05-08T21:17:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.712+0000 7ff8cb254700 1 --2- 172.21.15.33:0/522436996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c40692f0 0x7ff8c410de60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.712+0000 7ff8cb254700 1 --2- 172.21.15.33:0/522436996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8c4107020 0x7ff8c4107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.712+0000 7ff8cb254700 1 --2- 172.21.15.33:0/522436996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8c4068950 0x7ff8c4068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.712+0000 7ff8cb254700 1 -- 172.21.15.33:0/522436996 >> 172.21.15.33:0/522436996 conn(0x7ff8c4075f80 msgr2=0x7ff8c4076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.712+0000 7ff8cb254700 1 -- 172.21.15.33:0/522436996 shutdown_connections 2024-05-08T21:17:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.712+0000 7ff8cb254700 1 -- 172.21.15.33:0/522436996 wait complete. 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8cb254700 1 Processor -- start 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8cb254700 1 -- start start 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8cb254700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c4068950 0x7ff8c40fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8cb254700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8c40692f0 0x7ff8c41004e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8cb254700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8c4107020 0x7ff8c4100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8cb254700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c41a57a0 con 0x7ff8c4068950 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8cb254700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c41a5910 con 0x7ff8c40692f0 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8cb254700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c41a5ad0 con 0x7ff8c4107020 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8c8ff0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c4068950 0x7ff8c40fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8c8ff0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c4068950 0x7ff8c40fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38782/0 (socket says 172.21.15.33:38782) 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8c8ff0700 1 -- 172.21.15.33:0/1860399666 learned_addr learned my addr 172.21.15.33:0/1860399666 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8c97f1700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8c4107020 0x7ff8c4100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.713+0000 7ff8c3fff700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8c40692f0 0x7ff8c41004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.714+0000 7ff8c8ff0700 1 -- 172.21.15.33:0/1860399666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8c4107020 msgr2=0x7ff8c4100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.714+0000 7ff8c8ff0700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8c4107020 0x7ff8c4100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.714+0000 7ff8c8ff0700 1 -- 172.21.15.33:0/1860399666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8c40692f0 msgr2=0x7ff8c41004e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.714+0000 7ff8c8ff0700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8c40692f0 0x7ff8c41004e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.714+0000 7ff8c8ff0700 1 -- 172.21.15.33:0/1860399666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc009d00 con 0x7ff8c4068950 2024-05-08T21:17:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.714+0000 7ff8c8ff0700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c4068950 0x7ff8c40fffa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800e9a0 tx=0x7ff8b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.714+0000 7ff8c1ffb700 1 -- 172.21.15.33:0/1860399666 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8009790 con 0x7ff8c4068950 2024-05-08T21:17:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.714+0000 7ff8cb254700 1 -- 172.21.15.33:0/1860399666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c41a5db0 con 0x7ff8c4068950 2024-05-08T21:17:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.714+0000 7ff8c1ffb700 1 -- 172.21.15.33:0/1860399666 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff8b8010430 con 0x7ff8c4068950 2024-05-08T21:17:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.714+0000 7ff8c1ffb700 1 -- 172.21.15.33:0/1860399666 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b800cea0 con 0x7ff8c4068950 2024-05-08T21:17:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.714+0000 7ff8cb254700 1 -- 172.21.15.33:0/1860399666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c41a6250 con 0x7ff8c4068950 2024-05-08T21:17:51.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.715+0000 7ff8cb254700 1 -- 172.21.15.33:0/1860399666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8c41026d0 con 0x7ff8c4068950 2024-05-08T21:17:51.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.716+0000 7ff8c1ffb700 1 -- 172.21.15.33:0/1860399666 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff8b80105a0 con 0x7ff8c4068950 2024-05-08T21:17:51.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.717+0000 7ff8c1ffb700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff8b40a11b0 0x7ff8b40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:51.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.717+0000 7ff8c3fff700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff8b40a11b0 0x7ff8b40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:51.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.717+0000 7ff8c1ffb700 1 -- 172.21.15.33:0/1860399666 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff8b8014070 con 0x7ff8c4068950 2024-05-08T21:17:51.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.717+0000 7ff8c1ffb700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393] conn(0x7ff8b40abd30 0x7ff8b40ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:51.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.717+0000 7ff8c1ffb700 1 -- 172.21.15.33:0/1860399666 --> [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8b40ae840 con 0x7ff8b40abd30 2024-05-08T21:17:51.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.717+0000 7ff8c1ffb700 1 -- 172.21.15.33:0/1860399666 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8b80cd3e0 con 0x7ff8c4068950 2024-05-08T21:17:51.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.718+0000 7ff8c97f1700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393] conn(0x7ff8b40abd30 0x7ff8b40ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:51.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.718+0000 7ff8c3fff700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff8b40a11b0 0x7ff8b40a3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff8c41016d0 tx=0x7ff8b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:51.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.718+0000 7ff8c97f1700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393] conn(0x7ff8b40abd30 0x7ff8b40ae170 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:51.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.719+0000 7ff8c1ffb700 1 -- 172.21.15.33:0/1860399666 <== osd.0 v2:172.21.15.33:6802/1942383393 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff8b40ae840 con 0x7ff8b40abd30 2024-05-08T21:17:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.736+0000 7ff8cb254700 1 -- 172.21.15.33:0/1860399666 --> [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff8c4110f70 con 0x7ff8b40abd30 2024-05-08T21:17:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.736+0000 7ff8c1ffb700 1 -- 172.21.15.33:0/1860399666 <== osd.0 v2:172.21.15.33:6802/1942383393 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff8c4110f70 con 0x7ff8b40abd30 2024-05-08T21:17:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.736+0000 7ff8cb254700 1 -- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393] conn(0x7ff8b40abd30 msgr2=0x7ff8b40ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.736+0000 7ff8cb254700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393] conn(0x7ff8b40abd30 0x7ff8b40ae170 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.737+0000 7ff8cb254700 1 -- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff8b40a11b0 msgr2=0x7ff8b40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.737+0000 7ff8cb254700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff8b40a11b0 0x7ff8b40a3670 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff8c41016d0 tx=0x7ff8b000b410 comp rx=0 tx=0).stop 2024-05-08T21:17:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.737+0000 7ff8cb254700 1 -- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c4068950 msgr2=0x7ff8c40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.737+0000 7ff8cb254700 1 --2- 172.21.15.33:0/1860399666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c4068950 0x7ff8c40fffa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800e9a0 tx=0x7ff8b800ecb0 comp rx=0 tx=0).stop 2024-05-08T21:17:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.737+0000 7ff8c97f1700 1 -- 172.21.15.33:0/1860399666 reap_dead start 2024-05-08T21:17:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.737+0000 7ff8cb254700 1 -- 172.21.15.33:0/1860399666 shutdown_connections 2024-05-08T21:17:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.737+0000 7ff8cb254700 1 -- 172.21.15.33:0/1860399666 >> 172.21.15.33:0/1860399666 conn(0x7ff8c4075f80 msgr2=0x7ff8c40fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.737+0000 7ff8cb254700 1 -- 172.21.15.33:0/1860399666 shutdown_connections 2024-05-08T21:17:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:51.737+0000 7ff8cb254700 1 -- 172.21.15.33:0/1860399666 wait complete. 2024-05-08T21:17:51.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:51 smithi071 ceph-mon[47074]: pgmap v252: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-08T21:17:51.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:51 smithi082 ceph-mon[47512]: pgmap v252: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-08T21:17:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.262+0000 7f9dd2460700 1 -- 172.21.15.33:0/954620651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dcc1084a0 msgr2=0x7f9dcc110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.262+0000 7f9dd2460700 1 --2- 172.21.15.33:0/954620651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dcc1084a0 0x7f9dcc110070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400b0a0 tx=0x7f9dc400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.262+0000 7f9dd2460700 1 -- 172.21.15.33:0/954620651 shutdown_connections 2024-05-08T21:17:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.262+0000 7f9dd2460700 1 --2- 172.21.15.33:0/954620651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dcc1084a0 0x7f9dcc110070 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:52.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.262+0000 7f9dd2460700 1 --2- 172.21.15.33:0/954620651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dcc1033c0 0x7f9dcc1037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.262+0000 7f9dd2460700 1 --2- 172.21.15.33:0/954620651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dcc103d70 0x7f9dcc107dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.262+0000 7f9dd2460700 1 -- 172.21.15.33:0/954620651 >> 172.21.15.33:0/954620651 conn(0x7f9dcc0fec50 msgr2=0x7f9dcc101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.262+0000 7f9dd2460700 1 -- 172.21.15.33:0/954620651 shutdown_connections 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.262+0000 7f9dd2460700 1 -- 172.21.15.33:0/954620651 wait complete. 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.263+0000 7f9dd2460700 1 Processor -- start 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.263+0000 7f9dd2460700 1 -- start start 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.263+0000 7f9dd2460700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dcc1033c0 0x7f9dcc199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.263+0000 7f9dd2460700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dcc103d70 0x7f9dcc199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.263+0000 7f9dd2460700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dcc1084a0 0x7f9dcc19dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.263+0000 7f9dd2460700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc19e4f0 con 0x7f9dcc103d70 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.263+0000 7f9dd2460700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc19e660 con 0x7f9dcc1084a0 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.263+0000 7f9dd2460700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc19e820 con 0x7f9dcc1033c0 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.263+0000 7f9dcb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dcc103d70 0x7f9dcc199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.263+0000 7f9dcb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dcc103d70 0x7f9dcc199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38800/0 (socket says 172.21.15.33:38800) 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.263+0000 7f9dcb7fe700 1 -- 172.21.15.33:0/132449766 learned_addr learned my addr 172.21.15.33:0/132449766 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.263+0000 7f9dcbfff700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dcc1033c0 0x7f9dcc199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dd09fd700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dcc1084a0 0x7f9dcc19dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dcb7fe700 1 -- 172.21.15.33:0/132449766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dcc1033c0 msgr2=0x7f9dcc199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dcb7fe700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dcc1033c0 0x7f9dcc199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dcb7fe700 1 -- 172.21.15.33:0/132449766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dcc1084a0 msgr2=0x7f9dcc19dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dcb7fe700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dcc1084a0 0x7f9dcc19dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dcb7fe700 1 -- 172.21.15.33:0/132449766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4009d00 con 0x7f9dcc103d70 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dcbfff700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dcc1033c0 0x7f9dcc199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dd09fd700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dcc1084a0 0x7f9dcc19dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dcb7fe700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dcc103d70 0x7f9dcc199bd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000e970 tx=0x7f9dc000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dc97fa700 1 -- 172.21.15.33:0/132449766 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0009790 con 0x7f9dcc103d70 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dc97fa700 1 -- 172.21.15.33:0/132449766 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9dc000eec0 con 0x7f9dcc103d70 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dd2460700 1 -- 172.21.15.33:0/132449766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc19eb00 con 0x7f9dcc103d70 2024-05-08T21:17:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dc97fa700 1 -- 172.21.15.33:0/132449766 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00054b0 con 0x7f9dcc103d70 2024-05-08T21:17:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.264+0000 7f9dd2460700 1 -- 172.21.15.33:0/132449766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dcc1a6360 con 0x7f9dcc103d70 2024-05-08T21:17:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.265+0000 7f9dd2460700 1 -- 172.21.15.33:0/132449766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9dcc113200 con 0x7f9dcc103d70 2024-05-08T21:17:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.266+0000 7f9dc97fa700 1 -- 172.21.15.33:0/132449766 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9dc0005610 con 0x7f9dcc103d70 2024-05-08T21:17:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.267+0000 7f9dc97fa700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9db40a11b0 0x7f9db40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.267+0000 7f9dc97fa700 1 -- 172.21.15.33:0/132449766 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9dc0014070 con 0x7f9dcc103d70 2024-05-08T21:17:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.267+0000 7f9dcbfff700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9db40a11b0 0x7f9db40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.267+0000 7f9dc97fa700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764] conn(0x7f9db40abd30 0x7f9db40ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:52.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.267+0000 7f9dc97fa700 1 -- 172.21.15.33:0/132449766 --> [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9db40ae840 con 0x7f9db40abd30 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.267+0000 7f9dc97fa700 1 -- 172.21.15.33:0/132449766 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dc00cd470 con 0x7f9dcc103d70 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.268+0000 7f9dcbfff700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9db40a11b0 0x7f9db40a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc011490 tx=0x7f9dbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.268+0000 7f9dd09fd700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764] conn(0x7f9db40abd30 0x7f9db40ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.268+0000 7f9dd09fd700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764] conn(0x7f9db40abd30 0x7f9db40ae170 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.269+0000 7f9dc97fa700 1 -- 172.21.15.33:0/132449766 <== osd.4 v2:172.21.15.71:6808/4006373764 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9db40ae840 con 0x7f9db40abd30 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.286+0000 7f9dd2460700 1 -- 172.21.15.33:0/132449766 --> [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9dcc0616a0 con 0x7f9db40abd30 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.286+0000 7f9dc97fa700 1 -- 172.21.15.33:0/132449766 <== osd.4 v2:172.21.15.71:6808/4006373764 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9dcc0616a0 con 0x7f9db40abd30 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.287+0000 7f9dd2460700 1 -- 172.21.15.33:0/132449766 >> [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764] conn(0x7f9db40abd30 msgr2=0x7f9db40ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.287+0000 7f9dd2460700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764] conn(0x7f9db40abd30 0x7f9db40ae170 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.287+0000 7f9dd2460700 1 -- 172.21.15.33:0/132449766 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9db40a11b0 msgr2=0x7f9db40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.287+0000 7f9dd2460700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9db40a11b0 0x7f9db40a3670 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc011490 tx=0x7f9dbc009260 comp rx=0 tx=0).stop 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.287+0000 7f9dd2460700 1 -- 172.21.15.33:0/132449766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dcc103d70 msgr2=0x7f9dcc199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.287+0000 7f9dd2460700 1 --2- 172.21.15.33:0/132449766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dcc103d70 0x7f9dcc199bd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000e970 tx=0x7f9dc000ec80 comp rx=0 tx=0).stop 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.287+0000 7f9dd09fd700 1 -- 172.21.15.33:0/132449766 reap_dead start 2024-05-08T21:17:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.287+0000 7f9dd2460700 1 -- 172.21.15.33:0/132449766 shutdown_connections 2024-05-08T21:17:52.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.287+0000 7f9dd2460700 1 -- 172.21.15.33:0/132449766 >> 172.21.15.33:0/132449766 conn(0x7f9dcc0fec50 msgr2=0x7f9dcc100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:52.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.287+0000 7f9dd2460700 1 -- 172.21.15.33:0/132449766 shutdown_connections 2024-05-08T21:17:52.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.287+0000 7f9dd2460700 1 -- 172.21.15.33:0/132449766 wait complete. 2024-05-08T21:17:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.836+0000 7f47a74d5700 1 -- 172.21.15.33:0/2162394240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a00ff940 msgr2=0x7f47a00ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.836+0000 7f47a74d5700 1 --2- 172.21.15.33:0/2162394240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a00ff940 0x7f47a00ffdc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4790009990 tx=0x7f4790009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.836+0000 7f47a74d5700 1 -- 172.21.15.33:0/2162394240 shutdown_connections 2024-05-08T21:17:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.836+0000 7f47a74d5700 1 --2- 172.21.15.33:0/2162394240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47a0100300 0x7f47a010c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.836+0000 7f47a74d5700 1 --2- 172.21.15.33:0/2162394240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a0104650 0x7f47a0104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.836+0000 7f47a74d5700 1 --2- 172.21.15.33:0/2162394240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a00ff940 0x7f47a00ffdc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.836+0000 7f47a74d5700 1 -- 172.21.15.33:0/2162394240 >> 172.21.15.33:0/2162394240 conn(0x7f47a0075260 msgr2=0x7f47a0075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.836+0000 7f47a74d5700 1 -- 172.21.15.33:0/2162394240 shutdown_connections 2024-05-08T21:17:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.836+0000 7f47a74d5700 1 -- 172.21.15.33:0/2162394240 wait complete. 2024-05-08T21:17:53.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.837+0000 7f47a74d5700 1 Processor -- start 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.837+0000 7f47a74d5700 1 -- start start 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.837+0000 7f47a74d5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a00ff940 0x7f47a0197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.837+0000 7f47a74d5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47a0100300 0x7f47a0197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.837+0000 7f47a74d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a0104650 0x7f47a019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.837+0000 7f47a74d5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a019c120 con 0x7f47a0104650 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.837+0000 7f47a74d5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a019c290 con 0x7f47a0100300 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.837+0000 7f47a74d5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a019c450 con 0x7f47a00ff940 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.837+0000 7f47a5a72700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a0104650 0x7f47a019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.837+0000 7f47a5a72700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a0104650 0x7f47a019bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38818/0 (socket says 172.21.15.33:38818) 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.837+0000 7f47a5a72700 1 -- 172.21.15.33:0/297741642 learned_addr learned my addr 172.21.15.33:0/297741642 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47a4a70700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47a0100300 0x7f47a0197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47a5271700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a00ff940 0x7f47a0197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47a5a72700 1 -- 172.21.15.33:0/297741642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a00ff940 msgr2=0x7f47a0197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47a5a72700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47a00ff940 0x7f47a0197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47a5a72700 1 -- 172.21.15.33:0/297741642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47a0100300 msgr2=0x7f47a0197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47a5a72700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47a0100300 0x7f47a0197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47a5a72700 1 -- 172.21.15.33:0/297741642 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47900095f0 con 0x7f47a0104650 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47a4a70700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47a0100300 0x7f47a0197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47a5a72700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a0104650 0x7f47a019bb10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4798007ee0 tx=0x7f479800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47967fc700 1 -- 172.21.15.33:0/297741642 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479800ddd0 con 0x7f47a0104650 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47967fc700 1 -- 172.21.15.33:0/297741642 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f479800f040 con 0x7f47a0104650 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47a74d5700 1 -- 172.21.15.33:0/297741642 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a019c730 con 0x7f47a0104650 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47967fc700 1 -- 172.21.15.33:0/297741642 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798004870 con 0x7f47a0104650 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.838+0000 7f47a74d5700 1 -- 172.21.15.33:0/297741642 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a01a3f90 con 0x7f47a0104650 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.839+0000 7f47a74d5700 1 -- 172.21.15.33:0/297741642 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47a0100d60 con 0x7f47a0104650 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.841+0000 7f47967fc700 1 -- 172.21.15.33:0/297741642 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4798003a00 con 0x7f47a0104650 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.841+0000 7f47967fc700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f478c0a11b0 0x7f478c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:53.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.841+0000 7f47a5271700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f478c0a11b0 0x7f478c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.841+0000 7f47967fc700 1 -- 172.21.15.33:0/297741642 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f47980cd1c0 con 0x7f47a0104650 2024-05-08T21:17:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.842+0000 7f47967fc700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799] conn(0x7f478c0abd30 0x7f478c0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.842+0000 7f47967fc700 1 -- 172.21.15.33:0/297741642 --> [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f478c0ae840 con 0x7f478c0abd30 2024-05-08T21:17:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.842+0000 7f47967fc700 1 -- 172.21.15.33:0/297741642 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47980cd5d0 con 0x7f47a0104650 2024-05-08T21:17:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.842+0000 7f47a4a70700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799] conn(0x7f478c0abd30 0x7f478c0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.842+0000 7f47a5271700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f478c0a11b0 0x7f478c0a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f479c005950 tx=0x7f479c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.842+0000 7f47a4a70700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799] conn(0x7f478c0abd30 0x7f478c0ae170 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:53.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.843+0000 7f47967fc700 1 -- 172.21.15.33:0/297741642 <== osd.3 v2:172.21.15.33:6810/2632370799 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f478c0ae840 con 0x7f478c0abd30 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.861+0000 7f47a74d5700 1 -- 172.21.15.33:0/297741642 --> [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f47a010ff80 con 0x7f478c0abd30 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.861+0000 7f47967fc700 1 -- 172.21.15.33:0/297741642 <== osd.3 v2:172.21.15.33:6810/2632370799 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f47a010ff80 con 0x7f478c0abd30 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.862+0000 7f47a74d5700 1 -- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799] conn(0x7f478c0abd30 msgr2=0x7f478c0ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.862+0000 7f47a74d5700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799] conn(0x7f478c0abd30 0x7f478c0ae170 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.862+0000 7f47a74d5700 1 -- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f478c0a11b0 msgr2=0x7f478c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.862+0000 7f47a74d5700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f478c0a11b0 0x7f478c0a3670 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f479c005950 tx=0x7f479c0058e0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.862+0000 7f47a74d5700 1 -- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a0104650 msgr2=0x7f47a019bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.862+0000 7f47a74d5700 1 --2- 172.21.15.33:0/297741642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a0104650 0x7f47a019bb10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4798007ee0 tx=0x7f479800d3b0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.862+0000 7f47a5a72700 1 -- 172.21.15.33:0/297741642 reap_dead start 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.862+0000 7f47a74d5700 1 -- 172.21.15.33:0/297741642 shutdown_connections 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.862+0000 7f47a74d5700 1 -- 172.21.15.33:0/297741642 >> 172.21.15.33:0/297741642 conn(0x7f47a0075260 msgr2=0x7f47a010d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.862+0000 7f47a74d5700 1 -- 172.21.15.33:0/297741642 shutdown_connections 2024-05-08T21:17:53.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:52.862+0000 7f47a74d5700 1 -- 172.21.15.33:0/297741642 wait complete. 2024-05-08T21:17:53.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:53 smithi033 ceph-mon[42528]: pgmap v253: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-05-08T21:17:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:53 smithi071 ceph-mon[47074]: pgmap v253: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-05-08T21:17:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:53 smithi082 ceph-mon[47512]: pgmap v253: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-05-08T21:17:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.944+0000 7f4b94cfd700 1 -- 172.21.15.33:0/2270316955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b90069210 msgr2=0x7f4b90069690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.944+0000 7f4b94cfd700 1 --2- 172.21.15.33:0/2270316955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b90069210 0x7f4b90069690 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4b840099c0 tx=0x7f4b84009cd0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.944+0000 7f4b94cfd700 1 -- 172.21.15.33:0/2270316955 shutdown_connections 2024-05-08T21:17:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.944+0000 7f4b94cfd700 1 --2- 172.21.15.33:0/2270316955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90069bd0 0x7f4b9010de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.944+0000 7f4b94cfd700 1 --2- 172.21.15.33:0/2270316955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b90106fe0 0x7f4b901073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.944+0000 7f4b94cfd700 1 --2- 172.21.15.33:0/2270316955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b90069210 0x7f4b90069690 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.944+0000 7f4b94cfd700 1 -- 172.21.15.33:0/2270316955 >> 172.21.15.33:0/2270316955 conn(0x7f4b90076e70 msgr2=0x7f4b90077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.944+0000 7f4b94cfd700 1 -- 172.21.15.33:0/2270316955 shutdown_connections 2024-05-08T21:17:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.944+0000 7f4b94cfd700 1 -- 172.21.15.33:0/2270316955 wait complete. 2024-05-08T21:17:53.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.945+0000 7f4b94cfd700 1 Processor -- start 2024-05-08T21:17:53.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.945+0000 7f4b94cfd700 1 -- start start 2024-05-08T21:17:53.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.945+0000 7f4b94cfd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b90069210 0x7f4b90197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:53.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.945+0000 7f4b94cfd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b90069bd0 0x7f4b90197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:53.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b94cfd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90106fe0 0x7f4b9019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:53.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b94cfd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9019c120 con 0x7f4b90069210 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b94cfd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9019c290 con 0x7f4b90106fe0 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b94cfd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9019c450 con 0x7f4b90069bd0 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b8e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b90069210 0x7f4b90197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b8e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b90069210 0x7f4b90197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38844/0 (socket says 172.21.15.33:38844) 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b8e59c700 1 -- 172.21.15.33:0/1576231145 learned_addr learned my addr 172.21.15.33:0/1576231145 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b8dd9b700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b90069bd0 0x7f4b90197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b8ed9d700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90106fe0 0x7f4b9019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b8e59c700 1 -- 172.21.15.33:0/1576231145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b90069bd0 msgr2=0x7f4b90197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b8e59c700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b90069bd0 0x7f4b90197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b8e59c700 1 -- 172.21.15.33:0/1576231145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90106fe0 msgr2=0x7f4b9019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b8e59c700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90106fe0 0x7f4b9019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b8e59c700 1 -- 172.21.15.33:0/1576231145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840095f0 con 0x7f4b90069210 2024-05-08T21:17:53.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b8e59c700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b90069210 0x7f4b90197220 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000d950 tx=0x7f4b8000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b7f7fe700 1 -- 172.21.15.33:0/1576231145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000f890 con 0x7f4b90069210 2024-05-08T21:17:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b7f7fe700 1 -- 172.21.15.33:0/1576231145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b8000fe80 con 0x7f4b90069210 2024-05-08T21:17:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b94cfd700 1 -- 172.21.15.33:0/1576231145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9019c730 con 0x7f4b90069210 2024-05-08T21:17:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.946+0000 7f4b7f7fe700 1 -- 172.21.15.33:0/1576231145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000e3b0 con 0x7f4b90069210 2024-05-08T21:17:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.947+0000 7f4b94cfd700 1 -- 172.21.15.33:0/1576231145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b901a3f90 con 0x7f4b90069210 2024-05-08T21:17:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.947+0000 7f4b94cfd700 1 -- 172.21.15.33:0/1576231145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b90102690 con 0x7f4b90069210 2024-05-08T21:17:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.949+0000 7f4b7f7fe700 1 -- 172.21.15.33:0/1576231145 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4b80010460 con 0x7f4b90069210 2024-05-08T21:17:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.949+0000 7f4b7f7fe700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4b780a11b0 0x7f4b780a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.950+0000 7f4b8dd9b700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4b780a11b0 0x7f4b780a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.950+0000 7f4b7f7fe700 1 -- 172.21.15.33:0/1576231145 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4b800ccf50 con 0x7f4b90069210 2024-05-08T21:17:53.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.950+0000 7f4b7f7fe700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719] conn(0x7f4b780abd30 0x7f4b780ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:53.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.950+0000 7f4b7f7fe700 1 -- 172.21.15.33:0/1576231145 --> [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b780ae840 con 0x7f4b780abd30 2024-05-08T21:17:53.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.950+0000 7f4b7f7fe700 1 -- 172.21.15.33:0/1576231145 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b800cd360 con 0x7f4b90069210 2024-05-08T21:17:53.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.950+0000 7f4b8dd9b700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4b780a11b0 0x7f4b780a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4b84000c00 tx=0x7f4b84005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:53.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.950+0000 7f4b8ed9d700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719] conn(0x7f4b780abd30 0x7f4b780ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:53.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.950+0000 7f4b8ed9d700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719] conn(0x7f4b780abd30 0x7f4b780ae170 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:53.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.951+0000 7f4b7f7fe700 1 -- 172.21.15.33:0/1576231145 <== osd.5 v2:172.21.15.82:6808/1310343719 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f4b780ae840 con 0x7f4b780abd30 2024-05-08T21:17:53.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.968+0000 7f4b94cfd700 1 -- 172.21.15.33:0/1576231145 --> [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b90110f90 con 0x7f4b780abd30 2024-05-08T21:17:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.969+0000 7f4b7f7fe700 1 -- 172.21.15.33:0/1576231145 <== osd.5 v2:172.21.15.82:6808/1310343719 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4b90110f90 con 0x7f4b780abd30 2024-05-08T21:17:53.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.969+0000 7f4b94cfd700 1 -- 172.21.15.33:0/1576231145 >> [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719] conn(0x7f4b780abd30 msgr2=0x7f4b780ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:53.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.969+0000 7f4b94cfd700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719] conn(0x7f4b780abd30 0x7f4b780ae170 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:53.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.969+0000 7f4b94cfd700 1 -- 172.21.15.33:0/1576231145 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4b780a11b0 msgr2=0x7f4b780a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:53.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.969+0000 7f4b94cfd700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4b780a11b0 0x7f4b780a3670 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4b84000c00 tx=0x7f4b84005a90 comp rx=0 tx=0).stop 2024-05-08T21:17:53.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.969+0000 7f4b94cfd700 1 -- 172.21.15.33:0/1576231145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b90069210 msgr2=0x7f4b90197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:53.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.969+0000 7f4b94cfd700 1 --2- 172.21.15.33:0/1576231145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b90069210 0x7f4b90197220 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000d950 tx=0x7f4b8000dc60 comp rx=0 tx=0).stop 2024-05-08T21:17:53.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.969+0000 7f4b8ed9d700 1 -- 172.21.15.33:0/1576231145 reap_dead start 2024-05-08T21:17:53.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.969+0000 7f4b94cfd700 1 -- 172.21.15.33:0/1576231145 shutdown_connections 2024-05-08T21:17:53.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.969+0000 7f4b94cfd700 1 -- 172.21.15.33:0/1576231145 >> 172.21.15.33:0/1576231145 conn(0x7f4b90076e70 msgr2=0x7f4b900fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:53.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.970+0000 7f4b94cfd700 1 -- 172.21.15.33:0/1576231145 shutdown_connections 2024-05-08T21:17:53.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:53.970+0000 7f4b94cfd700 1 -- 172.21.15.33:0/1576231145 wait complete. 2024-05-08T21:17:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.242+0000 7f4653688700 1 -- 172.21.15.33:0/1351015868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c0695e0 msgr2=0x7f464c069a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:54.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.242+0000 7f4653688700 1 --2- 172.21.15.33:0/1351015868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c0695e0 0x7f464c069a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f463c0099c0 tx=0x7f463c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:17:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.243+0000 7f4653688700 1 -- 172.21.15.33:0/1351015868 shutdown_connections 2024-05-08T21:17:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.243+0000 7f4653688700 1 --2- 172.21.15.33:0/1351015868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f464c069fa0 0x7f464c10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.243+0000 7f4653688700 1 --2- 172.21.15.33:0/1351015868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c107020 0x7f464c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.243+0000 7f4653688700 1 --2- 172.21.15.33:0/1351015868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c0695e0 0x7f464c069a60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.243+0000 7f4653688700 1 -- 172.21.15.33:0/1351015868 >> 172.21.15.33:0/1351015868 conn(0x7f464c076c50 msgr2=0x7f464c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.243+0000 7f4653688700 1 -- 172.21.15.33:0/1351015868 shutdown_connections 2024-05-08T21:17:54.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.243+0000 7f4653688700 1 -- 172.21.15.33:0/1351015868 wait complete. 2024-05-08T21:17:54.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.243+0000 7f4653688700 1 Processor -- start 2024-05-08T21:17:54.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.243+0000 7f4653688700 1 -- start start 2024-05-08T21:17:54.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4653688700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c0695e0 0x7f464c1002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:54.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4653688700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f464c069fa0 0x7f464c100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:54.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4653688700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c107020 0x7f464c100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:54.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4653688700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c1a56c0 con 0x7f464c0695e0 2024-05-08T21:17:54.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4653688700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c1a5830 con 0x7f464c107020 2024-05-08T21:17:54.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4653688700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c1a59f0 con 0x7f464c069fa0 2024-05-08T21:17:54.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4651424700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c0695e0 0x7f464c1002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:54.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4651424700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c0695e0 0x7f464c1002c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38862/0 (socket says 172.21.15.33:38862) 2024-05-08T21:17:54.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4651424700 1 -- 172.21.15.33:0/3692666506 learned_addr learned my addr 172.21.15.33:0/3692666506 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:54.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4651c25700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c107020 0x7f464c100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:54.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4650c23700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f464c069fa0 0x7f464c100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:54.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4651424700 1 -- 172.21.15.33:0/3692666506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f464c069fa0 msgr2=0x7f464c100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:54.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4651424700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f464c069fa0 0x7f464c100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:54.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4651424700 1 -- 172.21.15.33:0/3692666506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c107020 msgr2=0x7f464c100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:54.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4651424700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f464c107020 0x7f464c100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:54.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4651424700 1 -- 172.21.15.33:0/3692666506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c0095f0 con 0x7f464c0695e0 2024-05-08T21:17:54.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.244+0000 7f4651424700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c0695e0 0x7f464c1002c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f464800d980 tx=0x7f464800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:54.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.245+0000 7f46427fc700 1 -- 172.21.15.33:0/3692666506 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4648004220 con 0x7f464c0695e0 2024-05-08T21:17:54.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.245+0000 7f46427fc700 1 -- 172.21.15.33:0/3692666506 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4648007640 con 0x7f464c0695e0 2024-05-08T21:17:54.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.245+0000 7f4653688700 1 -- 172.21.15.33:0/3692666506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c1a5cd0 con 0x7f464c0695e0 2024-05-08T21:17:54.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.245+0000 7f46427fc700 1 -- 172.21.15.33:0/3692666506 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46480055e0 con 0x7f464c0695e0 2024-05-08T21:17:54.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.245+0000 7f4653688700 1 -- 172.21.15.33:0/3692666506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c1a6180 con 0x7f464c0695e0 2024-05-08T21:17:54.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.245+0000 7f4653688700 1 -- 172.21.15.33:0/3692666506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f464c068e90 con 0x7f464c0695e0 2024-05-08T21:17:54.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.247+0000 7f46427fc700 1 -- 172.21.15.33:0/3692666506 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f4648004750 con 0x7f464c0695e0 2024-05-08T21:17:54.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.248+0000 7f46427fc700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f46380a11b0 0x7f46380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:54.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.248+0000 7f4650c23700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f46380a11b0 0x7f46380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.248+0000 7f46427fc700 1 -- 172.21.15.33:0/3692666506 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f46480cd070 con 0x7f464c0695e0 2024-05-08T21:17:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.248+0000 7f46427fc700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386] conn(0x7f46380abd30 0x7f46380ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.248+0000 7f46427fc700 1 -- 172.21.15.33:0/3692666506 --> [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46380ae840 con 0x7f46380abd30 2024-05-08T21:17:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.248+0000 7f46427fc700 1 -- 172.21.15.33:0/3692666506 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46480cd480 con 0x7f464c0695e0 2024-05-08T21:17:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.248+0000 7f4650c23700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f46380a11b0 0x7f46380a3670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f464c1019a0 tx=0x7f463c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.248+0000 7f4651c25700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386] conn(0x7f46380abd30 0x7f46380ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.249+0000 7f4651c25700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386] conn(0x7f46380abd30 0x7f46380ae170 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:54.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.250+0000 7f46427fc700 1 -- 172.21.15.33:0/3692666506 <== osd.11 v2:172.21.15.82:6824/1998522386 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f46380ae840 con 0x7f46380abd30 2024-05-08T21:17:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.267+0000 7f4653688700 1 -- 172.21.15.33:0/3692666506 --> [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f464c111050 con 0x7f46380abd30 2024-05-08T21:17:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.267+0000 7f46427fc700 1 -- 172.21.15.33:0/3692666506 <== osd.11 v2:172.21.15.82:6824/1998522386 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f464c111050 con 0x7f46380abd30 2024-05-08T21:17:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.267+0000 7f4653688700 1 -- 172.21.15.33:0/3692666506 >> [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386] conn(0x7f46380abd30 msgr2=0x7f46380ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.267+0000 7f4653688700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386] conn(0x7f46380abd30 0x7f46380ae170 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.268+0000 7f4653688700 1 -- 172.21.15.33:0/3692666506 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f46380a11b0 msgr2=0x7f46380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.268+0000 7f4653688700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f46380a11b0 0x7f46380a3670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f464c1019a0 tx=0x7f463c005fb0 comp rx=0 tx=0).stop 2024-05-08T21:17:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.268+0000 7f4653688700 1 -- 172.21.15.33:0/3692666506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c0695e0 msgr2=0x7f464c1002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.268+0000 7f4653688700 1 --2- 172.21.15.33:0/3692666506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c0695e0 0x7f464c1002c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f464800d980 tx=0x7f464800dc90 comp rx=0 tx=0).stop 2024-05-08T21:17:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.268+0000 7f4651c25700 1 -- 172.21.15.33:0/3692666506 reap_dead start 2024-05-08T21:17:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.268+0000 7f4653688700 1 -- 172.21.15.33:0/3692666506 shutdown_connections 2024-05-08T21:17:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.268+0000 7f4653688700 1 -- 172.21.15.33:0/3692666506 >> 172.21.15.33:0/3692666506 conn(0x7f464c076c50 msgr2=0x7f464c0ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.268+0000 7f4653688700 1 -- 172.21.15.33:0/3692666506 shutdown_connections 2024-05-08T21:17:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:54.268+0000 7f4653688700 1 -- 172.21.15.33:0/3692666506 wait complete. 2024-05-08T21:17:54.360 INFO:teuthology.orchestra.run.smithi033.stdout:103079215153 2024-05-08T21:17:54.360 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-08T21:17:54.811 INFO:teuthology.orchestra.run.smithi033.stdout:296352743446 2024-05-08T21:17:54.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-08T21:17:55.019 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:55.428 INFO:teuthology.orchestra.run.smithi033.stdout:154618822698 2024-05-08T21:17:55.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-08T21:17:55.601 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:55 smithi033 ceph-mon[42528]: pgmap v254: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-08T21:17:55.792 INFO:teuthology.orchestra.run.smithi033.stdout:34359738426 2024-05-08T21:17:55.792 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-08T21:17:55.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:55 smithi071 ceph-mon[47074]: pgmap v254: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-08T21:17:55.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:55 smithi082 ceph-mon[47512]: pgmap v254: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-05-08T21:17:56.076 INFO:teuthology.orchestra.run.smithi033.stdout:326417514517 2024-05-08T21:17:56.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-08T21:17:56.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.217+0000 7fb8cf88c700 1 -- 172.21.15.33:0/3584830891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c8105f00 msgr2=0x7fb8c81062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:56.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.217+0000 7fb8cf88c700 1 --2- 172.21.15.33:0/3584830891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c8105f00 0x7fb8c81062e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009990 tx=0x7fb8c4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.217+0000 7fb8cf88c700 1 -- 172.21.15.33:0/3584830891 shutdown_connections 2024-05-08T21:17:56.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.217+0000 7fb8cf88c700 1 --2- 172.21.15.33:0/3584830891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8c8100890 0x7fb8c810cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.217+0000 7fb8cf88c700 1 --2- 172.21.15.33:0/3584830891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c8105f00 0x7fb8c81062e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.217+0000 7fb8cf88c700 1 --2- 172.21.15.33:0/3584830891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8c80ffed0 0x7fb8c8100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.217+0000 7fb8cf88c700 1 -- 172.21.15.33:0/3584830891 >> 172.21.15.33:0/3584830891 conn(0x7fb8c8075f80 msgr2=0x7fb8c8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:56.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.217+0000 7fb8cf88c700 1 -- 172.21.15.33:0/3584830891 shutdown_connections 2024-05-08T21:17:56.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.217+0000 7fb8cf88c700 1 -- 172.21.15.33:0/3584830891 wait complete. 2024-05-08T21:17:56.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.218+0000 7fb8cf88c700 1 Processor -- start 2024-05-08T21:17:56.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.218+0000 7fb8cf88c700 1 -- start start 2024-05-08T21:17:56.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cf88c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8c80ffed0 0x7fb8c81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:56.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cf88c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c8100890 0x7fb8c8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:56.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cf88c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8c8105f00 0x7fb8c81a3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cf88c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c8198fd0 con 0x7fb8c8100890 2024-05-08T21:17:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cf88c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c81a3790 con 0x7fb8c80ffed0 2024-05-08T21:17:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cf88c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c81a38d0 con 0x7fb8c8105f00 2024-05-08T21:17:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cce27700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c8100890 0x7fb8c8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cce27700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c8100890 0x7fb8c8198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38870/0 (socket says 172.21.15.33:38870) 2024-05-08T21:17:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cce27700 1 -- 172.21.15.33:0/2832698193 learned_addr learned my addr 172.21.15.33:0/2832698193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cd628700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8c80ffed0 0x7fb8c81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cde29700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8c8105f00 0x7fb8c81a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cce27700 1 -- 172.21.15.33:0/2832698193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8c8105f00 msgr2=0x7fb8c81a3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cce27700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8c8105f00 0x7fb8c81a3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cce27700 1 -- 172.21.15.33:0/2832698193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8c80ffed0 msgr2=0x7fb8c81981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:56.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cce27700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8c80ffed0 0x7fb8c81981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cce27700 1 -- 172.21.15.33:0/2832698193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c40095f0 con 0x7fb8c8100890 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8cce27700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c8100890 0x7fb8c8198710 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800d980 tx=0x7fb8b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.219+0000 7fb8be7fc700 1 -- 172.21.15.33:0/2832698193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8004220 con 0x7fb8c8100890 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.220+0000 7fb8be7fc700 1 -- 172.21.15.33:0/2832698193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8b8007640 con 0x7fb8c8100890 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.220+0000 7fb8cf88c700 1 -- 172.21.15.33:0/2832698193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c81a3b50 con 0x7fb8c8100890 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.220+0000 7fb8be7fc700 1 -- 172.21.15.33:0/2832698193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8005660 con 0x7fb8c8100890 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.220+0000 7fb8cf88c700 1 -- 172.21.15.33:0/2832698193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c81a4050 con 0x7fb8c8100890 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.221+0000 7fb8cf88c700 1 -- 172.21.15.33:0/2832698193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb8c810ff30 con 0x7fb8c8100890 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.222+0000 7fb8be7fc700 1 -- 172.21.15.33:0/2832698193 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb8b8004750 con 0x7fb8c8100890 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.222+0000 7fb8be7fc700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8b40a11b0 0x7fb8b40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.222+0000 7fb8be7fc700 1 -- 172.21.15.33:0/2832698193 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb8b80ccfe0 con 0x7fb8c8100890 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.223+0000 7fb8cd628700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8b40a11b0 0x7fb8b40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:56.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.223+0000 7fb8be7fc700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395] conn(0x7fb8b40abd30 0x7fb8b40ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.223+0000 7fb8be7fc700 1 -- 172.21.15.33:0/2832698193 --> [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8b40ae840 con 0x7fb8b40abd30 2024-05-08T21:17:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.223+0000 7fb8be7fc700 1 -- 172.21.15.33:0/2832698193 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8b80cd3f0 con 0x7fb8c8100890 2024-05-08T21:17:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.223+0000 7fb8cd628700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8b40a11b0 0x7fb8b40a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4006010 tx=0x7fb8c400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.223+0000 7fb8cde29700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395] conn(0x7fb8b40abd30 0x7fb8b40ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.223+0000 7fb8cde29700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395] conn(0x7fb8b40abd30 0x7fb8b40ae170 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.224+0000 7fb8be7fc700 1 -- 172.21.15.33:0/2832698193 <== osd.8 v2:172.21.15.82:6816/1451701395 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb8b40ae840 con 0x7fb8b40abd30 2024-05-08T21:17:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.242+0000 7fb8cf88c700 1 -- 172.21.15.33:0/2832698193 --> [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb8c8061620 con 0x7fb8b40abd30 2024-05-08T21:17:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.242+0000 7fb8be7fc700 1 -- 172.21.15.33:0/2832698193 <== osd.8 v2:172.21.15.82:6816/1451701395 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb8c8061620 con 0x7fb8b40abd30 2024-05-08T21:17:56.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.243+0000 7fb8cf88c700 1 -- 172.21.15.33:0/2832698193 >> [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395] conn(0x7fb8b40abd30 msgr2=0x7fb8b40ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:56.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.243+0000 7fb8cf88c700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395] conn(0x7fb8b40abd30 0x7fb8b40ae170 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.243+0000 7fb8cf88c700 1 -- 172.21.15.33:0/2832698193 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8b40a11b0 msgr2=0x7fb8b40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:56.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.243+0000 7fb8cf88c700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8b40a11b0 0x7fb8b40a3670 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4006010 tx=0x7fb8c400b260 comp rx=0 tx=0).stop 2024-05-08T21:17:56.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.243+0000 7fb8cf88c700 1 -- 172.21.15.33:0/2832698193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c8100890 msgr2=0x7fb8c8198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:56.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.243+0000 7fb8cf88c700 1 --2- 172.21.15.33:0/2832698193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8c8100890 0x7fb8c8198710 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800d980 tx=0x7fb8b800dc90 comp rx=0 tx=0).stop 2024-05-08T21:17:56.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.243+0000 7fb8cde29700 1 -- 172.21.15.33:0/2832698193 reap_dead start 2024-05-08T21:17:56.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.243+0000 7fb8cf88c700 1 -- 172.21.15.33:0/2832698193 shutdown_connections 2024-05-08T21:17:56.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.243+0000 7fb8cf88c700 1 -- 172.21.15.33:0/2832698193 >> 172.21.15.33:0/2832698193 conn(0x7fb8c8075f80 msgr2=0x7fb8c80fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:56.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.243+0000 7fb8cf88c700 1 -- 172.21.15.33:0/2832698193 shutdown_connections 2024-05-08T21:17:56.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.243+0000 7fb8cf88c700 1 -- 172.21.15.33:0/2832698193 wait complete. 2024-05-08T21:17:56.630 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-08T21:17:56.630 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-08T21:17:56.630 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-05-08T21:17:56.631 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-05-08T21:17:56.631 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-08T21:17:56.631 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-08T21:17:56.631 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-05-08T21:17:56.631 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-08T21:17:56.631 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-05-08T21:17:56.631 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:56 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-08T21:17:56.694 INFO:teuthology.orchestra.run.smithi033.stdout:133143986220 2024-05-08T21:17:56.695 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-08T21:17:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-08T21:17:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-08T21:17:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-05-08T21:17:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-05-08T21:17:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-08T21:17:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-08T21:17:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-05-08T21:17:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-08T21:17:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-05-08T21:17:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:56 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-08T21:17:56.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-08T21:17:56.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-05-08T21:17:56.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-05-08T21:17:56.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-05-08T21:17:56.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-05-08T21:17:56.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-05-08T21:17:56.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]: dispatch 2024-05-08T21:17:56.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-08T21:17:56.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-05-08T21:17:56.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:56 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-05-08T21:17:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.877+0000 7f70570a6700 1 -- 172.21.15.33:0/1430878854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7050103990 msgr2=0x7f7050103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.877+0000 7f70570a6700 1 --2- 172.21.15.33:0/1430878854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7050103990 0x7f7050103e40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7048009f90 tx=0x7f704800b330 comp rx=0 tx=0).stop 2024-05-08T21:17:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.877+0000 7f70570a6700 1 -- 172.21.15.33:0/1430878854 shutdown_connections 2024-05-08T21:17:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.877+0000 7f70570a6700 1 --2- 172.21.15.33:0/1430878854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7050103990 0x7f7050103e40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.877+0000 7f70570a6700 1 --2- 172.21.15.33:0/1430878854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7050105fa0 0x7f70500691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.877+0000 7f70570a6700 1 --2- 172.21.15.33:0/1430878854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7050069720 0x7f7050069ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.877+0000 7f70570a6700 1 -- 172.21.15.33:0/1430878854 >> 172.21.15.33:0/1430878854 conn(0x7f7050076e70 msgr2=0x7f7050077280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.877+0000 7f70570a6700 1 -- 172.21.15.33:0/1430878854 shutdown_connections 2024-05-08T21:17:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.877+0000 7f70570a6700 1 -- 172.21.15.33:0/1430878854 wait complete. 2024-05-08T21:17:56.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.878+0000 7f70570a6700 1 Processor -- start 2024-05-08T21:17:56.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.879+0000 7f70570a6700 1 -- start start 2024-05-08T21:17:56.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.879+0000 7f70570a6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7050069720 0x7f70501a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:56.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.879+0000 7f70570a6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7050103990 0x7f70501a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:56.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.879+0000 7f70570a6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7050105fa0 0x7f705019d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:56.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.879+0000 7f70570a6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705019de80 con 0x7f7050103990 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.879+0000 7f70570a6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705019dff0 con 0x7f7050069720 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.879+0000 7f70570a6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705019e1b0 con 0x7f7050105fa0 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.879+0000 7f7055643700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7050105fa0 0x7f705019d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.879+0000 7f7055643700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7050105fa0 0x7f705019d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:57210/0 (socket says 172.21.15.33:57210) 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.879+0000 7f7055643700 1 -- 172.21.15.33:0/2072532131 learned_addr learned my addr 172.21.15.33:0/2072532131 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.879+0000 7f704ffff700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7050103990 0x7f70501a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.880+0000 7f7054e42700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7050069720 0x7f70501a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.880+0000 7f7055643700 1 -- 172.21.15.33:0/2072532131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7050069720 msgr2=0x7f70501a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.880+0000 7f7055643700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7050069720 0x7f70501a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.880+0000 7f7055643700 1 -- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7050103990 msgr2=0x7f70501a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.880+0000 7f7055643700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7050103990 0x7f70501a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.880+0000 7f7055643700 1 -- 172.21.15.33:0/2072532131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70400095f0 con 0x7f7050105fa0 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.880+0000 7f7055643700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7050105fa0 0x7f705019d810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7048009790 tx=0x7f7048007910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:56.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.880+0000 7f704dffb700 1 -- 172.21.15.33:0/2072532131 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704800e040 con 0x7f7050105fa0 2024-05-08T21:17:56.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.880+0000 7f704dffb700 1 -- 172.21.15.33:0/2072532131 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7048004e90 con 0x7f7050105fa0 2024-05-08T21:17:56.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.880+0000 7f704dffb700 1 -- 172.21.15.33:0/2072532131 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704801b450 con 0x7f7050105fa0 2024-05-08T21:17:56.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.880+0000 7f70570a6700 1 -- 172.21.15.33:0/2072532131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7048009c40 con 0x7f7050105fa0 2024-05-08T21:17:56.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.881+0000 7f70570a6700 1 -- 172.21.15.33:0/2072532131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70501aa4f0 con 0x7f7050105fa0 2024-05-08T21:17:56.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.881+0000 7f70570a6700 1 -- 172.21.15.33:0/2072532131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f70501175d0 con 0x7f7050105fa0 2024-05-08T21:17:56.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.883+0000 7f704dffb700 1 -- 172.21.15.33:0/2072532131 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f704801d480 con 0x7f7050105fa0 2024-05-08T21:17:56.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.884+0000 7f704dffb700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f703c0a11b0 0x7f703c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:56.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.884+0000 7f704dffb700 1 -- 172.21.15.33:0/2072532131 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f70480cecb0 con 0x7f7050105fa0 2024-05-08T21:17:56.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.884+0000 7f704dffb700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180] conn(0x7f703c0ac1d0 0x7f703c0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:56.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.884+0000 7f704dffb700 1 -- 172.21.15.33:0/2072532131 --> [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f703c0aed40 con 0x7f703c0ac1d0 2024-05-08T21:17:56.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.884+0000 7f704dffb700 1 -- 172.21.15.33:0/2072532131 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70480cf020 con 0x7f7050105fa0 2024-05-08T21:17:56.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.884+0000 7f704ffff700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180] conn(0x7f703c0ac1d0 0x7f703c0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:56.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.885+0000 7f7054e42700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f703c0a11b0 0x7f703c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:56.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.885+0000 7f704ffff700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180] conn(0x7f703c0ac1d0 0x7f703c0ae670 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:56.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.886+0000 7f7054e42700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f703c0a11b0 0x7f703c0a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7038009bd0 tx=0x7f7038009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:56.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.886+0000 7f704dffb700 1 -- 172.21.15.33:0/2072532131 <== osd.9 v2:172.21.15.33:6826/4123781180 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f703c0aed40 con 0x7f703c0ac1d0 2024-05-08T21:17:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.905+0000 7f70570a6700 1 -- 172.21.15.33:0/2072532131 --> [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f70500ff850 con 0x7f703c0ac1d0 2024-05-08T21:17:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.906+0000 7f704dffb700 1 -- 172.21.15.33:0/2072532131 <== osd.9 v2:172.21.15.33:6826/4123781180 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f70500ff850 con 0x7f703c0ac1d0 2024-05-08T21:17:56.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.907+0000 7f70570a6700 1 -- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180] conn(0x7f703c0ac1d0 msgr2=0x7f703c0ae670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.907+0000 7f70570a6700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180] conn(0x7f703c0ac1d0 0x7f703c0ae670 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.909+0000 7f70570a6700 1 -- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f703c0a11b0 msgr2=0x7f703c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.909+0000 7f70570a6700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f703c0a11b0 0x7f703c0a3670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7038009bd0 tx=0x7f7038009260 comp rx=0 tx=0).stop 2024-05-08T21:17:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.909+0000 7f70570a6700 1 -- 172.21.15.33:0/2072532131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7050105fa0 msgr2=0x7f705019d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.909+0000 7f70570a6700 1 --2- 172.21.15.33:0/2072532131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7050105fa0 0x7f705019d810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7048009790 tx=0x7f7048007910 comp rx=0 tx=0).stop 2024-05-08T21:17:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.909+0000 7f7055643700 1 -- 172.21.15.33:0/2072532131 reap_dead start 2024-05-08T21:17:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.909+0000 7f70570a6700 1 -- 172.21.15.33:0/2072532131 shutdown_connections 2024-05-08T21:17:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.909+0000 7f70570a6700 1 -- 172.21.15.33:0/2072532131 >> 172.21.15.33:0/2072532131 conn(0x7f7050076e70 msgr2=0x7f70500fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.909+0000 7f70570a6700 1 -- 172.21.15.33:0/2072532131 shutdown_connections 2024-05-08T21:17:56.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:56.909+0000 7f70570a6700 1 -- 172.21.15.33:0/2072532131 wait complete. 2024-05-08T21:17:57.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:57 smithi033 ceph-mon[42528]: pgmap v255: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-08T21:17:57.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-08T21:17:57.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-08T21:17:57.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-05-08T21:17:57.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-05-08T21:17:57.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-08T21:17:57.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-08T21:17:57.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-05-08T21:17:57.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-08T21:17:57.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-05-08T21:17:57.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-08T21:17:57.606 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:57 smithi033 ceph-mon[42528]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T21:17:57.771 INFO:teuthology.orchestra.run.smithi033.stdout:274877906971 2024-05-08T21:17:57.771 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-08T21:17:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:57 smithi071 ceph-mon[47074]: pgmap v255: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-08T21:17:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-08T21:17:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-08T21:17:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-05-08T21:17:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-05-08T21:17:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-08T21:17:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-08T21:17:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-05-08T21:17:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-08T21:17:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-05-08T21:17:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-08T21:17:57.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:57 smithi071 ceph-mon[47074]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T21:17:57.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:57 smithi082 ceph-mon[47512]: pgmap v255: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-05-08T21:17:57.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-08T21:17:57.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-05-08T21:17:57.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-05-08T21:17:57.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-05-08T21:17:57.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-05-08T21:17:57.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-05-08T21:17:57.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8]}]': finished 2024-05-08T21:17:57.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-08T21:17:57.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-05-08T21:17:57.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-05-08T21:17:57.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:57 smithi082 ceph-mon[47512]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T21:17:58.011 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:58.012 INFO:teuthology.orchestra.run.smithi033.stdout:240518168608 2024-05-08T21:17:58.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-08T21:17:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.113+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/986707852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f8069270 msgr2=0x7fd1f810d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.113+0000 7fd1fd1c0700 1 --2- 172.21.15.33:0/986707852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f8069270 0x7fd1f810d540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009990 tx=0x7fd1ec009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.113+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/986707852 shutdown_connections 2024-05-08T21:17:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.113+0000 7fd1fd1c0700 1 --2- 172.21.15.33:0/986707852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f810da80 0x7fd1f810fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.113+0000 7fd1fd1c0700 1 --2- 172.21.15.33:0/986707852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f8068950 0x7fd1f8068d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.113+0000 7fd1fd1c0700 1 --2- 172.21.15.33:0/986707852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f8069270 0x7fd1f810d540 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.113+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/986707852 >> 172.21.15.33:0/986707852 conn(0x7fd1f8075f80 msgr2=0x7fd1f8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.113+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/986707852 shutdown_connections 2024-05-08T21:17:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.113+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/986707852 wait complete. 2024-05-08T21:17:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.114+0000 7fd1fd1c0700 1 Processor -- start 2024-05-08T21:17:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.114+0000 7fd1fd1c0700 1 -- start start 2024-05-08T21:17:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1fd1c0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f8068950 0x7fd1f8199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1fd1c0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8069270 0x7fd1f8199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1fd1c0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f810da80 0x7fd1f819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1fd1c0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f819e480 con 0x7fd1f8069270 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1fd1c0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f819e5f0 con 0x7fd1f8068950 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1fd1c0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f819e7b0 con 0x7fd1f810da80 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1f659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8069270 0x7fd1f8199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1f6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f8068950 0x7fd1f8199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1f6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f8068950 0x7fd1f8199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:45256/0 (socket says 172.21.15.33:45256) 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1f6d9d700 1 -- 172.21.15.33:0/4008870804 learned_addr learned my addr 172.21.15.33:0/4008870804 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1f759e700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f810da80 0x7fd1f819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1f6d9d700 1 -- 172.21.15.33:0/4008870804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f810da80 msgr2=0x7fd1f819de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1f6d9d700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f810da80 0x7fd1f819de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1f6d9d700 1 -- 172.21.15.33:0/4008870804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8069270 msgr2=0x7fd1f8199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:58.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1f6d9d700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8069270 0x7fd1f8199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1f6d9d700 1 -- 172.21.15.33:0/4008870804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0095f0 con 0x7fd1f8068950 2024-05-08T21:17:58.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.115+0000 7fd1f659c700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8069270 0x7fd1f8199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:17:58.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.116+0000 7fd1f6d9d700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f8068950 0x7fd1f8199580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800e9a0 tx=0x7fd1e800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:58.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.116+0000 7fd1dffff700 1 -- 172.21.15.33:0/4008870804 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8009790 con 0x7fd1f8068950 2024-05-08T21:17:58.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.116+0000 7fd1dffff700 1 -- 172.21.15.33:0/4008870804 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1e8010430 con 0x7fd1f8068950 2024-05-08T21:17:58.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.116+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/4008870804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f819ea90 con 0x7fd1f8068950 2024-05-08T21:17:58.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.116+0000 7fd1dffff700 1 -- 172.21.15.33:0/4008870804 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800cea0 con 0x7fd1f8068950 2024-05-08T21:17:58.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.116+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/4008870804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f81a62f0 con 0x7fd1f8068950 2024-05-08T21:17:58.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.117+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/4008870804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f8108ca0 con 0x7fd1f8068950 2024-05-08T21:17:58.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.119+0000 7fd1dffff700 1 -- 172.21.15.33:0/4008870804 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd1e80105a0 con 0x7fd1f8068950 2024-05-08T21:17:58.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.119+0000 7fd1dffff700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd1e00a11b0 0x7fd1e00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:58.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.120+0000 7fd1f659c700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd1e00a11b0 0x7fd1e00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:58.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.120+0000 7fd1dffff700 1 -- 172.21.15.33:0/4008870804 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd1e8014070 con 0x7fd1f8068950 2024-05-08T21:17:58.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.120+0000 7fd1f659c700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd1e00a11b0 0x7fd1e00a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd1f819aae0 tx=0x7fd1ec00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:58.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.123+0000 7fd1dffff700 1 -- 172.21.15.33:0/4008870804 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd1e8096470 con 0x7fd1f8068950 2024-05-08T21:17:58.146 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:58.219 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:17:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.254+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/330696338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c64068e90 msgr2=0x7f0c64069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.254+0000 7f0c6a7ea700 1 --2- 172.21.15.33:0/330696338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c64068e90 0x7f0c64069310 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009990 tx=0x7f0c58009ca0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.254+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/330696338 shutdown_connections 2024-05-08T21:17:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.254+0000 7f0c6a7ea700 1 --2- 172.21.15.33:0/330696338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c641039d0 0x7f0c64103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.254+0000 7f0c6a7ea700 1 --2- 172.21.15.33:0/330696338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c64105ec0 0x7f0c64068950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.254+0000 7f0c6a7ea700 1 --2- 172.21.15.33:0/330696338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c64068e90 0x7f0c64069310 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.254+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/330696338 >> 172.21.15.33:0/330696338 conn(0x7f0c64075f80 msgr2=0x7f0c64076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.254+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/330696338 shutdown_connections 2024-05-08T21:17:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.254+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/330696338 wait complete. 2024-05-08T21:17:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.255+0000 7f0c6a7ea700 1 Processor -- start 2024-05-08T21:17:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.255+0000 7f0c6a7ea700 1 -- start start 2024-05-08T21:17:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.255+0000 7f0c6a7ea700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c64068e90 0x7f0c64072610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.255+0000 7f0c6a7ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c641039d0 0x7f0c64072b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.255+0000 7f0c6a7ea700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c64105ec0 0x7f0c6406b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.255+0000 7f0c6a7ea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6406bff0 con 0x7f0c641039d0 2024-05-08T21:17:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.255+0000 7f0c6a7ea700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6406c160 con 0x7f0c64068e90 2024-05-08T21:17:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.255+0000 7f0c6a7ea700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6406c320 con 0x7f0c64105ec0 2024-05-08T21:17:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.256+0000 7f0c637fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c641039d0 0x7f0c64072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.256+0000 7f0c637fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c641039d0 0x7f0c64072b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38912/0 (socket says 172.21.15.33:38912) 2024-05-08T21:17:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.256+0000 7f0c637fe700 1 -- 172.21.15.33:0/1339333628 learned_addr learned my addr 172.21.15.33:0/1339333628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:17:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.256+0000 7f0c68d87700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c64105ec0 0x7f0c6406b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.256+0000 7f0c63fff700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c64068e90 0x7f0c64072610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.256+0000 7f0c637fe700 1 -- 172.21.15.33:0/1339333628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c64105ec0 msgr2=0x7f0c6406b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.256+0000 7f0c637fe700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c64105ec0 0x7f0c6406b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.256+0000 7f0c637fe700 1 -- 172.21.15.33:0/1339333628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c64068e90 msgr2=0x7f0c64072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.256+0000 7f0c637fe700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c64068e90 0x7f0c64072610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.256+0000 7f0c637fe700 1 -- 172.21.15.33:0/1339333628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c580095f0 con 0x7f0c641039d0 2024-05-08T21:17:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.256+0000 7f0c637fe700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c641039d0 0x7f0c64072b90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0c580098e0 tx=0x7f0c58003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.256+0000 7f0c617fa700 1 -- 172.21.15.33:0/1339333628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5801d070 con 0x7f0c641039d0 2024-05-08T21:17:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.257+0000 7f0c617fa700 1 -- 172.21.15.33:0/1339333628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c5800bcb0 con 0x7f0c641039d0 2024-05-08T21:17:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.257+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/1339333628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c640701b0 con 0x7f0c641039d0 2024-05-08T21:17:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.257+0000 7f0c617fa700 1 -- 172.21.15.33:0/1339333628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800fc90 con 0x7f0c641039d0 2024-05-08T21:17:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.257+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/1339333628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c64070650 con 0x7f0c641039d0 2024-05-08T21:17:58.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.257+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/1339333628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c641176c0 con 0x7f0c641039d0 2024-05-08T21:17:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.260+0000 7f0c617fa700 1 -- 172.21.15.33:0/1339333628 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0c5800f460 con 0x7f0c641039d0 2024-05-08T21:17:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.260+0000 7f0c617fa700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0c4c0a11b0 0x7f0c4c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.261+0000 7f0c617fa700 1 -- 172.21.15.33:0/1339333628 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0c580cdff0 con 0x7f0c641039d0 2024-05-08T21:17:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.261+0000 7f0c63fff700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0c4c0a11b0 0x7f0c4c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.261+0000 7f0c617fa700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909] conn(0x7f0c4c0ac0a0 0x7f0c4c0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:17:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.261+0000 7f0c617fa700 1 -- 172.21.15.33:0/1339333628 --> [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c4c0aec10 con 0x7f0c4c0ac0a0 2024-05-08T21:17:58.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.261+0000 7f0c617fa700 1 -- 172.21.15.33:0/1339333628 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c580ce3e0 con 0x7f0c641039d0 2024-05-08T21:17:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.261+0000 7f0c68d87700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909] conn(0x7f0c4c0ac0a0 0x7f0c4c0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:17:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.261+0000 7f0c63fff700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0c4c0a11b0 0x7f0c4c0a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c54006fd0 tx=0x7f0c54009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:58.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.262+0000 7f0c68d87700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909] conn(0x7f0c4c0ac0a0 0x7f0c4c0ae540 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:17:58.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.262+0000 7f0c617fa700 1 -- 172.21.15.33:0/1339333628 <== osd.7 v2:172.21.15.71:6816/1213981909 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f0c4c0aec10 con 0x7f0c4c0ac0a0 2024-05-08T21:17:58.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.279+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/1339333628 --> [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c64102300 con 0x7f0c4c0ac0a0 2024-05-08T21:17:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.280+0000 7f0c617fa700 1 -- 172.21.15.33:0/1339333628 <== osd.7 v2:172.21.15.71:6816/1213981909 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0c64102300 con 0x7f0c4c0ac0a0 2024-05-08T21:17:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.280+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/1339333628 >> [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909] conn(0x7f0c4c0ac0a0 msgr2=0x7f0c4c0ae540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.280+0000 7f0c6a7ea700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909] conn(0x7f0c4c0ac0a0 0x7f0c4c0ae540 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.280+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/1339333628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0c4c0a11b0 msgr2=0x7f0c4c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.280+0000 7f0c6a7ea700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0c4c0a11b0 0x7f0c4c0a3670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c54006fd0 tx=0x7f0c54009190 comp rx=0 tx=0).stop 2024-05-08T21:17:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.280+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/1339333628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c641039d0 msgr2=0x7f0c64072b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.280+0000 7f0c6a7ea700 1 --2- 172.21.15.33:0/1339333628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c641039d0 0x7f0c64072b90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0c580098e0 tx=0x7f0c58003920 comp rx=0 tx=0).stop 2024-05-08T21:17:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.280+0000 7f0c68d87700 1 -- 172.21.15.33:0/1339333628 reap_dead start 2024-05-08T21:17:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.280+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/1339333628 shutdown_connections 2024-05-08T21:17:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.280+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/1339333628 >> 172.21.15.33:0/1339333628 conn(0x7f0c64075f80 msgr2=0x7f0c641009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.281+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/1339333628 shutdown_connections 2024-05-08T21:17:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.281+0000 7f0c6a7ea700 1 -- 172.21.15.33:0/1339333628 wait complete. 2024-05-08T21:17:58.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.369+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/4008870804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd1f81004b0 con 0x7fd1f8068950 2024-05-08T21:17:58.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.370+0000 7fd1dffff700 1 -- 172.21.15.33:0/4008870804 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd1e8026020 con 0x7fd1f8068950 2024-05-08T21:17:58.373 INFO:teuthology.orchestra.run.smithi033.stdout:73014444085 2024-05-08T21:17:58.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.373+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/4008870804 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd1e00a11b0 msgr2=0x7fd1e00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:58.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.373+0000 7fd1fd1c0700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd1e00a11b0 0x7fd1e00a3670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd1f819aae0 tx=0x7fd1ec00b260 comp rx=0 tx=0).stop 2024-05-08T21:17:58.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.373+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/4008870804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f8068950 msgr2=0x7fd1f8199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:17:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.373+0000 7fd1fd1c0700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f8068950 0x7fd1f8199580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800e9a0 tx=0x7fd1e800ecb0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.373+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/4008870804 shutdown_connections 2024-05-08T21:17:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.373+0000 7fd1fd1c0700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f810da80 0x7fd1f819de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.373+0000 7fd1fd1c0700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd1e00a11b0 0x7fd1e00a3670 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.373+0000 7fd1fd1c0700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1f8068950 0x7fd1f8199580 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.373+0000 7fd1fd1c0700 1 --2- 172.21.15.33:0/4008870804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd1f8069270 0x7fd1f8199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:17:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.373+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/4008870804 >> 172.21.15.33:0/4008870804 conn(0x7fd1f8075f80 msgr2=0x7fd1f810f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:17:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.373+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/4008870804 shutdown_connections 2024-05-08T21:17:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:17:58.373+0000 7fd1fd1c0700 1 -- 172.21.15.33:0/4008870804 wait complete. 2024-05-08T21:17:58.702 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:58 smithi033 ceph-mon[42528]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T21:17:58.702 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:58 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4008870804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T21:17:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:58 smithi071 ceph-mon[47074]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T21:17:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:58 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/4008870804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T21:17:58.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:58 smithi082 ceph-mon[47512]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T21:17:58.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:58 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/4008870804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T21:17:58.935 INFO:teuthology.orchestra.run.smithi033.stdout:210453397539 2024-05-08T21:17:58.936 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-08T21:17:59.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:17:59 smithi033 ceph-mon[42528]: pgmap v258: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s 2024-05-08T21:17:59.463 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444083 got 73014444085 for osd.2 2024-05-08T21:17:59.463 DEBUG:teuthology.parallel:result is None 2024-05-08T21:17:59.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:17:59 smithi071 ceph-mon[47074]: pgmap v258: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s 2024-05-08T21:17:59.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:17:59 smithi082 ceph-mon[47512]: pgmap v258: 129 pgs: 3 peering, 126 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 8 op/s 2024-05-08T21:18:00.302 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:00.556 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.724+0000 7f93b18af700 1 -- 172.21.15.33:0/1210600512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93ac0ffba0 msgr2=0x7f93ac0fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.724+0000 7f93b18af700 1 --2- 172.21.15.33:0/1210600512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93ac0ffba0 0x7f93ac0fff80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f939c00b0d0 tx=0x7f939c00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.724+0000 7f93b18af700 1 -- 172.21.15.33:0/1210600512 shutdown_connections 2024-05-08T21:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.724+0000 7f93b18af700 1 --2- 172.21.15.33:0/1210600512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93ac1063c0 0x7f93ac10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.724+0000 7f93b18af700 1 --2- 172.21.15.33:0/1210600512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93ac0ffba0 0x7f93ac0fff80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.724+0000 7f93b18af700 1 --2- 172.21.15.33:0/1210600512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac100550 0x7f93ac105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.724+0000 7f93b18af700 1 -- 172.21.15.33:0/1210600512 >> 172.21.15.33:0/1210600512 conn(0x7f93ac075260 msgr2=0x7f93ac075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.725+0000 7f93b18af700 1 -- 172.21.15.33:0/1210600512 shutdown_connections 2024-05-08T21:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.725+0000 7f93b18af700 1 -- 172.21.15.33:0/1210600512 wait complete. 2024-05-08T21:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.725+0000 7f93b18af700 1 Processor -- start 2024-05-08T21:18:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.725+0000 7f93b18af700 1 -- start start 2024-05-08T21:18:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93b18af700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93ac0ffba0 0x7f93ac1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93b18af700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac100550 0x7f93ac197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93b18af700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93ac1063c0 0x7f93ac19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93b18af700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac19c2c0 con 0x7f93ac0ffba0 2024-05-08T21:18:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93b18af700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac19c430 con 0x7f93ac1063c0 2024-05-08T21:18:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93b18af700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac19c5f0 con 0x7f93ac100550 2024-05-08T21:18:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93aaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93ac0ffba0 0x7f93ac1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93aaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93ac0ffba0 0x7f93ac1973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44090/0 (socket says 172.21.15.33:44090) 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93aaffd700 1 -- 172.21.15.33:0/731771544 learned_addr learned my addr 172.21.15.33:0/731771544 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93aa7fc700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac100550 0x7f93ac197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93ab7fe700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93ac1063c0 0x7f93ac19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93aaffd700 1 -- 172.21.15.33:0/731771544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac100550 msgr2=0x7f93ac197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93aaffd700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac100550 0x7f93ac197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93aaffd700 1 -- 172.21.15.33:0/731771544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93ac1063c0 msgr2=0x7f93ac19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93aaffd700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93ac1063c0 0x7f93ac19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93aaffd700 1 -- 172.21.15.33:0/731771544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c009d00 con 0x7f93ac0ffba0 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93aa7fc700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac100550 0x7f93ac197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93ab7fe700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93ac1063c0 0x7f93ac19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.726+0000 7f93aaffd700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93ac0ffba0 0x7f93ac1973c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f939c004fd0 tx=0x7f939c007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.727+0000 7f93b08ad700 1 -- 172.21.15.33:0/731771544 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c00e040 con 0x7f93ac0ffba0 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.727+0000 7f93b08ad700 1 -- 172.21.15.33:0/731771544 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f939c00be50 con 0x7f93ac0ffba0 2024-05-08T21:18:00.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.727+0000 7f93b18af700 1 -- 172.21.15.33:0/731771544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ac19c870 con 0x7f93ac0ffba0 2024-05-08T21:18:00.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.727+0000 7f93b08ad700 1 -- 172.21.15.33:0/731771544 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c0052a0 con 0x7f93ac0ffba0 2024-05-08T21:18:00.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.727+0000 7f93b18af700 1 -- 172.21.15.33:0/731771544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ac1a4130 con 0x7f93ac0ffba0 2024-05-08T21:18:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.728+0000 7f93b18af700 1 -- 172.21.15.33:0/731771544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ac1095a0 con 0x7f93ac0ffba0 2024-05-08T21:18:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.729+0000 7f93b08ad700 1 -- 172.21.15.33:0/731771544 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f939c01b450 con 0x7f93ac0ffba0 2024-05-08T21:18:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.729+0000 7f93b08ad700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f93980a11b0 0x7f93980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.730+0000 7f93aa7fc700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f93980a11b0 0x7f93980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.730+0000 7f93b08ad700 1 -- 172.21.15.33:0/731771544 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f939c0ce300 con 0x7f93ac0ffba0 2024-05-08T21:18:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.730+0000 7f93aa7fc700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f93980a11b0 0x7f93980a3670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f93ac198920 tx=0x7f9394009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:00.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.735+0000 7f93b08ad700 1 -- 172.21.15.33:0/731771544 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f939c0976b0 con 0x7f93ac0ffba0 2024-05-08T21:18:00.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.948+0000 7faf99b7f700 1 -- 172.21.15.33:0/3451449184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf94069b60 msgr2=0x7faf94069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:00.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.948+0000 7faf99b7f700 1 --2- 172.21.15.33:0/3451449184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf94069b60 0x7faf94069fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf880098f0 tx=0x7faf88009c00 comp rx=0 tx=0).stop 2024-05-08T21:18:00.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.948+0000 7faf99b7f700 1 -- 172.21.15.33:0/3451449184 shutdown_connections 2024-05-08T21:18:00.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.948+0000 7faf99b7f700 1 --2- 172.21.15.33:0/3451449184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf9410d860 0x7faf9410fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.948+0000 7faf99b7f700 1 --2- 172.21.15.33:0/3451449184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94101ee0 0x7faf94069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.948+0000 7faf99b7f700 1 --2- 172.21.15.33:0/3451449184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf94069b60 0x7faf94069fe0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.948+0000 7faf99b7f700 1 -- 172.21.15.33:0/3451449184 >> 172.21.15.33:0/3451449184 conn(0x7faf94076e50 msgr2=0x7faf94077260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:00.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.948+0000 7faf99b7f700 1 -- 172.21.15.33:0/3451449184 shutdown_connections 2024-05-08T21:18:00.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.948+0000 7faf99b7f700 1 -- 172.21.15.33:0/3451449184 wait complete. 2024-05-08T21:18:00.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf99b7f700 1 Processor -- start 2024-05-08T21:18:00.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf99b7f700 1 -- start start 2024-05-08T21:18:00.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf99b7f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf94069b60 0x7faf941995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:00.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf99b7f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94101ee0 0x7faf94199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:00.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf99b7f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf9410d860 0x7faf9419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:00.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf99b7f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9419e4a0 con 0x7faf94101ee0 2024-05-08T21:18:00.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf99b7f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9419e610 con 0x7faf94069b60 2024-05-08T21:18:00.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf99b7f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9419e7d0 con 0x7faf9410d860 2024-05-08T21:18:00.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf92ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94101ee0 0x7faf94199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:00.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf92ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94101ee0 0x7faf94199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44106/0 (socket says 172.21.15.33:44106) 2024-05-08T21:18:00.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf937fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf94069b60 0x7faf941995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:00.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf92ffd700 1 -- 172.21.15.33:0/2940816517 learned_addr learned my addr 172.21.15.33:0/2940816517 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:00.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.949+0000 7faf93fff700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf9410d860 0x7faf9419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:00.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.950+0000 7faf937fe700 1 -- 172.21.15.33:0/2940816517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf9410d860 msgr2=0x7faf9419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:00.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.950+0000 7faf937fe700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf9410d860 0x7faf9419de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.950+0000 7faf937fe700 1 -- 172.21.15.33:0/2940816517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94101ee0 msgr2=0x7faf94199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:00.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.950+0000 7faf937fe700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94101ee0 0x7faf94199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.950+0000 7faf937fe700 1 -- 172.21.15.33:0/2940816517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c009c40 con 0x7faf94069b60 2024-05-08T21:18:00.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.950+0000 7faf92ffd700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94101ee0 0x7faf94199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:00.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.950+0000 7faf937fe700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf94069b60 0x7faf941995f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf8400e9c0 tx=0x7faf8400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:00.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.950+0000 7faf90ff9700 1 -- 172.21.15.33:0/2940816517 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8400c9c0 con 0x7faf94069b60 2024-05-08T21:18:00.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.950+0000 7faf99b7f700 1 -- 172.21.15.33:0/2940816517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf88009520 con 0x7faf94069b60 2024-05-08T21:18:00.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.950+0000 7faf90ff9700 1 -- 172.21.15.33:0/2940816517 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf84010430 con 0x7faf94069b60 2024-05-08T21:18:00.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.950+0000 7faf90ff9700 1 -- 172.21.15.33:0/2940816517 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84018680 con 0x7faf94069b60 2024-05-08T21:18:00.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.950+0000 7faf99b7f700 1 -- 172.21.15.33:0/2940816517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf941a6140 con 0x7faf94069b60 2024-05-08T21:18:00.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.951+0000 7faf99b7f700 1 -- 172.21.15.33:0/2940816517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf941004e0 con 0x7faf94069b60 2024-05-08T21:18:00.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.953+0000 7faf90ff9700 1 -- 172.21.15.33:0/2940816517 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faf840105a0 con 0x7faf94069b60 2024-05-08T21:18:00.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.953+0000 7faf90ff9700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faf7c0a11b0 0x7faf7c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:00.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.954+0000 7faf90ff9700 1 -- 172.21.15.33:0/2940816517 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faf84014070 con 0x7faf94069b60 2024-05-08T21:18:00.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.954+0000 7faf92ffd700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faf7c0a11b0 0x7faf7c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:00.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.954+0000 7faf92ffd700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faf7c0a11b0 0x7faf7c0a3670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faf9419ab00 tx=0x7faf88019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:00.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.961+0000 7faf90ff9700 1 -- 172.21.15.33:0/2940816517 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faf84096310 con 0x7faf94069b60 2024-05-08T21:18:00.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.969+0000 7f93b18af700 1 -- 172.21.15.33:0/731771544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f93ac1986d0 con 0x7f93ac0ffba0 2024-05-08T21:18:00.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.969+0000 7f93b08ad700 1 -- 172.21.15.33:0/731771544 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f939c09acd0 con 0x7f93ac0ffba0 2024-05-08T21:18:00.972 INFO:teuthology.orchestra.run.smithi033.stdout:51539607609 2024-05-08T21:18:00.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.972+0000 7f93b18af700 1 -- 172.21.15.33:0/731771544 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f93980a11b0 msgr2=0x7f93980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:00.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.972+0000 7f93b18af700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f93980a11b0 0x7f93980a3670 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f93ac198920 tx=0x7f9394009260 comp rx=0 tx=0).stop 2024-05-08T21:18:00.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.972+0000 7f93b18af700 1 -- 172.21.15.33:0/731771544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93ac0ffba0 msgr2=0x7f93ac1973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:00.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.972+0000 7f93b18af700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93ac0ffba0 0x7f93ac1973c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f939c004fd0 tx=0x7f939c007ad0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.976+0000 7f93b18af700 1 -- 172.21.15.33:0/731771544 shutdown_connections 2024-05-08T21:18:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.976+0000 7f93b18af700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f93980a11b0 0x7f93980a3670 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.976+0000 7f93b18af700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93ac1063c0 0x7f93ac19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.976+0000 7f93b18af700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93ac0ffba0 0x7f93ac1973c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.976+0000 7f93b18af700 1 --2- 172.21.15.33:0/731771544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac100550 0x7f93ac197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.976+0000 7f93b18af700 1 -- 172.21.15.33:0/731771544 >> 172.21.15.33:0/731771544 conn(0x7f93ac075260 msgr2=0x7f93ac105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.976+0000 7f93b18af700 1 -- 172.21.15.33:0/731771544 shutdown_connections 2024-05-08T21:18:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:00.976+0000 7f93b18af700 1 -- 172.21.15.33:0/731771544 wait complete. 2024-05-08T21:18:01.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.190+0000 7faf99b7f700 1 -- 172.21.15.33:0/2940816517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7faf9419a950 con 0x7faf94069b60 2024-05-08T21:18:01.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.190+0000 7faf90ff9700 1 -- 172.21.15.33:0/2940816517 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf84026090 con 0x7faf94069b60 2024-05-08T21:18:01.193 INFO:teuthology.orchestra.run.smithi033.stdout:188978561064 2024-05-08T21:18:01.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.193+0000 7faf99b7f700 1 -- 172.21.15.33:0/2940816517 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faf7c0a11b0 msgr2=0x7faf7c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:01.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.193+0000 7faf99b7f700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faf7c0a11b0 0x7faf7c0a3670 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faf9419ab00 tx=0x7faf88019040 comp rx=0 tx=0).stop 2024-05-08T21:18:01.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.193+0000 7faf99b7f700 1 -- 172.21.15.33:0/2940816517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf94069b60 msgr2=0x7faf941995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:01.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.193+0000 7faf99b7f700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf94069b60 0x7faf941995f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf8400e9c0 tx=0x7faf8400ecd0 comp rx=0 tx=0).stop 2024-05-08T21:18:01.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.193+0000 7faf99b7f700 1 -- 172.21.15.33:0/2940816517 shutdown_connections 2024-05-08T21:18:01.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.193+0000 7faf99b7f700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf9410d860 0x7faf9419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:01.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.193+0000 7faf99b7f700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faf7c0a11b0 0x7faf7c0a3670 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:01.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.193+0000 7faf99b7f700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf94069b60 0x7faf941995f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:01.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.193+0000 7faf99b7f700 1 --2- 172.21.15.33:0/2940816517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94101ee0 0x7faf94199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:01.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.193+0000 7faf99b7f700 1 -- 172.21.15.33:0/2940816517 >> 172.21.15.33:0/2940816517 conn(0x7faf94076e50 msgr2=0x7faf940fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:01.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.193+0000 7faf99b7f700 1 -- 172.21.15.33:0/2940816517 shutdown_connections 2024-05-08T21:18:01.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:01.193+0000 7faf99b7f700 1 -- 172.21.15.33:0/2940816517 wait complete. 2024-05-08T21:18:01.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:01 smithi033 ceph-mon[42528]: pgmap v259: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 83 B/s, 0 objects/s recovering 2024-05-08T21:18:01.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:01 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:18:01.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:01 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/731771544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T21:18:01.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:01 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2940816517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T21:18:01.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:01 smithi071 ceph-mon[47074]: pgmap v259: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 83 B/s, 0 objects/s recovering 2024-05-08T21:18:01.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:01 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:18:01.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:01 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/731771544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T21:18:01.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:01 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2940816517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T21:18:01.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:01 smithi082 ceph-mon[47512]: pgmap v259: 129 pgs: 129 active+clean; 642 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 83 B/s, 0 objects/s recovering 2024-05-08T21:18:01.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:01 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:18:01.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:01 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/731771544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T21:18:01.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:01 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2940816517' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T21:18:01.925 INFO:tasks.cephadm.ceph_manager.ceph:need seq 51539607607 got 51539607609 for osd.1 2024-05-08T21:18:01.925 DEBUG:teuthology.parallel:result is None 2024-05-08T21:18:02.094 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561061 got 188978561064 for osd.6 2024-05-08T21:18:02.094 DEBUG:teuthology.parallel:result is None 2024-05-08T21:18:02.115 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:02.685 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:02.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:02 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:02.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:02 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:02.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:02 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:02.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:02 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:02.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:02 smithi033 ceph-mon[42528]: pgmap v260: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 383 B/s wr, 9 op/s; 155 B/s, 1 objects/s recovering 2024-05-08T21:18:03.022 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:03.174 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:03.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:02 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:03.383 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:02 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:03.383 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:02 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:03.383 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:02 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:03.383 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:02 smithi071 ceph-mon[47074]: pgmap v260: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 383 B/s wr, 9 op/s; 155 B/s, 1 objects/s recovering 2024-05-08T21:18:03.384 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:02 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:03.384 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:02 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:03.384 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:02 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:03.385 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:02 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:03.385 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:02 smithi082 ceph-mon[47512]: pgmap v260: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 383 B/s wr, 9 op/s; 155 B/s, 1 objects/s recovering 2024-05-08T21:18:03.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.377+0000 7f5ffe14e700 1 -- 172.21.15.33:0/1013792873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff8068950 msgr2=0x7f5ff8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:03.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.377+0000 7f5ffe14e700 1 --2- 172.21.15.33:0/1013792873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff8068950 0x7f5ff8068db0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009990 tx=0x7f5fe0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:03.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.377+0000 7f5ffe14e700 1 -- 172.21.15.33:0/1013792873 shutdown_connections 2024-05-08T21:18:03.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.377+0000 7f5ffe14e700 1 --2- 172.21.15.33:0/1013792873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff80692f0 0x7f5ff810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:03.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.377+0000 7f5ffe14e700 1 --2- 172.21.15.33:0/1013792873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff8107020 0x7f5ff8107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:03.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.377+0000 7f5ffe14e700 1 --2- 172.21.15.33:0/1013792873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff8068950 0x7f5ff8068db0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:03.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.377+0000 7f5ffe14e700 1 -- 172.21.15.33:0/1013792873 >> 172.21.15.33:0/1013792873 conn(0x7f5ff8075f80 msgr2=0x7f5ff8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:03.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.377+0000 7f5ffe14e700 1 -- 172.21.15.33:0/1013792873 shutdown_connections 2024-05-08T21:18:03.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.377+0000 7f5ffe14e700 1 -- 172.21.15.33:0/1013792873 wait complete. 2024-05-08T21:18:03.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.378+0000 7f5ffe14e700 1 Processor -- start 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ffe14e700 1 -- start start 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ffe14e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff8068950 0x7f5ff81972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ffe14e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80692f0 0x7f5ff8197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ffe14e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff8107020 0x7f5ff819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ffe14e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff819c1c0 con 0x7f5ff80692f0 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ffe14e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff819c330 con 0x7f5ff8068950 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ffe14e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff819c4f0 con 0x7f5ff8107020 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ff6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80692f0 0x7f5ff8197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ff6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80692f0 0x7f5ff8197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44118/0 (socket says 172.21.15.33:44118) 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ff6ffd700 1 -- 172.21.15.33:0/2427805130 learned_addr learned my addr 172.21.15.33:0/2427805130 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ff77fe700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff8068950 0x7f5ff81972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ff7fff700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff8107020 0x7f5ff819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:03.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ff6ffd700 1 -- 172.21.15.33:0/2427805130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff8107020 msgr2=0x7f5ff819bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ff6ffd700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff8107020 0x7f5ff819bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ff6ffd700 1 -- 172.21.15.33:0/2427805130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff8068950 msgr2=0x7f5ff81972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ff6ffd700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff8068950 0x7f5ff81972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ff6ffd700 1 -- 172.21.15.33:0/2427805130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe00095f0 con 0x7f5ff80692f0 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.379+0000 7f5ff7fff700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff8107020 0x7f5ff819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.380+0000 7f5ff6ffd700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80692f0 0x7f5ff8197800 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0004760 tx=0x7f5fe0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.380+0000 7f5ff4ff9700 1 -- 172.21.15.33:0/2427805130 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe001d070 con 0x7f5ff80692f0 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.380+0000 7f5ffe14e700 1 -- 172.21.15.33:0/2427805130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff819c770 con 0x7f5ff80692f0 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.380+0000 7f5ff4ff9700 1 -- 172.21.15.33:0/2427805130 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fe0005a00 con 0x7f5ff80692f0 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.380+0000 7f5ff4ff9700 1 -- 172.21.15.33:0/2427805130 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0005280 con 0x7f5ff80692f0 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.380+0000 7f5ffe14e700 1 -- 172.21.15.33:0/2427805130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff81a3e60 con 0x7f5ff80692f0 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.381+0000 7f5ffe14e700 1 -- 172.21.15.33:0/2427805130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff8109540 con 0x7f5ff80692f0 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.382+0000 7f5ff4ff9700 1 -- 172.21.15.33:0/2427805130 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5fe00053e0 con 0x7f5ff80692f0 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.383+0000 7f5ff4ff9700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5fe40a11b0 0x7f5fe40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.383+0000 7f5ff77fe700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5fe40a11b0 0x7f5fe40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.383+0000 7f5ff4ff9700 1 -- 172.21.15.33:0/2427805130 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5fe00cdd90 con 0x7f5ff80692f0 2024-05-08T21:18:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.384+0000 7f5ff77fe700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5fe40a11b0 0x7f5fe40a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009550 tx=0x7f5fe80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:03.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.388+0000 7f5ff4ff9700 1 -- 172.21.15.33:0/2427805130 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5fe005bd60 con 0x7f5ff80692f0 2024-05-08T21:18:03.543 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:03.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.625+0000 7f5ffe14e700 1 -- 172.21.15.33:0/2427805130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5ff81026d0 con 0x7f5ff80692f0 2024-05-08T21:18:03.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.625+0000 7f5ff4ff9700 1 -- 172.21.15.33:0/2427805130 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5fe00d1030 con 0x7f5ff80692f0 2024-05-08T21:18:03.629 INFO:teuthology.orchestra.run.smithi033.stdout:154618822700 2024-05-08T21:18:03.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.628+0000 7f5ffe14e700 1 -- 172.21.15.33:0/2427805130 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5fe40a11b0 msgr2=0x7f5fe40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:03.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.628+0000 7f5ffe14e700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5fe40a11b0 0x7f5fe40a3670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009550 tx=0x7f5fe80091b0 comp rx=0 tx=0).stop 2024-05-08T21:18:03.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.628+0000 7f5ffe14e700 1 -- 172.21.15.33:0/2427805130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80692f0 msgr2=0x7f5ff8197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:03.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.628+0000 7f5ffe14e700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80692f0 0x7f5ff8197800 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0004760 tx=0x7f5fe0004790 comp rx=0 tx=0).stop 2024-05-08T21:18:03.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.628+0000 7f5ffe14e700 1 -- 172.21.15.33:0/2427805130 shutdown_connections 2024-05-08T21:18:03.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.628+0000 7f5ffe14e700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff8107020 0x7f5ff819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:03.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.628+0000 7f5ffe14e700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5fe40a11b0 0x7f5fe40a3670 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:03.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.628+0000 7f5ffe14e700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ff8068950 0x7f5ff81972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:03.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.628+0000 7f5ffe14e700 1 --2- 172.21.15.33:0/2427805130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff80692f0 0x7f5ff8197800 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:03.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.628+0000 7f5ffe14e700 1 -- 172.21.15.33:0/2427805130 >> 172.21.15.33:0/2427805130 conn(0x7f5ff8075f80 msgr2=0x7f5ff80fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:03.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.628+0000 7f5ffe14e700 1 -- 172.21.15.33:0/2427805130 shutdown_connections 2024-05-08T21:18:03.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:03.628+0000 7f5ffe14e700 1 -- 172.21.15.33:0/2427805130 wait complete. 2024-05-08T21:18:03.748 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:04.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.055+0000 7ff5d22c9700 1 -- 172.21.15.33:0/1086876797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5cc0ffed0 msgr2=0x7ff5cc100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:04.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.055+0000 7ff5d22c9700 1 --2- 172.21.15.33:0/1086876797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5cc0ffed0 0x7ff5cc100350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009990 tx=0x7ff5c0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:04.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.055+0000 7ff5d22c9700 1 -- 172.21.15.33:0/1086876797 shutdown_connections 2024-05-08T21:18:04.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.055+0000 7ff5d22c9700 1 --2- 172.21.15.33:0/1086876797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc100890 0x7ff5cc10cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:04.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.055+0000 7ff5d22c9700 1 --2- 172.21.15.33:0/1086876797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc105f00 0x7ff5cc1062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:04.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.055+0000 7ff5d22c9700 1 --2- 172.21.15.33:0/1086876797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5cc0ffed0 0x7ff5cc100350 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:04.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.055+0000 7ff5d22c9700 1 -- 172.21.15.33:0/1086876797 >> 172.21.15.33:0/1086876797 conn(0x7ff5cc075f80 msgr2=0x7ff5cc076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:04.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.055+0000 7ff5d22c9700 1 -- 172.21.15.33:0/1086876797 shutdown_connections 2024-05-08T21:18:04.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.055+0000 7ff5d22c9700 1 -- 172.21.15.33:0/1086876797 wait complete. 2024-05-08T21:18:04.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.056+0000 7ff5d22c9700 1 Processor -- start 2024-05-08T21:18:04.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.056+0000 7ff5d22c9700 1 -- start start 2024-05-08T21:18:04.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.056+0000 7ff5d22c9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc0ffed0 0x7ff5cc196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:04.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.056+0000 7ff5d22c9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5cc100890 0x7ff5cc196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:04.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.056+0000 7ff5d22c9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc105f00 0x7ff5cc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:04.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.056+0000 7ff5d22c9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc19b110 con 0x7ff5cc100890 2024-05-08T21:18:04.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.056+0000 7ff5d22c9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc19b280 con 0x7ff5cc105f00 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.056+0000 7ff5d22c9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc19b440 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.057+0000 7ff5cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5cc100890 0x7ff5cc196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.057+0000 7ff5cbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc0ffed0 0x7ff5cc196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.057+0000 7ff5cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5cc100890 0x7ff5cc196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44128/0 (socket says 172.21.15.33:44128) 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.057+0000 7ff5cb7fe700 1 -- 172.21.15.33:0/185404572 learned_addr learned my addr 172.21.15.33:0/185404572 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.057+0000 7ff5d0866700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc105f00 0x7ff5cc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.057+0000 7ff5cbfff700 1 -- 172.21.15.33:0/185404572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc105f00 msgr2=0x7ff5cc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.057+0000 7ff5cbfff700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc105f00 0x7ff5cc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.057+0000 7ff5cbfff700 1 -- 172.21.15.33:0/185404572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5cc100890 msgr2=0x7ff5cc196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.057+0000 7ff5cbfff700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5cc100890 0x7ff5cc196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.057+0000 7ff5cbfff700 1 -- 172.21.15.33:0/185404572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c00095f0 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.057+0000 7ff5cb7fe700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5cc100890 0x7ff5cc196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:04.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.057+0000 7ff5cbfff700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc0ffed0 0x7ff5cc196210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00e9f0 tx=0x7ff5bc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:04.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.058+0000 7ff5c97fa700 1 -- 172.21.15.33:0/185404572 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc00c9c0 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.058+0000 7ff5d22c9700 1 -- 172.21.15.33:0/185404572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5cc19b720 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.058+0000 7ff5c97fa700 1 -- 172.21.15.33:0/185404572 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5bc010430 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.058+0000 7ff5c97fa700 1 -- 172.21.15.33:0/185404572 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc018680 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.058+0000 7ff5d22c9700 1 -- 172.21.15.33:0/185404572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5cc068b20 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.059+0000 7ff5d22c9700 1 -- 172.21.15.33:0/185404572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5cc1015b0 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.060+0000 7ff5c97fa700 1 -- 172.21.15.33:0/185404572 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff5bc0105a0 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.060+0000 7ff5c97fa700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff5b40a11b0 0x7ff5b40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:04.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.061+0000 7ff5cb7fe700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff5b40a11b0 0x7ff5b40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:04.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.061+0000 7ff5c97fa700 1 -- 172.21.15.33:0/185404572 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff5bc014070 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.061+0000 7ff5cb7fe700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff5b40a11b0 0x7ff5b40a3670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc197770 tx=0x7ff5c001c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.064+0000 7ff5c97fa700 1 -- 172.21.15.33:0/185404572 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff5bc096250 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:03 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2427805130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T21:18:04.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.294+0000 7ff5d22c9700 1 -- 172.21.15.33:0/185404572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff5cc197580 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.295+0000 7ff5c97fa700 1 -- 172.21.15.33:0/185404572 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff5bc099870 con 0x7ff5cc0ffed0 2024-05-08T21:18:04.298 INFO:teuthology.orchestra.run.smithi033.stdout:34359738428 2024-05-08T21:18:04.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.297+0000 7ff5d22c9700 1 -- 172.21.15.33:0/185404572 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff5b40a11b0 msgr2=0x7ff5b40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:04.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.297+0000 7ff5d22c9700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff5b40a11b0 0x7ff5b40a3670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc197770 tx=0x7ff5c001c040 comp rx=0 tx=0).stop 2024-05-08T21:18:04.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.297+0000 7ff5d22c9700 1 -- 172.21.15.33:0/185404572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc0ffed0 msgr2=0x7ff5cc196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:04.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.297+0000 7ff5d22c9700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc0ffed0 0x7ff5cc196210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00e9f0 tx=0x7ff5bc00ed00 comp rx=0 tx=0).stop 2024-05-08T21:18:04.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.297+0000 7ff5d22c9700 1 -- 172.21.15.33:0/185404572 shutdown_connections 2024-05-08T21:18:04.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.297+0000 7ff5d22c9700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff5b40a11b0 0x7ff5b40a3670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:04.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.297+0000 7ff5d22c9700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5cc105f00 0x7ff5cc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:04.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.297+0000 7ff5d22c9700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc0ffed0 0x7ff5cc196210 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:04.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.297+0000 7ff5d22c9700 1 --2- 172.21.15.33:0/185404572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5cc100890 0x7ff5cc196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:04.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.297+0000 7ff5d22c9700 1 -- 172.21.15.33:0/185404572 >> 172.21.15.33:0/185404572 conn(0x7ff5cc075f80 msgr2=0x7ff5cc0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:04.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.297+0000 7ff5d22c9700 1 -- 172.21.15.33:0/185404572 shutdown_connections 2024-05-08T21:18:04.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.297+0000 7ff5d22c9700 1 -- 172.21.15.33:0/185404572 wait complete. 2024-05-08T21:18:04.320 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:03 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2427805130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T21:18:04.353 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:03 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2427805130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T21:18:04.998 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:04 smithi033 ceph-mon[42528]: pgmap v261: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 383 B/s wr, 9 op/s; 155 B/s, 1 objects/s recovering 2024-05-08T21:18:04.998 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:04 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/185404572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T21:18:04.998 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:04 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/47158285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T21:18:05.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.479+0000 7fd781300700 1 -- 172.21.15.33:0/2173422042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd77c100ff0 msgr2=0x7fd77c101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:05.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.479+0000 7fd781300700 1 --2- 172.21.15.33:0/2173422042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd77c100ff0 0x7fd77c101450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd770009990 tx=0x7fd770009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:05.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.479+0000 7fd781300700 1 -- 172.21.15.33:0/2173422042 shutdown_connections 2024-05-08T21:18:05.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.479+0000 7fd781300700 1 --2- 172.21.15.33:0/2173422042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c101990 0x7fd77c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:05.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.479+0000 7fd781300700 1 --2- 172.21.15.33:0/2173422042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c107030 0x7fd77c107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:05.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.479+0000 7fd781300700 1 --2- 172.21.15.33:0/2173422042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd77c100ff0 0x7fd77c101450 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:05.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.479+0000 7fd781300700 1 -- 172.21.15.33:0/2173422042 >> 172.21.15.33:0/2173422042 conn(0x7fd77c078190 msgr2=0x7fd77c0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:05.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.479+0000 7fd781300700 1 -- 172.21.15.33:0/2173422042 shutdown_connections 2024-05-08T21:18:05.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.479+0000 7fd781300700 1 -- 172.21.15.33:0/2173422042 wait complete. 2024-05-08T21:18:05.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.480+0000 7fd781300700 1 Processor -- start 2024-05-08T21:18:05.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.480+0000 7fd781300700 1 -- start start 2024-05-08T21:18:05.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.480+0000 7fd781300700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c100ff0 0x7fd77c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:05.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.480+0000 7fd781300700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c101990 0x7fd77c197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:05.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.480+0000 7fd781300700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd77c107030 0x7fd77c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:05.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd781300700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c19c240 con 0x7fd77c107030 2024-05-08T21:18:05.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd781300700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c19c3b0 con 0x7fd77c101990 2024-05-08T21:18:05.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd781300700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c19c570 con 0x7fd77c100ff0 2024-05-08T21:18:05.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd77c107030 0x7fd77c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:05.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd77c107030 0x7fd77c19bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44144/0 (socket says 172.21.15.33:44144) 2024-05-08T21:18:05.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77b7fe700 1 -- 172.21.15.33:0/47158285 learned_addr learned my addr 172.21.15.33:0/47158285 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77a7fc700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c101990 0x7fd77c197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77affd700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c100ff0 0x7fd77c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77b7fe700 1 -- 172.21.15.33:0/47158285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c100ff0 msgr2=0x7fd77c197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77b7fe700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c100ff0 0x7fd77c197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77b7fe700 1 -- 172.21.15.33:0/47158285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c101990 msgr2=0x7fd77c197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77b7fe700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c101990 0x7fd77c197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77b7fe700 1 -- 172.21.15.33:0/47158285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7700095f0 con 0x7fd77c107030 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77a7fc700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c101990 0x7fd77c197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77affd700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c100ff0 0x7fd77c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.481+0000 7fd77b7fe700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd77c107030 0x7fd77c19bc30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd774007f10 tx=0x7fd77400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.482+0000 7fd763fff700 1 -- 172.21.15.33:0/47158285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774017070 con 0x7fd77c107030 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.482+0000 7fd781300700 1 -- 172.21.15.33:0/47158285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77c19c850 con 0x7fd77c107030 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.482+0000 7fd763fff700 1 -- 172.21.15.33:0/47158285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd77400f040 con 0x7fd77c107030 2024-05-08T21:18:05.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.482+0000 7fd763fff700 1 -- 172.21.15.33:0/47158285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774005570 con 0x7fd77c107030 2024-05-08T21:18:05.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.482+0000 7fd781300700 1 -- 172.21.15.33:0/47158285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c1a40b0 con 0x7fd77c107030 2024-05-08T21:18:05.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.483+0000 7fd781300700 1 -- 172.21.15.33:0/47158285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77c109550 con 0x7fd77c107030 2024-05-08T21:18:05.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.485+0000 7fd763fff700 1 -- 172.21.15.33:0/47158285 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd77400db20 con 0x7fd77c107030 2024-05-08T21:18:05.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.485+0000 7fd763fff700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7640a11b0 0x7fd7640a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:05.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.485+0000 7fd77affd700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7640a11b0 0x7fd7640a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:05.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.485+0000 7fd763fff700 1 -- 172.21.15.33:0/47158285 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd7740ccaf0 con 0x7fd77c107030 2024-05-08T21:18:05.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.486+0000 7fd77affd700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7640a11b0 0x7fd7640a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009fd0 tx=0x7fd76c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:05.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.489+0000 7fd763fff700 1 -- 172.21.15.33:0/47158285 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd774095db0 con 0x7fd77c107030 2024-05-08T21:18:05.101 INFO:teuthology.orchestra.run.smithi033.stdout:103079215155 2024-05-08T21:18:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.712+0000 7fd781300700 1 -- 172.21.15.33:0/47158285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd77c1026e0 con 0x7fd77c107030 2024-05-08T21:18:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.713+0000 7fd763fff700 1 -- 172.21.15.33:0/47158285 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd7740993d0 con 0x7fd77c107030 2024-05-08T21:18:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.715+0000 7fd781300700 1 -- 172.21.15.33:0/47158285 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7640a11b0 msgr2=0x7fd7640a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.715+0000 7fd781300700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7640a11b0 0x7fd7640a3670 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009fd0 tx=0x7fd76c009260 comp rx=0 tx=0).stop 2024-05-08T21:18:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.715+0000 7fd781300700 1 -- 172.21.15.33:0/47158285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd77c107030 msgr2=0x7fd77c19bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.715+0000 7fd781300700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd77c107030 0x7fd77c19bc30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd774007f10 tx=0x7fd77400d3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.715+0000 7fd781300700 1 -- 172.21.15.33:0/47158285 shutdown_connections 2024-05-08T21:18:05.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.715+0000 7fd781300700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd77c107030 0x7fd77c19bc30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:05.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.715+0000 7fd781300700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd77c100ff0 0x7fd77c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:05.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.715+0000 7fd781300700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7640a11b0 0x7fd7640a3670 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:05.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.715+0000 7fd781300700 1 --2- 172.21.15.33:0/47158285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c101990 0x7fd77c197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:05.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.715+0000 7fd781300700 1 -- 172.21.15.33:0/47158285 >> 172.21.15.33:0/47158285 conn(0x7fd77c078190 msgr2=0x7fd77c0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:05.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.715+0000 7fd781300700 1 -- 172.21.15.33:0/47158285 shutdown_connections 2024-05-08T21:18:05.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:04.715+0000 7fd781300700 1 -- 172.21.15.33:0/47158285 wait complete. 2024-05-08T21:18:05.320 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:04 smithi071 ceph-mon[47074]: pgmap v261: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 383 B/s wr, 9 op/s; 155 B/s, 1 objects/s recovering 2024-05-08T21:18:05.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:04 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/185404572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T21:18:05.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:04 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/47158285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T21:18:05.353 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:04 smithi082 ceph-mon[47512]: pgmap v261: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 383 B/s wr, 9 op/s; 155 B/s, 1 objects/s recovering 2024-05-08T21:18:05.353 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:04 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/185404572' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T21:18:05.353 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:04 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/47158285' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T21:18:05.355 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738426 got 34359738428 for osd.0 2024-05-08T21:18:05.355 DEBUG:teuthology.parallel:result is None 2024-05-08T21:18:06.562 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822698 got 154618822700 for osd.5 2024-05-08T21:18:06.562 DEBUG:teuthology.parallel:result is None 2024-05-08T21:18:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.764+0000 7f6adff19700 1 -- 172.21.15.33:0/2886629553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad81001a0 msgr2=0x7f6ad8100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.764+0000 7f6adff19700 1 --2- 172.21.15.33:0/2886629553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad81001a0 0x7f6ad8100620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009990 tx=0x7f6ad4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.764+0000 7f6adff19700 1 -- 172.21.15.33:0/2886629553 shutdown_connections 2024-05-08T21:18:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.764+0000 7f6adff19700 1 --2- 172.21.15.33:0/2886629553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ad8100b60 0x7f6ad810cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.764+0000 7f6adff19700 1 --2- 172.21.15.33:0/2886629553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ad8104e10 0x7f6ad81051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.764+0000 7f6adff19700 1 --2- 172.21.15.33:0/2886629553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad81001a0 0x7f6ad8100620 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.764+0000 7f6adff19700 1 -- 172.21.15.33:0/2886629553 >> 172.21.15.33:0/2886629553 conn(0x7f6ad8076d60 msgr2=0x7f6ad8077170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.764+0000 7f6adff19700 1 -- 172.21.15.33:0/2886629553 shutdown_connections 2024-05-08T21:18:06.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.765+0000 7f6adff19700 1 -- 172.21.15.33:0/2886629553 wait complete. 2024-05-08T21:18:06.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.765+0000 7f6adff19700 1 Processor -- start 2024-05-08T21:18:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.765+0000 7f6adff19700 1 -- start start 2024-05-08T21:18:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6adff19700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ad81001a0 0x7f6ad8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6adff19700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad8100b60 0x7f6ad8197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6adff19700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ad8104e10 0x7f6ad819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6adff19700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad819c240 con 0x7f6ad8100b60 2024-05-08T21:18:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6adff19700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad819c3b0 con 0x7f6ad81001a0 2024-05-08T21:18:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6adff19700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad819c570 con 0x7f6ad8104e10 2024-05-08T21:18:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6add4b4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad8100b60 0x7f6ad8197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6add4b4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad8100b60 0x7f6ad8197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44164/0 (socket says 172.21.15.33:44164) 2024-05-08T21:18:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6add4b4700 1 -- 172.21.15.33:0/923092528 learned_addr learned my addr 172.21.15.33:0/923092528 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6ade4b6700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ad8104e10 0x7f6ad819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:06.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6addcb5700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ad81001a0 0x7f6ad8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6ade4b6700 1 -- 172.21.15.33:0/923092528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ad81001a0 msgr2=0x7f6ad8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6ade4b6700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ad81001a0 0x7f6ad8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6ade4b6700 1 -- 172.21.15.33:0/923092528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad8100b60 msgr2=0x7f6ad8197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6ade4b6700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad8100b60 0x7f6ad8197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.766+0000 7f6ade4b6700 1 -- 172.21.15.33:0/923092528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad40095f0 con 0x7f6ad8104e10 2024-05-08T21:18:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.767+0000 7f6ade4b6700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ad8104e10 0x7f6ad819bc30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0007f10 tx=0x7f6ad000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.767+0000 7f6acaffd700 1 -- 172.21.15.33:0/923092528 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0004db0 con 0x7f6ad8104e10 2024-05-08T21:18:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.767+0000 7f6adff19700 1 -- 172.21.15.33:0/923092528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad819c850 con 0x7f6ad8104e10 2024-05-08T21:18:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.767+0000 7f6adff19700 1 -- 172.21.15.33:0/923092528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad81a40b0 con 0x7f6ad8104e10 2024-05-08T21:18:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.767+0000 7f6acaffd700 1 -- 172.21.15.33:0/923092528 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ad000f040 con 0x7f6ad8104e10 2024-05-08T21:18:06.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.767+0000 7f6acaffd700 1 -- 172.21.15.33:0/923092528 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0005890 con 0x7f6ad8104e10 2024-05-08T21:18:06.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.769+0000 7f6adff19700 1 -- 172.21.15.33:0/923092528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc0052f0 con 0x7f6ad8104e10 2024-05-08T21:18:06.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.770+0000 7f6acaffd700 1 -- 172.21.15.33:0/923092528 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6ad0005000 con 0x7f6ad8104e10 2024-05-08T21:18:06.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.770+0000 7f6acaffd700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6ac40a1230 0x7f6ac40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:06.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.770+0000 7f6acaffd700 1 -- 172.21.15.33:0/923092528 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ad00cc500 con 0x7f6ad8104e10 2024-05-08T21:18:06.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.770+0000 7f6addcb5700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6ac40a1230 0x7f6ac40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:06.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.771+0000 7f6addcb5700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6ac40a1230 0x7f6ac40a36f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009c60 tx=0x7f6acc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:06.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.775+0000 7f6acaffd700 1 -- 172.21.15.33:0/923092528 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6ad0095820 con 0x7f6ad8104e10 2024-05-08T21:18:07.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.996+0000 7f6adff19700 1 -- 172.21.15.33:0/923092528 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f6abc005160 con 0x7f6ad8104e10 2024-05-08T21:18:07.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.997+0000 7f6acaffd700 1 -- 172.21.15.33:0/923092528 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f6ad0098e40 con 0x7f6ad8104e10 2024-05-08T21:18:07.001 INFO:teuthology.orchestra.run.smithi033.stdout:296352743449 2024-05-08T21:18:07.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.999+0000 7f6adff19700 1 -- 172.21.15.33:0/923092528 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6ac40a1230 msgr2=0x7f6ac40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:07.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.999+0000 7f6adff19700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6ac40a1230 0x7f6ac40a36f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009c60 tx=0x7f6acc009260 comp rx=0 tx=0).stop 2024-05-08T21:18:07.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.999+0000 7f6adff19700 1 -- 172.21.15.33:0/923092528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ad8104e10 msgr2=0x7f6ad819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:07.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.999+0000 7f6adff19700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ad8104e10 0x7f6ad819bc30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0007f10 tx=0x7f6ad000d3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:07.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.999+0000 7f6adff19700 1 -- 172.21.15.33:0/923092528 shutdown_connections 2024-05-08T21:18:07.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.999+0000 7f6adff19700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ad8104e10 0x7f6ad819bc30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.999+0000 7f6adff19700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6ac40a1230 0x7f6ac40a36f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.999+0000 7f6adff19700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ad81001a0 0x7f6ad8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.999+0000 7f6adff19700 1 --2- 172.21.15.33:0/923092528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad8100b60 0x7f6ad8197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.999+0000 7f6adff19700 1 -- 172.21.15.33:0/923092528 >> 172.21.15.33:0/923092528 conn(0x7f6ad8076d60 msgr2=0x7f6ad80698c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.999+0000 7f6adff19700 1 -- 172.21.15.33:0/923092528 shutdown_connections 2024-05-08T21:18:07.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:06.999+0000 7f6adff19700 1 -- 172.21.15.33:0/923092528 wait complete. 2024-05-08T21:18:07.168 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215153 got 103079215155 for osd.3 2024-05-08T21:18:07.169 DEBUG:teuthology.parallel:result is None 2024-05-08T21:18:07.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:07 smithi033 ceph-mon[42528]: pgmap v262: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 316 B/s wr, 6 op/s; 128 B/s, 1 objects/s recovering 2024-05-08T21:18:07.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:07 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/923092528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T21:18:07.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:07 smithi071 ceph-mon[47074]: pgmap v262: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 316 B/s wr, 6 op/s; 128 B/s, 1 objects/s recovering 2024-05-08T21:18:07.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:07 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/923092528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T21:18:07.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:07 smithi082 ceph-mon[47512]: pgmap v262: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 316 B/s wr, 6 op/s; 128 B/s, 1 objects/s recovering 2024-05-08T21:18:07.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:07 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/923092528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T21:18:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.036+0000 7f9ad0cee700 1 -- 172.21.15.33:0/1567884377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9acc105f00 msgr2=0x7f9acc1062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.036+0000 7f9ad0cee700 1 --2- 172.21.15.33:0/1567884377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9acc105f00 0x7f9acc1062e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009990 tx=0x7f9abc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.036+0000 7f9ad0cee700 1 -- 172.21.15.33:0/1567884377 shutdown_connections 2024-05-08T21:18:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.036+0000 7f9ad0cee700 1 --2- 172.21.15.33:0/1567884377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9acc100890 0x7f9acc10cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.036+0000 7f9ad0cee700 1 --2- 172.21.15.33:0/1567884377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9acc105f00 0x7f9acc1062e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.036+0000 7f9ad0cee700 1 --2- 172.21.15.33:0/1567884377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9acc0ffed0 0x7f9acc100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.036+0000 7f9ad0cee700 1 -- 172.21.15.33:0/1567884377 >> 172.21.15.33:0/1567884377 conn(0x7f9acc075f80 msgr2=0x7f9acc076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.036+0000 7f9ad0cee700 1 -- 172.21.15.33:0/1567884377 shutdown_connections 2024-05-08T21:18:08.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.036+0000 7f9ad0cee700 1 -- 172.21.15.33:0/1567884377 wait complete. 2024-05-08T21:18:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.037+0000 7f9ad0cee700 1 Processor -- start 2024-05-08T21:18:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.037+0000 7f9ad0cee700 1 -- start start 2024-05-08T21:18:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.037+0000 7f9ad0cee700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9acc0ffed0 0x7f9acc197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.037+0000 7f9ad0cee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9acc100890 0x7f9acc197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.037+0000 7f9ad0cee700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9acc105f00 0x7f9acc19bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.037+0000 7f9ad0cee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc19c200 con 0x7f9acc100890 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.037+0000 7f9ad0cee700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc19c370 con 0x7f9acc0ffed0 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.037+0000 7f9ad0cee700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc19c530 con 0x7f9acc105f00 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.037+0000 7f9ac9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9acc100890 0x7f9acc197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.037+0000 7f9ac9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9acc100890 0x7f9acc197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44208/0 (socket says 172.21.15.33:44208) 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.037+0000 7f9ac9d9b700 1 -- 172.21.15.33:0/626186309 learned_addr learned my addr 172.21.15.33:0/626186309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.037+0000 7f9aca59c700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9acc0ffed0 0x7f9acc197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.038+0000 7f9acad9d700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9acc105f00 0x7f9acc19bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.038+0000 7f9aca59c700 1 -- 172.21.15.33:0/626186309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9acc105f00 msgr2=0x7f9acc19bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.038+0000 7f9aca59c700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9acc105f00 0x7f9acc19bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.038+0000 7f9aca59c700 1 -- 172.21.15.33:0/626186309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9acc100890 msgr2=0x7f9acc197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.038+0000 7f9aca59c700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9acc100890 0x7f9acc197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.038+0000 7f9aca59c700 1 -- 172.21.15.33:0/626186309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc0095f0 con 0x7f9acc0ffed0 2024-05-08T21:18:08.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.038+0000 7f9aca59c700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9acc0ffed0 0x7f9acc197350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0098e0 tx=0x7f9abc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.038+0000 7f9abb7fe700 1 -- 172.21.15.33:0/626186309 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc01d070 con 0x7f9acc0ffed0 2024-05-08T21:18:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.038+0000 7f9abb7fe700 1 -- 172.21.15.33:0/626186309 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9abc00fb00 con 0x7f9acc0ffed0 2024-05-08T21:18:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.038+0000 7f9ad0cee700 1 -- 172.21.15.33:0/626186309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc19c7b0 con 0x7f9acc0ffed0 2024-05-08T21:18:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.038+0000 7f9abb7fe700 1 -- 172.21.15.33:0/626186309 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc022a70 con 0x7f9acc0ffed0 2024-05-08T21:18:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.038+0000 7f9ad0cee700 1 -- 172.21.15.33:0/626186309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc1a4070 con 0x7f9acc0ffed0 2024-05-08T21:18:08.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.039+0000 7f9ad0cee700 1 -- 172.21.15.33:0/626186309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc1015b0 con 0x7f9acc0ffed0 2024-05-08T21:18:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.041+0000 7f9abb7fe700 1 -- 172.21.15.33:0/626186309 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9abc00fc70 con 0x7f9acc0ffed0 2024-05-08T21:18:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.041+0000 7f9abb7fe700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9ab40a11b0 0x7f9ab40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.042+0000 7f9abb7fe700 1 -- 172.21.15.33:0/626186309 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9abc0ceb50 con 0x7f9acc0ffed0 2024-05-08T21:18:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.042+0000 7f9ac9d9b700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9ab40a11b0 0x7f9ab40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:08.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.042+0000 7f9ac9d9b700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9ab40a11b0 0x7f9ab40a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9acc198860 tx=0x7f9ac000b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:08.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.046+0000 7f9abb7fe700 1 -- 172.21.15.33:0/626186309 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9abc097ea0 con 0x7f9acc0ffed0 2024-05-08T21:18:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.974+0000 7faa9f680700 1 -- 172.21.15.33:0/3410992872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa980692f0 msgr2=0x7faa9810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.974+0000 7faa9f680700 1 --2- 172.21.15.33:0/3410992872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa980692f0 0x7faa9810de60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faa90009990 tx=0x7faa90009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.974+0000 7faa9f680700 1 -- 172.21.15.33:0/3410992872 shutdown_connections 2024-05-08T21:18:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.974+0000 7faa9f680700 1 --2- 172.21.15.33:0/3410992872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa980692f0 0x7faa9810de60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.974+0000 7faa9f680700 1 --2- 172.21.15.33:0/3410992872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa98107020 0x7faa98107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.974+0000 7faa9f680700 1 --2- 172.21.15.33:0/3410992872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa98068950 0x7faa98068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.974+0000 7faa9f680700 1 -- 172.21.15.33:0/3410992872 >> 172.21.15.33:0/3410992872 conn(0x7faa98075f80 msgr2=0x7faa98076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.974+0000 7faa9f680700 1 -- 172.21.15.33:0/3410992872 shutdown_connections 2024-05-08T21:18:08.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.975+0000 7faa9f680700 1 -- 172.21.15.33:0/3410992872 wait complete. 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.975+0000 7faa9f680700 1 Processor -- start 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.975+0000 7faa9f680700 1 -- start start 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.975+0000 7faa9f680700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa98068950 0x7faa98197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.975+0000 7faa9f680700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa980692f0 0x7faa98197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.975+0000 7faa9f680700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa98107020 0x7faa9819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.975+0000 7faa9f680700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9819c130 con 0x7faa98068950 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.975+0000 7faa9f680700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9819c2a0 con 0x7faa980692f0 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.975+0000 7faa9f680700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9819c460 con 0x7faa98107020 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa9dc1d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa98107020 0x7faa9819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa9dc1d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa98107020 0x7faa9819bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:44108/0 (socket says 172.21.15.33:44108) 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa9dc1d700 1 -- 172.21.15.33:0/3786624593 learned_addr learned my addr 172.21.15.33:0/3786624593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa9cc1b700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa980692f0 0x7faa98197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:08.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa9dc1d700 1 -- 172.21.15.33:0/3786624593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa980692f0 msgr2=0x7faa98197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa9dc1d700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa980692f0 0x7faa98197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa9dc1d700 1 -- 172.21.15.33:0/3786624593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa98068950 msgr2=0x7faa98197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa9dc1d700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa98068950 0x7faa98197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa9dc1d700 1 -- 172.21.15.33:0/3786624593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa900095f0 con 0x7faa98107020 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa9dc1d700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa98107020 0x7faa9819bb20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa900092e0 tx=0x7faa90004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa967fc700 1 -- 172.21.15.33:0/3786624593 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9001d070 con 0x7faa98107020 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa9f680700 1 -- 172.21.15.33:0/3786624593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9819c6e0 con 0x7faa98107020 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa967fc700 1 -- 172.21.15.33:0/3786624593 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa90005030 con 0x7faa98107020 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.976+0000 7faa967fc700 1 -- 172.21.15.33:0/3786624593 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa900049e0 con 0x7faa98107020 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.977+0000 7faa9f680700 1 -- 172.21.15.33:0/3786624593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa981a3fa0 con 0x7faa98107020 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.977+0000 7faa9f680700 1 -- 172.21.15.33:0/3786624593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa98109540 con 0x7faa98107020 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.979+0000 7faa967fc700 1 -- 172.21.15.33:0/3786624593 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7faa900051a0 con 0x7faa98107020 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.979+0000 7faa967fc700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faa800a11b0 0x7faa800a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.980+0000 7faa9d41c700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faa800a11b0 0x7faa800a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:08.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.980+0000 7faa967fc700 1 -- 172.21.15.33:0/3786624593 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faa900ce160 con 0x7faa98107020 2024-05-08T21:18:08.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.980+0000 7faa9d41c700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faa800a11b0 0x7faa800a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faa88009710 tx=0x7faa88009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:08.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:07.983+0000 7faa967fc700 1 -- 172.21.15.33:0/3786624593 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7faa900974f0 con 0x7faa98107020 2024-05-08T21:18:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.223+0000 7faa9f680700 1 -- 172.21.15.33:0/3786624593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7faa981026d0 con 0x7faa98107020 2024-05-08T21:18:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.224+0000 7faa967fc700 1 -- 172.21.15.33:0/3786624593 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa9009ab10 con 0x7faa98107020 2024-05-08T21:18:08.228 INFO:teuthology.orchestra.run.smithi033.stdout:240518168610 2024-05-08T21:18:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.227+0000 7faa9f680700 1 -- 172.21.15.33:0/3786624593 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faa800a11b0 msgr2=0x7faa800a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.227+0000 7faa9f680700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faa800a11b0 0x7faa800a3670 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faa88009710 tx=0x7faa88009620 comp rx=0 tx=0).stop 2024-05-08T21:18:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.227+0000 7faa9f680700 1 -- 172.21.15.33:0/3786624593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa98107020 msgr2=0x7faa9819bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.227+0000 7faa9f680700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa98107020 0x7faa9819bb20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa900092e0 tx=0x7faa90004730 comp rx=0 tx=0).stop 2024-05-08T21:18:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.227+0000 7faa9f680700 1 -- 172.21.15.33:0/3786624593 shutdown_connections 2024-05-08T21:18:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.227+0000 7faa9f680700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa98107020 0x7faa9819bb20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.227+0000 7faa9f680700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa98068950 0x7faa98197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.227+0000 7faa9f680700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faa800a11b0 0x7faa800a3670 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.227+0000 7faa9f680700 1 --2- 172.21.15.33:0/3786624593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa980692f0 0x7faa98197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.227+0000 7faa9f680700 1 -- 172.21.15.33:0/3786624593 >> 172.21.15.33:0/3786624593 conn(0x7faa98075f80 msgr2=0x7faa980fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.230+0000 7faa9f680700 1 -- 172.21.15.33:0/3786624593 shutdown_connections 2024-05-08T21:18:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.230+0000 7faa9f680700 1 -- 172.21.15.33:0/3786624593 wait complete. 2024-05-08T21:18:08.257 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743446 got 296352743449 for osd.10 2024-05-08T21:18:08.258 DEBUG:teuthology.parallel:result is None 2024-05-08T21:18:08.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.293+0000 7f9ad0cee700 1 -- 172.21.15.33:0/626186309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9acc198610 con 0x7f9acc0ffed0 2024-05-08T21:18:08.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f9abb7fe700 1 -- 172.21.15.33:0/626186309 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9abc09b4c0 con 0x7f9acc0ffed0 2024-05-08T21:18:08.298 INFO:teuthology.orchestra.run.smithi033.stdout:210453397541 2024-05-08T21:18:08.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f9ad0cee700 1 -- 172.21.15.33:0/626186309 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9ab40a11b0 msgr2=0x7f9ab40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f9ad0cee700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9ab40a11b0 0x7f9ab40a3670 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9acc198860 tx=0x7f9ac000b500 comp rx=0 tx=0).stop 2024-05-08T21:18:08.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f9ad0cee700 1 -- 172.21.15.33:0/626186309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9acc0ffed0 msgr2=0x7f9acc197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f9ad0cee700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9acc0ffed0 0x7f9acc197350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0098e0 tx=0x7f9abc009580 comp rx=0 tx=0).stop 2024-05-08T21:18:08.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f9ad0cee700 1 -- 172.21.15.33:0/626186309 shutdown_connections 2024-05-08T21:18:08.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f9ad0cee700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9acc105f00 0x7f9acc19bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f9ad0cee700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9ab40a11b0 0x7f9ab40a3670 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f9ad0cee700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9acc0ffed0 0x7f9acc197350 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f9ad0cee700 1 --2- 172.21.15.33:0/626186309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9acc100890 0x7f9acc197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f9ad0cee700 1 -- 172.21.15.33:0/626186309 >> 172.21.15.33:0/626186309 conn(0x7f9acc075f80 msgr2=0x7f9acc068bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:08.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f9ad0cee700 1 -- 172.21.15.33:0/626186309 shutdown_connections 2024-05-08T21:18:08.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f9ad0cee700 1 -- 172.21.15.33:0/626186309 wait complete. 2024-05-08T21:18:08.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.292+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/1344416818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bb8107020 msgr2=0x7f1bb8107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.292+0000 7f1bbfe9c700 1 --2- 172.21.15.33:0/1344416818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bb8107020 0x7f1bb8107400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009990 tx=0x7f1bb4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.292+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/1344416818 shutdown_connections 2024-05-08T21:18:08.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.292+0000 7f1bbfe9c700 1 --2- 172.21.15.33:0/1344416818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb80692f0 0x7f1bb810de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.292+0000 7f1bbfe9c700 1 --2- 172.21.15.33:0/1344416818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bb8107020 0x7f1bb8107400 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.292+0000 7f1bbfe9c700 1 --2- 172.21.15.33:0/1344416818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bb8068950 0x7f1bb8068db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.292+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/1344416818 >> 172.21.15.33:0/1344416818 conn(0x7f1bb8075f80 msgr2=0x7f1bb8076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.292+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/1344416818 shutdown_connections 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.292+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/1344416818 wait complete. 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.293+0000 7f1bbfe9c700 1 Processor -- start 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.293+0000 7f1bbfe9c700 1 -- start start 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.293+0000 7f1bbfe9c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bb8068950 0x7f1bb81972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.293+0000 7f1bbfe9c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bb80692f0 0x7f1bb8197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.293+0000 7f1bbfe9c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb8107020 0x7f1bb819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.293+0000 7f1bbfe9c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb819c1c0 con 0x7f1bb80692f0 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.293+0000 7f1bbfe9c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb819c330 con 0x7f1bb8068950 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.293+0000 7f1bbfe9c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb819c4f0 con 0x7f1bb8107020 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.293+0000 7f1bbd437700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bb80692f0 0x7f1bb8197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f1bbd437700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bb80692f0 0x7f1bb8197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44224/0 (socket says 172.21.15.33:44224) 2024-05-08T21:18:08.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f1bbd437700 1 -- 172.21.15.33:0/2376066443 learned_addr learned my addr 172.21.15.33:0/2376066443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f1bbe439700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb8107020 0x7f1bb819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f1bbdc38700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bb8068950 0x7f1bb81972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f1bbd437700 1 -- 172.21.15.33:0/2376066443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb8107020 msgr2=0x7f1bb819bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f1bbd437700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb8107020 0x7f1bb819bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f1bbd437700 1 -- 172.21.15.33:0/2376066443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bb8068950 msgr2=0x7f1bb81972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f1bbd437700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bb8068950 0x7f1bb81972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f1bbd437700 1 -- 172.21.15.33:0/2376066443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40095f0 con 0x7f1bb80692f0 2024-05-08T21:18:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f1bbe439700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb8107020 0x7f1bb819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f1bbd437700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bb80692f0 0x7f1bb8197800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800d950 tx=0x7f1ba800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.294+0000 7f1baeffd700 1 -- 172.21.15.33:0/2376066443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800f890 con 0x7f1bb80692f0 2024-05-08T21:18:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.295+0000 7f1baeffd700 1 -- 172.21.15.33:0/2376066443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ba800fe80 con 0x7f1bb80692f0 2024-05-08T21:18:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.295+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/2376066443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb819c7d0 con 0x7f1bb80692f0 2024-05-08T21:18:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.295+0000 7f1baeffd700 1 -- 172.21.15.33:0/2376066443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba800e3b0 con 0x7f1bb80692f0 2024-05-08T21:18:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.295+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/2376066443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb81a4030 con 0x7f1bb80692f0 2024-05-08T21:18:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.296+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/2376066443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb8109540 con 0x7f1bb80692f0 2024-05-08T21:18:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.297+0000 7f1baeffd700 1 -- 172.21.15.33:0/2376066443 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1ba8010460 con 0x7f1bb80692f0 2024-05-08T21:18:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.298+0000 7f1baeffd700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1ba40a11b0 0x7f1ba40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.298+0000 7f1baeffd700 1 -- 172.21.15.33:0/2376066443 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1ba80ccb10 con 0x7f1bb80692f0 2024-05-08T21:18:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.298+0000 7f1bbdc38700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1ba40a11b0 0x7f1ba40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.298+0000 7f1bbdc38700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1ba40a11b0 0x7f1ba40a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4006010 tx=0x7f1bb4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.305+0000 7f1baeffd700 1 -- 172.21.15.33:0/2376066443 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1ba8095e30 con 0x7f1bb80692f0 2024-05-08T21:18:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.537+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/2376066443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f1bb81985d0 con 0x7f1bb80692f0 2024-05-08T21:18:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.537+0000 7f1baeffd700 1 -- 172.21.15.33:0/2376066443 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ba8016090 con 0x7f1bb80692f0 2024-05-08T21:18:08.745 INFO:teuthology.orchestra.run.smithi033.stdout:274877906973 2024-05-08T21:18:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.539+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/2376066443 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1ba40a11b0 msgr2=0x7f1ba40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.539+0000 7f1bbfe9c700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1ba40a11b0 0x7f1ba40a3670 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4006010 tx=0x7f1bb4005960 comp rx=0 tx=0).stop 2024-05-08T21:18:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.539+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/2376066443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bb80692f0 msgr2=0x7f1bb8197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.539+0000 7f1bbfe9c700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bb80692f0 0x7f1bb8197800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800d950 tx=0x7f1ba800dc60 comp rx=0 tx=0).stop 2024-05-08T21:18:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.539+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/2376066443 shutdown_connections 2024-05-08T21:18:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.539+0000 7f1bbfe9c700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb8107020 0x7f1bb819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.539+0000 7f1bbfe9c700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1ba40a11b0 0x7f1ba40a3670 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.539+0000 7f1bbfe9c700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bb8068950 0x7f1bb81972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.539+0000 7f1bbfe9c700 1 --2- 172.21.15.33:0/2376066443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bb80692f0 0x7f1bb8197800 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.539+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/2376066443 >> 172.21.15.33:0/2376066443 conn(0x7f1bb8075f80 msgr2=0x7f1bb80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.539+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/2376066443 shutdown_connections 2024-05-08T21:18:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:08.539+0000 7f1bbfe9c700 1 -- 172.21.15.33:0/2376066443 wait complete. 2024-05-08T21:18:09.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:09 smithi033 ceph-mon[42528]: pgmap v263: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 286 B/s wr, 8 op/s; 116 B/s, 0 objects/s recovering 2024-05-08T21:18:09.195 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168608 got 240518168610 for osd.8 2024-05-08T21:18:09.195 DEBUG:teuthology.parallel:result is None 2024-05-08T21:18:09.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:09 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3786624593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T21:18:09.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:09 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/626186309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T21:18:09.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:09 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2376066443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T21:18:09.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:09 smithi071 ceph-mon[47074]: pgmap v263: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 286 B/s wr, 8 op/s; 116 B/s, 0 objects/s recovering 2024-05-08T21:18:09.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:09 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3786624593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T21:18:09.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:09 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/626186309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T21:18:09.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:09 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2376066443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T21:18:09.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:09 smithi082 ceph-mon[47512]: pgmap v263: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 286 B/s wr, 8 op/s; 116 B/s, 0 objects/s recovering 2024-05-08T21:18:09.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:09 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3786624593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T21:18:09.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:09 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/626186309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T21:18:09.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:09 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2376066443' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T21:18:09.709 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906971 got 274877906973 for osd.9 2024-05-08T21:18:09.709 DEBUG:teuthology.parallel:result is None 2024-05-08T21:18:09.936 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397539 got 210453397541 for osd.7 2024-05-08T21:18:09.936 DEBUG:teuthology.parallel:result is None 2024-05-08T21:18:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.840+0000 7f5147c7b700 1 -- 172.21.15.33:0/3726906675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51401063c0 msgr2=0x7f514010dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.840+0000 7f5147c7b700 1 --2- 172.21.15.33:0/3726906675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51401063c0 0x7f514010dee0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f513800b0d0 tx=0x7f513800b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.840+0000 7f5147c7b700 1 -- 172.21.15.33:0/3726906675 shutdown_connections 2024-05-08T21:18:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.840+0000 7f5147c7b700 1 --2- 172.21.15.33:0/3726906675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51401063c0 0x7f514010dee0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.840+0000 7f5147c7b700 1 --2- 172.21.15.33:0/3726906675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51400ffba0 0x7f51400fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:09.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.840+0000 7f5147c7b700 1 --2- 172.21.15.33:0/3726906675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5140100550 0x7f5140105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.840+0000 7f5147c7b700 1 -- 172.21.15.33:0/3726906675 >> 172.21.15.33:0/3726906675 conn(0x7f5140075260 msgr2=0x7f5140075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.840+0000 7f5147c7b700 1 -- 172.21.15.33:0/3726906675 shutdown_connections 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.840+0000 7f5147c7b700 1 -- 172.21.15.33:0/3726906675 wait complete. 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5147c7b700 1 Processor -- start 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5147c7b700 1 -- start start 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5147c7b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51400ffba0 0x7f5140100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5147c7b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5140100550 0x7f5140101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5147c7b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51401063c0 0x7f51401015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5147c7b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5140102ae0 con 0x7f51401063c0 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5147c7b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51401a9cc0 con 0x7f51400ffba0 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5147c7b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51401a9e80 con 0x7f5140100550 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5146218700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51401063c0 0x7f51401015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5146218700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51401063c0 0x7f51401015a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44236/0 (socket says 172.21.15.33:44236) 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5146218700 1 -- 172.21.15.33:0/1904670078 learned_addr learned my addr 172.21.15.33:0/1904670078 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:09.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5145a17700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51400ffba0 0x7f5140100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.841+0000 7f5145216700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5140100550 0x7f5140101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.842+0000 7f5146218700 1 -- 172.21.15.33:0/1904670078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5140100550 msgr2=0x7f5140101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.842+0000 7f5146218700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5140100550 0x7f5140101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.842+0000 7f5146218700 1 -- 172.21.15.33:0/1904670078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51400ffba0 msgr2=0x7f5140100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.842+0000 7f5146218700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51400ffba0 0x7f5140100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.842+0000 7f5146218700 1 -- 172.21.15.33:0/1904670078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5138009d00 con 0x7f51401063c0 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.842+0000 7f5146218700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51401063c0 0x7f51401015a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5138006070 tx=0x7f5138007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.842+0000 7f5136ffd700 1 -- 172.21.15.33:0/1904670078 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513800e040 con 0x7f51401063c0 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.842+0000 7f5136ffd700 1 -- 172.21.15.33:0/1904670078 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51380094c0 con 0x7f51401063c0 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.842+0000 7f5147c7b700 1 -- 172.21.15.33:0/1904670078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51401aa100 con 0x7f51401063c0 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.842+0000 7f5136ffd700 1 -- 172.21.15.33:0/1904670078 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513801b6b0 con 0x7f51401063c0 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.842+0000 7f5147c7b700 1 -- 172.21.15.33:0/1904670078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51401aa570 con 0x7f51401063c0 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.843+0000 7f5147c7b700 1 -- 172.21.15.33:0/1904670078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51401095a0 con 0x7f51401063c0 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.845+0000 7f5136ffd700 1 -- 172.21.15.33:0/1904670078 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f513801d480 con 0x7f51401063c0 2024-05-08T21:18:09.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.846+0000 7f5136ffd700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f512c0a11b0 0x7f512c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:09.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.846+0000 7f5145a17700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f512c0a11b0 0x7f512c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:09.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.846+0000 7f5136ffd700 1 -- 172.21.15.33:0/1904670078 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f51380ce580 con 0x7f51401063c0 2024-05-08T21:18:09.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.846+0000 7f5145a17700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f512c0a11b0 0x7f512c0a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f513c005950 tx=0x7f513c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:09.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:09.852+0000 7f5136ffd700 1 -- 172.21.15.33:0/1904670078 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5138097930 con 0x7f51401063c0 2024-05-08T21:18:10.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.102+0000 7f5147c7b700 1 -- 172.21.15.33:0/1904670078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f5140102000 con 0x7f51401063c0 2024-05-08T21:18:10.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.102+0000 7f5136ffd700 1 -- 172.21.15.33:0/1904670078 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f513809af50 con 0x7f51401063c0 2024-05-08T21:18:10.105 INFO:teuthology.orchestra.run.smithi033.stdout:326417514519 2024-05-08T21:18:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.105+0000 7f5147c7b700 1 -- 172.21.15.33:0/1904670078 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f512c0a11b0 msgr2=0x7f512c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.105+0000 7f5147c7b700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f512c0a11b0 0x7f512c0a3670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f513c005950 tx=0x7f513c00b410 comp rx=0 tx=0).stop 2024-05-08T21:18:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.105+0000 7f5147c7b700 1 -- 172.21.15.33:0/1904670078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51401063c0 msgr2=0x7f51401015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:10.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.105+0000 7f5147c7b700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51401063c0 0x7f51401015a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5138006070 tx=0x7f5138007cc0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.105+0000 7f5147c7b700 1 -- 172.21.15.33:0/1904670078 shutdown_connections 2024-05-08T21:18:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.105+0000 7f5147c7b700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51401063c0 0x7f51401015a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.105+0000 7f5147c7b700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f512c0a11b0 0x7f512c0a3670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.105+0000 7f5147c7b700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51400ffba0 0x7f5140100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.105+0000 7f5147c7b700 1 --2- 172.21.15.33:0/1904670078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5140100550 0x7f5140101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.105+0000 7f5147c7b700 1 -- 172.21.15.33:0/1904670078 >> 172.21.15.33:0/1904670078 conn(0x7f5140075260 msgr2=0x7f51400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.105+0000 7f5147c7b700 1 -- 172.21.15.33:0/1904670078 shutdown_connections 2024-05-08T21:18:10.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.105+0000 7f5147c7b700 1 -- 172.21.15.33:0/1904670078 wait complete. 2024-05-08T21:18:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.185+0000 7f5a64bfa700 1 -- 172.21.15.33:0/529104057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a60105f00 msgr2=0x7f5a601062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.185+0000 7f5a64bfa700 1 --2- 172.21.15.33:0/529104057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a60105f00 0x7f5a601062e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009990 tx=0x7f5a50009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.186+0000 7f5a64bfa700 1 -- 172.21.15.33:0/529104057 shutdown_connections 2024-05-08T21:18:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.186+0000 7f5a64bfa700 1 --2- 172.21.15.33:0/529104057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a60100890 0x7f5a6010cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.186+0000 7f5a64bfa700 1 --2- 172.21.15.33:0/529104057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a60105f00 0x7f5a601062e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.186+0000 7f5a64bfa700 1 --2- 172.21.15.33:0/529104057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a600ffed0 0x7f5a60100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.186+0000 7f5a64bfa700 1 -- 172.21.15.33:0/529104057 >> 172.21.15.33:0/529104057 conn(0x7f5a60075f80 msgr2=0x7f5a60076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.186+0000 7f5a64bfa700 1 -- 172.21.15.33:0/529104057 shutdown_connections 2024-05-08T21:18:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.186+0000 7f5a64bfa700 1 -- 172.21.15.33:0/529104057 wait complete. 2024-05-08T21:18:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.187+0000 7f5a64bfa700 1 Processor -- start 2024-05-08T21:18:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.187+0000 7f5a64bfa700 1 -- start start 2024-05-08T21:18:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.187+0000 7f5a64bfa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a600ffed0 0x7f5a601981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.187+0000 7f5a64bfa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a60100890 0x7f5a60198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.187+0000 7f5a64bfa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a60105f00 0x7f5a601a3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.187+0000 7f5a64bfa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60199000 con 0x7f5a600ffed0 2024-05-08T21:18:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.187+0000 7f5a64bfa700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a601a3790 con 0x7f5a60105f00 2024-05-08T21:18:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.187+0000 7f5a64bfa700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a601a3900 con 0x7f5a60100890 2024-05-08T21:18:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.188+0000 7f5a5e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a600ffed0 0x7f5a601981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.188+0000 7f5a5ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a60105f00 0x7f5a601a3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.188+0000 7f5a5e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a600ffed0 0x7f5a601981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44254/0 (socket says 172.21.15.33:44254) 2024-05-08T21:18:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.188+0000 7f5a5e59c700 1 -- 172.21.15.33:0/4193978837 learned_addr learned my addr 172.21.15.33:0/4193978837 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.188+0000 7f5a5dd9b700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a60100890 0x7f5a60198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.188+0000 7f5a5e59c700 1 -- 172.21.15.33:0/4193978837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a60100890 msgr2=0x7f5a60198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.188+0000 7f5a5e59c700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a60100890 0x7f5a60198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.189+0000 7f5a5e59c700 1 -- 172.21.15.33:0/4193978837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a60105f00 msgr2=0x7f5a601a3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.189+0000 7f5a5e59c700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a60105f00 0x7f5a601a3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.189+0000 7f5a5e59c700 1 -- 172.21.15.33:0/4193978837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a500095f0 con 0x7f5a600ffed0 2024-05-08T21:18:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.189+0000 7f5a5ed9d700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a60105f00 0x7f5a601a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.189+0000 7f5a5e59c700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a600ffed0 0x7f5a601981d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5a500098e0 tx=0x7f5a50005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.189+0000 7f5a4f7fe700 1 -- 172.21.15.33:0/4193978837 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5001d070 con 0x7f5a600ffed0 2024-05-08T21:18:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.189+0000 7f5a4f7fe700 1 -- 172.21.15.33:0/4193978837 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a5000fb00 con 0x7f5a600ffed0 2024-05-08T21:18:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.189+0000 7f5a64bfa700 1 -- 172.21.15.33:0/4193978837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a601a3b20 con 0x7f5a600ffed0 2024-05-08T21:18:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.189+0000 7f5a4f7fe700 1 -- 172.21.15.33:0/4193978837 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50022aa0 con 0x7f5a600ffed0 2024-05-08T21:18:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.189+0000 7f5a64bfa700 1 -- 172.21.15.33:0/4193978837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a601a3f90 con 0x7f5a600ffed0 2024-05-08T21:18:10.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.191+0000 7f5a64bfa700 1 -- 172.21.15.33:0/4193978837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a60108420 con 0x7f5a600ffed0 2024-05-08T21:18:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.198+0000 7f5a4f7fe700 1 -- 172.21.15.33:0/4193978837 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5a5000fc70 con 0x7f5a600ffed0 2024-05-08T21:18:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.199+0000 7f5a4f7fe700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5a480a1230 0x7f5a480a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:10.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.199+0000 7f5a4f7fe700 1 -- 172.21.15.33:0/4193978837 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5a500ceb80 con 0x7f5a600ffed0 2024-05-08T21:18:10.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.201+0000 7f5a5dd9b700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5a480a1230 0x7f5a480a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:10.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.202+0000 7f5a4f7fe700 1 -- 172.21.15.33:0/4193978837 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5a50097f30 con 0x7f5a600ffed0 2024-05-08T21:18:10.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.208+0000 7f5a5dd9b700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5a480a1230 0x7f5a480a36f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5a60069110 tx=0x7f5a5400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:10.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.430+0000 7f5a64bfa700 1 -- 172.21.15.33:0/4193978837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5a6004ec90 con 0x7f5a600ffed0 2024-05-08T21:18:10.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.431+0000 7f5a4f7fe700 1 -- 172.21.15.33:0/4193978837 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a5009b550 con 0x7f5a600ffed0 2024-05-08T21:18:10.433 INFO:teuthology.orchestra.run.smithi033.stdout:133143986224 2024-05-08T21:18:10.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.433+0000 7f5a64bfa700 1 -- 172.21.15.33:0/4193978837 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5a480a1230 msgr2=0x7f5a480a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:10.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.433+0000 7f5a64bfa700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5a480a1230 0x7f5a480a36f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5a60069110 tx=0x7f5a5400b410 comp rx=0 tx=0).stop 2024-05-08T21:18:10.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.433+0000 7f5a64bfa700 1 -- 172.21.15.33:0/4193978837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a600ffed0 msgr2=0x7f5a601981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:10.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.433+0000 7f5a64bfa700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a600ffed0 0x7f5a601981d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5a500098e0 tx=0x7f5a50005740 comp rx=0 tx=0).stop 2024-05-08T21:18:10.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.433+0000 7f5a64bfa700 1 -- 172.21.15.33:0/4193978837 shutdown_connections 2024-05-08T21:18:10.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.433+0000 7f5a64bfa700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5a480a1230 0x7f5a480a36f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.433+0000 7f5a64bfa700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a60105f00 0x7f5a601a3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.433+0000 7f5a64bfa700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a600ffed0 0x7f5a601981d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.433+0000 7f5a64bfa700 1 --2- 172.21.15.33:0/4193978837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a60100890 0x7f5a60198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.433+0000 7f5a64bfa700 1 -- 172.21.15.33:0/4193978837 >> 172.21.15.33:0/4193978837 conn(0x7f5a60075f80 msgr2=0x7f5a600feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.433+0000 7f5a64bfa700 1 -- 172.21.15.33:0/4193978837 shutdown_connections 2024-05-08T21:18:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:10.433+0000 7f5a64bfa700 1 -- 172.21.15.33:0/4193978837 wait complete. 2024-05-08T21:18:10.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:10 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1904670078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T21:18:10.508 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514517 got 326417514519 for osd.11 2024-05-08T21:18:10.508 DEBUG:teuthology.parallel:result is None 2024-05-08T21:18:10.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:10 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1904670078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T21:18:10.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:10 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1904670078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T21:18:10.893 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986220 got 133143986224 for osd.4 2024-05-08T21:18:10.893 DEBUG:teuthology.parallel:result is None 2024-05-08T21:18:10.893 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T21:18:10.893 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T21:18:11.150 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:11 smithi033 ceph-mon[42528]: pgmap v264: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s; 103 B/s, 0 objects/s recovering 2024-05-08T21:18:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:11 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4193978837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T21:18:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:11.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:11 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:11.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:11 smithi071 ceph-mon[47074]: pgmap v264: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s; 103 B/s, 0 objects/s recovering 2024-05-08T21:18:11.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:11 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/4193978837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T21:18:11.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:11.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:11 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:11.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:11 smithi082 ceph-mon[47512]: pgmap v264: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s; 103 B/s, 0 objects/s recovering 2024-05-08T21:18:11.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:11 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/4193978837' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T21:18:11.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:11.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:11 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.283+0000 7f58b0eb2700 1 -- 172.21.15.33:0/2756917430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58ac107030 msgr2=0x7f58ac107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.283+0000 7f58b0eb2700 1 --2- 172.21.15.33:0/2756917430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58ac107030 0x7f58ac107410 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f589c009990 tx=0x7f589c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.283+0000 7f58b0eb2700 1 -- 172.21.15.33:0/2756917430 shutdown_connections 2024-05-08T21:18:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.283+0000 7f58b0eb2700 1 --2- 172.21.15.33:0/2756917430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac101990 0x7f58ac10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.283+0000 7f58b0eb2700 1 --2- 172.21.15.33:0/2756917430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58ac107030 0x7f58ac107410 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.283+0000 7f58b0eb2700 1 --2- 172.21.15.33:0/2756917430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac100ff0 0x7f58ac101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.283+0000 7f58b0eb2700 1 -- 172.21.15.33:0/2756917430 >> 172.21.15.33:0/2756917430 conn(0x7f58ac078190 msgr2=0x7f58ac0785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.283+0000 7f58b0eb2700 1 -- 172.21.15.33:0/2756917430 shutdown_connections 2024-05-08T21:18:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.283+0000 7f58b0eb2700 1 -- 172.21.15.33:0/2756917430 wait complete. 2024-05-08T21:18:12.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.284+0000 7f58b0eb2700 1 Processor -- start 2024-05-08T21:18:12.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.284+0000 7f58b0eb2700 1 -- start start 2024-05-08T21:18:12.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.284+0000 7f58b0eb2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac100ff0 0x7f58ac0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:12.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.284+0000 7f58b0eb2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac101990 0x7f58ac0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58b0eb2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58ac107030 0x7f58ac073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58b0eb2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac1a5710 con 0x7f58ac107030 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58b0eb2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac1a5880 con 0x7f58ac101990 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58b0eb2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac1a5a40 con 0x7f58ac100ff0 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58aad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58ac107030 0x7f58ac073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58aad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58ac107030 0x7f58ac073d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48718/0 (socket says 172.21.15.33:48718) 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58aad9d700 1 -- 172.21.15.33:0/1344696564 learned_addr learned my addr 172.21.15.33:0/1344696564 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58a9d9b700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac101990 0x7f58ac0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58aa59c700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac100ff0 0x7f58ac0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58aad9d700 1 -- 172.21.15.33:0/1344696564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac100ff0 msgr2=0x7f58ac0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58aad9d700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac100ff0 0x7f58ac0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58aad9d700 1 -- 172.21.15.33:0/1344696564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac101990 msgr2=0x7f58ac0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58aad9d700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac101990 0x7f58ac0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58aad9d700 1 -- 172.21.15.33:0/1344696564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c0095f0 con 0x7f58ac107030 2024-05-08T21:18:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58aad9d700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58ac107030 0x7f58ac073d20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f58a4007f10 tx=0x7f58a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f589b7fe700 1 -- 172.21.15.33:0/1344696564 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4009ea0 con 0x7f58ac107030 2024-05-08T21:18:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f589b7fe700 1 -- 172.21.15.33:0/1344696564 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58a400f040 con 0x7f58ac107030 2024-05-08T21:18:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f589b7fe700 1 -- 172.21.15.33:0/1344696564 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4004640 con 0x7f58ac107030 2024-05-08T21:18:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.285+0000 7f58b0eb2700 1 -- 172.21.15.33:0/1344696564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ac1a5d20 con 0x7f58ac107030 2024-05-08T21:18:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.286+0000 7f58b0eb2700 1 -- 172.21.15.33:0/1344696564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac1a61d0 con 0x7f58ac107030 2024-05-08T21:18:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.287+0000 7f58b0eb2700 1 -- 172.21.15.33:0/1344696564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac109550 con 0x7f58ac107030 2024-05-08T21:18:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.288+0000 7f589b7fe700 1 -- 172.21.15.33:0/1344696564 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f58a40047a0 con 0x7f58ac107030 2024-05-08T21:18:12.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.289+0000 7f589b7fe700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f58940a11b0 0x7f58940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:12.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.289+0000 7f589b7fe700 1 -- 172.21.15.33:0/1344696564 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f58a40cc4c0 con 0x7f58ac107030 2024-05-08T21:18:12.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.289+0000 7f58aa59c700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f58940a11b0 0x7f58940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.289+0000 7f58aa59c700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f58940a11b0 0x7f58940a3670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f589c00b5c0 tx=0x7f589c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.294+0000 7f589b7fe700 1 -- 172.21.15.33:0/1344696564 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f58a40957e0 con 0x7f58ac107030 2024-05-08T21:18:12.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:18:12.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:18:12.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:12.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.502+0000 7f58b0eb2700 1 -- 172.21.15.33:0/1344696564 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58ac0747a0 con 0x7f58940a11b0 2024-05-08T21:18:12.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.516+0000 7f589b7fe700 1 -- 172.21.15.33:0/1344696564 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426561 (secure 0 0 0) 0x7f58ac0747a0 con 0x7f58940a11b0 2024-05-08T21:18:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.520+0000 7f58b0eb2700 1 -- 172.21.15.33:0/1344696564 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f58940a11b0 msgr2=0x7f58940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.520+0000 7f58b0eb2700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f58940a11b0 0x7f58940a3670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f589c00b5c0 tx=0x7f589c005fb0 comp rx=0 tx=0).stop 2024-05-08T21:18:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.520+0000 7f58b0eb2700 1 -- 172.21.15.33:0/1344696564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58ac107030 msgr2=0x7f58ac073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.520+0000 7f58b0eb2700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58ac107030 0x7f58ac073d20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f58a4007f10 tx=0x7f58a400d3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.521+0000 7f58b0eb2700 1 -- 172.21.15.33:0/1344696564 shutdown_connections 2024-05-08T21:18:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.521+0000 7f58b0eb2700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58ac107030 0x7f58ac073d20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.521+0000 7f58b0eb2700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac100ff0 0x7f58ac0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.521+0000 7f58b0eb2700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f58940a11b0 0x7f58940a3670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.521+0000 7f58b0eb2700 1 --2- 172.21.15.33:0/1344696564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac101990 0x7f58ac0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:12.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.521+0000 7f58b0eb2700 1 -- 172.21.15.33:0/1344696564 >> 172.21.15.33:0/1344696564 conn(0x7f58ac078190 msgr2=0x7f58ac0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:12.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.521+0000 7f58b0eb2700 1 -- 172.21.15.33:0/1344696564 shutdown_connections 2024-05-08T21:18:12.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:12.521+0000 7f58b0eb2700 1 -- 172.21.15.33:0/1344696564 wait complete. 2024-05-08T21:18:12.529 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T21:18:12.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:18:12.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:18:12.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:12.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:18:12.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:18:12.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:18:12.952 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-05-08T21:18:12.132557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659514,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":544,"num_read_kb":392,"num_write":422,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1181982,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374832,"kb_used_data":13696,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124566992,"statfs":{"total":1151940427776,"available":1151556599808,"internally_reserved":0,"allocated":14024704,"data_stored":7746134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":67,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":574,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003899"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418982+0000","last_change":"2024-05-08T21:17:03.243903+0000","last_active":"2024-05-08T21:17:56.418982+0000","last_peered":"2024-05-08T21:17:56.418982+0000","last_clean":"2024-05-08T21:17:56.418982+0000","last_became_active":"2024-05-08T21:17:03.240821+0000","last_became_peered":"2024-05-08T21:17:03.240821+0000","last_unstale":"2024-05-08T21:17:56.418982+0000","last_undegraded":"2024-05-08T21:17:56.418982+0000","last_fullsized":"2024-05-08T21:17:56.418982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:01.878614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423327+0000","last_change":"2024-05-08T21:17:03.243119+0000","last_active":"2024-05-08T21:17:56.423327+0000","last_peered":"2024-05-08T21:17:56.423327+0000","last_clean":"2024-05-08T21:17:56.423327+0000","last_became_active":"2024-05-08T21:17:03.242772+0000","last_became_peered":"2024-05-08T21:17:03.242772+0000","last_unstale":"2024-05-08T21:17:56.423327+0000","last_undegraded":"2024-05-08T21:17:56.423327+0000","last_fullsized":"2024-05-08T21:17:56.423327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:20.550646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423287+0000","last_change":"2024-05-08T21:17:03.237843+0000","last_active":"2024-05-08T21:17:56.423287+0000","last_peered":"2024-05-08T21:17:56.423287+0000","last_clean":"2024-05-08T21:17:56.423287+0000","last_became_active":"2024-05-08T21:17:03.237567+0000","last_became_peered":"2024-05-08T21:17:03.237567+0000","last_unstale":"2024-05-08T21:17:56.423287+0000","last_undegraded":"2024-05-08T21:17:56.423287+0000","last_fullsized":"2024-05-08T21:17:56.423287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:23.856461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.430448+0000","last_change":"2024-05-08T21:17:57.430448+0000","last_active":"2024-05-08T21:17:57.430448+0000","last_peered":"2024-05-08T21:17:57.430448+0000","last_clean":"2024-05-08T21:17:57.430448+0000","last_became_active":"2024-05-08T21:17:57.430195+0000","last_became_peered":"2024-05-08T21:17:57.430195+0000","last_unstale":"2024-05-08T21:17:57.430448+0000","last_undegraded":"2024-05-08T21:17:57.430448+0000","last_fullsized":"2024-05-08T21:17:57.430448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:41.536610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416067+0000","last_change":"2024-05-08T21:17:03.252246+0000","last_active":"2024-05-08T21:17:56.416067+0000","last_peered":"2024-05-08T21:17:56.416067+0000","last_clean":"2024-05-08T21:17:56.416067+0000","last_became_active":"2024-05-08T21:17:03.251898+0000","last_became_peered":"2024-05-08T21:17:03.251898+0000","last_unstale":"2024-05-08T21:17:56.416067+0000","last_undegraded":"2024-05-08T21:17:56.416067+0000","last_fullsized":"2024-05-08T21:17:56.416067+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:08.996033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417704+0000","last_change":"2024-05-08T21:17:03.238422+0000","last_active":"2024-05-08T21:17:56.417704+0000","last_peered":"2024-05-08T21:17:56.417704+0000","last_clean":"2024-05-08T21:17:56.417704+0000","last_became_active":"2024-05-08T21:17:03.238096+0000","last_became_peered":"2024-05-08T21:17:03.238096+0000","last_unstale":"2024-05-08T21:17:56.417704+0000","last_undegraded":"2024-05-08T21:17:56.417704+0000","last_fullsized":"2024-05-08T21:17:56.417704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:54.635089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419866+0000","last_change":"2024-05-08T21:17:03.244315+0000","last_active":"2024-05-08T21:17:56.419866+0000","last_peered":"2024-05-08T21:17:56.419866+0000","last_clean":"2024-05-08T21:17:56.419866+0000","last_became_active":"2024-05-08T21:17:03.243939+0000","last_became_peered":"2024-05-08T21:17:03.243939+0000","last_unstale":"2024-05-08T21:17:56.419866+0000","last_undegraded":"2024-05-08T21:17:56.419866+0000","last_fullsized":"2024-05-08T21:17:56.419866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:24.371495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415520+0000","last_change":"2024-05-08T21:17:03.240144+0000","last_active":"2024-05-08T21:17:56.415520+0000","last_peered":"2024-05-08T21:17:56.415520+0000","last_clean":"2024-05-08T21:17:56.415520+0000","last_became_active":"2024-05-08T21:17:03.239493+0000","last_became_peered":"2024-05-08T21:17:03.239493+0000","last_unstale":"2024-05-08T21:17:56.415520+0000","last_undegraded":"2024-05-08T21:17:56.415520+0000","last_fullsized":"2024-05-08T21:17:56.415520+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:21.374357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424317+0000","last_change":"2024-05-08T21:17:03.256706+0000","last_active":"2024-05-08T21:17:56.424317+0000","last_peered":"2024-05-08T21:17:56.424317+0000","last_clean":"2024-05-08T21:17:56.424317+0000","last_became_active":"2024-05-08T21:17:03.255345+0000","last_became_peered":"2024-05-08T21:17:03.255345+0000","last_unstale":"2024-05-08T21:17:56.424317+0000","last_undegraded":"2024-05-08T21:17:56.424317+0000","last_fullsized":"2024-05-08T21:17:56.424317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:09:50.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.430334+0000","last_change":"2024-05-08T21:17:57.430334+0000","last_active":"2024-05-08T21:17:57.430334+0000","last_peered":"2024-05-08T21:17:57.430334+0000","last_clean":"2024-05-08T21:17:57.430334+0000","last_became_active":"2024-05-08T21:17:57.429924+0000","last_became_peered":"2024-05-08T21:17:57.429924+0000","last_unstale":"2024-05-08T21:17:57.430334+0000","last_undegraded":"2024-05-08T21:17:57.430334+0000","last_fullsized":"2024-05-08T21:17:57.430334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.901755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419810+0000","last_change":"2024-05-08T21:17:03.253996+0000","last_active":"2024-05-08T21:17:56.419810+0000","last_peered":"2024-05-08T21:17:56.419810+0000","last_clean":"2024-05-08T21:17:56.419810+0000","last_became_active":"2024-05-08T21:17:03.253686+0000","last_became_peered":"2024-05-08T21:17:03.253686+0000","last_unstale":"2024-05-08T21:17:56.419810+0000","last_undegraded":"2024-05-08T21:17:56.419810+0000","last_fullsized":"2024-05-08T21:17:56.419810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:17.995692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423092+0000","last_change":"2024-05-08T21:17:03.254357+0000","last_active":"2024-05-08T21:17:56.423092+0000","last_peered":"2024-05-08T21:17:56.423092+0000","last_clean":"2024-05-08T21:17:56.423092+0000","last_became_active":"2024-05-08T21:17:03.254013+0000","last_became_peered":"2024-05-08T21:17:03.254013+0000","last_unstale":"2024-05-08T21:17:56.423092+0000","last_undegraded":"2024-05-08T21:17:56.423092+0000","last_fullsized":"2024-05-08T21:17:56.423092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:21.480612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.413995+0000","last_change":"2024-05-08T21:17:03.244190+0000","last_active":"2024-05-08T21:17:56.413995+0000","last_peered":"2024-05-08T21:17:56.413995+0000","last_clean":"2024-05-08T21:17:56.413995+0000","last_became_active":"2024-05-08T21:17:03.243989+0000","last_became_peered":"2024-05-08T21:17:03.243989+0000","last_unstale":"2024-05-08T21:17:56.413995+0000","last_undegraded":"2024-05-08T21:17:56.413995+0000","last_fullsized":"2024-05-08T21:17:56.413995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:45.059343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422938+0000","last_change":"2024-05-08T21:17:03.242996+0000","last_active":"2024-05-08T21:17:56.422938+0000","last_peered":"2024-05-08T21:17:56.422938+0000","last_clean":"2024-05-08T21:17:56.422938+0000","last_became_active":"2024-05-08T21:17:03.242648+0000","last_became_peered":"2024-05-08T21:17:03.242648+0000","last_unstale":"2024-05-08T21:17:56.422938+0000","last_undegraded":"2024-05-08T21:17:56.422938+0000","last_fullsized":"2024-05-08T21:17:56.422938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:47:19.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422908+0000","last_change":"2024-05-08T21:17:03.244970+0000","last_active":"2024-05-08T21:17:56.422908+0000","last_peered":"2024-05-08T21:17:56.422908+0000","last_clean":"2024-05-08T21:17:56.422908+0000","last_became_active":"2024-05-08T21:17:03.244604+0000","last_became_peered":"2024-05-08T21:17:03.244604+0000","last_unstale":"2024-05-08T21:17:56.422908+0000","last_undegraded":"2024-05-08T21:17:56.422908+0000","last_fullsized":"2024-05-08T21:17:56.422908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:32.484155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422727+0000","last_change":"2024-05-08T21:17:03.254628+0000","last_active":"2024-05-08T21:17:56.422727+0000","last_peered":"2024-05-08T21:17:56.422727+0000","last_clean":"2024-05-08T21:17:56.422727+0000","last_became_active":"2024-05-08T21:17:03.254305+0000","last_became_peered":"2024-05-08T21:17:03.254305+0000","last_unstale":"2024-05-08T21:17:56.422727+0000","last_undegraded":"2024-05-08T21:17:56.422727+0000","last_fullsized":"2024-05-08T21:17:56.422727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:52.727821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418732+0000","last_change":"2024-05-08T21:17:03.243406+0000","last_active":"2024-05-08T21:17:56.418732+0000","last_peered":"2024-05-08T21:17:56.418732+0000","last_clean":"2024-05-08T21:17:56.418732+0000","last_became_active":"2024-05-08T21:17:03.240379+0000","last_became_peered":"2024-05-08T21:17:03.240379+0000","last_unstale":"2024-05-08T21:17:56.418732+0000","last_undegraded":"2024-05-08T21:17:56.418732+0000","last_fullsized":"2024-05-08T21:17:56.418732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:49.897459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420035+0000","last_change":"2024-05-08T21:17:03.239664+0000","last_active":"2024-05-08T21:17:56.420035+0000","last_peered":"2024-05-08T21:17:56.420035+0000","last_clean":"2024-05-08T21:17:56.420035+0000","last_became_active":"2024-05-08T21:17:03.239347+0000","last_became_peered":"2024-05-08T21:17:03.239347+0000","last_unstale":"2024-05-08T21:17:56.420035+0000","last_undegraded":"2024-05-08T21:17:56.420035+0000","last_fullsized":"2024-05-08T21:17:56.420035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:06:37.557835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423543+0000","last_change":"2024-05-08T21:17:03.249424+0000","last_active":"2024-05-08T21:17:56.423543+0000","last_peered":"2024-05-08T21:17:56.423543+0000","last_clean":"2024-05-08T21:17:56.423543+0000","last_became_active":"2024-05-08T21:17:03.248971+0000","last_became_peered":"2024-05-08T21:17:03.248971+0000","last_unstale":"2024-05-08T21:17:56.423543+0000","last_undegraded":"2024-05-08T21:17:56.423543+0000","last_fullsized":"2024-05-08T21:17:56.423543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:08.096762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422849+0000","last_change":"2024-05-08T21:17:07.272414+0000","last_active":"2024-05-08T21:17:56.422849+0000","last_peered":"2024-05-08T21:17:56.422849+0000","last_clean":"2024-05-08T21:17:56.422849+0000","last_became_active":"2024-05-08T21:17:07.271660+0000","last_became_peered":"2024-05-08T21:17:07.271660+0000","last_unstale":"2024-05-08T21:17:56.422849+0000","last_undegraded":"2024-05-08T21:17:56.422849+0000","last_fullsized":"2024-05-08T21:17:56.422849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:15.697910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:07.488173+0000","last_change":"2024-05-08T21:17:57.483017+0000","last_active":"2024-05-08T21:18:07.488173+0000","last_peered":"2024-05-08T21:18:07.488173+0000","last_clean":"2024-05-08T21:18:07.488173+0000","last_became_active":"2024-05-08T21:17:57.473420+0000","last_became_peered":"2024-05-08T21:17:57.473420+0000","last_unstale":"2024-05-08T21:18:07.488173+0000","last_undegraded":"2024-05-08T21:18:07.488173+0000","last_fullsized":"2024-05-08T21:18:07.488173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:42.727117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417325+0000","last_change":"2024-05-08T21:17:03.240780+0000","last_active":"2024-05-08T21:17:56.417325+0000","last_peered":"2024-05-08T21:17:56.417325+0000","last_clean":"2024-05-08T21:17:56.417325+0000","last_became_active":"2024-05-08T21:17:03.240472+0000","last_became_peered":"2024-05-08T21:17:03.240472+0000","last_unstale":"2024-05-08T21:17:56.417325+0000","last_undegraded":"2024-05-08T21:17:56.417325+0000","last_fullsized":"2024-05-08T21:17:56.417325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.089760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.421952+0000","last_change":"2024-05-08T21:17:07.272308+0000","last_active":"2024-05-08T21:17:56.421952+0000","last_peered":"2024-05-08T21:17:56.421952+0000","last_clean":"2024-05-08T21:17:56.421952+0000","last_became_active":"2024-05-08T21:17:07.271869+0000","last_became_peered":"2024-05-08T21:17:07.271869+0000","last_unstale":"2024-05-08T21:17:56.421952+0000","last_undegraded":"2024-05-08T21:17:56.421952+0000","last_fullsized":"2024-05-08T21:17:56.421952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:31.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417727+0000","last_change":"2024-05-08T21:17:02.736446+0000","last_active":"2024-05-08T21:17:56.417727+0000","last_peered":"2024-05-08T21:17:56.417727+0000","last_clean":"2024-05-08T21:17:56.417727+0000","last_became_active":"2024-05-08T21:17:02.736187+0000","last_became_peered":"2024-05-08T21:17:02.736187+0000","last_unstale":"2024-05-08T21:17:56.417727+0000","last_undegraded":"2024-05-08T21:17:56.417727+0000","last_fullsized":"2024-05-08T21:17:56.417727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:29.302558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423206+0000","last_change":"2024-05-08T21:17:03.251896+0000","last_active":"2024-05-08T21:17:56.423206+0000","last_peered":"2024-05-08T21:17:56.423206+0000","last_clean":"2024-05-08T21:17:56.423206+0000","last_became_active":"2024-05-08T21:17:03.251495+0000","last_became_peered":"2024-05-08T21:17:03.251495+0000","last_unstale":"2024-05-08T21:17:56.423206+0000","last_undegraded":"2024-05-08T21:17:56.423206+0000","last_fullsized":"2024-05-08T21:17:56.423206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:50.774848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422223+0000","last_change":"2024-05-08T21:17:07.272150+0000","last_active":"2024-05-08T21:17:56.422223+0000","last_peered":"2024-05-08T21:17:56.422223+0000","last_clean":"2024-05-08T21:17:56.422223+0000","last_became_active":"2024-05-08T21:17:07.271727+0000","last_became_peered":"2024-05-08T21:17:07.271727+0000","last_unstale":"2024-05-08T21:17:56.422223+0000","last_undegraded":"2024-05-08T21:17:56.422223+0000","last_fullsized":"2024-05-08T21:17:56.422223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.058277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416860+0000","last_change":"2024-05-08T21:17:02.733611+0000","last_active":"2024-05-08T21:17:56.416860+0000","last_peered":"2024-05-08T21:17:56.416860+0000","last_clean":"2024-05-08T21:17:56.416860+0000","last_became_active":"2024-05-08T21:17:02.733306+0000","last_became_peered":"2024-05-08T21:17:02.733306+0000","last_unstale":"2024-05-08T21:17:56.416860+0000","last_undegraded":"2024-05-08T21:17:56.416860+0000","last_fullsized":"2024-05-08T21:17:56.416860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:33.304753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423294+0000","last_change":"2024-05-08T21:17:03.248480+0000","last_active":"2024-05-08T21:17:56.423294+0000","last_peered":"2024-05-08T21:17:56.423294+0000","last_clean":"2024-05-08T21:17:56.423294+0000","last_became_active":"2024-05-08T21:17:03.248126+0000","last_became_peered":"2024-05-08T21:17:03.248126+0000","last_unstale":"2024-05-08T21:17:56.423294+0000","last_undegraded":"2024-05-08T21:17:56.423294+0000","last_fullsized":"2024-05-08T21:17:56.423294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:08.785761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422472+0000","last_change":"2024-05-08T21:17:07.272344+0000","last_active":"2024-05-08T21:17:56.422472+0000","last_peered":"2024-05-08T21:17:56.422472+0000","last_clean":"2024-05-08T21:17:56.422472+0000","last_became_active":"2024-05-08T21:17:07.271629+0000","last_became_peered":"2024-05-08T21:17:07.271629+0000","last_unstale":"2024-05-08T21:17:56.422472+0000","last_undegraded":"2024-05-08T21:17:56.422472+0000","last_fullsized":"2024-05-08T21:17:56.422472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:15.052748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416295+0000","last_change":"2024-05-08T21:17:02.730364+0000","last_active":"2024-05-08T21:17:56.416295+0000","last_peered":"2024-05-08T21:17:56.416295+0000","last_clean":"2024-05-08T21:17:56.416295+0000","last_became_active":"2024-05-08T21:17:02.728854+0000","last_became_peered":"2024-05-08T21:17:02.728854+0000","last_unstale":"2024-05-08T21:17:56.416295+0000","last_undegraded":"2024-05-08T21:17:56.416295+0000","last_fullsized":"2024-05-08T21:17:56.416295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:09.552016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419025+0000","last_change":"2024-05-08T21:17:03.252713+0000","last_active":"2024-05-08T21:17:56.419025+0000","last_peered":"2024-05-08T21:17:56.419025+0000","last_clean":"2024-05-08T21:17:56.419025+0000","last_became_active":"2024-05-08T21:17:03.252388+0000","last_became_peered":"2024-05-08T21:17:03.252388+0000","last_unstale":"2024-05-08T21:17:56.419025+0000","last_undegraded":"2024-05-08T21:17:56.419025+0000","last_fullsized":"2024-05-08T21:17:56.419025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:09.114064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423183+0000","last_change":"2024-05-08T21:17:07.261112+0000","last_active":"2024-05-08T21:17:56.423183+0000","last_peered":"2024-05-08T21:17:56.423183+0000","last_clean":"2024-05-08T21:17:56.423183+0000","last_became_active":"2024-05-08T21:17:07.260713+0000","last_became_peered":"2024-05-08T21:17:07.260713+0000","last_unstale":"2024-05-08T21:17:56.423183+0000","last_undegraded":"2024-05-08T21:17:56.423183+0000","last_fullsized":"2024-05-08T21:17:56.423183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:37.262113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:09.451541+0000","last_change":"2024-05-08T21:17:57.436812+0000","last_active":"2024-05-08T21:18:09.451541+0000","last_peered":"2024-05-08T21:18:09.451541+0000","last_clean":"2024-05-08T21:18:09.451541+0000","last_became_active":"2024-05-08T21:17:57.432431+0000","last_became_peered":"2024-05-08T21:17:57.432431+0000","last_unstale":"2024-05-08T21:18:09.451541+0000","last_undegraded":"2024-05-08T21:18:09.451541+0000","last_fullsized":"2024-05-08T21:18:09.451541+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:46.062273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":574,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417176+0000","last_change":"2024-05-08T21:17:03.251245+0000","last_active":"2024-05-08T21:17:56.417176+0000","last_peered":"2024-05-08T21:17:56.417176+0000","last_clean":"2024-05-08T21:17:56.417176+0000","last_became_active":"2024-05-08T21:17:03.250296+0000","last_became_peered":"2024-05-08T21:17:03.250296+0000","last_unstale":"2024-05-08T21:17:56.417176+0000","last_undegraded":"2024-05-08T21:17:56.417176+0000","last_fullsized":"2024-05-08T21:17:56.417176+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:28.391927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423437+0000","last_change":"2024-05-08T21:17:07.266752+0000","last_active":"2024-05-08T21:17:56.423437+0000","last_peered":"2024-05-08T21:17:56.423437+0000","last_clean":"2024-05-08T21:17:56.423437+0000","last_became_active":"2024-05-08T21:17:07.266367+0000","last_became_peered":"2024-05-08T21:17:07.266367+0000","last_unstale":"2024-05-08T21:17:56.423437+0000","last_undegraded":"2024-05-08T21:17:56.423437+0000","last_fullsized":"2024-05-08T21:17:56.423437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:49.874073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417579+0000","last_change":"2024-05-08T21:17:02.728113+0000","last_active":"2024-05-08T21:17:56.417579+0000","last_peered":"2024-05-08T21:17:56.417579+0000","last_clean":"2024-05-08T21:17:56.417579+0000","last_became_active":"2024-05-08T21:17:02.727754+0000","last_became_peered":"2024-05-08T21:17:02.727754+0000","last_unstale":"2024-05-08T21:17:56.417579+0000","last_undegraded":"2024-05-08T21:17:56.417579+0000","last_fullsized":"2024-05-08T21:17:56.417579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:53.789218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418802+0000","last_change":"2024-05-08T21:17:03.245482+0000","last_active":"2024-05-08T21:17:56.418802+0000","last_peered":"2024-05-08T21:17:56.418802+0000","last_clean":"2024-05-08T21:17:56.418802+0000","last_became_active":"2024-05-08T21:17:03.245144+0000","last_became_peered":"2024-05-08T21:17:03.245144+0000","last_unstale":"2024-05-08T21:17:56.418802+0000","last_undegraded":"2024-05-08T21:17:56.418802+0000","last_fullsized":"2024-05-08T21:17:56.418802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:04.051234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418636+0000","last_change":"2024-05-08T21:17:07.268505+0000","last_active":"2024-05-08T21:17:56.418636+0000","last_peered":"2024-05-08T21:17:56.418636+0000","last_clean":"2024-05-08T21:17:56.418636+0000","last_became_active":"2024-05-08T21:17:07.268157+0000","last_became_peered":"2024-05-08T21:17:07.268157+0000","last_unstale":"2024-05-08T21:17:56.418636+0000","last_undegraded":"2024-05-08T21:17:56.418636+0000","last_fullsized":"2024-05-08T21:17:56.418636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:55.220521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412606+0000","last_change":"2024-05-08T21:17:02.727264+0000","last_active":"2024-05-08T21:17:56.412606+0000","last_peered":"2024-05-08T21:17:56.412606+0000","last_clean":"2024-05-08T21:17:56.412606+0000","last_became_active":"2024-05-08T21:17:02.726922+0000","last_became_peered":"2024-05-08T21:17:02.726922+0000","last_unstale":"2024-05-08T21:17:56.412606+0000","last_undegraded":"2024-05-08T21:17:56.412606+0000","last_fullsized":"2024-05-08T21:17:56.412606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:27.410231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415635+0000","last_change":"2024-05-08T21:17:03.233910+0000","last_active":"2024-05-08T21:17:56.415635+0000","last_peered":"2024-05-08T21:17:56.415635+0000","last_clean":"2024-05-08T21:17:56.415635+0000","last_became_active":"2024-05-08T21:17:03.233562+0000","last_became_peered":"2024-05-08T21:17:03.233562+0000","last_unstale":"2024-05-08T21:17:56.415635+0000","last_undegraded":"2024-05-08T21:17:56.415635+0000","last_fullsized":"2024-05-08T21:17:56.415635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:40.309400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423244+0000","last_change":"2024-05-08T21:17:07.264992+0000","last_active":"2024-05-08T21:17:56.423244+0000","last_peered":"2024-05-08T21:17:56.423244+0000","last_clean":"2024-05-08T21:17:56.423244+0000","last_became_active":"2024-05-08T21:17:07.264498+0000","last_became_peered":"2024-05-08T21:17:07.264498+0000","last_unstale":"2024-05-08T21:17:56.423244+0000","last_undegraded":"2024-05-08T21:17:56.423244+0000","last_fullsized":"2024-05-08T21:17:56.423244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:23:26.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418593+0000","last_change":"2024-05-08T21:17:02.725531+0000","last_active":"2024-05-08T21:17:56.418593+0000","last_peered":"2024-05-08T21:17:56.418593+0000","last_clean":"2024-05-08T21:17:56.418593+0000","last_became_active":"2024-05-08T21:17:02.725235+0000","last_became_peered":"2024-05-08T21:17:02.725235+0000","last_unstale":"2024-05-08T21:17:56.418593+0000","last_undegraded":"2024-05-08T21:17:56.418593+0000","last_fullsized":"2024-05-08T21:17:56.418593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:27.265590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419530+0000","last_change":"2024-05-08T21:17:03.236393+0000","last_active":"2024-05-08T21:17:56.419530+0000","last_peered":"2024-05-08T21:17:56.419530+0000","last_clean":"2024-05-08T21:17:56.419530+0000","last_became_active":"2024-05-08T21:17:03.236092+0000","last_became_peered":"2024-05-08T21:17:03.236092+0000","last_unstale":"2024-05-08T21:17:56.419530+0000","last_undegraded":"2024-05-08T21:17:56.419530+0000","last_fullsized":"2024-05-08T21:17:56.419530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:58.232669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417799+0000","last_change":"2024-05-08T21:17:07.271374+0000","last_active":"2024-05-08T21:17:56.417799+0000","last_peered":"2024-05-08T21:17:56.417799+0000","last_clean":"2024-05-08T21:17:56.417799+0000","last_became_active":"2024-05-08T21:17:07.270778+0000","last_became_peered":"2024-05-08T21:17:07.270778+0000","last_unstale":"2024-05-08T21:17:56.417799+0000","last_undegraded":"2024-05-08T21:17:56.417799+0000","last_fullsized":"2024-05-08T21:17:56.417799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:50.212213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":102,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:10.379619+0000","last_change":"2024-05-08T21:17:02.725624+0000","last_active":"2024-05-08T21:18:10.379619+0000","last_peered":"2024-05-08T21:18:10.379619+0000","last_clean":"2024-05-08T21:18:10.379619+0000","last_became_active":"2024-05-08T21:17:02.725367+0000","last_became_peered":"2024-05-08T21:17:02.725367+0000","last_unstale":"2024-05-08T21:18:10.379619+0000","last_undegraded":"2024-05-08T21:18:10.379619+0000","last_fullsized":"2024-05-08T21:18:10.379619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:19.573931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423993+0000","last_change":"2024-05-08T21:17:03.248729+0000","last_active":"2024-05-08T21:17:56.423993+0000","last_peered":"2024-05-08T21:17:56.423993+0000","last_clean":"2024-05-08T21:17:56.423993+0000","last_became_active":"2024-05-08T21:17:03.247899+0000","last_became_peered":"2024-05-08T21:17:03.247899+0000","last_unstale":"2024-05-08T21:17:56.423993+0000","last_undegraded":"2024-05-08T21:17:56.423993+0000","last_fullsized":"2024-05-08T21:17:56.423993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:27:57.012249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419291+0000","last_change":"2024-05-08T21:17:07.261257+0000","last_active":"2024-05-08T21:17:56.419291+0000","last_peered":"2024-05-08T21:17:56.419291+0000","last_clean":"2024-05-08T21:17:56.419291+0000","last_became_active":"2024-05-08T21:17:07.260785+0000","last_became_peered":"2024-05-08T21:17:07.260785+0000","last_unstale":"2024-05-08T21:17:56.419291+0000","last_undegraded":"2024-05-08T21:17:56.419291+0000","last_fullsized":"2024-05-08T21:17:56.419291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:58.469134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419901+0000","last_change":"2024-05-08T21:17:02.729569+0000","last_active":"2024-05-08T21:17:56.419901+0000","last_peered":"2024-05-08T21:17:56.419901+0000","last_clean":"2024-05-08T21:17:56.419901+0000","last_became_active":"2024-05-08T21:17:02.729096+0000","last_became_peered":"2024-05-08T21:17:02.729096+0000","last_unstale":"2024-05-08T21:17:56.419901+0000","last_undegraded":"2024-05-08T21:17:56.419901+0000","last_fullsized":"2024-05-08T21:17:56.419901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:30.668169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423406+0000","last_change":"2024-05-08T21:17:03.246387+0000","last_active":"2024-05-08T21:17:56.423406+0000","last_peered":"2024-05-08T21:17:56.423406+0000","last_clean":"2024-05-08T21:17:56.423406+0000","last_became_active":"2024-05-08T21:17:03.244658+0000","last_became_peered":"2024-05-08T21:17:03.244658+0000","last_unstale":"2024-05-08T21:17:56.423406+0000","last_undegraded":"2024-05-08T21:17:56.423406+0000","last_fullsized":"2024-05-08T21:17:56.423406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:02.602080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.473101+0000","last_change":"2024-05-08T21:17:57.473101+0000","last_active":"2024-05-08T21:17:57.473101+0000","last_peered":"2024-05-08T21:17:57.473101+0000","last_clean":"2024-05-08T21:17:57.473101+0000","last_became_active":"2024-05-08T21:17:57.472563+0000","last_became_peered":"2024-05-08T21:17:57.472563+0000","last_unstale":"2024-05-08T21:17:57.473101+0000","last_undegraded":"2024-05-08T21:17:57.473101+0000","last_fullsized":"2024-05-08T21:17:57.473101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:23.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418913+0000","last_change":"2024-05-08T21:17:02.732738+0000","last_active":"2024-05-08T21:17:56.418913+0000","last_peered":"2024-05-08T21:17:56.418913+0000","last_clean":"2024-05-08T21:17:56.418913+0000","last_became_active":"2024-05-08T21:17:02.731733+0000","last_became_peered":"2024-05-08T21:17:02.731733+0000","last_unstale":"2024-05-08T21:17:56.418913+0000","last_undegraded":"2024-05-08T21:17:56.418913+0000","last_fullsized":"2024-05-08T21:17:56.418913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:24.201171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423479+0000","last_change":"2024-05-08T21:17:03.254405+0000","last_active":"2024-05-08T21:17:56.423479+0000","last_peered":"2024-05-08T21:17:56.423479+0000","last_clean":"2024-05-08T21:17:56.423479+0000","last_became_active":"2024-05-08T21:17:03.252137+0000","last_became_peered":"2024-05-08T21:17:03.252137+0000","last_unstale":"2024-05-08T21:17:56.423479+0000","last_undegraded":"2024-05-08T21:17:56.423479+0000","last_fullsized":"2024-05-08T21:17:56.423479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:15.472225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.431574+0000","last_change":"2024-05-08T21:17:57.431091+0000","last_active":"2024-05-08T21:17:57.431574+0000","last_peered":"2024-05-08T21:17:57.431574+0000","last_clean":"2024-05-08T21:17:57.431574+0000","last_became_active":"2024-05-08T21:17:57.430668+0000","last_became_peered":"2024-05-08T21:17:57.430668+0000","last_unstale":"2024-05-08T21:17:57.431574+0000","last_undegraded":"2024-05-08T21:17:57.431574+0000","last_fullsized":"2024-05-08T21:17:57.431574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:33.208801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"86'36","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:10.452461+0000","last_change":"2024-05-08T21:17:02.723975+0000","last_active":"2024-05-08T21:18:10.452461+0000","last_peered":"2024-05-08T21:18:10.452461+0000","last_clean":"2024-05-08T21:18:10.452461+0000","last_became_active":"2024-05-08T21:17:02.723631+0000","last_became_peered":"2024-05-08T21:17:02.723631+0000","last_unstale":"2024-05-08T21:18:10.452461+0000","last_undegraded":"2024-05-08T21:18:10.452461+0000","last_fullsized":"2024-05-08T21:18:10.452461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:17.676007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62289,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":88,"num_read_kb":104,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416534+0000","last_change":"2024-05-08T21:17:03.251570+0000","last_active":"2024-05-08T21:17:56.416534+0000","last_peered":"2024-05-08T21:17:56.416534+0000","last_clean":"2024-05-08T21:17:56.416534+0000","last_became_active":"2024-05-08T21:17:03.251134+0000","last_became_peered":"2024-05-08T21:17:03.251134+0000","last_unstale":"2024-05-08T21:17:56.416534+0000","last_undegraded":"2024-05-08T21:17:56.416534+0000","last_fullsized":"2024-05-08T21:17:56.416534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:28:09.339958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419635+0000","last_change":"2024-05-08T21:17:07.272051+0000","last_active":"2024-05-08T21:17:56.419635+0000","last_peered":"2024-05-08T21:17:56.419635+0000","last_clean":"2024-05-08T21:17:56.419635+0000","last_became_active":"2024-05-08T21:17:07.271616+0000","last_became_peered":"2024-05-08T21:17:07.271616+0000","last_unstale":"2024-05-08T21:17:56.419635+0000","last_undegraded":"2024-05-08T21:17:56.419635+0000","last_fullsized":"2024-05-08T21:17:56.419635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.364731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423259+0000","last_change":"2024-05-08T21:17:02.736000+0000","last_active":"2024-05-08T21:17:56.423259+0000","last_peered":"2024-05-08T21:17:56.423259+0000","last_clean":"2024-05-08T21:17:56.423259+0000","last_became_active":"2024-05-08T21:17:02.735629+0000","last_became_peered":"2024-05-08T21:17:02.735629+0000","last_unstale":"2024-05-08T21:17:56.423259+0000","last_undegraded":"2024-05-08T21:17:56.423259+0000","last_fullsized":"2024-05-08T21:17:56.423259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:21.842135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412729+0000","last_change":"2024-05-08T21:17:03.237144+0000","last_active":"2024-05-08T21:17:56.412729+0000","last_peered":"2024-05-08T21:17:56.412729+0000","last_clean":"2024-05-08T21:17:56.412729+0000","last_became_active":"2024-05-08T21:17:03.236792+0000","last_became_peered":"2024-05-08T21:17:03.236792+0000","last_unstale":"2024-05-08T21:17:56.412729+0000","last_undegraded":"2024-05-08T21:17:56.412729+0000","last_fullsized":"2024-05-08T21:17:56.412729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:04.520986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.627656+0000","last_change":"2024-05-08T21:17:57.627656+0000","last_active":"2024-05-08T21:17:57.627656+0000","last_peered":"2024-05-08T21:17:57.627656+0000","last_clean":"2024-05-08T21:17:57.627656+0000","last_became_active":"2024-05-08T21:17:57.627108+0000","last_became_peered":"2024-05-08T21:17:57.627108+0000","last_unstale":"2024-05-08T21:17:57.627656+0000","last_undegraded":"2024-05-08T21:17:57.627656+0000","last_fullsized":"2024-05-08T21:17:57.627656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:24.025145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419802+0000","last_change":"2024-05-08T21:17:02.727748+0000","last_active":"2024-05-08T21:17:56.419802+0000","last_peered":"2024-05-08T21:17:56.419802+0000","last_clean":"2024-05-08T21:17:56.419802+0000","last_became_active":"2024-05-08T21:17:02.727444+0000","last_became_peered":"2024-05-08T21:17:02.727444+0000","last_unstale":"2024-05-08T21:17:56.419802+0000","last_undegraded":"2024-05-08T21:17:56.419802+0000","last_fullsized":"2024-05-08T21:17:56.419802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:47.939235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420245+0000","last_change":"2024-05-08T21:17:03.247185+0000","last_active":"2024-05-08T21:17:56.420245+0000","last_peered":"2024-05-08T21:17:56.420245+0000","last_clean":"2024-05-08T21:17:56.420245+0000","last_became_active":"2024-05-08T21:17:03.246880+0000","last_became_peered":"2024-05-08T21:17:03.246880+0000","last_unstale":"2024-05-08T21:17:56.420245+0000","last_undegraded":"2024-05-08T21:17:56.420245+0000","last_fullsized":"2024-05-08T21:17:56.420245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:38.978353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420018+0000","last_change":"2024-05-08T21:17:07.273710+0000","last_active":"2024-05-08T21:17:56.420018+0000","last_peered":"2024-05-08T21:17:56.420018+0000","last_clean":"2024-05-08T21:17:56.420018+0000","last_became_active":"2024-05-08T21:17:07.273119+0000","last_became_peered":"2024-05-08T21:17:07.273119+0000","last_unstale":"2024-05-08T21:17:56.420018+0000","last_undegraded":"2024-05-08T21:17:56.420018+0000","last_fullsized":"2024-05-08T21:17:56.420018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:30.589440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:06.360264+0000","last_change":"2024-05-08T21:17:02.725508+0000","last_active":"2024-05-08T21:18:06.360264+0000","last_peered":"2024-05-08T21:18:06.360264+0000","last_clean":"2024-05-08T21:18:06.360264+0000","last_became_active":"2024-05-08T21:17:02.725107+0000","last_became_peered":"2024-05-08T21:17:02.725107+0000","last_unstale":"2024-05-08T21:18:06.360264+0000","last_undegraded":"2024-05-08T21:18:06.360264+0000","last_fullsized":"2024-05-08T21:18:06.360264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:13.503702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420302+0000","last_change":"2024-05-08T21:17:03.245372+0000","last_active":"2024-05-08T21:17:56.420302+0000","last_peered":"2024-05-08T21:17:56.420302+0000","last_clean":"2024-05-08T21:17:56.420302+0000","last_became_active":"2024-05-08T21:17:03.245054+0000","last_became_peered":"2024-05-08T21:17:03.245054+0000","last_unstale":"2024-05-08T21:17:56.420302+0000","last_undegraded":"2024-05-08T21:17:56.420302+0000","last_fullsized":"2024-05-08T21:17:56.420302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:30:25.900846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414821+0000","last_change":"2024-05-08T21:17:07.272081+0000","last_active":"2024-05-08T21:17:56.414821+0000","last_peered":"2024-05-08T21:17:56.414821+0000","last_clean":"2024-05-08T21:17:56.414821+0000","last_became_active":"2024-05-08T21:17:07.271560+0000","last_became_peered":"2024-05-08T21:17:07.271560+0000","last_unstale":"2024-05-08T21:17:56.414821+0000","last_undegraded":"2024-05-08T21:17:56.414821+0000","last_fullsized":"2024-05-08T21:17:56.414821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:00.490798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416027+0000","last_change":"2024-05-08T21:17:02.734420+0000","last_active":"2024-05-08T21:17:56.416027+0000","last_peered":"2024-05-08T21:17:56.416027+0000","last_clean":"2024-05-08T21:17:56.416027+0000","last_became_active":"2024-05-08T21:17:02.734071+0000","last_became_peered":"2024-05-08T21:17:02.734071+0000","last_unstale":"2024-05-08T21:17:56.416027+0000","last_undegraded":"2024-05-08T21:17:56.416027+0000","last_fullsized":"2024-05-08T21:17:56.416027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:28.537464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419127+0000","last_change":"2024-05-08T21:17:03.250706+0000","last_active":"2024-05-08T21:17:56.419127+0000","last_peered":"2024-05-08T21:17:56.419127+0000","last_clean":"2024-05-08T21:17:56.419127+0000","last_became_active":"2024-05-08T21:17:03.250408+0000","last_became_peered":"2024-05-08T21:17:03.250408+0000","last_unstale":"2024-05-08T21:17:56.419127+0000","last_undegraded":"2024-05-08T21:17:56.419127+0000","last_fullsized":"2024-05-08T21:17:56.419127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:43.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418530+0000","last_change":"2024-05-08T21:17:07.272066+0000","last_active":"2024-05-08T21:17:56.418530+0000","last_peered":"2024-05-08T21:17:56.418530+0000","last_clean":"2024-05-08T21:17:56.418530+0000","last_became_active":"2024-05-08T21:17:07.271628+0000","last_became_peered":"2024-05-08T21:17:07.271628+0000","last_unstale":"2024-05-08T21:17:56.418530+0000","last_undegraded":"2024-05-08T21:17:56.418530+0000","last_fullsized":"2024-05-08T21:17:56.418530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:27.731995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416588+0000","last_change":"2024-05-08T21:17:02.728464+0000","last_active":"2024-05-08T21:17:56.416588+0000","last_peered":"2024-05-08T21:17:56.416588+0000","last_clean":"2024-05-08T21:17:56.416588+0000","last_became_active":"2024-05-08T21:17:02.727914+0000","last_became_peered":"2024-05-08T21:17:02.727914+0000","last_unstale":"2024-05-08T21:17:56.416588+0000","last_undegraded":"2024-05-08T21:17:56.416588+0000","last_fullsized":"2024-05-08T21:17:56.416588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:55.948661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418448+0000","last_change":"2024-05-08T21:17:03.245578+0000","last_active":"2024-05-08T21:17:56.418448+0000","last_peered":"2024-05-08T21:17:56.418448+0000","last_clean":"2024-05-08T21:17:56.418448+0000","last_became_active":"2024-05-08T21:17:03.245256+0000","last_became_peered":"2024-05-08T21:17:03.245256+0000","last_unstale":"2024-05-08T21:17:56.418448+0000","last_undegraded":"2024-05-08T21:17:56.418448+0000","last_fullsized":"2024-05-08T21:17:56.418448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417048+0000","last_change":"2024-05-08T21:17:07.439525+0000","last_active":"2024-05-08T21:17:56.417048+0000","last_peered":"2024-05-08T21:17:56.417048+0000","last_clean":"2024-05-08T21:17:56.417048+0000","last_became_active":"2024-05-08T21:17:07.438987+0000","last_became_peered":"2024-05-08T21:17:07.438987+0000","last_unstale":"2024-05-08T21:17:56.417048+0000","last_undegraded":"2024-05-08T21:17:56.417048+0000","last_fullsized":"2024-05-08T21:17:56.417048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:57:45.373981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.432999+0000","last_change":"2024-05-08T21:17:57.432999+0000","last_active":"2024-05-08T21:17:57.432999+0000","last_peered":"2024-05-08T21:17:57.432999+0000","last_clean":"2024-05-08T21:17:57.432999+0000","last_became_active":"2024-05-08T21:17:57.432567+0000","last_became_peered":"2024-05-08T21:17:57.432567+0000","last_unstale":"2024-05-08T21:17:57.432999+0000","last_undegraded":"2024-05-08T21:17:57.432999+0000","last_fullsized":"2024-05-08T21:17:57.432999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:33:00.188046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417907+0000","last_change":"2024-05-08T21:17:03.235556+0000","last_active":"2024-05-08T21:17:56.417907+0000","last_peered":"2024-05-08T21:17:56.417907+0000","last_clean":"2024-05-08T21:17:56.417907+0000","last_became_active":"2024-05-08T21:17:03.235214+0000","last_became_peered":"2024-05-08T21:17:03.235214+0000","last_unstale":"2024-05-08T21:17:56.417907+0000","last_undegraded":"2024-05-08T21:17:56.417907+0000","last_fullsized":"2024-05-08T21:17:56.417907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:45.042596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414845+0000","last_change":"2024-05-08T21:17:07.266080+0000","last_active":"2024-05-08T21:17:56.414845+0000","last_peered":"2024-05-08T21:17:56.414845+0000","last_clean":"2024-05-08T21:17:56.414845+0000","last_became_active":"2024-05-08T21:17:07.265605+0000","last_became_peered":"2024-05-08T21:17:07.265605+0000","last_unstale":"2024-05-08T21:17:56.414845+0000","last_undegraded":"2024-05-08T21:17:56.414845+0000","last_fullsized":"2024-05-08T21:17:56.414845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:49:48.387170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":63,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:09.491615+0000","last_change":"2024-05-08T21:17:02.728743+0000","last_active":"2024-05-08T21:18:09.491615+0000","last_peered":"2024-05-08T21:18:09.491615+0000","last_clean":"2024-05-08T21:18:09.491615+0000","last_became_active":"2024-05-08T21:17:02.728330+0000","last_became_peered":"2024-05-08T21:17:02.728330+0000","last_unstale":"2024-05-08T21:18:09.491615+0000","last_undegraded":"2024-05-08T21:18:09.491615+0000","last_fullsized":"2024-05-08T21:18:09.491615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:43.865117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417780+0000","last_change":"2024-05-08T21:17:03.245672+0000","last_active":"2024-05-08T21:17:56.417780+0000","last_peered":"2024-05-08T21:17:56.417780+0000","last_clean":"2024-05-08T21:17:56.417780+0000","last_became_active":"2024-05-08T21:17:03.245318+0000","last_became_peered":"2024-05-08T21:17:03.245318+0000","last_unstale":"2024-05-08T21:17:56.417780+0000","last_undegraded":"2024-05-08T21:17:56.417780+0000","last_fullsized":"2024-05-08T21:17:56.417780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:43.030011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414114+0000","last_change":"2024-05-08T21:17:07.439289+0000","last_active":"2024-05-08T21:17:56.414114+0000","last_peered":"2024-05-08T21:17:56.414114+0000","last_clean":"2024-05-08T21:17:56.414114+0000","last_became_active":"2024-05-08T21:17:07.438874+0000","last_became_peered":"2024-05-08T21:17:07.438874+0000","last_unstale":"2024-05-08T21:17:56.414114+0000","last_undegraded":"2024-05-08T21:17:56.414114+0000","last_fullsized":"2024-05-08T21:17:56.414114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:14.820087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419309+0000","last_change":"2024-05-08T21:17:03.247081+0000","last_active":"2024-05-08T21:17:56.419309+0000","last_peered":"2024-05-08T21:17:56.419309+0000","last_clean":"2024-05-08T21:17:56.419309+0000","last_became_active":"2024-05-08T21:17:03.246418+0000","last_became_peered":"2024-05-08T21:17:03.246418+0000","last_unstale":"2024-05-08T21:17:56.419309+0000","last_undegraded":"2024-05-08T21:17:56.419309+0000","last_fullsized":"2024-05-08T21:17:56.419309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:03.993770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":62,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:07.361065+0000","last_change":"2024-05-08T21:17:02.730764+0000","last_active":"2024-05-08T21:18:07.361065+0000","last_peered":"2024-05-08T21:18:07.361065+0000","last_clean":"2024-05-08T21:18:07.361065+0000","last_became_active":"2024-05-08T21:17:02.730494+0000","last_became_peered":"2024-05-08T21:17:02.730494+0000","last_unstale":"2024-05-08T21:18:07.361065+0000","last_undegraded":"2024-05-08T21:18:07.361065+0000","last_fullsized":"2024-05-08T21:18:07.361065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:50.070368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419669+0000","last_change":"2024-05-08T21:17:07.266424+0000","last_active":"2024-05-08T21:17:56.419669+0000","last_peered":"2024-05-08T21:17:56.419669+0000","last_clean":"2024-05-08T21:17:56.419669+0000","last_became_active":"2024-05-08T21:17:07.266144+0000","last_became_peered":"2024-05-08T21:17:07.266144+0000","last_unstale":"2024-05-08T21:17:56.419669+0000","last_undegraded":"2024-05-08T21:17:56.419669+0000","last_fullsized":"2024-05-08T21:17:56.419669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:22.285749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417254+0000","last_change":"2024-05-08T21:17:03.250461+0000","last_active":"2024-05-08T21:17:56.417254+0000","last_peered":"2024-05-08T21:17:56.417254+0000","last_clean":"2024-05-08T21:17:56.417254+0000","last_became_active":"2024-05-08T21:17:03.249974+0000","last_became_peered":"2024-05-08T21:17:03.249974+0000","last_unstale":"2024-05-08T21:17:56.417254+0000","last_undegraded":"2024-05-08T21:17:56.417254+0000","last_fullsized":"2024-05-08T21:17:56.417254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:25:58.759668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422681+0000","last_change":"2024-05-08T21:17:02.725648+0000","last_active":"2024-05-08T21:17:56.422681+0000","last_peered":"2024-05-08T21:17:56.422681+0000","last_clean":"2024-05-08T21:17:56.422681+0000","last_became_active":"2024-05-08T21:17:02.725297+0000","last_became_peered":"2024-05-08T21:17:02.725297+0000","last_unstale":"2024-05-08T21:17:56.422681+0000","last_undegraded":"2024-05-08T21:17:56.422681+0000","last_fullsized":"2024-05-08T21:17:56.422681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:41.462830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":328,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420115+0000","last_change":"2024-05-08T21:16:15.194478+0000","last_active":"2024-05-08T21:17:56.420115+0000","last_peered":"2024-05-08T21:17:56.420115+0000","last_clean":"2024-05-08T21:17:56.420115+0000","last_became_active":"2024-05-08T21:16:15.193964+0000","last_became_peered":"2024-05-08T21:16:15.193964+0000","last_unstale":"2024-05-08T21:17:56.420115+0000","last_undegraded":"2024-05-08T21:17:56.420115+0000","last_fullsized":"2024-05-08T21:17:56.420115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_clean_scrub_stamp":"2024-05-08T21:13:48.584026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:28.091200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.628348+0000","last_change":"2024-05-08T21:17:57.627889+0000","last_active":"2024-05-08T21:17:57.628348+0000","last_peered":"2024-05-08T21:17:57.628348+0000","last_clean":"2024-05-08T21:17:57.628348+0000","last_became_active":"2024-05-08T21:17:57.627502+0000","last_became_peered":"2024-05-08T21:17:57.627502+0000","last_unstale":"2024-05-08T21:17:57.628348+0000","last_undegraded":"2024-05-08T21:17:57.628348+0000","last_fullsized":"2024-05-08T21:17:57.628348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:10.025347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412481+0000","last_change":"2024-05-08T21:17:03.246034+0000","last_active":"2024-05-08T21:17:56.412481+0000","last_peered":"2024-05-08T21:17:56.412481+0000","last_clean":"2024-05-08T21:17:56.412481+0000","last_became_active":"2024-05-08T21:17:03.245620+0000","last_became_peered":"2024-05-08T21:17:03.245620+0000","last_unstale":"2024-05-08T21:17:56.412481+0000","last_undegraded":"2024-05-08T21:17:56.412481+0000","last_fullsized":"2024-05-08T21:17:56.412481+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:02.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420424+0000","last_change":"2024-05-08T21:17:02.731260+0000","last_active":"2024-05-08T21:17:56.420424+0000","last_peered":"2024-05-08T21:17:56.420424+0000","last_clean":"2024-05-08T21:17:56.420424+0000","last_became_active":"2024-05-08T21:17:02.730879+0000","last_became_peered":"2024-05-08T21:17:02.730879+0000","last_unstale":"2024-05-08T21:17:56.420424+0000","last_undegraded":"2024-05-08T21:17:56.420424+0000","last_fullsized":"2024-05-08T21:17:56.420424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:59.030674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417621+0000","last_change":"2024-05-08T21:17:07.269791+0000","last_active":"2024-05-08T21:17:56.417621+0000","last_peered":"2024-05-08T21:17:56.417621+0000","last_clean":"2024-05-08T21:17:56.417621+0000","last_became_active":"2024-05-08T21:17:07.269345+0000","last_became_peered":"2024-05-08T21:17:07.269345+0000","last_unstale":"2024-05-08T21:17:56.417621+0000","last_undegraded":"2024-05-08T21:17:56.417621+0000","last_fullsized":"2024-05-08T21:17:56.417621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:42.866864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416940+0000","last_change":"2024-05-08T21:17:02.735423+0000","last_active":"2024-05-08T21:17:56.416940+0000","last_peered":"2024-05-08T21:17:56.416940+0000","last_clean":"2024-05-08T21:17:56.416940+0000","last_became_active":"2024-05-08T21:17:02.735158+0000","last_became_peered":"2024-05-08T21:17:02.735158+0000","last_unstale":"2024-05-08T21:17:56.416940+0000","last_undegraded":"2024-05-08T21:17:56.416940+0000","last_fullsized":"2024-05-08T21:17:56.416940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:19.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415012+0000","last_change":"2024-05-08T21:17:03.239316+0000","last_active":"2024-05-08T21:17:56.415012+0000","last_peered":"2024-05-08T21:17:56.415012+0000","last_clean":"2024-05-08T21:17:56.415012+0000","last_became_active":"2024-05-08T21:17:03.239018+0000","last_became_peered":"2024-05-08T21:17:03.239018+0000","last_unstale":"2024-05-08T21:17:56.415012+0000","last_undegraded":"2024-05-08T21:17:56.415012+0000","last_fullsized":"2024-05-08T21:17:56.415012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:36.952155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415569+0000","last_change":"2024-05-08T21:17:07.439338+0000","last_active":"2024-05-08T21:17:56.415569+0000","last_peered":"2024-05-08T21:17:56.415569+0000","last_clean":"2024-05-08T21:17:56.415569+0000","last_became_active":"2024-05-08T21:17:07.438703+0000","last_became_peered":"2024-05-08T21:17:07.438703+0000","last_unstale":"2024-05-08T21:17:56.415569+0000","last_undegraded":"2024-05-08T21:17:56.415569+0000","last_fullsized":"2024-05-08T21:17:56.415569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:50.997138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420144+0000","last_change":"2024-05-08T21:17:02.729012+0000","last_active":"2024-05-08T21:17:56.420144+0000","last_peered":"2024-05-08T21:17:56.420144+0000","last_clean":"2024-05-08T21:17:56.420144+0000","last_became_active":"2024-05-08T21:17:02.728606+0000","last_became_peered":"2024-05-08T21:17:02.728606+0000","last_unstale":"2024-05-08T21:17:56.420144+0000","last_undegraded":"2024-05-08T21:17:56.420144+0000","last_fullsized":"2024-05-08T21:17:56.420144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.140609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416523+0000","last_change":"2024-05-08T21:17:03.234712+0000","last_active":"2024-05-08T21:17:56.416523+0000","last_peered":"2024-05-08T21:17:56.416523+0000","last_clean":"2024-05-08T21:17:56.416523+0000","last_became_active":"2024-05-08T21:17:03.234404+0000","last_became_peered":"2024-05-08T21:17:03.234404+0000","last_unstale":"2024-05-08T21:17:56.416523+0000","last_undegraded":"2024-05-08T21:17:56.416523+0000","last_fullsized":"2024-05-08T21:17:56.416523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:41.479470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422480+0000","last_change":"2024-05-08T21:17:07.259120+0000","last_active":"2024-05-08T21:17:56.422480+0000","last_peered":"2024-05-08T21:17:56.422480+0000","last_clean":"2024-05-08T21:17:56.422480+0000","last_became_active":"2024-05-08T21:17:07.258696+0000","last_became_peered":"2024-05-08T21:17:07.258696+0000","last_unstale":"2024-05-08T21:17:56.422480+0000","last_undegraded":"2024-05-08T21:17:56.422480+0000","last_fullsized":"2024-05-08T21:17:56.422480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:26.950767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423353+0000","last_change":"2024-05-08T21:17:02.727795+0000","last_active":"2024-05-08T21:17:56.423353+0000","last_peered":"2024-05-08T21:17:56.423353+0000","last_clean":"2024-05-08T21:17:56.423353+0000","last_became_active":"2024-05-08T21:17:02.727553+0000","last_became_peered":"2024-05-08T21:17:02.727553+0000","last_unstale":"2024-05-08T21:17:56.423353+0000","last_undegraded":"2024-05-08T21:17:56.423353+0000","last_fullsized":"2024-05-08T21:17:56.423353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:51.540221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423413+0000","last_change":"2024-05-08T21:17:03.247270+0000","last_active":"2024-05-08T21:17:56.423413+0000","last_peered":"2024-05-08T21:17:56.423413+0000","last_clean":"2024-05-08T21:17:56.423413+0000","last_became_active":"2024-05-08T21:17:03.246853+0000","last_became_peered":"2024-05-08T21:17:03.246853+0000","last_unstale":"2024-05-08T21:17:56.423413+0000","last_undegraded":"2024-05-08T21:17:56.423413+0000","last_fullsized":"2024-05-08T21:17:56.423413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:46.224088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422880+0000","last_change":"2024-05-08T21:17:07.439920+0000","last_active":"2024-05-08T21:17:56.422880+0000","last_peered":"2024-05-08T21:17:56.422880+0000","last_clean":"2024-05-08T21:17:56.422880+0000","last_became_active":"2024-05-08T21:17:07.439459+0000","last_became_peered":"2024-05-08T21:17:07.439459+0000","last_unstale":"2024-05-08T21:17:56.422880+0000","last_undegraded":"2024-05-08T21:17:56.422880+0000","last_fullsized":"2024-05-08T21:17:56.422880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:03.008930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422835+0000","last_change":"2024-05-08T21:17:02.731889+0000","last_active":"2024-05-08T21:17:56.422835+0000","last_peered":"2024-05-08T21:17:56.422835+0000","last_clean":"2024-05-08T21:17:56.422835+0000","last_became_active":"2024-05-08T21:17:02.731554+0000","last_became_peered":"2024-05-08T21:17:02.731554+0000","last_unstale":"2024-05-08T21:17:56.422835+0000","last_undegraded":"2024-05-08T21:17:56.422835+0000","last_fullsized":"2024-05-08T21:17:56.422835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:25.223003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417051+0000","last_change":"2024-05-08T21:17:03.252672+0000","last_active":"2024-05-08T21:17:56.417051+0000","last_peered":"2024-05-08T21:17:56.417051+0000","last_clean":"2024-05-08T21:17:56.417051+0000","last_became_active":"2024-05-08T21:17:03.251684+0000","last_became_peered":"2024-05-08T21:17:03.251684+0000","last_unstale":"2024-05-08T21:17:56.417051+0000","last_undegraded":"2024-05-08T21:17:56.417051+0000","last_fullsized":"2024-05-08T21:17:56.417051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:06.460419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420488+0000","last_change":"2024-05-08T21:17:07.273478+0000","last_active":"2024-05-08T21:17:56.420488+0000","last_peered":"2024-05-08T21:17:56.420488+0000","last_clean":"2024-05-08T21:17:56.420488+0000","last_became_active":"2024-05-08T21:17:07.272525+0000","last_became_peered":"2024-05-08T21:17:07.272525+0000","last_unstale":"2024-05-08T21:17:56.420488+0000","last_undegraded":"2024-05-08T21:17:56.420488+0000","last_fullsized":"2024-05-08T21:17:56.420488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:51:05.107387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417904+0000","last_change":"2024-05-08T21:17:02.728858+0000","last_active":"2024-05-08T21:17:56.417904+0000","last_peered":"2024-05-08T21:17:56.417904+0000","last_clean":"2024-05-08T21:17:56.417904+0000","last_became_active":"2024-05-08T21:17:02.728516+0000","last_became_peered":"2024-05-08T21:17:02.728516+0000","last_unstale":"2024-05-08T21:17:56.417904+0000","last_undegraded":"2024-05-08T21:17:56.417904+0000","last_fullsized":"2024-05-08T21:17:56.417904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:40.263983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418555+0000","last_change":"2024-05-08T21:17:03.239354+0000","last_active":"2024-05-08T21:17:56.418555+0000","last_peered":"2024-05-08T21:17:56.418555+0000","last_clean":"2024-05-08T21:17:56.418555+0000","last_became_active":"2024-05-08T21:17:03.239020+0000","last_became_peered":"2024-05-08T21:17:03.239020+0000","last_unstale":"2024-05-08T21:17:56.418555+0000","last_undegraded":"2024-05-08T21:17:56.418555+0000","last_fullsized":"2024-05-08T21:17:56.418555+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:02:43.362016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422887+0000","last_change":"2024-05-08T21:17:07.262401+0000","last_active":"2024-05-08T21:17:56.422887+0000","last_peered":"2024-05-08T21:17:56.422887+0000","last_clean":"2024-05-08T21:17:56.422887+0000","last_became_active":"2024-05-08T21:17:07.261946+0000","last_became_peered":"2024-05-08T21:17:07.261946+0000","last_unstale":"2024-05-08T21:17:56.422887+0000","last_undegraded":"2024-05-08T21:17:56.422887+0000","last_fullsized":"2024-05-08T21:17:56.422887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:56.565604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.438045+0000","last_change":"2024-05-08T21:17:57.437573+0000","last_active":"2024-05-08T21:17:57.438045+0000","last_peered":"2024-05-08T21:17:57.438045+0000","last_clean":"2024-05-08T21:17:57.438045+0000","last_became_active":"2024-05-08T21:17:57.432315+0000","last_became_peered":"2024-05-08T21:17:57.432315+0000","last_unstale":"2024-05-08T21:17:57.438045+0000","last_undegraded":"2024-05-08T21:17:57.438045+0000","last_fullsized":"2024-05-08T21:17:57.438045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:56:08.847290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417312+0000","last_change":"2024-05-08T21:17:03.250382+0000","last_active":"2024-05-08T21:17:56.417312+0000","last_peered":"2024-05-08T21:17:56.417312+0000","last_clean":"2024-05-08T21:17:56.417312+0000","last_became_active":"2024-05-08T21:17:03.250031+0000","last_became_peered":"2024-05-08T21:17:03.250031+0000","last_unstale":"2024-05-08T21:17:56.417312+0000","last_undegraded":"2024-05-08T21:17:56.417312+0000","last_fullsized":"2024-05-08T21:17:56.417312+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:59.961012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422877+0000","last_change":"2024-05-08T21:17:07.439341+0000","last_active":"2024-05-08T21:17:56.422877+0000","last_peered":"2024-05-08T21:17:56.422877+0000","last_clean":"2024-05-08T21:17:56.422877+0000","last_became_active":"2024-05-08T21:17:07.438896+0000","last_became_peered":"2024-05-08T21:17:07.438896+0000","last_unstale":"2024-05-08T21:17:56.422877+0000","last_undegraded":"2024-05-08T21:17:56.422877+0000","last_fullsized":"2024-05-08T21:17:56.422877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:55.180686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416194+0000","last_change":"2024-05-08T21:17:02.732967+0000","last_active":"2024-05-08T21:17:56.416194+0000","last_peered":"2024-05-08T21:17:56.416194+0000","last_clean":"2024-05-08T21:17:56.416194+0000","last_became_active":"2024-05-08T21:17:02.732616+0000","last_became_peered":"2024-05-08T21:17:02.732616+0000","last_unstale":"2024-05-08T21:17:56.416194+0000","last_undegraded":"2024-05-08T21:17:56.416194+0000","last_fullsized":"2024-05-08T21:17:56.416194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:50:26.116841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420367+0000","last_change":"2024-05-08T21:17:03.241974+0000","last_active":"2024-05-08T21:17:56.420367+0000","last_peered":"2024-05-08T21:17:56.420367+0000","last_clean":"2024-05-08T21:17:56.420367+0000","last_became_active":"2024-05-08T21:17:03.241627+0000","last_became_peered":"2024-05-08T21:17:03.241627+0000","last_unstale":"2024-05-08T21:17:56.420367+0000","last_undegraded":"2024-05-08T21:17:56.420367+0000","last_fullsized":"2024-05-08T21:17:56.420367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:57.351078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424107+0000","last_change":"2024-05-08T21:17:07.259593+0000","last_active":"2024-05-08T21:17:56.424107+0000","last_peered":"2024-05-08T21:17:56.424107+0000","last_clean":"2024-05-08T21:17:56.424107+0000","last_became_active":"2024-05-08T21:17:07.258362+0000","last_became_peered":"2024-05-08T21:17:07.258362+0000","last_unstale":"2024-05-08T21:17:56.424107+0000","last_undegraded":"2024-05-08T21:17:56.424107+0000","last_fullsized":"2024-05-08T21:17:56.424107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:07.877264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:10.380480+0000","last_change":"2024-05-08T21:17:02.731685+0000","last_active":"2024-05-08T21:18:10.380480+0000","last_peered":"2024-05-08T21:18:10.380480+0000","last_clean":"2024-05-08T21:18:10.380480+0000","last_became_active":"2024-05-08T21:17:02.731293+0000","last_became_peered":"2024-05-08T21:17:02.731293+0000","last_unstale":"2024-05-08T21:18:10.380480+0000","last_undegraded":"2024-05-08T21:18:10.380480+0000","last_fullsized":"2024-05-08T21:18:10.380480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:36.532962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419974+0000","last_change":"2024-05-08T21:17:03.250090+0000","last_active":"2024-05-08T21:17:56.419974+0000","last_peered":"2024-05-08T21:17:56.419974+0000","last_clean":"2024-05-08T21:17:56.419974+0000","last_became_active":"2024-05-08T21:17:03.249744+0000","last_became_peered":"2024-05-08T21:17:03.249744+0000","last_unstale":"2024-05-08T21:17:56.419974+0000","last_undegraded":"2024-05-08T21:17:56.419974+0000","last_fullsized":"2024-05-08T21:17:56.419974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:43.155040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422926+0000","last_change":"2024-05-08T21:17:07.264101+0000","last_active":"2024-05-08T21:17:56.422926+0000","last_peered":"2024-05-08T21:17:56.422926+0000","last_clean":"2024-05-08T21:17:56.422926+0000","last_became_active":"2024-05-08T21:17:07.263720+0000","last_became_peered":"2024-05-08T21:17:07.263720+0000","last_unstale":"2024-05-08T21:17:56.422926+0000","last_undegraded":"2024-05-08T21:17:56.422926+0000","last_fullsized":"2024-05-08T21:17:56.422926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:16.943044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419108+0000","last_change":"2024-05-08T21:17:02.726763+0000","last_active":"2024-05-08T21:17:56.419108+0000","last_peered":"2024-05-08T21:17:56.419108+0000","last_clean":"2024-05-08T21:17:56.419108+0000","last_became_active":"2024-05-08T21:17:02.726343+0000","last_became_peered":"2024-05-08T21:17:02.726343+0000","last_unstale":"2024-05-08T21:17:56.419108+0000","last_undegraded":"2024-05-08T21:17:56.419108+0000","last_fullsized":"2024-05-08T21:17:56.419108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:03.504667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420530+0000","last_change":"2024-05-08T21:17:03.244468+0000","last_active":"2024-05-08T21:17:56.420530+0000","last_peered":"2024-05-08T21:17:56.420530+0000","last_clean":"2024-05-08T21:17:56.420530+0000","last_became_active":"2024-05-08T21:17:03.244132+0000","last_became_peered":"2024-05-08T21:17:03.244132+0000","last_unstale":"2024-05-08T21:17:56.420530+0000","last_undegraded":"2024-05-08T21:17:56.420530+0000","last_fullsized":"2024-05-08T21:17:56.420530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:17.423657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418261+0000","last_change":"2024-05-08T21:17:07.260981+0000","last_active":"2024-05-08T21:17:56.418261+0000","last_peered":"2024-05-08T21:17:56.418261+0000","last_clean":"2024-05-08T21:17:56.418261+0000","last_became_active":"2024-05-08T21:17:07.260582+0000","last_became_peered":"2024-05-08T21:17:07.260582+0000","last_unstale":"2024-05-08T21:17:56.418261+0000","last_undegraded":"2024-05-08T21:17:56.418261+0000","last_fullsized":"2024-05-08T21:17:56.418261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:25.319411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414755+0000","last_change":"2024-05-08T21:17:02.732531+0000","last_active":"2024-05-08T21:17:56.414755+0000","last_peered":"2024-05-08T21:17:56.414755+0000","last_clean":"2024-05-08T21:17:56.414755+0000","last_became_active":"2024-05-08T21:17:02.732121+0000","last_became_peered":"2024-05-08T21:17:02.732121+0000","last_unstale":"2024-05-08T21:17:56.414755+0000","last_undegraded":"2024-05-08T21:17:56.414755+0000","last_fullsized":"2024-05-08T21:17:56.414755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:25.482458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420635+0000","last_change":"2024-05-08T21:17:03.249676+0000","last_active":"2024-05-08T21:17:56.420635+0000","last_peered":"2024-05-08T21:17:56.420635+0000","last_clean":"2024-05-08T21:17:56.420635+0000","last_became_active":"2024-05-08T21:17:03.249120+0000","last_became_peered":"2024-05-08T21:17:03.249120+0000","last_unstale":"2024-05-08T21:17:56.420635+0000","last_undegraded":"2024-05-08T21:17:56.420635+0000","last_fullsized":"2024-05-08T21:17:56.420635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:58:40.678800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418654+0000","last_change":"2024-05-08T21:17:03.245585+0000","last_active":"2024-05-08T21:17:56.418654+0000","last_peered":"2024-05-08T21:17:56.418654+0000","last_clean":"2024-05-08T21:17:56.418654+0000","last_became_active":"2024-05-08T21:17:03.245274+0000","last_became_peered":"2024-05-08T21:17:03.245274+0000","last_unstale":"2024-05-08T21:17:56.418654+0000","last_undegraded":"2024-05-08T21:17:56.418654+0000","last_fullsized":"2024-05-08T21:17:56.418654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:32.003962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423037+0000","last_change":"2024-05-08T21:17:03.246622+0000","last_active":"2024-05-08T21:17:56.423037+0000","last_peered":"2024-05-08T21:17:56.423037+0000","last_clean":"2024-05-08T21:17:56.423037+0000","last_became_active":"2024-05-08T21:17:03.246309+0000","last_became_peered":"2024-05-08T21:17:03.246309+0000","last_unstale":"2024-05-08T21:17:56.423037+0000","last_undegraded":"2024-05-08T21:17:56.423037+0000","last_fullsized":"2024-05-08T21:17:56.423037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:32.439389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422776+0000","last_change":"2024-05-08T21:17:03.243460+0000","last_active":"2024-05-08T21:17:56.422776+0000","last_peered":"2024-05-08T21:17:56.422776+0000","last_clean":"2024-05-08T21:17:56.422776+0000","last_became_active":"2024-05-08T21:17:03.243097+0000","last_became_peered":"2024-05-08T21:17:03.243097+0000","last_unstale":"2024-05-08T21:17:56.422776+0000","last_undegraded":"2024-05-08T21:17:56.422776+0000","last_fullsized":"2024-05-08T21:17:56.422776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:52:14.492180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423038+0000","last_change":"2024-05-08T21:17:03.245077+0000","last_active":"2024-05-08T21:17:56.423038+0000","last_peered":"2024-05-08T21:17:56.423038+0000","last_clean":"2024-05-08T21:17:56.423038+0000","last_became_active":"2024-05-08T21:17:03.244719+0000","last_became_peered":"2024-05-08T21:17:03.244719+0000","last_unstale":"2024-05-08T21:17:56.423038+0000","last_undegraded":"2024-05-08T21:17:56.423038+0000","last_fullsized":"2024-05-08T21:17:56.423038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.388318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415101+0000","last_change":"2024-05-08T21:17:03.245261+0000","last_active":"2024-05-08T21:17:56.415101+0000","last_peered":"2024-05-08T21:17:56.415101+0000","last_clean":"2024-05-08T21:17:56.415101+0000","last_became_active":"2024-05-08T21:17:03.244892+0000","last_became_peered":"2024-05-08T21:17:03.244892+0000","last_unstale":"2024-05-08T21:17:56.415101+0000","last_undegraded":"2024-05-08T21:17:56.415101+0000","last_fullsized":"2024-05-08T21:17:56.415101+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:21.075634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418251+0000","last_change":"2024-05-08T21:17:03.246250+0000","last_active":"2024-05-08T21:17:56.418251+0000","last_peered":"2024-05-08T21:17:56.418251+0000","last_clean":"2024-05-08T21:17:56.418251+0000","last_became_active":"2024-05-08T21:17:03.245807+0000","last_became_peered":"2024-05-08T21:17:03.245807+0000","last_unstale":"2024-05-08T21:17:56.418251+0000","last_undegraded":"2024-05-08T21:17:56.418251+0000","last_fullsized":"2024-05-08T21:17:56.418251+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:20.618260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423477+0000","last_change":"2024-05-08T21:17:03.254559+0000","last_active":"2024-05-08T21:17:56.423477+0000","last_peered":"2024-05-08T21:17:56.423477+0000","last_clean":"2024-05-08T21:17:56.423477+0000","last_became_active":"2024-05-08T21:17:03.254231+0000","last_became_peered":"2024-05-08T21:17:03.254231+0000","last_unstale":"2024-05-08T21:17:56.423477+0000","last_undegraded":"2024-05-08T21:17:56.423477+0000","last_fullsized":"2024-05-08T21:17:56.423477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:47.996927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418065+0000","last_change":"2024-05-08T21:17:03.242479+0000","last_active":"2024-05-08T21:17:56.418065+0000","last_peered":"2024-05-08T21:17:56.418065+0000","last_clean":"2024-05-08T21:17:56.418065+0000","last_became_active":"2024-05-08T21:17:03.241397+0000","last_became_peered":"2024-05-08T21:17:03.241397+0000","last_unstale":"2024-05-08T21:17:56.418065+0000","last_undegraded":"2024-05-08T21:17:56.418065+0000","last_fullsized":"2024-05-08T21:17:56.418065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:39.029139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419220+0000","last_change":"2024-05-08T21:17:03.244208+0000","last_active":"2024-05-08T21:17:56.419220+0000","last_peered":"2024-05-08T21:17:56.419220+0000","last_clean":"2024-05-08T21:17:56.419220+0000","last_became_active":"2024-05-08T21:17:03.243828+0000","last_became_peered":"2024-05-08T21:17:03.243828+0000","last_unstale":"2024-05-08T21:17:56.419220+0000","last_undegraded":"2024-05-08T21:17:56.419220+0000","last_fullsized":"2024-05-08T21:17:56.419220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:39.936750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418012+0000","last_change":"2024-05-08T21:17:03.240135+0000","last_active":"2024-05-08T21:17:56.418012+0000","last_peered":"2024-05-08T21:17:56.418012+0000","last_clean":"2024-05-08T21:17:56.418012+0000","last_became_active":"2024-05-08T21:17:03.239757+0000","last_became_peered":"2024-05-08T21:17:03.239757+0000","last_unstale":"2024-05-08T21:17:56.418012+0000","last_undegraded":"2024-05-08T21:17:56.418012+0000","last_fullsized":"2024-05-08T21:17:56.418012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:46.066111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424382+0000","last_change":"2024-05-08T21:17:03.256254+0000","last_active":"2024-05-08T21:17:56.424382+0000","last_peered":"2024-05-08T21:17:56.424382+0000","last_clean":"2024-05-08T21:17:56.424382+0000","last_became_active":"2024-05-08T21:17:03.255310+0000","last_became_peered":"2024-05-08T21:17:03.255310+0000","last_unstale":"2024-05-08T21:17:56.424382+0000","last_undegraded":"2024-05-08T21:17:56.424382+0000","last_fullsized":"2024-05-08T21:17:56.424382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:59.571958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412843+0000","last_change":"2024-05-08T21:17:03.241901+0000","last_active":"2024-05-08T21:17:56.412843+0000","last_peered":"2024-05-08T21:17:56.412843+0000","last_clean":"2024-05-08T21:17:56.412843+0000","last_became_active":"2024-05-08T21:17:03.241402+0000","last_became_peered":"2024-05-08T21:17:03.241402+0000","last_unstale":"2024-05-08T21:17:56.412843+0000","last_undegraded":"2024-05-08T21:17:56.412843+0000","last_fullsized":"2024-05-08T21:17:56.412843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:59.202987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417882+0000","last_change":"2024-05-08T21:17:03.243050+0000","last_active":"2024-05-08T21:17:56.417882+0000","last_peered":"2024-05-08T21:17:56.417882+0000","last_clean":"2024-05-08T21:17:56.417882+0000","last_became_active":"2024-05-08T21:17:03.240676+0000","last_became_peered":"2024-05-08T21:17:03.240676+0000","last_unstale":"2024-05-08T21:17:56.417882+0000","last_undegraded":"2024-05-08T21:17:56.417882+0000","last_fullsized":"2024-05-08T21:17:56.417882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:19:53.981559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68912,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":418,"num_read_kb":283,"num_write":239,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1246,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":206736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":483699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.69299999999999995}]},{"osd":1,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53800000000000003}]},{"osd":2,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.62},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.753}]},{"osd":7,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71499999999999997}]},{"osd":8,"last update":"Wed May 8 21:18:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.81299999999999994}]},{"osd":9,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.83699999999999997}]},{"osd":10,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.80100000000000005}]}]},{"osd":10,"up_from":69,"seq":296352743451,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36856,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708296,"statfs":{"total":95995035648,"available":95957295104,"internally_reserved":0,"allocated":1679360,"data_stored":1132080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.376}]},{"osd":2,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.53100000000000003}]},{"osd":3,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54400000000000004}]},{"osd":4,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.39800000000000002}]},{"osd":5,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57299999999999995}]},{"osd":6,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.625},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":8,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.61599999999999999}]},{"osd":9,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.68400000000000005}]},{"osd":11,"last update":"Wed May 8 21:17:31 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.67200000000000004}]}]},{"osd":9,"up_from":64,"seq":274877906974,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":487380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.48099999999999998,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65000000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.5,"15min":0.50800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63700000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.442}]},{"osd":1,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.84399999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81100000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.46300000000000002}]},{"osd":2,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76500000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63400000000000001,"15min":0.64300000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":1.048,"15min":1.048},"last":0.61499999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.496,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.77800000000000002,"15min":0.79500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92300000000000004,"5min":1.234,"15min":1.234},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.056,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.875},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.47099999999999997}]},{"osd":7,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.81599999999999995,"15min":0.83099999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96099999999999997,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.84199999999999997,"15min":0.86399999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.73499999999999999}]},{"osd":8,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.80100000000000005,"15min":0.82299999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93600000000000005,"5min":1.101,"15min":1.101},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.88600000000000001,"15min":0.90700000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.004,"5min":1.589,"15min":1.589},"last":0.77000000000000002}]},{"osd":10,"last update":"Wed May 8 21:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":8,"up_from":56,"seq":240518168611,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":482659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.78200000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.442},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41399999999999998}]},{"osd":1,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.33500000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79200000000000004,"5min":0.87,"15min":0.87},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75900000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.68000000000000005}]},{"osd":2,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.315,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.64500000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79800000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.879,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.501}]},{"osd":4,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626}]},{"osd":5,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.503},"min":{"1min":0.38700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.65600000000000003}]},{"osd":6,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.50700000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.503,"5min":0.373,"15min":0.373},"max":{"1min":0.93500000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69999999999999996}]},{"osd":9,"last update":"Wed May 8 21:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70199999999999996,"15min":0.68899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.745,"15min":0.73499999999999999},"min":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.96999999999999997,"5min":1.04,"15min":1.04},"last":0.67300000000000004}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.71099999999999997}]},{"osd":11,"last update":"Wed May 8 21:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.754},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.76400000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397542,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1597440,"data_stored":1079020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.26300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.499},{"interface":"front","average":{"1min":0.441,"5min":0.40400000000000003,"15min":0.39800000000000002},"min":{"1min":0.27700000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.65900000000000003}]},{"osd":3,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.63800000000000001}]},{"osd":4,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.33600000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.745},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.316,"5min":0.315,"15min":0.315},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.46800000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.45900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.50800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.754},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.79800000000000004}]},{"osd":8,"last update":"Wed May 8 21:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.501,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.52500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76800000000000002}]},{"osd":9,"last update":"Wed May 8 21:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67200000000000004,"15min":0.67600000000000005},"min":{"1min":0.47799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.81999999999999995,"5min":1.008,"15min":1.008},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.86499999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.82099999999999995}]},{"osd":10,"last update":"Wed May 8 21:17:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.78400000000000003}]},{"osd":11,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.86099999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561066,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":478740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.376}]},{"osd":1,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.502,"5min":0.434,"15min":0.434},"max":{"1min":1,"5min":1,"15min":1},"last":0.68700000000000006}]},{"osd":2,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.50900000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.749},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.57399999999999995}]},{"osd":3,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.441},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.60999999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.45700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.71499999999999997}]},{"osd":5,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.56000000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.72399999999999998,"15min":0.71199999999999997},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.70299999999999996}]},{"osd":7,"last update":"Wed May 8 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90900000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0169999999999999,"5min":1.034,"15min":1.034},"last":0.79800000000000004}]},{"osd":8,"last update":"Wed May 8 21:17:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.94799999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.65300000000000002}]},{"osd":9,"last update":"Wed May 8 21:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.80400000000000005,"15min":0.79300000000000004},"min":{"1min":0.54800000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.76700000000000002},"min":{"1min":0.625,"5min":0.498,"15min":0.498},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73499999999999999}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.85699999999999998}]},{"osd":11,"last update":"Wed May 8 21:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.877},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.76400000000000001}]}]},{"osd":1,"up_from":12,"seq":51539607611,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":542731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55000000000000004,"15min":0.52700000000000002},"min":{"1min":0.36099999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89800000000000002,"5min":1.548,"15min":1.548},"last":0.44700000000000001}]},{"osd":2,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.54000000000000004,"15min":0.50700000000000001},"min":{"1min":0.41699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.52900000000000003,"15min":0.496},"min":{"1min":0.40600000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80900000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68400000000000005}]},{"osd":3,"last update":"Wed May 8 21:17:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.55300000000000005,"15min":0.51600000000000001},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96199999999999997,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.60999999999999999}]},{"osd":4,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.435,"15min":0.41099999999999998},"min":{"1min":0.315,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.70599999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.432,"15min":0.41699999999999998},"min":{"1min":0.318,"5min":0.253,"15min":0.253},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed May 8 21:17:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.56299999999999994,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56100000000000005,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.63300000000000001}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.746},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64300000000000002,"15min":0.61199999999999999},"min":{"1min":0.502,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89100000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92400000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.36799999999999999,"5min":0.308,"15min":0.308},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65200000000000002}]},{"osd":8,"last update":"Wed May 8 21:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64300000000000002,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.50900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96599999999999997,"5min":1.607,"15min":1.607},"last":0.76000000000000001}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70099999999999996,"15min":0.69099999999999995},"min":{"1min":0.54300000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73299999999999998,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.84199999999999997}]},{"osd":10,"last update":"Wed May 8 21:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.78600000000000003}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.82499999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":1011712,"data_stored":480589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 21:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.40699999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.53000000000000003},"min":{"1min":0.434,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.86099999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed May 8 21:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.748,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.66900000000000004}]},{"osd":3,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.438}]},{"osd":4,"last update":"Wed May 8 21:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78300000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59999999999999998,"15min":0.58099999999999996},"min":{"1min":0.44,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.50600000000000001}]},{"osd":5,"last update":"Wed May 8 21:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86399999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.76400000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.872}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.745},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.42999999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89700000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.52600000000000002}]},{"osd":7,"last update":"Wed May 8 21:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.98099999999999998}]},{"osd":8,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94999999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.751,"15min":0.75800000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.91700000000000004}]},{"osd":9,"last update":"Wed May 8 21:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77400000000000002},"min":{"1min":0.65100000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97799999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.76200000000000001}]},{"osd":10,"last update":"Wed May 8 21:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.81100000000000005}]},{"osd":11,"last update":"Wed May 8 21:17:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.94499999999999995}]}]},{"osd":2,"up_from":17,"seq":73014444089,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1069056,"data_stored":542050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.56799999999999995,"15min":0.52100000000000002},"min":{"1min":0.32600000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":6.7270000000000003,"5min":6.7270000000000003,"15min":6.7270000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.51200000000000001},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.72099999999999997}]},{"osd":1,"last update":"Wed May 8 21:17:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.44500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.747},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.42899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68500000000000005,"5min":0.752,"15min":0.752},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65600000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53500000000000003}]},{"osd":4,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed May 8 21:17:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.42899999999999999,"15min":0.40799999999999997},"min":{"1min":0.29799999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.44,"15min":0.41599999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.37}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.55400000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.212,"5min":0.71199999999999997,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":7.7779999999999996,"5min":7.7779999999999996,"15min":7.7779999999999996},"last":0.68799999999999994}]},{"osd":7,"last update":"Wed May 8 21:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73499999999999999}]},{"osd":8,"last update":"Wed May 8 21:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.57799999999999996,"15min":0.56200000000000006},"min":{"1min":0.35799999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48699999999999999}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.67700000000000005,"15min":0.65800000000000003},"min":{"1min":0.66800000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.56799999999999995,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.82899999999999996}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.81699999999999995}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59799999999999998}]}]},{"osd":3,"up_from":24,"seq":103079215157,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":999424,"data_stored":478496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.40500000000000003,"15min":0.38600000000000001},"min":{"1min":0.36199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35599999999999998}]},{"osd":1,"last update":"Wed May 8 21:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.52500000000000002,"15min":0.51100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53500000000000003,"15min":0.51500000000000001},"min":{"1min":0.40999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.45300000000000001}]},{"osd":2,"last update":"Wed May 8 21:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.39700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.314},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.32900000000000001}]},{"osd":4,"last update":"Wed May 8 21:17:27 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.57499999999999996,"15min":0.54100000000000004},"min":{"1min":0.47899999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.88300000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60699999999999998}]},{"osd":5,"last update":"Wed May 8 21:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.496},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.61199999999999999},"min":{"1min":0.46800000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Wed May 8 21:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.504,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.55100000000000005}]},{"osd":7,"last update":"Wed May 8 21:17:23 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.748}]},{"osd":8,"last update":"Wed May 8 21:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79800000000000004}]},{"osd":9,"last update":"Wed May 8 21:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.628}]},{"osd":10,"last update":"Wed May 8 21:17:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.65600000000000003}]},{"osd":11,"last update":"Wed May 8 21:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.72399999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986224,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":483388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.373,"15min":0.36299999999999999},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.51300000000000001}]},{"osd":2,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72899999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.64800000000000002}]},{"osd":3,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.52100000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.54100000000000004}]},{"osd":5,"last update":"Wed May 8 21:17:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.376,"5min":0.371,"15min":0.371},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.57599999999999996,"15min":0.53800000000000003},"min":{"1min":0.54600000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59999999999999998,"15min":0.57999999999999996},"min":{"1min":0.51200000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82399999999999995,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.626}]},{"osd":7,"last update":"Wed May 8 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.5,"15min":0.47999999999999998},"min":{"1min":0.34300000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.53400000000000003,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.67600000000000005}]},{"osd":8,"last update":"Wed May 8 21:17:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70699999999999996,"15min":0.69599999999999995},"min":{"1min":0.58299999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.77400000000000002}]},{"osd":9,"last update":"Wed May 8 21:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.747,"15min":0.747},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.747},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.876,"5min":1.347,"15min":1.347},"last":0.75700000000000001}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.70999999999999996}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79500000000000004}]}]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36792,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708360,"statfs":{"total":95995035648,"available":95957360640,"internally_reserved":0,"allocated":1613824,"data_stored":1075302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.36799999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.622,"5min":0.877,"15min":0.877},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.51700000000000002},"min":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67500000000000004,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.52200000000000002}]},{"osd":1,"last update":"Wed May 8 21:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.42099999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.41199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71699999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed May 8 21:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40300000000000002,"15min":0.39900000000000002},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61199999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.32000000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.59999999999999998}]},{"osd":3,"last update":"Wed May 8 21:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80300000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.72199999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.747}]},{"osd":4,"last update":"Wed May 8 21:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.436,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84199999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76000000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71299999999999997}]},{"osd":6,"last update":"Wed May 8 21:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.75700000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.623,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93300000000000005,"5min":1.01,"15min":1.01},"last":0.67200000000000004}]},{"osd":7,"last update":"Wed May 8 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81999999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.81399999999999995}]},{"osd":8,"last update":"Wed May 8 21:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91800000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86799999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.76900000000000002}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.55100000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88900000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.83999999999999997}]},{"osd":10,"last update":"Wed May 8 21:17:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.86099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":63977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T21:18:12.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T21:18:13.217 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:13.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:13 smithi071 ceph-mon[47074]: pgmap v265: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s; 47 B/s, 0 objects/s recovering 2024-05-08T21:18:13.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:13 smithi082 ceph-mon[47512]: pgmap v265: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s; 47 B/s, 0 objects/s recovering 2024-05-08T21:18:13.682 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:13 smithi033 ceph-mon[42528]: pgmap v265: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s; 47 B/s, 0 objects/s recovering 2024-05-08T21:18:14.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.212+0000 7f6df3eb8700 1 -- 172.21.15.33:0/494850837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec0695e0 msgr2=0x7f6dec069a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:14.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.212+0000 7f6df3eb8700 1 --2- 172.21.15.33:0/494850837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec0695e0 0x7f6dec069a60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0099c0 tx=0x7f6ddc009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:14.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.212+0000 7f6df3eb8700 1 -- 172.21.15.33:0/494850837 shutdown_connections 2024-05-08T21:18:14.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.212+0000 7f6df3eb8700 1 --2- 172.21.15.33:0/494850837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec069fa0 0x7f6dec10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:14.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.212+0000 7f6df3eb8700 1 --2- 172.21.15.33:0/494850837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec107020 0x7f6dec107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:14.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.212+0000 7f6df3eb8700 1 --2- 172.21.15.33:0/494850837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec0695e0 0x7f6dec069a60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:14.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.212+0000 7f6df3eb8700 1 -- 172.21.15.33:0/494850837 >> 172.21.15.33:0/494850837 conn(0x7f6dec076c50 msgr2=0x7f6dec077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.212+0000 7f6df3eb8700 1 -- 172.21.15.33:0/494850837 shutdown_connections 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.212+0000 7f6df3eb8700 1 -- 172.21.15.33:0/494850837 wait complete. 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.213+0000 7f6df3eb8700 1 Processor -- start 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.213+0000 7f6df3eb8700 1 -- start start 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.213+0000 7f6df3eb8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec0695e0 0x7f6dec197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.213+0000 7f6df3eb8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec069fa0 0x7f6dec1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.213+0000 7f6df3eb8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec107020 0x7f6dec19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.213+0000 7f6df3eb8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec19c1c0 con 0x7f6dec0695e0 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.213+0000 7f6df3eb8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec19c330 con 0x7f6dec069fa0 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.213+0000 7f6df3eb8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec19c4f0 con 0x7f6dec107020 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.213+0000 7f6df1c54700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec0695e0 0x7f6dec197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.213+0000 7f6df1c54700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec0695e0 0x7f6dec197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48732/0 (socket says 172.21.15.33:48732) 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.213+0000 7f6df1c54700 1 -- 172.21.15.33:0/707421861 learned_addr learned my addr 172.21.15.33:0/707421861 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6df1453700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec069fa0 0x7f6dec1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:14.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6df2455700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec107020 0x7f6dec19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:14.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6df1c54700 1 -- 172.21.15.33:0/707421861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec107020 msgr2=0x7f6dec19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:14.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6df1c54700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec107020 0x7f6dec19bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:14.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6df1c54700 1 -- 172.21.15.33:0/707421861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec069fa0 msgr2=0x7f6dec1978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:14.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6df1c54700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec069fa0 0x7f6dec1978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:14.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6df1c54700 1 -- 172.21.15.33:0/707421861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc0095f0 con 0x7f6dec0695e0 2024-05-08T21:18:14.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6df1c54700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec0695e0 0x7f6dec197380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6de800e9a0 tx=0x7f6de800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:14.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6de2ffd700 1 -- 172.21.15.33:0/707421861 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8009790 con 0x7f6dec0695e0 2024-05-08T21:18:14.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6de2ffd700 1 -- 172.21.15.33:0/707421861 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6de8010430 con 0x7f6dec0695e0 2024-05-08T21:18:14.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6df3eb8700 1 -- 172.21.15.33:0/707421861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dec19c7d0 con 0x7f6dec0695e0 2024-05-08T21:18:14.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6de2ffd700 1 -- 172.21.15.33:0/707421861 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800cea0 con 0x7f6dec0695e0 2024-05-08T21:18:14.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.214+0000 7f6df3eb8700 1 -- 172.21.15.33:0/707421861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dec1a4050 con 0x7f6dec0695e0 2024-05-08T21:18:14.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.215+0000 7f6df3eb8700 1 -- 172.21.15.33:0/707421861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dec109520 con 0x7f6dec0695e0 2024-05-08T21:18:14.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.217+0000 7f6de2ffd700 1 -- 172.21.15.33:0/707421861 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f6de80105a0 con 0x7f6dec0695e0 2024-05-08T21:18:14.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.217+0000 7f6de2ffd700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6dd80a1230 0x7f6dd80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:14.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.217+0000 7f6df1453700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6dd80a1230 0x7f6dd80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:14.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.217+0000 7f6de2ffd700 1 -- 172.21.15.33:0/707421861 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6de8014070 con 0x7f6dec0695e0 2024-05-08T21:18:14.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.218+0000 7f6df1453700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6dd80a1230 0x7f6dd80a36f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc000c00 tx=0x7f6ddc005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:14.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.221+0000 7f6de2ffd700 1 -- 172.21.15.33:0/707421861 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6de8096350 con 0x7f6dec0695e0 2024-05-08T21:18:14.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.430+0000 7f6df3eb8700 1 -- 172.21.15.33:0/707421861 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6dec1985d0 con 0x7f6dd80a1230 2024-05-08T21:18:14.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.443+0000 7f6de2ffd700 1 -- 172.21.15.33:0/707421861 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426626 (secure 0 0 0) 0x7f6dec1985d0 con 0x7f6dd80a1230 2024-05-08T21:18:14.451 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.447+0000 7f6df3eb8700 1 -- 172.21.15.33:0/707421861 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6dd80a1230 msgr2=0x7f6dd80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.447+0000 7f6df3eb8700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6dd80a1230 0x7f6dd80a36f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc000c00 tx=0x7f6ddc005a90 comp rx=0 tx=0).stop 2024-05-08T21:18:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.447+0000 7f6df3eb8700 1 -- 172.21.15.33:0/707421861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec0695e0 msgr2=0x7f6dec197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.447+0000 7f6df3eb8700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec0695e0 0x7f6dec197380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6de800e9a0 tx=0x7f6de800ecb0 comp rx=0 tx=0).stop 2024-05-08T21:18:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.447+0000 7f6df3eb8700 1 -- 172.21.15.33:0/707421861 shutdown_connections 2024-05-08T21:18:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.447+0000 7f6df3eb8700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec107020 0x7f6dec19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.447+0000 7f6df3eb8700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dec0695e0 0x7f6dec197380 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.447+0000 7f6df3eb8700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6dd80a1230 0x7f6dd80a36f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.447+0000 7f6df3eb8700 1 --2- 172.21.15.33:0/707421861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dec069fa0 0x7f6dec1978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.447+0000 7f6df3eb8700 1 -- 172.21.15.33:0/707421861 >> 172.21.15.33:0/707421861 conn(0x7f6dec076c50 msgr2=0x7f6dec0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.447+0000 7f6df3eb8700 1 -- 172.21.15.33:0/707421861 shutdown_connections 2024-05-08T21:18:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:14.447+0000 7f6df3eb8700 1 -- 172.21.15.33:0/707421861 wait complete. 2024-05-08T21:18:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T21:18:14.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:14 smithi033 ceph-mon[42528]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:14.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:14 smithi071 ceph-mon[47074]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:14.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:14 smithi082 ceph-mon[47512]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-05-08T21:18:14.133113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659514,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":554,"num_read_kb":397,"num_write":422,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1181982,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374832,"kb_used_data":13696,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124566992,"statfs":{"total":1151940427776,"available":1151556599808,"internally_reserved":0,"allocated":14024704,"data_stored":7746134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003775"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418982+0000","last_change":"2024-05-08T21:17:03.243903+0000","last_active":"2024-05-08T21:17:56.418982+0000","last_peered":"2024-05-08T21:17:56.418982+0000","last_clean":"2024-05-08T21:17:56.418982+0000","last_became_active":"2024-05-08T21:17:03.240821+0000","last_became_peered":"2024-05-08T21:17:03.240821+0000","last_unstale":"2024-05-08T21:17:56.418982+0000","last_undegraded":"2024-05-08T21:17:56.418982+0000","last_fullsized":"2024-05-08T21:17:56.418982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:01.878614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423327+0000","last_change":"2024-05-08T21:17:03.243119+0000","last_active":"2024-05-08T21:17:56.423327+0000","last_peered":"2024-05-08T21:17:56.423327+0000","last_clean":"2024-05-08T21:17:56.423327+0000","last_became_active":"2024-05-08T21:17:03.242772+0000","last_became_peered":"2024-05-08T21:17:03.242772+0000","last_unstale":"2024-05-08T21:17:56.423327+0000","last_undegraded":"2024-05-08T21:17:56.423327+0000","last_fullsized":"2024-05-08T21:17:56.423327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:20.550646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423287+0000","last_change":"2024-05-08T21:17:03.237843+0000","last_active":"2024-05-08T21:17:56.423287+0000","last_peered":"2024-05-08T21:17:56.423287+0000","last_clean":"2024-05-08T21:17:56.423287+0000","last_became_active":"2024-05-08T21:17:03.237567+0000","last_became_peered":"2024-05-08T21:17:03.237567+0000","last_unstale":"2024-05-08T21:17:56.423287+0000","last_undegraded":"2024-05-08T21:17:56.423287+0000","last_fullsized":"2024-05-08T21:17:56.423287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:23.856461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.430448+0000","last_change":"2024-05-08T21:17:57.430448+0000","last_active":"2024-05-08T21:17:57.430448+0000","last_peered":"2024-05-08T21:17:57.430448+0000","last_clean":"2024-05-08T21:17:57.430448+0000","last_became_active":"2024-05-08T21:17:57.430195+0000","last_became_peered":"2024-05-08T21:17:57.430195+0000","last_unstale":"2024-05-08T21:17:57.430448+0000","last_undegraded":"2024-05-08T21:17:57.430448+0000","last_fullsized":"2024-05-08T21:17:57.430448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:41.536610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416067+0000","last_change":"2024-05-08T21:17:03.252246+0000","last_active":"2024-05-08T21:17:56.416067+0000","last_peered":"2024-05-08T21:17:56.416067+0000","last_clean":"2024-05-08T21:17:56.416067+0000","last_became_active":"2024-05-08T21:17:03.251898+0000","last_became_peered":"2024-05-08T21:17:03.251898+0000","last_unstale":"2024-05-08T21:17:56.416067+0000","last_undegraded":"2024-05-08T21:17:56.416067+0000","last_fullsized":"2024-05-08T21:17:56.416067+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:08.996033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417704+0000","last_change":"2024-05-08T21:17:03.238422+0000","last_active":"2024-05-08T21:17:56.417704+0000","last_peered":"2024-05-08T21:17:56.417704+0000","last_clean":"2024-05-08T21:17:56.417704+0000","last_became_active":"2024-05-08T21:17:03.238096+0000","last_became_peered":"2024-05-08T21:17:03.238096+0000","last_unstale":"2024-05-08T21:17:56.417704+0000","last_undegraded":"2024-05-08T21:17:56.417704+0000","last_fullsized":"2024-05-08T21:17:56.417704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:54.635089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419866+0000","last_change":"2024-05-08T21:17:03.244315+0000","last_active":"2024-05-08T21:17:56.419866+0000","last_peered":"2024-05-08T21:17:56.419866+0000","last_clean":"2024-05-08T21:17:56.419866+0000","last_became_active":"2024-05-08T21:17:03.243939+0000","last_became_peered":"2024-05-08T21:17:03.243939+0000","last_unstale":"2024-05-08T21:17:56.419866+0000","last_undegraded":"2024-05-08T21:17:56.419866+0000","last_fullsized":"2024-05-08T21:17:56.419866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:24.371495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415520+0000","last_change":"2024-05-08T21:17:03.240144+0000","last_active":"2024-05-08T21:17:56.415520+0000","last_peered":"2024-05-08T21:17:56.415520+0000","last_clean":"2024-05-08T21:17:56.415520+0000","last_became_active":"2024-05-08T21:17:03.239493+0000","last_became_peered":"2024-05-08T21:17:03.239493+0000","last_unstale":"2024-05-08T21:17:56.415520+0000","last_undegraded":"2024-05-08T21:17:56.415520+0000","last_fullsized":"2024-05-08T21:17:56.415520+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:21.374357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424317+0000","last_change":"2024-05-08T21:17:03.256706+0000","last_active":"2024-05-08T21:17:56.424317+0000","last_peered":"2024-05-08T21:17:56.424317+0000","last_clean":"2024-05-08T21:17:56.424317+0000","last_became_active":"2024-05-08T21:17:03.255345+0000","last_became_peered":"2024-05-08T21:17:03.255345+0000","last_unstale":"2024-05-08T21:17:56.424317+0000","last_undegraded":"2024-05-08T21:17:56.424317+0000","last_fullsized":"2024-05-08T21:17:56.424317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:09:50.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.430334+0000","last_change":"2024-05-08T21:17:57.430334+0000","last_active":"2024-05-08T21:17:57.430334+0000","last_peered":"2024-05-08T21:17:57.430334+0000","last_clean":"2024-05-08T21:17:57.430334+0000","last_became_active":"2024-05-08T21:17:57.429924+0000","last_became_peered":"2024-05-08T21:17:57.429924+0000","last_unstale":"2024-05-08T21:17:57.430334+0000","last_undegraded":"2024-05-08T21:17:57.430334+0000","last_fullsized":"2024-05-08T21:17:57.430334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.901755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419810+0000","last_change":"2024-05-08T21:17:03.253996+0000","last_active":"2024-05-08T21:17:56.419810+0000","last_peered":"2024-05-08T21:17:56.419810+0000","last_clean":"2024-05-08T21:17:56.419810+0000","last_became_active":"2024-05-08T21:17:03.253686+0000","last_became_peered":"2024-05-08T21:17:03.253686+0000","last_unstale":"2024-05-08T21:17:56.419810+0000","last_undegraded":"2024-05-08T21:17:56.419810+0000","last_fullsized":"2024-05-08T21:17:56.419810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:17.995692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423092+0000","last_change":"2024-05-08T21:17:03.254357+0000","last_active":"2024-05-08T21:17:56.423092+0000","last_peered":"2024-05-08T21:17:56.423092+0000","last_clean":"2024-05-08T21:17:56.423092+0000","last_became_active":"2024-05-08T21:17:03.254013+0000","last_became_peered":"2024-05-08T21:17:03.254013+0000","last_unstale":"2024-05-08T21:17:56.423092+0000","last_undegraded":"2024-05-08T21:17:56.423092+0000","last_fullsized":"2024-05-08T21:17:56.423092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:21.480612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.413995+0000","last_change":"2024-05-08T21:17:03.244190+0000","last_active":"2024-05-08T21:17:56.413995+0000","last_peered":"2024-05-08T21:17:56.413995+0000","last_clean":"2024-05-08T21:17:56.413995+0000","last_became_active":"2024-05-08T21:17:03.243989+0000","last_became_peered":"2024-05-08T21:17:03.243989+0000","last_unstale":"2024-05-08T21:17:56.413995+0000","last_undegraded":"2024-05-08T21:17:56.413995+0000","last_fullsized":"2024-05-08T21:17:56.413995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:45.059343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422938+0000","last_change":"2024-05-08T21:17:03.242996+0000","last_active":"2024-05-08T21:17:56.422938+0000","last_peered":"2024-05-08T21:17:56.422938+0000","last_clean":"2024-05-08T21:17:56.422938+0000","last_became_active":"2024-05-08T21:17:03.242648+0000","last_became_peered":"2024-05-08T21:17:03.242648+0000","last_unstale":"2024-05-08T21:17:56.422938+0000","last_undegraded":"2024-05-08T21:17:56.422938+0000","last_fullsized":"2024-05-08T21:17:56.422938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:47:19.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422908+0000","last_change":"2024-05-08T21:17:03.244970+0000","last_active":"2024-05-08T21:17:56.422908+0000","last_peered":"2024-05-08T21:17:56.422908+0000","last_clean":"2024-05-08T21:17:56.422908+0000","last_became_active":"2024-05-08T21:17:03.244604+0000","last_became_peered":"2024-05-08T21:17:03.244604+0000","last_unstale":"2024-05-08T21:17:56.422908+0000","last_undegraded":"2024-05-08T21:17:56.422908+0000","last_fullsized":"2024-05-08T21:17:56.422908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:32.484155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422727+0000","last_change":"2024-05-08T21:17:03.254628+0000","last_active":"2024-05-08T21:17:56.422727+0000","last_peered":"2024-05-08T21:17:56.422727+0000","last_clean":"2024-05-08T21:17:56.422727+0000","last_became_active":"2024-05-08T21:17:03.254305+0000","last_became_peered":"2024-05-08T21:17:03.254305+0000","last_unstale":"2024-05-08T21:17:56.422727+0000","last_undegraded":"2024-05-08T21:17:56.422727+0000","last_fullsized":"2024-05-08T21:17:56.422727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:52.727821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418732+0000","last_change":"2024-05-08T21:17:03.243406+0000","last_active":"2024-05-08T21:17:56.418732+0000","last_peered":"2024-05-08T21:17:56.418732+0000","last_clean":"2024-05-08T21:17:56.418732+0000","last_became_active":"2024-05-08T21:17:03.240379+0000","last_became_peered":"2024-05-08T21:17:03.240379+0000","last_unstale":"2024-05-08T21:17:56.418732+0000","last_undegraded":"2024-05-08T21:17:56.418732+0000","last_fullsized":"2024-05-08T21:17:56.418732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:49.897459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420035+0000","last_change":"2024-05-08T21:17:03.239664+0000","last_active":"2024-05-08T21:17:56.420035+0000","last_peered":"2024-05-08T21:17:56.420035+0000","last_clean":"2024-05-08T21:17:56.420035+0000","last_became_active":"2024-05-08T21:17:03.239347+0000","last_became_peered":"2024-05-08T21:17:03.239347+0000","last_unstale":"2024-05-08T21:17:56.420035+0000","last_undegraded":"2024-05-08T21:17:56.420035+0000","last_fullsized":"2024-05-08T21:17:56.420035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:06:37.557835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423543+0000","last_change":"2024-05-08T21:17:03.249424+0000","last_active":"2024-05-08T21:17:56.423543+0000","last_peered":"2024-05-08T21:17:56.423543+0000","last_clean":"2024-05-08T21:17:56.423543+0000","last_became_active":"2024-05-08T21:17:03.248971+0000","last_became_peered":"2024-05-08T21:17:03.248971+0000","last_unstale":"2024-05-08T21:17:56.423543+0000","last_undegraded":"2024-05-08T21:17:56.423543+0000","last_fullsized":"2024-05-08T21:17:56.423543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:08.096762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422849+0000","last_change":"2024-05-08T21:17:07.272414+0000","last_active":"2024-05-08T21:17:56.422849+0000","last_peered":"2024-05-08T21:17:56.422849+0000","last_clean":"2024-05-08T21:17:56.422849+0000","last_became_active":"2024-05-08T21:17:07.271660+0000","last_became_peered":"2024-05-08T21:17:07.271660+0000","last_unstale":"2024-05-08T21:17:56.422849+0000","last_undegraded":"2024-05-08T21:17:56.422849+0000","last_fullsized":"2024-05-08T21:17:56.422849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:15.697910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:07.488173+0000","last_change":"2024-05-08T21:17:57.483017+0000","last_active":"2024-05-08T21:18:07.488173+0000","last_peered":"2024-05-08T21:18:07.488173+0000","last_clean":"2024-05-08T21:18:07.488173+0000","last_became_active":"2024-05-08T21:17:57.473420+0000","last_became_peered":"2024-05-08T21:17:57.473420+0000","last_unstale":"2024-05-08T21:18:07.488173+0000","last_undegraded":"2024-05-08T21:18:07.488173+0000","last_fullsized":"2024-05-08T21:18:07.488173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:42.727117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417325+0000","last_change":"2024-05-08T21:17:03.240780+0000","last_active":"2024-05-08T21:17:56.417325+0000","last_peered":"2024-05-08T21:17:56.417325+0000","last_clean":"2024-05-08T21:17:56.417325+0000","last_became_active":"2024-05-08T21:17:03.240472+0000","last_became_peered":"2024-05-08T21:17:03.240472+0000","last_unstale":"2024-05-08T21:17:56.417325+0000","last_undegraded":"2024-05-08T21:17:56.417325+0000","last_fullsized":"2024-05-08T21:17:56.417325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.089760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.421952+0000","last_change":"2024-05-08T21:17:07.272308+0000","last_active":"2024-05-08T21:17:56.421952+0000","last_peered":"2024-05-08T21:17:56.421952+0000","last_clean":"2024-05-08T21:17:56.421952+0000","last_became_active":"2024-05-08T21:17:07.271869+0000","last_became_peered":"2024-05-08T21:17:07.271869+0000","last_unstale":"2024-05-08T21:17:56.421952+0000","last_undegraded":"2024-05-08T21:17:56.421952+0000","last_fullsized":"2024-05-08T21:17:56.421952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:31.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417727+0000","last_change":"2024-05-08T21:17:02.736446+0000","last_active":"2024-05-08T21:17:56.417727+0000","last_peered":"2024-05-08T21:17:56.417727+0000","last_clean":"2024-05-08T21:17:56.417727+0000","last_became_active":"2024-05-08T21:17:02.736187+0000","last_became_peered":"2024-05-08T21:17:02.736187+0000","last_unstale":"2024-05-08T21:17:56.417727+0000","last_undegraded":"2024-05-08T21:17:56.417727+0000","last_fullsized":"2024-05-08T21:17:56.417727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:29.302558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423206+0000","last_change":"2024-05-08T21:17:03.251896+0000","last_active":"2024-05-08T21:17:56.423206+0000","last_peered":"2024-05-08T21:17:56.423206+0000","last_clean":"2024-05-08T21:17:56.423206+0000","last_became_active":"2024-05-08T21:17:03.251495+0000","last_became_peered":"2024-05-08T21:17:03.251495+0000","last_unstale":"2024-05-08T21:17:56.423206+0000","last_undegraded":"2024-05-08T21:17:56.423206+0000","last_fullsized":"2024-05-08T21:17:56.423206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:50.774848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422223+0000","last_change":"2024-05-08T21:17:07.272150+0000","last_active":"2024-05-08T21:17:56.422223+0000","last_peered":"2024-05-08T21:17:56.422223+0000","last_clean":"2024-05-08T21:17:56.422223+0000","last_became_active":"2024-05-08T21:17:07.271727+0000","last_became_peered":"2024-05-08T21:17:07.271727+0000","last_unstale":"2024-05-08T21:17:56.422223+0000","last_undegraded":"2024-05-08T21:17:56.422223+0000","last_fullsized":"2024-05-08T21:17:56.422223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.058277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416860+0000","last_change":"2024-05-08T21:17:02.733611+0000","last_active":"2024-05-08T21:17:56.416860+0000","last_peered":"2024-05-08T21:17:56.416860+0000","last_clean":"2024-05-08T21:17:56.416860+0000","last_became_active":"2024-05-08T21:17:02.733306+0000","last_became_peered":"2024-05-08T21:17:02.733306+0000","last_unstale":"2024-05-08T21:17:56.416860+0000","last_undegraded":"2024-05-08T21:17:56.416860+0000","last_fullsized":"2024-05-08T21:17:56.416860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:33.304753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423294+0000","last_change":"2024-05-08T21:17:03.248480+0000","last_active":"2024-05-08T21:17:56.423294+0000","last_peered":"2024-05-08T21:17:56.423294+0000","last_clean":"2024-05-08T21:17:56.423294+0000","last_became_active":"2024-05-08T21:17:03.248126+0000","last_became_peered":"2024-05-08T21:17:03.248126+0000","last_unstale":"2024-05-08T21:17:56.423294+0000","last_undegraded":"2024-05-08T21:17:56.423294+0000","last_fullsized":"2024-05-08T21:17:56.423294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:08.785761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422472+0000","last_change":"2024-05-08T21:17:07.272344+0000","last_active":"2024-05-08T21:17:56.422472+0000","last_peered":"2024-05-08T21:17:56.422472+0000","last_clean":"2024-05-08T21:17:56.422472+0000","last_became_active":"2024-05-08T21:17:07.271629+0000","last_became_peered":"2024-05-08T21:17:07.271629+0000","last_unstale":"2024-05-08T21:17:56.422472+0000","last_undegraded":"2024-05-08T21:17:56.422472+0000","last_fullsized":"2024-05-08T21:17:56.422472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:15.052748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416295+0000","last_change":"2024-05-08T21:17:02.730364+0000","last_active":"2024-05-08T21:17:56.416295+0000","last_peered":"2024-05-08T21:17:56.416295+0000","last_clean":"2024-05-08T21:17:56.416295+0000","last_became_active":"2024-05-08T21:17:02.728854+0000","last_became_peered":"2024-05-08T21:17:02.728854+0000","last_unstale":"2024-05-08T21:17:56.416295+0000","last_undegraded":"2024-05-08T21:17:56.416295+0000","last_fullsized":"2024-05-08T21:17:56.416295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:09.552016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419025+0000","last_change":"2024-05-08T21:17:03.252713+0000","last_active":"2024-05-08T21:17:56.419025+0000","last_peered":"2024-05-08T21:17:56.419025+0000","last_clean":"2024-05-08T21:17:56.419025+0000","last_became_active":"2024-05-08T21:17:03.252388+0000","last_became_peered":"2024-05-08T21:17:03.252388+0000","last_unstale":"2024-05-08T21:17:56.419025+0000","last_undegraded":"2024-05-08T21:17:56.419025+0000","last_fullsized":"2024-05-08T21:17:56.419025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:09.114064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423183+0000","last_change":"2024-05-08T21:17:07.261112+0000","last_active":"2024-05-08T21:17:56.423183+0000","last_peered":"2024-05-08T21:17:56.423183+0000","last_clean":"2024-05-08T21:17:56.423183+0000","last_became_active":"2024-05-08T21:17:07.260713+0000","last_became_peered":"2024-05-08T21:17:07.260713+0000","last_unstale":"2024-05-08T21:17:56.423183+0000","last_undegraded":"2024-05-08T21:17:56.423183+0000","last_fullsized":"2024-05-08T21:17:56.423183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:37.262113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:09.451541+0000","last_change":"2024-05-08T21:17:57.436812+0000","last_active":"2024-05-08T21:18:09.451541+0000","last_peered":"2024-05-08T21:18:09.451541+0000","last_clean":"2024-05-08T21:18:09.451541+0000","last_became_active":"2024-05-08T21:17:57.432431+0000","last_became_peered":"2024-05-08T21:17:57.432431+0000","last_unstale":"2024-05-08T21:18:09.451541+0000","last_undegraded":"2024-05-08T21:18:09.451541+0000","last_fullsized":"2024-05-08T21:18:09.451541+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:46.062273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":574,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417176+0000","last_change":"2024-05-08T21:17:03.251245+0000","last_active":"2024-05-08T21:17:56.417176+0000","last_peered":"2024-05-08T21:17:56.417176+0000","last_clean":"2024-05-08T21:17:56.417176+0000","last_became_active":"2024-05-08T21:17:03.250296+0000","last_became_peered":"2024-05-08T21:17:03.250296+0000","last_unstale":"2024-05-08T21:17:56.417176+0000","last_undegraded":"2024-05-08T21:17:56.417176+0000","last_fullsized":"2024-05-08T21:17:56.417176+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:28.391927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423437+0000","last_change":"2024-05-08T21:17:07.266752+0000","last_active":"2024-05-08T21:17:56.423437+0000","last_peered":"2024-05-08T21:17:56.423437+0000","last_clean":"2024-05-08T21:17:56.423437+0000","last_became_active":"2024-05-08T21:17:07.266367+0000","last_became_peered":"2024-05-08T21:17:07.266367+0000","last_unstale":"2024-05-08T21:17:56.423437+0000","last_undegraded":"2024-05-08T21:17:56.423437+0000","last_fullsized":"2024-05-08T21:17:56.423437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:49.874073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417579+0000","last_change":"2024-05-08T21:17:02.728113+0000","last_active":"2024-05-08T21:17:56.417579+0000","last_peered":"2024-05-08T21:17:56.417579+0000","last_clean":"2024-05-08T21:17:56.417579+0000","last_became_active":"2024-05-08T21:17:02.727754+0000","last_became_peered":"2024-05-08T21:17:02.727754+0000","last_unstale":"2024-05-08T21:17:56.417579+0000","last_undegraded":"2024-05-08T21:17:56.417579+0000","last_fullsized":"2024-05-08T21:17:56.417579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:53.789218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418802+0000","last_change":"2024-05-08T21:17:03.245482+0000","last_active":"2024-05-08T21:17:56.418802+0000","last_peered":"2024-05-08T21:17:56.418802+0000","last_clean":"2024-05-08T21:17:56.418802+0000","last_became_active":"2024-05-08T21:17:03.245144+0000","last_became_peered":"2024-05-08T21:17:03.245144+0000","last_unstale":"2024-05-08T21:17:56.418802+0000","last_undegraded":"2024-05-08T21:17:56.418802+0000","last_fullsized":"2024-05-08T21:17:56.418802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:04.051234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418636+0000","last_change":"2024-05-08T21:17:07.268505+0000","last_active":"2024-05-08T21:17:56.418636+0000","last_peered":"2024-05-08T21:17:56.418636+0000","last_clean":"2024-05-08T21:17:56.418636+0000","last_became_active":"2024-05-08T21:17:07.268157+0000","last_became_peered":"2024-05-08T21:17:07.268157+0000","last_unstale":"2024-05-08T21:17:56.418636+0000","last_undegraded":"2024-05-08T21:17:56.418636+0000","last_fullsized":"2024-05-08T21:17:56.418636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:55.220521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412606+0000","last_change":"2024-05-08T21:17:02.727264+0000","last_active":"2024-05-08T21:17:56.412606+0000","last_peered":"2024-05-08T21:17:56.412606+0000","last_clean":"2024-05-08T21:17:56.412606+0000","last_became_active":"2024-05-08T21:17:02.726922+0000","last_became_peered":"2024-05-08T21:17:02.726922+0000","last_unstale":"2024-05-08T21:17:56.412606+0000","last_undegraded":"2024-05-08T21:17:56.412606+0000","last_fullsized":"2024-05-08T21:17:56.412606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:27.410231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415635+0000","last_change":"2024-05-08T21:17:03.233910+0000","last_active":"2024-05-08T21:17:56.415635+0000","last_peered":"2024-05-08T21:17:56.415635+0000","last_clean":"2024-05-08T21:17:56.415635+0000","last_became_active":"2024-05-08T21:17:03.233562+0000","last_became_peered":"2024-05-08T21:17:03.233562+0000","last_unstale":"2024-05-08T21:17:56.415635+0000","last_undegraded":"2024-05-08T21:17:56.415635+0000","last_fullsized":"2024-05-08T21:17:56.415635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:40.309400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423244+0000","last_change":"2024-05-08T21:17:07.264992+0000","last_active":"2024-05-08T21:17:56.423244+0000","last_peered":"2024-05-08T21:17:56.423244+0000","last_clean":"2024-05-08T21:17:56.423244+0000","last_became_active":"2024-05-08T21:17:07.264498+0000","last_became_peered":"2024-05-08T21:17:07.264498+0000","last_unstale":"2024-05-08T21:17:56.423244+0000","last_undegraded":"2024-05-08T21:17:56.423244+0000","last_fullsized":"2024-05-08T21:17:56.423244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:23:26.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418593+0000","last_change":"2024-05-08T21:17:02.725531+0000","last_active":"2024-05-08T21:17:56.418593+0000","last_peered":"2024-05-08T21:17:56.418593+0000","last_clean":"2024-05-08T21:17:56.418593+0000","last_became_active":"2024-05-08T21:17:02.725235+0000","last_became_peered":"2024-05-08T21:17:02.725235+0000","last_unstale":"2024-05-08T21:17:56.418593+0000","last_undegraded":"2024-05-08T21:17:56.418593+0000","last_fullsized":"2024-05-08T21:17:56.418593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:27.265590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419530+0000","last_change":"2024-05-08T21:17:03.236393+0000","last_active":"2024-05-08T21:17:56.419530+0000","last_peered":"2024-05-08T21:17:56.419530+0000","last_clean":"2024-05-08T21:17:56.419530+0000","last_became_active":"2024-05-08T21:17:03.236092+0000","last_became_peered":"2024-05-08T21:17:03.236092+0000","last_unstale":"2024-05-08T21:17:56.419530+0000","last_undegraded":"2024-05-08T21:17:56.419530+0000","last_fullsized":"2024-05-08T21:17:56.419530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:58.232669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417799+0000","last_change":"2024-05-08T21:17:07.271374+0000","last_active":"2024-05-08T21:17:56.417799+0000","last_peered":"2024-05-08T21:17:56.417799+0000","last_clean":"2024-05-08T21:17:56.417799+0000","last_became_active":"2024-05-08T21:17:07.270778+0000","last_became_peered":"2024-05-08T21:17:07.270778+0000","last_unstale":"2024-05-08T21:17:56.417799+0000","last_undegraded":"2024-05-08T21:17:56.417799+0000","last_fullsized":"2024-05-08T21:17:56.417799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:50.212213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":102,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:10.379619+0000","last_change":"2024-05-08T21:17:02.725624+0000","last_active":"2024-05-08T21:18:10.379619+0000","last_peered":"2024-05-08T21:18:10.379619+0000","last_clean":"2024-05-08T21:18:10.379619+0000","last_became_active":"2024-05-08T21:17:02.725367+0000","last_became_peered":"2024-05-08T21:17:02.725367+0000","last_unstale":"2024-05-08T21:18:10.379619+0000","last_undegraded":"2024-05-08T21:18:10.379619+0000","last_fullsized":"2024-05-08T21:18:10.379619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:19.573931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423993+0000","last_change":"2024-05-08T21:17:03.248729+0000","last_active":"2024-05-08T21:17:56.423993+0000","last_peered":"2024-05-08T21:17:56.423993+0000","last_clean":"2024-05-08T21:17:56.423993+0000","last_became_active":"2024-05-08T21:17:03.247899+0000","last_became_peered":"2024-05-08T21:17:03.247899+0000","last_unstale":"2024-05-08T21:17:56.423993+0000","last_undegraded":"2024-05-08T21:17:56.423993+0000","last_fullsized":"2024-05-08T21:17:56.423993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:27:57.012249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419291+0000","last_change":"2024-05-08T21:17:07.261257+0000","last_active":"2024-05-08T21:17:56.419291+0000","last_peered":"2024-05-08T21:17:56.419291+0000","last_clean":"2024-05-08T21:17:56.419291+0000","last_became_active":"2024-05-08T21:17:07.260785+0000","last_became_peered":"2024-05-08T21:17:07.260785+0000","last_unstale":"2024-05-08T21:17:56.419291+0000","last_undegraded":"2024-05-08T21:17:56.419291+0000","last_fullsized":"2024-05-08T21:17:56.419291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:58.469134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419901+0000","last_change":"2024-05-08T21:17:02.729569+0000","last_active":"2024-05-08T21:17:56.419901+0000","last_peered":"2024-05-08T21:17:56.419901+0000","last_clean":"2024-05-08T21:17:56.419901+0000","last_became_active":"2024-05-08T21:17:02.729096+0000","last_became_peered":"2024-05-08T21:17:02.729096+0000","last_unstale":"2024-05-08T21:17:56.419901+0000","last_undegraded":"2024-05-08T21:17:56.419901+0000","last_fullsized":"2024-05-08T21:17:56.419901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:30.668169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423406+0000","last_change":"2024-05-08T21:17:03.246387+0000","last_active":"2024-05-08T21:17:56.423406+0000","last_peered":"2024-05-08T21:17:56.423406+0000","last_clean":"2024-05-08T21:17:56.423406+0000","last_became_active":"2024-05-08T21:17:03.244658+0000","last_became_peered":"2024-05-08T21:17:03.244658+0000","last_unstale":"2024-05-08T21:17:56.423406+0000","last_undegraded":"2024-05-08T21:17:56.423406+0000","last_fullsized":"2024-05-08T21:17:56.423406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:02.602080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.473101+0000","last_change":"2024-05-08T21:17:57.473101+0000","last_active":"2024-05-08T21:17:57.473101+0000","last_peered":"2024-05-08T21:17:57.473101+0000","last_clean":"2024-05-08T21:17:57.473101+0000","last_became_active":"2024-05-08T21:17:57.472563+0000","last_became_peered":"2024-05-08T21:17:57.472563+0000","last_unstale":"2024-05-08T21:17:57.473101+0000","last_undegraded":"2024-05-08T21:17:57.473101+0000","last_fullsized":"2024-05-08T21:17:57.473101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:23.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418913+0000","last_change":"2024-05-08T21:17:02.732738+0000","last_active":"2024-05-08T21:17:56.418913+0000","last_peered":"2024-05-08T21:17:56.418913+0000","last_clean":"2024-05-08T21:17:56.418913+0000","last_became_active":"2024-05-08T21:17:02.731733+0000","last_became_peered":"2024-05-08T21:17:02.731733+0000","last_unstale":"2024-05-08T21:17:56.418913+0000","last_undegraded":"2024-05-08T21:17:56.418913+0000","last_fullsized":"2024-05-08T21:17:56.418913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:24.201171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423479+0000","last_change":"2024-05-08T21:17:03.254405+0000","last_active":"2024-05-08T21:17:56.423479+0000","last_peered":"2024-05-08T21:17:56.423479+0000","last_clean":"2024-05-08T21:17:56.423479+0000","last_became_active":"2024-05-08T21:17:03.252137+0000","last_became_peered":"2024-05-08T21:17:03.252137+0000","last_unstale":"2024-05-08T21:17:56.423479+0000","last_undegraded":"2024-05-08T21:17:56.423479+0000","last_fullsized":"2024-05-08T21:17:56.423479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:15.472225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.431574+0000","last_change":"2024-05-08T21:17:57.431091+0000","last_active":"2024-05-08T21:17:57.431574+0000","last_peered":"2024-05-08T21:17:57.431574+0000","last_clean":"2024-05-08T21:17:57.431574+0000","last_became_active":"2024-05-08T21:17:57.430668+0000","last_became_peered":"2024-05-08T21:17:57.430668+0000","last_unstale":"2024-05-08T21:17:57.431574+0000","last_undegraded":"2024-05-08T21:17:57.431574+0000","last_fullsized":"2024-05-08T21:17:57.431574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:33.208801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"86'36","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:10.452461+0000","last_change":"2024-05-08T21:17:02.723975+0000","last_active":"2024-05-08T21:18:10.452461+0000","last_peered":"2024-05-08T21:18:10.452461+0000","last_clean":"2024-05-08T21:18:10.452461+0000","last_became_active":"2024-05-08T21:17:02.723631+0000","last_became_peered":"2024-05-08T21:17:02.723631+0000","last_unstale":"2024-05-08T21:18:10.452461+0000","last_undegraded":"2024-05-08T21:18:10.452461+0000","last_fullsized":"2024-05-08T21:18:10.452461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:17.676007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62289,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":88,"num_read_kb":104,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416534+0000","last_change":"2024-05-08T21:17:03.251570+0000","last_active":"2024-05-08T21:17:56.416534+0000","last_peered":"2024-05-08T21:17:56.416534+0000","last_clean":"2024-05-08T21:17:56.416534+0000","last_became_active":"2024-05-08T21:17:03.251134+0000","last_became_peered":"2024-05-08T21:17:03.251134+0000","last_unstale":"2024-05-08T21:17:56.416534+0000","last_undegraded":"2024-05-08T21:17:56.416534+0000","last_fullsized":"2024-05-08T21:17:56.416534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:28:09.339958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419635+0000","last_change":"2024-05-08T21:17:07.272051+0000","last_active":"2024-05-08T21:17:56.419635+0000","last_peered":"2024-05-08T21:17:56.419635+0000","last_clean":"2024-05-08T21:17:56.419635+0000","last_became_active":"2024-05-08T21:17:07.271616+0000","last_became_peered":"2024-05-08T21:17:07.271616+0000","last_unstale":"2024-05-08T21:17:56.419635+0000","last_undegraded":"2024-05-08T21:17:56.419635+0000","last_fullsized":"2024-05-08T21:17:56.419635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.364731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423259+0000","last_change":"2024-05-08T21:17:02.736000+0000","last_active":"2024-05-08T21:17:56.423259+0000","last_peered":"2024-05-08T21:17:56.423259+0000","last_clean":"2024-05-08T21:17:56.423259+0000","last_became_active":"2024-05-08T21:17:02.735629+0000","last_became_peered":"2024-05-08T21:17:02.735629+0000","last_unstale":"2024-05-08T21:17:56.423259+0000","last_undegraded":"2024-05-08T21:17:56.423259+0000","last_fullsized":"2024-05-08T21:17:56.423259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:21.842135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412729+0000","last_change":"2024-05-08T21:17:03.237144+0000","last_active":"2024-05-08T21:17:56.412729+0000","last_peered":"2024-05-08T21:17:56.412729+0000","last_clean":"2024-05-08T21:17:56.412729+0000","last_became_active":"2024-05-08T21:17:03.236792+0000","last_became_peered":"2024-05-08T21:17:03.236792+0000","last_unstale":"2024-05-08T21:17:56.412729+0000","last_undegraded":"2024-05-08T21:17:56.412729+0000","last_fullsized":"2024-05-08T21:17:56.412729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:04.520986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.627656+0000","last_change":"2024-05-08T21:17:57.627656+0000","last_active":"2024-05-08T21:17:57.627656+0000","last_peered":"2024-05-08T21:17:57.627656+0000","last_clean":"2024-05-08T21:17:57.627656+0000","last_became_active":"2024-05-08T21:17:57.627108+0000","last_became_peered":"2024-05-08T21:17:57.627108+0000","last_unstale":"2024-05-08T21:17:57.627656+0000","last_undegraded":"2024-05-08T21:17:57.627656+0000","last_fullsized":"2024-05-08T21:17:57.627656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:24.025145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419802+0000","last_change":"2024-05-08T21:17:02.727748+0000","last_active":"2024-05-08T21:17:56.419802+0000","last_peered":"2024-05-08T21:17:56.419802+0000","last_clean":"2024-05-08T21:17:56.419802+0000","last_became_active":"2024-05-08T21:17:02.727444+0000","last_became_peered":"2024-05-08T21:17:02.727444+0000","last_unstale":"2024-05-08T21:17:56.419802+0000","last_undegraded":"2024-05-08T21:17:56.419802+0000","last_fullsized":"2024-05-08T21:17:56.419802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:47.939235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420245+0000","last_change":"2024-05-08T21:17:03.247185+0000","last_active":"2024-05-08T21:17:56.420245+0000","last_peered":"2024-05-08T21:17:56.420245+0000","last_clean":"2024-05-08T21:17:56.420245+0000","last_became_active":"2024-05-08T21:17:03.246880+0000","last_became_peered":"2024-05-08T21:17:03.246880+0000","last_unstale":"2024-05-08T21:17:56.420245+0000","last_undegraded":"2024-05-08T21:17:56.420245+0000","last_fullsized":"2024-05-08T21:17:56.420245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:38.978353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420018+0000","last_change":"2024-05-08T21:17:07.273710+0000","last_active":"2024-05-08T21:17:56.420018+0000","last_peered":"2024-05-08T21:17:56.420018+0000","last_clean":"2024-05-08T21:17:56.420018+0000","last_became_active":"2024-05-08T21:17:07.273119+0000","last_became_peered":"2024-05-08T21:17:07.273119+0000","last_unstale":"2024-05-08T21:17:56.420018+0000","last_undegraded":"2024-05-08T21:17:56.420018+0000","last_fullsized":"2024-05-08T21:17:56.420018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:30.589440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:11.367386+0000","last_change":"2024-05-08T21:17:02.725508+0000","last_active":"2024-05-08T21:18:11.367386+0000","last_peered":"2024-05-08T21:18:11.367386+0000","last_clean":"2024-05-08T21:18:11.367386+0000","last_became_active":"2024-05-08T21:17:02.725107+0000","last_became_peered":"2024-05-08T21:17:02.725107+0000","last_unstale":"2024-05-08T21:18:11.367386+0000","last_undegraded":"2024-05-08T21:18:11.367386+0000","last_fullsized":"2024-05-08T21:18:11.367386+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:13.503702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420302+0000","last_change":"2024-05-08T21:17:03.245372+0000","last_active":"2024-05-08T21:17:56.420302+0000","last_peered":"2024-05-08T21:17:56.420302+0000","last_clean":"2024-05-08T21:17:56.420302+0000","last_became_active":"2024-05-08T21:17:03.245054+0000","last_became_peered":"2024-05-08T21:17:03.245054+0000","last_unstale":"2024-05-08T21:17:56.420302+0000","last_undegraded":"2024-05-08T21:17:56.420302+0000","last_fullsized":"2024-05-08T21:17:56.420302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:30:25.900846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414821+0000","last_change":"2024-05-08T21:17:07.272081+0000","last_active":"2024-05-08T21:17:56.414821+0000","last_peered":"2024-05-08T21:17:56.414821+0000","last_clean":"2024-05-08T21:17:56.414821+0000","last_became_active":"2024-05-08T21:17:07.271560+0000","last_became_peered":"2024-05-08T21:17:07.271560+0000","last_unstale":"2024-05-08T21:17:56.414821+0000","last_undegraded":"2024-05-08T21:17:56.414821+0000","last_fullsized":"2024-05-08T21:17:56.414821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:00.490798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416027+0000","last_change":"2024-05-08T21:17:02.734420+0000","last_active":"2024-05-08T21:17:56.416027+0000","last_peered":"2024-05-08T21:17:56.416027+0000","last_clean":"2024-05-08T21:17:56.416027+0000","last_became_active":"2024-05-08T21:17:02.734071+0000","last_became_peered":"2024-05-08T21:17:02.734071+0000","last_unstale":"2024-05-08T21:17:56.416027+0000","last_undegraded":"2024-05-08T21:17:56.416027+0000","last_fullsized":"2024-05-08T21:17:56.416027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:28.537464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419127+0000","last_change":"2024-05-08T21:17:03.250706+0000","last_active":"2024-05-08T21:17:56.419127+0000","last_peered":"2024-05-08T21:17:56.419127+0000","last_clean":"2024-05-08T21:17:56.419127+0000","last_became_active":"2024-05-08T21:17:03.250408+0000","last_became_peered":"2024-05-08T21:17:03.250408+0000","last_unstale":"2024-05-08T21:17:56.419127+0000","last_undegraded":"2024-05-08T21:17:56.419127+0000","last_fullsized":"2024-05-08T21:17:56.419127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:43.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418530+0000","last_change":"2024-05-08T21:17:07.272066+0000","last_active":"2024-05-08T21:17:56.418530+0000","last_peered":"2024-05-08T21:17:56.418530+0000","last_clean":"2024-05-08T21:17:56.418530+0000","last_became_active":"2024-05-08T21:17:07.271628+0000","last_became_peered":"2024-05-08T21:17:07.271628+0000","last_unstale":"2024-05-08T21:17:56.418530+0000","last_undegraded":"2024-05-08T21:17:56.418530+0000","last_fullsized":"2024-05-08T21:17:56.418530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:27.731995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416588+0000","last_change":"2024-05-08T21:17:02.728464+0000","last_active":"2024-05-08T21:17:56.416588+0000","last_peered":"2024-05-08T21:17:56.416588+0000","last_clean":"2024-05-08T21:17:56.416588+0000","last_became_active":"2024-05-08T21:17:02.727914+0000","last_became_peered":"2024-05-08T21:17:02.727914+0000","last_unstale":"2024-05-08T21:17:56.416588+0000","last_undegraded":"2024-05-08T21:17:56.416588+0000","last_fullsized":"2024-05-08T21:17:56.416588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:55.948661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418448+0000","last_change":"2024-05-08T21:17:03.245578+0000","last_active":"2024-05-08T21:17:56.418448+0000","last_peered":"2024-05-08T21:17:56.418448+0000","last_clean":"2024-05-08T21:17:56.418448+0000","last_became_active":"2024-05-08T21:17:03.245256+0000","last_became_peered":"2024-05-08T21:17:03.245256+0000","last_unstale":"2024-05-08T21:17:56.418448+0000","last_undegraded":"2024-05-08T21:17:56.418448+0000","last_fullsized":"2024-05-08T21:17:56.418448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417048+0000","last_change":"2024-05-08T21:17:07.439525+0000","last_active":"2024-05-08T21:17:56.417048+0000","last_peered":"2024-05-08T21:17:56.417048+0000","last_clean":"2024-05-08T21:17:56.417048+0000","last_became_active":"2024-05-08T21:17:07.438987+0000","last_became_peered":"2024-05-08T21:17:07.438987+0000","last_unstale":"2024-05-08T21:17:56.417048+0000","last_undegraded":"2024-05-08T21:17:56.417048+0000","last_fullsized":"2024-05-08T21:17:56.417048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:57:45.373981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.432999+0000","last_change":"2024-05-08T21:17:57.432999+0000","last_active":"2024-05-08T21:17:57.432999+0000","last_peered":"2024-05-08T21:17:57.432999+0000","last_clean":"2024-05-08T21:17:57.432999+0000","last_became_active":"2024-05-08T21:17:57.432567+0000","last_became_peered":"2024-05-08T21:17:57.432567+0000","last_unstale":"2024-05-08T21:17:57.432999+0000","last_undegraded":"2024-05-08T21:17:57.432999+0000","last_fullsized":"2024-05-08T21:17:57.432999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:33:00.188046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417907+0000","last_change":"2024-05-08T21:17:03.235556+0000","last_active":"2024-05-08T21:17:56.417907+0000","last_peered":"2024-05-08T21:17:56.417907+0000","last_clean":"2024-05-08T21:17:56.417907+0000","last_became_active":"2024-05-08T21:17:03.235214+0000","last_became_peered":"2024-05-08T21:17:03.235214+0000","last_unstale":"2024-05-08T21:17:56.417907+0000","last_undegraded":"2024-05-08T21:17:56.417907+0000","last_fullsized":"2024-05-08T21:17:56.417907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:45.042596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414845+0000","last_change":"2024-05-08T21:17:07.266080+0000","last_active":"2024-05-08T21:17:56.414845+0000","last_peered":"2024-05-08T21:17:56.414845+0000","last_clean":"2024-05-08T21:17:56.414845+0000","last_became_active":"2024-05-08T21:17:07.265605+0000","last_became_peered":"2024-05-08T21:17:07.265605+0000","last_unstale":"2024-05-08T21:17:56.414845+0000","last_undegraded":"2024-05-08T21:17:56.414845+0000","last_fullsized":"2024-05-08T21:17:56.414845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:49:48.387170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":63,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:09.491615+0000","last_change":"2024-05-08T21:17:02.728743+0000","last_active":"2024-05-08T21:18:09.491615+0000","last_peered":"2024-05-08T21:18:09.491615+0000","last_clean":"2024-05-08T21:18:09.491615+0000","last_became_active":"2024-05-08T21:17:02.728330+0000","last_became_peered":"2024-05-08T21:17:02.728330+0000","last_unstale":"2024-05-08T21:18:09.491615+0000","last_undegraded":"2024-05-08T21:18:09.491615+0000","last_fullsized":"2024-05-08T21:18:09.491615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:43.865117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417780+0000","last_change":"2024-05-08T21:17:03.245672+0000","last_active":"2024-05-08T21:17:56.417780+0000","last_peered":"2024-05-08T21:17:56.417780+0000","last_clean":"2024-05-08T21:17:56.417780+0000","last_became_active":"2024-05-08T21:17:03.245318+0000","last_became_peered":"2024-05-08T21:17:03.245318+0000","last_unstale":"2024-05-08T21:17:56.417780+0000","last_undegraded":"2024-05-08T21:17:56.417780+0000","last_fullsized":"2024-05-08T21:17:56.417780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:43.030011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414114+0000","last_change":"2024-05-08T21:17:07.439289+0000","last_active":"2024-05-08T21:17:56.414114+0000","last_peered":"2024-05-08T21:17:56.414114+0000","last_clean":"2024-05-08T21:17:56.414114+0000","last_became_active":"2024-05-08T21:17:07.438874+0000","last_became_peered":"2024-05-08T21:17:07.438874+0000","last_unstale":"2024-05-08T21:17:56.414114+0000","last_undegraded":"2024-05-08T21:17:56.414114+0000","last_fullsized":"2024-05-08T21:17:56.414114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:14.820087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419309+0000","last_change":"2024-05-08T21:17:03.247081+0000","last_active":"2024-05-08T21:17:56.419309+0000","last_peered":"2024-05-08T21:17:56.419309+0000","last_clean":"2024-05-08T21:17:56.419309+0000","last_became_active":"2024-05-08T21:17:03.246418+0000","last_became_peered":"2024-05-08T21:17:03.246418+0000","last_unstale":"2024-05-08T21:17:56.419309+0000","last_undegraded":"2024-05-08T21:17:56.419309+0000","last_fullsized":"2024-05-08T21:17:56.419309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:03.993770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:12.368114+0000","last_change":"2024-05-08T21:17:02.730764+0000","last_active":"2024-05-08T21:18:12.368114+0000","last_peered":"2024-05-08T21:18:12.368114+0000","last_clean":"2024-05-08T21:18:12.368114+0000","last_became_active":"2024-05-08T21:17:02.730494+0000","last_became_peered":"2024-05-08T21:17:02.730494+0000","last_unstale":"2024-05-08T21:18:12.368114+0000","last_undegraded":"2024-05-08T21:18:12.368114+0000","last_fullsized":"2024-05-08T21:18:12.368114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:50.070368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419669+0000","last_change":"2024-05-08T21:17:07.266424+0000","last_active":"2024-05-08T21:17:56.419669+0000","last_peered":"2024-05-08T21:17:56.419669+0000","last_clean":"2024-05-08T21:17:56.419669+0000","last_became_active":"2024-05-08T21:17:07.266144+0000","last_became_peered":"2024-05-08T21:17:07.266144+0000","last_unstale":"2024-05-08T21:17:56.419669+0000","last_undegraded":"2024-05-08T21:17:56.419669+0000","last_fullsized":"2024-05-08T21:17:56.419669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:22.285749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417254+0000","last_change":"2024-05-08T21:17:03.250461+0000","last_active":"2024-05-08T21:17:56.417254+0000","last_peered":"2024-05-08T21:17:56.417254+0000","last_clean":"2024-05-08T21:17:56.417254+0000","last_became_active":"2024-05-08T21:17:03.249974+0000","last_became_peered":"2024-05-08T21:17:03.249974+0000","last_unstale":"2024-05-08T21:17:56.417254+0000","last_undegraded":"2024-05-08T21:17:56.417254+0000","last_fullsized":"2024-05-08T21:17:56.417254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:25:58.759668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422681+0000","last_change":"2024-05-08T21:17:02.725648+0000","last_active":"2024-05-08T21:17:56.422681+0000","last_peered":"2024-05-08T21:17:56.422681+0000","last_clean":"2024-05-08T21:17:56.422681+0000","last_became_active":"2024-05-08T21:17:02.725297+0000","last_became_peered":"2024-05-08T21:17:02.725297+0000","last_unstale":"2024-05-08T21:17:56.422681+0000","last_undegraded":"2024-05-08T21:17:56.422681+0000","last_fullsized":"2024-05-08T21:17:56.422681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:41.462830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":328,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420115+0000","last_change":"2024-05-08T21:16:15.194478+0000","last_active":"2024-05-08T21:17:56.420115+0000","last_peered":"2024-05-08T21:17:56.420115+0000","last_clean":"2024-05-08T21:17:56.420115+0000","last_became_active":"2024-05-08T21:16:15.193964+0000","last_became_peered":"2024-05-08T21:16:15.193964+0000","last_unstale":"2024-05-08T21:17:56.420115+0000","last_undegraded":"2024-05-08T21:17:56.420115+0000","last_fullsized":"2024-05-08T21:17:56.420115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_clean_scrub_stamp":"2024-05-08T21:13:48.584026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:28.091200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.628348+0000","last_change":"2024-05-08T21:17:57.627889+0000","last_active":"2024-05-08T21:17:57.628348+0000","last_peered":"2024-05-08T21:17:57.628348+0000","last_clean":"2024-05-08T21:17:57.628348+0000","last_became_active":"2024-05-08T21:17:57.627502+0000","last_became_peered":"2024-05-08T21:17:57.627502+0000","last_unstale":"2024-05-08T21:17:57.628348+0000","last_undegraded":"2024-05-08T21:17:57.628348+0000","last_fullsized":"2024-05-08T21:17:57.628348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:10.025347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412481+0000","last_change":"2024-05-08T21:17:03.246034+0000","last_active":"2024-05-08T21:17:56.412481+0000","last_peered":"2024-05-08T21:17:56.412481+0000","last_clean":"2024-05-08T21:17:56.412481+0000","last_became_active":"2024-05-08T21:17:03.245620+0000","last_became_peered":"2024-05-08T21:17:03.245620+0000","last_unstale":"2024-05-08T21:17:56.412481+0000","last_undegraded":"2024-05-08T21:17:56.412481+0000","last_fullsized":"2024-05-08T21:17:56.412481+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:02.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420424+0000","last_change":"2024-05-08T21:17:02.731260+0000","last_active":"2024-05-08T21:17:56.420424+0000","last_peered":"2024-05-08T21:17:56.420424+0000","last_clean":"2024-05-08T21:17:56.420424+0000","last_became_active":"2024-05-08T21:17:02.730879+0000","last_became_peered":"2024-05-08T21:17:02.730879+0000","last_unstale":"2024-05-08T21:17:56.420424+0000","last_undegraded":"2024-05-08T21:17:56.420424+0000","last_fullsized":"2024-05-08T21:17:56.420424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:59.030674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417621+0000","last_change":"2024-05-08T21:17:07.269791+0000","last_active":"2024-05-08T21:17:56.417621+0000","last_peered":"2024-05-08T21:17:56.417621+0000","last_clean":"2024-05-08T21:17:56.417621+0000","last_became_active":"2024-05-08T21:17:07.269345+0000","last_became_peered":"2024-05-08T21:17:07.269345+0000","last_unstale":"2024-05-08T21:17:56.417621+0000","last_undegraded":"2024-05-08T21:17:56.417621+0000","last_fullsized":"2024-05-08T21:17:56.417621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:42.866864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416940+0000","last_change":"2024-05-08T21:17:02.735423+0000","last_active":"2024-05-08T21:17:56.416940+0000","last_peered":"2024-05-08T21:17:56.416940+0000","last_clean":"2024-05-08T21:17:56.416940+0000","last_became_active":"2024-05-08T21:17:02.735158+0000","last_became_peered":"2024-05-08T21:17:02.735158+0000","last_unstale":"2024-05-08T21:17:56.416940+0000","last_undegraded":"2024-05-08T21:17:56.416940+0000","last_fullsized":"2024-05-08T21:17:56.416940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:19.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415012+0000","last_change":"2024-05-08T21:17:03.239316+0000","last_active":"2024-05-08T21:17:56.415012+0000","last_peered":"2024-05-08T21:17:56.415012+0000","last_clean":"2024-05-08T21:17:56.415012+0000","last_became_active":"2024-05-08T21:17:03.239018+0000","last_became_peered":"2024-05-08T21:17:03.239018+0000","last_unstale":"2024-05-08T21:17:56.415012+0000","last_undegraded":"2024-05-08T21:17:56.415012+0000","last_fullsized":"2024-05-08T21:17:56.415012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:36.952155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415569+0000","last_change":"2024-05-08T21:17:07.439338+0000","last_active":"2024-05-08T21:17:56.415569+0000","last_peered":"2024-05-08T21:17:56.415569+0000","last_clean":"2024-05-08T21:17:56.415569+0000","last_became_active":"2024-05-08T21:17:07.438703+0000","last_became_peered":"2024-05-08T21:17:07.438703+0000","last_unstale":"2024-05-08T21:17:56.415569+0000","last_undegraded":"2024-05-08T21:17:56.415569+0000","last_fullsized":"2024-05-08T21:17:56.415569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:50.997138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420144+0000","last_change":"2024-05-08T21:17:02.729012+0000","last_active":"2024-05-08T21:17:56.420144+0000","last_peered":"2024-05-08T21:17:56.420144+0000","last_clean":"2024-05-08T21:17:56.420144+0000","last_became_active":"2024-05-08T21:17:02.728606+0000","last_became_peered":"2024-05-08T21:17:02.728606+0000","last_unstale":"2024-05-08T21:17:56.420144+0000","last_undegraded":"2024-05-08T21:17:56.420144+0000","last_fullsized":"2024-05-08T21:17:56.420144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.140609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416523+0000","last_change":"2024-05-08T21:17:03.234712+0000","last_active":"2024-05-08T21:17:56.416523+0000","last_peered":"2024-05-08T21:17:56.416523+0000","last_clean":"2024-05-08T21:17:56.416523+0000","last_became_active":"2024-05-08T21:17:03.234404+0000","last_became_peered":"2024-05-08T21:17:03.234404+0000","last_unstale":"2024-05-08T21:17:56.416523+0000","last_undegraded":"2024-05-08T21:17:56.416523+0000","last_fullsized":"2024-05-08T21:17:56.416523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:41.479470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422480+0000","last_change":"2024-05-08T21:17:07.259120+0000","last_active":"2024-05-08T21:17:56.422480+0000","last_peered":"2024-05-08T21:17:56.422480+0000","last_clean":"2024-05-08T21:17:56.422480+0000","last_became_active":"2024-05-08T21:17:07.258696+0000","last_became_peered":"2024-05-08T21:17:07.258696+0000","last_unstale":"2024-05-08T21:17:56.422480+0000","last_undegraded":"2024-05-08T21:17:56.422480+0000","last_fullsized":"2024-05-08T21:17:56.422480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:26.950767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423353+0000","last_change":"2024-05-08T21:17:02.727795+0000","last_active":"2024-05-08T21:17:56.423353+0000","last_peered":"2024-05-08T21:17:56.423353+0000","last_clean":"2024-05-08T21:17:56.423353+0000","last_became_active":"2024-05-08T21:17:02.727553+0000","last_became_peered":"2024-05-08T21:17:02.727553+0000","last_unstale":"2024-05-08T21:17:56.423353+0000","last_undegraded":"2024-05-08T21:17:56.423353+0000","last_fullsized":"2024-05-08T21:17:56.423353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:51.540221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423413+0000","last_change":"2024-05-08T21:17:03.247270+0000","last_active":"2024-05-08T21:17:56.423413+0000","last_peered":"2024-05-08T21:17:56.423413+0000","last_clean":"2024-05-08T21:17:56.423413+0000","last_became_active":"2024-05-08T21:17:03.246853+0000","last_became_peered":"2024-05-08T21:17:03.246853+0000","last_unstale":"2024-05-08T21:17:56.423413+0000","last_undegraded":"2024-05-08T21:17:56.423413+0000","last_fullsized":"2024-05-08T21:17:56.423413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:46.224088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422880+0000","last_change":"2024-05-08T21:17:07.439920+0000","last_active":"2024-05-08T21:17:56.422880+0000","last_peered":"2024-05-08T21:17:56.422880+0000","last_clean":"2024-05-08T21:17:56.422880+0000","last_became_active":"2024-05-08T21:17:07.439459+0000","last_became_peered":"2024-05-08T21:17:07.439459+0000","last_unstale":"2024-05-08T21:17:56.422880+0000","last_undegraded":"2024-05-08T21:17:56.422880+0000","last_fullsized":"2024-05-08T21:17:56.422880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:03.008930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422835+0000","last_change":"2024-05-08T21:17:02.731889+0000","last_active":"2024-05-08T21:17:56.422835+0000","last_peered":"2024-05-08T21:17:56.422835+0000","last_clean":"2024-05-08T21:17:56.422835+0000","last_became_active":"2024-05-08T21:17:02.731554+0000","last_became_peered":"2024-05-08T21:17:02.731554+0000","last_unstale":"2024-05-08T21:17:56.422835+0000","last_undegraded":"2024-05-08T21:17:56.422835+0000","last_fullsized":"2024-05-08T21:17:56.422835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:25.223003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417051+0000","last_change":"2024-05-08T21:17:03.252672+0000","last_active":"2024-05-08T21:17:56.417051+0000","last_peered":"2024-05-08T21:17:56.417051+0000","last_clean":"2024-05-08T21:17:56.417051+0000","last_became_active":"2024-05-08T21:17:03.251684+0000","last_became_peered":"2024-05-08T21:17:03.251684+0000","last_unstale":"2024-05-08T21:17:56.417051+0000","last_undegraded":"2024-05-08T21:17:56.417051+0000","last_fullsized":"2024-05-08T21:17:56.417051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:06.460419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420488+0000","last_change":"2024-05-08T21:17:07.273478+0000","last_active":"2024-05-08T21:17:56.420488+0000","last_peered":"2024-05-08T21:17:56.420488+0000","last_clean":"2024-05-08T21:17:56.420488+0000","last_became_active":"2024-05-08T21:17:07.272525+0000","last_became_peered":"2024-05-08T21:17:07.272525+0000","last_unstale":"2024-05-08T21:17:56.420488+0000","last_undegraded":"2024-05-08T21:17:56.420488+0000","last_fullsized":"2024-05-08T21:17:56.420488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:51:05.107387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417904+0000","last_change":"2024-05-08T21:17:02.728858+0000","last_active":"2024-05-08T21:17:56.417904+0000","last_peered":"2024-05-08T21:17:56.417904+0000","last_clean":"2024-05-08T21:17:56.417904+0000","last_became_active":"2024-05-08T21:17:02.728516+0000","last_became_peered":"2024-05-08T21:17:02.728516+0000","last_unstale":"2024-05-08T21:17:56.417904+0000","last_undegraded":"2024-05-08T21:17:56.417904+0000","last_fullsized":"2024-05-08T21:17:56.417904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:40.263983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418555+0000","last_change":"2024-05-08T21:17:03.239354+0000","last_active":"2024-05-08T21:17:56.418555+0000","last_peered":"2024-05-08T21:17:56.418555+0000","last_clean":"2024-05-08T21:17:56.418555+0000","last_became_active":"2024-05-08T21:17:03.239020+0000","last_became_peered":"2024-05-08T21:17:03.239020+0000","last_unstale":"2024-05-08T21:17:56.418555+0000","last_undegraded":"2024-05-08T21:17:56.418555+0000","last_fullsized":"2024-05-08T21:17:56.418555+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:02:43.362016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422887+0000","last_change":"2024-05-08T21:17:07.262401+0000","last_active":"2024-05-08T21:17:56.422887+0000","last_peered":"2024-05-08T21:17:56.422887+0000","last_clean":"2024-05-08T21:17:56.422887+0000","last_became_active":"2024-05-08T21:17:07.261946+0000","last_became_peered":"2024-05-08T21:17:07.261946+0000","last_unstale":"2024-05-08T21:17:56.422887+0000","last_undegraded":"2024-05-08T21:17:56.422887+0000","last_fullsized":"2024-05-08T21:17:56.422887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:56.565604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.438045+0000","last_change":"2024-05-08T21:17:57.437573+0000","last_active":"2024-05-08T21:17:57.438045+0000","last_peered":"2024-05-08T21:17:57.438045+0000","last_clean":"2024-05-08T21:17:57.438045+0000","last_became_active":"2024-05-08T21:17:57.432315+0000","last_became_peered":"2024-05-08T21:17:57.432315+0000","last_unstale":"2024-05-08T21:17:57.438045+0000","last_undegraded":"2024-05-08T21:17:57.438045+0000","last_fullsized":"2024-05-08T21:17:57.438045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:56:08.847290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417312+0000","last_change":"2024-05-08T21:17:03.250382+0000","last_active":"2024-05-08T21:17:56.417312+0000","last_peered":"2024-05-08T21:17:56.417312+0000","last_clean":"2024-05-08T21:17:56.417312+0000","last_became_active":"2024-05-08T21:17:03.250031+0000","last_became_peered":"2024-05-08T21:17:03.250031+0000","last_unstale":"2024-05-08T21:17:56.417312+0000","last_undegraded":"2024-05-08T21:17:56.417312+0000","last_fullsized":"2024-05-08T21:17:56.417312+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:59.961012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422877+0000","last_change":"2024-05-08T21:17:07.439341+0000","last_active":"2024-05-08T21:17:56.422877+0000","last_peered":"2024-05-08T21:17:56.422877+0000","last_clean":"2024-05-08T21:17:56.422877+0000","last_became_active":"2024-05-08T21:17:07.438896+0000","last_became_peered":"2024-05-08T21:17:07.438896+0000","last_unstale":"2024-05-08T21:17:56.422877+0000","last_undegraded":"2024-05-08T21:17:56.422877+0000","last_fullsized":"2024-05-08T21:17:56.422877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:55.180686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416194+0000","last_change":"2024-05-08T21:17:02.732967+0000","last_active":"2024-05-08T21:17:56.416194+0000","last_peered":"2024-05-08T21:17:56.416194+0000","last_clean":"2024-05-08T21:17:56.416194+0000","last_became_active":"2024-05-08T21:17:02.732616+0000","last_became_peered":"2024-05-08T21:17:02.732616+0000","last_unstale":"2024-05-08T21:17:56.416194+0000","last_undegraded":"2024-05-08T21:17:56.416194+0000","last_fullsized":"2024-05-08T21:17:56.416194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:50:26.116841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420367+0000","last_change":"2024-05-08T21:17:03.241974+0000","last_active":"2024-05-08T21:17:56.420367+0000","last_peered":"2024-05-08T21:17:56.420367+0000","last_clean":"2024-05-08T21:17:56.420367+0000","last_became_active":"2024-05-08T21:17:03.241627+0000","last_became_peered":"2024-05-08T21:17:03.241627+0000","last_unstale":"2024-05-08T21:17:56.420367+0000","last_undegraded":"2024-05-08T21:17:56.420367+0000","last_fullsized":"2024-05-08T21:17:56.420367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:57.351078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424107+0000","last_change":"2024-05-08T21:17:07.259593+0000","last_active":"2024-05-08T21:17:56.424107+0000","last_peered":"2024-05-08T21:17:56.424107+0000","last_clean":"2024-05-08T21:17:56.424107+0000","last_became_active":"2024-05-08T21:17:07.258362+0000","last_became_peered":"2024-05-08T21:17:07.258362+0000","last_unstale":"2024-05-08T21:17:56.424107+0000","last_undegraded":"2024-05-08T21:17:56.424107+0000","last_fullsized":"2024-05-08T21:17:56.424107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:07.877264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"85'6","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:10.380480+0000","last_change":"2024-05-08T21:17:02.731685+0000","last_active":"2024-05-08T21:18:10.380480+0000","last_peered":"2024-05-08T21:18:10.380480+0000","last_clean":"2024-05-08T21:18:10.380480+0000","last_became_active":"2024-05-08T21:17:02.731293+0000","last_became_peered":"2024-05-08T21:17:02.731293+0000","last_unstale":"2024-05-08T21:18:10.380480+0000","last_undegraded":"2024-05-08T21:18:10.380480+0000","last_fullsized":"2024-05-08T21:18:10.380480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:36.532962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419974+0000","last_change":"2024-05-08T21:17:03.250090+0000","last_active":"2024-05-08T21:17:56.419974+0000","last_peered":"2024-05-08T21:17:56.419974+0000","last_clean":"2024-05-08T21:17:56.419974+0000","last_became_active":"2024-05-08T21:17:03.249744+0000","last_became_peered":"2024-05-08T21:17:03.249744+0000","last_unstale":"2024-05-08T21:17:56.419974+0000","last_undegraded":"2024-05-08T21:17:56.419974+0000","last_fullsized":"2024-05-08T21:17:56.419974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:43.155040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422926+0000","last_change":"2024-05-08T21:17:07.264101+0000","last_active":"2024-05-08T21:17:56.422926+0000","last_peered":"2024-05-08T21:17:56.422926+0000","last_clean":"2024-05-08T21:17:56.422926+0000","last_became_active":"2024-05-08T21:17:07.263720+0000","last_became_peered":"2024-05-08T21:17:07.263720+0000","last_unstale":"2024-05-08T21:17:56.422926+0000","last_undegraded":"2024-05-08T21:17:56.422926+0000","last_fullsized":"2024-05-08T21:17:56.422926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:16.943044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419108+0000","last_change":"2024-05-08T21:17:02.726763+0000","last_active":"2024-05-08T21:17:56.419108+0000","last_peered":"2024-05-08T21:17:56.419108+0000","last_clean":"2024-05-08T21:17:56.419108+0000","last_became_active":"2024-05-08T21:17:02.726343+0000","last_became_peered":"2024-05-08T21:17:02.726343+0000","last_unstale":"2024-05-08T21:17:56.419108+0000","last_undegraded":"2024-05-08T21:17:56.419108+0000","last_fullsized":"2024-05-08T21:17:56.419108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:03.504667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420530+0000","last_change":"2024-05-08T21:17:03.244468+0000","last_active":"2024-05-08T21:17:56.420530+0000","last_peered":"2024-05-08T21:17:56.420530+0000","last_clean":"2024-05-08T21:17:56.420530+0000","last_became_active":"2024-05-08T21:17:03.244132+0000","last_became_peered":"2024-05-08T21:17:03.244132+0000","last_unstale":"2024-05-08T21:17:56.420530+0000","last_undegraded":"2024-05-08T21:17:56.420530+0000","last_fullsized":"2024-05-08T21:17:56.420530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:17.423657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418261+0000","last_change":"2024-05-08T21:17:07.260981+0000","last_active":"2024-05-08T21:17:56.418261+0000","last_peered":"2024-05-08T21:17:56.418261+0000","last_clean":"2024-05-08T21:17:56.418261+0000","last_became_active":"2024-05-08T21:17:07.260582+0000","last_became_peered":"2024-05-08T21:17:07.260582+0000","last_unstale":"2024-05-08T21:17:56.418261+0000","last_undegraded":"2024-05-08T21:17:56.418261+0000","last_fullsized":"2024-05-08T21:17:56.418261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:25.319411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414755+0000","last_change":"2024-05-08T21:17:02.732531+0000","last_active":"2024-05-08T21:17:56.414755+0000","last_peered":"2024-05-08T21:17:56.414755+0000","last_clean":"2024-05-08T21:17:56.414755+0000","last_became_active":"2024-05-08T21:17:02.732121+0000","last_became_peered":"2024-05-08T21:17:02.732121+0000","last_unstale":"2024-05-08T21:17:56.414755+0000","last_undegraded":"2024-05-08T21:17:56.414755+0000","last_fullsized":"2024-05-08T21:17:56.414755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:25.482458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420635+0000","last_change":"2024-05-08T21:17:03.249676+0000","last_active":"2024-05-08T21:17:56.420635+0000","last_peered":"2024-05-08T21:17:56.420635+0000","last_clean":"2024-05-08T21:17:56.420635+0000","last_became_active":"2024-05-08T21:17:03.249120+0000","last_became_peered":"2024-05-08T21:17:03.249120+0000","last_unstale":"2024-05-08T21:17:56.420635+0000","last_undegraded":"2024-05-08T21:17:56.420635+0000","last_fullsized":"2024-05-08T21:17:56.420635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:58:40.678800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418654+0000","last_change":"2024-05-08T21:17:03.245585+0000","last_active":"2024-05-08T21:17:56.418654+0000","last_peered":"2024-05-08T21:17:56.418654+0000","last_clean":"2024-05-08T21:17:56.418654+0000","last_became_active":"2024-05-08T21:17:03.245274+0000","last_became_peered":"2024-05-08T21:17:03.245274+0000","last_unstale":"2024-05-08T21:17:56.418654+0000","last_undegraded":"2024-05-08T21:17:56.418654+0000","last_fullsized":"2024-05-08T21:17:56.418654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:32.003962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423037+0000","last_change":"2024-05-08T21:17:03.246622+0000","last_active":"2024-05-08T21:17:56.423037+0000","last_peered":"2024-05-08T21:17:56.423037+0000","last_clean":"2024-05-08T21:17:56.423037+0000","last_became_active":"2024-05-08T21:17:03.246309+0000","last_became_peered":"2024-05-08T21:17:03.246309+0000","last_unstale":"2024-05-08T21:17:56.423037+0000","last_undegraded":"2024-05-08T21:17:56.423037+0000","last_fullsized":"2024-05-08T21:17:56.423037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:32.439389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422776+0000","last_change":"2024-05-08T21:17:03.243460+0000","last_active":"2024-05-08T21:17:56.422776+0000","last_peered":"2024-05-08T21:17:56.422776+0000","last_clean":"2024-05-08T21:17:56.422776+0000","last_became_active":"2024-05-08T21:17:03.243097+0000","last_became_peered":"2024-05-08T21:17:03.243097+0000","last_unstale":"2024-05-08T21:17:56.422776+0000","last_undegraded":"2024-05-08T21:17:56.422776+0000","last_fullsized":"2024-05-08T21:17:56.422776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:52:14.492180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423038+0000","last_change":"2024-05-08T21:17:03.245077+0000","last_active":"2024-05-08T21:17:56.423038+0000","last_peered":"2024-05-08T21:17:56.423038+0000","last_clean":"2024-05-08T21:17:56.423038+0000","last_became_active":"2024-05-08T21:17:03.244719+0000","last_became_peered":"2024-05-08T21:17:03.244719+0000","last_unstale":"2024-05-08T21:17:56.423038+0000","last_undegraded":"2024-05-08T21:17:56.423038+0000","last_fullsized":"2024-05-08T21:17:56.423038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.388318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415101+0000","last_change":"2024-05-08T21:17:03.245261+0000","last_active":"2024-05-08T21:17:56.415101+0000","last_peered":"2024-05-08T21:17:56.415101+0000","last_clean":"2024-05-08T21:17:56.415101+0000","last_became_active":"2024-05-08T21:17:03.244892+0000","last_became_peered":"2024-05-08T21:17:03.244892+0000","last_unstale":"2024-05-08T21:17:56.415101+0000","last_undegraded":"2024-05-08T21:17:56.415101+0000","last_fullsized":"2024-05-08T21:17:56.415101+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:21.075634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418251+0000","last_change":"2024-05-08T21:17:03.246250+0000","last_active":"2024-05-08T21:17:56.418251+0000","last_peered":"2024-05-08T21:17:56.418251+0000","last_clean":"2024-05-08T21:17:56.418251+0000","last_became_active":"2024-05-08T21:17:03.245807+0000","last_became_peered":"2024-05-08T21:17:03.245807+0000","last_unstale":"2024-05-08T21:17:56.418251+0000","last_undegraded":"2024-05-08T21:17:56.418251+0000","last_fullsized":"2024-05-08T21:17:56.418251+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:20.618260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423477+0000","last_change":"2024-05-08T21:17:03.254559+0000","last_active":"2024-05-08T21:17:56.423477+0000","last_peered":"2024-05-08T21:17:56.423477+0000","last_clean":"2024-05-08T21:17:56.423477+0000","last_became_active":"2024-05-08T21:17:03.254231+0000","last_became_peered":"2024-05-08T21:17:03.254231+0000","last_unstale":"2024-05-08T21:17:56.423477+0000","last_undegraded":"2024-05-08T21:17:56.423477+0000","last_fullsized":"2024-05-08T21:17:56.423477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:47.996927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418065+0000","last_change":"2024-05-08T21:17:03.242479+0000","last_active":"2024-05-08T21:17:56.418065+0000","last_peered":"2024-05-08T21:17:56.418065+0000","last_clean":"2024-05-08T21:17:56.418065+0000","last_became_active":"2024-05-08T21:17:03.241397+0000","last_became_peered":"2024-05-08T21:17:03.241397+0000","last_unstale":"2024-05-08T21:17:56.418065+0000","last_undegraded":"2024-05-08T21:17:56.418065+0000","last_fullsized":"2024-05-08T21:17:56.418065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:39.029139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419220+0000","last_change":"2024-05-08T21:17:03.244208+0000","last_active":"2024-05-08T21:17:56.419220+0000","last_peered":"2024-05-08T21:17:56.419220+0000","last_clean":"2024-05-08T21:17:56.419220+0000","last_became_active":"2024-05-08T21:17:03.243828+0000","last_became_peered":"2024-05-08T21:17:03.243828+0000","last_unstale":"2024-05-08T21:17:56.419220+0000","last_undegraded":"2024-05-08T21:17:56.419220+0000","last_fullsized":"2024-05-08T21:17:56.419220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:39.936750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418012+0000","last_change":"2024-05-08T21:17:03.240135+0000","last_active":"2024-05-08T21:17:56.418012+0000","last_peered":"2024-05-08T21:17:56.418012+0000","last_clean":"2024-05-08T21:17:56.418012+0000","last_became_active":"2024-05-08T21:17:03.239757+0000","last_became_peered":"2024-05-08T21:17:03.239757+0000","last_unstale":"2024-05-08T21:17:56.418012+0000","last_undegraded":"2024-05-08T21:17:56.418012+0000","last_fullsized":"2024-05-08T21:17:56.418012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:46.066111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424382+0000","last_change":"2024-05-08T21:17:03.256254+0000","last_active":"2024-05-08T21:17:56.424382+0000","last_peered":"2024-05-08T21:17:56.424382+0000","last_clean":"2024-05-08T21:17:56.424382+0000","last_became_active":"2024-05-08T21:17:03.255310+0000","last_became_peered":"2024-05-08T21:17:03.255310+0000","last_unstale":"2024-05-08T21:17:56.424382+0000","last_undegraded":"2024-05-08T21:17:56.424382+0000","last_fullsized":"2024-05-08T21:17:56.424382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:59.571958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412843+0000","last_change":"2024-05-08T21:17:03.241901+0000","last_active":"2024-05-08T21:17:56.412843+0000","last_peered":"2024-05-08T21:17:56.412843+0000","last_clean":"2024-05-08T21:17:56.412843+0000","last_became_active":"2024-05-08T21:17:03.241402+0000","last_became_peered":"2024-05-08T21:17:03.241402+0000","last_unstale":"2024-05-08T21:17:56.412843+0000","last_undegraded":"2024-05-08T21:17:56.412843+0000","last_fullsized":"2024-05-08T21:17:56.412843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:59.202987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417882+0000","last_change":"2024-05-08T21:17:03.243050+0000","last_active":"2024-05-08T21:17:56.417882+0000","last_peered":"2024-05-08T21:17:56.417882+0000","last_clean":"2024-05-08T21:17:56.417882+0000","last_became_active":"2024-05-08T21:17:03.240676+0000","last_became_peered":"2024-05-08T21:17:03.240676+0000","last_unstale":"2024-05-08T21:17:56.417882+0000","last_undegraded":"2024-05-08T21:17:56.417882+0000","last_fullsized":"2024-05-08T21:17:56.417882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:19:53.981559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68912,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":428,"num_read_kb":288,"num_write":239,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1246,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":206736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514520,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":483699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.69299999999999995}]},{"osd":1,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53800000000000003}]},{"osd":2,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.67800000000000005}]},{"osd":5,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.62},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.753}]},{"osd":7,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71499999999999997}]},{"osd":8,"last update":"Wed May 8 21:18:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.81299999999999994}]},{"osd":9,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.83699999999999997}]},{"osd":10,"last update":"Wed May 8 21:17:32 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.80100000000000005}]}]},{"osd":10,"up_from":69,"seq":296352743451,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36856,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708296,"statfs":{"total":95995035648,"available":95957295104,"internally_reserved":0,"allocated":1679360,"data_stored":1132080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.55600000000000005}]},{"osd":1,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.376}]},{"osd":2,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.53100000000000003}]},{"osd":3,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.54400000000000004}]},{"osd":4,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.39800000000000002}]},{"osd":5,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57299999999999995}]},{"osd":6,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.625},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":8,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.61599999999999999}]},{"osd":9,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.68400000000000005}]},{"osd":11,"last update":"Wed May 8 21:17:31 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.67200000000000004}]}]},{"osd":9,"up_from":64,"seq":274877906974,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":487380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.48099999999999998,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65000000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.5,"15min":0.50800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63700000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.442}]},{"osd":1,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.84399999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81100000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.46300000000000002}]},{"osd":2,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76500000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63400000000000001,"15min":0.64300000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.58599999999999997}]},{"osd":3,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":1.048,"15min":1.048},"last":0.61499999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.496,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.77800000000000002,"15min":0.79500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92300000000000004,"5min":1.234,"15min":1.234},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.056,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.875},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.47099999999999997}]},{"osd":7,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.81599999999999995,"15min":0.83099999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96099999999999997,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.84199999999999997,"15min":0.86399999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.73499999999999999}]},{"osd":8,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.80100000000000005,"15min":0.82299999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93600000000000005,"5min":1.101,"15min":1.101},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.88600000000000001,"15min":0.90700000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.004,"5min":1.589,"15min":1.589},"last":0.77000000000000002}]},{"osd":10,"last update":"Wed May 8 21:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":8,"up_from":56,"seq":240518168611,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":482659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.78200000000000003,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.442},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41399999999999998}]},{"osd":1,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.33500000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79200000000000004,"5min":0.87,"15min":0.87},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75900000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.68000000000000005}]},{"osd":2,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.315,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.64500000000000002,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.438},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.51800000000000002}]},{"osd":3,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79800000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.45900000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.879,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.501}]},{"osd":4,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.35999999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.41899999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.626}]},{"osd":5,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51300000000000001,"15min":0.503},"min":{"1min":0.38700000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54500000000000004,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.65600000000000003}]},{"osd":6,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.67200000000000004,"15min":0.66800000000000004},"min":{"1min":0.50700000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.503,"5min":0.373,"15min":0.373},"max":{"1min":0.93500000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69999999999999996}]},{"osd":9,"last update":"Wed May 8 21:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70199999999999996,"15min":0.68899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.745,"15min":0.73499999999999999},"min":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.96999999999999997,"5min":1.04,"15min":1.04},"last":0.67300000000000004}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.71099999999999997}]},{"osd":11,"last update":"Wed May 8 21:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.754},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.76400000000000001}]}]},{"osd":7,"up_from":49,"seq":210453397542,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1597440,"data_stored":1079020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.40300000000000002,"15min":0.39700000000000002},"min":{"1min":0.26300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.499},{"interface":"front","average":{"1min":0.441,"5min":0.40400000000000003,"15min":0.39800000000000002},"min":{"1min":0.27700000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57699999999999996,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54600000000000004,"15min":0.54000000000000004},"min":{"1min":0.437,"5min":0.378,"15min":0.378},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.65900000000000003}]},{"osd":3,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.37,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.63800000000000001}]},{"osd":4,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.33600000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.745},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.316,"5min":0.315,"15min":0.315},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.71099999999999997}]},{"osd":5,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.46800000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.45900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.50800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.754},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.79800000000000004}]},{"osd":8,"last update":"Wed May 8 21:17:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.501,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.52500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76800000000000002}]},{"osd":9,"last update":"Wed May 8 21:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67200000000000004,"15min":0.67600000000000005},"min":{"1min":0.47799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.81999999999999995,"5min":1.008,"15min":1.008},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.86499999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.82099999999999995}]},{"osd":10,"last update":"Wed May 8 21:17:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.78400000000000003}]},{"osd":11,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.86099999999999999}]}]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":478740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.39400000000000002}]},{"osd":1,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.502,"5min":0.434,"15min":0.434},"max":{"1min":1,"5min":1,"15min":1},"last":0.66900000000000004}]},{"osd":2,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.50900000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.629}]},{"osd":4,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.45700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.59499999999999997}]},{"osd":5,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.56000000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.72399999999999998,"15min":0.71199999999999997},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.55200000000000005}]},{"osd":7,"last update":"Wed May 8 21:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.75,"15min":0.75600000000000001},"min":{"1min":0.56699999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90900000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.64200000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0169999999999999,"5min":1.034,"15min":1.034},"last":0.73599999999999999}]},{"osd":8,"last update":"Wed May 8 21:17:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.94799999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.69899999999999995}]},{"osd":9,"last update":"Wed May 8 21:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.80400000000000005,"15min":0.79300000000000004},"min":{"1min":0.54800000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.76700000000000002},"min":{"1min":0.625,"5min":0.498,"15min":0.498},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.79400000000000004}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.753},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.77000000000000002}]},{"osd":11,"last update":"Wed May 8 21:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.68000000000000005}]}]},{"osd":1,"up_from":12,"seq":51539607611,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":542731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.52700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55000000000000004,"15min":0.52700000000000002},"min":{"1min":0.36099999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89800000000000002,"5min":1.548,"15min":1.548},"last":0.44700000000000001}]},{"osd":2,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.54000000000000004,"15min":0.50700000000000001},"min":{"1min":0.41699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.52900000000000003,"15min":0.496},"min":{"1min":0.40600000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80900000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68400000000000005}]},{"osd":3,"last update":"Wed May 8 21:17:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.52900000000000003,"15min":0.50700000000000001},"min":{"1min":0.38500000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.55300000000000005,"15min":0.51600000000000001},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96199999999999997,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.60999999999999999}]},{"osd":4,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.435,"15min":0.41099999999999998},"min":{"1min":0.315,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.70599999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.432,"15min":0.41699999999999998},"min":{"1min":0.318,"5min":0.253,"15min":0.253},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed May 8 21:17:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.56299999999999994,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.56100000000000005,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.63300000000000001}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.746},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64300000000000002,"15min":0.61199999999999999},"min":{"1min":0.502,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89100000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed May 8 21:17:19 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.92400000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.36799999999999999,"5min":0.308,"15min":0.308},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.65200000000000002}]},{"osd":8,"last update":"Wed May 8 21:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.64300000000000002,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.50900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96599999999999997,"5min":1.607,"15min":1.607},"last":0.76000000000000001}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70099999999999996,"15min":0.69099999999999995},"min":{"1min":0.54300000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73299999999999998,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.84199999999999997}]},{"osd":10,"last update":"Wed May 8 21:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.78600000000000003}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.82499999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":1011712,"data_stored":480589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 21:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.54700000000000004,"15min":0.53100000000000003},"min":{"1min":0.40699999999999997,"5min":0.315,"15min":0.315},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.53000000000000003},"min":{"1min":0.434,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.86099999999999999,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed May 8 21:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.748,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.48999999999999999}]},{"osd":3,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.39700000000000002}]},{"osd":4,"last update":"Wed May 8 21:17:26 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78300000000000003,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.59999999999999998,"15min":0.58099999999999996},"min":{"1min":0.44,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.61699999999999999}]},{"osd":5,"last update":"Wed May 8 21:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86399999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.76400000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.42999999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89700000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.85599999999999998}]},{"osd":7,"last update":"Wed May 8 21:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.80300000000000005}]},{"osd":8,"last update":"Wed May 8 21:17:39 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94999999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.751,"15min":0.75800000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84699999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.75700000000000001}]},{"osd":9,"last update":"Wed May 8 21:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77400000000000002},"min":{"1min":0.65100000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97799999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.71899999999999997}]},{"osd":10,"last update":"Wed May 8 21:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.74099999999999999}]},{"osd":11,"last update":"Wed May 8 21:17:31 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.879}]}]},{"osd":2,"up_from":17,"seq":73014444089,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1069056,"data_stored":542050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.56799999999999995,"15min":0.52100000000000002},"min":{"1min":0.32600000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":6.7270000000000003,"5min":6.7270000000000003,"15min":6.7270000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.51200000000000001},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.72099999999999997}]},{"osd":1,"last update":"Wed May 8 21:17:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.44500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.747},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.42899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68500000000000005,"5min":0.752,"15min":0.752},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65600000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.53500000000000003}]},{"osd":4,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61299999999999999}]},{"osd":5,"last update":"Wed May 8 21:17:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.42899999999999999,"15min":0.40799999999999997},"min":{"1min":0.29799999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.44,"15min":0.41599999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.37}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.55400000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.212,"5min":0.71199999999999997,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":7.7779999999999996,"5min":7.7779999999999996,"15min":7.7779999999999996},"last":0.68799999999999994}]},{"osd":7,"last update":"Wed May 8 21:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73499999999999999}]},{"osd":8,"last update":"Wed May 8 21:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.57799999999999996,"15min":0.56200000000000006},"min":{"1min":0.35799999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57399999999999995,"15min":0.56200000000000006},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48699999999999999}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.67700000000000005,"15min":0.65800000000000003},"min":{"1min":0.66800000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.56799999999999995,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.82899999999999996}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.81699999999999995}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59799999999999998}]}]},{"osd":3,"up_from":24,"seq":103079215157,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":999424,"data_stored":478496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.40500000000000003,"15min":0.38600000000000001},"min":{"1min":0.36199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35599999999999998}]},{"osd":1,"last update":"Wed May 8 21:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.52500000000000002,"15min":0.51100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.53500000000000003,"15min":0.51500000000000001},"min":{"1min":0.40999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.45300000000000001}]},{"osd":2,"last update":"Wed May 8 21:17:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57199999999999995,"15min":0.55600000000000005},"min":{"1min":0.39700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.314},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.32900000000000001}]},{"osd":4,"last update":"Wed May 8 21:17:27 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.57499999999999996,"15min":0.54100000000000004},"min":{"1min":0.47899999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.88300000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.60699999999999998}]},{"osd":5,"last update":"Wed May 8 21:17:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.56200000000000006},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.496},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.61199999999999999},"min":{"1min":0.46800000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Wed May 8 21:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.504,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.55100000000000005}]},{"osd":7,"last update":"Wed May 8 21:17:23 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.52500000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.748}]},{"osd":8,"last update":"Wed May 8 21:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79800000000000004}]},{"osd":9,"last update":"Wed May 8 21:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.628}]},{"osd":10,"last update":"Wed May 8 21:17:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.65600000000000003}]},{"osd":11,"last update":"Wed May 8 21:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.72399999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986225,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":483388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.373,"15min":0.36299999999999999},"min":{"1min":0.27700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.39600000000000002,"15min":0.39100000000000001},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.39400000000000002}]},{"osd":2,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.56000000000000005,"15min":0.54400000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72899999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.746}]},{"osd":3,"last update":"Wed May 8 21:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.52100000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.46700000000000003}]},{"osd":5,"last update":"Wed May 8 21:17:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.376,"5min":0.371,"15min":0.371},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.57599999999999996,"15min":0.53800000000000003},"min":{"1min":0.54600000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59999999999999998,"15min":0.57999999999999996},"min":{"1min":0.51200000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82399999999999995,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.60899999999999999}]},{"osd":7,"last update":"Wed May 8 21:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.5,"15min":0.47999999999999998},"min":{"1min":0.34300000000000003,"5min":0.307,"15min":0.307},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.53400000000000003,"15min":0.51200000000000001},"min":{"1min":0.36599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.54700000000000004}]},{"osd":8,"last update":"Wed May 8 21:17:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70699999999999996,"15min":0.69599999999999995},"min":{"1min":0.58299999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94099999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.72999999999999998}]},{"osd":9,"last update":"Wed May 8 21:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.747,"15min":0.747},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.876,"5min":1.347,"15min":1.347},"last":0.56200000000000006}]},{"osd":10,"last update":"Wed May 8 21:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.59099999999999997}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.76500000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36792,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708360,"statfs":{"total":95995035648,"available":95957360640,"internally_reserved":0,"allocated":1613824,"data_stored":1075302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.52300000000000002},"min":{"1min":0.36799999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.622,"5min":0.877,"15min":0.877},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.51700000000000002},"min":{"1min":0.38500000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67500000000000004,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.52200000000000002}]},{"osd":1,"last update":"Wed May 8 21:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.54400000000000004},"min":{"1min":0.42099999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77600000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.41199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71699999999999997,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed May 8 21:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40300000000000002,"15min":0.39900000000000002},"min":{"1min":0.28000000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61199999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.40600000000000003},"min":{"1min":0.32000000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.59999999999999998}]},{"osd":3,"last update":"Wed May 8 21:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.80300000000000005,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.39100000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.72199999999999998,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.747}]},{"osd":4,"last update":"Wed May 8 21:17:47 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.56599999999999995,"15min":0.56200000000000006},"min":{"1min":0.436,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84199999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.44900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76000000000000001,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.71299999999999997}]},{"osd":6,"last update":"Wed May 8 21:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.75700000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.623,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93300000000000005,"5min":1.01,"15min":1.01},"last":0.67200000000000004}]},{"osd":7,"last update":"Wed May 8 21:17:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81999999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.81399999999999995}]},{"osd":8,"last update":"Wed May 8 21:17:36 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91800000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.38100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86799999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.76900000000000002}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.55100000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88900000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.83999999999999997}]},{"osd":10,"last update":"Wed May 8 21:17:11 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.86099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":63977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T21:18:15.045 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T21:18:15.045 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T21:18:15.045 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-08T21:18:15.045 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-08T21:18:15.340 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:15.522 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:15 smithi033 ceph-mon[42528]: pgmap v266: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:15.522 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:15 smithi033 ceph-mon[42528]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:15.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:15 smithi071 ceph-mon[47074]: pgmap v266: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:15.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:15 smithi071 ceph-mon[47074]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:15.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:15 smithi082 ceph-mon[47512]: pgmap v266: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:15.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:15 smithi082 ceph-mon[47512]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.428+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/2902202004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0dc0ffed0 msgr2=0x7fd0dc100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.428+0000 7fd0e0cf9700 1 --2- 172.21.15.33:0/2902202004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0dc0ffed0 0x7fd0dc100350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009990 tx=0x7fd0d0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.428+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/2902202004 shutdown_connections 2024-05-08T21:18:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.428+0000 7fd0e0cf9700 1 --2- 172.21.15.33:0/2902202004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0dc100890 0x7fd0dc10cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.428+0000 7fd0e0cf9700 1 --2- 172.21.15.33:0/2902202004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc105f00 0x7fd0dc1062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.428+0000 7fd0e0cf9700 1 --2- 172.21.15.33:0/2902202004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0dc0ffed0 0x7fd0dc100350 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.428+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/2902202004 >> 172.21.15.33:0/2902202004 conn(0x7fd0dc075f80 msgr2=0x7fd0dc076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.428+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/2902202004 shutdown_connections 2024-05-08T21:18:16.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.428+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/2902202004 wait complete. 2024-05-08T21:18:16.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.429+0000 7fd0e0cf9700 1 Processor -- start 2024-05-08T21:18:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.429+0000 7fd0e0cf9700 1 -- start start 2024-05-08T21:18:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.429+0000 7fd0e0cf9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0dc0ffed0 0x7fd0dc197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.429+0000 7fd0e0cf9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc100890 0x7fd0dc197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.429+0000 7fd0e0cf9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0dc105f00 0x7fd0dc19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.429+0000 7fd0e0cf9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc19c1b0 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.429+0000 7fd0e0cf9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc19c320 con 0x7fd0dc100890 2024-05-08T21:18:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.429+0000 7fd0e0cf9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc19c4e0 con 0x7fd0dc105f00 2024-05-08T21:18:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.429+0000 7fd0da59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0dc0ffed0 0x7fd0dc197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0da59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0dc0ffed0 0x7fd0dc197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48744/0 (socket says 172.21.15.33:48744) 2024-05-08T21:18:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0da59c700 1 -- 172.21.15.33:0/4286017814 learned_addr learned my addr 172.21.15.33:0/4286017814 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:16.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0d9d9b700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc100890 0x7fd0dc197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0dad9d700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0dc105f00 0x7fd0dc19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0da59c700 1 -- 172.21.15.33:0/4286017814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0dc105f00 msgr2=0x7fd0dc19bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0da59c700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0dc105f00 0x7fd0dc19bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0da59c700 1 -- 172.21.15.33:0/4286017814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc100890 msgr2=0x7fd0dc197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0da59c700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc100890 0x7fd0dc197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0da59c700 1 -- 172.21.15.33:0/4286017814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d00095f0 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0d9d9b700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc100890 0x7fd0dc197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0da59c700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0dc0ffed0 0x7fd0dc197300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c9b0 tx=0x7fd0cc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0cb7fe700 1 -- 172.21.15.33:0/4286017814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc007b00 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/4286017814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0dc19c7c0 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0cb7fe700 1 -- 172.21.15.33:0/4286017814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0cc007c60 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0cb7fe700 1 -- 172.21.15.33:0/4286017814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc018660 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.430+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/4286017814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0dc1a4020 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.431+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/4286017814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0dc1015b0 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.433+0000 7fd0cb7fe700 1 -- 172.21.15.33:0/4286017814 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd0cc00f450 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.433+0000 7fd0cb7fe700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd0c40a11b0 0x7fd0c40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:16.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.433+0000 7fd0d9d9b700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd0c40a11b0 0x7fd0c40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:16.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.433+0000 7fd0cb7fe700 1 -- 172.21.15.33:0/4286017814 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd0cc0cd000 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.434+0000 7fd0d9d9b700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd0c40a11b0 0x7fd0c40a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc198810 tx=0x7fd0d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:16.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.437+0000 7fd0cb7fe700 1 -- 172.21.15.33:0/4286017814 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd0cc0963b0 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.694+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/4286017814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd0dc04ec60 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.695+0000 7fd0cb7fe700 1 -- 172.21.15.33:0/4286017814 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd0cc0999d0 con 0x7fd0dc0ffed0 2024-05-08T21:18:16.698 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:16.698 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T21:18:16.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.697+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/4286017814 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd0c40a11b0 msgr2=0x7fd0c40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:16.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.697+0000 7fd0e0cf9700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd0c40a11b0 0x7fd0c40a3670 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc198810 tx=0x7fd0d0005960 comp rx=0 tx=0).stop 2024-05-08T21:18:16.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.697+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/4286017814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0dc0ffed0 msgr2=0x7fd0dc197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:16.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.697+0000 7fd0e0cf9700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0dc0ffed0 0x7fd0dc197300 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c9b0 tx=0x7fd0cc00cd70 comp rx=0 tx=0).stop 2024-05-08T21:18:16.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.698+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/4286017814 shutdown_connections 2024-05-08T21:18:16.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.698+0000 7fd0e0cf9700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0dc105f00 0x7fd0dc19bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:16.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.698+0000 7fd0e0cf9700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0dc0ffed0 0x7fd0dc197300 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:16.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.698+0000 7fd0e0cf9700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd0c40a11b0 0x7fd0c40a3670 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:16.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.698+0000 7fd0e0cf9700 1 --2- 172.21.15.33:0/4286017814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc100890 0x7fd0dc197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:16.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.698+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/4286017814 >> 172.21.15.33:0/4286017814 conn(0x7fd0dc075f80 msgr2=0x7fd0dc068bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:16.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.698+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/4286017814 shutdown_connections 2024-05-08T21:18:16.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:16.698+0000 7fd0e0cf9700 1 -- 172.21.15.33:0/4286017814 wait complete. 2024-05-08T21:18:17.127 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-08T21:18:17.128 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:18:17.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.275+0000 7fdacea0f700 1 -- 172.21.15.33:0/565298598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac80ff160 msgr2=0x7fdac80ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:17.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.275+0000 7fdacea0f700 1 --2- 172.21.15.33:0/565298598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac80ff160 0x7fdac80ff580 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009990 tx=0x7fdab8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.275+0000 7fdacea0f700 1 -- 172.21.15.33:0/565298598 shutdown_connections 2024-05-08T21:18:17.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.275+0000 7fdacea0f700 1 --2- 172.21.15.33:0/565298598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdac8102ad0 0x7fdac8109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.275+0000 7fdacea0f700 1 --2- 172.21.15.33:0/565298598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac80ff160 0x7fdac80ff580 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.275+0000 7fdacea0f700 1 --2- 172.21.15.33:0/565298598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac8102130 0x7fdac8102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.275+0000 7fdacea0f700 1 -- 172.21.15.33:0/565298598 >> 172.21.15.33:0/565298598 conn(0x7fdac80faa90 msgr2=0x7fdac80fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:17.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.275+0000 7fdacea0f700 1 -- 172.21.15.33:0/565298598 shutdown_connections 2024-05-08T21:18:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.275+0000 7fdacea0f700 1 -- 172.21.15.33:0/565298598 wait complete. 2024-05-08T21:18:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdacea0f700 1 Processor -- start 2024-05-08T21:18:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdacea0f700 1 -- start start 2024-05-08T21:18:17.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdacea0f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdac80ff160 0x7fdac8101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:17.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdacea0f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac8102130 0x7fdac81000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:17.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdacea0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac8102ad0 0x7fdac8100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:17.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdacea0f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8100dc0 con 0x7fdac8102ad0 2024-05-08T21:18:17.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdacea0f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8100f30 con 0x7fdac80ff160 2024-05-08T21:18:17.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdacea0f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac81a0db0 con 0x7fdac8102130 2024-05-08T21:18:17.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdaccfac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac8102ad0 0x7fdac8100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:17.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdaccfac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac8102ad0 0x7fdac8100620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48764/0 (socket says 172.21.15.33:48764) 2024-05-08T21:18:17.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdaccfac700 1 -- 172.21.15.33:0/1074093927 learned_addr learned my addr 172.21.15.33:0/1074093927 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:17.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdac77fe700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac8102130 0x7fdac81000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:17.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.276+0000 7fdac7fff700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdac80ff160 0x7fdac8101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:17.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.277+0000 7fdaccfac700 1 -- 172.21.15.33:0/1074093927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac8102130 msgr2=0x7fdac81000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:17.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.277+0000 7fdaccfac700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac8102130 0x7fdac81000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.277+0000 7fdaccfac700 1 -- 172.21.15.33:0/1074093927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdac80ff160 msgr2=0x7fdac8101a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:17.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.277+0000 7fdaccfac700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdac80ff160 0x7fdac8101a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.277+0000 7fdaccfac700 1 -- 172.21.15.33:0/1074093927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab80095f0 con 0x7fdac8102ad0 2024-05-08T21:18:17.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.277+0000 7fdac77fe700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac8102130 0x7fdac81000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:17.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.277+0000 7fdaccfac700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac8102ad0 0x7fdac8100620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdac000ef30 tx=0x7fdac000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:17.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.277+0000 7fdac57fa700 1 -- 172.21.15.33:0/1074093927 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0018070 con 0x7fdac8102ad0 2024-05-08T21:18:17.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.277+0000 7fdac57fa700 1 -- 172.21.15.33:0/1074093927 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdac000f040 con 0x7fdac8102ad0 2024-05-08T21:18:17.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.277+0000 7fdacea0f700 1 -- 172.21.15.33:0/1074093927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac81a0fe0 con 0x7fdac8102ad0 2024-05-08T21:18:17.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.277+0000 7fdac57fa700 1 -- 172.21.15.33:0/1074093927 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac000cb60 con 0x7fdac8102ad0 2024-05-08T21:18:17.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.277+0000 7fdacea0f700 1 -- 172.21.15.33:0/1074093927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac81a1460 con 0x7fdac8102ad0 2024-05-08T21:18:17.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.278+0000 7fdacea0f700 1 -- 172.21.15.33:0/1074093927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac8108460 con 0x7fdac8102ad0 2024-05-08T21:18:17.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.279+0000 7fdac57fa700 1 -- 172.21.15.33:0/1074093927 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdac000cd20 con 0x7fdac8102ad0 2024-05-08T21:18:17.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.280+0000 7fdac57fa700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdab00a1230 0x7fdab00a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:17.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.280+0000 7fdac57fa700 1 -- 172.21.15.33:0/1074093927 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdac00ccd10 con 0x7fdac8102ad0 2024-05-08T21:18:17.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.280+0000 7fdac7fff700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdab00a1230 0x7fdab00a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:17.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.281+0000 7fdac7fff700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdab00a1230 0x7fdab00a36f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdab8006010 tx=0x7fdab800b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:17.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.284+0000 7fdac57fa700 1 -- 172.21.15.33:0/1074093927 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdac00960c0 con 0x7fdac8102ad0 2024-05-08T21:18:17.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:17 smithi033 ceph-mon[42528]: pgmap v267: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:17.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:17 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4286017814' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T21:18:17.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.504+0000 7fdacea0f700 1 -- 172.21.15.33:0/1074093927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdac8067480 con 0x7fdac8102ad0 2024-05-08T21:18:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.505+0000 7fdac57fa700 1 -- 172.21.15.33:0/1074093927 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24374 (secure 0 0 0) 0x7fdac001ce80 con 0x7fdac8102ad0 2024-05-08T21:18:17.508 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:17.508 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:57.410336+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:18:17.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.508+0000 7fdacea0f700 1 -- 172.21.15.33:0/1074093927 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdab00a1230 msgr2=0x7fdab00a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:17.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.508+0000 7fdacea0f700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdab00a1230 0x7fdab00a36f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdab8006010 tx=0x7fdab800b610 comp rx=0 tx=0).stop 2024-05-08T21:18:17.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.508+0000 7fdacea0f700 1 -- 172.21.15.33:0/1074093927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac8102ad0 msgr2=0x7fdac8100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:17.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.508+0000 7fdacea0f700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac8102ad0 0x7fdac8100620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdac000ef30 tx=0x7fdac000c3a0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.508+0000 7fdacea0f700 1 -- 172.21.15.33:0/1074093927 shutdown_connections 2024-05-08T21:18:17.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.508+0000 7fdacea0f700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdac8102ad0 0x7fdac8100620 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.508+0000 7fdacea0f700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdab00a1230 0x7fdab00a36f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.508+0000 7fdacea0f700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdac80ff160 0x7fdac8101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.508+0000 7fdacea0f700 1 --2- 172.21.15.33:0/1074093927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdac8102130 0x7fdac81000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.508+0000 7fdacea0f700 1 -- 172.21.15.33:0/1074093927 >> 172.21.15.33:0/1074093927 conn(0x7fdac80faa90 msgr2=0x7fdac81055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:17.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.508+0000 7fdacea0f700 1 -- 172.21.15.33:0/1074093927 shutdown_connections 2024-05-08T21:18:17.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.508+0000 7fdacea0f700 1 -- 172.21.15.33:0/1074093927 wait complete. 2024-05-08T21:18:17.525 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:17:01.282803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:17:02.555552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T21:17:05.686277+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T21:18:17.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:18:17.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:17 smithi071 ceph-mon[47074]: pgmap v267: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:17.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:17 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/4286017814' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T21:18:17.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:17 smithi082 ceph-mon[47512]: pgmap v267: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:17.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:17 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/4286017814' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T21:18:17.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.662+0000 7f1ddb17e700 1 -- 172.21.15.33:0/356461110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40ffd90 msgr2=0x7f1dd41001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.662+0000 7f1ddb17e700 1 --2- 172.21.15.33:0/356461110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40ffd90 0x7f1dd41001b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80099c0 tx=0x7f1dc8009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.662+0000 7f1ddb17e700 1 -- 172.21.15.33:0/356461110 shutdown_connections 2024-05-08T21:18:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.662+0000 7f1ddb17e700 1 --2- 172.21.15.33:0/356461110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4101920 0x7f1dd41081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.662+0000 7f1ddb17e700 1 --2- 172.21.15.33:0/356461110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40ffd90 0x7f1dd41001b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.662+0000 7f1ddb17e700 1 --2- 172.21.15.33:0/356461110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dd4100f80 0x7f1dd41013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.662+0000 7f1ddb17e700 1 -- 172.21.15.33:0/356461110 >> 172.21.15.33:0/356461110 conn(0x7f1dd40fb300 msgr2=0x7f1dd40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.662+0000 7f1ddb17e700 1 -- 172.21.15.33:0/356461110 shutdown_connections 2024-05-08T21:18:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.662+0000 7f1ddb17e700 1 -- 172.21.15.33:0/356461110 wait complete. 2024-05-08T21:18:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1ddb17e700 1 Processor -- start 2024-05-08T21:18:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1ddb17e700 1 -- start start 2024-05-08T21:18:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1ddb17e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40ffd90 0x7f1dd4103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1ddb17e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dd4100f80 0x7f1dd4102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1ddb17e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4101920 0x7f1dd4102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1ddb17e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4103220 con 0x7f1dd40ffd90 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1ddb17e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd4103390 con 0x7f1dd4101920 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1ddb17e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd41a0de0 con 0x7f1dd4100f80 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1dd8f1a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40ffd90 0x7f1dd4103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1dd8f1a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40ffd90 0x7f1dd4103eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48774/0 (socket says 172.21.15.33:48774) 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1dd8f1a700 1 -- 172.21.15.33:0/2396471201 learned_addr learned my addr 172.21.15.33:0/2396471201 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1dd971b700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4101920 0x7f1dd4102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1dd3fff700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dd4100f80 0x7f1dd4102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1dd8f1a700 1 -- 172.21.15.33:0/2396471201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dd4100f80 msgr2=0x7f1dd4102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1dd8f1a700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dd4100f80 0x7f1dd4102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1dd8f1a700 1 -- 172.21.15.33:0/2396471201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4101920 msgr2=0x7f1dd4102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.663+0000 7f1dd8f1a700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4101920 0x7f1dd4102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.664+0000 7f1dd8f1a700 1 -- 172.21.15.33:0/2396471201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc80095f0 con 0x7f1dd40ffd90 2024-05-08T21:18:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.664+0000 7f1dd3fff700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dd4100f80 0x7f1dd4102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:18:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.664+0000 7f1dd8f1a700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40ffd90 0x7f1dd4103eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009990 tx=0x7f1dc800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.664+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/2396471201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc801d070 con 0x7f1dd40ffd90 2024-05-08T21:18:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.664+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/2396471201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1dc800bd10 con 0x7f1dd40ffd90 2024-05-08T21:18:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.664+0000 7f1ddb17e700 1 -- 172.21.15.33:0/2396471201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd41a1010 con 0x7f1dd40ffd90 2024-05-08T21:18:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.664+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/2396471201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8017650 con 0x7f1dd40ffd90 2024-05-08T21:18:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.664+0000 7f1ddb17e700 1 -- 172.21.15.33:0/2396471201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd41a13d0 con 0x7f1dd40ffd90 2024-05-08T21:18:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.665+0000 7f1ddb17e700 1 -- 172.21.15.33:0/2396471201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd4067480 con 0x7f1dd40ffd90 2024-05-08T21:18:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.666+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/2396471201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1dc8004030 con 0x7f1dd40ffd90 2024-05-08T21:18:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.667+0000 7f1dd1ffb700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1dc40a11b0 0x7f1dc40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.667+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/2396471201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1dc80ce460 con 0x7f1dd40ffd90 2024-05-08T21:18:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.667+0000 7f1dd3fff700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1dc40a11b0 0x7f1dc40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:17.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.668+0000 7f1dd3fff700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1dc40a11b0 0x7f1dc40a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0005fd0 tx=0x7f1dc0005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:17.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.671+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/2396471201 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1dc80977b0 con 0x7f1dd40ffd90 2024-05-08T21:18:17.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.889+0000 7f1ddb17e700 1 -- 172.21.15.33:0/2396471201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1dd410b650 con 0x7f1dd40ffd90 2024-05-08T21:18:17.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.889+0000 7f1dd1ffb700 1 -- 172.21.15.33:0/2396471201 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1dc809add0 con 0x7f1dd40ffd90 2024-05-08T21:18:17.895 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:18:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.891+0000 7f1ddb17e700 1 -- 172.21.15.33:0/2396471201 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1dc40a11b0 msgr2=0x7f1dc40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.891+0000 7f1ddb17e700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1dc40a11b0 0x7f1dc40a3670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0005fd0 tx=0x7f1dc0005ee0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.891+0000 7f1ddb17e700 1 -- 172.21.15.33:0/2396471201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40ffd90 msgr2=0x7f1dd4103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.891+0000 7f1ddb17e700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40ffd90 0x7f1dd4103eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009990 tx=0x7f1dc800fab0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.891+0000 7f1ddb17e700 1 -- 172.21.15.33:0/2396471201 shutdown_connections 2024-05-08T21:18:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.891+0000 7f1ddb17e700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1dc40a11b0 0x7f1dc40a3670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.891+0000 7f1ddb17e700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4101920 0x7f1dd4102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.891+0000 7f1ddb17e700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd40ffd90 0x7f1dd4103eb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.892+0000 7f1ddb17e700 1 --2- 172.21.15.33:0/2396471201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dd4100f80 0x7f1dd4102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.892+0000 7f1ddb17e700 1 -- 172.21.15.33:0/2396471201 >> 172.21.15.33:0/2396471201 conn(0x7f1dd40fb300 msgr2=0x7f1dd4069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.892+0000 7f1ddb17e700 1 -- 172.21.15.33:0/2396471201 shutdown_connections 2024-05-08T21:18:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:17.892+0000 7f1ddb17e700 1 -- 172.21.15.33:0/2396471201 wait complete. 2024-05-08T21:18:17.908 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:18:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.046+0000 7f48b5337700 1 -- 172.21.15.33:0/3093693890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0102990 msgr2=0x7f48b01092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.046+0000 7f48b5337700 1 --2- 172.21.15.33:0/3093693890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0102990 0x7f48b01092b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f48a800b0a0 tx=0x7f48a800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.046+0000 7f48b5337700 1 -- 172.21.15.33:0/3093693890 shutdown_connections 2024-05-08T21:18:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.046+0000 7f48b5337700 1 --2- 172.21.15.33:0/3093693890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0102990 0x7f48b01092b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.046+0000 7f48b5337700 1 --2- 172.21.15.33:0/3093693890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48b0100e80 0x7f48b01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.046+0000 7f48b5337700 1 --2- 172.21.15.33:0/3093693890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0101fd0 0x7f48b0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.046+0000 7f48b5337700 1 -- 172.21.15.33:0/3093693890 >> 172.21.15.33:0/3093693890 conn(0x7f48b00fc3f0 msgr2=0x7f48b00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.046+0000 7f48b5337700 1 -- 172.21.15.33:0/3093693890 shutdown_connections 2024-05-08T21:18:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.046+0000 7f48b5337700 1 -- 172.21.15.33:0/3093693890 wait complete. 2024-05-08T21:18:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48b5337700 1 Processor -- start 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48b5337700 1 -- start start 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48b5337700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48b0100e80 0x7f48b0198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48b5337700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0101fd0 0x7f48b0199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48b5337700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0102990 0x7f48b01a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48b5337700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b01a0b20 con 0x7f48b0101fd0 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48b5337700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b01a0c60 con 0x7f48b0100e80 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48b5337700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b01a0df0 con 0x7f48b0102990 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48ae7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0101fd0 0x7f48b0199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48ae7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0101fd0 0x7f48b0199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48798/0 (socket says 172.21.15.33:48798) 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48ae7fc700 1 -- 172.21.15.33:0/2656476848 learned_addr learned my addr 172.21.15.33:0/2656476848 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48af7fe700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0102990 0x7f48b01a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.047+0000 7f48aeffd700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48b0100e80 0x7f48b0198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.048+0000 7f48ae7fc700 1 -- 172.21.15.33:0/2656476848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0102990 msgr2=0x7f48b01a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.048+0000 7f48ae7fc700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0102990 0x7f48b01a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.048+0000 7f48ae7fc700 1 -- 172.21.15.33:0/2656476848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48b0100e80 msgr2=0x7f48b0198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.048+0000 7f48ae7fc700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48b0100e80 0x7f48b0198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.048+0000 7f48ae7fc700 1 -- 172.21.15.33:0/2656476848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a8009d00 con 0x7f48b0101fd0 2024-05-08T21:18:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.048+0000 7f48ae7fc700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0101fd0 0x7f48b0199530 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f48a400e9a0 tx=0x7f48a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.048+0000 7f4897fff700 1 -- 172.21.15.33:0/2656476848 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4009790 con 0x7f48b0101fd0 2024-05-08T21:18:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.048+0000 7f4897fff700 1 -- 172.21.15.33:0/2656476848 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48a400cbd0 con 0x7f48b0101fd0 2024-05-08T21:18:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.048+0000 7f48b5337700 1 -- 172.21.15.33:0/2656476848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b01a10d0 con 0x7f48b0101fd0 2024-05-08T21:18:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.048+0000 7f4897fff700 1 -- 172.21.15.33:0/2656476848 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4018680 con 0x7f48b0101fd0 2024-05-08T21:18:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.048+0000 7f48b5337700 1 -- 172.21.15.33:0/2656476848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b01a15e0 con 0x7f48b0101fd0 2024-05-08T21:18:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.049+0000 7f48b5337700 1 -- 172.21.15.33:0/2656476848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b0068160 con 0x7f48b0101fd0 2024-05-08T21:18:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.050+0000 7f4897fff700 1 -- 172.21.15.33:0/2656476848 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f48a400cd40 con 0x7f48b0101fd0 2024-05-08T21:18:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.051+0000 7f4897fff700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48980a11b0 0x7f48980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.051+0000 7f48aeffd700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48980a11b0 0x7f48980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.051+0000 7f4897fff700 1 -- 172.21.15.33:0/2656476848 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f48a4014070 con 0x7f48b0101fd0 2024-05-08T21:18:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.051+0000 7f48aeffd700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48980a11b0 0x7f48980a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009bf0 tx=0x7f48a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:18.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.055+0000 7f4897fff700 1 -- 172.21.15.33:0/2656476848 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f48a40966d0 con 0x7f48b0101fd0 2024-05-08T21:18:18.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.276+0000 7f48b5337700 1 -- 172.21.15.33:0/2656476848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48b010c6c0 con 0x7f48b0101fd0 2024-05-08T21:18:18.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.276+0000 7f4897fff700 1 -- 172.21.15.33:0/2656476848 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f48a4099cf0 con 0x7f48b0101fd0 2024-05-08T21:18:18.279 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:18.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.279+0000 7f48b5337700 1 -- 172.21.15.33:0/2656476848 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48980a11b0 msgr2=0x7f48980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.279+0000 7f48b5337700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48980a11b0 0x7f48980a3670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009bf0 tx=0x7f48a0009260 comp rx=0 tx=0).stop 2024-05-08T21:18:18.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.279+0000 7f48b5337700 1 -- 172.21.15.33:0/2656476848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0101fd0 msgr2=0x7f48b0199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.279+0000 7f48b5337700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0101fd0 0x7f48b0199530 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f48a400e9a0 tx=0x7f48a400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.279+0000 7f48b5337700 1 -- 172.21.15.33:0/2656476848 shutdown_connections 2024-05-08T21:18:18.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.279+0000 7f48b5337700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0102990 0x7f48b01a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.279+0000 7f48b5337700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48980a11b0 0x7f48980a3670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.279+0000 7f48b5337700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48b0100e80 0x7f48b0198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.279+0000 7f48b5337700 1 --2- 172.21.15.33:0/2656476848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0101fd0 0x7f48b0199530 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.279+0000 7f48b5337700 1 -- 172.21.15.33:0/2656476848 >> 172.21.15.33:0/2656476848 conn(0x7f48b00fc3f0 msgr2=0x7f48b00731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:18.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.279+0000 7f48b5337700 1 -- 172.21.15.33:0/2656476848 shutdown_connections 2024-05-08T21:18:18.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.279+0000 7f48b5337700 1 -- 172.21.15.33:0/2656476848 wait complete. 2024-05-08T21:18:18.296 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:18:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.435+0000 7fa8292e1700 1 -- 172.21.15.33:0/178458996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101920 msgr2=0x7fa824103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.435+0000 7fa8292e1700 1 --2- 172.21.15.33:0/178458996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101920 0x7fa824103d10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009990 tx=0x7fa80c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.435+0000 7fa8292e1700 1 -- 172.21.15.33:0/178458996 shutdown_connections 2024-05-08T21:18:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.435+0000 7fa8292e1700 1 --2- 172.21.15.33:0/178458996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa824106b80 0x7fa824108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.435+0000 7fa8292e1700 1 --2- 172.21.15.33:0/178458996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101920 0x7fa824103d10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.435+0000 7fa8292e1700 1 --2- 172.21.15.33:0/178458996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa824104250 0x7fa824106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.435+0000 7fa8292e1700 1 -- 172.21.15.33:0/178458996 >> 172.21.15.33:0/178458996 conn(0x7fa8240fb330 msgr2=0x7fa8240fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.435+0000 7fa8292e1700 1 -- 172.21.15.33:0/178458996 shutdown_connections 2024-05-08T21:18:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.435+0000 7fa8292e1700 1 -- 172.21.15.33:0/178458996 wait complete. 2024-05-08T21:18:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.436+0000 7fa8292e1700 1 Processor -- start 2024-05-08T21:18:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.436+0000 7fa8292e1700 1 -- start start 2024-05-08T21:18:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.436+0000 7fa8292e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101920 0x7fa824196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.436+0000 7fa8292e1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa824104250 0x7fa824197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.436+0000 7fa8292e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa824106b80 0x7fa82419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.436+0000 7fa8292e1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82419e830 con 0x7fa824101920 2024-05-08T21:18:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.436+0000 7fa8292e1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82419e9a0 con 0x7fa824104250 2024-05-08T21:18:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.436+0000 7fa8292e1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82419eb60 con 0x7fa824106b80 2024-05-08T21:18:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.436+0000 7fa822ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101920 0x7fa824196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.436+0000 7fa822ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101920 0x7fa824196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48812/0 (socket says 172.21.15.33:48812) 2024-05-08T21:18:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.436+0000 7fa822ffd700 1 -- 172.21.15.33:0/2991778744 learned_addr learned my addr 172.21.15.33:0/2991778744 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa8227fc700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa824104250 0x7fa824197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa8237fe700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa824106b80 0x7fa82419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa822ffd700 1 -- 172.21.15.33:0/2991778744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa824106b80 msgr2=0x7fa82419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa822ffd700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa824106b80 0x7fa82419e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa822ffd700 1 -- 172.21.15.33:0/2991778744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa824104250 msgr2=0x7fa824197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa822ffd700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa824104250 0x7fa824197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa822ffd700 1 -- 172.21.15.33:0/2991778744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c0095f0 con 0x7fa824101920 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa822ffd700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101920 0x7fa824196d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa80c006010 tx=0x7fa80c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa81bfff700 1 -- 172.21.15.33:0/2991778744 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c01d070 con 0x7fa824101920 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa8292e1700 1 -- 172.21.15.33:0/2991778744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82419ede0 con 0x7fa824101920 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa81bfff700 1 -- 172.21.15.33:0/2991778744 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa80c00bdb0 con 0x7fa824101920 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa8292e1700 1 -- 172.21.15.33:0/2991778744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82419f1b0 con 0x7fa824101920 2024-05-08T21:18:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.437+0000 7fa81bfff700 1 -- 172.21.15.33:0/2991778744 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c00fd40 con 0x7fa824101920 2024-05-08T21:18:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.438+0000 7fa819ffb700 1 -- 172.21.15.33:0/2991778744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8080052f0 con 0x7fa824101920 2024-05-08T21:18:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.439+0000 7fa81bfff700 1 -- 172.21.15.33:0/2991778744 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa80c00f460 con 0x7fa824101920 2024-05-08T21:18:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.440+0000 7fa81bfff700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa8100a11b0 0x7fa8100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.440+0000 7fa8227fc700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa8100a11b0 0x7fa8100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.440+0000 7fa81bfff700 1 -- 172.21.15.33:0/2991778744 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa80c0cdf50 con 0x7fa824101920 2024-05-08T21:18:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.441+0000 7fa8227fc700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa8100a11b0 0x7fa8100a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa814009c60 tx=0x7fa814009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.444+0000 7fa81bfff700 1 -- 172.21.15.33:0/2991778744 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa80c097300 con 0x7fa824101920 2024-05-08T21:18:18.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:18 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1074093927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:18 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2396471201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:18 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1074093927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:18 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2396471201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.663+0000 7fa819ffb700 1 -- 172.21.15.33:0/2991778744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa808005b20 con 0x7fa824101920 2024-05-08T21:18:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.664+0000 7fa81bfff700 1 -- 172.21.15.33:0/2991778744 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa80c09a920 con 0x7fa824101920 2024-05-08T21:18:18.666 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T21:18:18.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.666+0000 7fa819ffb700 1 -- 172.21.15.33:0/2991778744 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa8100a11b0 msgr2=0x7fa8100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.666+0000 7fa819ffb700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa8100a11b0 0x7fa8100a3670 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa814009c60 tx=0x7fa814009260 comp rx=0 tx=0).stop 2024-05-08T21:18:18.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.666+0000 7fa819ffb700 1 -- 172.21.15.33:0/2991778744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101920 msgr2=0x7fa824196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.666+0000 7fa819ffb700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101920 0x7fa824196d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa80c006010 tx=0x7fa80c004730 comp rx=0 tx=0).stop 2024-05-08T21:18:18.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.666+0000 7fa819ffb700 1 -- 172.21.15.33:0/2991778744 shutdown_connections 2024-05-08T21:18:18.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.666+0000 7fa819ffb700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa824106b80 0x7fa82419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.666+0000 7fa819ffb700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa824101920 0x7fa824196d40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.666+0000 7fa819ffb700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa8100a11b0 0x7fa8100a3670 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.666+0000 7fa819ffb700 1 --2- 172.21.15.33:0/2991778744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa824104250 0x7fa824197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.666+0000 7fa819ffb700 1 -- 172.21.15.33:0/2991778744 >> 172.21.15.33:0/2991778744 conn(0x7fa8240fb330 msgr2=0x7fa824103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:18.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.667+0000 7fa819ffb700 1 -- 172.21.15.33:0/2991778744 shutdown_connections 2024-05-08T21:18:18.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.667+0000 7fa819ffb700 1 -- 172.21.15.33:0/2991778744 wait complete. 2024-05-08T21:18:18.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:18 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1074093927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:18.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:18 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2396471201' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:18.684 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T21:18:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.823+0000 7f01d5c73700 1 -- 172.21.15.33:0/933847889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01d0106b40 msgr2=0x7f01d0108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.823+0000 7f01d5c73700 1 --2- 172.21.15.33:0/933847889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01d0106b40 0x7f01d0108fb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f01c800b0a0 tx=0x7f01c800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.823+0000 7f01d5c73700 1 -- 172.21.15.33:0/933847889 shutdown_connections 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.823+0000 7f01d5c73700 1 --2- 172.21.15.33:0/933847889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01d0106b40 0x7f01d0108fb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.823+0000 7f01d5c73700 1 --2- 172.21.15.33:0/933847889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d01018e0 0x7f01d0103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.823+0000 7f01d5c73700 1 --2- 172.21.15.33:0/933847889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d0104210 0x7f01d0106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.823+0000 7f01d5c73700 1 -- 172.21.15.33:0/933847889 >> 172.21.15.33:0/933847889 conn(0x7f01d00fb290 msgr2=0x7f01d00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.823+0000 7f01d5c73700 1 -- 172.21.15.33:0/933847889 shutdown_connections 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.823+0000 7f01d5c73700 1 -- 172.21.15.33:0/933847889 wait complete. 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.824+0000 7f01d5c73700 1 Processor -- start 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.824+0000 7f01d5c73700 1 -- start start 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.824+0000 7f01d5c73700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01d01018e0 0x7f01d0198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01d5c73700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d0104210 0x7f01d0199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01d5c73700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d0106b40 0x7f01d01a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01d5c73700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d01a0b10 con 0x7f01d01018e0 2024-05-08T21:18:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01d5c73700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d01a0c50 con 0x7f01d0104210 2024-05-08T21:18:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01d5c73700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d01a0de0 con 0x7f01d0106b40 2024-05-08T21:18:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01d01018e0 0x7f01d0198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01cf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01d01018e0 0x7f01d0198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48830/0 (socket says 172.21.15.33:48830) 2024-05-08T21:18:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01cf7fe700 1 -- 172.21.15.33:0/1058587908 learned_addr learned my addr 172.21.15.33:0/1058587908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01ceffd700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d0104210 0x7f01d0199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01cffff700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d0106b40 0x7f01d01a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01cf7fe700 1 -- 172.21.15.33:0/1058587908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d0106b40 msgr2=0x7f01d01a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01cf7fe700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d0106b40 0x7f01d01a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01cf7fe700 1 -- 172.21.15.33:0/1058587908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d0104210 msgr2=0x7f01d0199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01cf7fe700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d0104210 0x7f01d0199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01cf7fe700 1 -- 172.21.15.33:0/1058587908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c8009d00 con 0x7f01d01018e0 2024-05-08T21:18:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01cf7fe700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01d01018e0 0x7f01d0198fe0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f01c000e9a0 tx=0x7f01c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:18.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01ccff9700 1 -- 172.21.15.33:0/1058587908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0009790 con 0x7f01d01018e0 2024-05-08T21:18:18.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01ccff9700 1 -- 172.21.15.33:0/1058587908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01c000cbd0 con 0x7f01d01018e0 2024-05-08T21:18:18.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01ccff9700 1 -- 172.21.15.33:0/1058587908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0018680 con 0x7f01d01018e0 2024-05-08T21:18:18.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.825+0000 7f01d5c73700 1 -- 172.21.15.33:0/1058587908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d01a10c0 con 0x7f01d01018e0 2024-05-08T21:18:18.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.826+0000 7f01d5c73700 1 -- 172.21.15.33:0/1058587908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d01a15d0 con 0x7f01d01018e0 2024-05-08T21:18:18.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.826+0000 7f01d5c73700 1 -- 172.21.15.33:0/1058587908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d01086c0 con 0x7f01d01018e0 2024-05-08T21:18:18.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.828+0000 7f01ccff9700 1 -- 172.21.15.33:0/1058587908 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f01c000cd40 con 0x7f01d01018e0 2024-05-08T21:18:18.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.828+0000 7f01ccff9700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f01b80a1230 0x7f01b80a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:18.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.829+0000 7f01ccff9700 1 -- 172.21.15.33:0/1058587908 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f01c0014070 con 0x7f01d01018e0 2024-05-08T21:18:18.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.829+0000 7f01ceffd700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f01b80a1230 0x7f01b80a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:18.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.829+0000 7f01ceffd700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f01b80a1230 0x7f01b80a36f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009c60 tx=0x7f01c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:18.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:18.832+0000 7f01ccff9700 1 -- 172.21.15.33:0/1058587908 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f01c0096670 con 0x7f01d01018e0 2024-05-08T21:18:19.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.050+0000 7f01d5c73700 1 -- 172.21.15.33:0/1058587908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f01d0067d10 con 0x7f01d01018e0 2024-05-08T21:18:19.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.050+0000 7f01ccff9700 1 -- 172.21.15.33:0/1058587908 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f01c0099c90 con 0x7f01d01018e0 2024-05-08T21:18:19.053 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:19.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.053+0000 7f01d5c73700 1 -- 172.21.15.33:0/1058587908 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f01b80a1230 msgr2=0x7f01b80a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.053+0000 7f01d5c73700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f01b80a1230 0x7f01b80a36f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009c60 tx=0x7f01c4009260 comp rx=0 tx=0).stop 2024-05-08T21:18:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.053+0000 7f01d5c73700 1 -- 172.21.15.33:0/1058587908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01d01018e0 msgr2=0x7f01d0198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.053+0000 7f01d5c73700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01d01018e0 0x7f01d0198fe0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f01c000e9a0 tx=0x7f01c000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.053+0000 7f01d5c73700 1 -- 172.21.15.33:0/1058587908 shutdown_connections 2024-05-08T21:18:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.053+0000 7f01d5c73700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d0106b40 0x7f01d01a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.053+0000 7f01d5c73700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01d01018e0 0x7f01d0198fe0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.053+0000 7f01d5c73700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f01b80a1230 0x7f01b80a36f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.053+0000 7f01d5c73700 1 --2- 172.21.15.33:0/1058587908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01d0104210 0x7f01d0199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.053+0000 7f01d5c73700 1 -- 172.21.15.33:0/1058587908 >> 172.21.15.33:0/1058587908 conn(0x7f01d00fb290 msgr2=0x7f01d01010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.053+0000 7f01d5c73700 1 -- 172.21.15.33:0/1058587908 shutdown_connections 2024-05-08T21:18:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.053+0000 7f01d5c73700 1 -- 172.21.15.33:0/1058587908 wait complete. 2024-05-08T21:18:19.069 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:18:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.205+0000 7f3bbdd56700 1 -- 172.21.15.33:0/2575534895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb8101fd0 msgr2=0x7f3bb8102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.205+0000 7f3bbdd56700 1 --2- 172.21.15.33:0/2575534895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb8101fd0 0x7f3bb8102450 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009990 tx=0x7f3ba8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.205+0000 7f3bbdd56700 1 -- 172.21.15.33:0/2575534895 shutdown_connections 2024-05-08T21:18:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.205+0000 7f3bbdd56700 1 --2- 172.21.15.33:0/2575534895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb8102990 0x7f3bb81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.205+0000 7f3bbdd56700 1 --2- 172.21.15.33:0/2575534895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb8100e80 0x7f3bb81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.205+0000 7f3bbdd56700 1 --2- 172.21.15.33:0/2575534895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb8101fd0 0x7f3bb8102450 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.205+0000 7f3bbdd56700 1 -- 172.21.15.33:0/2575534895 >> 172.21.15.33:0/2575534895 conn(0x7f3bb80fc410 msgr2=0x7f3bb80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.205+0000 7f3bbdd56700 1 -- 172.21.15.33:0/2575534895 shutdown_connections 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.205+0000 7f3bbdd56700 1 -- 172.21.15.33:0/2575534895 wait complete. 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bbdd56700 1 Processor -- start 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bbdd56700 1 -- start start 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bbdd56700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb8100e80 0x7f3bb8196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bbdd56700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb8101fd0 0x7f3bb8197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bbdd56700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb8102990 0x7f3bb819e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bbdd56700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb819e820 con 0x7f3bb8101fd0 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bbdd56700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb819e990 con 0x7f3bb8100e80 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bbdd56700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb819eb50 con 0x7f3bb8102990 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bb6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb8101fd0 0x7f3bb8197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bb6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb8101fd0 0x7f3bb8197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48850/0 (socket says 172.21.15.33:48850) 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bb6ffd700 1 -- 172.21.15.33:0/297256117 learned_addr learned my addr 172.21.15.33:0/297256117 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bb77fe700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb8100e80 0x7f3bb8196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.206+0000 7f3bb7fff700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb8102990 0x7f3bb819e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.207+0000 7f3bb6ffd700 1 -- 172.21.15.33:0/297256117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb8102990 msgr2=0x7f3bb819e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.207+0000 7f3bb6ffd700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb8102990 0x7f3bb819e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.207+0000 7f3bb6ffd700 1 -- 172.21.15.33:0/297256117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb8100e80 msgr2=0x7f3bb8196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.207+0000 7f3bb6ffd700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb8100e80 0x7f3bb8196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.207+0000 7f3bb6ffd700 1 -- 172.21.15.33:0/297256117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80095f0 con 0x7f3bb8101fd0 2024-05-08T21:18:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.207+0000 7f3bb6ffd700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb8101fd0 0x7f3bb8197240 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800b5c0 tx=0x7f3ba8003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.207+0000 7f3bb4ff9700 1 -- 172.21.15.33:0/297256117 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba801d070 con 0x7f3bb8101fd0 2024-05-08T21:18:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.207+0000 7f3bb4ff9700 1 -- 172.21.15.33:0/297256117 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ba800bd10 con 0x7f3bb8101fd0 2024-05-08T21:18:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.207+0000 7f3bbdd56700 1 -- 172.21.15.33:0/297256117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb819edd0 con 0x7f3bb8101fd0 2024-05-08T21:18:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.207+0000 7f3bb4ff9700 1 -- 172.21.15.33:0/297256117 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8022c50 con 0x7f3bb8101fd0 2024-05-08T21:18:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.207+0000 7f3bbdd56700 1 -- 172.21.15.33:0/297256117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb819f1a0 con 0x7f3bb8101fd0 2024-05-08T21:18:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.208+0000 7f3bbdd56700 1 -- 172.21.15.33:0/297256117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb8108360 con 0x7f3bb8101fd0 2024-05-08T21:18:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.209+0000 7f3bb4ff9700 1 -- 172.21.15.33:0/297256117 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3ba8022470 con 0x7f3bb8101fd0 2024-05-08T21:18:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.210+0000 7f3bb4ff9700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ba40a11b0 0x7f3ba40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.210+0000 7f3bb77fe700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ba40a11b0 0x7f3ba40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.210+0000 7f3bb4ff9700 1 -- 172.21.15.33:0/297256117 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3ba80cde30 con 0x7f3bb8101fd0 2024-05-08T21:18:19.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.211+0000 7f3bb77fe700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ba40a11b0 0x7f3ba40a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80fe880 tx=0x7f3ba000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.215+0000 7f3bb4ff9700 1 -- 172.21.15.33:0/297256117 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3ba8097180 con 0x7f3bb8101fd0 2024-05-08T21:18:19.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:19 smithi033 ceph-mon[42528]: pgmap v268: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:18:19.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:19 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2656476848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:19.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:19 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2991778744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:19.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:19 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1058587908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:19.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.490+0000 7f3bbdd56700 1 -- 172.21.15.33:0/297256117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bb810c6c0 con 0x7f3bb8101fd0 2024-05-08T21:18:19.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.491+0000 7f3bb4ff9700 1 -- 172.21.15.33:0/297256117 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10899 (secure 0 0 0) 0x7f3ba809a7a0 con 0x7f3bb8101fd0 2024-05-08T21:18:19.494 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:19.494 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:17:32.303648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:18:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.494+0000 7f3bbdd56700 1 -- 172.21.15.33:0/297256117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ba40a11b0 msgr2=0x7f3ba40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.494+0000 7f3bbdd56700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ba40a11b0 0x7f3ba40a3670 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80fe880 tx=0x7f3ba000b410 comp rx=0 tx=0).stop 2024-05-08T21:18:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.494+0000 7f3bbdd56700 1 -- 172.21.15.33:0/297256117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb8101fd0 msgr2=0x7f3bb8197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.494+0000 7f3bbdd56700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb8101fd0 0x7f3bb8197240 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800b5c0 tx=0x7f3ba8003a50 comp rx=0 tx=0).stop 2024-05-08T21:18:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.494+0000 7f3bbdd56700 1 -- 172.21.15.33:0/297256117 shutdown_connections 2024-05-08T21:18:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.494+0000 7f3bbdd56700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bb8102990 0x7f3bb819e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.494+0000 7f3bbdd56700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ba40a11b0 0x7f3ba40a3670 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.494+0000 7f3bbdd56700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb8100e80 0x7f3bb8196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.494+0000 7f3bbdd56700 1 --2- 172.21.15.33:0/297256117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bb8101fd0 0x7f3bb8197240 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.494+0000 7f3bbdd56700 1 -- 172.21.15.33:0/297256117 >> 172.21.15.33:0/297256117 conn(0x7f3bb80fc410 msgr2=0x7f3bb81054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.494+0000 7f3bbdd56700 1 -- 172.21.15.33:0/297256117 shutdown_connections 2024-05-08T21:18:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.494+0000 7f3bbdd56700 1 -- 172.21.15.33:0/297256117 wait complete. 2024-05-08T21:18:19.498 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-05-08T21:18:19.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:18:19.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:19 smithi071 ceph-mon[47074]: pgmap v268: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:18:19.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:19 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2656476848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:19.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:19 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2991778744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:19.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:19 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1058587908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:19.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:19 smithi082 ceph-mon[47512]: pgmap v268: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:18:19.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:19 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2656476848' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:19.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:19 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2991778744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:19.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:19 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1058587908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.647+0000 7fdf2f75a700 1 -- 172.21.15.33:0/957409978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf28100e80 msgr2=0x7fdf281012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.647+0000 7fdf2f75a700 1 --2- 172.21.15.33:0/957409978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf28100e80 0x7fdf281012a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009990 tx=0x7fdf24009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.647+0000 7fdf2f75a700 1 -- 172.21.15.33:0/957409978 shutdown_connections 2024-05-08T21:18:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.647+0000 7fdf2f75a700 1 --2- 172.21.15.33:0/957409978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf28102990 0x7fdf281092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.647+0000 7fdf2f75a700 1 --2- 172.21.15.33:0/957409978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf28100e80 0x7fdf281012a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.647+0000 7fdf2f75a700 1 --2- 172.21.15.33:0/957409978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf28101fd0 0x7fdf28102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.647+0000 7fdf2f75a700 1 -- 172.21.15.33:0/957409978 >> 172.21.15.33:0/957409978 conn(0x7fdf280fc410 msgr2=0x7fdf280fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.647+0000 7fdf2f75a700 1 -- 172.21.15.33:0/957409978 shutdown_connections 2024-05-08T21:18:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.647+0000 7fdf2f75a700 1 -- 172.21.15.33:0/957409978 wait complete. 2024-05-08T21:18:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.647+0000 7fdf2f75a700 1 Processor -- start 2024-05-08T21:18:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2f75a700 1 -- start start 2024-05-08T21:18:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2f75a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf28100e80 0x7fdf28198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2f75a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf28101fd0 0x7fdf28199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2f75a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf28102990 0x7fdf281a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2f75a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281a0a50 con 0x7fdf28100e80 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2f75a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281a0b90 con 0x7fdf28101fd0 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2f75a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281a0d20 con 0x7fdf28102990 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2d4f6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf28100e80 0x7fdf28198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2d4f6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf28100e80 0x7fdf28198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48864/0 (socket says 172.21.15.33:48864) 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2d4f6700 1 -- 172.21.15.33:0/3457861894 learned_addr learned my addr 172.21.15.33:0/3457861894 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2dcf7700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf28102990 0x7fdf281a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2ccf5700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf28101fd0 0x7fdf28199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2d4f6700 1 -- 172.21.15.33:0/3457861894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf28102990 msgr2=0x7fdf281a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2d4f6700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf28102990 0x7fdf281a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2d4f6700 1 -- 172.21.15.33:0/3457861894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf28101fd0 msgr2=0x7fdf28199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2d4f6700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf28101fd0 0x7fdf28199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2d4f6700 1 -- 172.21.15.33:0/3457861894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf240095f0 con 0x7fdf28100e80 2024-05-08T21:18:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.648+0000 7fdf2d4f6700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf28100e80 0x7fdf28198fc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdf240059d0 tx=0x7fdf24004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.649+0000 7fdf1e7fc700 1 -- 172.21.15.33:0/3457861894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2401d070 con 0x7fdf28100e80 2024-05-08T21:18:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.649+0000 7fdf1e7fc700 1 -- 172.21.15.33:0/3457861894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf2400bdd0 con 0x7fdf28100e80 2024-05-08T21:18:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.649+0000 7fdf2f75a700 1 -- 172.21.15.33:0/3457861894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf281a0fa0 con 0x7fdf28100e80 2024-05-08T21:18:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.649+0000 7fdf1e7fc700 1 -- 172.21.15.33:0/3457861894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24022d40 con 0x7fdf28100e80 2024-05-08T21:18:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.649+0000 7fdf2f75a700 1 -- 172.21.15.33:0/3457861894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf281a1310 con 0x7fdf28100e80 2024-05-08T21:18:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.650+0000 7fdf2f75a700 1 -- 172.21.15.33:0/3457861894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf28067d80 con 0x7fdf28100e80 2024-05-08T21:18:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.652+0000 7fdf1e7fc700 1 -- 172.21.15.33:0/3457861894 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdf24022500 con 0x7fdf28100e80 2024-05-08T21:18:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.652+0000 7fdf1e7fc700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdf140a1230 0x7fdf140a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.652+0000 7fdf2ccf5700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdf140a1230 0x7fdf140a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.652+0000 7fdf1e7fc700 1 -- 172.21.15.33:0/3457861894 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdf240ce2b0 con 0x7fdf28100e80 2024-05-08T21:18:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.653+0000 7fdf2ccf5700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdf140a1230 0x7fdf140a36f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf18005950 tx=0x7fdf1800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:19.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.656+0000 7fdf1e7fc700 1 -- 172.21.15.33:0/3457861894 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdf24097660 con 0x7fdf28100e80 2024-05-08T21:18:19.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.879+0000 7fdf2f75a700 1 -- 172.21.15.33:0/3457861894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf2810c530 con 0x7fdf28100e80 2024-05-08T21:18:19.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.881+0000 7fdf1e7fc700 1 -- 172.21.15.33:0/3457861894 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24374 (secure 0 0 0) 0x7fdf2409ac80 con 0x7fdf28100e80 2024-05-08T21:18:19.883 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:19.883 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:57.410336+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:18:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.883+0000 7fdf2f75a700 1 -- 172.21.15.33:0/3457861894 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdf140a1230 msgr2=0x7fdf140a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.883+0000 7fdf2f75a700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdf140a1230 0x7fdf140a36f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdf18005950 tx=0x7fdf1800b410 comp rx=0 tx=0).stop 2024-05-08T21:18:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.883+0000 7fdf2f75a700 1 -- 172.21.15.33:0/3457861894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf28100e80 msgr2=0x7fdf28198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.883+0000 7fdf2f75a700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf28100e80 0x7fdf28198fc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdf240059d0 tx=0x7fdf24004710 comp rx=0 tx=0).stop 2024-05-08T21:18:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.883+0000 7fdf2f75a700 1 -- 172.21.15.33:0/3457861894 shutdown_connections 2024-05-08T21:18:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.883+0000 7fdf2f75a700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf28102990 0x7fdf281a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.883+0000 7fdf2f75a700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf28100e80 0x7fdf28198fc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.883+0000 7fdf2f75a700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdf140a1230 0x7fdf140a36f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.883+0000 7fdf2f75a700 1 --2- 172.21.15.33:0/3457861894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf28101fd0 0x7fdf28199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:19.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.883+0000 7fdf2f75a700 1 -- 172.21.15.33:0/3457861894 >> 172.21.15.33:0/3457861894 conn(0x7fdf280fc410 msgr2=0x7fdf28107820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.883+0000 7fdf2f75a700 1 -- 172.21.15.33:0/3457861894 shutdown_connections 2024-05-08T21:18:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:19.883+0000 7fdf2f75a700 1 -- 172.21.15.33:0/3457861894 wait complete. 2024-05-08T21:18:19.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:17:01.282803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:17:02.555552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T21:17:05.686277+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T21:18:19.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:18:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.038+0000 7fc9de2fe700 1 -- 172.21.15.33:0/865504597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d8101920 msgr2=0x7fc9d81081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.038+0000 7fc9de2fe700 1 --2- 172.21.15.33:0/865504597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d8101920 0x7fc9d81081d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000b0d0 tx=0x7fc9d000b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.038+0000 7fc9de2fe700 1 -- 172.21.15.33:0/865504597 shutdown_connections 2024-05-08T21:18:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.038+0000 7fc9de2fe700 1 --2- 172.21.15.33:0/865504597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d8101920 0x7fc9d81081d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.038+0000 7fc9de2fe700 1 --2- 172.21.15.33:0/865504597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9d80ffd90 0x7fc9d81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.038+0000 7fc9de2fe700 1 --2- 172.21.15.33:0/865504597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d8100f80 0x7fc9d81013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.038+0000 7fc9de2fe700 1 -- 172.21.15.33:0/865504597 >> 172.21.15.33:0/865504597 conn(0x7fc9d80fb300 msgr2=0x7fc9d80fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:20.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.038+0000 7fc9de2fe700 1 -- 172.21.15.33:0/865504597 shutdown_connections 2024-05-08T21:18:20.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.038+0000 7fc9de2fe700 1 -- 172.21.15.33:0/865504597 wait complete. 2024-05-08T21:18:20.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.039+0000 7fc9de2fe700 1 Processor -- start 2024-05-08T21:18:20.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.039+0000 7fc9de2fe700 1 -- start start 2024-05-08T21:18:20.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.039+0000 7fc9de2fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d80ffd90 0x7fc9d8196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:20.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.039+0000 7fc9de2fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9d8100f80 0x7fc9d8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:20.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.039+0000 7fc9de2fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d8101920 0x7fc9d819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:20.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.039+0000 7fc9de2fe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d819e8f0 con 0x7fc9d8101920 2024-05-08T21:18:20.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.039+0000 7fc9de2fe700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d819ea60 con 0x7fc9d8100f80 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.039+0000 7fc9de2fe700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d819ec20 con 0x7fc9d80ffd90 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.039+0000 7fc9dc89b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d8101920 0x7fc9d819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.039+0000 7fc9dc89b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d8101920 0x7fc9d819e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48878/0 (socket says 172.21.15.33:48878) 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.039+0000 7fc9dc89b700 1 -- 172.21.15.33:0/4111613901 learned_addr learned my addr 172.21.15.33:0/4111613901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9d7fff700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d80ffd90 0x7fc9d8196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9d77fe700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9d8100f80 0x7fc9d8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9dc89b700 1 -- 172.21.15.33:0/4111613901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d80ffd90 msgr2=0x7fc9d8196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9dc89b700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d80ffd90 0x7fc9d8196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9dc89b700 1 -- 172.21.15.33:0/4111613901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9d8100f80 msgr2=0x7fc9d8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9dc89b700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9d8100f80 0x7fc9d8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9dc89b700 1 -- 172.21.15.33:0/4111613901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d0009d00 con 0x7fc9d8101920 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9d77fe700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9d8100f80 0x7fc9d8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9dc89b700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d8101920 0x7fc9d819e320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0007af0 tx=0x7fc9d0007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9d57fa700 1 -- 172.21.15.33:0/4111613901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d000e040 con 0x7fc9d8101920 2024-05-08T21:18:20.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9d57fa700 1 -- 172.21.15.33:0/4111613901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9d001db00 con 0x7fc9d8101920 2024-05-08T21:18:20.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9de2fe700 1 -- 172.21.15.33:0/4111613901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d819eea0 con 0x7fc9d8101920 2024-05-08T21:18:20.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9d57fa700 1 -- 172.21.15.33:0/4111613901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d001b990 con 0x7fc9d8101920 2024-05-08T21:18:20.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.040+0000 7fc9de2fe700 1 -- 172.21.15.33:0/4111613901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d819f210 con 0x7fc9d8101920 2024-05-08T21:18:20.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.041+0000 7fc9de2fe700 1 -- 172.21.15.33:0/4111613901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d8067480 con 0x7fc9d8101920 2024-05-08T21:18:20.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.042+0000 7fc9d57fa700 1 -- 172.21.15.33:0/4111613901 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc9d001dc70 con 0x7fc9d8101920 2024-05-08T21:18:20.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.043+0000 7fc9d57fa700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9c00a11b0 0x7fc9c00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:20.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.043+0000 7fc9d57fa700 1 -- 172.21.15.33:0/4111613901 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc9d00ce810 con 0x7fc9d8101920 2024-05-08T21:18:20.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.043+0000 7fc9d7fff700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9c00a11b0 0x7fc9c00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:20.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.044+0000 7fc9d7fff700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9c00a11b0 0x7fc9c00a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8198280 tx=0x7fc9c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:20.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.047+0000 7fc9d57fa700 1 -- 172.21.15.33:0/4111613901 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc9d0097bc0 con 0x7fc9d8101920 2024-05-08T21:18:20.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.270+0000 7fc9de2fe700 1 -- 172.21.15.33:0/4111613901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9d810b630 con 0x7fc9d8101920 2024-05-08T21:18:20.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.271+0000 7fc9d57fa700 1 -- 172.21.15.33:0/4111613901 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9d009b1e0 con 0x7fc9d8101920 2024-05-08T21:18:20.273 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:18:20.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.273+0000 7fc9de2fe700 1 -- 172.21.15.33:0/4111613901 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9c00a11b0 msgr2=0x7fc9c00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.273+0000 7fc9de2fe700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9c00a11b0 0x7fc9c00a3670 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8198280 tx=0x7fc9c8009260 comp rx=0 tx=0).stop 2024-05-08T21:18:20.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.273+0000 7fc9de2fe700 1 -- 172.21.15.33:0/4111613901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d8101920 msgr2=0x7fc9d819e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.273+0000 7fc9de2fe700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d8101920 0x7fc9d819e320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0007af0 tx=0x7fc9d0007850 comp rx=0 tx=0).stop 2024-05-08T21:18:20.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.273+0000 7fc9de2fe700 1 -- 172.21.15.33:0/4111613901 shutdown_connections 2024-05-08T21:18:20.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.273+0000 7fc9de2fe700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9d8101920 0x7fc9d819e320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.273+0000 7fc9de2fe700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9d80ffd90 0x7fc9d8196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.273+0000 7fc9de2fe700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9c00a11b0 0x7fc9c00a3670 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.273+0000 7fc9de2fe700 1 --2- 172.21.15.33:0/4111613901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9d8100f80 0x7fc9d8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.273+0000 7fc9de2fe700 1 -- 172.21.15.33:0/4111613901 >> 172.21.15.33:0/4111613901 conn(0x7fc9d80fb300 msgr2=0x7fc9d8069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:20.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.274+0000 7fc9de2fe700 1 -- 172.21.15.33:0/4111613901 shutdown_connections 2024-05-08T21:18:20.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.274+0000 7fc9de2fe700 1 -- 172.21.15.33:0/4111613901 wait complete. 2024-05-08T21:18:20.291 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:18:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.429+0000 7f46d7b53700 1 -- 172.21.15.33:0/1021341878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d0102990 msgr2=0x7f46d01092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.429+0000 7f46d7b53700 1 --2- 172.21.15.33:0/1021341878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d0102990 0x7f46d01092b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f46c800b0a0 tx=0x7f46c800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.429+0000 7f46d7b53700 1 -- 172.21.15.33:0/1021341878 shutdown_connections 2024-05-08T21:18:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.429+0000 7f46d7b53700 1 --2- 172.21.15.33:0/1021341878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d0102990 0x7f46d01092b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.429+0000 7f46d7b53700 1 --2- 172.21.15.33:0/1021341878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46d0100e80 0x7f46d01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.429+0000 7f46d7b53700 1 --2- 172.21.15.33:0/1021341878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d0101fd0 0x7f46d0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.429+0000 7f46d7b53700 1 -- 172.21.15.33:0/1021341878 >> 172.21.15.33:0/1021341878 conn(0x7f46d00fc3d0 msgr2=0x7f46d00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.429+0000 7f46d7b53700 1 -- 172.21.15.33:0/1021341878 shutdown_connections 2024-05-08T21:18:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.429+0000 7f46d7b53700 1 -- 172.21.15.33:0/1021341878 wait complete. 2024-05-08T21:18:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.430+0000 7f46d7b53700 1 Processor -- start 2024-05-08T21:18:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.430+0000 7f46d7b53700 1 -- start start 2024-05-08T21:18:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d7b53700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d0100e80 0x7f46d0196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d7b53700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46d0101fd0 0x7f46d01973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d7b53700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d0102990 0x7f46d019e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d7b53700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d019e950 con 0x7f46d0102990 2024-05-08T21:18:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d7b53700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d019ea90 con 0x7f46d0101fd0 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d7b53700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d019ec20 con 0x7f46d0100e80 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d60f0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d0102990 0x7f46d019e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d60f0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d0102990 0x7f46d019e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50530/0 (socket says 172.21.15.33:50530) 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d60f0700 1 -- 172.21.15.33:0/2665922916 learned_addr learned my addr 172.21.15.33:0/2665922916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d50ee700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46d0101fd0 0x7f46d01973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d58ef700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d0100e80 0x7f46d0196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d60f0700 1 -- 172.21.15.33:0/2665922916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d0100e80 msgr2=0x7f46d0196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d60f0700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d0100e80 0x7f46d0196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d60f0700 1 -- 172.21.15.33:0/2665922916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46d0101fd0 msgr2=0x7f46d01973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d60f0700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46d0101fd0 0x7f46d01973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d60f0700 1 -- 172.21.15.33:0/2665922916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c8009d00 con 0x7f46d0102990 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46d60f0700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d0102990 0x7f46d019e380 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f46c8007950 tx=0x7f46c8007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.431+0000 7f46c2ffd700 1 -- 172.21.15.33:0/2665922916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c800e040 con 0x7f46d0102990 2024-05-08T21:18:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.432+0000 7f46c2ffd700 1 -- 172.21.15.33:0/2665922916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46c8004c50 con 0x7f46d0102990 2024-05-08T21:18:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.432+0000 7f46d7b53700 1 -- 172.21.15.33:0/2665922916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d019eea0 con 0x7f46d0102990 2024-05-08T21:18:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.432+0000 7f46c2ffd700 1 -- 172.21.15.33:0/2665922916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c8005040 con 0x7f46d0102990 2024-05-08T21:18:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.432+0000 7f46d7b53700 1 -- 172.21.15.33:0/2665922916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d019f2c0 con 0x7f46d0102990 2024-05-08T21:18:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.433+0000 7f46d7b53700 1 -- 172.21.15.33:0/2665922916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d0108340 con 0x7f46d0102990 2024-05-08T21:18:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.434+0000 7f46c2ffd700 1 -- 172.21.15.33:0/2665922916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f46c801d470 con 0x7f46d0102990 2024-05-08T21:18:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.434+0000 7f46c2ffd700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f46bc0a11b0 0x7f46bc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.435+0000 7f46d58ef700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f46bc0a11b0 0x7f46bc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.435+0000 7f46c2ffd700 1 -- 172.21.15.33:0/2665922916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f46c80ce970 con 0x7f46d0102990 2024-05-08T21:18:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.435+0000 7f46d58ef700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f46bc0a11b0 0x7f46bc0a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009c60 tx=0x7f46c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.439+0000 7f46c2ffd700 1 -- 172.21.15.33:0/2665922916 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f46c8097ed0 con 0x7f46d0102990 2024-05-08T21:18:20.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:20 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/297256117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:20.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:20 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3457861894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:20.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:20 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/4111613901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:20.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:20 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/297256117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:20.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:20 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3457861894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:20.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:20 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/4111613901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:20.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.655+0000 7f46d7b53700 1 -- 172.21.15.33:0/2665922916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f46d010c580 con 0x7f46d0102990 2024-05-08T21:18:20.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.656+0000 7f46c2ffd700 1 -- 172.21.15.33:0/2665922916 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f46c809b4f0 con 0x7f46d0102990 2024-05-08T21:18:20.658 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.658+0000 7f46d7b53700 1 -- 172.21.15.33:0/2665922916 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f46bc0a11b0 msgr2=0x7f46bc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.658+0000 7f46d7b53700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f46bc0a11b0 0x7f46bc0a3670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009c60 tx=0x7f46c4009260 comp rx=0 tx=0).stop 2024-05-08T21:18:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.658+0000 7f46d7b53700 1 -- 172.21.15.33:0/2665922916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d0102990 msgr2=0x7f46d019e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.658+0000 7f46d7b53700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d0102990 0x7f46d019e380 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f46c8007950 tx=0x7f46c8007870 comp rx=0 tx=0).stop 2024-05-08T21:18:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.658+0000 7f46d7b53700 1 -- 172.21.15.33:0/2665922916 shutdown_connections 2024-05-08T21:18:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.658+0000 7f46d7b53700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46d0102990 0x7f46d019e380 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.658+0000 7f46d7b53700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46d0100e80 0x7f46d0196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.658+0000 7f46d7b53700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f46bc0a11b0 0x7f46bc0a3670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.658+0000 7f46d7b53700 1 --2- 172.21.15.33:0/2665922916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46d0101fd0 0x7f46d01973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.658+0000 7f46d7b53700 1 -- 172.21.15.33:0/2665922916 >> 172.21.15.33:0/2665922916 conn(0x7f46d00fc3d0 msgr2=0x7f46d0105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.658+0000 7f46d7b53700 1 -- 172.21.15.33:0/2665922916 shutdown_connections 2024-05-08T21:18:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.658+0000 7f46d7b53700 1 -- 172.21.15.33:0/2665922916 wait complete. 2024-05-08T21:18:20.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:20 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/297256117' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:20.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:20 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3457861894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:20.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:20 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4111613901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:20.674 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:18:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.812+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/1595496556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e0100fb0 msgr2=0x7ff2e0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.812+0000 7ff2e4b2b700 1 --2- 172.21.15.33:0/1595496556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e0100fb0 0x7ff2e0101410 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00099c0 tx=0x7ff2d0009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.812+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/1595496556 shutdown_connections 2024-05-08T21:18:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.812+0000 7ff2e4b2b700 1 --2- 172.21.15.33:0/1595496556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e0101950 0x7ff2e0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.812+0000 7ff2e4b2b700 1 --2- 172.21.15.33:0/1595496556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2e00ffdc0 0x7ff2e01001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.812+0000 7ff2e4b2b700 1 --2- 172.21.15.33:0/1595496556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e0100fb0 0x7ff2e0101410 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.812+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/1595496556 >> 172.21.15.33:0/1595496556 conn(0x7ff2e00fb330 msgr2=0x7ff2e00fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.812+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/1595496556 shutdown_connections 2024-05-08T21:18:20.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.812+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/1595496556 wait complete. 2024-05-08T21:18:20.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.813+0000 7ff2e4b2b700 1 Processor -- start 2024-05-08T21:18:20.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.813+0000 7ff2e4b2b700 1 -- start start 2024-05-08T21:18:20.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.813+0000 7ff2e4b2b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e00ffdc0 0x7ff2e0103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:20.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.813+0000 7ff2e4b2b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e0100fb0 0x7ff2e0102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:20.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.813+0000 7ff2e4b2b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2e0101950 0x7ff2e01028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:20.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.813+0000 7ff2e4b2b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e0102fd0 con 0x7ff2e0100fb0 2024-05-08T21:18:20.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.813+0000 7ff2e4b2b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e0103140 con 0x7ff2e00ffdc0 2024-05-08T21:18:20.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.813+0000 7ff2e4b2b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e01a0e10 con 0x7ff2e0101950 2024-05-08T21:18:20.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.813+0000 7ff2ddd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e0100fb0 0x7ff2e0102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:20.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.813+0000 7ff2ddd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e0100fb0 0x7ff2e0102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50542/0 (socket says 172.21.15.33:50542) 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2ddd9b700 1 -- 172.21.15.33:0/2031062831 learned_addr learned my addr 172.21.15.33:0/2031062831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2ded9d700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2e0101950 0x7ff2e01028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2de59c700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e00ffdc0 0x7ff2e0103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2ddd9b700 1 -- 172.21.15.33:0/2031062831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2e0101950 msgr2=0x7ff2e01028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2ddd9b700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2e0101950 0x7ff2e01028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2ddd9b700 1 -- 172.21.15.33:0/2031062831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e00ffdc0 msgr2=0x7ff2e0103cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2ddd9b700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e00ffdc0 0x7ff2e0103cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2ddd9b700 1 -- 172.21.15.33:0/2031062831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d00095f0 con 0x7ff2e0100fb0 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2ded9d700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2e0101950 0x7ff2e01028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2ddd9b700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e0100fb0 0x7ff2e0102390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0000c00 tx=0x7ff2d000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2d77fe700 1 -- 172.21.15.33:0/2031062831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d001d070 con 0x7ff2e0100fb0 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2d77fe700 1 -- 172.21.15.33:0/2031062831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2d0005030 con 0x7ff2e0100fb0 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/2031062831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e01a1010 con 0x7ff2e0100fb0 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2d77fe700 1 -- 172.21.15.33:0/2031062831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0004b10 con 0x7ff2e0100fb0 2024-05-08T21:18:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.814+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/2031062831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e01a1490 con 0x7ff2e0100fb0 2024-05-08T21:18:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.815+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/2031062831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e0067480 con 0x7ff2e0100fb0 2024-05-08T21:18:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.816+0000 7ff2d77fe700 1 -- 172.21.15.33:0/2031062831 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff2d00051a0 con 0x7ff2e0100fb0 2024-05-08T21:18:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.817+0000 7ff2d77fe700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff2cc0a1230 0x7ff2cc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:20.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.817+0000 7ff2de59c700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff2cc0a1230 0x7ff2cc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:20.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.817+0000 7ff2d77fe700 1 -- 172.21.15.33:0/2031062831 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff2d00cd9c0 con 0x7ff2e0100fb0 2024-05-08T21:18:20.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.818+0000 7ff2de59c700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff2cc0a1230 0x7ff2cc0a36f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0100e10 tx=0x7ff2c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:20.822+0000 7ff2d77fe700 1 -- 172.21.15.33:0/2031062831 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff2d0096d70 con 0x7ff2e0100fb0 2024-05-08T21:18:21.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.040+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/2031062831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2e01a1750 con 0x7ff2e0100fb0 2024-05-08T21:18:21.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.041+0000 7ff2d77fe700 1 -- 172.21.15.33:0/2031062831 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2d009a390 con 0x7ff2e0100fb0 2024-05-08T21:18:21.043 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T21:18:21.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.043+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/2031062831 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff2cc0a1230 msgr2=0x7ff2cc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:21.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.043+0000 7ff2e4b2b700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff2cc0a1230 0x7ff2cc0a36f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0100e10 tx=0x7ff2c8009260 comp rx=0 tx=0).stop 2024-05-08T21:18:21.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.043+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/2031062831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e0100fb0 msgr2=0x7ff2e0102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:21.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.043+0000 7ff2e4b2b700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e0100fb0 0x7ff2e0102390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0000c00 tx=0x7ff2d000f690 comp rx=0 tx=0).stop 2024-05-08T21:18:21.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.043+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/2031062831 shutdown_connections 2024-05-08T21:18:21.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.043+0000 7ff2e4b2b700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2e0101950 0x7ff2e01028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.043+0000 7ff2e4b2b700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff2cc0a1230 0x7ff2cc0a36f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.043+0000 7ff2e4b2b700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2e00ffdc0 0x7ff2e0103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.043+0000 7ff2e4b2b700 1 --2- 172.21.15.33:0/2031062831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2e0100fb0 0x7ff2e0102390 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.043+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/2031062831 >> 172.21.15.33:0/2031062831 conn(0x7ff2e00fb330 msgr2=0x7ff2e0069020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:21.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.043+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/2031062831 shutdown_connections 2024-05-08T21:18:21.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.043+0000 7ff2e4b2b700 1 -- 172.21.15.33:0/2031062831 wait complete. 2024-05-08T21:18:21.059 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T21:18:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.198+0000 7ff47c7eb700 1 -- 172.21.15.33:0/3232137965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4740ffd60 msgr2=0x7ff474100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.198+0000 7ff47c7eb700 1 --2- 172.21.15.33:0/3232137965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4740ffd60 0x7ff474100180 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff4700099c0 tx=0x7ff470009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.198+0000 7ff47c7eb700 1 -- 172.21.15.33:0/3232137965 shutdown_connections 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.198+0000 7ff47c7eb700 1 --2- 172.21.15.33:0/3232137965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4741018f0 0x7ff4741081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.198+0000 7ff47c7eb700 1 --2- 172.21.15.33:0/3232137965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4740ffd60 0x7ff474100180 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.198+0000 7ff47c7eb700 1 --2- 172.21.15.33:0/3232137965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff474100f50 0x7ff4741013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.198+0000 7ff47c7eb700 1 -- 172.21.15.33:0/3232137965 >> 172.21.15.33:0/3232137965 conn(0x7ff4740fb290 msgr2=0x7ff4740fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.198+0000 7ff47c7eb700 1 -- 172.21.15.33:0/3232137965 shutdown_connections 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.198+0000 7ff47c7eb700 1 -- 172.21.15.33:0/3232137965 wait complete. 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.198+0000 7ff47c7eb700 1 Processor -- start 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff47c7eb700 1 -- start start 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff47c7eb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4740ffd60 0x7ff474198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff47c7eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff474100f50 0x7ff474199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff47c7eb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4741018f0 0x7ff4741a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff47c7eb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4741a0a80 con 0x7ff474100f50 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff47c7eb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4741a0bc0 con 0x7ff4740ffd60 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff47c7eb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4741a0d50 con 0x7ff4741018f0 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff479d86700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff474100f50 0x7ff474199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff479d86700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff474100f50 0x7ff474199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50564/0 (socket says 172.21.15.33:50564) 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff479d86700 1 -- 172.21.15.33:0/2385780219 learned_addr learned my addr 172.21.15.33:0/2385780219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff47a587700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4740ffd60 0x7ff474198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff47ad88700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4741018f0 0x7ff4741a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff479d86700 1 -- 172.21.15.33:0/2385780219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4741018f0 msgr2=0x7ff4741a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff479d86700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4741018f0 0x7ff4741a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff479d86700 1 -- 172.21.15.33:0/2385780219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4740ffd60 msgr2=0x7ff474198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff479d86700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4740ffd60 0x7ff474198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff479d86700 1 -- 172.21.15.33:0/2385780219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4700095f0 con 0x7ff474100f50 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.199+0000 7ff479d86700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff474100f50 0x7ff474199490 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff46400d950 tx=0x7ff46400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.200+0000 7ff46b7fe700 1 -- 172.21.15.33:0/2385780219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464009940 con 0x7ff474100f50 2024-05-08T21:18:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.200+0000 7ff46b7fe700 1 -- 172.21.15.33:0/2385780219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff464010460 con 0x7ff474100f50 2024-05-08T21:18:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.200+0000 7ff47c7eb700 1 -- 172.21.15.33:0/2385780219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4741a0fd0 con 0x7ff474100f50 2024-05-08T21:18:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.200+0000 7ff46b7fe700 1 -- 172.21.15.33:0/2385780219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46400f620 con 0x7ff474100f50 2024-05-08T21:18:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.200+0000 7ff47c7eb700 1 -- 172.21.15.33:0/2385780219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4741a14b0 con 0x7ff474100f50 2024-05-08T21:18:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.201+0000 7ff47c7eb700 1 -- 172.21.15.33:0/2385780219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff474067d10 con 0x7ff474100f50 2024-05-08T21:18:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.202+0000 7ff46b7fe700 1 -- 172.21.15.33:0/2385780219 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff4640105d0 con 0x7ff474100f50 2024-05-08T21:18:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.203+0000 7ff46b7fe700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4600a11b0 0x7ff4600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.203+0000 7ff47a587700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4600a11b0 0x7ff4600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.203+0000 7ff46b7fe700 1 -- 172.21.15.33:0/2385780219 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff4640ccbe0 con 0x7ff474100f50 2024-05-08T21:18:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.203+0000 7ff47a587700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4600a11b0 0x7ff4600a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff470000c00 tx=0x7ff470003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.207+0000 7ff46b7fe700 1 -- 172.21.15.33:0/2385780219 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff4640cce60 con 0x7ff474100f50 2024-05-08T21:18:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.423+0000 7ff47c7eb700 1 -- 172.21.15.33:0/2385780219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff47410b670 con 0x7ff474100f50 2024-05-08T21:18:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.423+0000 7ff46b7fe700 1 -- 172.21.15.33:0/2385780219 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff4640160a0 con 0x7ff474100f50 2024-05-08T21:18:21.426 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.426+0000 7ff47c7eb700 1 -- 172.21.15.33:0/2385780219 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4600a11b0 msgr2=0x7ff4600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.426+0000 7ff47c7eb700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4600a11b0 0x7ff4600a3670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff470000c00 tx=0x7ff470003680 comp rx=0 tx=0).stop 2024-05-08T21:18:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.426+0000 7ff47c7eb700 1 -- 172.21.15.33:0/2385780219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff474100f50 msgr2=0x7ff474199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.426+0000 7ff47c7eb700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff474100f50 0x7ff474199490 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff46400d950 tx=0x7ff46400dc60 comp rx=0 tx=0).stop 2024-05-08T21:18:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.426+0000 7ff47c7eb700 1 -- 172.21.15.33:0/2385780219 shutdown_connections 2024-05-08T21:18:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.426+0000 7ff47c7eb700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4741018f0 0x7ff4741a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.426+0000 7ff47c7eb700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4600a11b0 0x7ff4600a3670 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.426+0000 7ff47c7eb700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4740ffd60 0x7ff474198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.426+0000 7ff47c7eb700 1 --2- 172.21.15.33:0/2385780219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff474100f50 0x7ff474199490 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.426+0000 7ff47c7eb700 1 -- 172.21.15.33:0/2385780219 >> 172.21.15.33:0/2385780219 conn(0x7ff4740fb290 msgr2=0x7ff474068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.426+0000 7ff47c7eb700 1 -- 172.21.15.33:0/2385780219 shutdown_connections 2024-05-08T21:18:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.426+0000 7ff47c7eb700 1 -- 172.21.15.33:0/2385780219 wait complete. 2024-05-08T21:18:21.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:21 smithi033 ceph-mon[42528]: pgmap v269: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:18:21.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:21 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2665922916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:21.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:21 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2031062831' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:21.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:18:21.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:21 smithi071 ceph-mon[47074]: pgmap v269: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:18:21.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:21 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2665922916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:21.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:21 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2031062831' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.578+0000 7f87c0963700 1 -- 172.21.15.33:0/846070556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87bc101910 msgr2=0x7f87bc1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.578+0000 7f87c0963700 1 --2- 172.21.15.33:0/846070556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87bc101910 0x7f87bc1081c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f87b400b0d0 tx=0x7f87b400b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.578+0000 7f87c0963700 1 -- 172.21.15.33:0/846070556 shutdown_connections 2024-05-08T21:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.578+0000 7f87c0963700 1 --2- 172.21.15.33:0/846070556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87bc101910 0x7f87bc1081c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.578+0000 7f87c0963700 1 --2- 172.21.15.33:0/846070556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87bc0ffd80 0x7f87bc1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.578+0000 7f87c0963700 1 --2- 172.21.15.33:0/846070556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc100f70 0x7f87bc1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.578+0000 7f87c0963700 1 -- 172.21.15.33:0/846070556 >> 172.21.15.33:0/846070556 conn(0x7f87bc0fb310 msgr2=0x7f87bc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.578+0000 7f87c0963700 1 -- 172.21.15.33:0/846070556 shutdown_connections 2024-05-08T21:18:21.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.578+0000 7f87c0963700 1 -- 172.21.15.33:0/846070556 wait complete. 2024-05-08T21:18:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87c0963700 1 Processor -- start 2024-05-08T21:18:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87c0963700 1 -- start start 2024-05-08T21:18:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87c0963700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc0ffd80 0x7f87bc195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87c0963700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87bc100f70 0x7f87bc196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87c0963700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87bc101910 0x7f87bc19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87c0963700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc19d830 con 0x7f87bc101910 2024-05-08T21:18:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87c0963700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc19d9a0 con 0x7f87bc100f70 2024-05-08T21:18:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87c0963700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc19db60 con 0x7f87bc0ffd80 2024-05-08T21:18:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87bad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87bc101910 0x7f87bc19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87bad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87bc101910 0x7f87bc19d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50586/0 (socket says 172.21.15.33:50586) 2024-05-08T21:18:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87bad9d700 1 -- 172.21.15.33:0/1572024757 learned_addr learned my addr 172.21.15.33:0/1572024757 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87b9d9b700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87bc100f70 0x7f87bc196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.579+0000 7f87ba59c700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc0ffd80 0x7f87bc195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.580+0000 7f87bad9d700 1 -- 172.21.15.33:0/1572024757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc0ffd80 msgr2=0x7f87bc195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.580+0000 7f87bad9d700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc0ffd80 0x7f87bc195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.580+0000 7f87bad9d700 1 -- 172.21.15.33:0/1572024757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87bc100f70 msgr2=0x7f87bc196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.580+0000 7f87bad9d700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87bc100f70 0x7f87bc196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.580+0000 7f87bad9d700 1 -- 172.21.15.33:0/1572024757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b4009d00 con 0x7f87bc101910 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.580+0000 7f87b9d9b700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87bc100f70 0x7f87bc196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.580+0000 7f87bad9d700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87bc101910 0x7f87bc19d260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f87b4007af0 tx=0x7f87b400baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.580+0000 7f87ab7fe700 1 -- 172.21.15.33:0/1572024757 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400e040 con 0x7f87bc101910 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.580+0000 7f87c0963700 1 -- 172.21.15.33:0/1572024757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc19dde0 con 0x7f87bc101910 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.580+0000 7f87ab7fe700 1 -- 172.21.15.33:0/1572024757 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f87b401db00 con 0x7f87bc101910 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.580+0000 7f87ab7fe700 1 -- 172.21.15.33:0/1572024757 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b401baf0 con 0x7f87bc101910 2024-05-08T21:18:21.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.580+0000 7f87c0963700 1 -- 172.21.15.33:0/1572024757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc19e200 con 0x7f87bc101910 2024-05-08T21:18:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.581+0000 7f87c0963700 1 -- 172.21.15.33:0/1572024757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87bc107250 con 0x7f87bc101910 2024-05-08T21:18:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.582+0000 7f87ab7fe700 1 -- 172.21.15.33:0/1572024757 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f87b401dc70 con 0x7f87bc101910 2024-05-08T21:18:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.583+0000 7f87ab7fe700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f87a40a11b0 0x7f87a40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:21.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.583+0000 7f87ab7fe700 1 -- 172.21.15.33:0/1572024757 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f87b40ceb30 con 0x7f87bc101910 2024-05-08T21:18:21.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.583+0000 7f87ba59c700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f87a40a11b0 0x7f87a40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:21.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.584+0000 7f87ba59c700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f87a40a11b0 0x7f87a40a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0fd720 tx=0x7f87ac00b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:21.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.587+0000 7f87ab7fe700 1 -- 172.21.15.33:0/1572024757 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f87b4097ee0 con 0x7f87bc101910 2024-05-08T21:18:21.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:21 smithi082 ceph-mon[47512]: pgmap v269: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:18:21.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:21 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2665922916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:21.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:21 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2031062831' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:21.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.855+0000 7f87c0963700 1 -- 172.21.15.33:0/1572024757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87bc067d10 con 0x7f87bc101910 2024-05-08T21:18:21.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.856+0000 7f87ab7fe700 1 -- 172.21.15.33:0/1572024757 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10899 (secure 0 0 0) 0x7f87b409b500 con 0x7f87bc101910 2024-05-08T21:18:21.859 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:21.859 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:17:32.303648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:18:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.859+0000 7f87c0963700 1 -- 172.21.15.33:0/1572024757 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f87a40a11b0 msgr2=0x7f87a40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.859+0000 7f87c0963700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f87a40a11b0 0x7f87a40a3670 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0fd720 tx=0x7f87ac00b5d0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.859+0000 7f87c0963700 1 -- 172.21.15.33:0/1572024757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87bc101910 msgr2=0x7f87bc19d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.859+0000 7f87c0963700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87bc101910 0x7f87bc19d260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f87b4007af0 tx=0x7f87b400baa0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.859+0000 7f87c0963700 1 -- 172.21.15.33:0/1572024757 shutdown_connections 2024-05-08T21:18:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.859+0000 7f87c0963700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87bc101910 0x7f87bc19d260 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.859+0000 7f87c0963700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87bc0ffd80 0x7f87bc195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.859+0000 7f87c0963700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f87a40a11b0 0x7f87a40a3670 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.859+0000 7f87c0963700 1 --2- 172.21.15.33:0/1572024757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87bc100f70 0x7f87bc196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.859+0000 7f87c0963700 1 -- 172.21.15.33:0/1572024757 >> 172.21.15.33:0/1572024757 conn(0x7f87bc0fb310 msgr2=0x7f87bc104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.859+0000 7f87c0963700 1 -- 172.21.15.33:0/1572024757 shutdown_connections 2024-05-08T21:18:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:21.859+0000 7f87c0963700 1 -- 172.21.15.33:0/1572024757 wait complete. 2024-05-08T21:18:21.863 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-05-08T21:18:21.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:18:22.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.011+0000 7fa73b69f700 1 -- 172.21.15.33:0/321413254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa734102990 msgr2=0x7fa7341092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.011+0000 7fa73b69f700 1 --2- 172.21.15.33:0/321413254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa734102990 0x7fa7341092b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00b0a0 tx=0x7fa72c00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.011+0000 7fa73b69f700 1 -- 172.21.15.33:0/321413254 shutdown_connections 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.011+0000 7fa73b69f700 1 --2- 172.21.15.33:0/321413254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa734102990 0x7fa7341092b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.011+0000 7fa73b69f700 1 --2- 172.21.15.33:0/321413254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa734100e80 0x7fa7341012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.011+0000 7fa73b69f700 1 --2- 172.21.15.33:0/321413254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa734101fd0 0x7fa734102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.011+0000 7fa73b69f700 1 -- 172.21.15.33:0/321413254 >> 172.21.15.33:0/321413254 conn(0x7fa7340fc410 msgr2=0x7fa7340fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.011+0000 7fa73b69f700 1 -- 172.21.15.33:0/321413254 shutdown_connections 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.011+0000 7fa73b69f700 1 -- 172.21.15.33:0/321413254 wait complete. 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.011+0000 7fa73b69f700 1 Processor -- start 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa73b69f700 1 -- start start 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa73b69f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa734100e80 0x7fa734199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa73b69f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa734101fd0 0x7fa734199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa73b69f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa734102990 0x7fa7341a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa73b69f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7341a0b50 con 0x7fa734102990 2024-05-08T21:18:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa73b69f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7341a0c90 con 0x7fa734101fd0 2024-05-08T21:18:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa73b69f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7341a0e20 con 0x7fa734100e80 2024-05-08T21:18:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa739c3c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa734102990 0x7fa7341a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa738c3a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa734101fd0 0x7fa734199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa738c3a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa734101fd0 0x7fa734199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:60116/0 (socket says 172.21.15.33:60116) 2024-05-08T21:18:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa738c3a700 1 -- 172.21.15.33:0/3319958517 learned_addr learned my addr 172.21.15.33:0/3319958517 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa73943b700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa734100e80 0x7fa734199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa738c3a700 1 -- 172.21.15.33:0/3319958517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa734100e80 msgr2=0x7fa734199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa738c3a700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa734100e80 0x7fa734199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa738c3a700 1 -- 172.21.15.33:0/3319958517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa734102990 msgr2=0x7fa7341a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa738c3a700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa734102990 0x7fa7341a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa738c3a700 1 -- 172.21.15.33:0/3319958517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa730009520 con 0x7fa734101fd0 2024-05-08T21:18:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa73943b700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa734100e80 0x7fa734199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.012+0000 7fa739c3c700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa734102990 0x7fa7341a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.013+0000 7fa738c3a700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa734101fd0 0x7fa734199560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa72400e9a0 tx=0x7fa72400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.013+0000 7fa72a7fc700 1 -- 172.21.15.33:0/3319958517 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724009790 con 0x7fa734101fd0 2024-05-08T21:18:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.013+0000 7fa73b69f700 1 -- 172.21.15.33:0/3319958517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c009d00 con 0x7fa734101fd0 2024-05-08T21:18:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.013+0000 7fa72a7fc700 1 -- 172.21.15.33:0/3319958517 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa724010430 con 0x7fa734101fd0 2024-05-08T21:18:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.013+0000 7fa73b69f700 1 -- 172.21.15.33:0/3319958517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7341a14d0 con 0x7fa734101fd0 2024-05-08T21:18:22.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.013+0000 7fa72a7fc700 1 -- 172.21.15.33:0/3319958517 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72400cea0 con 0x7fa734101fd0 2024-05-08T21:18:22.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.014+0000 7fa73b69f700 1 -- 172.21.15.33:0/3319958517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa734067480 con 0x7fa734101fd0 2024-05-08T21:18:22.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.015+0000 7fa72a7fc700 1 -- 172.21.15.33:0/3319958517 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fa7240105a0 con 0x7fa734101fd0 2024-05-08T21:18:22.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.016+0000 7fa72a7fc700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7200a1230 0x7fa7200a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.016+0000 7fa73943b700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7200a1230 0x7fa7200a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.016+0000 7fa72a7fc700 1 -- 172.21.15.33:0/3319958517 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa724014070 con 0x7fa734101fd0 2024-05-08T21:18:22.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.017+0000 7fa73943b700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7200a1230 0x7fa7200a36f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa730009fd0 tx=0x7fa730009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:22.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.020+0000 7fa72a7fc700 1 -- 172.21.15.33:0/3319958517 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa7240963f0 con 0x7fa734101fd0 2024-05-08T21:18:22.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.238+0000 7fa73b69f700 1 -- 172.21.15.33:0/3319958517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa73410c700 con 0x7fa734101fd0 2024-05-08T21:18:22.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.240+0000 7fa72a7fc700 1 -- 172.21.15.33:0/3319958517 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24374 (secure 0 0 0) 0x7fa724026020 con 0x7fa734101fd0 2024-05-08T21:18:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:22.243 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:57.410336+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:18:22.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.243+0000 7fa73b69f700 1 -- 172.21.15.33:0/3319958517 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7200a1230 msgr2=0x7fa7200a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.243+0000 7fa73b69f700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7200a1230 0x7fa7200a36f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa730009fd0 tx=0x7fa730009260 comp rx=0 tx=0).stop 2024-05-08T21:18:22.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.243+0000 7fa73b69f700 1 -- 172.21.15.33:0/3319958517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa734101fd0 msgr2=0x7fa734199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.243+0000 7fa73b69f700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa734101fd0 0x7fa734199560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa72400e9a0 tx=0x7fa72400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.243+0000 7fa73b69f700 1 -- 172.21.15.33:0/3319958517 shutdown_connections 2024-05-08T21:18:22.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.243+0000 7fa73b69f700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa734102990 0x7fa7341a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.243+0000 7fa73b69f700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa734100e80 0x7fa734199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.243+0000 7fa73b69f700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7200a1230 0x7fa7200a36f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.243+0000 7fa73b69f700 1 --2- 172.21.15.33:0/3319958517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa734101fd0 0x7fa734199560 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.243+0000 7fa73b69f700 1 -- 172.21.15.33:0/3319958517 >> 172.21.15.33:0/3319958517 conn(0x7fa7340fc410 msgr2=0x7fa7340731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:22.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.243+0000 7fa73b69f700 1 -- 172.21.15.33:0/3319958517 shutdown_connections 2024-05-08T21:18:22.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.243+0000 7fa73b69f700 1 -- 172.21.15.33:0/3319958517 wait complete. 2024-05-08T21:18:22.259 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:18:22.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.397+0000 7f95e5eed700 1 -- 172.21.15.33:0/161623603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e0100f70 msgr2=0x7f95e01013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.397+0000 7f95e5eed700 1 --2- 172.21.15.33:0/161623603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e0100f70 0x7f95e01013d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f95d40099c0 tx=0x7f95d4009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.397+0000 7f95e5eed700 1 -- 172.21.15.33:0/161623603 shutdown_connections 2024-05-08T21:18:22.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.397+0000 7f95e5eed700 1 --2- 172.21.15.33:0/161623603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e0101910 0x7f95e01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.397+0000 7f95e5eed700 1 --2- 172.21.15.33:0/161623603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95e00ffd80 0x7f95e01001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.397+0000 7f95e5eed700 1 --2- 172.21.15.33:0/161623603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e0100f70 0x7f95e01013d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.397+0000 7f95e5eed700 1 -- 172.21.15.33:0/161623603 >> 172.21.15.33:0/161623603 conn(0x7f95e00fb310 msgr2=0x7f95e00fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:22.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.397+0000 7f95e5eed700 1 -- 172.21.15.33:0/161623603 shutdown_connections 2024-05-08T21:18:22.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.397+0000 7f95e5eed700 1 -- 172.21.15.33:0/161623603 wait complete. 2024-05-08T21:18:22.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.397+0000 7f95e5eed700 1 Processor -- start 2024-05-08T21:18:22.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.398+0000 7f95e5eed700 1 -- start start 2024-05-08T21:18:22.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.398+0000 7f95e5eed700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e00ffd80 0x7f95e0195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.398+0000 7f95e5eed700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95e0100f70 0x7f95e01960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.398+0000 7f95e5eed700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e0101910 0x7f95e019d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.398+0000 7f95e5eed700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e019d690 con 0x7f95e00ffd80 2024-05-08T21:18:22.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.398+0000 7f95e5eed700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e019d7d0 con 0x7f95e0100f70 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.398+0000 7f95e5eed700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e019d960 con 0x7f95e0101910 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.398+0000 7f95df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e00ffd80 0x7f95e0195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.398+0000 7f95df7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e00ffd80 0x7f95e0195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50624/0 (socket says 172.21.15.33:50624) 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.398+0000 7f95df7fe700 1 -- 172.21.15.33:0/1709972412 learned_addr learned my addr 172.21.15.33:0/1709972412 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.398+0000 7f95deffd700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95e0100f70 0x7f95e01960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.398+0000 7f95dffff700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e0101910 0x7f95e019d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.399+0000 7f95df7fe700 1 -- 172.21.15.33:0/1709972412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e0101910 msgr2=0x7f95e019d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.399+0000 7f95df7fe700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e0101910 0x7f95e019d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.399+0000 7f95df7fe700 1 -- 172.21.15.33:0/1709972412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95e0100f70 msgr2=0x7f95e01960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.399+0000 7f95df7fe700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95e0100f70 0x7f95e01960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.399+0000 7f95df7fe700 1 -- 172.21.15.33:0/1709972412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40095f0 con 0x7f95e00ffd80 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.399+0000 7f95df7fe700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e00ffd80 0x7f95e0195b60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f95d000c990 tx=0x7f95d000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.399+0000 7f95dcff9700 1 -- 172.21.15.33:0/1709972412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0007940 con 0x7f95e00ffd80 2024-05-08T21:18:22.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.399+0000 7f95dcff9700 1 -- 172.21.15.33:0/1709972412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f95d000f450 con 0x7f95e00ffd80 2024-05-08T21:18:22.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.399+0000 7f95dcff9700 1 -- 172.21.15.33:0/1709972412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0018660 con 0x7f95e00ffd80 2024-05-08T21:18:22.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.399+0000 7f95e5eed700 1 -- 172.21.15.33:0/1709972412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e019dc40 con 0x7f95e00ffd80 2024-05-08T21:18:22.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.399+0000 7f95e5eed700 1 -- 172.21.15.33:0/1709972412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e019e100 con 0x7f95e00ffd80 2024-05-08T21:18:22.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.400+0000 7f95e5eed700 1 -- 172.21.15.33:0/1709972412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e0107200 con 0x7f95e00ffd80 2024-05-08T21:18:22.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.401+0000 7f95dcff9700 1 -- 172.21.15.33:0/1709972412 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f95d000f5c0 con 0x7f95e00ffd80 2024-05-08T21:18:22.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.402+0000 7f95dcff9700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95c80a11b0 0x7f95c80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.402+0000 7f95deffd700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95c80a11b0 0x7f95c80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.402+0000 7f95dcff9700 1 -- 172.21.15.33:0/1709972412 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f95d00ccc50 con 0x7f95e00ffd80 2024-05-08T21:18:22.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.403+0000 7f95deffd700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95c80a11b0 0x7f95c80a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009990 tx=0x7f95d4005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:22.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.408+0000 7f95dcff9700 1 -- 172.21.15.33:0/1709972412 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f95d0095f70 con 0x7f95e00ffd80 2024-05-08T21:18:22.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:22 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2385780219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:22.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:22 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1572024757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:22.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:22 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3319958517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:22.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:22 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2385780219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:22.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:22 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1572024757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:22.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:22 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3319958517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.680+0000 7f95e5eed700 1 -- 172.21.15.33:0/1709972412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95e0067d10 con 0x7f95e00ffd80 2024-05-08T21:18:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.681+0000 7f95dcff9700 1 -- 172.21.15.33:0/1709972412 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10899 (secure 0 0 0) 0x7f95d0099590 con 0x7f95e00ffd80 2024-05-08T21:18:22.683 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:22.684 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:17:32.303648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:18:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.684+0000 7f95e5eed700 1 -- 172.21.15.33:0/1709972412 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95c80a11b0 msgr2=0x7f95c80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.684+0000 7f95e5eed700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95c80a11b0 0x7f95c80a3670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009990 tx=0x7f95d4005e80 comp rx=0 tx=0).stop 2024-05-08T21:18:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.684+0000 7f95e5eed700 1 -- 172.21.15.33:0/1709972412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e00ffd80 msgr2=0x7f95e0195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.684+0000 7f95e5eed700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e00ffd80 0x7f95e0195b60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f95d000c990 tx=0x7f95d000cca0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.684+0000 7f95e5eed700 1 -- 172.21.15.33:0/1709972412 shutdown_connections 2024-05-08T21:18:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.684+0000 7f95e5eed700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e0101910 0x7f95e019d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.684+0000 7f95e5eed700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e00ffd80 0x7f95e0195b60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.684+0000 7f95e5eed700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95c80a11b0 0x7f95c80a3670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.684+0000 7f95e5eed700 1 --2- 172.21.15.33:0/1709972412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95e0100f70 0x7f95e01960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.684+0000 7f95e5eed700 1 -- 172.21.15.33:0/1709972412 >> 172.21.15.33:0/1709972412 conn(0x7f95e00fb310 msgr2=0x7f95e0104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.684+0000 7f95e5eed700 1 -- 172.21.15.33:0/1709972412 shutdown_connections 2024-05-08T21:18:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.684+0000 7f95e5eed700 1 -- 172.21.15.33:0/1709972412 wait complete. 2024-05-08T21:18:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-05-08T21:18:22.699 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:22 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2385780219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:22.699 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:22 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1572024757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:22.699 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:22 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3319958517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:22.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T21:17:03.219607+0000', 'modified': '2024-05-08T21:17:32.303648+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24563, 'mds_1': 24521, 'mds_2': 24499, 'mds_3': 24497, 'mds_4': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6841/2132539478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6840', 'nonce': 2132539478}, {'type': 'v1', 'addr': '172.21.15.33:6841', 'nonce': 2132539478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24451': {'gid': 24451, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.82:6833/3382287612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6832', 'nonce': 3382287612}, {'type': 'v1', 'addr': '172.21.15.82:6833', 'nonce': 3382287612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24460': {'gid': 24460, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6835/82548780', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6834', 'nonce': 82548780}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 82548780}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24461': {'gid': 24461, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6833/497942273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 497942273}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 497942273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24472': {'gid': 24472, 'name': 'd', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6837/362093971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6836', 'nonce': 362093971}, {'type': 'v1', 'addr': '172.21.15.33:6837', 'nonce': 362093971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6835/1877231014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6834', 'nonce': 1877231014}, {'type': 'v1', 'addr': '172.21.15.82:6835', 'nonce': 1877231014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24497': {'gid': 24497, 'name': 'e', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.71:6835/3686264359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 3686264359}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 3686264359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.82:6837/2409190198', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 2409190198}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 2409190198}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.71:6837/1668078897', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6836', 'nonce': 1668078897}, {'type': 'v1', 'addr': '172.21.15.71:6837', 'nonce': 1668078897}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6839/1515211151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1515211151}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1515211151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T21:18:22.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-08T21:18:22.701 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-05-08T21:18:22.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.839+0000 7fe4f839c700 1 -- 172.21.15.33:0/3281780662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0101920 msgr2=0x7fe4f01081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.839+0000 7fe4f839c700 1 --2- 172.21.15.33:0/3281780662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0101920 0x7fe4f01081d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800b0d0 tx=0x7fe4e800b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.839+0000 7fe4f839c700 1 -- 172.21.15.33:0/3281780662 shutdown_connections 2024-05-08T21:18:22.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.839+0000 7fe4f839c700 1 --2- 172.21.15.33:0/3281780662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f0101920 0x7fe4f01081d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.839+0000 7fe4f839c700 1 --2- 172.21.15.33:0/3281780662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f00ffd90 0x7fe4f01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.839+0000 7fe4f839c700 1 --2- 172.21.15.33:0/3281780662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4f0100f80 0x7fe4f01013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.839+0000 7fe4f839c700 1 -- 172.21.15.33:0/3281780662 >> 172.21.15.33:0/3281780662 conn(0x7fe4f00fb300 msgr2=0x7fe4f00fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:22.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.839+0000 7fe4f839c700 1 -- 172.21.15.33:0/3281780662 shutdown_connections 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.839+0000 7fe4f839c700 1 -- 172.21.15.33:0/3281780662 wait complete. 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f839c700 1 Processor -- start 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f839c700 1 -- start start 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f839c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f00ffd90 0x7fe4f0198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f839c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4f0100f80 0x7fe4f0199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f839c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0101920 0x7fe4f01a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f839c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01a0b20 con 0x7fe4f00ffd90 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f839c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01a0c60 con 0x7fe4f0100f80 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f839c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01a0df0 con 0x7fe4f0101920 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f6138700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f00ffd90 0x7fe4f0198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f6138700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f00ffd90 0x7fe4f0198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50636/0 (socket says 172.21.15.33:50636) 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f6138700 1 -- 172.21.15.33:0/2565603405 learned_addr learned my addr 172.21.15.33:0/2565603405 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f5937700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4f0100f80 0x7fe4f0199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f6939700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0101920 0x7fe4f01a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f6138700 1 -- 172.21.15.33:0/2565603405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0101920 msgr2=0x7fe4f01a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f6138700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0101920 0x7fe4f01a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f6138700 1 -- 172.21.15.33:0/2565603405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4f0100f80 msgr2=0x7fe4f0199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f6138700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4f0100f80 0x7fe4f0199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f6138700 1 -- 172.21.15.33:0/2565603405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e8009d00 con 0x7fe4f00ffd90 2024-05-08T21:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.840+0000 7fe4f5937700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4f0100f80 0x7fe4f0199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.841+0000 7fe4f6138700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f00ffd90 0x7fe4f0198ff0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00e970 tx=0x7fe4ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.841+0000 7fe4e77fe700 1 -- 172.21.15.33:0/2565603405 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec009790 con 0x7fe4f00ffd90 2024-05-08T21:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.841+0000 7fe4e77fe700 1 -- 172.21.15.33:0/2565603405 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4ec00eec0 con 0x7fe4f00ffd90 2024-05-08T21:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.841+0000 7fe4f839c700 1 -- 172.21.15.33:0/2565603405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f01a10d0 con 0x7fe4f00ffd90 2024-05-08T21:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.841+0000 7fe4e77fe700 1 -- 172.21.15.33:0/2565603405 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec003c10 con 0x7fe4f00ffd90 2024-05-08T21:18:22.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.841+0000 7fe4f839c700 1 -- 172.21.15.33:0/2565603405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01a15e0 con 0x7fe4f00ffd90 2024-05-08T21:18:22.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.843+0000 7fe4e77fe700 1 -- 172.21.15.33:0/2565603405 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe4ec003e50 con 0x7fe4f00ffd90 2024-05-08T21:18:22.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.844+0000 7fe4e77fe700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe4dc0a1300 0x7fe4dc0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.844+0000 7fe4e77fe700 1 -- 172.21.15.33:0/2565603405 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe4ec014070 con 0x7fe4f00ffd90 2024-05-08T21:18:22.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.844+0000 7fe4f5937700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe4dc0a1300 0x7fe4dc0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.844+0000 7fe4f5937700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe4dc0a1300 0x7fe4dc0a37c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009cb0 tx=0x7fe4e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:22.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.845+0000 7fe504556b80 1 Processor -- start 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.845+0000 7fe504556b80 1 -- start start 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.845+0000 7fe504556b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfc3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.845+0000 7fe504556b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfc42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.845+0000 7fe504556b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfc4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.845+0000 7fe504556b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cabcf1a100 con 0x55cabcfc34f0 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.845+0000 7fe504556b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cabcf17ac0 con 0x55cabcfc4810 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.845+0000 7fe504556b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cabcf133b0 con 0x55cabcfc3e50 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4f6138700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfc3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4f6138700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfc3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50640/0 (socket says 172.21.15.33:50640) 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4f6138700 1 -- 172.21.15.33:0/244111687 learned_addr learned my addr 172.21.15.33:0/244111687 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4f5937700 1 --2- 172.21.15.33:0/244111687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfc42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4f6939700 1 --2- 172.21.15.33:0/244111687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfc4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4f6138700 1 -- 172.21.15.33:0/244111687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 msgr2=0x55cabcfc42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4f6138700 1 --2- 172.21.15.33:0/244111687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfc42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4f6138700 1 -- 172.21.15.33:0/244111687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 msgr2=0x55cabcfc4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4f6138700 1 --2- 172.21.15.33:0/244111687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfc4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4f6138700 1 -- 172.21.15.33:0/244111687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cabcd94360 con 0x55cabcfc34f0 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4f6939700 1 --2- 172.21.15.33:0/244111687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfc4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4f6138700 1 --2- 172.21.15.33:0/244111687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfc3910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0a1e60 tx=0x7fe4ec099b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be47c54c3ab7fdfc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4e4da4700 1 -- 172.21.15.33:0/244111687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec09a610 con 0x55cabcfc34f0 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4e4da4700 1 -- 172.21.15.33:0/244111687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4ec099f10 con 0x55cabcfc34f0 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.846+0000 7fe4e4da4700 1 -- 172.21.15.33:0/244111687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0cae40 con 0x55cabcfc34f0 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.847+0000 7fe504556b80 1 -- 172.21.15.33:0/244111687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 msgr2=0x55cabcfc3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.847+0000 7fe504556b80 1 --2- 172.21.15.33:0/244111687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfc3910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0a1e60 tx=0x7fe4ec099b20 comp rx=0 tx=0).stop 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.847+0000 7fe504556b80 1 -- 172.21.15.33:0/244111687 shutdown_connections 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.847+0000 7fe504556b80 1 --2- 172.21.15.33:0/244111687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfc4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.847+0000 7fe504556b80 1 --2- 172.21.15.33:0/244111687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfc3910 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.847+0000 7fe504556b80 1 --2- 172.21.15.33:0/244111687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfc42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.847+0000 7fe504556b80 1 -- 172.21.15.33:0/244111687 >> 172.21.15.33:0/244111687 conn(0x55cabcf211c0 msgr2=0x55cabcfbdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.847+0000 7fe504556b80 1 -- 172.21.15.33:0/244111687 shutdown_connections 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.847+0000 7fe504556b80 1 -- 172.21.15.33:0/244111687 wait complete. 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe504556b80 1 -- start start 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe504556b80 1 Processor -- start 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe504556b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe504556b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfcd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe504556b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfd4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe504556b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cabceaf510 con 0x55cabcfc34f0 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe504556b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cabceaf1b0 con 0x55cabcfc4810 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe504556b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cabcd96650 con 0x55cabcfc3e50 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe504556b80 10 client.0 _ll_register_callbacks cb 0x55cabceb8ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:22.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe4f6138700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe4f6138700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50654/0 (socket says 172.21.15.33:50654) 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe4f6138700 1 -- 172.21.15.33:0/754099381 learned_addr learned my addr 172.21.15.33:0/754099381 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe4f6939700 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfd4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe4f5937700 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfcd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe4f6138700 1 -- 172.21.15.33:0/754099381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 msgr2=0x55cabcfcd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe4f6138700 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfcd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe4f6138700 1 -- 172.21.15.33:0/754099381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 msgr2=0x55cabcfd4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe4f6138700 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfd4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe4f6138700 1 -- 172.21.15.33:0/754099381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01a15e0 con 0x55cabcfc34f0 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.848+0000 7fe4f6939700 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfd4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe4f6138700 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfccdc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0ca650 tx=0x7fe4ec003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e54ce2eaa08148a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe4d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe4d9ffb700 1 -- 172.21.15.33:0/754099381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec09a610 con 0x55cabcfc34f0 2024-05-08T21:18:22.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe4d9ffb700 1 -- 172.21.15.33:0/754099381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe4ec09dd40 con 0x55cabcfc34f0 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe504556b80 1 -- 172.21.15.33:0/754099381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cabce6c220 con 0x55cabcfc34f0 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe4d9ffb700 1 -- 172.21.15.33:0/754099381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4ec0b82f0 con 0x55cabcfc34f0 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe504556b80 10 client.15210 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe504556b80 1 -- 172.21.15.33:0/754099381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cabcd40b10 con 0x55cabcfc34f0 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe4d9ffb700 1 -- 172.21.15.33:0/754099381 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cabce6c220 con 0x55cabcfc34f0 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe4d9ffb700 1 -- 172.21.15.33:0/754099381 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe4ec095d00 con 0x55cabcfc34f0 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe504556b80 10 client.15210 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe504556b80 10 client.15210 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe504556b80 20 client.15210 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe504556b80 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55cabcfd79e0 0x55cabcfdcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe504556b80 4 client.15210 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.849+0000 7fe504556b80 1 -- 172.21.15.33:0/754099381 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cabcf927c0 con 0x55cabcfd79e0 2024-05-08T21:18:22.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.850+0000 7fe4f5937700 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55cabcfd79e0 0x55cabcfdcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.850+0000 7fe4d9ffb700 10 client.15210 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.850+0000 7fe4f5937700 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55cabcfd79e0 0x55cabcfdcca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10550212b8a26340 server_cookie=af66f0e4610950a9 in_seq=0 out_seq=0 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe4d9ffb700 1 -- 172.21.15.33:0/754099381 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cabcf927c0 con 0x55cabcfd79e0 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe4d9ffb700 10 client.15210 handle_command_reply: tid=0 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe504556b80 1 client.15210 shutdown 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe504556b80 1 -- 172.21.15.33:0/754099381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 msgr2=0x55cabcfccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe504556b80 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfccdc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0ca650 tx=0x7fe4ec003d90 comp rx=0 tx=0).stop 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe504556b80 1 -- 172.21.15.33:0/754099381 shutdown_connections 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe504556b80 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfd4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe504556b80 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfccdc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe504556b80 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55cabcfd79e0 0x55cabcfdcca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe504556b80 1 --2- 172.21.15.33:0/754099381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfcd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe504556b80 1 -- 172.21.15.33:0/754099381 >> 172.21.15.33:0/754099381 conn(0x55cabcf211c0 msgr2=0x55cabcfbd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:22.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe4d9ffb700 0 client.15210 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:18:22.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe504556b80 1 -- 172.21.15.33:0/754099381 shutdown_connections 2024-05-08T21:18:22.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe504556b80 1 -- 172.21.15.33:0/754099381 wait complete. 2024-05-08T21:18:22.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.851+0000 7fe504556b80 20 client.15210 trim_cache size 0 max 16384 2024-05-08T21:18:22.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.870+0000 7fe504556b80 1 Processor -- start 2024-05-08T21:18:22.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.870+0000 7fe504556b80 1 -- start start 2024-05-08T21:18:22.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe504556b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc34f0 0x55cabcfd5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe504556b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfc38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe504556b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc4810 0x55cabcfd49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe504556b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cabce6c220 con 0x55cabcfc4810 2024-05-08T21:18:22.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe504556b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cabcfbeb80 con 0x55cabcfc34f0 2024-05-08T21:18:22.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe504556b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cabcf0c150 con 0x55cabcfc3e50 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe4f6939700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc4810 0x55cabcfd49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe4f5937700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfc38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe4f6939700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc4810 0x55cabcfd49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50664/0 (socket says 172.21.15.33:50664) 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe4f6939700 1 -- 172.21.15.33:0/2510635179 learned_addr learned my addr 172.21.15.33:0/2510635179 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe4f6138700 1 --2- 172.21.15.33:0/2510635179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc34f0 0x55cabcfd5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe4f6939700 1 -- 172.21.15.33:0/2510635179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 msgr2=0x55cabcfc38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe4f6939700 1 --2- 172.21.15.33:0/2510635179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfc38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe4f6939700 1 -- 172.21.15.33:0/2510635179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc34f0 msgr2=0x55cabcfd5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe4f6939700 1 --2- 172.21.15.33:0/2510635179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc34f0 0x55cabcfd5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe4f6939700 1 -- 172.21.15.33:0/2510635179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cabced9510 con 0x55cabcfc4810 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe4f6939700 1 --2- 172.21.15.33:0/2510635179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc4810 0x55cabcfd49f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800b0d0 tx=0x7fe4e800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131b46347ad107e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.871+0000 7fe4e55a5700 1 -- 172.21.15.33:0/2510635179 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e800e040 con 0x55cabcfc4810 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.872+0000 7fe4e55a5700 1 -- 172.21.15.33:0/2510635179 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4e8007b90 con 0x55cabcfc4810 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.872+0000 7fe4e55a5700 1 -- 172.21.15.33:0/2510635179 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8003d70 con 0x55cabcfc4810 2024-05-08T21:18:22.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.872+0000 7fe504556b80 1 -- 172.21.15.33:0/2510635179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc4810 msgr2=0x55cabcfd49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.872+0000 7fe504556b80 1 --2- 172.21.15.33:0/2510635179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc4810 0x55cabcfd49f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe4e800b0d0 tx=0x7fe4e800b040 comp rx=0 tx=0).stop 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.872+0000 7fe504556b80 1 -- 172.21.15.33:0/2510635179 shutdown_connections 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.872+0000 7fe504556b80 1 --2- 172.21.15.33:0/2510635179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc4810 0x55cabcfd49f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.872+0000 7fe504556b80 1 --2- 172.21.15.33:0/2510635179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc34f0 0x55cabcfd5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.872+0000 7fe504556b80 1 --2- 172.21.15.33:0/2510635179 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfc38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.872+0000 7fe504556b80 1 -- 172.21.15.33:0/2510635179 >> 172.21.15.33:0/2510635179 conn(0x55cabcf211c0 msgr2=0x55cabcfc4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.872+0000 7fe504556b80 1 -- 172.21.15.33:0/2510635179 shutdown_connections 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.872+0000 7fe504556b80 1 -- 172.21.15.33:0/2510635179 wait complete. 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.872+0000 7fe504556b80 1 -- start start 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe504556b80 1 Processor -- start 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe504556b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe504556b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfcea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe504556b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfd34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe504556b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cabcfd3a10 con 0x55cabcfc34f0 2024-05-08T21:18:22.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe504556b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cabcfd3b80 con 0x55cabcfc4810 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe504556b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cabcfd3d40 con 0x55cabcfc3e50 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe504556b80 10 client.0 _ll_register_callbacks cb 0x55cabceb8ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4f6138700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4f6138700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50670/0 (socket says 172.21.15.33:50670) 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4f6138700 1 -- 172.21.15.33:0/1736987828 learned_addr learned my addr 172.21.15.33:0/1736987828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4f5937700 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfcea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4f6939700 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfd34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4f6138700 1 -- 172.21.15.33:0/1736987828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 msgr2=0x55cabcfcea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4f6138700 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfcea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4f6138700 1 -- 172.21.15.33:0/1736987828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 msgr2=0x55cabcfd34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4f6138700 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfd34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4f6138700 1 -- 172.21.15.33:0/1736987828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f0108b20 con 0x55cabcfc34f0 2024-05-08T21:18:22.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4f6939700 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfd34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4f6138700 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfce4f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0960b0 tx=0x7fe4ec00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90d273598e0708f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.873+0000 7fe4d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe4d9ffb700 1 -- 172.21.15.33:0/1736987828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0b2ea0 con 0x55cabcfc34f0 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe4d9ffb700 1 -- 172.21.15.33:0/1736987828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe4ec0a2390 con 0x55cabcfc34f0 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe504556b80 1 -- 172.21.15.33:0/1736987828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4ec099f10 con 0x55cabcfc34f0 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe4d9ffb700 1 -- 172.21.15.33:0/1736987828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4ec09a610 con 0x55cabcfc34f0 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe504556b80 10 client.15216 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe504556b80 1 -- 172.21.15.33:0/1736987828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cabcd98c80 con 0x55cabcfc34f0 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe4d9ffb700 1 -- 172.21.15.33:0/1736987828 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe4ec099f10 con 0x55cabcfc34f0 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe4d9ffb700 1 -- 172.21.15.33:0/1736987828 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe4ec0a1f00 con 0x55cabcfc34f0 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe504556b80 10 client.15216 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe504556b80 10 client.15216 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe504556b80 20 client.15216 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe504556b80 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55cabd032500 0x55cabd02efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:22.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe504556b80 4 client.15216 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.874+0000 7fe504556b80 1 -- 172.21.15.33:0/1736987828 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cabcf0be70 con 0x55cabd032500 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe4f5937700 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55cabd032500 0x55cabd02efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe4d9ffb700 10 client.15216 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe4f5937700 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55cabd032500 0x55cabd02efe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=151bbe3c4c5e4a4b server_cookie=e46fda65c51ace4b in_seq=0 out_seq=0 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe4d9ffb700 1 -- 172.21.15.33:0/1736987828 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55cabcf0be70 con 0x55cabd032500 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe4d9ffb700 10 client.15216 handle_command_reply: tid=0 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe504556b80 1 client.15216 shutdown 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe504556b80 1 -- 172.21.15.33:0/1736987828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 msgr2=0x55cabcfce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe504556b80 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfce4f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0960b0 tx=0x7fe4ec00cbd0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe504556b80 1 -- 172.21.15.33:0/1736987828 shutdown_connections 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe504556b80 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cabcfc4810 0x55cabcfd34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe504556b80 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cabcfc34f0 0x55cabcfce4f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe504556b80 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55cabd032500 0x55cabd02efe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe504556b80 1 --2- 172.21.15.33:0/1736987828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cabcfc3e50 0x55cabcfcea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe4d9ffb700 0 client.15216 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:18:22.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe504556b80 1 -- 172.21.15.33:0/1736987828 >> 172.21.15.33:0/1736987828 conn(0x55cabcf211c0 msgr2=0x55cabcfd4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:22.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe504556b80 1 -- 172.21.15.33:0/1736987828 shutdown_connections 2024-05-08T21:18:22.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.875+0000 7fe504556b80 1 -- 172.21.15.33:0/1736987828 wait complete. 2024-05-08T21:18:22.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe504556b80 20 client.15216 trim_cache size 0 max 16384 2024-05-08T21:18:22.885 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:22.885 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:18:22.885 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 4, 2024-05-08T21:18:22.885 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24451, 2024-05-08T21:18:22.885 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-08T21:18:22.885 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-08T21:18:22.886 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-08T21:18:22.886 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 51.575345710000001, 2024-05-08T21:18:22.886 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-05-08T21:18:22.886 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-05-08T21:18:22.886 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-05-08T21:18:22.886 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 105.481903468 2024-05-08T21:18:22.886 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe4f839c700 1 -- 172.21.15.33:0/2565603405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe4dc0a1300 msgr2=0x7fe4dc0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe4f839c700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe4dc0a1300 0x7fe4dc0a37c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009cb0 tx=0x7fe4e0009260 comp rx=0 tx=0).stop 2024-05-08T21:18:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe4f839c700 1 -- 172.21.15.33:0/2565603405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f00ffd90 msgr2=0x7fe4f0198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe4f839c700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f00ffd90 0x7fe4f0198ff0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00e970 tx=0x7fe4ec00ec80 comp rx=0 tx=0).stop 2024-05-08T21:18:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe4f839c700 1 -- 172.21.15.33:0/2565603405 shutdown_connections 2024-05-08T21:18:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe4f839c700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0101920 0x7fe4f01a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe4f839c700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4f00ffd90 0x7fe4f0198ff0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe4f839c700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe4dc0a1300 0x7fe4dc0a37c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe4f839c700 1 --2- 172.21.15.33:0/2565603405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4f0100f80 0x7fe4f0199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe4f839c700 1 -- 172.21.15.33:0/2565603405 >> 172.21.15.33:0/2565603405 conn(0x7fe4f00fb300 msgr2=0x7fe4f0069060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe4f839c700 1 -- 172.21.15.33:0/2565603405 shutdown_connections 2024-05-08T21:18:22.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:22.876+0000 7fe4f839c700 1 -- 172.21.15.33:0/2565603405 wait complete. 2024-05-08T21:18:22.892 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-05-08T21:18:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.028+0000 7fc5f6450700 1 -- 172.21.15.33:0/79041504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0101960 msgr2=0x7fc5f0103d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.028+0000 7fc5f6450700 1 --2- 172.21.15.33:0/79041504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0101960 0x7fc5f0103d50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009990 tx=0x7fc5e0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.028+0000 7fc5f6450700 1 -- 172.21.15.33:0/79041504 shutdown_connections 2024-05-08T21:18:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.028+0000 7fc5f6450700 1 --2- 172.21.15.33:0/79041504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f0106bc0 0x7fc5f0108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.028+0000 7fc5f6450700 1 --2- 172.21.15.33:0/79041504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0101960 0x7fc5f0103d50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.028+0000 7fc5f6450700 1 --2- 172.21.15.33:0/79041504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f0104290 0x7fc5f0106680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.028+0000 7fc5f6450700 1 -- 172.21.15.33:0/79041504 >> 172.21.15.33:0/79041504 conn(0x7fc5f00fb310 msgr2=0x7fc5f00fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.028+0000 7fc5f6450700 1 -- 172.21.15.33:0/79041504 shutdown_connections 2024-05-08T21:18:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.028+0000 7fc5f6450700 1 -- 172.21.15.33:0/79041504 wait complete. 2024-05-08T21:18:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5f6450700 1 Processor -- start 2024-05-08T21:18:23.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5f6450700 1 -- start start 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5f6450700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f0101960 0x7fc5f0101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5f6450700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0104290 0x7fc5f00ff910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5f6450700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f0106bc0 0x7fc5f00ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5f6450700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01005f0 con 0x7fc5f0104290 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5f6450700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0100760 con 0x7fc5f0101960 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5f6450700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01a0d50 con 0x7fc5f0106bc0 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0104290 0x7fc5f00ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0104290 0x7fc5f00ff910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50698/0 (socket says 172.21.15.33:50698) 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5ef7fe700 1 -- 172.21.15.33:0/1138981310 learned_addr learned my addr 172.21.15.33:0/1138981310 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5f49ed700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f0106bc0 0x7fc5f00ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5effff700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f0101960 0x7fc5f0101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5ef7fe700 1 -- 172.21.15.33:0/1138981310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f0106bc0 msgr2=0x7fc5f00ffe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f0106bc0 0x7fc5f00ffe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5ef7fe700 1 -- 172.21.15.33:0/1138981310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f0101960 msgr2=0x7fc5f0101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f0101960 0x7fc5f0101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5ef7fe700 1 -- 172.21.15.33:0/1138981310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e00095f0 con 0x7fc5f0104290 2024-05-08T21:18:23.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.029+0000 7fc5effff700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f0101960 0x7fc5f0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:23.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.030+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0104290 0x7fc5f00ff910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400e9a0 tx=0x7fc5e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.030+0000 7fc5ed7fa700 1 -- 172.21.15.33:0/1138981310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4009790 con 0x7fc5f0104290 2024-05-08T21:18:23.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.030+0000 7fc5ed7fa700 1 -- 172.21.15.33:0/1138981310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5e4010430 con 0x7fc5f0104290 2024-05-08T21:18:23.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.030+0000 7fc5f6450700 1 -- 172.21.15.33:0/1138981310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f01a0fe0 con 0x7fc5f0104290 2024-05-08T21:18:23.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.030+0000 7fc5ed7fa700 1 -- 172.21.15.33:0/1138981310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4018680 con 0x7fc5f0104290 2024-05-08T21:18:23.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.030+0000 7fc5f6450700 1 -- 172.21.15.33:0/1138981310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f01a1460 con 0x7fc5f0104290 2024-05-08T21:18:23.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.032+0000 7fc5ed7fa700 1 -- 172.21.15.33:0/1138981310 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc5e40105a0 con 0x7fc5f0104290 2024-05-08T21:18:23.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.033+0000 7fc5ed7fa700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5d80a11b0 0x7fc5d80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.033+0000 7fc5effff700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5d80a11b0 0x7fc5d80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.033+0000 7fc5ed7fa700 1 -- 172.21.15.33:0/1138981310 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc5e4014070 con 0x7fc5f0104290 2024-05-08T21:18:23.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.034+0000 7fc5effff700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5d80a11b0 0x7fc5d80a3670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000b670 tx=0x7fc5e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.034+0000 7fc60260ab80 1 Processor -- start 2024-05-08T21:18:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.034+0000 7fc60260ab80 1 -- start start 2024-05-08T21:18:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.034+0000 7fc60260ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282b9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.034+0000 7fc60260ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 0x5602282ba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.034+0000 7fc60260ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282ba810 0x5602282bac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc60260ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560228210100 con 0x5602282b9e50 2024-05-08T21:18:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc60260ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022820dac0 con 0x5602282ba810 2024-05-08T21:18:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc60260ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602282093b0 con 0x5602282b94f0 2024-05-08T21:18:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 0x5602282ba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 0x5602282ba2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50700/0 (socket says 172.21.15.33:50700) 2024-05-08T21:18:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5ef7fe700 1 -- 172.21.15.33:0/3395718442 learned_addr learned my addr 172.21.15.33:0/3395718442 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5f49ed700 1 --2- 172.21.15.33:0/3395718442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282ba810 0x5602282bac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5effff700 1 --2- 172.21.15.33:0/3395718442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282b9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5ef7fe700 1 -- 172.21.15.33:0/3395718442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 msgr2=0x5602282b9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/3395718442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282b9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5ef7fe700 1 -- 172.21.15.33:0/3395718442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282ba810 msgr2=0x5602282bac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/3395718442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282ba810 0x5602282bac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5ef7fe700 1 -- 172.21.15.33:0/3395718442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56022808a360 con 0x5602282b9e50 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5f49ed700 1 --2- 172.21.15.33:0/3395718442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282ba810 0x5602282bac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/3395718442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 0x5602282ba2d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4099c90 tx=0x7fc5e40998f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778a590bd990a455 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5d67fc700 1 -- 172.21.15.33:0/3395718442 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4099f90 con 0x5602282b9e50 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5d67fc700 1 -- 172.21.15.33:0/3395718442 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5e409ddc0 con 0x5602282b9e50 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.035+0000 7fc5d67fc700 1 -- 172.21.15.33:0/3395718442 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40a1f80 con 0x5602282b9e50 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.036+0000 7fc60260ab80 1 -- 172.21.15.33:0/3395718442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 msgr2=0x5602282ba2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.036+0000 7fc60260ab80 1 --2- 172.21.15.33:0/3395718442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 0x5602282ba2d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4099c90 tx=0x7fc5e40998f0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.036+0000 7fc60260ab80 1 -- 172.21.15.33:0/3395718442 shutdown_connections 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.036+0000 7fc60260ab80 1 --2- 172.21.15.33:0/3395718442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282ba810 0x5602282bac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.036+0000 7fc60260ab80 1 --2- 172.21.15.33:0/3395718442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282b9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.036+0000 7fc60260ab80 1 --2- 172.21.15.33:0/3395718442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 0x5602282ba2d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.036+0000 7fc60260ab80 1 -- 172.21.15.33:0/3395718442 >> 172.21.15.33:0/3395718442 conn(0x5602282171c0 msgr2=0x5602282b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.036+0000 7fc60260ab80 1 -- 172.21.15.33:0/3395718442 shutdown_connections 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.036+0000 7fc60260ab80 1 -- 172.21.15.33:0/3395718442 wait complete. 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc60260ab80 1 -- start start 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc60260ab80 1 Processor -- start 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc60260ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b94f0 0x5602282c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc60260ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282b9e50 0x5602282c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc60260ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282ba810 0x5602282ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc60260ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602281a5510 con 0x5602282b94f0 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc60260ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602281a51b0 con 0x5602282b9e50 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc60260ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56022808c650 con 0x5602282ba810 2024-05-08T21:18:23.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc60260ab80 10 client.0 _ll_register_callbacks cb 0x5602281aeae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc5effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b94f0 0x5602282c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc5effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b94f0 0x5602282c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50712/0 (socket says 172.21.15.33:50712) 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc5effff700 1 -- 172.21.15.33:0/821169450 learned_addr learned my addr 172.21.15.33:0/821169450 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.037+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282b9e50 0x5602282c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5f49ed700 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282ba810 0x5602282ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5effff700 1 -- 172.21.15.33:0/821169450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282ba810 msgr2=0x5602282ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5effff700 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282ba810 0x5602282ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5effff700 1 -- 172.21.15.33:0/821169450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282b9e50 msgr2=0x5602282c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5effff700 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282b9e50 0x5602282c32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5effff700 1 -- 172.21.15.33:0/821169450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f01a1460 con 0x5602282b94f0 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5f49ed700 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282ba810 0x5602282ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5effff700 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b94f0 0x5602282c2da0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009990 tx=0x7fc5e0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209923124936fdf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:23.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5cf7fe700 1 -- 172.21.15.33:0/821169450 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0004830 con 0x5602282b94f0 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5cf7fe700 1 -- 172.21.15.33:0/821169450 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc5e0004990 con 0x5602282b94f0 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc60260ab80 1 -- 172.21.15.33:0/821169450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560228162220 con 0x5602282b94f0 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5cf7fe700 1 -- 172.21.15.33:0/821169450 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5e0004b90 con 0x5602282b94f0 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc60260ab80 10 client.15228 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc60260ab80 1 -- 172.21.15.33:0/821169450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x560228036b10 con 0x5602282b94f0 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.038+0000 7fc5cf7fe700 1 -- 172.21.15.33:0/821169450 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560228162220 con 0x5602282b94f0 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.039+0000 7fc5cf7fe700 1 -- 172.21.15.33:0/821169450 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc5e000f580 con 0x5602282b94f0 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.039+0000 7fc60260ab80 10 client.15228 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.039+0000 7fc60260ab80 10 client.15228 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.039+0000 7fc60260ab80 20 client.15228 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.039+0000 7fc60260ab80 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5602282cd4c0 0x5602282d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.039+0000 7fc60260ab80 4 client.15228 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.039+0000 7fc60260ab80 1 -- 172.21.15.33:0/821169450 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602282887c0 con 0x5602282cd4c0 2024-05-08T21:18:23.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.039+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5602282cd4c0 0x5602282d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.040+0000 7fc5cf7fe700 10 client.15228 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:18:23.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.040+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5602282cd4c0 0x5602282d32c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d886cf68931ddc4c server_cookie=77924af16ca50134 in_seq=0 out_seq=0 2024-05-08T21:18:23.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.040+0000 7fc5cf7fe700 1 -- 172.21.15.33:0/821169450 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fc5e405fe90 con 0x5602282cd4c0 2024-05-08T21:18:23.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.040+0000 7fc5cf7fe700 10 client.15228 handle_command_reply: tid=0 2024-05-08T21:18:23.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc60260ab80 1 client.15228 shutdown 2024-05-08T21:18:23.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc60260ab80 1 -- 172.21.15.33:0/821169450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b94f0 msgr2=0x5602282c2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc60260ab80 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b94f0 0x5602282c2da0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009990 tx=0x7fc5e0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc60260ab80 1 -- 172.21.15.33:0/821169450 shutdown_connections 2024-05-08T21:18:23.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc60260ab80 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5602282cd4c0 0x5602282d32c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc60260ab80 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282ba810 0x5602282ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc60260ab80 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b94f0 0x5602282c2da0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc5cf7fe700 0 client.15228 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:18:23.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc60260ab80 1 --2- 172.21.15.33:0/821169450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282b9e50 0x5602282c32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc60260ab80 1 -- 172.21.15.33:0/821169450 >> 172.21.15.33:0/821169450 conn(0x5602282171c0 msgr2=0x5602282b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc60260ab80 1 -- 172.21.15.33:0/821169450 shutdown_connections 2024-05-08T21:18:23.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc60260ab80 1 -- 172.21.15.33:0/821169450 wait complete. 2024-05-08T21:18:23.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.041+0000 7fc60260ab80 20 client.15228 trim_cache size 0 max 16384 2024-05-08T21:18:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.060+0000 7fc60260ab80 1 Processor -- start 2024-05-08T21:18:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.060+0000 7fc60260ab80 1 -- start start 2024-05-08T21:18:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.060+0000 7fc60260ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.060+0000 7fc60260ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 0x5602282b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.060+0000 7fc60260ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282ba810 0x5602282ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.060+0000 7fc60260ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560228202150 con 0x5602282b9e50 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.060+0000 7fc60260ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560228206010 con 0x5602282ba810 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.060+0000 7fc60260ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602281a85a0 con 0x5602282b94f0 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 0x5602282b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 0x5602282b98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50720/0 (socket says 172.21.15.33:50720) 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5ef7fe700 1 -- 172.21.15.33:0/3242460645 learned_addr learned my addr 172.21.15.33:0/3242460645 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5effff700 1 --2- 172.21.15.33:0/3242460645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5f49ed700 1 --2- 172.21.15.33:0/3242460645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282ba810 0x5602282ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5ef7fe700 1 -- 172.21.15.33:0/3242460645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 msgr2=0x5602282c1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/3242460645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282c1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5ef7fe700 1 -- 172.21.15.33:0/3242460645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282ba810 msgr2=0x5602282ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/3242460645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282ba810 0x5602282ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5ef7fe700 1 -- 172.21.15.33:0/3242460645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602281cf510 con 0x5602282b9e50 2024-05-08T21:18:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5f49ed700 1 --2- 172.21.15.33:0/3242460645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282ba810 0x5602282ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/3242460645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 0x5602282b98d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400e970 tx=0x7fc5e40af040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfe2d9468cef873 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5d6ffd700 1 -- 172.21.15.33:0/3242460645 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40b8050 con 0x5602282b9e50 2024-05-08T21:18:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5d6ffd700 1 -- 172.21.15.33:0/3242460645 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5e405fe90 con 0x5602282b9e50 2024-05-08T21:18:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.061+0000 7fc5d6ffd700 1 -- 172.21.15.33:0/3242460645 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4099f90 con 0x5602282b9e50 2024-05-08T21:18:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.062+0000 7fc60260ab80 1 -- 172.21.15.33:0/3242460645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 msgr2=0x5602282b98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.062+0000 7fc60260ab80 1 --2- 172.21.15.33:0/3242460645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 0x5602282b98d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400e970 tx=0x7fc5e40af040 comp rx=0 tx=0).stop 2024-05-08T21:18:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.062+0000 7fc60260ab80 1 -- 172.21.15.33:0/3242460645 shutdown_connections 2024-05-08T21:18:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.062+0000 7fc60260ab80 1 --2- 172.21.15.33:0/3242460645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282ba810 0x5602282ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.062+0000 7fc60260ab80 1 --2- 172.21.15.33:0/3242460645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282c1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.062+0000 7fc60260ab80 1 --2- 172.21.15.33:0/3242460645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282b9e50 0x5602282b98d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.062+0000 7fc60260ab80 1 -- 172.21.15.33:0/3242460645 >> 172.21.15.33:0/3242460645 conn(0x5602282171c0 msgr2=0x5602282b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.062+0000 7fc60260ab80 1 -- 172.21.15.33:0/3242460645 shutdown_connections 2024-05-08T21:18:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.062+0000 7fc60260ab80 1 -- 172.21.15.33:0/3242460645 wait complete. 2024-05-08T21:18:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.062+0000 7fc60260ab80 1 -- start start 2024-05-08T21:18:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc60260ab80 1 Processor -- start 2024-05-08T21:18:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc60260ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc60260ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282b9e50 0x5602282b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc60260ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282ba810 0x5602282c9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc60260ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602282c98e0 con 0x5602282ba810 2024-05-08T21:18:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc60260ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602282c9a50 con 0x5602282b9e50 2024-05-08T21:18:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc60260ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602282c9c10 con 0x5602282b94f0 2024-05-08T21:18:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc60260ab80 10 client.0 _ll_register_callbacks cb 0x5602281aeae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc5f49ed700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282ba810 0x5602282c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc5f49ed700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282ba810 0x5602282c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50722/0 (socket says 172.21.15.33:50722) 2024-05-08T21:18:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc5f49ed700 1 -- 172.21.15.33:0/2845625596 learned_addr learned my addr 172.21.15.33:0/2845625596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc5effff700 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.063+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282b9e50 0x5602282b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5f49ed700 1 -- 172.21.15.33:0/2845625596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 msgr2=0x5602282b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5f49ed700 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5f49ed700 1 -- 172.21.15.33:0/2845625596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282b9e50 msgr2=0x5602282b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5f49ed700 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282b9e50 0x5602282b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5f49ed700 1 -- 172.21.15.33:0/2845625596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56022808a360 con 0x5602282ba810 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5effff700 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5ef7fe700 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282b9e50 0x5602282b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5f49ed700 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282ba810 0x5602282c9370 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8003fd0 tx=0x7fc5e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782a0bcdc033970d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5cf7fe700 1 -- 172.21.15.33:0/2845625596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8003d30 con 0x5602282ba810 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5cf7fe700 1 -- 172.21.15.33:0/2845625596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc5e8004db0 con 0x5602282ba810 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc60260ab80 1 -- 172.21.15.33:0/2845625596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5e409ddc0 con 0x5602282ba810 2024-05-08T21:18:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5cf7fe700 1 -- 172.21.15.33:0/2845625596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5e8017620 con 0x5602282ba810 2024-05-08T21:18:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc60260ab80 10 client.15234 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc60260ab80 1 -- 172.21.15.33:0/2845625596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56022808ec80 con 0x5602282ba810 2024-05-08T21:18:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5cf7fe700 1 -- 172.21.15.33:0/2845625596 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5e409ddc0 con 0x5602282ba810 2024-05-08T21:18:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc5cf7fe700 1 -- 172.21.15.33:0/2845625596 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc5e801d030 con 0x5602282ba810 2024-05-08T21:18:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.064+0000 7fc60260ab80 10 client.15234 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.065+0000 7fc60260ab80 10 client.15234 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-08T21:18:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.065+0000 7fc60260ab80 20 client.15234 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.065+0000 7fc60260ab80 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5602283291b0 0x560228325b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.065+0000 7fc60260ab80 4 client.15234 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:18:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.065+0000 7fc60260ab80 1 -- 172.21.15.33:0/2845625596 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560228201e70 con 0x5602283291b0 2024-05-08T21:18:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.065+0000 7fc5effff700 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5602283291b0 0x560228325b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.065+0000 7fc5effff700 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5602283291b0 0x560228325b00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68b27d4899b60418 server_cookie=d422c5a39d3335d9 in_seq=0 out_seq=0 2024-05-08T21:18:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.065+0000 7fc5cf7fe700 10 client.15234 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:18:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.065+0000 7fc5cf7fe700 1 -- 172.21.15.33:0/2845625596 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560228201e70 con 0x5602283291b0 2024-05-08T21:18:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.065+0000 7fc5cf7fe700 10 client.15234 handle_command_reply: tid=0 2024-05-08T21:18:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.065+0000 7fc60260ab80 1 client.15234 shutdown 2024-05-08T21:18:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc60260ab80 1 -- 172.21.15.33:0/2845625596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282ba810 msgr2=0x5602282c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc60260ab80 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282ba810 0x5602282c9370 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8003fd0 tx=0x7fc5e80042f0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc60260ab80 1 -- 172.21.15.33:0/2845625596 shutdown_connections 2024-05-08T21:18:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc60260ab80 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602282ba810 0x5602282c9370 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc60260ab80 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5602283291b0 0x560228325b00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc60260ab80 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5602282b94f0 0x5602282b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc60260ab80 1 --2- 172.21.15.33:0/2845625596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602282b9e50 0x5602282b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc5cf7fe700 0 client.15234 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:18:23.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc60260ab80 1 -- 172.21.15.33:0/2845625596 >> 172.21.15.33:0/2845625596 conn(0x5602282171c0 msgr2=0x5602282c0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc60260ab80 1 -- 172.21.15.33:0/2845625596 shutdown_connections 2024-05-08T21:18:23.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc60260ab80 1 -- 172.21.15.33:0/2845625596 wait complete. 2024-05-08T21:18:23.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc60260ab80 20 client.15234 trim_cache size 0 max 16384 2024-05-08T21:18:23.075 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:18:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 3, 2024-05-08T21:18:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24497, 2024-05-08T21:18:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-08T21:18:23.075 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-08T21:18:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-08T21:18:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 52.720051482000002, 2024-05-08T21:18:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-05-08T21:18:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-05-08T21:18:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-05-08T21:18:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 98.464041425000005 2024-05-08T21:18:23.076 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc5f6450700 1 -- 172.21.15.33:0/1138981310 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5d80a11b0 msgr2=0x7fc5d80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc5f6450700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5d80a11b0 0x7fc5d80a3670 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000b670 tx=0x7fc5e0005fb0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc5f6450700 1 -- 172.21.15.33:0/1138981310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0104290 msgr2=0x7fc5f00ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc5f6450700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0104290 0x7fc5f00ff910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400e9a0 tx=0x7fc5e400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc5f6450700 1 -- 172.21.15.33:0/1138981310 shutdown_connections 2024-05-08T21:18:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc5f6450700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f0106bc0 0x7fc5f00ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc5f6450700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5d80a11b0 0x7fc5d80a3670 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc5f6450700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f0101960 0x7fc5f0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc5f6450700 1 --2- 172.21.15.33:0/1138981310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0104290 0x7fc5f00ff910 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.066+0000 7fc5f6450700 1 -- 172.21.15.33:0/1138981310 >> 172.21.15.33:0/1138981310 conn(0x7fc5f00fb310 msgr2=0x7fc5f0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.067+0000 7fc5f6450700 1 -- 172.21.15.33:0/1138981310 shutdown_connections 2024-05-08T21:18:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.067+0000 7fc5f6450700 1 -- 172.21.15.33:0/1138981310 wait complete. 2024-05-08T21:18:23.082 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T21:18:23.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.219+0000 7f176b603700 1 -- 172.21.15.33:0/2435476356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764101fd0 msgr2=0x7f1764102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.219+0000 7f176b603700 1 --2- 172.21.15.33:0/2435476356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764101fd0 0x7f1764102450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f17540098f0 tx=0x7f1754009c00 comp rx=0 tx=0).stop 2024-05-08T21:18:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.219+0000 7f176b603700 1 -- 172.21.15.33:0/2435476356 shutdown_connections 2024-05-08T21:18:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.219+0000 7f176b603700 1 --2- 172.21.15.33:0/2435476356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1764102990 0x7f17641092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.219+0000 7f176b603700 1 --2- 172.21.15.33:0/2435476356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1764100e80 0x7f17641012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.219+0000 7f176b603700 1 --2- 172.21.15.33:0/2435476356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764101fd0 0x7f1764102450 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.219+0000 7f176b603700 1 -- 172.21.15.33:0/2435476356 >> 172.21.15.33:0/2435476356 conn(0x7f17640fc3d0 msgr2=0x7f17640fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.219+0000 7f176b603700 1 -- 172.21.15.33:0/2435476356 shutdown_connections 2024-05-08T21:18:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.219+0000 7f176b603700 1 -- 172.21.15.33:0/2435476356 wait complete. 2024-05-08T21:18:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.219+0000 7f176b603700 1 Processor -- start 2024-05-08T21:18:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176b603700 1 -- start start 2024-05-08T21:18:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176b603700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1764100e80 0x7f1764198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176b603700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764101fd0 0x7f17641993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176b603700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1764102990 0x7f17641a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176b603700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17641a09d0 con 0x7f1764100e80 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176b603700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17641a0b10 con 0x7f1764102990 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176b603700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17641a0ca0 con 0x7f1764101fd0 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176939f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1764100e80 0x7f1764198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176939f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1764100e80 0x7f1764198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50738/0 (socket says 172.21.15.33:50738) 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176939f700 1 -- 172.21.15.33:0/3383510630 learned_addr learned my addr 172.21.15.33:0/3383510630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f1769ba0700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1764102990 0x7f17641a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f1768b9e700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764101fd0 0x7f17641993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176939f700 1 -- 172.21.15.33:0/3383510630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764101fd0 msgr2=0x7f17641993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176939f700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764101fd0 0x7f17641993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176939f700 1 -- 172.21.15.33:0/3383510630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1764102990 msgr2=0x7f17641a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176939f700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1764102990 0x7f17641a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.220+0000 7f176939f700 1 -- 172.21.15.33:0/3383510630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1754009520 con 0x7f1764100e80 2024-05-08T21:18:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.221+0000 7f176939f700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1764100e80 0x7f1764198ea0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f176000e9a0 tx=0x7f176000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.221+0000 7f175a7fc700 1 -- 172.21.15.33:0/3383510630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760009790 con 0x7f1764100e80 2024-05-08T21:18:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.221+0000 7f175a7fc700 1 -- 172.21.15.33:0/3383510630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1760010430 con 0x7f1764100e80 2024-05-08T21:18:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.221+0000 7f176b603700 1 -- 172.21.15.33:0/3383510630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17641a0f80 con 0x7f1764100e80 2024-05-08T21:18:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.221+0000 7f175a7fc700 1 -- 172.21.15.33:0/3383510630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176000cea0 con 0x7f1764100e80 2024-05-08T21:18:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.221+0000 7f176b603700 1 -- 172.21.15.33:0/3383510630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17641a13a0 con 0x7f1764100e80 2024-05-08T21:18:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.223+0000 7f175a7fc700 1 -- 172.21.15.33:0/3383510630 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f17600105a0 con 0x7f1764100e80 2024-05-08T21:18:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.224+0000 7f175a7fc700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17500a11b0 0x7f17500a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.224+0000 7f175a7fc700 1 -- 172.21.15.33:0/3383510630 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1760014070 con 0x7f1764100e80 2024-05-08T21:18:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.224+0000 7f1768b9e700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17500a11b0 0x7f17500a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.225+0000 7f1768b9e700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17500a11b0 0x7f17500a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1754005e90 tx=0x7f1754019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.225+0000 7f17777bdb80 1 Processor -- start 2024-05-08T21:18:23.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.225+0000 7f17777bdb80 1 -- start start 2024-05-08T21:18:23.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.225+0000 7f17777bdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56492324f4f0 0x56492324f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.225+0000 7f17777bdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 0x5649232502d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.225+0000 7f17777bdb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 0x564923250c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.225+0000 7f17777bdb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649231a6100 con 0x564923250810 2024-05-08T21:18:23.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.225+0000 7f17777bdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649231a3ac0 con 0x56492324f4f0 2024-05-08T21:18:23.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.225+0000 7f17777bdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492319f3b0 con 0x56492324fe50 2024-05-08T21:18:23.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.225+0000 7f1769ba0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 0x564923250c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.225+0000 7f1769ba0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 0x564923250c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50754/0 (socket says 172.21.15.33:50754) 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f1769ba0700 1 -- 172.21.15.33:0/1411538744 learned_addr learned my addr 172.21.15.33:0/1411538744 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f1768b9e700 1 --2- 172.21.15.33:0/1411538744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 0x5649232502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f176939f700 1 --2- 172.21.15.33:0/1411538744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56492324f4f0 0x56492324f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f1769ba0700 1 -- 172.21.15.33:0/1411538744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 msgr2=0x5649232502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f1769ba0700 1 --2- 172.21.15.33:0/1411538744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 0x5649232502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f1769ba0700 1 -- 172.21.15.33:0/1411538744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56492324f4f0 msgr2=0x56492324f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f1769ba0700 1 --2- 172.21.15.33:0/1411538744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56492324f4f0 0x56492324f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f1769ba0700 1 -- 172.21.15.33:0/1411538744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564923020360 con 0x564923250810 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f176939f700 1 --2- 172.21.15.33:0/1411538744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56492324f4f0 0x56492324f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f1769ba0700 1 --2- 172.21.15.33:0/1411538744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 0x564923250c90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f175c011670 tx=0x7f175c0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6680efef1c0e5979 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f174f7fe700 1 -- 172.21.15.33:0/1411538744 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c00e040 con 0x564923250810 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f174f7fe700 1 -- 172.21.15.33:0/1411538744 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f175c00f040 con 0x564923250810 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.226+0000 7f174f7fe700 1 -- 172.21.15.33:0/1411538744 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c004570 con 0x564923250810 2024-05-08T21:18:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.227+0000 7f17777bdb80 1 -- 172.21.15.33:0/1411538744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 msgr2=0x564923250c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.227+0000 7f17777bdb80 1 --2- 172.21.15.33:0/1411538744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 0x564923250c90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f175c011670 tx=0x7f175c0099e0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.227+0000 7f17777bdb80 1 -- 172.21.15.33:0/1411538744 shutdown_connections 2024-05-08T21:18:23.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.227+0000 7f17777bdb80 1 --2- 172.21.15.33:0/1411538744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 0x564923250c90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.227+0000 7f17777bdb80 1 --2- 172.21.15.33:0/1411538744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56492324f4f0 0x56492324f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.227+0000 7f17777bdb80 1 --2- 172.21.15.33:0/1411538744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 0x5649232502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.227+0000 7f17777bdb80 1 -- 172.21.15.33:0/1411538744 >> 172.21.15.33:0/1411538744 conn(0x5649231ad1c0 msgr2=0x564923249d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.227+0000 7f17777bdb80 1 -- 172.21.15.33:0/1411538744 shutdown_connections 2024-05-08T21:18:23.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.227+0000 7f17777bdb80 1 -- 172.21.15.33:0/1411538744 wait complete. 2024-05-08T21:18:23.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.227+0000 7f17777bdb80 1 -- start start 2024-05-08T21:18:23.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.228+0000 7f17777bdb80 1 Processor -- start 2024-05-08T21:18:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.228+0000 7f17777bdb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324f4f0 0x564923258d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.228+0000 7f17777bdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 0x564923259290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.228+0000 7f17777bdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564923250810 0x5649232602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.228+0000 7f17777bdb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492313b510 con 0x56492324f4f0 2024-05-08T21:18:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.228+0000 7f17777bdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492313b1b0 con 0x564923250810 2024-05-08T21:18:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.228+0000 7f17777bdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564923022650 con 0x56492324fe50 2024-05-08T21:18:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.228+0000 7f17777bdb80 10 client.0 _ll_register_callbacks cb 0x564923144ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.228+0000 7f176939f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324f4f0 0x564923258d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.228+0000 7f176939f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324f4f0 0x564923258d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50766/0 (socket says 172.21.15.33:50766) 2024-05-08T21:18:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.228+0000 7f176939f700 1 -- 172.21.15.33:0/2398336995 learned_addr learned my addr 172.21.15.33:0/2398336995 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f1768b9e700 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 0x564923259290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f1769ba0700 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564923250810 0x5649232602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f176939f700 1 -- 172.21.15.33:0/2398336995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 msgr2=0x564923259290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f176939f700 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 0x564923259290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f176939f700 1 -- 172.21.15.33:0/2398336995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564923250810 msgr2=0x5649232602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f176939f700 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564923250810 0x5649232602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f176939f700 1 -- 172.21.15.33:0/2398336995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c011350 con 0x56492324f4f0 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f176939f700 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324f4f0 0x564923258d50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1760000c00 tx=0x7f176009a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5174156bf285cf5e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f174cff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f174cff9700 1 -- 172.21.15.33:0/2398336995 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760099f30 con 0x56492324f4f0 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f174cff9700 1 -- 172.21.15.33:0/2398336995 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f176009dd60 con 0x56492324f4f0 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f17777bdb80 1 -- 172.21.15.33:0/2398336995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649230f8220 con 0x56492324f4f0 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f174cff9700 1 -- 172.21.15.33:0/2398336995 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17600a23b0 con 0x56492324f4f0 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f17777bdb80 10 client.15246 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.229+0000 7f17777bdb80 1 -- 172.21.15.33:0/2398336995 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564922fccb10 con 0x56492324f4f0 2024-05-08T21:18:23.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.230+0000 7f174cff9700 1 -- 172.21.15.33:0/2398336995 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5649230f8220 con 0x56492324f4f0 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.230+0000 7f174cff9700 1 -- 172.21.15.33:0/2398336995 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1760094b40 con 0x56492324f4f0 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.230+0000 7f17777bdb80 10 client.15246 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.230+0000 7f17777bdb80 10 client.15246 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.230+0000 7f17777bdb80 20 client.15246 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.230+0000 7f17777bdb80 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x564923263f00 0x564923269200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.230+0000 7f17777bdb80 4 client.15246 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.230+0000 7f17777bdb80 1 -- 172.21.15.33:0/2398336995 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56492321e7c0 con 0x564923263f00 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.230+0000 7f1768b9e700 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x564923263f00 0x564923269200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.230+0000 7f174cff9700 10 client.15246 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.230+0000 7f1768b9e700 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x564923263f00 0x564923269200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c8226c2d77662db server_cookie=7af063c10c8a5f23 in_seq=0 out_seq=0 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.231+0000 7f174cff9700 1 -- 172.21.15.33:0/2398336995 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56492321e7c0 con 0x564923263f00 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.231+0000 7f174cff9700 10 client.15246 handle_command_reply: tid=0 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.231+0000 7f17777bdb80 1 client.15246 shutdown 2024-05-08T21:18:23.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.231+0000 7f17777bdb80 1 -- 172.21.15.33:0/2398336995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324f4f0 msgr2=0x564923258d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.231+0000 7f17777bdb80 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324f4f0 0x564923258d50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1760000c00 tx=0x7f176009a470 comp rx=0 tx=0).stop 2024-05-08T21:18:23.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.231+0000 7f17777bdb80 1 -- 172.21.15.33:0/2398336995 shutdown_connections 2024-05-08T21:18:23.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.231+0000 7f17777bdb80 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x564923263f00 0x564923269200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.231+0000 7f17777bdb80 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564923250810 0x5649232602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.231+0000 7f17777bdb80 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324f4f0 0x564923258d50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.231+0000 7f174cff9700 0 client.15246 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:18:23.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.231+0000 7f17777bdb80 1 --2- 172.21.15.33:0/2398336995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 0x564923259290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.231+0000 7f17777bdb80 1 -- 172.21.15.33:0/2398336995 >> 172.21.15.33:0/2398336995 conn(0x5649231ad1c0 msgr2=0x564923249550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.232+0000 7f17777bdb80 1 -- 172.21.15.33:0/2398336995 shutdown_connections 2024-05-08T21:18:23.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.232+0000 7f17777bdb80 1 -- 172.21.15.33:0/2398336995 wait complete. 2024-05-08T21:18:23.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.232+0000 7f17777bdb80 20 client.15246 trim_cache size 0 max 16384 2024-05-08T21:18:23.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.254+0000 7f17777bdb80 1 Processor -- start 2024-05-08T21:18:23.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.254+0000 7f17777bdb80 1 -- start start 2024-05-08T21:18:23.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.254+0000 7f17777bdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324f4f0 0x56492324f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.254+0000 7f17777bdb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324fe50 0x564923260290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.254+0000 7f17777bdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564923250810 0x564923258260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.254+0000 7f17777bdb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564923257d90 con 0x56492324fe50 2024-05-08T21:18:23.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.254+0000 7f17777bdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492315d2b0 con 0x564923250810 2024-05-08T21:18:23.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.254+0000 7f17777bdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564923198150 con 0x56492324f4f0 2024-05-08T21:18:23.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f1768b9e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324fe50 0x564923260290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f176939f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324f4f0 0x56492324f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f1769ba0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564923250810 0x564923258260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f1768b9e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324fe50 0x564923260290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50776/0 (socket says 172.21.15.33:50776) 2024-05-08T21:18:23.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f1768b9e700 1 -- 172.21.15.33:0/1229277265 learned_addr learned my addr 172.21.15.33:0/1229277265 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f176939f700 1 -- 172.21.15.33:0/1229277265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564923250810 msgr2=0x564923258260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f176939f700 1 --2- 172.21.15.33:0/1229277265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564923250810 0x564923258260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f176939f700 1 -- 172.21.15.33:0/1229277265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324fe50 msgr2=0x564923260290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f176939f700 1 --2- 172.21.15.33:0/1229277265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324fe50 0x564923260290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f176939f700 1 -- 172.21.15.33:0/1229277265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564923165510 con 0x56492324f4f0 2024-05-08T21:18:23.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f176939f700 1 --2- 172.21.15.33:0/1229277265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324f4f0 0x56492324f8d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f17600cab70 tx=0x7f17600963a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=575a2b476437795e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f174ffff700 1 -- 172.21.15.33:0/1229277265 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760099d00 con 0x56492324f4f0 2024-05-08T21:18:23.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f174ffff700 1 -- 172.21.15.33:0/1229277265 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f176009dd60 con 0x56492324f4f0 2024-05-08T21:18:23.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.255+0000 7f174ffff700 1 -- 172.21.15.33:0/1229277265 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760094b40 con 0x56492324f4f0 2024-05-08T21:18:23.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.256+0000 7f17777bdb80 1 -- 172.21.15.33:0/1229277265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324f4f0 msgr2=0x56492324f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.256+0000 7f17777bdb80 1 --2- 172.21.15.33:0/1229277265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324f4f0 0x56492324f8d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f17600cab70 tx=0x7f17600963a0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.256+0000 7f17777bdb80 1 -- 172.21.15.33:0/1229277265 shutdown_connections 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.256+0000 7f17777bdb80 1 --2- 172.21.15.33:0/1229277265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564923250810 0x564923258260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.256+0000 7f17777bdb80 1 --2- 172.21.15.33:0/1229277265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324f4f0 0x56492324f8d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.256+0000 7f17777bdb80 1 --2- 172.21.15.33:0/1229277265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56492324fe50 0x564923260290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.256+0000 7f17777bdb80 1 -- 172.21.15.33:0/1229277265 >> 172.21.15.33:0/1229277265 conn(0x5649231ad1c0 msgr2=0x564923250bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.256+0000 7f17777bdb80 1 -- 172.21.15.33:0/1229277265 shutdown_connections 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.256+0000 7f17777bdb80 1 -- 172.21.15.33:0/1229277265 wait complete. 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.256+0000 7f17777bdb80 1 -- start start 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f17777bdb80 1 Processor -- start 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f17777bdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56492324f4f0 0x56492325a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f17777bdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 0x56492325a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f17777bdb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 0x56492325f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f17777bdb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492325fa40 con 0x564923250810 2024-05-08T21:18:23.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f17777bdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492325fb80 con 0x56492324f4f0 2024-05-08T21:18:23.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f17777bdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56492325fd40 con 0x56492324fe50 2024-05-08T21:18:23.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f17777bdb80 10 client.0 _ll_register_callbacks cb 0x564923144ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:23.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f1769ba0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 0x56492325f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f1769ba0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 0x56492325f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50778/0 (socket says 172.21.15.33:50778) 2024-05-08T21:18:23.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f1769ba0700 1 -- 172.21.15.33:0/663314178 learned_addr learned my addr 172.21.15.33:0/663314178 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f176939f700 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56492324f4f0 0x56492325a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f1768b9e700 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 0x56492325a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f1769ba0700 1 -- 172.21.15.33:0/663314178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 msgr2=0x56492325a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f1769ba0700 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 0x56492325a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f1769ba0700 1 -- 172.21.15.33:0/663314178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56492324f4f0 msgr2=0x56492325a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f1769ba0700 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56492324f4f0 0x56492325a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.257+0000 7f1769ba0700 1 -- 172.21.15.33:0/663314178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564922fccb10 con 0x564923250810 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.258+0000 7f1769ba0700 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 0x56492325f500 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f175c011640 tx=0x7f175c009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2443e19f3850c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.258+0000 7f174cff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.258+0000 7f174cff9700 1 -- 172.21.15.33:0/663314178 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c00e040 con 0x564923250810 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.258+0000 7f174cff9700 1 -- 172.21.15.33:0/663314178 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f175c005590 con 0x564923250810 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.258+0000 7f17777bdb80 1 -- 172.21.15.33:0/663314178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f175c00f040 con 0x564923250810 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.258+0000 7f174cff9700 1 -- 172.21.15.33:0/663314178 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f175c005790 con 0x564923250810 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.258+0000 7f17777bdb80 10 client.15252 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.258+0000 7f17777bdb80 1 -- 172.21.15.33:0/663314178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x564923024c80 con 0x564923250810 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.258+0000 7f174cff9700 1 -- 172.21.15.33:0/663314178 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f175c00f040 con 0x564923250810 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.258+0000 7f174cff9700 1 -- 172.21.15.33:0/663314178 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f175c015450 con 0x564923250810 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.259+0000 7f17777bdb80 10 client.15252 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.259+0000 7f17777bdb80 10 client.15252 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.259+0000 7f17777bdb80 20 client.15252 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.259+0000 7f17777bdb80 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5649232bf180 0x5649232bbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.259+0000 7f17777bdb80 4 client.15252 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.259+0000 7f17777bdb80 1 -- 172.21.15.33:0/663314178 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564923197e70 con 0x5649232bf180 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.259+0000 7f176939f700 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5649232bf180 0x5649232bbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.259+0000 7f174cff9700 10 client.15252 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.259+0000 7f176939f700 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5649232bf180 0x5649232bbbb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e607edca0cfcd87b server_cookie=b6711f379faf34df in_seq=0 out_seq=0 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.259+0000 7f174cff9700 1 -- 172.21.15.33:0/663314178 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x564923197e70 con 0x5649232bf180 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.259+0000 7f174cff9700 10 client.15252 handle_command_reply: tid=0 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.259+0000 7f17777bdb80 1 client.15252 shutdown 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f17777bdb80 1 -- 172.21.15.33:0/663314178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 msgr2=0x56492325f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f17777bdb80 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 0x56492325f500 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f175c011640 tx=0x7f175c009590 comp rx=0 tx=0).stop 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f17777bdb80 1 -- 172.21.15.33:0/663314178 shutdown_connections 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f17777bdb80 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5649232bf180 0x5649232bbbb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f17777bdb80 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564923250810 0x56492325f500 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f17777bdb80 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56492324f4f0 0x56492325a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f17777bdb80 1 --2- 172.21.15.33:0/663314178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56492324fe50 0x56492325a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f174cff9700 0 client.15252 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:18:23.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f17777bdb80 1 -- 172.21.15.33:0/663314178 >> 172.21.15.33:0/663314178 conn(0x5649231ad1c0 msgr2=0x564923258260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f17777bdb80 1 -- 172.21.15.33:0/663314178 shutdown_connections 2024-05-08T21:18:23.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f17777bdb80 1 -- 172.21.15.33:0/663314178 wait complete. 2024-05-08T21:18:23.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f17777bdb80 20 client.15252 trim_cache size 0 max 16384 2024-05-08T21:18:23.269 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:23.269 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:18:23.269 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-05-08T21:18:23.269 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24499, 2024-05-08T21:18:23.269 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-08T21:18:23.269 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-08T21:18:23.270 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-08T21:18:23.270 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 53.928662572, 2024-05-08T21:18:23.270 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-05-08T21:18:23.270 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-05-08T21:18:23.270 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-05-08T21:18:23.270 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 95.177152551000006 2024-05-08T21:18:23.270 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:23.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f176b603700 1 -- 172.21.15.33:0/3383510630 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17500a11b0 msgr2=0x7f17500a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f176b603700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17500a11b0 0x7f17500a3670 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1754005e90 tx=0x7f1754019040 comp rx=0 tx=0).stop 2024-05-08T21:18:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f176b603700 1 -- 172.21.15.33:0/3383510630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1764100e80 msgr2=0x7f1764198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f176b603700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1764100e80 0x7f1764198ea0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f176000e9a0 tx=0x7f176000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f176b603700 1 -- 172.21.15.33:0/3383510630 shutdown_connections 2024-05-08T21:18:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f176b603700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17500a11b0 0x7f17500a3670 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f176b603700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1764102990 0x7f17641a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f176b603700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1764100e80 0x7f1764198ea0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f176b603700 1 --2- 172.21.15.33:0/3383510630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1764101fd0 0x7f17641993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.260+0000 7f176b603700 1 -- 172.21.15.33:0/3383510630 >> 172.21.15.33:0/3383510630 conn(0x7f17640fc3d0 msgr2=0x7f17641078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.261+0000 7f176b603700 1 -- 172.21.15.33:0/3383510630 shutdown_connections 2024-05-08T21:18:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.261+0000 7f176b603700 1 -- 172.21.15.33:0/3383510630 wait complete. 2024-05-08T21:18:23.276 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-08T21:18:23.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.412+0000 7fc9e84af700 1 -- 172.21.15.33:0/3110057592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e0100e70 msgr2=0x7fc9e0101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.412+0000 7fc9e84af700 1 --2- 172.21.15.33:0/3110057592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e0100e70 0x7fc9e0101290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009990 tx=0x7fc9dc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.412+0000 7fc9e84af700 1 -- 172.21.15.33:0/3110057592 shutdown_connections 2024-05-08T21:18:23.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.412+0000 7fc9e84af700 1 --2- 172.21.15.33:0/3110057592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9e0102980 0x7fc9e01092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.412+0000 7fc9e84af700 1 --2- 172.21.15.33:0/3110057592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e0100e70 0x7fc9e0101290 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.412+0000 7fc9e84af700 1 --2- 172.21.15.33:0/3110057592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9e0101fc0 0x7fc9e0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.412+0000 7fc9e84af700 1 -- 172.21.15.33:0/3110057592 >> 172.21.15.33:0/3110057592 conn(0x7fc9e00fc400 msgr2=0x7fc9e00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.412+0000 7fc9e84af700 1 -- 172.21.15.33:0/3110057592 shutdown_connections 2024-05-08T21:18:23.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.412+0000 7fc9e84af700 1 -- 172.21.15.33:0/3110057592 wait complete. 2024-05-08T21:18:23.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e84af700 1 Processor -- start 2024-05-08T21:18:23.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e84af700 1 -- start start 2024-05-08T21:18:23.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e84af700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9e0100e70 0x7fc9e0198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e84af700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e0101fc0 0x7fc9e0199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e84af700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9e0102980 0x7fc9e01a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e84af700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e01a0a20 con 0x7fc9e0101fc0 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e84af700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e01a0b60 con 0x7fc9e0102980 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e84af700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e01a0cf0 con 0x7fc9e0100e70 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e5a4a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e0101fc0 0x7fc9e0199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e5a4a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e0101fc0 0x7fc9e0199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50792/0 (socket says 172.21.15.33:50792) 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e5a4a700 1 -- 172.21.15.33:0/3872829180 learned_addr learned my addr 172.21.15.33:0/3872829180 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e6a4c700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9e0102980 0x7fc9e01a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.413+0000 7fc9e624b700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9e0100e70 0x7fc9e0198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.414+0000 7fc9e5a4a700 1 -- 172.21.15.33:0/3872829180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9e0100e70 msgr2=0x7fc9e0198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.414+0000 7fc9e5a4a700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9e0100e70 0x7fc9e0198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.414+0000 7fc9e5a4a700 1 -- 172.21.15.33:0/3872829180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9e0102980 msgr2=0x7fc9e01a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.414+0000 7fc9e5a4a700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9e0102980 0x7fc9e01a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.414+0000 7fc9e5a4a700 1 -- 172.21.15.33:0/3872829180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc0095f0 con 0x7fc9e0101fc0 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.414+0000 7fc9e624b700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9e0100e70 0x7fc9e0198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.414+0000 7fc9e5a4a700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e0101fc0 0x7fc9e0199430 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400d950 tx=0x7fc9d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.414+0000 7fc9d37fe700 1 -- 172.21.15.33:0/3872829180 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4009940 con 0x7fc9e0101fc0 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.414+0000 7fc9d37fe700 1 -- 172.21.15.33:0/3872829180 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9d4010460 con 0x7fc9e0101fc0 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.414+0000 7fc9e84af700 1 -- 172.21.15.33:0/3872829180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e01a0fd0 con 0x7fc9e0101fc0 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.414+0000 7fc9d37fe700 1 -- 172.21.15.33:0/3872829180 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d400f620 con 0x7fc9e0101fc0 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.414+0000 7fc9e84af700 1 -- 172.21.15.33:0/3872829180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e01a1450 con 0x7fc9e0101fc0 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.417+0000 7fc9d37fe700 1 -- 172.21.15.33:0/3872829180 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc9d40105d0 con 0x7fc9e0101fc0 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.417+0000 7fc9d37fe700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9cc0a11b0 0x7fc9cc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.418+0000 7fc9e624b700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9cc0a11b0 0x7fc9cc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.418+0000 7fc9d37fe700 1 -- 172.21.15.33:0/3872829180 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc9d40cc600 con 0x7fc9e0101fc0 2024-05-08T21:18:23.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.418+0000 7fc9e624b700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9cc0a11b0 0x7fc9cc0a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00b5c0 tx=0x7fc9dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.418+0000 7fc9f4669b80 1 Processor -- start 2024-05-08T21:18:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.419+0000 7fc9f4669b80 1 -- start start 2024-05-08T21:18:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.419+0000 7fc9f4669b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524b4f0 0x555fc524b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.419+0000 7fc9f4669b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc524c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.419+0000 7fc9f4669b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 0x555fc524cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.419+0000 7fc9f4669b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fc51a2100 con 0x555fc524c810 2024-05-08T21:18:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.419+0000 7fc9f4669b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fc519fac0 con 0x555fc524be50 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.419+0000 7fc9f4669b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fc519b3b0 con 0x555fc524b4f0 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.419+0000 7fc9e6a4c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 0x555fc524cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.419+0000 7fc9e6a4c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 0x555fc524cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50806/0 (socket says 172.21.15.33:50806) 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.419+0000 7fc9e6a4c700 1 -- 172.21.15.33:0/1196323074 learned_addr learned my addr 172.21.15.33:0/1196323074 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.419+0000 7fc9e5a4a700 1 --2- 172.21.15.33:0/1196323074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc524c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.419+0000 7fc9e624b700 1 --2- 172.21.15.33:0/1196323074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524b4f0 0x555fc524b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.420+0000 7fc9e6a4c700 1 -- 172.21.15.33:0/1196323074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524b4f0 msgr2=0x555fc524b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.420+0000 7fc9e6a4c700 1 --2- 172.21.15.33:0/1196323074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524b4f0 0x555fc524b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.420+0000 7fc9e6a4c700 1 -- 172.21.15.33:0/1196323074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 msgr2=0x555fc524c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.420+0000 7fc9e6a4c700 1 --2- 172.21.15.33:0/1196323074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc524c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.420+0000 7fc9e6a4c700 1 -- 172.21.15.33:0/1196323074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fc501c360 con 0x555fc524c810 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.420+0000 7fc9e5a4a700 1 --2- 172.21.15.33:0/1196323074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc524c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.420+0000 7fc9e6a4c700 1 --2- 172.21.15.33:0/1196323074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 0x555fc524cc90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80098f0 tx=0x7fc9d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205e1b84034924a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.420+0000 7fc9d0da4700 1 -- 172.21.15.33:0/1196323074 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8019070 con 0x555fc524c810 2024-05-08T21:18:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.420+0000 7fc9d0da4700 1 -- 172.21.15.33:0/1196323074 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9d800f040 con 0x555fc524c810 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.420+0000 7fc9d0da4700 1 -- 172.21.15.33:0/1196323074 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80079c0 con 0x555fc524c810 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.421+0000 7fc9f4669b80 1 -- 172.21.15.33:0/1196323074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 msgr2=0x555fc524cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.421+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/1196323074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 0x555fc524cc90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80098f0 tx=0x7fc9d8009c00 comp rx=0 tx=0).stop 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.421+0000 7fc9f4669b80 1 -- 172.21.15.33:0/1196323074 shutdown_connections 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.421+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/1196323074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 0x555fc524cc90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.421+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/1196323074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524b4f0 0x555fc524b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.421+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/1196323074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc524c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.421+0000 7fc9f4669b80 1 -- 172.21.15.33:0/1196323074 >> 172.21.15.33:0/1196323074 conn(0x555fc51a91c0 msgr2=0x555fc5245d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.421+0000 7fc9f4669b80 1 -- 172.21.15.33:0/1196323074 shutdown_connections 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.421+0000 7fc9f4669b80 1 -- 172.21.15.33:0/1196323074 wait complete. 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.421+0000 7fc9f4669b80 1 -- start start 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.421+0000 7fc9f4669b80 1 Processor -- start 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9f4669b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524b4f0 0x555fc5254da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9f4669b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc52552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9f4669b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 0x555fc525c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9f4669b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fc5137510 con 0x555fc524c810 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9f4669b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fc51371b0 con 0x555fc524be50 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9f4669b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fc501e650 con 0x555fc524b4f0 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9f4669b80 10 client.0 _ll_register_callbacks cb 0x555fc5140ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9e6a4c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 0x555fc525c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9e6a4c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 0x555fc525c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50822/0 (socket says 172.21.15.33:50822) 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9e6a4c700 1 -- 172.21.15.33:0/2722820150 learned_addr learned my addr 172.21.15.33:0/2722820150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9e5a4a700 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc52552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9e624b700 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524b4f0 0x555fc5254da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9e6a4c700 1 -- 172.21.15.33:0/2722820150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524b4f0 msgr2=0x555fc5254da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9e6a4c700 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524b4f0 0x555fc5254da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9e6a4c700 1 -- 172.21.15.33:0/2722820150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 msgr2=0x555fc52552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9e6a4c700 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc52552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9e6a4c700 1 -- 172.21.15.33:0/2722820150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e01a1450 con 0x555fc524c810 2024-05-08T21:18:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9e624b700 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524b4f0 0x555fc5254da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9e6a4c700 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 0x555fc525c320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8004fa0 tx=0x7fc9d8004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52913a442ef5002 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9c9ffb700 1 -- 172.21.15.33:0/2722820150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8019040 con 0x555fc524c810 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.422+0000 7fc9c9ffb700 1 -- 172.21.15.33:0/2722820150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc9d8007b50 con 0x555fc524c810 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2722820150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fc50f4220 con 0x555fc524c810 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9c9ffb700 1 -- 172.21.15.33:0/2722820150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9d8007d10 con 0x555fc524c810 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9f4669b80 10 client.15264 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2722820150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fc9d8003c50 con 0x555fc524c810 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9c9ffb700 1 -- 172.21.15.33:0/2722820150 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555fc50f4220 con 0x555fc524c810 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9c9ffb700 1 -- 172.21.15.33:0/2722820150 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9d801f450 con 0x555fc524c810 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9f4669b80 10 client.15264 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9f4669b80 10 client.15264 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9f4669b80 20 client.15264 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x555fc525ff20 0x555fc5265200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9f4669b80 4 client.15264 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2722820150 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fc521a7c0 con 0x555fc525ff20 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.423+0000 7fc9e624b700 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x555fc525ff20 0x555fc5265200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.424+0000 7fc9c9ffb700 10 client.15264 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.424+0000 7fc9e624b700 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x555fc525ff20 0x555fc5265200 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13a8f3ae2388d01d server_cookie=848dd592e54d0e0 in_seq=0 out_seq=0 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9c9ffb700 1 -- 172.21.15.33:0/2722820150 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555fc521a7c0 con 0x555fc525ff20 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9c9ffb700 10 client.15264 handle_command_reply: tid=0 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9f4669b80 1 client.15264 shutdown 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2722820150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 msgr2=0x555fc525c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 0x555fc525c320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8004fa0 tx=0x7fc9d8004700 comp rx=0 tx=0).stop 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2722820150 shutdown_connections 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x555fc525ff20 0x555fc5265200 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524c810 0x555fc525c320 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524b4f0 0x555fc5254da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9c9ffb700 0 client.15264 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:18:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/2722820150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc52552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2722820150 >> 172.21.15.33:0/2722820150 conn(0x555fc51a91c0 msgr2=0x555fc524a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2722820150 shutdown_connections 2024-05-08T21:18:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2722820150 wait complete. 2024-05-08T21:18:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.425+0000 7fc9f4669b80 20 client.15264 trim_cache size 0 max 16384 2024-05-08T21:18:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.444+0000 7fc9f4669b80 1 Processor -- start 2024-05-08T21:18:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.444+0000 7fc9f4669b80 1 -- start start 2024-05-08T21:18:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.444+0000 7fc9f4669b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524b4f0 0x555fc524b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.444+0000 7fc9f4669b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524be50 0x555fc525c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.444+0000 7fc9f4669b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524c810 0x555fc5252da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.444+0000 7fc9f4669b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fc5242d70 con 0x555fc524be50 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.444+0000 7fc9f4669b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fc521a7c0 con 0x555fc524b4f0 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.444+0000 7fc9f4669b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fc50f4220 con 0x555fc524c810 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.444+0000 7fc9e5a4a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524be50 0x555fc525c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9e5a4a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524be50 0x555fc525c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50832/0 (socket says 172.21.15.33:50832) 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9e5a4a700 1 -- 172.21.15.33:0/924795464 learned_addr learned my addr 172.21.15.33:0/924795464 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9e624b700 1 --2- 172.21.15.33:0/924795464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524b4f0 0x555fc524b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9e6a4c700 1 --2- 172.21.15.33:0/924795464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524c810 0x555fc5252da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9e5a4a700 1 -- 172.21.15.33:0/924795464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524c810 msgr2=0x555fc5252da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9e5a4a700 1 --2- 172.21.15.33:0/924795464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524c810 0x555fc5252da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9e5a4a700 1 -- 172.21.15.33:0/924795464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524b4f0 msgr2=0x555fc524b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9e5a4a700 1 --2- 172.21.15.33:0/924795464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524b4f0 0x555fc524b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9e5a4a700 1 -- 172.21.15.33:0/924795464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fc4fc8b10 con 0x555fc524be50 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9e624b700 1 --2- 172.21.15.33:0/924795464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524b4f0 0x555fc524b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9e5a4a700 1 --2- 172.21.15.33:0/924795464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524be50 0x555fc525c2e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4099570 tx=0x7fc9d4084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537ea739f2326e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9d15a5700 1 -- 172.21.15.33:0/924795464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40108d0 con 0x555fc524be50 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9d15a5700 1 -- 172.21.15.33:0/924795464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9d4099ce0 con 0x555fc524be50 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.445+0000 7fc9d15a5700 1 -- 172.21.15.33:0/924795464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40c9ef0 con 0x555fc524be50 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 -- 172.21.15.33:0/924795464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524be50 msgr2=0x555fc525c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/924795464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524be50 0x555fc525c2e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4099570 tx=0x7fc9d4084040 comp rx=0 tx=0).stop 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 -- 172.21.15.33:0/924795464 shutdown_connections 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/924795464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524c810 0x555fc5252da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/924795464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524b4f0 0x555fc524b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/924795464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524be50 0x555fc525c2e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 -- 172.21.15.33:0/924795464 >> 172.21.15.33:0/924795464 conn(0x555fc51a91c0 msgr2=0x555fc52421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 -- 172.21.15.33:0/924795464 shutdown_connections 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 -- 172.21.15.33:0/924795464 wait complete. 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 -- start start 2024-05-08T21:18:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 Processor -- start 2024-05-08T21:18:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524b4f0 0x555fc5259270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc525b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524c810 0x555fc525bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fc525c3e0 con 0x555fc524b4f0 2024-05-08T21:18:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fc525c550 con 0x555fc524be50 2024-05-08T21:18:23.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fc525c710 con 0x555fc524c810 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.446+0000 7fc9f4669b80 10 client.0 _ll_register_callbacks cb 0x555fc5140ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9e624b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524b4f0 0x555fc5259270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9e624b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524b4f0 0x555fc5259270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50842/0 (socket says 172.21.15.33:50842) 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9e624b700 1 -- 172.21.15.33:0/2209792052 learned_addr learned my addr 172.21.15.33:0/2209792052 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9e5a4a700 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc525b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9e6a4c700 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524c810 0x555fc525bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9e624b700 1 -- 172.21.15.33:0/2209792052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524c810 msgr2=0x555fc525bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9e624b700 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524c810 0x555fc525bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9e624b700 1 -- 172.21.15.33:0/2209792052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 msgr2=0x555fc525b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9e624b700 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc525b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9e624b700 1 -- 172.21.15.33:0/2209792052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e01a0fd0 con 0x555fc524b4f0 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9e6a4c700 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524c810 0x555fc525bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9e624b700 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524b4f0 0x555fc5259270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009990 tx=0x7fc9dc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5fc27a08367c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9c9ffb700 1 -- 172.21.15.33:0/2209792052 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc02b040 con 0x555fc524b4f0 2024-05-08T21:18:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9c9ffb700 1 -- 172.21.15.33:0/2209792052 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc9dc0050c0 con 0x555fc524b4f0 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2209792052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9d800f040 con 0x555fc524b4f0 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9c9ffb700 1 -- 172.21.15.33:0/2209792052 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9dc0348e0 con 0x555fc524b4f0 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.447+0000 7fc9f4669b80 10 client.15270 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.448+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2209792052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555fc5161510 con 0x555fc524b4f0 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.448+0000 7fc9c9ffb700 1 -- 172.21.15.33:0/2209792052 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9d800f040 con 0x555fc524b4f0 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.448+0000 7fc9c9ffb700 1 -- 172.21.15.33:0/2209792052 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9dc004cc0 con 0x555fc524b4f0 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.448+0000 7fc9f4669b80 10 client.15270 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.448+0000 7fc9f4669b80 10 client.15270 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.448+0000 7fc9f4669b80 20 client.15270 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.448+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x555fc52bba80 0x555fc52b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.448+0000 7fc9f4669b80 4 client.15270 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.448+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2209792052 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555fc5193e70 con 0x555fc52bba80 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.448+0000 7fc9e5a4a700 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x555fc52bba80 0x555fc52b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.448+0000 7fc9e5a4a700 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x555fc52bba80 0x555fc52b83f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61327570eab02ff6 server_cookie=cbb8cc3673ee4cb6 in_seq=0 out_seq=0 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.448+0000 7fc9c9ffb700 10 client.15270 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9c9ffb700 1 -- 172.21.15.33:0/2209792052 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x555fc5193e70 con 0x555fc52bba80 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9c9ffb700 10 client.15270 handle_command_reply: tid=0 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9f4669b80 1 client.15270 shutdown 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2209792052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524b4f0 msgr2=0x555fc5259270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524b4f0 0x555fc5259270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009990 tx=0x7fc9dc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2209792052 shutdown_connections 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x555fc52bba80 0x555fc52b83f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555fc524c810 0x555fc525bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fc524b4f0 0x555fc5259270 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9c9ffb700 0 client.15270 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9f4669b80 1 --2- 172.21.15.33:0/2209792052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fc524be50 0x555fc525b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2209792052 >> 172.21.15.33:0/2209792052 conn(0x555fc51a91c0 msgr2=0x555fc524f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2209792052 shutdown_connections 2024-05-08T21:18:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9f4669b80 1 -- 172.21.15.33:0/2209792052 wait complete. 2024-05-08T21:18:23.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.449+0000 7fc9f4669b80 20 client.15270 trim_cache size 0 max 16384 2024-05-08T21:18:23.458 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:18:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-05-08T21:18:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24521, 2024-05-08T21:18:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-08T21:18:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-08T21:18:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-08T21:18:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 55.125387101999998, 2024-05-08T21:18:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-05-08T21:18:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-05-08T21:18:23.458 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-05-08T21:18:23.459 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 93.581320767999998 2024-05-08T21:18:23.459 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.450+0000 7fc9e84af700 1 -- 172.21.15.33:0/3872829180 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9cc0a11b0 msgr2=0x7fc9cc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.450+0000 7fc9e84af700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9cc0a11b0 0x7fc9cc0a3670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00b5c0 tx=0x7fc9dc005fb0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.450+0000 7fc9e84af700 1 -- 172.21.15.33:0/3872829180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e0101fc0 msgr2=0x7fc9e0199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.450+0000 7fc9e84af700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e0101fc0 0x7fc9e0199430 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400d950 tx=0x7fc9d400dc60 comp rx=0 tx=0).stop 2024-05-08T21:18:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.450+0000 7fc9e84af700 1 -- 172.21.15.33:0/3872829180 shutdown_connections 2024-05-08T21:18:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.450+0000 7fc9e84af700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9cc0a11b0 0x7fc9cc0a3670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.450+0000 7fc9e84af700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9e0102980 0x7fc9e01a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.450+0000 7fc9e84af700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9e0100e70 0x7fc9e0198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.450+0000 7fc9e84af700 1 --2- 172.21.15.33:0/3872829180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9e0101fc0 0x7fc9e0199430 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.450+0000 7fc9e84af700 1 -- 172.21.15.33:0/3872829180 >> 172.21.15.33:0/3872829180 conn(0x7fc9e00fc400 msgr2=0x7fc9e0107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.450+0000 7fc9e84af700 1 -- 172.21.15.33:0/3872829180 shutdown_connections 2024-05-08T21:18:23.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.450+0000 7fc9e84af700 1 -- 172.21.15.33:0/3872829180 wait complete. 2024-05-08T21:18:23.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:23 smithi033 ceph-mon[42528]: pgmap v270: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:18:23.460 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:23 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1709972412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:23.466 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T21:18:23.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:23 smithi071 ceph-mon[47074]: pgmap v270: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:18:23.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:23 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1709972412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:23.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:23 smithi082 ceph-mon[47512]: pgmap v270: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:18:23.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:23 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1709972412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.602+0000 7fbabaf4e700 1 -- 172.21.15.33:0/2611628352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab4100e70 msgr2=0x7fbab4101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.602+0000 7fbabaf4e700 1 --2- 172.21.15.33:0/2611628352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab4100e70 0x7fbab4101290 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009990 tx=0x7fbaa8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.602+0000 7fbabaf4e700 1 -- 172.21.15.33:0/2611628352 shutdown_connections 2024-05-08T21:18:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.602+0000 7fbabaf4e700 1 --2- 172.21.15.33:0/2611628352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbab4102980 0x7fbab41092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.602+0000 7fbabaf4e700 1 --2- 172.21.15.33:0/2611628352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab4100e70 0x7fbab4101290 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.602+0000 7fbabaf4e700 1 --2- 172.21.15.33:0/2611628352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbab4101fc0 0x7fbab4102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.602+0000 7fbabaf4e700 1 -- 172.21.15.33:0/2611628352 >> 172.21.15.33:0/2611628352 conn(0x7fbab40fc400 msgr2=0x7fbab40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.602+0000 7fbabaf4e700 1 -- 172.21.15.33:0/2611628352 shutdown_connections 2024-05-08T21:18:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.602+0000 7fbabaf4e700 1 -- 172.21.15.33:0/2611628352 wait complete. 2024-05-08T21:18:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbabaf4e700 1 Processor -- start 2024-05-08T21:18:23.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbabaf4e700 1 -- start start 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbabaf4e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbab4100e70 0x7fbab4196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbabaf4e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbab4101fc0 0x7fbab41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbabaf4e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab4102980 0x7fbab419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbabaf4e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab419e8c0 con 0x7fbab4102980 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbabaf4e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab419ea00 con 0x7fbab4101fc0 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbabaf4e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab419eb90 con 0x7fbab4100e70 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbab94eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab4102980 0x7fbab419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbab94eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab4102980 0x7fbab419e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50872/0 (socket says 172.21.15.33:50872) 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbab94eb700 1 -- 172.21.15.33:0/118079620 learned_addr learned my addr 172.21.15.33:0/118079620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbab8cea700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbab4100e70 0x7fbab4196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.603+0000 7fbab3fff700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbab4101fc0 0x7fbab41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.604+0000 7fbab94eb700 1 -- 172.21.15.33:0/118079620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbab4100e70 msgr2=0x7fbab4196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.604+0000 7fbab94eb700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbab4100e70 0x7fbab4196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.604+0000 7fbab94eb700 1 -- 172.21.15.33:0/118079620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbab4101fc0 msgr2=0x7fbab41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.604+0000 7fbab94eb700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbab4101fc0 0x7fbab41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.604+0000 7fbab94eb700 1 -- 172.21.15.33:0/118079620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac009c40 con 0x7fbab4102980 2024-05-08T21:18:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.604+0000 7fbab94eb700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab4102980 0x7fbab419e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00c370 tx=0x7fbaac00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.604+0000 7fbab1ffb700 1 -- 172.21.15.33:0/118079620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac017070 con 0x7fbab4102980 2024-05-08T21:18:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.604+0000 7fbab1ffb700 1 -- 172.21.15.33:0/118079620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbaac00f040 con 0x7fbab4102980 2024-05-08T21:18:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.604+0000 7fbab1ffb700 1 -- 172.21.15.33:0/118079620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac00e050 con 0x7fbab4102980 2024-05-08T21:18:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.604+0000 7fbabaf4e700 1 -- 172.21.15.33:0/118079620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa80095f0 con 0x7fbab4102980 2024-05-08T21:18:23.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.604+0000 7fbabaf4e700 1 -- 172.21.15.33:0/118079620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab419f140 con 0x7fbab4102980 2024-05-08T21:18:23.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.606+0000 7fbab1ffb700 1 -- 172.21.15.33:0/118079620 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fbaac00cb60 con 0x7fbab4102980 2024-05-08T21:18:23.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.607+0000 7fbab1ffb700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbaa40a11b0 0x7fbaa40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.607+0000 7fbab8cea700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbaa40a11b0 0x7fbaa40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.607+0000 7fbab1ffb700 1 -- 172.21.15.33:0/118079620 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbaac0cc740 con 0x7fbab4102980 2024-05-08T21:18:23.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.608+0000 7fbab8cea700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbaa40a11b0 0x7fbaa40a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8006010 tx=0x7fbaa800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.608+0000 7fbac7108b80 1 Processor -- start 2024-05-08T21:18:23.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.608+0000 7fbac7108b80 1 -- start start 2024-05-08T21:18:23.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.608+0000 7fbac7108b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 0x561084f1cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.608+0000 7fbac7108b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 0x561084f1d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.608+0000 7fbac7108b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 0x561084f1e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.608+0000 7fbac7108b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084e6f100 con 0x561084f1c900 2024-05-08T21:18:23.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.608+0000 7fbac7108b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084e6cac0 con 0x561084f1dc20 2024-05-08T21:18:23.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.608+0000 7fbac7108b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084e683b0 con 0x561084f1d260 2024-05-08T21:18:23.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.608+0000 7fbab8cea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 0x561084f1cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fbab8cea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 0x561084f1cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50888/0 (socket says 172.21.15.33:50888) 2024-05-08T21:18:23.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fbab8cea700 1 -- 172.21.15.33:0/3508118977 learned_addr learned my addr 172.21.15.33:0/3508118977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fbab3fff700 1 --2- 172.21.15.33:0/3508118977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 0x561084f1d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fbab94eb700 1 --2- 172.21.15.33:0/3508118977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 0x561084f1e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fbab8cea700 1 -- 172.21.15.33:0/3508118977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 msgr2=0x561084f1d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fbab8cea700 1 --2- 172.21.15.33:0/3508118977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 0x561084f1d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fbab8cea700 1 -- 172.21.15.33:0/3508118977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 msgr2=0x561084f1e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fbab8cea700 1 --2- 172.21.15.33:0/3508118977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 0x561084f1e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fbab8cea700 1 -- 172.21.15.33:0/3508118977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561084ce9360 con 0x561084f1c900 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fbab94eb700 1 --2- 172.21.15.33:0/3508118977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 0x561084f1e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fbab8cea700 1 --2- 172.21.15.33:0/3508118977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 0x561084f1cd20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8015000 tx=0x7fbaa8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7970cebdd6f7f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fba9effd700 1 -- 172.21.15.33:0/3508118977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8003d10 con 0x561084f1c900 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fba9effd700 1 -- 172.21.15.33:0/3508118977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbaa801e040 con 0x561084f1c900 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.609+0000 7fba9effd700 1 -- 172.21.15.33:0/3508118977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa80048c0 con 0x561084f1c900 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.610+0000 7fbac7108b80 1 -- 172.21.15.33:0/3508118977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 msgr2=0x561084f1cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.610+0000 7fbac7108b80 1 --2- 172.21.15.33:0/3508118977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 0x561084f1cd20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8015000 tx=0x7fbaa8020000 comp rx=0 tx=0).stop 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.610+0000 7fbac7108b80 1 -- 172.21.15.33:0/3508118977 shutdown_connections 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.610+0000 7fbac7108b80 1 --2- 172.21.15.33:0/3508118977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 0x561084f1e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.610+0000 7fbac7108b80 1 --2- 172.21.15.33:0/3508118977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 0x561084f1cd20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.610+0000 7fbac7108b80 1 --2- 172.21.15.33:0/3508118977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 0x561084f1d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.610+0000 7fbac7108b80 1 -- 172.21.15.33:0/3508118977 >> 172.21.15.33:0/3508118977 conn(0x561084e761c0 msgr2=0x561084f12d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.610+0000 7fbac7108b80 1 -- 172.21.15.33:0/3508118977 shutdown_connections 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.610+0000 7fbac7108b80 1 -- 172.21.15.33:0/3508118977 wait complete. 2024-05-08T21:18:23.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbac7108b80 1 -- start start 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbac7108b80 1 Processor -- start 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbac7108b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 0x561084f26160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbac7108b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 0x561084f266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbac7108b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 0x561084f2d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbac7108b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084e04510 con 0x561084f1c900 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbac7108b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084e041b0 con 0x561084f1dc20 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbac7108b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084ceb650 con 0x561084f1d260 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbac7108b80 10 client.0 _ll_register_callbacks cb 0x561084e0dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbab8cea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 0x561084f26160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbab8cea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 0x561084f26160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50896/0 (socket says 172.21.15.33:50896) 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbab8cea700 1 -- 172.21.15.33:0/4075895301 learned_addr learned my addr 172.21.15.33:0/4075895301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbab3fff700 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 0x561084f266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.611+0000 7fbab94eb700 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 0x561084f2d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fbab8cea700 1 -- 172.21.15.33:0/4075895301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 msgr2=0x561084f266a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fbab8cea700 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 0x561084f266a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fbab8cea700 1 -- 172.21.15.33:0/4075895301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 msgr2=0x561084f2d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fbab8cea700 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 0x561084f2d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fbab8cea700 1 -- 172.21.15.33:0/4075895301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab4109bd0 con 0x561084f1c900 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fbab8cea700 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 0x561084f26160 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80098e0 tx=0x7fbaa8004eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecd8062f01c23a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fba93fff700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fba93fff700 1 -- 172.21.15.33:0/4075895301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa800b8b0 con 0x561084f1c900 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fba93fff700 1 -- 172.21.15.33:0/4075895301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbaa8004910 con 0x561084f1c900 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fbac7108b80 1 -- 172.21.15.33:0/4075895301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561084dc1220 con 0x561084f1c900 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fba93fff700 1 -- 172.21.15.33:0/4075895301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaa800ba10 con 0x561084f1c900 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fbac7108b80 10 client.15282 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fbac7108b80 1 -- 172.21.15.33:0/4075895301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561084c95b10 con 0x561084f1c900 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.612+0000 7fba93fff700 1 -- 172.21.15.33:0/4075895301 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561084dc1220 con 0x561084f1c900 2024-05-08T21:18:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.613+0000 7fba93fff700 1 -- 172.21.15.33:0/4075895301 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbaa802fca0 con 0x561084f1c900 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.613+0000 7fbac7108b80 10 client.15282 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.613+0000 7fbac7108b80 10 client.15282 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.613+0000 7fbac7108b80 20 client.15282 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.613+0000 7fbac7108b80 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561084f30ec0 0x561084f36180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.613+0000 7fbac7108b80 4 client.15282 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.613+0000 7fbac7108b80 1 -- 172.21.15.33:0/4075895301 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561084ee77c0 con 0x561084f30ec0 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.613+0000 7fbab3fff700 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561084f30ec0 0x561084f36180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.613+0000 7fba93fff700 10 client.15282 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.613+0000 7fbab3fff700 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561084f30ec0 0x561084f36180 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a949de0bbc23025 server_cookie=4cd4579dde388314 in_seq=0 out_seq=0 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.614+0000 7fba93fff700 1 -- 172.21.15.33:0/4075895301 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561084ee77c0 con 0x561084f30ec0 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.614+0000 7fba93fff700 10 client.15282 handle_command_reply: tid=0 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.614+0000 7fbac7108b80 1 client.15282 shutdown 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.614+0000 7fbac7108b80 1 -- 172.21.15.33:0/4075895301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 msgr2=0x561084f26160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.614+0000 7fbac7108b80 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 0x561084f26160 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80098e0 tx=0x7fbaa8004eb0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.614+0000 7fbac7108b80 1 -- 172.21.15.33:0/4075895301 shutdown_connections 2024-05-08T21:18:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.614+0000 7fbac7108b80 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561084f30ec0 0x561084f36180 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.614+0000 7fbac7108b80 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 0x561084f2d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.614+0000 7fbac7108b80 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1c900 0x561084f26160 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.614+0000 7fba93fff700 0 client.15282 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.614+0000 7fbac7108b80 1 --2- 172.21.15.33:0/4075895301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 0x561084f266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.614+0000 7fbac7108b80 1 -- 172.21.15.33:0/4075895301 >> 172.21.15.33:0/4075895301 conn(0x561084e761c0 msgr2=0x561084f12550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.615+0000 7fbac7108b80 1 -- 172.21.15.33:0/4075895301 shutdown_connections 2024-05-08T21:18:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.615+0000 7fbac7108b80 1 -- 172.21.15.33:0/4075895301 wait complete. 2024-05-08T21:18:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.615+0000 7fbac7108b80 20 client.15282 trim_cache size 0 max 16384 2024-05-08T21:18:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.634+0000 7fbac7108b80 1 Processor -- start 2024-05-08T21:18:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.634+0000 7fbac7108b80 1 -- start start 2024-05-08T21:18:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.634+0000 7fbac7108b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1c900 0x561084f13580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.634+0000 7fbac7108b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1d260 0x561084f2d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.634+0000 7fbac7108b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 0x561084f2dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.634+0000 7fbac7108b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084e262b0 con 0x561084f1d260 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.634+0000 7fbac7108b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084e61150 con 0x561084f1dc20 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.634+0000 7fbac7108b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084e65010 con 0x561084f1c900 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.634+0000 7fbab3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1d260 0x561084f2d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.634+0000 7fbab3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1d260 0x561084f2d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50912/0 (socket says 172.21.15.33:50912) 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.634+0000 7fbab3fff700 1 -- 172.21.15.33:0/4034071603 learned_addr learned my addr 172.21.15.33:0/4034071603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.634+0000 7fbab8cea700 1 --2- 172.21.15.33:0/4034071603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1c900 0x561084f13580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.635+0000 7fbab3fff700 1 -- 172.21.15.33:0/4034071603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1c900 msgr2=0x561084f13580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.635+0000 7fbab3fff700 1 --2- 172.21.15.33:0/4034071603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1c900 0x561084f13580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.635+0000 7fbab3fff700 1 -- 172.21.15.33:0/4034071603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 msgr2=0x561084f2dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.635+0000 7fbab3fff700 1 --2- 172.21.15.33:0/4034071603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 0x561084f2dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.635+0000 7fbab3fff700 1 -- 172.21.15.33:0/4034071603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561084e2e510 con 0x561084f1d260 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.635+0000 7fbab8cea700 1 --2- 172.21.15.33:0/4034071603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1c900 0x561084f13580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.635+0000 7fbab3fff700 1 --2- 172.21.15.33:0/4034071603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1d260 0x561084f2d6a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x561084e04290 tx=0x7fbaa000b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4258a53e8a300a38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.635+0000 7fba9f7fe700 1 -- 172.21.15.33:0/4034071603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0016030 con 0x561084f1d260 2024-05-08T21:18:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.635+0000 7fba9f7fe700 1 -- 172.21.15.33:0/4034071603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbaa0017040 con 0x561084f1d260 2024-05-08T21:18:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.635+0000 7fba9f7fe700 1 -- 172.21.15.33:0/4034071603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0005260 con 0x561084f1d260 2024-05-08T21:18:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.635+0000 7fbac7108b80 1 -- 172.21.15.33:0/4034071603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1d260 msgr2=0x561084f2d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.635+0000 7fbac7108b80 1 --2- 172.21.15.33:0/4034071603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1d260 0x561084f2d6a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x561084e04290 tx=0x7fbaa000b3c0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 -- 172.21.15.33:0/4034071603 shutdown_connections 2024-05-08T21:18:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 --2- 172.21.15.33:0/4034071603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1dc20 0x561084f2dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 --2- 172.21.15.33:0/4034071603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1c900 0x561084f13580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 --2- 172.21.15.33:0/4034071603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1d260 0x561084f2d6a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 -- 172.21.15.33:0/4034071603 >> 172.21.15.33:0/4034071603 conn(0x561084e761c0 msgr2=0x561084f1e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 -- 172.21.15.33:0/4034071603 shutdown_connections 2024-05-08T21:18:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 -- 172.21.15.33:0/4034071603 wait complete. 2024-05-08T21:18:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 -- start start 2024-05-08T21:18:23.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 Processor -- start 2024-05-08T21:18:23.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1c900 0x561084f27890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 0x561084f27df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1dc20 0x561084f2cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084f2d020 con 0x561084f1dc20 2024-05-08T21:18:23.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084f2d190 con 0x561084f1c900 2024-05-08T21:18:23.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561084f2d350 con 0x561084f1d260 2024-05-08T21:18:23.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.636+0000 7fbac7108b80 10 client.0 _ll_register_callbacks cb 0x561084e0dae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbab94eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1dc20 0x561084f2cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbab94eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1dc20 0x561084f2cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50920/0 (socket says 172.21.15.33:50920) 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbab94eb700 1 -- 172.21.15.33:0/991455621 learned_addr learned my addr 172.21.15.33:0/991455621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbab3fff700 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 0x561084f27df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbab8cea700 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1c900 0x561084f27890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbab94eb700 1 -- 172.21.15.33:0/991455621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 msgr2=0x561084f27df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbab94eb700 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 0x561084f27df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbab94eb700 1 -- 172.21.15.33:0/991455621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1c900 msgr2=0x561084f27890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbab94eb700 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1c900 0x561084f27890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbab94eb700 1 -- 172.21.15.33:0/991455621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa000b830 con 0x561084f1dc20 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbab94eb700 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1dc20 0x561084f2cae0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0c9db0 tx=0x7fbaac099c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1503f9871c080b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fba93fff700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fba93fff700 1 -- 172.21.15.33:0/991455621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0092e0 con 0x561084f1dc20 2024-05-08T21:18:23.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fba93fff700 1 -- 172.21.15.33:0/991455621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbaac0192c0 con 0x561084f1dc20 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbac7108b80 1 -- 172.21.15.33:0/991455621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbaa801e040 con 0x561084f1dc20 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fba93fff700 1 -- 172.21.15.33:0/991455621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaac00ce10 con 0x561084f1dc20 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbac7108b80 10 client.15288 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.637+0000 7fbac7108b80 1 -- 172.21.15.33:0/991455621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561084cedc80 con 0x561084f1dc20 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.638+0000 7fba93fff700 1 -- 172.21.15.33:0/991455621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbaa801e040 con 0x561084f1dc20 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.638+0000 7fba93fff700 1 -- 172.21.15.33:0/991455621 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbaac09d150 con 0x561084f1dc20 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.638+0000 7fbac7108b80 10 client.15288 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.638+0000 7fbac7108b80 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.638+0000 7fbac7108b80 20 client.15288 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.638+0000 7fbac7108b80 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561084f8c2b0 0x561084f88c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.638+0000 7fbac7108b80 4 client.15288 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.638+0000 7fbac7108b80 1 -- 172.21.15.33:0/991455621 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561084e60e70 con 0x561084f8c2b0 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.638+0000 7fbab8cea700 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561084f8c2b0 0x561084f88c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:23.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.638+0000 7fba93fff700 10 client.15288 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.638+0000 7fbab8cea700 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561084f8c2b0 0x561084f88c20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a33fd196d9f309 server_cookie=bd068536fa1075fb in_seq=0 out_seq=0 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fba93fff700 1 -- 172.21.15.33:0/991455621 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x561084e60e70 con 0x561084f8c2b0 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fba93fff700 10 client.15288 handle_command_reply: tid=0 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fbac7108b80 1 client.15288 shutdown 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fbac7108b80 1 -- 172.21.15.33:0/991455621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1dc20 msgr2=0x561084f2cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fbac7108b80 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1dc20 0x561084f2cae0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0c9db0 tx=0x7fbaac099c00 comp rx=0 tx=0).stop 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fbac7108b80 1 -- 172.21.15.33:0/991455621 shutdown_connections 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fbac7108b80 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561084f8c2b0 0x561084f88c20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fbac7108b80 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561084f1dc20 0x561084f2cae0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fbac7108b80 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561084f1c900 0x561084f27890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fba93fff700 0 client.15288 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fbac7108b80 1 --2- 172.21.15.33:0/991455621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561084f1d260 0x561084f27df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fbac7108b80 1 -- 172.21.15.33:0/991455621 >> 172.21.15.33:0/991455621 conn(0x561084e761c0 msgr2=0x561084f25320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fbac7108b80 1 -- 172.21.15.33:0/991455621 shutdown_connections 2024-05-08T21:18:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fbac7108b80 1 -- 172.21.15.33:0/991455621 wait complete. 2024-05-08T21:18:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.639+0000 7fbac7108b80 20 client.15288 trim_cache size 0 max 16384 2024-05-08T21:18:23.648 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:18:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-05-08T21:18:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24563, 2024-05-08T21:18:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-08T21:18:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-08T21:18:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-08T21:18:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 80.416706817999994, 2024-05-08T21:18:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-05-08T21:18:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-05-08T21:18:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-05-08T21:18:23.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 86.239535666999998 2024-05-08T21:18:23.649 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:23.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.640+0000 7fbabaf4e700 1 -- 172.21.15.33:0/118079620 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbaa40a11b0 msgr2=0x7fbaa40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.640+0000 7fbabaf4e700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbaa40a11b0 0x7fbaa40a3670 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8006010 tx=0x7fbaa800b5b0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.640+0000 7fbabaf4e700 1 -- 172.21.15.33:0/118079620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab4102980 msgr2=0x7fbab419e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:23.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.640+0000 7fbabaf4e700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab4102980 0x7fbab419e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00c370 tx=0x7fbaac00c680 comp rx=0 tx=0).stop 2024-05-08T21:18:23.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.640+0000 7fbabaf4e700 1 -- 172.21.15.33:0/118079620 shutdown_connections 2024-05-08T21:18:23.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.640+0000 7fbabaf4e700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbab4102980 0x7fbab419e2f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.640+0000 7fbabaf4e700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbab4100e70 0x7fbab4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.640+0000 7fbabaf4e700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbaa40a11b0 0x7fbaa40a3670 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.640+0000 7fbabaf4e700 1 --2- 172.21.15.33:0/118079620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbab4101fc0 0x7fbab41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:23.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.640+0000 7fbabaf4e700 1 -- 172.21.15.33:0/118079620 >> 172.21.15.33:0/118079620 conn(0x7fbab40fc400 msgr2=0x7fbab4105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:23.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.640+0000 7fbabaf4e700 1 -- 172.21.15.33:0/118079620 shutdown_connections 2024-05-08T21:18:23.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:23.640+0000 7fbabaf4e700 1 -- 172.21.15.33:0/118079620 wait complete. 2024-05-08T21:18:23.656 INFO:tasks.cephadm:Setup complete, yielding 2024-05-08T21:18:23.656 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T21:18:23.666 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T21:18:23.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-08T21:18:23.916 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:25.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.038+0000 7fb8eec84700 1 -- 172.21.15.33:0/494827206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8102ba0 msgr2=0x7fb8e8109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:25.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.038+0000 7fb8eec84700 1 --2- 172.21.15.33:0/494827206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8102ba0 0x7fb8e8109450 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000b0d0 tx=0x7fb8e000b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:25.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.038+0000 7fb8eec84700 1 -- 172.21.15.33:0/494827206 shutdown_connections 2024-05-08T21:18:25.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.038+0000 7fb8eec84700 1 --2- 172.21.15.33:0/494827206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8102ba0 0x7fb8e8109450 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:25.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.038+0000 7fb8eec84700 1 --2- 172.21.15.33:0/494827206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e8101000 0x7fb8e8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:25.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.038+0000 7fb8eec84700 1 --2- 172.21.15.33:0/494827206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e8102200 0x7fb8e8102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:25.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.038+0000 7fb8eec84700 1 -- 172.21.15.33:0/494827206 >> 172.21.15.33:0/494827206 conn(0x7fb8e80fc5c0 msgr2=0x7fb8e80fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:25.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.038+0000 7fb8eec84700 1 -- 172.21.15.33:0/494827206 shutdown_connections 2024-05-08T21:18:25.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.038+0000 7fb8eec84700 1 -- 172.21.15.33:0/494827206 wait complete. 2024-05-08T21:18:25.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.039+0000 7fb8eec84700 1 Processor -- start 2024-05-08T21:18:25.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.039+0000 7fb8eec84700 1 -- start start 2024-05-08T21:18:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eec84700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e8101000 0x7fb8e8196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eec84700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8102200 0x7fb8e8197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eec84700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e8102ba0 0x7fb8e819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eec84700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e819ea30 con 0x7fb8e8102200 2024-05-08T21:18:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eec84700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e819eb70 con 0x7fb8e8101000 2024-05-08T21:18:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eec84700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e819ed00 con 0x7fb8e8102ba0 2024-05-08T21:18:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8102200 0x7fb8e8197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8102200 0x7fb8e8197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50948/0 (socket says 172.21.15.33:50948) 2024-05-08T21:18:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eca20700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e8101000 0x7fb8e8196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8e7fff700 1 -- 172.21.15.33:0/3479031282 learned_addr learned my addr 172.21.15.33:0/3479031282 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8ed221700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e8102ba0 0x7fb8e819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eca20700 1 -- 172.21.15.33:0/3479031282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e8102ba0 msgr2=0x7fb8e819e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:25.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eca20700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e8102ba0 0x7fb8e819e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:25.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eca20700 1 -- 172.21.15.33:0/3479031282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8102200 msgr2=0x7fb8e8197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:25.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eca20700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8102200 0x7fb8e8197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:25.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eca20700 1 -- 172.21.15.33:0/3479031282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e0009d00 con 0x7fb8e8101000 2024-05-08T21:18:25.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8e7fff700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8102200 0x7fb8e8197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:25.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.040+0000 7fb8eca20700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e8101000 0x7fb8e8196ef0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00e9c0 tx=0x7fb8dc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:25.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.041+0000 7fb8e5ffb700 1 -- 172.21.15.33:0/3479031282 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc004220 con 0x7fb8e8101000 2024-05-08T21:18:25.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.041+0000 7fb8eec84700 1 -- 172.21.15.33:0/3479031282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e80732f0 con 0x7fb8e8101000 2024-05-08T21:18:25.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.041+0000 7fb8e5ffb700 1 -- 172.21.15.33:0/3479031282 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8dc004380 con 0x7fb8e8101000 2024-05-08T21:18:25.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.041+0000 7fb8e5ffb700 1 -- 172.21.15.33:0/3479031282 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc003b40 con 0x7fb8e8101000 2024-05-08T21:18:25.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.041+0000 7fb8eec84700 1 -- 172.21.15.33:0/3479031282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e8073840 con 0x7fb8e8101000 2024-05-08T21:18:25.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.042+0000 7fb8eec84700 1 -- 172.21.15.33:0/3479031282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e804ec60 con 0x7fb8e8101000 2024-05-08T21:18:25.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.044+0000 7fb8e5ffb700 1 -- 172.21.15.33:0/3479031282 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb8dc003ca0 con 0x7fb8e8101000 2024-05-08T21:18:25.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.044+0000 7fb8e5ffb700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8d80a11b0 0x7fb8d80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:25.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.044+0000 7fb8e5ffb700 1 -- 172.21.15.33:0/3479031282 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb8dc014070 con 0x7fb8e8101000 2024-05-08T21:18:25.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.044+0000 7fb8e7fff700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8d80a11b0 0x7fb8d80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:25.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.045+0000 7fb8e7fff700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8d80a11b0 0x7fb8d80a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4011550 tx=0x7fb8d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.049+0000 7fb8e5ffb700 1 -- 172.21.15.33:0/3479031282 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb8dc096310 con 0x7fb8e8101000 2024-05-08T21:18:25.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.281+0000 7fb8eec84700 1 -- 172.21.15.33:0/3479031282 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8e80613e0 con 0x7fb8d80a11b0 2024-05-08T21:18:25.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.282+0000 7fb8e5ffb700 1 -- 172.21.15.33:0/3479031282 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb8e80613e0 con 0x7fb8d80a11b0 2024-05-08T21:18:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:Backend: cephadm 2024-05-08T21:18:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:Available: Yes 2024-05-08T21:18:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:Paused: No 2024-05-08T21:18:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.284+0000 7fb8eec84700 1 -- 172.21.15.33:0/3479031282 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8d80a11b0 msgr2=0x7fb8d80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.284+0000 7fb8eec84700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8d80a11b0 0x7fb8d80a3670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4011550 tx=0x7fb8d4009260 comp rx=0 tx=0).stop 2024-05-08T21:18:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.284+0000 7fb8eec84700 1 -- 172.21.15.33:0/3479031282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e8101000 msgr2=0x7fb8e8196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.284+0000 7fb8eec84700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e8101000 0x7fb8e8196ef0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00e9c0 tx=0x7fb8dc00ecd0 comp rx=0 tx=0).stop 2024-05-08T21:18:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.284+0000 7fb8eec84700 1 -- 172.21.15.33:0/3479031282 shutdown_connections 2024-05-08T21:18:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.284+0000 7fb8eec84700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e8102ba0 0x7fb8e819e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.284+0000 7fb8eec84700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8d80a11b0 0x7fb8d80a3670 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.284+0000 7fb8eec84700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e8101000 0x7fb8e8196ef0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.284+0000 7fb8eec84700 1 --2- 172.21.15.33:0/3479031282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8102200 0x7fb8e8197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.284+0000 7fb8eec84700 1 -- 172.21.15.33:0/3479031282 >> 172.21.15.33:0/3479031282 conn(0x7fb8e80fc5c0 msgr2=0x7fb8e8105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.284+0000 7fb8eec84700 1 -- 172.21.15.33:0/3479031282 shutdown_connections 2024-05-08T21:18:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:25.284+0000 7fb8eec84700 1 -- 172.21.15.33:0/3479031282 wait complete. 2024-05-08T21:18:25.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:25 smithi071 ceph-mon[47074]: pgmap v271: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:25.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:25 smithi082 ceph-mon[47512]: pgmap v271: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:25.641 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:25 smithi033 ceph-mon[42528]: pgmap v271: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:25.699 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-08T21:18:25.956 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:26.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:26 smithi033 ceph-mon[42528]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:26.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:26 smithi071 ceph-mon[47074]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:26.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:26 smithi082 ceph-mon[47512]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.061+0000 7fb158d13700 1 -- 172.21.15.33:0/3084474211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb154101490 msgr2=0x7fb1541018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.061+0000 7fb158d13700 1 --2- 172.21.15.33:0/3084474211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb154101490 0x7fb1541018b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb144009990 tx=0x7fb144009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.061+0000 7fb158d13700 1 -- 172.21.15.33:0/3084474211 shutdown_connections 2024-05-08T21:18:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.061+0000 7fb158d13700 1 --2- 172.21.15.33:0/3084474211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb154100b70 0x7fb154108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.061+0000 7fb158d13700 1 --2- 172.21.15.33:0/3084474211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb154101490 0x7fb1541018b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.061+0000 7fb158d13700 1 --2- 172.21.15.33:0/3084474211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1541001d0 0x7fb154100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.061+0000 7fb158d13700 1 -- 172.21.15.33:0/3084474211 >> 172.21.15.33:0/3084474211 conn(0x7fb1540fac40 msgr2=0x7fb1540fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.061+0000 7fb158d13700 1 -- 172.21.15.33:0/3084474211 shutdown_connections 2024-05-08T21:18:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.061+0000 7fb158d13700 1 -- 172.21.15.33:0/3084474211 wait complete. 2024-05-08T21:18:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.062+0000 7fb158d13700 1 Processor -- start 2024-05-08T21:18:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.062+0000 7fb158d13700 1 -- start start 2024-05-08T21:18:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.062+0000 7fb158d13700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1541001d0 0x7fb154196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.062+0000 7fb158d13700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb154100b70 0x7fb154197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb158d13700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb154101490 0x7fb15419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb158d13700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15419ea20 con 0x7fb154101490 2024-05-08T21:18:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb158d13700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15419eb60 con 0x7fb154100b70 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb158d13700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15419ecf0 con 0x7fb1541001d0 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb152d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb154101490 0x7fb15419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb152d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb154101490 0x7fb15419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50968/0 (socket says 172.21.15.33:50968) 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb152d9d700 1 -- 172.21.15.33:0/2013613901 learned_addr learned my addr 172.21.15.33:0/2013613901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb15259c700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1541001d0 0x7fb154196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb151d9b700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb154100b70 0x7fb154197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb152d9d700 1 -- 172.21.15.33:0/2013613901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1541001d0 msgr2=0x7fb154196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb152d9d700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1541001d0 0x7fb154196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb152d9d700 1 -- 172.21.15.33:0/2013613901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb154100b70 msgr2=0x7fb154197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb152d9d700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb154100b70 0x7fb154197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb152d9d700 1 -- 172.21.15.33:0/2013613901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1440095f0 con 0x7fb154101490 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb152d9d700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb154101490 0x7fb15419e450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb14c007f10 tx=0x7fb14c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.063+0000 7fb1437fe700 1 -- 172.21.15.33:0/2013613901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c00dd20 con 0x7fb154101490 2024-05-08T21:18:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.064+0000 7fb158d13700 1 -- 172.21.15.33:0/2013613901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15419efd0 con 0x7fb154101490 2024-05-08T21:18:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.064+0000 7fb1437fe700 1 -- 172.21.15.33:0/2013613901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb14c00f040 con 0x7fb154101490 2024-05-08T21:18:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.064+0000 7fb1437fe700 1 -- 172.21.15.33:0/2013613901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c004870 con 0x7fb154101490 2024-05-08T21:18:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.064+0000 7fb158d13700 1 -- 172.21.15.33:0/2013613901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15419f440 con 0x7fb154101490 2024-05-08T21:18:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.065+0000 7fb158d13700 1 -- 172.21.15.33:0/2013613901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15404ec60 con 0x7fb154101490 2024-05-08T21:18:27.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.066+0000 7fb1437fe700 1 -- 172.21.15.33:0/2013613901 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb14c003a00 con 0x7fb154101490 2024-05-08T21:18:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.066+0000 7fb1437fe700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb13c0a1230 0x7fb13c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.067+0000 7fb15259c700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb13c0a1230 0x7fb13c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.067+0000 7fb1437fe700 1 -- 172.21.15.33:0/2013613901 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb14c0cc650 con 0x7fb154101490 2024-05-08T21:18:27.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.067+0000 7fb15259c700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb13c0a1230 0x7fb13c0a36f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb144000c00 tx=0x7fb144005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:27.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.072+0000 7fb1437fe700 1 -- 172.21.15.33:0/2013613901 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb14c095910 con 0x7fb154101490 2024-05-08T21:18:27.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.297+0000 7fb158d13700 1 -- 172.21.15.33:0/2013613901 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1540ff7d0 con 0x7fb13c0a1230 2024-05-08T21:18:27.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.313+0000 7fb1437fe700 1 -- 172.21.15.33:0/2013613901 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fb1540ff7d0 con 0x7fb13c0a1230 2024-05-08T21:18:27.315 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-08T21:18:27.315 INFO:teuthology.orchestra.run.smithi033.stdout:mds.a smithi033 running (92s) 16s ago 91s 17.3M - 18.2.2-1322-gd21e4f96 af23193ff691 04dd9f55aa25 2024-05-08T21:18:27.315 INFO:teuthology.orchestra.run.smithi033.stdout:mds.b smithi071 running (90s) 25s ago 89s 33.5M - 18.2.2-1322-gd21e4f96 af23193ff691 d00127dbd5c0 2024-05-08T21:18:27.315 INFO:teuthology.orchestra.run.smithi033.stdout:mds.c smithi082 running (99s) 25s ago 99s 17.5M - 18.2.2-1322-gd21e4f96 af23193ff691 49995cf0a2b7 2024-05-08T21:18:27.315 INFO:teuthology.orchestra.run.smithi033.stdout:mds.d smithi033 running (104s) 16s ago 104s 19.0M - 18.2.2-1322-gd21e4f96 af23193ff691 9af8667f927d 2024-05-08T21:18:27.315 INFO:teuthology.orchestra.run.smithi033.stdout:mds.e smithi071 running (102s) 25s ago 102s 17.9M - 18.2.2-1322-gd21e4f96 af23193ff691 8256d60fba74 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mds.f smithi082 running (101s) 25s ago 100s 17.7M - 18.2.2-1322-gd21e4f96 af23193ff691 3f16dded9fcf 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mds.g smithi033 running (93s) 16s ago 93s 17.5M - 18.2.2-1322-gd21e4f96 af23193ff691 05246cfe86e6 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mds.h smithi071 running (97s) 25s ago 97s 21.2M - 18.2.2-1322-gd21e4f96 af23193ff691 edc8e8e0bd38 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mds.i smithi082 running (95s) 25s ago 95s 16.5M - 18.2.2-1322-gd21e4f96 af23193ff691 c9f046ccb079 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mds.j smithi033 running (106s) 16s ago 106s 17.5M - 18.2.2-1322-gd21e4f96 af23193ff691 177d48c7fc70 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mds.k smithi071 running (108s) 25s ago 108s 19.8M - 18.2.2-1322-gd21e4f96 af23193ff691 72c661c2ce5f 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mds.l smithi082 running (110s) 25s ago 109s 17.8M - 18.2.2-1322-gd21e4f96 af23193ff691 ab47aa1857f2 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.x smithi033 *:9283,8765 running (7m) 16s ago 7m 531M - 18.2.2-1322-gd21e4f96 af23193ff691 df72257e92a5 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.y smithi071 *:8443,8765 running (5m) 25s ago 5m 461M - 18.2.2-1322-gd21e4f96 af23193ff691 31096a25dedc 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.z smithi082 *:8443,8765 running (5m) 25s ago 5m 461M - 18.2.2-1322-gd21e4f96 af23193ff691 714d0918feb8 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mon.a smithi033 running (7m) 16s ago 7m 90.7M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 5f32ed17a96f 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mon.b smithi071 running (5m) 25s ago 5m 65.2M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 db4306a53ea0 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:mon.c smithi082 running (6m) 25s ago 6m 61.6M 2048M 18.2.2-1322-gd21e4f96 af23193ff691 1e018b7ba582 2024-05-08T21:18:27.316 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (5m) 16s ago 5m 141M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 85e38e5f1da9 2024-05-08T21:18:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi071 running (5m) 25s ago 5m 76.0M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 5a2211a42550 2024-05-08T21:18:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi082 running (4m) 25s ago 4m 149M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 a4792ffcf0c9 2024-05-08T21:18:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi033 running (4m) 16s ago 4m 139M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 0063306a6916 2024-05-08T21:18:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi071 running (4m) 25s ago 4m 63.1M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 bd8a87ee406a 2024-05-08T21:18:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi082 running (3m) 25s ago 3m 141M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 3fdf76c382b2 2024-05-08T21:18:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 smithi033 running (3m) 16s ago 3m 63.4M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 8ba6a32c7def 2024-05-08T21:18:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 smithi071 running (3m) 25s ago 3m 66.9M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 a0fbb19821bf 2024-05-08T21:18:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 smithi082 running (3m) 25s ago 3m 62.5M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 957ea4a9e030 2024-05-08T21:18:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 smithi033 running (2m) 16s ago 2m 59.5M 4279M 18.2.2-1322-gd21e4f96 af23193ff691 8747a50c60e7 2024-05-08T21:18:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 smithi071 running (2m) 25s ago 2m 136M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 494ce2ff1116 2024-05-08T21:18:27.317 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 smithi082 running (2m) 25s ago 2m 55.8M 4278M 18.2.2-1322-gd21e4f96 af23193ff691 f099b035511d 2024-05-08T21:18:27.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.315+0000 7fb158d13700 1 -- 172.21.15.33:0/2013613901 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb13c0a1230 msgr2=0x7fb13c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:27.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.315+0000 7fb158d13700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb13c0a1230 0x7fb13c0a36f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb144000c00 tx=0x7fb144005fb0 comp rx=0 tx=0).stop 2024-05-08T21:18:27.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.315+0000 7fb158d13700 1 -- 172.21.15.33:0/2013613901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb154101490 msgr2=0x7fb15419e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:27.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.315+0000 7fb158d13700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb154101490 0x7fb15419e450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb14c007f10 tx=0x7fb14c00d3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:27.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.315+0000 7fb158d13700 1 -- 172.21.15.33:0/2013613901 shutdown_connections 2024-05-08T21:18:27.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.315+0000 7fb158d13700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb154101490 0x7fb15419e450 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:27.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.315+0000 7fb158d13700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1541001d0 0x7fb154196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.315+0000 7fb158d13700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb13c0a1230 0x7fb13c0a36f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.315+0000 7fb158d13700 1 --2- 172.21.15.33:0/2013613901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb154100b70 0x7fb154197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.315+0000 7fb158d13700 1 -- 172.21.15.33:0/2013613901 >> 172.21.15.33:0/2013613901 conn(0x7fb1540fac40 msgr2=0x7fb1540ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.315+0000 7fb158d13700 1 -- 172.21.15.33:0/2013613901 shutdown_connections 2024-05-08T21:18:27.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:27.315+0000 7fb158d13700 1 -- 172.21.15.33:0/2013613901 wait complete. 2024-05-08T21:18:27.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:27 smithi071 ceph-mon[47074]: pgmap v272: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:27.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:27 smithi082 ceph-mon[47512]: pgmap v272: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:27.699 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:27 smithi033 ceph-mon[42528]: pgmap v272: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:27.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-08T21:18:28.008 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:28.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:28 smithi033 ceph-mon[42528]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:28.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:28 smithi071 ceph-mon[47074]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:28.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:28 smithi082 ceph-mon[47512]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.009+0000 7f0ba2a36700 1 -- 172.21.15.33:0/405846893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b9c1043c0 msgr2=0x7f0b9c1067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.009+0000 7f0ba2a36700 1 --2- 172.21.15.33:0/405846893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b9c1043c0 0x7f0b9c1067b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009990 tx=0x7f0b90009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.009+0000 7f0ba2a36700 1 -- 172.21.15.33:0/405846893 shutdown_connections 2024-05-08T21:18:29.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.009+0000 7f0ba2a36700 1 --2- 172.21.15.33:0/405846893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c106cf0 0x7f0b9c109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:29.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.009+0000 7f0ba2a36700 1 --2- 172.21.15.33:0/405846893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b9c101a90 0x7f0b9c103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:29.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.009+0000 7f0ba2a36700 1 --2- 172.21.15.33:0/405846893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b9c1043c0 0x7f0b9c1067b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:29.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.009+0000 7f0ba2a36700 1 -- 172.21.15.33:0/405846893 >> 172.21.15.33:0/405846893 conn(0x7f0b9c0fb4a0 msgr2=0x7f0b9c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:29.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.009+0000 7f0ba2a36700 1 -- 172.21.15.33:0/405846893 shutdown_connections 2024-05-08T21:18:29.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.010+0000 7f0ba2a36700 1 -- 172.21.15.33:0/405846893 wait complete. 2024-05-08T21:18:29.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.010+0000 7f0ba2a36700 1 Processor -- start 2024-05-08T21:18:29.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.010+0000 7f0ba2a36700 1 -- start start 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0ba2a36700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b9c101a90 0x7f0b9c101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0ba2a36700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b9c1043c0 0x7f0b9c0ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0ba2a36700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c106cf0 0x7f0b9c0fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0ba2a36700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c100660 con 0x7f0b9c1043c0 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0ba2a36700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1007d0 con 0x7f0b9c106cf0 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0ba2a36700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1a0f00 con 0x7f0b9c101a90 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0b9b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b9c1043c0 0x7f0b9c0ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0b9b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b9c1043c0 0x7f0b9c0ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50984/0 (socket says 172.21.15.33:50984) 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0b9b7fe700 1 -- 172.21.15.33:0/3894432056 learned_addr learned my addr 172.21.15.33:0/3894432056 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0b9bfff700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b9c101a90 0x7f0b9c101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0ba0fd3700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c106cf0 0x7f0b9c0fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0b9b7fe700 1 -- 172.21.15.33:0/3894432056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b9c101a90 msgr2=0x7f0b9c101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0b9b7fe700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b9c101a90 0x7f0b9c101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:29.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0b9b7fe700 1 -- 172.21.15.33:0/3894432056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c106cf0 msgr2=0x7f0b9c0fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:29.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0b9b7fe700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c106cf0 0x7f0b9c0fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:29.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0b9b7fe700 1 -- 172.21.15.33:0/3894432056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b900095f0 con 0x7f0b9c1043c0 2024-05-08T21:18:29.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0ba0fd3700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c106cf0 0x7f0b9c0fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:29.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.011+0000 7f0b9b7fe700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b9c1043c0 0x7f0b9c0ffa40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0b90000c00 tx=0x7f0b90003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:29.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.012+0000 7f0b997fa700 1 -- 172.21.15.33:0/3894432056 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9001d070 con 0x7f0b9c1043c0 2024-05-08T21:18:29.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.012+0000 7f0b997fa700 1 -- 172.21.15.33:0/3894432056 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b9000bd70 con 0x7f0b9c1043c0 2024-05-08T21:18:29.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.012+0000 7f0ba2a36700 1 -- 172.21.15.33:0/3894432056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c1a1100 con 0x7f0b9c1043c0 2024-05-08T21:18:29.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.012+0000 7f0b997fa700 1 -- 172.21.15.33:0/3894432056 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90022c60 con 0x7f0b9c1043c0 2024-05-08T21:18:29.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.012+0000 7f0ba2a36700 1 -- 172.21.15.33:0/3894432056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c1a1570 con 0x7f0b9c1043c0 2024-05-08T21:18:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.013+0000 7f0ba2a36700 1 -- 172.21.15.33:0/3894432056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c0fd020 con 0x7f0b9c1043c0 2024-05-08T21:18:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.014+0000 7f0b997fa700 1 -- 172.21.15.33:0/3894432056 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0b90022470 con 0x7f0b9c1043c0 2024-05-08T21:18:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.015+0000 7f0b997fa700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0b840a1230 0x7f0b840a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.015+0000 7f0b9bfff700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0b840a1230 0x7f0b840a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.015+0000 7f0b997fa700 1 -- 172.21.15.33:0/3894432056 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0b900ceaa0 con 0x7f0b9c1043c0 2024-05-08T21:18:29.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.016+0000 7f0b9bfff700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0b840a1230 0x7f0b840a36f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c005950 tx=0x7f0b8c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:29.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.019+0000 7f0b997fa700 1 -- 172.21.15.33:0/3894432056 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0b90097df0 con 0x7f0b9c1043c0 2024-05-08T21:18:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.243+0000 7f0ba2a36700 1 -- 172.21.15.33:0/3894432056 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b9c0613e0 con 0x7f0b840a1230 2024-05-08T21:18:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.250+0000 7f0b997fa700 1 -- 172.21.15.33:0/3894432056 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f0b9c0613e0 con 0x7f0b840a1230 2024-05-08T21:18:29.252 INFO:teuthology.orchestra.run.smithi033.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-08T21:18:29.252 INFO:teuthology.orchestra.run.smithi033.stdout:mds.all 12/12 27s ago 114s smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l;count:12 2024-05-08T21:18:29.252 INFO:teuthology.orchestra.run.smithi033.stdout:mgr 3/3 27s ago 5m smithi033=x;smithi071=y;smithi082=z;count:3 2024-05-08T21:18:29.252 INFO:teuthology.orchestra.run.smithi033.stdout:mon 3/3 27s ago 6m smithi033:172.21.15.33=a;smithi071:172.21.15.71=b;smithi082:172.21.15.82=c;count:3 2024-05-08T21:18:29.252 INFO:teuthology.orchestra.run.smithi033.stdout:osd 12 27s ago - 2024-05-08T21:18:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.252+0000 7f0ba2a36700 1 -- 172.21.15.33:0/3894432056 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0b840a1230 msgr2=0x7f0b840a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.252+0000 7f0ba2a36700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0b840a1230 0x7f0b840a36f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c005950 tx=0x7f0b8c00b410 comp rx=0 tx=0).stop 2024-05-08T21:18:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.252+0000 7f0ba2a36700 1 -- 172.21.15.33:0/3894432056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b9c1043c0 msgr2=0x7f0b9c0ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.252+0000 7f0ba2a36700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b9c1043c0 0x7f0b9c0ffa40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0b90000c00 tx=0x7f0b90003a50 comp rx=0 tx=0).stop 2024-05-08T21:18:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.252+0000 7f0ba2a36700 1 -- 172.21.15.33:0/3894432056 shutdown_connections 2024-05-08T21:18:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.252+0000 7f0ba2a36700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0b840a1230 0x7f0b840a36f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.252+0000 7f0ba2a36700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c106cf0 0x7f0b9c0fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.252+0000 7f0ba2a36700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b9c101a90 0x7f0b9c101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.252+0000 7f0ba2a36700 1 --2- 172.21.15.33:0/3894432056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b9c1043c0 0x7f0b9c0ffa40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.252+0000 7f0ba2a36700 1 -- 172.21.15.33:0/3894432056 >> 172.21.15.33:0/3894432056 conn(0x7f0b9c0fb4a0 msgr2=0x7f0b9c103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.252+0000 7f0ba2a36700 1 -- 172.21.15.33:0/3894432056 shutdown_connections 2024-05-08T21:18:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:29.252+0000 7f0ba2a36700 1 -- 172.21.15.33:0/3894432056 wait complete. 2024-05-08T21:18:29.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:29 smithi071 ceph-mon[47074]: pgmap v273: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:18:29.574 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:29 smithi033 ceph-mon[42528]: pgmap v273: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:18:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:29 smithi082 ceph-mon[47512]: pgmap v273: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:18:29.650 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-08T21:18:29.909 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:30.444 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:30 smithi033 ceph-mon[42528]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:30.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:30 smithi071 ceph-mon[47074]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:30.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:30 smithi082 ceph-mon[47512]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:31.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.064+0000 7feb29b57700 1 -- 172.21.15.33:0/2930543174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb24101000 msgr2=0x7feb24101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:31.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.064+0000 7feb29b57700 1 --2- 172.21.15.33:0/2930543174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb24101000 0x7feb24101420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0099c0 tx=0x7feb0c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.064+0000 7feb29b57700 1 -- 172.21.15.33:0/2930543174 shutdown_connections 2024-05-08T21:18:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.064+0000 7feb29b57700 1 --2- 172.21.15.33:0/2930543174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24102ba0 0x7feb24109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.064+0000 7feb29b57700 1 --2- 172.21.15.33:0/2930543174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb24101000 0x7feb24101420 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.064+0000 7feb29b57700 1 --2- 172.21.15.33:0/2930543174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb24102200 0x7feb24102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.064+0000 7feb29b57700 1 -- 172.21.15.33:0/2930543174 >> 172.21.15.33:0/2930543174 conn(0x7feb240fc5c0 msgr2=0x7feb240fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.064+0000 7feb29b57700 1 -- 172.21.15.33:0/2930543174 shutdown_connections 2024-05-08T21:18:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.064+0000 7feb29b57700 1 -- 172.21.15.33:0/2930543174 wait complete. 2024-05-08T21:18:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.065+0000 7feb29b57700 1 Processor -- start 2024-05-08T21:18:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.065+0000 7feb29b57700 1 -- start start 2024-05-08T21:18:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.065+0000 7feb29b57700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb24101000 0x7feb24196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.065+0000 7feb29b57700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb24102200 0x7feb24197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.065+0000 7feb29b57700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24102ba0 0x7feb2419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.065+0000 7feb29b57700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2419e9f0 con 0x7feb24102200 2024-05-08T21:18:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.065+0000 7feb29b57700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2419eb30 con 0x7feb24102ba0 2024-05-08T21:18:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.065+0000 7feb29b57700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2419ecc0 con 0x7feb24101000 2024-05-08T21:18:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb22ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb24102200 0x7feb24197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb22ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb24102200 0x7feb24197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40252/0 (socket says 172.21.15.33:40252) 2024-05-08T21:18:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb22ffd700 1 -- 172.21.15.33:0/278016370 learned_addr learned my addr 172.21.15.33:0/278016370 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb237fe700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb24101000 0x7feb24196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:31.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb23fff700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24102ba0 0x7feb2419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb22ffd700 1 -- 172.21.15.33:0/278016370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb24101000 msgr2=0x7feb24196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb22ffd700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb24101000 0x7feb24196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb22ffd700 1 -- 172.21.15.33:0/278016370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24102ba0 msgr2=0x7feb2419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb22ffd700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24102ba0 0x7feb2419e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb22ffd700 1 -- 172.21.15.33:0/278016370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c0095f0 con 0x7feb24102200 2024-05-08T21:18:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb22ffd700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb24102200 0x7feb24197440 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7feb1400e9d0 tx=0x7feb1400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb20ff9700 1 -- 172.21.15.33:0/278016370 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb14009790 con 0x7feb24102200 2024-05-08T21:18:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb20ff9700 1 -- 172.21.15.33:0/278016370 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb1400ee70 con 0x7feb24102200 2024-05-08T21:18:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb29b57700 1 -- 172.21.15.33:0/278016370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2419efa0 con 0x7feb24102200 2024-05-08T21:18:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb20ff9700 1 -- 172.21.15.33:0/278016370 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb14018420 con 0x7feb24102200 2024-05-08T21:18:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.066+0000 7feb29b57700 1 -- 172.21.15.33:0/278016370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2419f410 con 0x7feb24102200 2024-05-08T21:18:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.067+0000 7feb29b57700 1 -- 172.21.15.33:0/278016370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb241085d0 con 0x7feb24102200 2024-05-08T21:18:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.069+0000 7feb20ff9700 1 -- 172.21.15.33:0/278016370 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7feb1400cbd0 con 0x7feb24102200 2024-05-08T21:18:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.070+0000 7feb20ff9700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feb100a11b0 0x7feb100a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.070+0000 7feb237fe700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feb100a11b0 0x7feb100a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.070+0000 7feb20ff9700 1 -- 172.21.15.33:0/278016370 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb14014070 con 0x7feb24102200 2024-05-08T21:18:31.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.070+0000 7feb237fe700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feb100a11b0 0x7feb100a3670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feb0c000c00 tx=0x7feb0c003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:31.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.074+0000 7feb20ff9700 1 -- 172.21.15.33:0/278016370 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7feb14096690 con 0x7feb24102200 2024-05-08T21:18:31.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.299+0000 7feb29b57700 1 -- 172.21.15.33:0/278016370 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7feb240613e0 con 0x7feb100a11b0 2024-05-08T21:18:31.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.300+0000 7feb20ff9700 1 -- 172.21.15.33:0/278016370 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+187 (secure 0 0 0) 0x7feb240613e0 con 0x7feb100a11b0 2024-05-08T21:18:31.303 INFO:teuthology.orchestra.run.smithi033.stdout:HOST ADDR LABELS STATUS 2024-05-08T21:18:31.303 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 172.21.15.33 2024-05-08T21:18:31.303 INFO:teuthology.orchestra.run.smithi033.stdout:smithi071 172.21.15.71 2024-05-08T21:18:31.303 INFO:teuthology.orchestra.run.smithi033.stdout:smithi082 172.21.15.82 2024-05-08T21:18:31.303 INFO:teuthology.orchestra.run.smithi033.stdout:3 hosts in cluster 2024-05-08T21:18:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.303+0000 7feb29b57700 1 -- 172.21.15.33:0/278016370 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feb100a11b0 msgr2=0x7feb100a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.303+0000 7feb29b57700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feb100a11b0 0x7feb100a3670 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feb0c000c00 tx=0x7feb0c003680 comp rx=0 tx=0).stop 2024-05-08T21:18:31.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.303+0000 7feb29b57700 1 -- 172.21.15.33:0/278016370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb24102200 msgr2=0x7feb24197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.303+0000 7feb29b57700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb24102200 0x7feb24197440 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7feb1400e9d0 tx=0x7feb1400ece0 comp rx=0 tx=0).stop 2024-05-08T21:18:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.303+0000 7feb29b57700 1 -- 172.21.15.33:0/278016370 shutdown_connections 2024-05-08T21:18:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.303+0000 7feb29b57700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feb100a11b0 0x7feb100a3670 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.303+0000 7feb29b57700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb24102ba0 0x7feb2419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.303+0000 7feb29b57700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb24101000 0x7feb24196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.303+0000 7feb29b57700 1 --2- 172.21.15.33:0/278016370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb24102200 0x7feb24197440 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.303+0000 7feb29b57700 1 -- 172.21.15.33:0/278016370 >> 172.21.15.33:0/278016370 conn(0x7feb240fc5c0 msgr2=0x7feb24105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.303+0000 7feb29b57700 1 -- 172.21.15.33:0/278016370 shutdown_connections 2024-05-08T21:18:31.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:31.303+0000 7feb29b57700 1 -- 172.21.15.33:0/278016370 wait complete. 2024-05-08T21:18:31.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:31 smithi071 ceph-mon[47074]: pgmap v274: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:18:31.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:31 smithi082 ceph-mon[47512]: pgmap v274: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:18:31.648 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:31 smithi033 ceph-mon[42528]: pgmap v274: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:18:31.707 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-08T21:18:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:32.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:32 smithi033 ceph-mon[42528]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:32.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:32 smithi082 ceph-mon[47512]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:32.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:32 smithi071 ceph-mon[47074]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:33.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.062+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3560020753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a81001e0 msgr2=0x7fb6a8100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:33.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.062+0000 7fb6afbaf700 1 --2- 172.21.15.33:0/3560020753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a81001e0 0x7fb6a8100660 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb698009990 tx=0x7fb698009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:33.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.062+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3560020753 shutdown_connections 2024-05-08T21:18:33.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.062+0000 7fb6afbaf700 1 --2- 172.21.15.33:0/3560020753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6a8100ba0 0x7fb6a8101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:33.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.062+0000 7fb6afbaf700 1 --2- 172.21.15.33:0/3560020753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a80ff880 0x7fb6a80ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:33.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.062+0000 7fb6afbaf700 1 --2- 172.21.15.33:0/3560020753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a81001e0 0x7fb6a8100660 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:33.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.062+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3560020753 >> 172.21.15.33:0/3560020753 conn(0x7fb6a80fb4a0 msgr2=0x7fb6a80fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:33.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.062+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3560020753 shutdown_connections 2024-05-08T21:18:33.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.062+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3560020753 wait complete. 2024-05-08T21:18:33.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6afbaf700 1 Processor -- start 2024-05-08T21:18:33.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6afbaf700 1 -- start start 2024-05-08T21:18:33.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6afbaf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a80ff880 0x7fb6a8199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:33.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6afbaf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6a81001e0 0x7fb6a8199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:33.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6afbaf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8100ba0 0x7fb6a81a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:33.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6afbaf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a81a0b90 con 0x7fb6a8100ba0 2024-05-08T21:18:33.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6afbaf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a81a0cd0 con 0x7fb6a81001e0 2024-05-08T21:18:33.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6afbaf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a81a0e60 con 0x7fb6a80ff880 2024-05-08T21:18:33.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6ae14c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8100ba0 0x7fb6a81a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:33.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6ae14c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8100ba0 0x7fb6a81a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40268/0 (socket says 172.21.15.33:40268) 2024-05-08T21:18:33.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6ae14c700 1 -- 172.21.15.33:0/3870479682 learned_addr learned my addr 172.21.15.33:0/3870479682 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6ad14a700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6a81001e0 0x7fb6a8199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6ad94b700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a80ff880 0x7fb6a8199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6ae14c700 1 -- 172.21.15.33:0/3870479682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a80ff880 msgr2=0x7fb6a8199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6ae14c700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a80ff880 0x7fb6a8199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6ae14c700 1 -- 172.21.15.33:0/3870479682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6a81001e0 msgr2=0x7fb6a8199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6ae14c700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6a81001e0 0x7fb6a8199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6ae14c700 1 -- 172.21.15.33:0/3870479682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6980095f0 con 0x7fb6a8100ba0 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.063+0000 7fb6ad94b700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a80ff880 0x7fb6a8199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.064+0000 7fb6ae14c700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8100ba0 0x7fb6a81a05c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000add0 tx=0x7fb6a000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.064+0000 7fb69effd700 1 -- 172.21.15.33:0/3870479682 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a000f070 con 0x7fb6a8100ba0 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.064+0000 7fb69effd700 1 -- 172.21.15.33:0/3870479682 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6a0007b30 con 0x7fb6a8100ba0 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.064+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3870479682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a81a1140 con 0x7fb6a8100ba0 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.064+0000 7fb69effd700 1 -- 172.21.15.33:0/3870479682 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0005280 con 0x7fb6a8100ba0 2024-05-08T21:18:33.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.064+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3870479682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a81a15f0 con 0x7fb6a8100ba0 2024-05-08T21:18:33.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.065+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3870479682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a810a660 con 0x7fb6a8100ba0 2024-05-08T21:18:33.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.066+0000 7fb69effd700 1 -- 172.21.15.33:0/3870479682 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb6a000c940 con 0x7fb6a8100ba0 2024-05-08T21:18:33.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.067+0000 7fb69effd700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb6940a11b0 0x7fb6940a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:33.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.067+0000 7fb6ad94b700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb6940a11b0 0x7fb6940a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:33.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.067+0000 7fb69effd700 1 -- 172.21.15.33:0/3870479682 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb6a00cc980 con 0x7fb6a8100ba0 2024-05-08T21:18:33.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.067+0000 7fb6ad94b700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb6940a11b0 0x7fb6940a3670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4005950 tx=0x7fb6a40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:33.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.071+0000 7fb69effd700 1 -- 172.21.15.33:0/3870479682 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb6a0095d30 con 0x7fb6a8100ba0 2024-05-08T21:18:33.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.293+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3870479682 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6a80613e0 con 0x7fb6940a11b0 2024-05-08T21:18:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.295+0000 7fb69effd700 1 -- 172.21.15.33:0/3870479682 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fb6a80613e0 con 0x7fb6940a11b0 2024-05-08T21:18:33.297 INFO:teuthology.orchestra.run.smithi033.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-08T21:18:33.297 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-08T21:18:33.297 INFO:teuthology.orchestra.run.smithi033.stdout:smithi071 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040172400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-08T21:18:33.297 INFO:teuthology.orchestra.run.smithi033.stdout:smithi082 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400SD400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-05-08T21:18:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.297+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3870479682 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb6940a11b0 msgr2=0x7fb6940a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.297+0000 7fb6afbaf700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb6940a11b0 0x7fb6940a3670 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4005950 tx=0x7fb6a40058e0 comp rx=0 tx=0).stop 2024-05-08T21:18:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.297+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3870479682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8100ba0 msgr2=0x7fb6a81a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.297+0000 7fb6afbaf700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8100ba0 0x7fb6a81a05c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000add0 tx=0x7fb6a000c330 comp rx=0 tx=0).stop 2024-05-08T21:18:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.297+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3870479682 shutdown_connections 2024-05-08T21:18:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.297+0000 7fb6afbaf700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8100ba0 0x7fb6a81a05c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.297+0000 7fb6afbaf700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6a80ff880 0x7fb6a8199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.297+0000 7fb6afbaf700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb6940a11b0 0x7fb6940a3670 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.297+0000 7fb6afbaf700 1 --2- 172.21.15.33:0/3870479682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6a81001e0 0x7fb6a8199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:33.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.297+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3870479682 >> 172.21.15.33:0/3870479682 conn(0x7fb6a80fb4a0 msgr2=0x7fb6a80fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:33.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.297+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3870479682 shutdown_connections 2024-05-08T21:18:33.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:33.297+0000 7fb6afbaf700 1 -- 172.21.15.33:0/3870479682 wait complete. 2024-05-08T21:18:33.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:33 smithi082 ceph-mon[47512]: pgmap v275: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:18:33.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:33 smithi033 ceph-mon[42528]: pgmap v275: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:18:33.724 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T21:18:33.736 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T21:18:33.736 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-08T21:18:33.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:33 smithi071 ceph-mon[47074]: pgmap v275: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:18:33.998 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:34.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:34 smithi033 ceph-mon[42528]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:34.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:34 smithi082 ceph-mon[47512]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:34.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:34 smithi071 ceph-mon[47074]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T21:18:35.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.001+0000 7ff486fc8700 1 -- 172.21.15.33:0/1916246750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff480102200 msgr2=0x7ff480102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:35.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.001+0000 7ff486fc8700 1 --2- 172.21.15.33:0/1916246750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff480102200 0x7ff480102660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0099c0 tx=0x7ff46c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:35.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.001+0000 7ff486fc8700 1 -- 172.21.15.33:0/1916246750 shutdown_connections 2024-05-08T21:18:35.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.001+0000 7ff486fc8700 1 --2- 172.21.15.33:0/1916246750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff480102ba0 0x7ff480109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:35.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.001+0000 7ff486fc8700 1 --2- 172.21.15.33:0/1916246750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff480101000 0x7ff480101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.001+0000 7ff486fc8700 1 --2- 172.21.15.33:0/1916246750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff480102200 0x7ff480102660 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.001+0000 7ff486fc8700 1 -- 172.21.15.33:0/1916246750 >> 172.21.15.33:0/1916246750 conn(0x7ff4800fc5c0 msgr2=0x7ff4800fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.001+0000 7ff486fc8700 1 -- 172.21.15.33:0/1916246750 shutdown_connections 2024-05-08T21:18:35.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.002+0000 7ff486fc8700 1 -- 172.21.15.33:0/1916246750 wait complete. 2024-05-08T21:18:35.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.002+0000 7ff486fc8700 1 Processor -- start 2024-05-08T21:18:35.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.002+0000 7ff486fc8700 1 -- start start 2024-05-08T21:18:35.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.002+0000 7ff486fc8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff480101000 0x7ff480196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:35.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.002+0000 7ff486fc8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff480102200 0x7ff480197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:35.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.002+0000 7ff486fc8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff480102ba0 0x7ff48019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:35.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.002+0000 7ff486fc8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48019e980 con 0x7ff480101000 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.002+0000 7ff486fc8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48019eac0 con 0x7ff480102200 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.002+0000 7ff486fc8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48019ec50 con 0x7ff480102ba0 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff484d64700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff480101000 0x7ff480196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff484d64700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff480101000 0x7ff480196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40288/0 (socket says 172.21.15.33:40288) 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff484d64700 1 -- 172.21.15.33:0/1395215914 learned_addr learned my addr 172.21.15.33:0/1395215914 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff485565700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff480102ba0 0x7ff48019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff47ffff700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff480102200 0x7ff480197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff484d64700 1 -- 172.21.15.33:0/1395215914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff480102ba0 msgr2=0x7ff48019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff484d64700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff480102ba0 0x7ff48019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff484d64700 1 -- 172.21.15.33:0/1395215914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff480102200 msgr2=0x7ff480197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff484d64700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff480102200 0x7ff480197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff484d64700 1 -- 172.21.15.33:0/1395215914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c0095f0 con 0x7ff480101000 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff47ffff700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff480102200 0x7ff480197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff484d64700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff480101000 0x7ff480196e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff47400e9a0 tx=0x7ff47400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:35.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff47dffb700 1 -- 172.21.15.33:0/1395215914 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474009790 con 0x7ff480101000 2024-05-08T21:18:35.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff47dffb700 1 -- 172.21.15.33:0/1395215914 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4740044b0 con 0x7ff480101000 2024-05-08T21:18:35.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff486fc8700 1 -- 172.21.15.33:0/1395215914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48019ef30 con 0x7ff480101000 2024-05-08T21:18:35.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.003+0000 7ff47dffb700 1 -- 172.21.15.33:0/1395215914 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474003c60 con 0x7ff480101000 2024-05-08T21:18:35.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.004+0000 7ff486fc8700 1 -- 172.21.15.33:0/1395215914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48019f340 con 0x7ff480101000 2024-05-08T21:18:35.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.005+0000 7ff486fc8700 1 -- 172.21.15.33:0/1395215914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4801084d0 con 0x7ff480101000 2024-05-08T21:18:35.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.006+0000 7ff47dffb700 1 -- 172.21.15.33:0/1395215914 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff474003dc0 con 0x7ff480101000 2024-05-08T21:18:35.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.006+0000 7ff47dffb700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4700a11b0 0x7ff4700a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:35.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.006+0000 7ff47ffff700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4700a11b0 0x7ff4700a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:35.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.007+0000 7ff47dffb700 1 -- 172.21.15.33:0/1395215914 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff474014070 con 0x7ff480101000 2024-05-08T21:18:35.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.007+0000 7ff47ffff700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4700a11b0 0x7ff4700a3670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00b580 tx=0x7ff46c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:35.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.011+0000 7ff47dffb700 1 -- 172.21.15.33:0/1395215914 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff474096380 con 0x7ff480101000 2024-05-08T21:18:35.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.272+0000 7ff486fc8700 1 -- 172.21.15.33:0/1395215914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff48010c6c0 con 0x7ff480101000 2024-05-08T21:18:35.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.272+0000 7ff47dffb700 1 -- 172.21.15.33:0/1395215914 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+2915 (secure 0 0 0) 0x7ff4740999a0 con 0x7ff480101000 2024-05-08T21:18:35.278 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-05-08T21:18:35.278 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T21:18:35.278 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T21:18:35.278 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-05-08T21:18:35.278 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:35.278 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-05-08T21:18:35.278 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 28 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-05-08T21:17:03.219607+0000 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-05-08T21:17:32.303648+0000 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 300 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-05-08T21:18:35.279 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 5 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1,2,3,4 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=24563,1=24521,2=24499,3=24497,4=24451} 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3,4] 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 0 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b{0:24563} state up:active seq 3 addr [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.f{0:24487} state up:standby-replay seq 1 addr [v2:172.21.15.82:6834/1877231014,v1:172.21.15.82:6835/1877231014] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:18:35.280 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.h{1:24521} state up:active seq 11 addr [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d{1:24472} state up:standby-replay seq 1 addr [v2:172.21.15.33:6836/362093971,v1:172.21.15.33:6837/362093971] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c{2:24499} state up:active seq 12 addr [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.k{2:24461} state up:standby-replay seq 1 addr [v2:172.21.15.71:6832/497942273,v1:172.21.15.71:6833/497942273] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.e{3:24497} state up:active seq 13 addr [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.j{3:24460} state up:standby-replay seq 1 addr [v2:172.21.15.33:6834/82548780,v1:172.21.15.33:6835/82548780] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.l{4:24451} state up:active seq 15 addr [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a{4:14628} state up:standby-replay seq 1 addr [v2:172.21.15.33:6840/2132539478,v1:172.21.15.33:6841/2132539478] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.i{-1:14610} state up:standby seq 1 addr [v2:172.21.15.82:6838/1299124477,v1:172.21.15.82:6839/1299124477] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:18:35.281 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.g{-1:14616} state up:standby seq 1 addr [v2:172.21.15.33:6838/2620722111,v1:172.21.15.33:6839/2620722111] compat {c=[1],r=[1],i=[7ff]}] 2024-05-08T21:18:35.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.275+0000 7ff486fc8700 1 -- 172.21.15.33:0/1395215914 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4700a11b0 msgr2=0x7ff4700a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:35.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.275+0000 7ff486fc8700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4700a11b0 0x7ff4700a3670 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00b580 tx=0x7ff46c01a040 comp rx=0 tx=0).stop 2024-05-08T21:18:35.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.275+0000 7ff486fc8700 1 -- 172.21.15.33:0/1395215914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff480101000 msgr2=0x7ff480196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:35.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.275+0000 7ff486fc8700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff480101000 0x7ff480196e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff47400e9a0 tx=0x7ff47400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:18:35.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.275+0000 7ff486fc8700 1 -- 172.21.15.33:0/1395215914 shutdown_connections 2024-05-08T21:18:35.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.275+0000 7ff486fc8700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff480102ba0 0x7ff48019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.275+0000 7ff486fc8700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff480101000 0x7ff480196e40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.275+0000 7ff486fc8700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4700a11b0 0x7ff4700a3670 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.275+0000 7ff486fc8700 1 --2- 172.21.15.33:0/1395215914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff480102200 0x7ff480197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.275+0000 7ff486fc8700 1 -- 172.21.15.33:0/1395215914 >> 172.21.15.33:0/1395215914 conn(0x7ff4800fc5c0 msgr2=0x7ff480105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.275+0000 7ff486fc8700 1 -- 172.21.15.33:0/1395215914 shutdown_connections 2024-05-08T21:18:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:35.275+0000 7ff486fc8700 1 -- 172.21.15.33:0/1395215914 wait complete. 2024-05-08T21:18:35.283 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-05-08T21:18:35.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:35 smithi082 ceph-mon[47512]: pgmap v276: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:35.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:35 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1395215914' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T21:18:35.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:35 smithi033 ceph-mon[42528]: pgmap v276: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:35.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:35 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1395215914' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T21:18:35.682 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-08T21:18:35.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:35 smithi071 ceph-mon[47074]: pgmap v276: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:35.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:35 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1395215914' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T21:18:35.942 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.032+0000 7f1c30872700 1 -- 172.21.15.33:0/2341035875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c2c106cf0 msgr2=0x7f1c2c109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.032+0000 7f1c30872700 1 --2- 172.21.15.33:0/2341035875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c2c106cf0 0x7f1c2c109160 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400b0a0 tx=0x7f1c2400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.032+0000 7f1c30872700 1 -- 172.21.15.33:0/2341035875 shutdown_connections 2024-05-08T21:18:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.032+0000 7f1c30872700 1 --2- 172.21.15.33:0/2341035875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c2c106cf0 0x7f1c2c109160 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.032+0000 7f1c30872700 1 --2- 172.21.15.33:0/2341035875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c101a90 0x7f1c2c103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.032+0000 7f1c30872700 1 --2- 172.21.15.33:0/2341035875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c2c1043c0 0x7f1c2c1067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.032+0000 7f1c30872700 1 -- 172.21.15.33:0/2341035875 >> 172.21.15.33:0/2341035875 conn(0x7f1c2c0fb4a0 msgr2=0x7f1c2c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:37.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.033+0000 7f1c30872700 1 -- 172.21.15.33:0/2341035875 shutdown_connections 2024-05-08T21:18:37.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.033+0000 7f1c30872700 1 -- 172.21.15.33:0/2341035875 wait complete. 2024-05-08T21:18:37.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.034+0000 7f1c30872700 1 Processor -- start 2024-05-08T21:18:37.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.034+0000 7f1c30872700 1 -- start start 2024-05-08T21:18:37.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.034+0000 7f1c30872700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c101a90 0x7f1c2c196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:37.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.034+0000 7f1c30872700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c2c1043c0 0x7f1c2c1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:37.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.034+0000 7f1c30872700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c2c106cf0 0x7f1c2c19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:37.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.034+0000 7f1c30872700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c19e9e0 con 0x7f1c2c1043c0 2024-05-08T21:18:37.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.034+0000 7f1c30872700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c19eb20 con 0x7f1c2c101a90 2024-05-08T21:18:37.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.034+0000 7f1c30872700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c19ecb0 con 0x7f1c2c106cf0 2024-05-08T21:18:37.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.034+0000 7f1c21d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c2c1043c0 0x7f1c2c1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:37.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.034+0000 7f1c21d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c2c1043c0 0x7f1c2c1973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40316/0 (socket says 172.21.15.33:40316) 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.034+0000 7f1c21d9b700 1 -- 172.21.15.33:0/2813781856 learned_addr learned my addr 172.21.15.33:0/2813781856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.034+0000 7f1c2a59c700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c101a90 0x7f1c2c196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.035+0000 7f1c2ad9d700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c2c106cf0 0x7f1c2c19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.035+0000 7f1c21d9b700 1 -- 172.21.15.33:0/2813781856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c2c106cf0 msgr2=0x7f1c2c19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.035+0000 7f1c21d9b700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c2c106cf0 0x7f1c2c19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.035+0000 7f1c21d9b700 1 -- 172.21.15.33:0/2813781856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c101a90 msgr2=0x7f1c2c196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.035+0000 7f1c21d9b700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c101a90 0x7f1c2c196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.035+0000 7f1c21d9b700 1 -- 172.21.15.33:0/2813781856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c24009d00 con 0x7f1c2c1043c0 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.035+0000 7f1c21d9b700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c2c1043c0 0x7f1c2c1973e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400d9b0 tx=0x7f1c1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.035+0000 7f1c23fff700 1 -- 172.21.15.33:0/2813781856 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14004220 con 0x7f1c2c1043c0 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.035+0000 7f1c30872700 1 -- 172.21.15.33:0/2813781856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2c19ef90 con 0x7f1c2c1043c0 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.035+0000 7f1c23fff700 1 -- 172.21.15.33:0/2813781856 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c14007640 con 0x7f1c2c1043c0 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.035+0000 7f1c23fff700 1 -- 172.21.15.33:0/2813781856 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14005620 con 0x7f1c2c1043c0 2024-05-08T21:18:37.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.035+0000 7f1c30872700 1 -- 172.21.15.33:0/2813781856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c19f400 con 0x7f1c2c1043c0 2024-05-08T21:18:37.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.036+0000 7f1c30872700 1 -- 172.21.15.33:0/2813781856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c0fd020 con 0x7f1c2c1043c0 2024-05-08T21:18:37.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.037+0000 7f1c23fff700 1 -- 172.21.15.33:0/2813781856 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1c14004750 con 0x7f1c2c1043c0 2024-05-08T21:18:37.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.038+0000 7f1c23fff700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1c180a11b0 0x7f1c180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:37.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.038+0000 7f1c2a59c700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1c180a11b0 0x7f1c180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:37.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.038+0000 7f1c23fff700 1 -- 172.21.15.33:0/2813781856 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1c140cd060 con 0x7f1c2c1043c0 2024-05-08T21:18:37.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.039+0000 7f1c2a59c700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1c180a11b0 0x7f1c180a3670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c005950 tx=0x7f1c1c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:37.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.043+0000 7f1c23fff700 1 -- 172.21.15.33:0/2813781856 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1c14096320 con 0x7f1c2c1043c0 2024-05-08T21:18:37.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.271+0000 7f1c30872700 1 -- 172.21.15.33:0/2813781856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f1c2c10c6b0 con 0x7f1c2c1043c0 2024-05-08T21:18:37.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.272+0000 7f1c23fff700 1 -- 172.21.15.33:0/2813781856 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5584 (secure 0 0 0) 0x7f1c14016070 con 0x7f1c2c1043c0 2024-05-08T21:18:37.275 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 87 2024-05-08T21:18:37.275 INFO:teuthology.orchestra.run.smithi033.stdout:fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:18:37.275 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-05-08T21:11:07.208389+0000 2024-05-08T21:18:37.275 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-05-08T21:17:57.410336+0000 2024-05-08T21:18:37.275 INFO:teuthology.orchestra.run.smithi033.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T21:18:37.275 INFO:teuthology.orchestra.run.smithi033.stdout:crush_version 27 2024-05-08T21:18:37.275 INFO:teuthology.orchestra.run.smithi033.stdout:full_ratio 0.95 2024-05-08T21:18:37.275 INFO:teuthology.orchestra.run.smithi033.stdout:backfillfull_ratio 0.9 2024-05-08T21:18:37.275 INFO:teuthology.orchestra.run.smithi033.stdout:nearfull_ratio 0.85 2024-05-08T21:18:37.275 INFO:teuthology.orchestra.run.smithi033.stdout:require_min_compat_client luminous 2024-05-08T21:18:37.275 INFO:teuthology.orchestra.run.smithi033.stdout:min_compat_client luminous 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:require_osd_release reef 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:stretch_mode_enabled false 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:max_osd 12 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6802/1942383393,v1:172.21.15.33:6803/1942383393] [v2:172.21.15.33:6804/1942383393,v1:172.21.15.33:6805/1942383393] exists,up a37bcd68-6698-4832-8f60-78c1e15a7208 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 up in weight 1 up_from 12 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6800/2306412237,v1:172.21.15.71:6801/2306412237] [v2:172.21.15.71:6802/2306412237,v1:172.21.15.71:6803/2306412237] exists,up f789dc73-b4d8-49f3-9a6f-83388b7245da 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 up in weight 1 up_from 17 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6800/3057197540,v1:172.21.15.82:6801/3057197540] [v2:172.21.15.82:6802/3057197540,v1:172.21.15.82:6803/3057197540] exists,up b6c73166-7e6f-41d9-a8ad-713e7ddceb63 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 up in weight 1 up_from 24 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6810/2632370799,v1:172.21.15.33:6811/2632370799] [v2:172.21.15.33:6812/2632370799,v1:172.21.15.33:6813/2632370799] exists,up a1b61b4d-bbc1-448a-856a-5be6034cb981 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 up in weight 1 up_from 31 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6808/4006373764,v1:172.21.15.71:6809/4006373764] [v2:172.21.15.71:6810/4006373764,v1:172.21.15.71:6811/4006373764] exists,up 21a8fee3-514c-4bea-b364-566a8664ee55 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 up in weight 1 up_from 36 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6808/1310343719,v1:172.21.15.82:6809/1310343719] [v2:172.21.15.82:6810/1310343719,v1:172.21.15.82:6811/1310343719] exists,up 7669410f-f67f-41a0-9b5b-3cc04a4ab45b 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 up in weight 1 up_from 44 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6818/3543778506,v1:172.21.15.33:6819/3543778506] [v2:172.21.15.33:6820/3543778506,v1:172.21.15.33:6821/3543778506] exists,up b7ff5abb-f264-461b-8f31-afd52b29bd59 2024-05-08T21:18:37.276 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 up in weight 1 up_from 49 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6816/1213981909,v1:172.21.15.71:6817/1213981909] [v2:172.21.15.71:6818/1213981909,v1:172.21.15.71:6819/1213981909] exists,up c1de3175-dfdd-40ad-a3d3-52a45dec6b2e 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 up in weight 1 up_from 56 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6816/1451701395,v1:172.21.15.82:6817/1451701395] [v2:172.21.15.82:6818/1451701395,v1:172.21.15.82:6819/1451701395] exists,up 4a7620db-cbad-473d-a5fd-ab506474d661 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 up in weight 1 up_from 64 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6826/4123781180,v1:172.21.15.33:6827/4123781180] [v2:172.21.15.33:6828/4123781180,v1:172.21.15.33:6829/4123781180] exists,up 227000e2-dc41-45c2-84f6-4a4345b83328 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 up in weight 1 up_from 69 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6824/3245386576,v1:172.21.15.71:6825/3245386576] [v2:172.21.15.71:6826/3245386576,v1:172.21.15.71:6827/3245386576] exists,up 768308bc-54b7-43ff-af95-87a481d38bb0 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 up in weight 1 up_from 76 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6824/1998522386,v1:172.21.15.82:6825/1998522386] [v2:172.21.15.82:6826/1998522386,v1:172.21.15.82:6827/1998522386] exists,up 760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.4 [2,11] 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.a [2,11] 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.1b [2,4] 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.1f [8,1] 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.36 [4,9] 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.3c [4,9] 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.6 [5,8] 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.12 [9,10] 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.13 [9,8] 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 4.14 [9,8] 2024-05-08T21:18:37.277 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/3156563197 expires 2024-05-09T21:11:56.047045+0000 2024-05-08T21:18:37.278 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/3413344235 expires 2024-05-09T21:11:56.047045+0000 2024-05-08T21:18:37.278 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/3413344235 expires 2024-05-09T21:11:56.047045+0000 2024-05-08T21:18:37.278 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1762147139 expires 2024-05-09T21:11:56.047045+0000 2024-05-08T21:18:37.278 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/2047278949 expires 2024-05-09T21:11:32.156884+0000 2024-05-08T21:18:37.278 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1002276836 expires 2024-05-09T21:11:32.156884+0000 2024-05-08T21:18:37.278 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/3475366293 expires 2024-05-09T21:11:32.156884+0000 2024-05-08T21:18:37.278 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/3475366293 expires 2024-05-09T21:11:32.156884+0000 2024-05-08T21:18:37.278 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1708721255 expires 2024-05-09T21:11:32.156884+0000 2024-05-08T21:18:37.278 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/3864964868 expires 2024-05-09T21:11:56.047045+0000 2024-05-08T21:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.275+0000 7f1c30872700 1 -- 172.21.15.33:0/2813781856 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1c180a11b0 msgr2=0x7f1c180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.275+0000 7f1c30872700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1c180a11b0 0x7f1c180a3670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c005950 tx=0x7f1c1c00b410 comp rx=0 tx=0).stop 2024-05-08T21:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.275+0000 7f1c30872700 1 -- 172.21.15.33:0/2813781856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c2c1043c0 msgr2=0x7f1c2c1973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.275+0000 7f1c30872700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c2c1043c0 0x7f1c2c1973e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400d9b0 tx=0x7f1c1400dcc0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.275+0000 7f1c30872700 1 -- 172.21.15.33:0/2813781856 shutdown_connections 2024-05-08T21:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.275+0000 7f1c30872700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c2c106cf0 0x7f1c2c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.275+0000 7f1c30872700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1c180a11b0 0x7f1c180a3670 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.275+0000 7f1c30872700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c101a90 0x7f1c2c196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.275+0000 7f1c30872700 1 --2- 172.21.15.33:0/2813781856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c2c1043c0 0x7f1c2c1973e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.275+0000 7f1c30872700 1 -- 172.21.15.33:0/2813781856 >> 172.21.15.33:0/2813781856 conn(0x7f1c2c0fb4a0 msgr2=0x7f1c2c1024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.275+0000 7f1c30872700 1 -- 172.21.15.33:0/2813781856 shutdown_connections 2024-05-08T21:18:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.275+0000 7f1c30872700 1 -- 172.21.15.33:0/2813781856 wait complete. 2024-05-08T21:18:37.589 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:37 smithi033 ceph-mon[42528]: pgmap v277: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:37.589 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:37 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2813781856' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T21:18:37.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:37 smithi082 ceph-mon[47512]: pgmap v277: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:37.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:37 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2813781856' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T21:18:37.663 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-08T21:18:37.674 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:18:37.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:37 smithi071 ceph-mon[47074]: pgmap v277: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:37.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:37 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2813781856' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T21:18:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.823+0000 7f15aefb7700 1 -- 172.21.15.33:0/544116897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a8101fd0 msgr2=0x7f15a8102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.823+0000 7f15aefb7700 1 --2- 172.21.15.33:0/544116897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a8101fd0 0x7f15a8102450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1594009990 tx=0x7f1594009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.823+0000 7f15aefb7700 1 -- 172.21.15.33:0/544116897 shutdown_connections 2024-05-08T21:18:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.823+0000 7f15aefb7700 1 --2- 172.21.15.33:0/544116897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8102990 0x7f15a81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.823+0000 7f15aefb7700 1 --2- 172.21.15.33:0/544116897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a8100e80 0x7f15a81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.823+0000 7f15aefb7700 1 --2- 172.21.15.33:0/544116897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a8101fd0 0x7f15a8102450 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.823+0000 7f15aefb7700 1 -- 172.21.15.33:0/544116897 >> 172.21.15.33:0/544116897 conn(0x7f15a80fc410 msgr2=0x7f15a80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.823+0000 7f15aefb7700 1 -- 172.21.15.33:0/544116897 shutdown_connections 2024-05-08T21:18:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.823+0000 7f15aefb7700 1 -- 172.21.15.33:0/544116897 wait complete. 2024-05-08T21:18:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.823+0000 7f15aefb7700 1 Processor -- start 2024-05-08T21:18:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.823+0000 7f15aefb7700 1 -- start start 2024-05-08T21:18:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15aefb7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8100e80 0x7f15a8196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15aefb7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a8101fd0 0x7f15a8197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15aefb7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a8102990 0x7f15a819e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15aefb7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a819e890 con 0x7f15a8101fd0 2024-05-08T21:18:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15aefb7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a819ea00 con 0x7f15a8100e80 2024-05-08T21:18:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15aefb7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a819ebc0 con 0x7f15a8102990 2024-05-08T21:18:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a8101fd0 0x7f15a8197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a8101fd0 0x7f15a8197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40334/0 (socket says 172.21.15.33:40334) 2024-05-08T21:18:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15a7fff700 1 -- 172.21.15.33:0/959017928 learned_addr learned my addr 172.21.15.33:0/959017928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15ad554700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a8102990 0x7f15a819e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15acd53700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8100e80 0x7f15a8196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15a7fff700 1 -- 172.21.15.33:0/959017928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a8102990 msgr2=0x7f15a819e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15a7fff700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a8102990 0x7f15a819e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15a7fff700 1 -- 172.21.15.33:0/959017928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8100e80 msgr2=0x7f15a8196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15a7fff700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8100e80 0x7f15a8196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15a7fff700 1 -- 172.21.15.33:0/959017928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940095f0 con 0x7f15a8101fd0 2024-05-08T21:18:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15a7fff700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a8101fd0 0x7f15a8197330 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f1594005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.824+0000 7f15a5ffb700 1 -- 172.21.15.33:0/959017928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159401d070 con 0x7f15a8101fd0 2024-05-08T21:18:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.825+0000 7f15a5ffb700 1 -- 172.21.15.33:0/959017928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f159400bd10 con 0x7f15a8101fd0 2024-05-08T21:18:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.825+0000 7f15aefb7700 1 -- 172.21.15.33:0/959017928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a819ee40 con 0x7f15a8101fd0 2024-05-08T21:18:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.825+0000 7f15a5ffb700 1 -- 172.21.15.33:0/959017928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1594022b90 con 0x7f15a8101fd0 2024-05-08T21:18:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.825+0000 7f15aefb7700 1 -- 172.21.15.33:0/959017928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a819f1b0 con 0x7f15a8101fd0 2024-05-08T21:18:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.826+0000 7f15aefb7700 1 -- 172.21.15.33:0/959017928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a81083b0 con 0x7f15a8101fd0 2024-05-08T21:18:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.827+0000 7f15a5ffb700 1 -- 172.21.15.33:0/959017928 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f1594003e20 con 0x7f15a8101fd0 2024-05-08T21:18:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.828+0000 7f15a5ffb700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15980a11b0 0x7f15980a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.828+0000 7f15acd53700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15980a11b0 0x7f15980a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.828+0000 7f15a5ffb700 1 -- 172.21.15.33:0/959017928 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f15940ce610 con 0x7f15a8101fd0 2024-05-08T21:18:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.828+0000 7f15acd53700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15980a11b0 0x7f15980a3670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15a8198220 tx=0x7f159c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:37.832+0000 7f15a5ffb700 1 -- 172.21.15.33:0/959017928 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1594097960 con 0x7f15a8101fd0 2024-05-08T21:18:38.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.051+0000 7f15aefb7700 1 -- 172.21.15.33:0/959017928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15a8067480 con 0x7f15a8101fd0 2024-05-08T21:18:38.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.053+0000 7f15a5ffb700 1 -- 172.21.15.33:0/959017928 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24374 (secure 0 0 0) 0x7f159409af80 con 0x7f15a8101fd0 2024-05-08T21:18:38.056 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:38.056 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:57.410336+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:18:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.056+0000 7f15aefb7700 1 -- 172.21.15.33:0/959017928 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15980a11b0 msgr2=0x7f15980a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.056+0000 7f15aefb7700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15980a11b0 0x7f15980a3670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15a8198220 tx=0x7f159c0073d0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.056+0000 7f15aefb7700 1 -- 172.21.15.33:0/959017928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a8101fd0 msgr2=0x7f15a8197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.056+0000 7f15aefb7700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a8101fd0 0x7f15a8197330 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f1594005b00 comp rx=0 tx=0).stop 2024-05-08T21:18:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.056+0000 7f15aefb7700 1 -- 172.21.15.33:0/959017928 shutdown_connections 2024-05-08T21:18:38.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.056+0000 7f15aefb7700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a8102990 0x7f15a819e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.056+0000 7f15aefb7700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15980a11b0 0x7f15980a3670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.056+0000 7f15aefb7700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8100e80 0x7f15a8196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.056+0000 7f15aefb7700 1 --2- 172.21.15.33:0/959017928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a8101fd0 0x7f15a8197330 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.056+0000 7f15aefb7700 1 -- 172.21.15.33:0/959017928 >> 172.21.15.33:0/959017928 conn(0x7f15a80fc410 msgr2=0x7f15a81054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:38.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.056+0000 7f15aefb7700 1 -- 172.21.15.33:0/959017928 shutdown_connections 2024-05-08T21:18:38.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.056+0000 7f15aefb7700 1 -- 172.21.15.33:0/959017928 wait complete. 2024-05-08T21:18:38.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:17:01.282803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:17:02.555552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T21:17:05.686277+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T21:18:38.073 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:18:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.211+0000 7f275d414700 1 -- 172.21.15.33:0/1248942349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27581024f0 msgr2=0x7f2758102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.211+0000 7f275d414700 1 --2- 172.21.15.33:0/1248942349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27581024f0 0x7f2758102910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27480098f0 tx=0x7f2748009c00 comp rx=0 tx=0).stop 2024-05-08T21:18:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.211+0000 7f275d414700 1 -- 172.21.15.33:0/1248942349 shutdown_connections 2024-05-08T21:18:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.211+0000 7f275d414700 1 --2- 172.21.15.33:0/1248942349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27580ff900 0x7f27581094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.211+0000 7f275d414700 1 --2- 172.21.15.33:0/1248942349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27581024f0 0x7f2758102910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.211+0000 7f275d414700 1 --2- 172.21.15.33:0/1248942349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27580fef60 0x7f27580ff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.211+0000 7f275d414700 1 -- 172.21.15.33:0/1248942349 >> 172.21.15.33:0/1248942349 conn(0x7f27580faae0 msgr2=0x7f27580fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.211+0000 7f275d414700 1 -- 172.21.15.33:0/1248942349 shutdown_connections 2024-05-08T21:18:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.211+0000 7f275d414700 1 -- 172.21.15.33:0/1248942349 wait complete. 2024-05-08T21:18:38.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f275d414700 1 Processor -- start 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f275d414700 1 -- start start 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f275d414700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27580fef60 0x7f2758101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f275d414700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27580ff900 0x7f2758100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f275d414700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27581024f0 0x7f2758100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f275d414700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2758100f10 con 0x7f27580fef60 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f275d414700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2758101080 con 0x7f27581024f0 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f275d414700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27581a0df0 con 0x7f27580ff900 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f2756ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27580fef60 0x7f2758101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f2756ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27580fef60 0x7f2758101ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40346/0 (socket says 172.21.15.33:40346) 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f2756ffd700 1 -- 172.21.15.33:0/3232078832 learned_addr learned my addr 172.21.15.33:0/3232078832 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f27567fc700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27580ff900 0x7f2758100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.212+0000 7f27577fe700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27581024f0 0x7f2758100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.213+0000 7f2756ffd700 1 -- 172.21.15.33:0/3232078832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27580ff900 msgr2=0x7f2758100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.213+0000 7f2756ffd700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27580ff900 0x7f2758100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.213+0000 7f2756ffd700 1 -- 172.21.15.33:0/3232078832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27581024f0 msgr2=0x7f2758100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.213+0000 7f2756ffd700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27581024f0 0x7f2758100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.213+0000 7f2756ffd700 1 -- 172.21.15.33:0/3232078832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2750009d00 con 0x7f27580fef60 2024-05-08T21:18:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.213+0000 7f27567fc700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27580ff900 0x7f2758100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.213+0000 7f2756ffd700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27580fef60 0x7f2758101ba0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2748009810 tx=0x7f274800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.213+0000 7f273ffff700 1 -- 172.21.15.33:0/3232078832 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274801c070 con 0x7f27580fef60 2024-05-08T21:18:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.213+0000 7f273ffff700 1 -- 172.21.15.33:0/3232078832 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f274800f7d0 con 0x7f27580fef60 2024-05-08T21:18:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.213+0000 7f275d414700 1 -- 172.21.15.33:0/3232078832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2748009520 con 0x7f27580fef60 2024-05-08T21:18:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.213+0000 7f273ffff700 1 -- 172.21.15.33:0/3232078832 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748003970 con 0x7f27580fef60 2024-05-08T21:18:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.213+0000 7f275d414700 1 -- 172.21.15.33:0/3232078832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27581a1250 con 0x7f27580fef60 2024-05-08T21:18:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.214+0000 7f275d414700 1 -- 172.21.15.33:0/3232078832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2758108650 con 0x7f27580fef60 2024-05-08T21:18:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.215+0000 7f273ffff700 1 -- 172.21.15.33:0/3232078832 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2748020030 con 0x7f27580fef60 2024-05-08T21:18:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.216+0000 7f273ffff700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f27400a11b0 0x7f27400a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.216+0000 7f273ffff700 1 -- 172.21.15.33:0/3232078832 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f27480ce220 con 0x7f27580fef60 2024-05-08T21:18:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.216+0000 7f27567fc700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f27400a11b0 0x7f27400a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:38.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.217+0000 7f27567fc700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f27400a11b0 0x7f27400a3670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f274c005950 tx=0x7f274c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.220+0000 7f273ffff700 1 -- 172.21.15.33:0/3232078832 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2748097570 con 0x7f27580fef60 2024-05-08T21:18:38.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.438+0000 7f275d414700 1 -- 172.21.15.33:0/3232078832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f275810ceb0 con 0x7f27580fef60 2024-05-08T21:18:38.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.439+0000 7f273ffff700 1 -- 172.21.15.33:0/3232078832 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f274809ab90 con 0x7f27580fef60 2024-05-08T21:18:38.441 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:18:38.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.441+0000 7f275d414700 1 -- 172.21.15.33:0/3232078832 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f27400a11b0 msgr2=0x7f27400a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.441+0000 7f275d414700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f27400a11b0 0x7f27400a3670 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f274c005950 tx=0x7f274c00b410 comp rx=0 tx=0).stop 2024-05-08T21:18:38.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.441+0000 7f275d414700 1 -- 172.21.15.33:0/3232078832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27580fef60 msgr2=0x7f2758101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.441+0000 7f275d414700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27580fef60 0x7f2758101ba0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2748009810 tx=0x7f274800b790 comp rx=0 tx=0).stop 2024-05-08T21:18:38.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.441+0000 7f275d414700 1 -- 172.21.15.33:0/3232078832 shutdown_connections 2024-05-08T21:18:38.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.441+0000 7f275d414700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f27400a11b0 0x7f27400a3670 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.441+0000 7f275d414700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27581024f0 0x7f2758100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.441+0000 7f275d414700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27580fef60 0x7f2758101ba0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.441+0000 7f275d414700 1 --2- 172.21.15.33:0/3232078832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27580ff900 0x7f2758100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.441+0000 7f275d414700 1 -- 172.21.15.33:0/3232078832 >> 172.21.15.33:0/3232078832 conn(0x7f27580faae0 msgr2=0x7f2758105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:38.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.441+0000 7f275d414700 1 -- 172.21.15.33:0/3232078832 shutdown_connections 2024-05-08T21:18:38.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.441+0000 7f275d414700 1 -- 172.21.15.33:0/3232078832 wait complete. 2024-05-08T21:18:38.457 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:18:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.595+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/58618047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0106b80 msgr2=0x7f0bf0108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.595+0000 7f0bf7ca8700 1 --2- 172.21.15.33:0/58618047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0106b80 0x7f0bf0108fb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0be800b0a0 tx=0x7f0be800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.595+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/58618047 shutdown_connections 2024-05-08T21:18:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.595+0000 7f0bf7ca8700 1 --2- 172.21.15.33:0/58618047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0106b80 0x7f0bf0108fb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.595+0000 7f0bf7ca8700 1 --2- 172.21.15.33:0/58618047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bf0101920 0x7f0bf0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.595+0000 7f0bf7ca8700 1 --2- 172.21.15.33:0/58618047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bf0104250 0x7f0bf0106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.595+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/58618047 >> 172.21.15.33:0/58618047 conn(0x7f0bf00fb290 msgr2=0x7f0bf00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.596+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/58618047 shutdown_connections 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.596+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/58618047 wait complete. 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.596+0000 7f0bf7ca8700 1 Processor -- start 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.596+0000 7f0bf7ca8700 1 -- start start 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.596+0000 7f0bf7ca8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bf0101920 0x7f0bf0196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.596+0000 7f0bf7ca8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0104250 0x7f0bf0197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.596+0000 7f0bf7ca8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bf0106b80 0x7f0bf019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.596+0000 7f0bf7ca8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf019e8e0 con 0x7f0bf0104250 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.596+0000 7f0bf7ca8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf019ea50 con 0x7f0bf0106b80 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.596+0000 7f0bf7ca8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf019ec10 con 0x7f0bf0101920 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf5243700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0104250 0x7f0bf0197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf5243700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0104250 0x7f0bf0197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40360/0 (socket says 172.21.15.33:40360) 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf5243700 1 -- 172.21.15.33:0/3771511010 learned_addr learned my addr 172.21.15.33:0/3771511010 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf5a44700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bf0101920 0x7f0bf0196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf6245700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bf0106b80 0x7f0bf019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf5243700 1 -- 172.21.15.33:0/3771511010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bf0101920 msgr2=0x7f0bf0196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf5243700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bf0101920 0x7f0bf0196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf5243700 1 -- 172.21.15.33:0/3771511010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bf0106b80 msgr2=0x7f0bf019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf5243700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bf0106b80 0x7f0bf019e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf5243700 1 -- 172.21.15.33:0/3771511010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec009520 con 0x7f0bf0104250 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf5243700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0104250 0x7f0bf0197380 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0be000e9a0 tx=0x7f0be000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0be6ffd700 1 -- 172.21.15.33:0/3771511010 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0009790 con 0x7f0bf0104250 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0be6ffd700 1 -- 172.21.15.33:0/3771511010 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0be000cbd0 con 0x7f0bf0104250 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/3771511010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be8009d00 con 0x7f0bf0104250 2024-05-08T21:18:38.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0be6ffd700 1 -- 172.21.15.33:0/3771511010 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0018680 con 0x7f0bf0104250 2024-05-08T21:18:38.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.597+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/3771511010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf019f200 con 0x7f0bf0104250 2024-05-08T21:18:38.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:38 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/959017928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.598+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/3771511010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf00fce10 con 0x7f0bf0104250 2024-05-08T21:18:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.600+0000 7f0be6ffd700 1 -- 172.21.15.33:0/3771511010 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0be000cd40 con 0x7f0bf0104250 2024-05-08T21:18:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.600+0000 7f0be6ffd700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0bdc0a1230 0x7f0bdc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.600+0000 7f0bf5a44700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0bdc0a1230 0x7f0bdc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.600+0000 7f0be6ffd700 1 -- 172.21.15.33:0/3771511010 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0be0014070 con 0x7f0bf0104250 2024-05-08T21:18:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.601+0000 7f0bf5a44700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0bdc0a1230 0x7f0bdc0a36f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0198270 tx=0x7f0bec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:38.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.604+0000 7f0be6ffd700 1 -- 172.21.15.33:0/3771511010 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0be0096640 con 0x7f0bf0104250 2024-05-08T21:18:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:38 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/959017928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:38 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/959017928' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.825+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/3771511010 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0bf010c680 con 0x7f0bf0104250 2024-05-08T21:18:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.825+0000 7f0be6ffd700 1 -- 172.21.15.33:0/3771511010 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0be00260a0 con 0x7f0bf0104250 2024-05-08T21:18:38.828 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:38.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.828+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/3771511010 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0bdc0a1230 msgr2=0x7f0bdc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.828+0000 7f0bf7ca8700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0bdc0a1230 0x7f0bdc0a36f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0198270 tx=0x7f0bec009260 comp rx=0 tx=0).stop 2024-05-08T21:18:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.828+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/3771511010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0104250 msgr2=0x7f0bf0197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.828+0000 7f0bf7ca8700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0104250 0x7f0bf0197380 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0be000e9a0 tx=0x7f0be000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.828+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/3771511010 shutdown_connections 2024-05-08T21:18:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.828+0000 7f0bf7ca8700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0bdc0a1230 0x7f0bdc0a36f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.828+0000 7f0bf7ca8700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bf0106b80 0x7f0bf019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.828+0000 7f0bf7ca8700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bf0101920 0x7f0bf0196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.828+0000 7f0bf7ca8700 1 --2- 172.21.15.33:0/3771511010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf0104250 0x7f0bf0197380 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.828+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/3771511010 >> 172.21.15.33:0/3771511010 conn(0x7f0bf00fb290 msgr2=0x7f0bf01023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.828+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/3771511010 shutdown_connections 2024-05-08T21:18:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.828+0000 7f0bf7ca8700 1 -- 172.21.15.33:0/3771511010 wait complete. 2024-05-08T21:18:38.844 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:18:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.984+0000 7f8720b11700 1 -- 172.21.15.33:0/2399446236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f871c101ff0 msgr2=0x7f871c102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.984+0000 7f8720b11700 1 --2- 172.21.15.33:0/2399446236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f871c101ff0 0x7f871c102470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8710009990 tx=0x7f8710009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.984+0000 7f8720b11700 1 -- 172.21.15.33:0/2399446236 shutdown_connections 2024-05-08T21:18:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.984+0000 7f8720b11700 1 --2- 172.21.15.33:0/2399446236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f871c1029b0 0x7f871c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.984+0000 7f8720b11700 1 --2- 172.21.15.33:0/2399446236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f871c100ea0 0x7f871c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.984+0000 7f8720b11700 1 --2- 172.21.15.33:0/2399446236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f871c101ff0 0x7f871c102470 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.984+0000 7f8720b11700 1 -- 172.21.15.33:0/2399446236 >> 172.21.15.33:0/2399446236 conn(0x7f871c0fc430 msgr2=0x7f871c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.984+0000 7f8720b11700 1 -- 172.21.15.33:0/2399446236 shutdown_connections 2024-05-08T21:18:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.984+0000 7f8720b11700 1 -- 172.21.15.33:0/2399446236 wait complete. 2024-05-08T21:18:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f8720b11700 1 Processor -- start 2024-05-08T21:18:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f8720b11700 1 -- start start 2024-05-08T21:18:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f8720b11700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f871c100ea0 0x7f871c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f8720b11700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f871c101ff0 0x7f871c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f8720b11700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f871c1029b0 0x7f871c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f8720b11700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c074070 con 0x7f871c100ea0 2024-05-08T21:18:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f8720b11700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c0741e0 con 0x7f871c1029b0 2024-05-08T21:18:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f8720b11700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1a0d50 con 0x7f871c101ff0 2024-05-08T21:18:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f871a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f871c100ea0 0x7f871c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:38.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f871a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f871c100ea0 0x7f871c074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40376/0 (socket says 172.21.15.33:40376) 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f871a59c700 1 -- 172.21.15.33:0/3717786176 learned_addr learned my addr 172.21.15.33:0/3717786176 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f8719d9b700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f871c101ff0 0x7f871c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f871ad9d700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f871c1029b0 0x7f871c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f871a59c700 1 -- 172.21.15.33:0/3717786176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f871c101ff0 msgr2=0x7f871c073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f871a59c700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f871c101ff0 0x7f871c073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f871a59c700 1 -- 172.21.15.33:0/3717786176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f871c1029b0 msgr2=0x7f871c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f871a59c700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f871c1029b0 0x7f871c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.985+0000 7f871a59c700 1 -- 172.21.15.33:0/3717786176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8714009c40 con 0x7f871c100ea0 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.986+0000 7f871a59c700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f871c100ea0 0x7f871c074d00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f870c00e9a0 tx=0x7f870c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.986+0000 7f870b7fe700 1 -- 172.21.15.33:0/3717786176 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c009790 con 0x7f871c100ea0 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.986+0000 7f870b7fe700 1 -- 172.21.15.33:0/3717786176 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f870c010430 con 0x7f871c100ea0 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.986+0000 7f8720b11700 1 -- 172.21.15.33:0/3717786176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87100095f0 con 0x7f871c100ea0 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.986+0000 7f870b7fe700 1 -- 172.21.15.33:0/3717786176 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c018680 con 0x7f871c100ea0 2024-05-08T21:18:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.986+0000 7f8720b11700 1 -- 172.21.15.33:0/3717786176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c1a1270 con 0x7f871c100ea0 2024-05-08T21:18:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.987+0000 7f8720b11700 1 -- 172.21.15.33:0/3717786176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c108350 con 0x7f871c100ea0 2024-05-08T21:18:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.988+0000 7f870b7fe700 1 -- 172.21.15.33:0/3717786176 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f870c0105a0 con 0x7f871c100ea0 2024-05-08T21:18:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.989+0000 7f870b7fe700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f87040a11b0 0x7f87040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.989+0000 7f870b7fe700 1 -- 172.21.15.33:0/3717786176 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f870c014070 con 0x7f871c100ea0 2024-05-08T21:18:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.989+0000 7f8719d9b700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f87040a11b0 0x7f87040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.989+0000 7f8719d9b700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f87040a11b0 0x7f87040a3670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f871000b610 tx=0x7f871000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:38.993+0000 7f870b7fe700 1 -- 172.21.15.33:0/3717786176 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f870c096620 con 0x7f871c100ea0 2024-05-08T21:18:39.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.211+0000 7f8720b11700 1 -- 172.21.15.33:0/3717786176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f871c10c580 con 0x7f871c100ea0 2024-05-08T21:18:39.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.211+0000 7f870b7fe700 1 -- 172.21.15.33:0/3717786176 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f870c099c40 con 0x7f871c100ea0 2024-05-08T21:18:39.214 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T21:18:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.213+0000 7f8720b11700 1 -- 172.21.15.33:0/3717786176 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f87040a11b0 msgr2=0x7f87040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.213+0000 7f8720b11700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f87040a11b0 0x7f87040a3670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f871000b610 tx=0x7f871000b260 comp rx=0 tx=0).stop 2024-05-08T21:18:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.213+0000 7f8720b11700 1 -- 172.21.15.33:0/3717786176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f871c100ea0 msgr2=0x7f871c074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.213+0000 7f8720b11700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f871c100ea0 0x7f871c074d00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f870c00e9a0 tx=0x7f870c00ed60 comp rx=0 tx=0).stop 2024-05-08T21:18:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.214+0000 7f8720b11700 1 -- 172.21.15.33:0/3717786176 shutdown_connections 2024-05-08T21:18:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.214+0000 7f8720b11700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f87040a11b0 0x7f87040a3670 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.214+0000 7f8720b11700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f871c1029b0 0x7f871c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.214+0000 7f8720b11700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f871c100ea0 0x7f871c074d00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.214+0000 7f8720b11700 1 --2- 172.21.15.33:0/3717786176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f871c101ff0 0x7f871c073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.214+0000 7f8720b11700 1 -- 172.21.15.33:0/3717786176 >> 172.21.15.33:0/3717786176 conn(0x7f871c0fc430 msgr2=0x7f871c105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:39.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.214+0000 7f8720b11700 1 -- 172.21.15.33:0/3717786176 shutdown_connections 2024-05-08T21:18:39.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.214+0000 7f8720b11700 1 -- 172.21.15.33:0/3717786176 wait complete. 2024-05-08T21:18:39.230 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T21:18:39.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.372+0000 7f9d223ae700 1 -- 172.21.15.33:0/2796749507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d1c101910 msgr2=0x7f9d1c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:39.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.372+0000 7f9d223ae700 1 --2- 172.21.15.33:0/2796749507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d1c101910 0x7f9d1c103d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009990 tx=0x7f9d0c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.372+0000 7f9d223ae700 1 -- 172.21.15.33:0/2796749507 shutdown_connections 2024-05-08T21:18:39.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.372+0000 7f9d223ae700 1 --2- 172.21.15.33:0/2796749507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d1c106b70 0x7f9d1c108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.372+0000 7f9d223ae700 1 --2- 172.21.15.33:0/2796749507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d1c101910 0x7f9d1c103d00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.372+0000 7f9d223ae700 1 --2- 172.21.15.33:0/2796749507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d1c104240 0x7f9d1c106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.372+0000 7f9d223ae700 1 -- 172.21.15.33:0/2796749507 >> 172.21.15.33:0/2796749507 conn(0x7f9d1c0fb300 msgr2=0x7f9d1c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:39.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.372+0000 7f9d223ae700 1 -- 172.21.15.33:0/2796749507 shutdown_connections 2024-05-08T21:18:39.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.372+0000 7f9d223ae700 1 -- 172.21.15.33:0/2796749507 wait complete. 2024-05-08T21:18:39.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.373+0000 7f9d223ae700 1 Processor -- start 2024-05-08T21:18:39.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.373+0000 7f9d223ae700 1 -- start start 2024-05-08T21:18:39.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.373+0000 7f9d223ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d1c101910 0x7f9d1c196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:39.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.373+0000 7f9d223ae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d1c104240 0x7f9d1c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:39.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.373+0000 7f9d223ae700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d1c106b70 0x7f9d1c19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:39.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.373+0000 7f9d223ae700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c19e830 con 0x7f9d1c101910 2024-05-08T21:18:39.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.373+0000 7f9d223ae700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c19e9a0 con 0x7f9d1c106b70 2024-05-08T21:18:39.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.373+0000 7f9d223ae700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c19eb60 con 0x7f9d1c104240 2024-05-08T21:18:39.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d1bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d1c101910 0x7f9d1c196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d1bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d1c101910 0x7f9d1c196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40388/0 (socket says 172.21.15.33:40388) 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d1bfff700 1 -- 172.21.15.33:0/2819890954 learned_addr learned my addr 172.21.15.33:0/2819890954 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d1b7fe700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d1c104240 0x7f9d1c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d2094b700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d1c106b70 0x7f9d1c19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d1bfff700 1 -- 172.21.15.33:0/2819890954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d1c104240 msgr2=0x7f9d1c197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d1bfff700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d1c104240 0x7f9d1c197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d1bfff700 1 -- 172.21.15.33:0/2819890954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d1c106b70 msgr2=0x7f9d1c19e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d1bfff700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d1c106b70 0x7f9d1c19e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d1bfff700 1 -- 172.21.15.33:0/2819890954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c0095f0 con 0x7f9d1c101910 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d1bfff700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d1c101910 0x7f9d1c196d40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00b2e0 tx=0x7f9d0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d197fa700 1 -- 172.21.15.33:0/2819890954 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c01d070 con 0x7f9d1c101910 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d197fa700 1 -- 172.21.15.33:0/2819890954 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d0c00bdb0 con 0x7f9d1c101910 2024-05-08T21:18:39.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d223ae700 1 -- 172.21.15.33:0/2819890954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c19ede0 con 0x7f9d1c101910 2024-05-08T21:18:39.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d197fa700 1 -- 172.21.15.33:0/2819890954 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00fdc0 con 0x7f9d1c101910 2024-05-08T21:18:39.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.374+0000 7f9d223ae700 1 -- 172.21.15.33:0/2819890954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c19f1b0 con 0x7f9d1c101910 2024-05-08T21:18:39.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.375+0000 7f9d223ae700 1 -- 172.21.15.33:0/2819890954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c0fce80 con 0x7f9d1c101910 2024-05-08T21:18:39.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.377+0000 7f9d197fa700 1 -- 172.21.15.33:0/2819890954 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9d0c00f560 con 0x7f9d1c101910 2024-05-08T21:18:39.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.377+0000 7f9d197fa700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9d040a11b0 0x7f9d040a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:39.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.377+0000 7f9d197fa700 1 -- 172.21.15.33:0/2819890954 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9d0c0ce600 con 0x7f9d1c101910 2024-05-08T21:18:39.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.377+0000 7f9d1b7fe700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9d040a11b0 0x7f9d040a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:39.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.378+0000 7f9d1b7fe700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9d040a11b0 0x7f9d040a3670 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009c60 tx=0x7f9d10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:39.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.382+0000 7f9d197fa700 1 -- 172.21.15.33:0/2819890954 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9d0c097ad0 con 0x7f9d1c101910 2024-05-08T21:18:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.598+0000 7f9d223ae700 1 -- 172.21.15.33:0/2819890954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9d1c10c6a0 con 0x7f9d1c101910 2024-05-08T21:18:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.599+0000 7f9d197fa700 1 -- 172.21.15.33:0/2819890954 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9d0c026020 con 0x7f9d1c101910 2024-05-08T21:18:39.601 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:39.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:39 smithi082 ceph-mon[47512]: pgmap v278: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:18:39.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:39 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3232078832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:39.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:39 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3771511010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:39.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:39 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3717786176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:39.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.601+0000 7f9d223ae700 1 -- 172.21.15.33:0/2819890954 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9d040a11b0 msgr2=0x7f9d040a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:39.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.601+0000 7f9d223ae700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9d040a11b0 0x7f9d040a3670 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009c60 tx=0x7f9d10009260 comp rx=0 tx=0).stop 2024-05-08T21:18:39.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.601+0000 7f9d223ae700 1 -- 172.21.15.33:0/2819890954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d1c101910 msgr2=0x7f9d1c196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:39.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.601+0000 7f9d223ae700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d1c101910 0x7f9d1c196d40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00b2e0 tx=0x7f9d0c004730 comp rx=0 tx=0).stop 2024-05-08T21:18:39.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.601+0000 7f9d223ae700 1 -- 172.21.15.33:0/2819890954 shutdown_connections 2024-05-08T21:18:39.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.601+0000 7f9d223ae700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9d040a11b0 0x7f9d040a3670 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.601+0000 7f9d223ae700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d1c106b70 0x7f9d1c19e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.601+0000 7f9d223ae700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d1c101910 0x7f9d1c196d40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.601+0000 7f9d223ae700 1 --2- 172.21.15.33:0/2819890954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d1c104240 0x7f9d1c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.601+0000 7f9d223ae700 1 -- 172.21.15.33:0/2819890954 >> 172.21.15.33:0/2819890954 conn(0x7f9d1c0fb300 msgr2=0x7f9d1c103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:39.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.601+0000 7f9d223ae700 1 -- 172.21.15.33:0/2819890954 shutdown_connections 2024-05-08T21:18:39.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.601+0000 7f9d223ae700 1 -- 172.21.15.33:0/2819890954 wait complete. 2024-05-08T21:18:39.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:39 smithi033 ceph-mon[42528]: pgmap v278: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:18:39.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:39 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3232078832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:39.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:39 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3771511010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:39.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:39 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3717786176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:39.617 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:18:39.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.753+0000 7fe672612700 1 -- 172.21.15.33:0/184186594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe66c106b60 msgr2=0x7fe66c108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.753+0000 7fe672612700 1 --2- 172.21.15.33:0/184186594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe66c106b60 0x7fe66c108fd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe66400b0a0 tx=0x7fe66400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.753+0000 7fe672612700 1 -- 172.21.15.33:0/184186594 shutdown_connections 2024-05-08T21:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.753+0000 7fe672612700 1 --2- 172.21.15.33:0/184186594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe66c106b60 0x7fe66c108fd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.753+0000 7fe672612700 1 --2- 172.21.15.33:0/184186594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe66c101900 0x7fe66c103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.753+0000 7fe672612700 1 --2- 172.21.15.33:0/184186594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe66c104230 0x7fe66c106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.753+0000 7fe672612700 1 -- 172.21.15.33:0/184186594 >> 172.21.15.33:0/184186594 conn(0x7fe66c0fb310 msgr2=0x7fe66c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.753+0000 7fe672612700 1 -- 172.21.15.33:0/184186594 shutdown_connections 2024-05-08T21:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.753+0000 7fe672612700 1 -- 172.21.15.33:0/184186594 wait complete. 2024-05-08T21:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.753+0000 7fe672612700 1 Processor -- start 2024-05-08T21:18:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.753+0000 7fe672612700 1 -- start start 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe672612700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe66c101900 0x7fe66c196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe672612700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe66c104230 0x7fe66c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe672612700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe66c106b60 0x7fe66c19e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe672612700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c19e870 con 0x7fe66c106b60 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe672612700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c19e9e0 con 0x7fe66c104230 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe672612700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c19eba0 con 0x7fe66c101900 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe670baf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe66c106b60 0x7fe66c19e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe670baf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe66c106b60 0x7fe66c19e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40400/0 (socket says 172.21.15.33:40400) 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe670baf700 1 -- 172.21.15.33:0/364522131 learned_addr learned my addr 172.21.15.33:0/364522131 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe66bfff700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe66c101900 0x7fe66c196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe66b7fe700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe66c104230 0x7fe66c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe670baf700 1 -- 172.21.15.33:0/364522131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe66c101900 msgr2=0x7fe66c196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe670baf700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe66c101900 0x7fe66c196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe670baf700 1 -- 172.21.15.33:0/364522131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe66c104230 msgr2=0x7fe66c1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe670baf700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe66c104230 0x7fe66c1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe670baf700 1 -- 172.21.15.33:0/364522131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe660009520 con 0x7fe66c106b60 2024-05-08T21:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.754+0000 7fe66b7fe700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe66c104230 0x7fe66c1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.755+0000 7fe670baf700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe66c106b60 0x7fe66c19e2a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe66400b780 tx=0x7fe66400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.755+0000 7fe6697fa700 1 -- 172.21.15.33:0/364522131 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66400e040 con 0x7fe66c106b60 2024-05-08T21:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.755+0000 7fe6697fa700 1 -- 172.21.15.33:0/364522131 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe664003c00 con 0x7fe66c106b60 2024-05-08T21:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.755+0000 7fe672612700 1 -- 172.21.15.33:0/364522131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe664009d00 con 0x7fe66c106b60 2024-05-08T21:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.755+0000 7fe6697fa700 1 -- 172.21.15.33:0/364522131 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6640052a0 con 0x7fe66c106b60 2024-05-08T21:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.755+0000 7fe672612700 1 -- 172.21.15.33:0/364522131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c19f0b0 con 0x7fe66c106b60 2024-05-08T21:18:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.756+0000 7fe672612700 1 -- 172.21.15.33:0/364522131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66c0fce90 con 0x7fe66c106b60 2024-05-08T21:18:39.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.757+0000 7fe6697fa700 1 -- 172.21.15.33:0/364522131 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe664005400 con 0x7fe66c106b60 2024-05-08T21:18:39.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.758+0000 7fe6697fa700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6540a11b0 0x7fe6540a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:39.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.758+0000 7fe6697fa700 1 -- 172.21.15.33:0/364522131 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe6640ce160 con 0x7fe66c106b60 2024-05-08T21:18:39.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.758+0000 7fe66bfff700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6540a11b0 0x7fe6540a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:39.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.758+0000 7fe66bfff700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6540a11b0 0x7fe6540a3670 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe66c198200 tx=0x7fe65c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:39.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:39.762+0000 7fe6697fa700 1 -- 172.21.15.33:0/364522131 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe664097510 con 0x7fe66c106b60 2024-05-08T21:18:39.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:39 smithi071 ceph-mon[47074]: pgmap v278: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:18:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:39 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3232078832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:39 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3771511010' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:39 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3717786176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:40.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.033+0000 7fe672612700 1 -- 172.21.15.33:0/364522131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe66c067d10 con 0x7fe66c106b60 2024-05-08T21:18:40.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.034+0000 7fe6697fa700 1 -- 172.21.15.33:0/364522131 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10899 (secure 0 0 0) 0x7fe66409ab30 con 0x7fe66c106b60 2024-05-08T21:18:40.036 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:40.037 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:17:32.303648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:18:40.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.036+0000 7fe672612700 1 -- 172.21.15.33:0/364522131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6540a11b0 msgr2=0x7fe6540a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.036+0000 7fe672612700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6540a11b0 0x7fe6540a3670 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe66c198200 tx=0x7fe65c009260 comp rx=0 tx=0).stop 2024-05-08T21:18:40.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.036+0000 7fe672612700 1 -- 172.21.15.33:0/364522131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe66c106b60 msgr2=0x7fe66c19e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.036+0000 7fe672612700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe66c106b60 0x7fe66c19e2a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe66400b780 tx=0x7fe66400bab0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.037+0000 7fe672612700 1 -- 172.21.15.33:0/364522131 shutdown_connections 2024-05-08T21:18:40.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.037+0000 7fe672612700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe66c106b60 0x7fe66c19e2a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.037+0000 7fe672612700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe66c101900 0x7fe66c196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.037+0000 7fe672612700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6540a11b0 0x7fe6540a3670 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.037+0000 7fe672612700 1 --2- 172.21.15.33:0/364522131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe66c104230 0x7fe66c1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.037+0000 7fe672612700 1 -- 172.21.15.33:0/364522131 >> 172.21.15.33:0/364522131 conn(0x7fe66c0fb310 msgr2=0x7fe66c102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:40.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.037+0000 7fe672612700 1 -- 172.21.15.33:0/364522131 shutdown_connections 2024-05-08T21:18:40.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.037+0000 7fe672612700 1 -- 172.21.15.33:0/364522131 wait complete. 2024-05-08T21:18:40.040 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-05-08T21:18:40.053 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:18:40.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.188+0000 7f7c83798700 1 -- 172.21.15.33:0/2063483325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c101910 msgr2=0x7f7c7c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.188+0000 7f7c83798700 1 --2- 172.21.15.33:0/2063483325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c101910 0x7f7c7c1081c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400b0d0 tx=0x7f7c7400b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.188+0000 7f7c83798700 1 -- 172.21.15.33:0/2063483325 shutdown_connections 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.188+0000 7f7c83798700 1 --2- 172.21.15.33:0/2063483325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c101910 0x7f7c7c1081c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.188+0000 7f7c83798700 1 --2- 172.21.15.33:0/2063483325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c7c0ffd80 0x7f7c7c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.188+0000 7f7c83798700 1 --2- 172.21.15.33:0/2063483325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c7c100f70 0x7f7c7c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.188+0000 7f7c83798700 1 -- 172.21.15.33:0/2063483325 >> 172.21.15.33:0/2063483325 conn(0x7f7c7c0fb310 msgr2=0x7f7c7c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.188+0000 7f7c83798700 1 -- 172.21.15.33:0/2063483325 shutdown_connections 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.188+0000 7f7c83798700 1 -- 172.21.15.33:0/2063483325 wait complete. 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c83798700 1 Processor -- start 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c83798700 1 -- start start 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c83798700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c7c0ffd80 0x7f7c7c103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c83798700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c7c100f70 0x7f7c7c102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c83798700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c101910 0x7f7c7c102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c83798700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c103030 con 0x7f7c7c101910 2024-05-08T21:18:40.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c83798700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c1031a0 con 0x7f7c7c100f70 2024-05-08T21:18:40.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c83798700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c1a0e60 con 0x7f7c7c0ffd80 2024-05-08T21:18:40.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c81d35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c101910 0x7f7c7c102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:40.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c81d35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c101910 0x7f7c7c102890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40408/0 (socket says 172.21.15.33:40408) 2024-05-08T21:18:40.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c81d35700 1 -- 172.21.15.33:0/665127043 learned_addr learned my addr 172.21.15.33:0/665127043 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:40.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c81534700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c7c0ffd80 0x7f7c7c103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:40.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c80d33700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c7c100f70 0x7f7c7c102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:40.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c81d35700 1 -- 172.21.15.33:0/665127043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c7c0ffd80 msgr2=0x7f7c7c103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c81d35700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c7c0ffd80 0x7f7c7c103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c81d35700 1 -- 172.21.15.33:0/665127043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c7c100f70 msgr2=0x7f7c7c102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c81d35700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c7c100f70 0x7f7c7c102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.189+0000 7f7c81d35700 1 -- 172.21.15.33:0/665127043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c74009d00 con 0x7f7c7c101910 2024-05-08T21:18:40.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.190+0000 7f7c81d35700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c101910 0x7f7c7c102890 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7c74007af0 tx=0x7f7c74007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:40.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.190+0000 7f7c727fc700 1 -- 172.21.15.33:0/665127043 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7400e040 con 0x7f7c7c101910 2024-05-08T21:18:40.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.190+0000 7f7c727fc700 1 -- 172.21.15.33:0/665127043 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c7400be30 con 0x7f7c7c101910 2024-05-08T21:18:40.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.190+0000 7f7c83798700 1 -- 172.21.15.33:0/665127043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7c1a1090 con 0x7f7c7c101910 2024-05-08T21:18:40.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.190+0000 7f7c727fc700 1 -- 172.21.15.33:0/665127043 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7401b9f0 con 0x7f7c7c101910 2024-05-08T21:18:40.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.190+0000 7f7c83798700 1 -- 172.21.15.33:0/665127043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7c1a14b0 con 0x7f7c7c101910 2024-05-08T21:18:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.191+0000 7f7c83798700 1 -- 172.21.15.33:0/665127043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c7c067d10 con 0x7f7c7c101910 2024-05-08T21:18:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.192+0000 7f7c727fc700 1 -- 172.21.15.33:0/665127043 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7c7401d670 con 0x7f7c7c101910 2024-05-08T21:18:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.193+0000 7f7c727fc700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c680a11b0 0x7f7c680a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.193+0000 7f7c81534700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c680a11b0 0x7f7c680a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.193+0000 7f7c727fc700 1 -- 172.21.15.33:0/665127043 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7c740cf170 con 0x7f7c7c101910 2024-05-08T21:18:40.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.193+0000 7f7c81534700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c680a11b0 0x7f7c680a3670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c100dd0 tx=0x7f7c7800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:40.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.197+0000 7f7c727fc700 1 -- 172.21.15.33:0/665127043 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7c74098520 con 0x7f7c7c101910 2024-05-08T21:18:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.417+0000 7f7c83798700 1 -- 172.21.15.33:0/665127043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c7c10b610 con 0x7f7c7c101910 2024-05-08T21:18:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.419+0000 7f7c727fc700 1 -- 172.21.15.33:0/665127043 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24374 (secure 0 0 0) 0x7f7c7401d920 con 0x7f7c7c101910 2024-05-08T21:18:40.422 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:40.422 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:57.410336+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:18:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.421+0000 7f7c83798700 1 -- 172.21.15.33:0/665127043 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c680a11b0 msgr2=0x7f7c680a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.421+0000 7f7c83798700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c680a11b0 0x7f7c680a3670 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c100dd0 tx=0x7f7c7800b410 comp rx=0 tx=0).stop 2024-05-08T21:18:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.421+0000 7f7c83798700 1 -- 172.21.15.33:0/665127043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c101910 msgr2=0x7f7c7c102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.421+0000 7f7c83798700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c101910 0x7f7c7c102890 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7c74007af0 tx=0x7f7c74007850 comp rx=0 tx=0).stop 2024-05-08T21:18:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.421+0000 7f7c83798700 1 -- 172.21.15.33:0/665127043 shutdown_connections 2024-05-08T21:18:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.421+0000 7f7c83798700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c101910 0x7f7c7c102890 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.421+0000 7f7c83798700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c7c0ffd80 0x7f7c7c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.421+0000 7f7c83798700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c680a11b0 0x7f7c680a3670 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.422+0000 7f7c83798700 1 --2- 172.21.15.33:0/665127043 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c7c100f70 0x7f7c7c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.422+0000 7f7c83798700 1 -- 172.21.15.33:0/665127043 >> 172.21.15.33:0/665127043 conn(0x7f7c7c0fb310 msgr2=0x7f7c7c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.422+0000 7f7c83798700 1 -- 172.21.15.33:0/665127043 shutdown_connections 2024-05-08T21:18:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.422+0000 7f7c83798700 1 -- 172.21.15.33:0/665127043 wait complete. 2024-05-08T21:18:40.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:17:01.282803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:17:02.555552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T21:17:05.686277+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T21:18:40.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:18:40.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.578+0000 7f4547725700 1 -- 172.21.15.33:0/789820900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4540101910 msgr2=0x7f4540103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.578+0000 7f4547725700 1 --2- 172.21.15.33:0/789820900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4540101910 0x7f4540103d00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f453c00b0a0 tx=0x7f453c00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.578+0000 7f4547725700 1 -- 172.21.15.33:0/789820900 shutdown_connections 2024-05-08T21:18:40.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.578+0000 7f4547725700 1 --2- 172.21.15.33:0/789820900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4540106b70 0x7f4540108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.578+0000 7f4547725700 1 --2- 172.21.15.33:0/789820900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4540101910 0x7f4540103d00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.578+0000 7f4547725700 1 --2- 172.21.15.33:0/789820900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4540104240 0x7f4540106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.578+0000 7f4547725700 1 -- 172.21.15.33:0/789820900 >> 172.21.15.33:0/789820900 conn(0x7f45400fb300 msgr2=0x7f45400fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:40.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.578+0000 7f4547725700 1 -- 172.21.15.33:0/789820900 shutdown_connections 2024-05-08T21:18:40.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.578+0000 7f4547725700 1 -- 172.21.15.33:0/789820900 wait complete. 2024-05-08T21:18:40.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f4547725700 1 Processor -- start 2024-05-08T21:18:40.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f4547725700 1 -- start start 2024-05-08T21:18:40.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f4547725700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4540101910 0x7f4540196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:40.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f4547725700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4540104240 0x7f4540197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:40.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f4547725700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4540106b70 0x7f454019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:40.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f4547725700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454019e8e0 con 0x7f4540106b70 2024-05-08T21:18:40.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f4547725700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454019ea50 con 0x7f4540104240 2024-05-08T21:18:40.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f4547725700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454019ec10 con 0x7f4540101910 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f4545cc2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4540106b70 0x7f454019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f4545cc2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4540106b70 0x7f454019e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54522/0 (socket says 172.21.15.33:54522) 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f4545cc2700 1 -- 172.21.15.33:0/3861179759 learned_addr learned my addr 172.21.15.33:0/3861179759 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f45454c1700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4540101910 0x7f4540196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.579+0000 7f4544cc0700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4540104240 0x7f4540197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.580+0000 7f4545cc2700 1 -- 172.21.15.33:0/3861179759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4540101910 msgr2=0x7f4540196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.580+0000 7f4545cc2700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4540101910 0x7f4540196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.580+0000 7f4545cc2700 1 -- 172.21.15.33:0/3861179759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4540104240 msgr2=0x7f4540197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.580+0000 7f4545cc2700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4540104240 0x7f4540197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.580+0000 7f4545cc2700 1 -- 172.21.15.33:0/3861179759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c009d00 con 0x7f4540106b70 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.580+0000 7f45454c1700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4540101910 0x7f4540196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.580+0000 7f4545cc2700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4540106b70 0x7f454019e310 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f453800c900 tx=0x7f453800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.580+0000 7f45367fc700 1 -- 172.21.15.33:0/3861179759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538007b00 con 0x7f4540106b70 2024-05-08T21:18:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.580+0000 7f45367fc700 1 -- 172.21.15.33:0/3861179759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4538007c60 con 0x7f4540106b70 2024-05-08T21:18:40.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.580+0000 7f45367fc700 1 -- 172.21.15.33:0/3861179759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538018660 con 0x7f4540106b70 2024-05-08T21:18:40.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.580+0000 7f4547725700 1 -- 172.21.15.33:0/3861179759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454019eef0 con 0x7f4540106b70 2024-05-08T21:18:40.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.580+0000 7f4547725700 1 -- 172.21.15.33:0/3861179759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454019f2c0 con 0x7f4540106b70 2024-05-08T21:18:40.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.581+0000 7f4547725700 1 -- 172.21.15.33:0/3861179759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45400fce80 con 0x7f4540106b70 2024-05-08T21:18:40.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.582+0000 7f45367fc700 1 -- 172.21.15.33:0/3861179759 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f453800f450 con 0x7f4540106b70 2024-05-08T21:18:40.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.583+0000 7f45367fc700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f452c0a11b0 0x7f452c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:40.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.583+0000 7f45454c1700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f452c0a11b0 0x7f452c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:40.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.583+0000 7f45367fc700 1 -- 172.21.15.33:0/3861179759 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f45380ccdd0 con 0x7f4540106b70 2024-05-08T21:18:40.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.584+0000 7f45454c1700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f452c0a11b0 0x7f452c0a3670 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f453c009fd0 tx=0x7f453c009c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:40.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.587+0000 7f45367fc700 1 -- 172.21.15.33:0/3861179759 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f45380960f0 con 0x7f4540106b70 2024-05-08T21:18:40.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:40 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2819890954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:40.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:40 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/364522131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:40.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:40 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2819890954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:40.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:40 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/364522131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:40.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.805+0000 7f4547725700 1 -- 172.21.15.33:0/3861179759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f454010c520 con 0x7f4540106b70 2024-05-08T21:18:40.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.805+0000 7f45367fc700 1 -- 172.21.15.33:0/3861179759 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f453800fe60 con 0x7f4540106b70 2024-05-08T21:18:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:18:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.808+0000 7f4547725700 1 -- 172.21.15.33:0/3861179759 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f452c0a11b0 msgr2=0x7f452c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.808+0000 7f4547725700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f452c0a11b0 0x7f452c0a3670 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f453c009fd0 tx=0x7f453c009c60 comp rx=0 tx=0).stop 2024-05-08T21:18:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.808+0000 7f4547725700 1 -- 172.21.15.33:0/3861179759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4540106b70 msgr2=0x7f454019e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.808+0000 7f4547725700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4540106b70 0x7f454019e310 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f453800c900 tx=0x7f453800ccc0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.808+0000 7f4547725700 1 -- 172.21.15.33:0/3861179759 shutdown_connections 2024-05-08T21:18:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.808+0000 7f4547725700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4540106b70 0x7f454019e310 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.808+0000 7f4547725700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4540101910 0x7f4540196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.808+0000 7f4547725700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f452c0a11b0 0x7f452c0a3670 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.808+0000 7f4547725700 1 --2- 172.21.15.33:0/3861179759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4540104240 0x7f4540197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.808+0000 7f4547725700 1 -- 172.21.15.33:0/3861179759 >> 172.21.15.33:0/3861179759 conn(0x7f45400fb300 msgr2=0x7f4540102370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:40.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.808+0000 7f4547725700 1 -- 172.21.15.33:0/3861179759 shutdown_connections 2024-05-08T21:18:40.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.808+0000 7f4547725700 1 -- 172.21.15.33:0/3861179759 wait complete. 2024-05-08T21:18:40.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:40 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2819890954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:40.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:40 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/364522131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:40.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:18:40.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.962+0000 7f78d3315700 1 -- 172.21.15.33:0/123848194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78cc1018e0 msgr2=0x7f78cc103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.962+0000 7f78d3315700 1 --2- 172.21.15.33:0/123848194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78cc1018e0 0x7f78cc103cd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009990 tx=0x7f78bc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.962+0000 7f78d3315700 1 -- 172.21.15.33:0/123848194 shutdown_connections 2024-05-08T21:18:40.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.962+0000 7f78d3315700 1 --2- 172.21.15.33:0/123848194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78cc106b40 0x7f78cc108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.962+0000 7f78d3315700 1 --2- 172.21.15.33:0/123848194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78cc1018e0 0x7f78cc103cd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.962+0000 7f78d3315700 1 --2- 172.21.15.33:0/123848194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78cc104210 0x7f78cc106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.962+0000 7f78d3315700 1 -- 172.21.15.33:0/123848194 >> 172.21.15.33:0/123848194 conn(0x7f78cc0fb290 msgr2=0x7f78cc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.962+0000 7f78d3315700 1 -- 172.21.15.33:0/123848194 shutdown_connections 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.963+0000 7f78d3315700 1 -- 172.21.15.33:0/123848194 wait complete. 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.963+0000 7f78d3315700 1 Processor -- start 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.963+0000 7f78d3315700 1 -- start start 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.963+0000 7f78d3315700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78cc1018e0 0x7f78cc101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.963+0000 7f78d3315700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78cc104210 0x7f78cc0ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.963+0000 7f78d3315700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78cc106b40 0x7f78cc0ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.963+0000 7f78d3315700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc100570 con 0x7f78cc1018e0 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.963+0000 7f78d3315700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1006e0 con 0x7f78cc104210 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.963+0000 7f78d3315700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1a0d30 con 0x7f78cc106b40 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d10b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78cc1018e0 0x7f78cc101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d10b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78cc1018e0 0x7f78cc101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54530/0 (socket says 172.21.15.33:54530) 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d10b1700 1 -- 172.21.15.33:0/4240399228 learned_addr learned my addr 172.21.15.33:0/4240399228 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d18b2700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78cc106b40 0x7f78cc0ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d08b0700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78cc104210 0x7f78cc0ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d10b1700 1 -- 172.21.15.33:0/4240399228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78cc106b40 msgr2=0x7f78cc0ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d10b1700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78cc106b40 0x7f78cc0ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d10b1700 1 -- 172.21.15.33:0/4240399228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78cc104210 msgr2=0x7f78cc0ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d10b1700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78cc104210 0x7f78cc0ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d10b1700 1 -- 172.21.15.33:0/4240399228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc0095f0 con 0x7f78cc1018e0 2024-05-08T21:18:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d10b1700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78cc1018e0 0x7f78cc101200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f78bc003ec0 tx=0x7f78bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78c27fc700 1 -- 172.21.15.33:0/4240399228 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc01d070 con 0x7f78cc1018e0 2024-05-08T21:18:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78c27fc700 1 -- 172.21.15.33:0/4240399228 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f78bc00bdb0 con 0x7f78cc1018e0 2024-05-08T21:18:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d3315700 1 -- 172.21.15.33:0/4240399228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78cc1a0f60 con 0x7f78cc1018e0 2024-05-08T21:18:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78c27fc700 1 -- 172.21.15.33:0/4240399228 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc00fda0 con 0x7f78cc1018e0 2024-05-08T21:18:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.964+0000 7f78d3315700 1 -- 172.21.15.33:0/4240399228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc1a12d0 con 0x7f78cc1018e0 2024-05-08T21:18:40.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.965+0000 7f78d3315700 1 -- 172.21.15.33:0/4240399228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78cc0fce10 con 0x7f78cc1018e0 2024-05-08T21:18:40.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.966+0000 7f78c27fc700 1 -- 172.21.15.33:0/4240399228 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f78bc00f460 con 0x7f78cc1018e0 2024-05-08T21:18:40.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.967+0000 7f78c27fc700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f78b80a11b0 0x7f78b80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:40.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.968+0000 7f78d08b0700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f78b80a11b0 0x7f78b80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:40.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.968+0000 7f78c27fc700 1 -- 172.21.15.33:0/4240399228 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f78bc0ce4d0 con 0x7f78cc1018e0 2024-05-08T21:18:40.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.968+0000 7f78d08b0700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f78b80a11b0 0x7f78b80a3670 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009bf0 tx=0x7f78c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:40.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:40.972+0000 7f78c27fc700 1 -- 172.21.15.33:0/4240399228 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f78bc097820 con 0x7f78cc1018e0 2024-05-08T21:18:41.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.191+0000 7f78d3315700 1 -- 172.21.15.33:0/4240399228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78cc10c4f0 con 0x7f78cc1018e0 2024-05-08T21:18:41.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.192+0000 7f78c27fc700 1 -- 172.21.15.33:0/4240399228 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78bc09ae40 con 0x7f78cc1018e0 2024-05-08T21:18:41.194 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:41.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.194+0000 7f78d3315700 1 -- 172.21.15.33:0/4240399228 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f78b80a11b0 msgr2=0x7f78b80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:41.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.194+0000 7f78d3315700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f78b80a11b0 0x7f78b80a3670 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009bf0 tx=0x7f78c8009260 comp rx=0 tx=0).stop 2024-05-08T21:18:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.194+0000 7f78d3315700 1 -- 172.21.15.33:0/4240399228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78cc1018e0 msgr2=0x7f78cc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.194+0000 7f78d3315700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78cc1018e0 0x7f78cc101200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f78bc003ec0 tx=0x7f78bc004730 comp rx=0 tx=0).stop 2024-05-08T21:18:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.194+0000 7f78d3315700 1 -- 172.21.15.33:0/4240399228 shutdown_connections 2024-05-08T21:18:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.194+0000 7f78d3315700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78cc106b40 0x7f78cc0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.194+0000 7f78d3315700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78cc1018e0 0x7f78cc101200 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.194+0000 7f78d3315700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f78b80a11b0 0x7f78b80a3670 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.194+0000 7f78d3315700 1 --2- 172.21.15.33:0/4240399228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78cc104210 0x7f78cc0ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.194+0000 7f78d3315700 1 -- 172.21.15.33:0/4240399228 >> 172.21.15.33:0/4240399228 conn(0x7f78cc0fb290 msgr2=0x7f78cc102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.194+0000 7f78d3315700 1 -- 172.21.15.33:0/4240399228 shutdown_connections 2024-05-08T21:18:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.194+0000 7f78d3315700 1 -- 172.21.15.33:0/4240399228 wait complete. 2024-05-08T21:18:41.211 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:18:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.351+0000 7fe79575f700 1 -- 172.21.15.33:0/1679673929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe790101960 msgr2=0x7fe790108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.351+0000 7fe79575f700 1 --2- 172.21.15.33:0/1679673929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe790101960 0x7fe790108210 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe788009fc0 tx=0x7fe78800b330 comp rx=0 tx=0).stop 2024-05-08T21:18:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.352+0000 7fe79575f700 1 -- 172.21.15.33:0/1679673929 shutdown_connections 2024-05-08T21:18:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.352+0000 7fe79575f700 1 --2- 172.21.15.33:0/1679673929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe790101960 0x7fe790108210 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.352+0000 7fe79575f700 1 --2- 172.21.15.33:0/1679673929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7900ffdd0 0x7fe7901001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.352+0000 7fe79575f700 1 --2- 172.21.15.33:0/1679673929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe790100fc0 0x7fe790101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.352+0000 7fe79575f700 1 -- 172.21.15.33:0/1679673929 >> 172.21.15.33:0/1679673929 conn(0x7fe7900fb340 msgr2=0x7fe7900fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.352+0000 7fe79575f700 1 -- 172.21.15.33:0/1679673929 shutdown_connections 2024-05-08T21:18:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.352+0000 7fe79575f700 1 -- 172.21.15.33:0/1679673929 wait complete. 2024-05-08T21:18:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.352+0000 7fe79575f700 1 Processor -- start 2024-05-08T21:18:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe79575f700 1 -- start start 2024-05-08T21:18:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe79575f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffdd0 0x7fe790069b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe79575f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe790100fc0 0x7fe79019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe79575f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe790101960 0x7fe79019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe79575f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79019e960 con 0x7fe790101960 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe79575f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79019ead0 con 0x7fe7900ffdd0 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe79575f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79019ec90 con 0x7fe790100fc0 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe78f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe790101960 0x7fe79019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe78f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe790101960 0x7fe79019e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54558/0 (socket says 172.21.15.33:54558) 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe78effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffdd0 0x7fe790069b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe78f7fe700 1 -- 172.21.15.33:0/3278382381 learned_addr learned my addr 172.21.15.33:0/3278382381 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe78e7fc700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe790100fc0 0x7fe79019bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe78f7fe700 1 -- 172.21.15.33:0/3278382381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe790100fc0 msgr2=0x7fe79019bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe78f7fe700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe790100fc0 0x7fe79019bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe78f7fe700 1 -- 172.21.15.33:0/3278382381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffdd0 msgr2=0x7fe790069b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe78f7fe700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffdd0 0x7fe790069b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe78f7fe700 1 -- 172.21.15.33:0/3278382381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7800095f0 con 0x7fe790101960 2024-05-08T21:18:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.353+0000 7fe78effd700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffdd0 0x7fe790069b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.354+0000 7fe78f7fe700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe790101960 0x7fe79019e170 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe788008d00 tx=0x7fe788008d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.354+0000 7fe777fff700 1 -- 172.21.15.33:0/3278382381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78800e040 con 0x7fe790101960 2024-05-08T21:18:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.354+0000 7fe777fff700 1 -- 172.21.15.33:0/3278382381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe788006020 con 0x7fe790101960 2024-05-08T21:18:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.354+0000 7fe79575f700 1 -- 172.21.15.33:0/3278382381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe788009c40 con 0x7fe790101960 2024-05-08T21:18:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.354+0000 7fe777fff700 1 -- 172.21.15.33:0/3278382381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788005040 con 0x7fe790101960 2024-05-08T21:18:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.354+0000 7fe79575f700 1 -- 172.21.15.33:0/3278382381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79019f1a0 con 0x7fe790101960 2024-05-08T21:18:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.355+0000 7fe79575f700 1 -- 172.21.15.33:0/3278382381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe790107250 con 0x7fe790101960 2024-05-08T21:18:41.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.356+0000 7fe777fff700 1 -- 172.21.15.33:0/3278382381 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe78801d470 con 0x7fe790101960 2024-05-08T21:18:41.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.357+0000 7fe777fff700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe7780a11b0 0x7fe7780a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:41.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.357+0000 7fe78effd700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe7780a11b0 0x7fe7780a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:41.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.357+0000 7fe777fff700 1 -- 172.21.15.33:0/3278382381 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe7880ce580 con 0x7fe790101960 2024-05-08T21:18:41.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.358+0000 7fe78effd700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe7780a11b0 0x7fe7780a3670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe790100e20 tx=0x7fe780009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:41.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.362+0000 7fe777fff700 1 -- 172.21.15.33:0/3278382381 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe788097930 con 0x7fe790101960 2024-05-08T21:18:41.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.584+0000 7fe79575f700 1 -- 172.21.15.33:0/3278382381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe79010b630 con 0x7fe790101960 2024-05-08T21:18:41.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.584+0000 7fe777fff700 1 -- 172.21.15.33:0/3278382381 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe78809af50 con 0x7fe790101960 2024-05-08T21:18:41.587 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T21:18:41.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.587+0000 7fe79575f700 1 -- 172.21.15.33:0/3278382381 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe7780a11b0 msgr2=0x7fe7780a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:41.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.587+0000 7fe79575f700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe7780a11b0 0x7fe7780a3670 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe790100e20 tx=0x7fe780009310 comp rx=0 tx=0).stop 2024-05-08T21:18:41.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.587+0000 7fe79575f700 1 -- 172.21.15.33:0/3278382381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe790101960 msgr2=0x7fe79019e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:41.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.587+0000 7fe79575f700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe790101960 0x7fe79019e170 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe788008d00 tx=0x7fe788008d30 comp rx=0 tx=0).stop 2024-05-08T21:18:41.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.587+0000 7fe79575f700 1 -- 172.21.15.33:0/3278382381 shutdown_connections 2024-05-08T21:18:41.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.587+0000 7fe79575f700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe790101960 0x7fe79019e170 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.587+0000 7fe79575f700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe7780a11b0 0x7fe7780a3670 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.587+0000 7fe79575f700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7900ffdd0 0x7fe790069b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.587+0000 7fe79575f700 1 --2- 172.21.15.33:0/3278382381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe790100fc0 0x7fe79019bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.587+0000 7fe79575f700 1 -- 172.21.15.33:0/3278382381 >> 172.21.15.33:0/3278382381 conn(0x7fe7900fb340 msgr2=0x7fe790104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:41.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.587+0000 7fe79575f700 1 -- 172.21.15.33:0/3278382381 shutdown_connections 2024-05-08T21:18:41.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.587+0000 7fe79575f700 1 -- 172.21.15.33:0/3278382381 wait complete. 2024-05-08T21:18:41.602 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:41 smithi033 ceph-mon[42528]: pgmap v279: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:18:41.602 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:41 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/665127043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:41.602 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:41 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3861179759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:41.602 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:41 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4240399228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:41.603 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T21:18:41.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:41 smithi082 ceph-mon[47512]: pgmap v279: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:18:41.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:41 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/665127043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:41.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:41 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3861179759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:41.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:41 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/4240399228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.743+0000 7fe6c7316700 1 -- 172.21.15.33:0/682348673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6c0100fb0 msgr2=0x7fe6c0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.743+0000 7fe6c7316700 1 --2- 172.21.15.33:0/682348673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6c0100fb0 0x7fe6c0101410 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0099c0 tx=0x7fe6bc009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.743+0000 7fe6c7316700 1 -- 172.21.15.33:0/682348673 shutdown_connections 2024-05-08T21:18:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.743+0000 7fe6c7316700 1 --2- 172.21.15.33:0/682348673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0101950 0x7fe6c0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.743+0000 7fe6c7316700 1 --2- 172.21.15.33:0/682348673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c00ffdc0 0x7fe6c01001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.743+0000 7fe6c7316700 1 --2- 172.21.15.33:0/682348673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6c0100fb0 0x7fe6c0101410 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.743+0000 7fe6c7316700 1 -- 172.21.15.33:0/682348673 >> 172.21.15.33:0/682348673 conn(0x7fe6c00fb330 msgr2=0x7fe6c00fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.743+0000 7fe6c7316700 1 -- 172.21.15.33:0/682348673 shutdown_connections 2024-05-08T21:18:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.743+0000 7fe6c7316700 1 -- 172.21.15.33:0/682348673 wait complete. 2024-05-08T21:18:41.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c7316700 1 Processor -- start 2024-05-08T21:18:41.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c7316700 1 -- start start 2024-05-08T21:18:41.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c7316700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c00ffdc0 0x7fe6c0103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:41.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c7316700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6c0100fb0 0x7fe6c0102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:41.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c7316700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0101950 0x7fe6c01028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:41.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c7316700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c0102fd0 con 0x7fe6c0100fb0 2024-05-08T21:18:41.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c7316700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c0103140 con 0x7fe6c00ffdc0 2024-05-08T21:18:41.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c7316700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c01a0e20 con 0x7fe6c0101950 2024-05-08T21:18:41.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c48b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6c0100fb0 0x7fe6c0102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:41.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c48b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6c0100fb0 0x7fe6c0102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54584/0 (socket says 172.21.15.33:54584) 2024-05-08T21:18:41.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c48b1700 1 -- 172.21.15.33:0/811504950 learned_addr learned my addr 172.21.15.33:0/811504950 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:41.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c58b3700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0101950 0x7fe6c01028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:41.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.744+0000 7fe6c50b2700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c00ffdc0 0x7fe6c0103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.745+0000 7fe6c48b1700 1 -- 172.21.15.33:0/811504950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0101950 msgr2=0x7fe6c01028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.745+0000 7fe6c48b1700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0101950 0x7fe6c01028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.745+0000 7fe6c48b1700 1 -- 172.21.15.33:0/811504950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c00ffdc0 msgr2=0x7fe6c0103cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.745+0000 7fe6c48b1700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c00ffdc0 0x7fe6c0103cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.745+0000 7fe6c48b1700 1 -- 172.21.15.33:0/811504950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc0095f0 con 0x7fe6c0100fb0 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.745+0000 7fe6c58b3700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0101950 0x7fe6c01028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.745+0000 7fe6c48b1700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6c0100fb0 0x7fe6c0102390 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000c00 tx=0x7fe6bc00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.745+0000 7fe6b67fc700 1 -- 172.21.15.33:0/811504950 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc01d070 con 0x7fe6c0100fb0 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.745+0000 7fe6b67fc700 1 -- 172.21.15.33:0/811504950 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6bc00bd30 con 0x7fe6c0100fb0 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.745+0000 7fe6b67fc700 1 -- 172.21.15.33:0/811504950 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc017650 con 0x7fe6c0100fb0 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.745+0000 7fe6c7316700 1 -- 172.21.15.33:0/811504950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c01a1020 con 0x7fe6c0100fb0 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.745+0000 7fe6c7316700 1 -- 172.21.15.33:0/811504950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c01a1440 con 0x7fe6c0100fb0 2024-05-08T21:18:41.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.746+0000 7fe6c7316700 1 -- 172.21.15.33:0/811504950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c0067480 con 0x7fe6c0100fb0 2024-05-08T21:18:41.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.747+0000 7fe6b67fc700 1 -- 172.21.15.33:0/811504950 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe6bc022470 con 0x7fe6c0100fb0 2024-05-08T21:18:41.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.748+0000 7fe6b67fc700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6ac0a1230 0x7fe6ac0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:41.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.748+0000 7fe6b67fc700 1 -- 172.21.15.33:0/811504950 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe6bc0ce550 con 0x7fe6c0100fb0 2024-05-08T21:18:41.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.748+0000 7fe6c50b2700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6ac0a1230 0x7fe6ac0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:41.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.749+0000 7fe6c50b2700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6ac0a1230 0x7fe6ac0a36f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0100e10 tx=0x7fe6b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:41.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.752+0000 7fe6b67fc700 1 -- 172.21.15.33:0/811504950 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe6bc097900 con 0x7fe6c0100fb0 2024-05-08T21:18:41.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:41 smithi071 ceph-mon[47074]: pgmap v279: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:18:41.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:41 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/665127043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:41.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:41 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3861179759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:41.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:41 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/4240399228' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.970+0000 7fe6c7316700 1 -- 172.21.15.33:0/811504950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe6c010b610 con 0x7fe6c0100fb0 2024-05-08T21:18:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.971+0000 7fe6b67fc700 1 -- 172.21.15.33:0/811504950 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe6bc027020 con 0x7fe6c0100fb0 2024-05-08T21:18:41.973 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.973+0000 7fe6c7316700 1 -- 172.21.15.33:0/811504950 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6ac0a1230 msgr2=0x7fe6ac0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.973+0000 7fe6c7316700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6ac0a1230 0x7fe6ac0a36f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0100e10 tx=0x7fe6b0009260 comp rx=0 tx=0).stop 2024-05-08T21:18:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.973+0000 7fe6c7316700 1 -- 172.21.15.33:0/811504950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6c0100fb0 msgr2=0x7fe6c0102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.973+0000 7fe6c7316700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6c0100fb0 0x7fe6c0102390 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000c00 tx=0x7fe6bc00fab0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.973+0000 7fe6c7316700 1 -- 172.21.15.33:0/811504950 shutdown_connections 2024-05-08T21:18:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.973+0000 7fe6c7316700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c0101950 0x7fe6c01028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.973+0000 7fe6c7316700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6ac0a1230 0x7fe6ac0a36f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.973+0000 7fe6c7316700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6c00ffdc0 0x7fe6c0103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.973+0000 7fe6c7316700 1 --2- 172.21.15.33:0/811504950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6c0100fb0 0x7fe6c0102390 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.973+0000 7fe6c7316700 1 -- 172.21.15.33:0/811504950 >> 172.21.15.33:0/811504950 conn(0x7fe6c00fb330 msgr2=0x7fe6c0069080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.974+0000 7fe6c7316700 1 -- 172.21.15.33:0/811504950 shutdown_connections 2024-05-08T21:18:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:41.974+0000 7fe6c7316700 1 -- 172.21.15.33:0/811504950 wait complete. 2024-05-08T21:18:41.990 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:18:42.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.124+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2421777670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5b4104230 msgr2=0x7fc5b4106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.124+0000 7fc5b8bf4700 1 --2- 172.21.15.33:0/2421777670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5b4104230 0x7fc5b4106620 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009990 tx=0x7fc59c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.124+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2421777670 shutdown_connections 2024-05-08T21:18:42.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.124+0000 7fc5b8bf4700 1 --2- 172.21.15.33:0/2421777670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b4106b60 0x7fc5b4108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.124+0000 7fc5b8bf4700 1 --2- 172.21.15.33:0/2421777670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b4101900 0x7fc5b4103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.124+0000 7fc5b8bf4700 1 --2- 172.21.15.33:0/2421777670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5b4104230 0x7fc5b4106620 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.124+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2421777670 >> 172.21.15.33:0/2421777670 conn(0x7fc5b40fb310 msgr2=0x7fc5b40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:42.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.124+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2421777670 shutdown_connections 2024-05-08T21:18:42.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.124+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2421777670 wait complete. 2024-05-08T21:18:42.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.125+0000 7fc5b8bf4700 1 Processor -- start 2024-05-08T21:18:42.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.125+0000 7fc5b8bf4700 1 -- start start 2024-05-08T21:18:42.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.125+0000 7fc5b8bf4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b4101900 0x7fc5b4196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.125+0000 7fc5b8bf4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5b4104230 0x7fc5b4197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.125+0000 7fc5b8bf4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b4106b60 0x7fc5b419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.125+0000 7fc5b8bf4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b419e800 con 0x7fc5b4104230 2024-05-08T21:18:42.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.125+0000 7fc5b8bf4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b419e970 con 0x7fc5b4106b60 2024-05-08T21:18:42.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.125+0000 7fc5b8bf4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b419eb30 con 0x7fc5b4101900 2024-05-08T21:18:42.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5a9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5b4104230 0x7fc5b4197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5a9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5b4104230 0x7fc5b4197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54604/0 (socket says 172.21.15.33:54604) 2024-05-08T21:18:42.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5a9d9b700 1 -- 172.21.15.33:0/2295732255 learned_addr learned my addr 172.21.15.33:0/2295732255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5b259c700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b4101900 0x7fc5b4196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5b2d9d700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b4106b60 0x7fc5b419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5a9d9b700 1 -- 172.21.15.33:0/2295732255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b4101900 msgr2=0x7fc5b4196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5a9d9b700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b4101900 0x7fc5b4196ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5a9d9b700 1 -- 172.21.15.33:0/2295732255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b4106b60 msgr2=0x7fc5b419e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5a9d9b700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b4106b60 0x7fc5b419e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5a9d9b700 1 -- 172.21.15.33:0/2295732255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c0095f0 con 0x7fc5b4104230 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5a9d9b700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5b4104230 0x7fc5b4197220 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005740 tx=0x7fc59c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5abfff700 1 -- 172.21.15.33:0/2295732255 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c01d070 con 0x7fc5b4104230 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2295732255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b419edb0 con 0x7fc5b4104230 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5abfff700 1 -- 172.21.15.33:0/2295732255 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc59c00be70 con 0x7fc5b4104230 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2295732255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b419f1d0 con 0x7fc5b4104230 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.126+0000 7fc5abfff700 1 -- 172.21.15.33:0/2295732255 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00fe50 con 0x7fc5b4104230 2024-05-08T21:18:42.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.127+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2295732255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b40fce90 con 0x7fc5b4104230 2024-05-08T21:18:42.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.129+0000 7fc5abfff700 1 -- 172.21.15.33:0/2295732255 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc59c00f640 con 0x7fc5b4104230 2024-05-08T21:18:42.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.129+0000 7fc5abfff700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5a00a11b0 0x7fc5a00a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.130+0000 7fc5b259c700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5a00a11b0 0x7fc5a00a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.130+0000 7fc5abfff700 1 -- 172.21.15.33:0/2295732255 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc59c0ce1e0 con 0x7fc5b4104230 2024-05-08T21:18:42.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.130+0000 7fc5b259c700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5a00a11b0 0x7fc5a00a3670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4103020 tx=0x7fc5a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:42.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.134+0000 7fc5abfff700 1 -- 172.21.15.33:0/2295732255 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc59c097530 con 0x7fc5b4104230 2024-05-08T21:18:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.403+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2295732255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5b4067d10 con 0x7fc5b4104230 2024-05-08T21:18:42.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.404+0000 7fc5abfff700 1 -- 172.21.15.33:0/2295732255 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10899 (secure 0 0 0) 0x7fc59c09ab50 con 0x7fc5b4104230 2024-05-08T21:18:42.406 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:42.406 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:17:32.303648+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:18:42.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.406+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2295732255 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5a00a11b0 msgr2=0x7fc5a00a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.406+0000 7fc5b8bf4700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5a00a11b0 0x7fc5a00a3670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4103020 tx=0x7fc5a4009260 comp rx=0 tx=0).stop 2024-05-08T21:18:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.406+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2295732255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5b4104230 msgr2=0x7fc5b4197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.406+0000 7fc5b8bf4700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5b4104230 0x7fc5b4197220 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005740 tx=0x7fc59c009580 comp rx=0 tx=0).stop 2024-05-08T21:18:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.406+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2295732255 shutdown_connections 2024-05-08T21:18:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.406+0000 7fc5b8bf4700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5a00a11b0 0x7fc5a00a3670 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.406+0000 7fc5b8bf4700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b4106b60 0x7fc5b419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.406+0000 7fc5b8bf4700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b4101900 0x7fc5b4196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.406+0000 7fc5b8bf4700 1 --2- 172.21.15.33:0/2295732255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5b4104230 0x7fc5b4197220 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.406+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2295732255 >> 172.21.15.33:0/2295732255 conn(0x7fc5b40fb310 msgr2=0x7fc5b4102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.406+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2295732255 shutdown_connections 2024-05-08T21:18:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.406+0000 7fc5b8bf4700 1 -- 172.21.15.33:0/2295732255 wait complete. 2024-05-08T21:18:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-05-08T21:18:42.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:18:42.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.558+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2785073041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c4104230 msgr2=0x7fe9c4106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.558+0000 7fe9c8dba700 1 --2- 172.21.15.33:0/2785073041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c4104230 0x7fe9c4106620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009990 tx=0x7fe9b8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.558+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2785073041 shutdown_connections 2024-05-08T21:18:42.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.558+0000 7fe9c8dba700 1 --2- 172.21.15.33:0/2785073041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c4106b60 0x7fe9c4108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.558+0000 7fe9c8dba700 1 --2- 172.21.15.33:0/2785073041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c4101900 0x7fe9c4103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.558+0000 7fe9c8dba700 1 --2- 172.21.15.33:0/2785073041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c4104230 0x7fe9c4106620 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.558+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2785073041 >> 172.21.15.33:0/2785073041 conn(0x7fe9c40fb310 msgr2=0x7fe9c40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:42.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.558+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2785073041 shutdown_connections 2024-05-08T21:18:42.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.558+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2785073041 wait complete. 2024-05-08T21:18:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.559+0000 7fe9c8dba700 1 Processor -- start 2024-05-08T21:18:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.559+0000 7fe9c8dba700 1 -- start start 2024-05-08T21:18:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.559+0000 7fe9c8dba700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c4101900 0x7fe9c4196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.559+0000 7fe9c8dba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c4104230 0x7fe9c41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.559+0000 7fe9c8dba700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c4106b60 0x7fe9c419e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.559+0000 7fe9c8dba700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c419e7a0 con 0x7fe9c4104230 2024-05-08T21:18:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.559+0000 7fe9c8dba700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c419e910 con 0x7fe9c4106b60 2024-05-08T21:18:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.559+0000 7fe9c8dba700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c419ead0 con 0x7fe9c4101900 2024-05-08T21:18:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.559+0000 7fe9c1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c4104230 0x7fe9c41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.559+0000 7fe9c1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c4104230 0x7fe9c41971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54614/0 (socket says 172.21.15.33:54614) 2024-05-08T21:18:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.559+0000 7fe9c1d9b700 1 -- 172.21.15.33:0/2464078900 learned_addr learned my addr 172.21.15.33:0/2464078900 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9c2d9d700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c4106b60 0x7fe9c419e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9c259c700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c4101900 0x7fe9c4196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9c1d9b700 1 -- 172.21.15.33:0/2464078900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c4101900 msgr2=0x7fe9c4196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9c1d9b700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c4101900 0x7fe9c4196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9c1d9b700 1 -- 172.21.15.33:0/2464078900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c4106b60 msgr2=0x7fe9c419e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9c1d9b700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c4106b60 0x7fe9c419e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9c1d9b700 1 -- 172.21.15.33:0/2464078900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b80095f0 con 0x7fe9c4104230 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9c1d9b700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c4104230 0x7fe9c41971f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80098e0 tx=0x7fe9b8003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9b37fe700 1 -- 172.21.15.33:0/2464078900 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b801d070 con 0x7fe9c4104230 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9b37fe700 1 -- 172.21.15.33:0/2464078900 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9b800bd90 con 0x7fe9c4104230 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2464078900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c419ed50 con 0x7fe9c4104230 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9b37fe700 1 -- 172.21.15.33:0/2464078900 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800f460 con 0x7fe9c4104230 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.560+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2464078900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c419f170 con 0x7fe9c4104230 2024-05-08T21:18:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.561+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2464078900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c40fce90 con 0x7fe9c4104230 2024-05-08T21:18:42.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.563+0000 7fe9b37fe700 1 -- 172.21.15.33:0/2464078900 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe9b8022470 con 0x7fe9c4104230 2024-05-08T21:18:42.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.563+0000 7fe9b37fe700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe9ac0a11b0 0x7fe9ac0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.563+0000 7fe9b37fe700 1 -- 172.21.15.33:0/2464078900 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe9b80ce4e0 con 0x7fe9c4104230 2024-05-08T21:18:42.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.563+0000 7fe9c259c700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe9ac0a11b0 0x7fe9ac0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.564+0000 7fe9c259c700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe9ac0a11b0 0x7fe9ac0a3670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4005950 tx=0x7fe9b4005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:42.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.567+0000 7fe9b37fe700 1 -- 172.21.15.33:0/2464078900 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe9b8097830 con 0x7fe9c4104230 2024-05-08T21:18:42.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:42 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3278382381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:42.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:42 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/811504950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:42.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.785+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2464078900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9c410c520 con 0x7fe9c4104230 2024-05-08T21:18:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.787+0000 7fe9b37fe700 1 -- 172.21.15.33:0/2464078900 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24374 (secure 0 0 0) 0x7fe9b809ae50 con 0x7fe9c4104230 2024-05-08T21:18:42.789 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:42.789 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:57.410336+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:18:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.789+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2464078900 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe9ac0a11b0 msgr2=0x7fe9ac0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.789+0000 7fe9c8dba700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe9ac0a11b0 0x7fe9ac0a3670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4005950 tx=0x7fe9b4005ed0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.789+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2464078900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c4104230 msgr2=0x7fe9c41971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.789+0000 7fe9c8dba700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c4104230 0x7fe9c41971f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80098e0 tx=0x7fe9b8003b00 comp rx=0 tx=0).stop 2024-05-08T21:18:42.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.789+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2464078900 shutdown_connections 2024-05-08T21:18:42.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.789+0000 7fe9c8dba700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe9ac0a11b0 0x7fe9ac0a3670 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.789+0000 7fe9c8dba700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c4106b60 0x7fe9c419e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.789+0000 7fe9c8dba700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9c4101900 0x7fe9c4196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.789+0000 7fe9c8dba700 1 --2- 172.21.15.33:0/2464078900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9c4104230 0x7fe9c41971f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.789+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2464078900 >> 172.21.15.33:0/2464078900 conn(0x7fe9c40fb310 msgr2=0x7fe9c4102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:42.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.789+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2464078900 shutdown_connections 2024-05-08T21:18:42.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.789+0000 7fe9c8dba700 1 -- 172.21.15.33:0/2464078900 wait complete. 2024-05-08T21:18:42.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T21:17:03.219607+0000', 'modified': '2024-05-08T21:17:32.303648+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24563, 'mds_1': 24521, 'mds_2': 24499, 'mds_3': 24497, 'mds_4': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6841/2132539478', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6840', 'nonce': 2132539478}, {'type': 'v1', 'addr': '172.21.15.33:6841', 'nonce': 2132539478}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24451': {'gid': 24451, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.82:6833/3382287612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6832', 'nonce': 3382287612}, {'type': 'v1', 'addr': '172.21.15.82:6833', 'nonce': 3382287612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24460': {'gid': 24460, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6835/82548780', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6834', 'nonce': 82548780}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 82548780}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24461': {'gid': 24461, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6833/497942273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 497942273}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 497942273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24472': {'gid': 24472, 'name': 'd', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6837/362093971', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6836', 'nonce': 362093971}, {'type': 'v1', 'addr': '172.21.15.33:6837', 'nonce': 362093971}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24487': {'gid': 24487, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6835/1877231014', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6834', 'nonce': 1877231014}, {'type': 'v1', 'addr': '172.21.15.82:6835', 'nonce': 1877231014}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24497': {'gid': 24497, 'name': 'e', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.71:6835/3686264359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 3686264359}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 3686264359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24499': {'gid': 24499, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.82:6837/2409190198', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 2409190198}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 2409190198}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.71:6837/1668078897', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6836', 'nonce': 1668078897}, {'type': 'v1', 'addr': '172.21.15.71:6837', 'nonce': 1668078897}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24563': {'gid': 24563, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6839/1515211151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1515211151}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1515211151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-05-08T21:18:42.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-08T21:18:42.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-05-08T21:18:42.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:42 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3278382381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:42.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:42 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/811504950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:42.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:42 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3278382381' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:42.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:42 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/811504950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:42.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.942+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/4291521289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf581018f0 msgr2=0x7fcf581081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.942+0000 7fcf5e2b8700 1 --2- 172.21.15.33:0/4291521289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf581018f0 0x7fcf581081a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000b0d0 tx=0x7fcf5000b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.942+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/4291521289 shutdown_connections 2024-05-08T21:18:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.942+0000 7fcf5e2b8700 1 --2- 172.21.15.33:0/4291521289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf581018f0 0x7fcf581081a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.942+0000 7fcf5e2b8700 1 --2- 172.21.15.33:0/4291521289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf580ffd60 0x7fcf58100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.942+0000 7fcf5e2b8700 1 --2- 172.21.15.33:0/4291521289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf58100f50 0x7fcf581013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.942+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/4291521289 >> 172.21.15.33:0/4291521289 conn(0x7fcf580fb290 msgr2=0x7fcf580fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.942+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/4291521289 shutdown_connections 2024-05-08T21:18:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.942+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/4291521289 wait complete. 2024-05-08T21:18:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.943+0000 7fcf5e2b8700 1 Processor -- start 2024-05-08T21:18:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.943+0000 7fcf5e2b8700 1 -- start start 2024-05-08T21:18:42.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.943+0000 7fcf5e2b8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf580ffd60 0x7fcf58196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.943+0000 7fcf5e2b8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf58100f50 0x7fcf58197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.943+0000 7fcf5e2b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf581018f0 0x7fcf5819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5e2b8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5819e940 con 0x7fcf581018f0 2024-05-08T21:18:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5e2b8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5819ea80 con 0x7fcf58100f50 2024-05-08T21:18:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5e2b8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5819ec10 con 0x7fcf580ffd60 2024-05-08T21:18:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5c855700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf581018f0 0x7fcf5819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5c855700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf581018f0 0x7fcf5819e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54636/0 (socket says 172.21.15.33:54636) 2024-05-08T21:18:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5c855700 1 -- 172.21.15.33:0/2131274553 learned_addr learned my addr 172.21.15.33:0/2131274553 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf57fff700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf580ffd60 0x7fcf58196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf577fe700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf58100f50 0x7fcf58197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5c855700 1 -- 172.21.15.33:0/2131274553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf580ffd60 msgr2=0x7fcf58196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5c855700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf580ffd60 0x7fcf58196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5c855700 1 -- 172.21.15.33:0/2131274553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf58100f50 msgr2=0x7fcf58197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5c855700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf58100f50 0x7fcf58197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5c855700 1 -- 172.21.15.33:0/2131274553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf50009d00 con 0x7fcf581018f0 2024-05-08T21:18:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf57fff700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf580ffd60 0x7fcf58196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf577fe700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf58100f50 0x7fcf58197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5c855700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf581018f0 0x7fcf5819e370 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcf50007af0 tx=0x7fcf5000baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf557fa700 1 -- 172.21.15.33:0/2131274553 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000e040 con 0x7fcf581018f0 2024-05-08T21:18:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf557fa700 1 -- 172.21.15.33:0/2131274553 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf5001db00 con 0x7fcf581018f0 2024-05-08T21:18:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf557fa700 1 -- 172.21.15.33:0/2131274553 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5001baf0 con 0x7fcf581018f0 2024-05-08T21:18:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.944+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/2131274553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5819ee90 con 0x7fcf581018f0 2024-05-08T21:18:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.945+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/2131274553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5819f2b0 con 0x7fcf581018f0 2024-05-08T21:18:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.947+0000 7fcf557fa700 1 -- 172.21.15.33:0/2131274553 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcf5001dc70 con 0x7fcf581018f0 2024-05-08T21:18:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.947+0000 7fcf557fa700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcf400a1230 0x7fcf400a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.947+0000 7fcf557fa700 1 -- 172.21.15.33:0/2131274553 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcf500ce340 con 0x7fcf581018f0 2024-05-08T21:18:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.948+0000 7fcf57fff700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcf400a1230 0x7fcf400a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.948+0000 7fcf57fff700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcf400a1230 0x7fcf400a36f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf48005950 tx=0x7fcf480058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.948+0000 7fcf6a472b80 1 Processor -- start 2024-05-08T21:18:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.948+0000 7fcf6a472b80 1 -- start start 2024-05-08T21:18:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.948+0000 7fcf6a472b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475b4f0 0x55c3b475b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.948+0000 7fcf6a472b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475be50 0x55c3b475c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.948+0000 7fcf6a472b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475c810 0x55c3b475cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.948+0000 7fcf6a472b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b46b2100 con 0x55c3b475be50 2024-05-08T21:18:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.948+0000 7fcf6a472b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b46afac0 con 0x55c3b475c810 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.948+0000 7fcf6a472b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b46ab3b0 con 0x55c3b475b4f0 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475be50 0x55c3b475c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475be50 0x55c3b475c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54648/0 (socket says 172.21.15.33:54648) 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf577fe700 1 -- 172.21.15.33:0/2562526887 learned_addr learned my addr 172.21.15.33:0/2562526887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf57fff700 1 --2- 172.21.15.33:0/2562526887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475b4f0 0x55c3b475b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf5c855700 1 --2- 172.21.15.33:0/2562526887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475c810 0x55c3b475cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf577fe700 1 -- 172.21.15.33:0/2562526887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475b4f0 msgr2=0x55c3b475b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf577fe700 1 --2- 172.21.15.33:0/2562526887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475b4f0 0x55c3b475b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf577fe700 1 -- 172.21.15.33:0/2562526887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475c810 msgr2=0x55c3b475cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf577fe700 1 --2- 172.21.15.33:0/2562526887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475c810 0x55c3b475cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf577fe700 1 -- 172.21.15.33:0/2562526887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3b452c360 con 0x55c3b475be50 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf577fe700 1 --2- 172.21.15.33:0/2562526887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475be50 0x55c3b475c2d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009de0 tx=0x7fcf4c0116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8f8a1ed32fa23d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf3e7fc700 1 -- 172.21.15.33:0/2562526887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c011e20 con 0x55c3b475be50 2024-05-08T21:18:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf3e7fc700 1 -- 172.21.15.33:0/2562526887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf4c009e70 con 0x55c3b475be50 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.949+0000 7fcf3e7fc700 1 -- 172.21.15.33:0/2562526887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00f730 con 0x55c3b475be50 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.950+0000 7fcf6a472b80 1 -- 172.21.15.33:0/2562526887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475be50 msgr2=0x55c3b475c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.950+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/2562526887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475be50 0x55c3b475c2d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009de0 tx=0x7fcf4c0116c0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.950+0000 7fcf6a472b80 1 -- 172.21.15.33:0/2562526887 shutdown_connections 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.950+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/2562526887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475c810 0x55c3b475cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.950+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/2562526887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475b4f0 0x55c3b475b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.950+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/2562526887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475be50 0x55c3b475c2d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.950+0000 7fcf6a472b80 1 -- 172.21.15.33:0/2562526887 >> 172.21.15.33:0/2562526887 conn(0x55c3b46b91c0 msgr2=0x55c3b4755d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.950+0000 7fcf6a472b80 1 -- 172.21.15.33:0/2562526887 shutdown_connections 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.950+0000 7fcf6a472b80 1 -- 172.21.15.33:0/2562526887 wait complete. 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf6a472b80 1 -- start start 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf6a472b80 1 Processor -- start 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf6a472b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475b4f0 0x55c3b4764d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf6a472b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475be50 0x55c3b47652d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf6a472b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 0x55c3b476c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf6a472b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b4647510 con 0x55c3b475c810 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf6a472b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b46471b0 con 0x55c3b475b4f0 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf6a472b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b452e650 con 0x55c3b475be50 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf6a472b80 10 client.0 _ll_register_callbacks cb 0x55c3b4650ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf5c855700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 0x55c3b476c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf5c855700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 0x55c3b476c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54660/0 (socket says 172.21.15.33:54660) 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf5c855700 1 -- 172.21.15.33:0/1709038890 learned_addr learned my addr 172.21.15.33:0/1709038890 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf57fff700 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475b4f0 0x55c3b4764d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.951+0000 7fcf577fe700 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475be50 0x55c3b47652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf5c855700 1 -- 172.21.15.33:0/1709038890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475be50 msgr2=0x55c3b47652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf5c855700 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475be50 0x55c3b47652d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf5c855700 1 -- 172.21.15.33:0/1709038890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475b4f0 msgr2=0x55c3b4764d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:42.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf5c855700 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475b4f0 0x55c3b4764d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf5c855700 1 -- 172.21.15.33:0/1709038890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4c009a10 con 0x55c3b475c810 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf5c855700 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 0x55c3b476c310 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcf500079c0 tx=0x7fcf5009b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe1fcb573a6bae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf377fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf377fe700 1 -- 172.21.15.33:0/1709038890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500860b0 con 0x55c3b475c810 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf377fe700 1 -- 172.21.15.33:0/1709038890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcf5009bab0 con 0x55c3b475c810 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf6a472b80 1 -- 172.21.15.33:0/1709038890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3b4604220 con 0x55c3b475c810 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf377fe700 1 -- 172.21.15.33:0/1709038890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf5009b3b0 con 0x55c3b475c810 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf6a472b80 10 client.15420 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf6a472b80 1 -- 172.21.15.33:0/1709038890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c3b44d8b10 con 0x55c3b475c810 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf377fe700 1 -- 172.21.15.33:0/1709038890 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c3b4604220 con 0x55c3b475c810 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.952+0000 7fcf377fe700 1 -- 172.21.15.33:0/1709038890 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcf5009a8a0 con 0x55c3b475c810 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.953+0000 7fcf6a472b80 10 client.15420 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.953+0000 7fcf6a472b80 10 client.15420 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-08T21:18:42.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.953+0000 7fcf6a472b80 20 client.15420 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:42.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.953+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c3b476fd90 0x55c3b4775090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.953+0000 7fcf6a472b80 4 client.15420 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:18:42.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.953+0000 7fcf6a472b80 1 -- 172.21.15.33:0/1709038890 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3b472a7c0 con 0x55c3b476fd90 2024-05-08T21:18:42.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.953+0000 7fcf57fff700 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c3b476fd90 0x55c3b4775090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.953+0000 7fcf377fe700 10 client.15420 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:18:42.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.953+0000 7fcf57fff700 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c3b476fd90 0x55c3b4775090 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a1859e5010a879d server_cookie=38a8b8e6b7646531 in_seq=0 out_seq=0 2024-05-08T21:18:42.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf377fe700 1 -- 172.21.15.33:0/1709038890 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3b472a7c0 con 0x55c3b476fd90 2024-05-08T21:18:42.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf377fe700 10 client.15420 handle_command_reply: tid=0 2024-05-08T21:18:42.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf6a472b80 1 client.15420 shutdown 2024-05-08T21:18:42.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf6a472b80 1 -- 172.21.15.33:0/1709038890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 msgr2=0x55c3b476c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 0x55c3b476c310 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcf500079c0 tx=0x7fcf5009b200 comp rx=0 tx=0).stop 2024-05-08T21:18:42.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf6a472b80 1 -- 172.21.15.33:0/1709038890 shutdown_connections 2024-05-08T21:18:42.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 0x55c3b476c310 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475b4f0 0x55c3b4764d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c3b476fd90 0x55c3b4775090 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/1709038890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475be50 0x55c3b47652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf377fe700 0 client.15420 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:18:42.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf6a472b80 1 -- 172.21.15.33:0/1709038890 >> 172.21.15.33:0/1709038890 conn(0x55c3b46b91c0 msgr2=0x55c3b4755550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:42.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf6a472b80 1 -- 172.21.15.33:0/1709038890 shutdown_connections 2024-05-08T21:18:42.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.954+0000 7fcf6a472b80 1 -- 172.21.15.33:0/1709038890 wait complete. 2024-05-08T21:18:42.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.955+0000 7fcf6a472b80 20 client.15420 trim_cache size 0 max 16384 2024-05-08T21:18:42.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.974+0000 7fcf6a472b80 1 Processor -- start 2024-05-08T21:18:42.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.974+0000 7fcf6a472b80 1 -- start start 2024-05-08T21:18:42.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.974+0000 7fcf6a472b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475b4f0 0x55c3b476c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.974+0000 7fcf6a472b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475be50 0x55c3b47642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.974+0000 7fcf6a472b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 0x55c3b476c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.974+0000 7fcf6a472b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b472a7c0 con 0x55c3b475c810 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.974+0000 7fcf6a472b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b4604220 con 0x55c3b475b4f0 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.974+0000 7fcf6a472b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b46692b0 con 0x55c3b475be50 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.974+0000 7fcf5c855700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 0x55c3b476c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.974+0000 7fcf5c855700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 0x55c3b476c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54670/0 (socket says 172.21.15.33:54670) 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.974+0000 7fcf5c855700 1 -- 172.21.15.33:0/3411621411 learned_addr learned my addr 172.21.15.33:0/3411621411 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.974+0000 7fcf577fe700 1 --2- 172.21.15.33:0/3411621411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475be50 0x55c3b47642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.975+0000 7fcf57fff700 1 --2- 172.21.15.33:0/3411621411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475b4f0 0x55c3b476c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.975+0000 7fcf5c855700 1 -- 172.21.15.33:0/3411621411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475be50 msgr2=0x55c3b47642a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.975+0000 7fcf5c855700 1 --2- 172.21.15.33:0/3411621411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475be50 0x55c3b47642a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.975+0000 7fcf5c855700 1 -- 172.21.15.33:0/3411621411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475b4f0 msgr2=0x55c3b476c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.975+0000 7fcf5c855700 1 --2- 172.21.15.33:0/3411621411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475b4f0 0x55c3b476c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.975+0000 7fcf5c855700 1 -- 172.21.15.33:0/3411621411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3b4671510 con 0x55c3b475c810 2024-05-08T21:18:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.975+0000 7fcf5c855700 1 --2- 172.21.15.33:0/3411621411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 0x55c3b476c2d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcf50097e20 tx=0x7fcf50099960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe96cc4bc225c940 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.975+0000 7fcf3effd700 1 -- 172.21.15.33:0/3411621411 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500860b0 con 0x55c3b475c810 2024-05-08T21:18:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.975+0000 7fcf3effd700 1 -- 172.21.15.33:0/3411621411 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf5009bab0 con 0x55c3b475c810 2024-05-08T21:18:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.975+0000 7fcf3effd700 1 -- 172.21.15.33:0/3411621411 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500a37a0 con 0x55c3b475c810 2024-05-08T21:18:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.976+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3411621411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 msgr2=0x55c3b476c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.976+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/3411621411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 0x55c3b476c2d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcf50097e20 tx=0x7fcf50099960 comp rx=0 tx=0).stop 2024-05-08T21:18:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.976+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3411621411 shutdown_connections 2024-05-08T21:18:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.976+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/3411621411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475c810 0x55c3b476c2d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.976+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/3411621411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475b4f0 0x55c3b476c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.976+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/3411621411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475be50 0x55c3b47642a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.976+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3411621411 >> 172.21.15.33:0/3411621411 conn(0x55c3b46b91c0 msgr2=0x55c3b47521c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.976+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3411621411 shutdown_connections 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.976+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3411621411 wait complete. 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.976+0000 7fcf6a472b80 1 -- start start 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.976+0000 7fcf6a472b80 1 Processor -- start 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf6a472b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475b4f0 0x55c3b47664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf6a472b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475be50 0x55c3b4766ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf6a472b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475c810 0x55c3b476b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf6a472b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b476bcd0 con 0x55c3b475b4f0 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf6a472b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b476be40 con 0x55c3b475be50 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf6a472b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3b476c000 con 0x55c3b475c810 2024-05-08T21:18:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf6a472b80 10 client.0 _ll_register_callbacks cb 0x55c3b4650ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf57fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475b4f0 0x55c3b47664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf57fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475b4f0 0x55c3b47664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54682/0 (socket says 172.21.15.33:54682) 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf57fff700 1 -- 172.21.15.33:0/3100338441 learned_addr learned my addr 172.21.15.33:0/3100338441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf577fe700 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475be50 0x55c3b4766ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf5c855700 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475c810 0x55c3b476b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf57fff700 1 -- 172.21.15.33:0/3100338441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475c810 msgr2=0x55c3b476b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf57fff700 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475c810 0x55c3b476b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf57fff700 1 -- 172.21.15.33:0/3100338441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475be50 msgr2=0x55c3b4766ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf57fff700 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475be50 0x55c3b4766ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf57fff700 1 -- 172.21.15.33:0/3100338441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3b44d8b10 con 0x55c3b475b4f0 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf57fff700 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475b4f0 0x55c3b47664f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55c3b46472c0 tx=0x7fcf48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aab63afe9904b12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf377fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.977+0000 7fcf377fe700 1 -- 172.21.15.33:0/3100338441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf48024040 con 0x55c3b475b4f0 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf377fe700 1 -- 172.21.15.33:0/3100338441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcf48011850 con 0x55c3b475b4f0 2024-05-08T21:18:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3100338441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf4c009e70 con 0x55c3b475b4f0 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf377fe700 1 -- 172.21.15.33:0/3100338441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf48011a10 con 0x55c3b475b4f0 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf6a472b80 10 client.15426 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3100338441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c3b4530c80 con 0x55c3b475b4f0 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf377fe700 1 -- 172.21.15.33:0/3100338441 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf4c009e70 con 0x55c3b475b4f0 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf377fe700 1 -- 172.21.15.33:0/3100338441 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcf4801fca0 con 0x55c3b475b4f0 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf6a472b80 10 client.15426 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf6a472b80 10 client.15426 resolve_mds: resolved name 'l' to daemon mds.l 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf6a472b80 20 client.15426 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c3b47cb340 0x55c3b47c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf6a472b80 4 client.15426 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3100338441 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c3b46a3e70 con 0x55c3b47cb340 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.978+0000 7fcf577fe700 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c3b47cb340 0x55c3b47c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:42.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf577fe700 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c3b47cb340 0x55c3b47c7c90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34e4c1e2d957ee2c server_cookie=fbe87509c519c76c in_seq=0 out_seq=0 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf377fe700 10 client.15426 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf377fe700 1 -- 172.21.15.33:0/3100338441 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55c3b46a3e70 con 0x55c3b47cb340 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf377fe700 10 client.15426 handle_command_reply: tid=0 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf6a472b80 1 client.15426 shutdown 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3100338441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475b4f0 msgr2=0x55c3b47664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475b4f0 0x55c3b47664f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55c3b46472c0 tx=0x7fcf48004750 comp rx=0 tx=0).stop 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3100338441 shutdown_connections 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c3b47cb340 0x55c3b47c7c90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3b475c810 0x55c3b476b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3b475b4f0 0x55c3b47664f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf6a472b80 1 --2- 172.21.15.33:0/3100338441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3b475be50 0x55c3b4766ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf377fe700 0 client.15426 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3100338441 >> 172.21.15.33:0/3100338441 conn(0x55c3b46b91c0 msgr2=0x55c3b476c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3100338441 shutdown_connections 2024-05-08T21:18:42.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf6a472b80 1 -- 172.21.15.33:0/3100338441 wait complete. 2024-05-08T21:18:42.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.979+0000 7fcf6a472b80 20 client.15426 trim_cache size 0 max 16384 2024-05-08T21:18:42.988 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:42.988 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:18:42.988 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 4, 2024-05-08T21:18:42.988 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24451, 2024-05-08T21:18:42.988 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-08T21:18:42.988 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-08T21:18:42.988 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-08T21:18:42.988 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 71.679152612999999, 2024-05-08T21:18:42.988 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-05-08T21:18:42.988 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-05-08T21:18:42.989 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-05-08T21:18:42.989 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 125.585710531 2024-05-08T21:18:42.989 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:42.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.980+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/2131274553 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcf400a1230 msgr2=0x7fcf400a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.980+0000 7fcf5e2b8700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcf400a1230 0x7fcf400a36f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcf48005950 tx=0x7fcf480058e0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.980+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/2131274553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf581018f0 msgr2=0x7fcf5819e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.980+0000 7fcf5e2b8700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf581018f0 0x7fcf5819e370 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcf50007af0 tx=0x7fcf5000baa0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.980+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/2131274553 shutdown_connections 2024-05-08T21:18:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.980+0000 7fcf5e2b8700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf581018f0 0x7fcf5819e370 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.980+0000 7fcf5e2b8700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf580ffd60 0x7fcf58196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.980+0000 7fcf5e2b8700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcf400a1230 0x7fcf400a36f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.980+0000 7fcf5e2b8700 1 --2- 172.21.15.33:0/2131274553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf58100f50 0x7fcf58197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.980+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/2131274553 >> 172.21.15.33:0/2131274553 conn(0x7fcf580fb290 msgr2=0x7fcf58068250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.980+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/2131274553 shutdown_connections 2024-05-08T21:18:42.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:42.980+0000 7fcf5e2b8700 1 -- 172.21.15.33:0/2131274553 wait complete. 2024-05-08T21:18:42.996 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-05-08T21:18:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.131+0000 7f8501ce1700 1 -- 172.21.15.33:0/3030287393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84fc100f90 msgr2=0x7f84fc1013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.131+0000 7f8501ce1700 1 --2- 172.21.15.33:0/3030287393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84fc100f90 0x7f84fc1013f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f84e40099c0 tx=0x7f84e4009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.131+0000 7f8501ce1700 1 -- 172.21.15.33:0/3030287393 shutdown_connections 2024-05-08T21:18:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.131+0000 7f8501ce1700 1 --2- 172.21.15.33:0/3030287393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84fc101930 0x7f84fc1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.131+0000 7f8501ce1700 1 --2- 172.21.15.33:0/3030287393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84fc0ffda0 0x7f84fc1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.131+0000 7f8501ce1700 1 --2- 172.21.15.33:0/3030287393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84fc100f90 0x7f84fc1013f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.131+0000 7f8501ce1700 1 -- 172.21.15.33:0/3030287393 >> 172.21.15.33:0/3030287393 conn(0x7f84fc0fb330 msgr2=0x7f84fc0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.131+0000 7f8501ce1700 1 -- 172.21.15.33:0/3030287393 shutdown_connections 2024-05-08T21:18:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.131+0000 7f8501ce1700 1 -- 172.21.15.33:0/3030287393 wait complete. 2024-05-08T21:18:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f8501ce1700 1 Processor -- start 2024-05-08T21:18:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f8501ce1700 1 -- start start 2024-05-08T21:18:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f8501ce1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84fc0ffda0 0x7f84fc196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f8501ce1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84fc100f90 0x7f84fc197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f8501ce1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84fc101930 0x7f84fc19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f8501ce1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc19e820 con 0x7f84fc100f90 2024-05-08T21:18:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f8501ce1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc19e990 con 0x7f84fc0ffda0 2024-05-08T21:18:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f8501ce1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc19eb50 con 0x7f84fc101930 2024-05-08T21:18:43.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f84faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84fc100f90 0x7f84fc197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f84faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84fc100f90 0x7f84fc197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54690/0 (socket says 172.21.15.33:54690) 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f84faffd700 1 -- 172.21.15.33:0/854012042 learned_addr learned my addr 172.21.15.33:0/854012042 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84fc0ffda0 0x7f84fc196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.132+0000 7f84fbfff700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84fc101930 0x7f84fc19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.133+0000 7f84faffd700 1 -- 172.21.15.33:0/854012042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84fc101930 msgr2=0x7f84fc19e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.133+0000 7f84faffd700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84fc101930 0x7f84fc19e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.133+0000 7f84faffd700 1 -- 172.21.15.33:0/854012042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84fc0ffda0 msgr2=0x7f84fc196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.133+0000 7f84faffd700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84fc0ffda0 0x7f84fc196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.133+0000 7f84faffd700 1 -- 172.21.15.33:0/854012042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e40095f0 con 0x7f84fc100f90 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.133+0000 7f84fbfff700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84fc101930 0x7f84fc19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.133+0000 7f84faffd700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84fc100f90 0x7f84fc197240 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f84e4005910 tx=0x7f84e4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.133+0000 7f84f8ff9700 1 -- 172.21.15.33:0/854012042 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e401d070 con 0x7f84fc100f90 2024-05-08T21:18:43.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.133+0000 7f8501ce1700 1 -- 172.21.15.33:0/854012042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc19edd0 con 0x7f84fc100f90 2024-05-08T21:18:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.133+0000 7f84f8ff9700 1 -- 172.21.15.33:0/854012042 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84e400bd30 con 0x7f84fc100f90 2024-05-08T21:18:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.133+0000 7f84f8ff9700 1 -- 172.21.15.33:0/854012042 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4022d80 con 0x7f84fc100f90 2024-05-08T21:18:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.133+0000 7f8501ce1700 1 -- 172.21.15.33:0/854012042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc19f1a0 con 0x7f84fc100f90 2024-05-08T21:18:43.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.135+0000 7f84f8ff9700 1 -- 172.21.15.33:0/854012042 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f84e4022470 con 0x7f84fc100f90 2024-05-08T21:18:43.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.136+0000 7f84f8ff9700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f84e80a11b0 0x7f84e80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.136+0000 7f84f8ff9700 1 -- 172.21.15.33:0/854012042 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f84e40cdc90 con 0x7f84fc100f90 2024-05-08T21:18:43.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.136+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f84e80a11b0 0x7f84e80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f84e80a11b0 0x7f84e80a3670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0fd780 tx=0x7f84ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f850de9bb80 1 Processor -- start 2024-05-08T21:18:43.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f850de9bb80 1 -- start start 2024-05-08T21:18:43.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f850de9bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 0x55e438019910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f850de9bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e438019e50 0x55e43801a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f850de9bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e43801a810 0x55e43801ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f850de9bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e437f70100 con 0x55e4380194f0 2024-05-08T21:18:43.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f850de9bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e437f6dac0 con 0x55e438019e50 2024-05-08T21:18:43.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f850de9bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e437f693b0 con 0x55e43801a810 2024-05-08T21:18:43.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 0x55e438019910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 0x55e438019910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54700/0 (socket says 172.21.15.33:54700) 2024-05-08T21:18:43.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f84fb7fe700 1 -- 172.21.15.33:0/4284406908 learned_addr learned my addr 172.21.15.33:0/4284406908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f84fbfff700 1 --2- 172.21.15.33:0/4284406908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e43801a810 0x55e43801ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.137+0000 7f84faffd700 1 --2- 172.21.15.33:0/4284406908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e438019e50 0x55e43801a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.138+0000 7f84fb7fe700 1 -- 172.21.15.33:0/4284406908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e43801a810 msgr2=0x55e43801ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.138+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/4284406908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e43801a810 0x55e43801ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.138+0000 7f84fb7fe700 1 -- 172.21.15.33:0/4284406908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e438019e50 msgr2=0x55e43801a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.138+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/4284406908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e438019e50 0x55e43801a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.138+0000 7f84fb7fe700 1 -- 172.21.15.33:0/4284406908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e437dea360 con 0x55e4380194f0 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.138+0000 7f84faffd700 1 --2- 172.21.15.33:0/4284406908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e438019e50 0x55e43801a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.138+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/4284406908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 0x55e438019910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f84ec010f70 tx=0x7f84ec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e744e8971cfc332 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.138+0000 7f84f1ffb700 1 -- 172.21.15.33:0/4284406908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec011db0 con 0x55e4380194f0 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.138+0000 7f84f1ffb700 1 -- 172.21.15.33:0/4284406908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84ec015070 con 0x55e4380194f0 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.138+0000 7f84f1ffb700 1 -- 172.21.15.33:0/4284406908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec003d60 con 0x55e4380194f0 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.139+0000 7f850de9bb80 1 -- 172.21.15.33:0/4284406908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 msgr2=0x55e438019910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.139+0000 7f850de9bb80 1 --2- 172.21.15.33:0/4284406908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 0x55e438019910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f84ec010f70 tx=0x7f84ec013000 comp rx=0 tx=0).stop 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.139+0000 7f850de9bb80 1 -- 172.21.15.33:0/4284406908 shutdown_connections 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.139+0000 7f850de9bb80 1 --2- 172.21.15.33:0/4284406908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e43801a810 0x55e43801ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.139+0000 7f850de9bb80 1 --2- 172.21.15.33:0/4284406908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 0x55e438019910 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.139+0000 7f850de9bb80 1 --2- 172.21.15.33:0/4284406908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e438019e50 0x55e43801a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.139+0000 7f850de9bb80 1 -- 172.21.15.33:0/4284406908 >> 172.21.15.33:0/4284406908 conn(0x55e437f771c0 msgr2=0x55e438013d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.139+0000 7f850de9bb80 1 -- 172.21.15.33:0/4284406908 shutdown_connections 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.139+0000 7f850de9bb80 1 -- 172.21.15.33:0/4284406908 wait complete. 2024-05-08T21:18:43.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.139+0000 7f850de9bb80 1 -- start start 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f850de9bb80 1 Processor -- start 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f850de9bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4380194f0 0x55e438022db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f850de9bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 0x55e4380232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f850de9bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e43802a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f850de9bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e437f05510 con 0x55e438019e50 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f850de9bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e437f051b0 con 0x55e43801a810 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f850de9bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e437dec650 con 0x55e4380194f0 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f850de9bb80 10 client.0 _ll_register_callbacks cb 0x55e437f0eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 0x55e4380232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4380194f0 0x55e438022db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4380194f0 0x55e438022db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49774/0 (socket says 172.21.15.33:49774) 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 0x55e4380232f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54702/0 (socket says 172.21.15.33:54702) 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84fb7fe700 1 -- 172.21.15.33:0/2242576215 learned_addr learned my addr 172.21.15.33:0/2242576215 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84fbfff700 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e43802a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84faffd700 1 -- 172.21.15.33:0/2242576215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4380194f0 msgr2=0x55e438022db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84faffd700 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4380194f0 0x55e438022db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84faffd700 1 -- 172.21.15.33:0/2242576215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 msgr2=0x55e43802a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84faffd700 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e43802a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84faffd700 1 -- 172.21.15.33:0/2242576215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ec010c20 con 0x55e438019e50 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84fbfff700 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e43802a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4380194f0 0x55e438022db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84faffd700 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 0x55e4380232f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f84e40cb2d0 tx=0x7f84e409b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d917853fca12adb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.140+0000 7f84daffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f84daffd700 1 -- 172.21.15.33:0/2242576215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e40a2e00 con 0x55e438019e50 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f84daffd700 1 -- 172.21.15.33:0/2242576215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f84e409ead0 con 0x55e438019e50 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f850de9bb80 1 -- 172.21.15.33:0/2242576215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e437ec2220 con 0x55e438019e50 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f84daffd700 1 -- 172.21.15.33:0/2242576215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e40a3090 con 0x55e438019e50 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f850de9bb80 10 client.15438 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f850de9bb80 1 -- 172.21.15.33:0/2242576215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e437d96b10 con 0x55e438019e50 2024-05-08T21:18:43.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f84daffd700 1 -- 172.21.15.33:0/2242576215 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84e40b7080 con 0x55e438019e50 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f84daffd700 1 -- 172.21.15.33:0/2242576215 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84e40b3240 con 0x55e438019e50 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f850de9bb80 10 client.15438 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f850de9bb80 10 client.15438 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f850de9bb80 20 client.15438 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f850de9bb80 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e43802dc20 0x55e438032ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f850de9bb80 4 client.15438 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.141+0000 7f850de9bb80 1 -- 172.21.15.33:0/2242576215 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e437fe87c0 con 0x55e43802dc20 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.142+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e43802dc20 0x55e438032ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.142+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e43802dc20 0x55e438032ee0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd2a3b360a8532f5 server_cookie=603f94855b62a86a in_seq=0 out_seq=0 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.142+0000 7f84daffd700 10 client.15438 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f84daffd700 1 -- 172.21.15.33:0/2242576215 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e437fe87c0 con 0x55e43802dc20 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f84daffd700 10 client.15438 handle_command_reply: tid=0 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f850de9bb80 1 client.15438 shutdown 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f850de9bb80 1 -- 172.21.15.33:0/2242576215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 msgr2=0x55e4380232f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f850de9bb80 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 0x55e4380232f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f84e40cb2d0 tx=0x7f84e409b1e0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f850de9bb80 1 -- 172.21.15.33:0/2242576215 shutdown_connections 2024-05-08T21:18:43.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f850de9bb80 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e43802a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f850de9bb80 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e43802dc20 0x55e438032ee0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f850de9bb80 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4380194f0 0x55e438022db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f850de9bb80 1 --2- 172.21.15.33:0/2242576215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 0x55e4380232f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f84daffd700 0 client.15438 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:18:43.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f850de9bb80 1 -- 172.21.15.33:0/2242576215 >> 172.21.15.33:0/2242576215 conn(0x55e437f771c0 msgr2=0x55e438013550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f850de9bb80 1 -- 172.21.15.33:0/2242576215 shutdown_connections 2024-05-08T21:18:43.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f850de9bb80 1 -- 172.21.15.33:0/2242576215 wait complete. 2024-05-08T21:18:43.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.143+0000 7f850de9bb80 20 client.15438 trim_cache size 0 max 16384 2024-05-08T21:18:43.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.162+0000 7f850de9bb80 1 Processor -- start 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.162+0000 7f850de9bb80 1 -- start start 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.162+0000 7f850de9bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4380194f0 0x55e4380233f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.162+0000 7f850de9bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 0x55e4380222c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.162+0000 7f850de9bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e43802a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.162+0000 7f850de9bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e437fe87c0 con 0x55e438019e50 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.162+0000 7f850de9bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e437f272b0 con 0x55e43801a810 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.162+0000 7f850de9bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e437f66010 con 0x55e4380194f0 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.162+0000 7f84faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 0x55e4380222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.162+0000 7f84faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 0x55e4380222c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54706/0 (socket says 172.21.15.33:54706) 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.162+0000 7f84faffd700 1 -- 172.21.15.33:0/4232223124 learned_addr learned my addr 172.21.15.33:0/4232223124 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.163+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/4232223124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4380194f0 0x55e4380233f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.163+0000 7f84fbfff700 1 --2- 172.21.15.33:0/4232223124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e43802a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.163+0000 7f84faffd700 1 -- 172.21.15.33:0/4232223124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4380194f0 msgr2=0x55e4380233f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.163+0000 7f84faffd700 1 --2- 172.21.15.33:0/4232223124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4380194f0 0x55e4380233f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.163+0000 7f84faffd700 1 -- 172.21.15.33:0/4232223124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 msgr2=0x55e43802a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.163+0000 7f84faffd700 1 --2- 172.21.15.33:0/4232223124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e43802a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.163+0000 7f84faffd700 1 -- 172.21.15.33:0/4232223124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e437f2f510 con 0x55e438019e50 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.163+0000 7f84fbfff700 1 --2- 172.21.15.33:0/4232223124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e43802a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.163+0000 7f84faffd700 1 --2- 172.21.15.33:0/4232223124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 0x55e4380222c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009fd0 tx=0x7f84e409ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a0143fe8a3eb98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.163+0000 7f84f27fc700 1 -- 172.21.15.33:0/4232223124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e409b420 con 0x55e438019e50 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.163+0000 7f84f27fc700 1 -- 172.21.15.33:0/4232223124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e437ec2220 con 0x55e438019e50 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.163+0000 7f84f27fc700 1 -- 172.21.15.33:0/4232223124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e40aeae0 con 0x55e438019e50 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.164+0000 7f850de9bb80 1 -- 172.21.15.33:0/4232223124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 msgr2=0x55e4380222c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.164+0000 7f850de9bb80 1 --2- 172.21.15.33:0/4232223124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 0x55e4380222c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009fd0 tx=0x7f84e409ae20 comp rx=0 tx=0).stop 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.164+0000 7f850de9bb80 1 -- 172.21.15.33:0/4232223124 shutdown_connections 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.164+0000 7f850de9bb80 1 --2- 172.21.15.33:0/4232223124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e43802a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.164+0000 7f850de9bb80 1 --2- 172.21.15.33:0/4232223124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e4380194f0 0x55e4380233f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.164+0000 7f850de9bb80 1 --2- 172.21.15.33:0/4232223124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e438019e50 0x55e4380222c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.164+0000 7f850de9bb80 1 -- 172.21.15.33:0/4232223124 >> 172.21.15.33:0/4232223124 conn(0x55e437f771c0 msgr2=0x55e43801abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.164+0000 7f850de9bb80 1 -- 172.21.15.33:0/4232223124 shutdown_connections 2024-05-08T21:18:43.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.164+0000 7f850de9bb80 1 -- 172.21.15.33:0/4232223124 wait complete. 2024-05-08T21:18:43.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.164+0000 7f850de9bb80 1 -- start start 2024-05-08T21:18:43.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.164+0000 7f850de9bb80 1 Processor -- start 2024-05-08T21:18:43.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f850de9bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 0x55e438024470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f850de9bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e438019e50 0x55e4380249b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f850de9bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e438029600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f850de9bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e438029b40 con 0x55e4380194f0 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f850de9bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e438029cb0 con 0x55e43801a810 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f850de9bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e438029e70 con 0x55e438019e50 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f850de9bb80 10 client.0 _ll_register_callbacks cb 0x55e437f0eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 0x55e438024470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 0x55e438024470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54712/0 (socket says 172.21.15.33:54712) 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f84fb7fe700 1 -- 172.21.15.33:0/1883794628 learned_addr learned my addr 172.21.15.33:0/1883794628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f84fbfff700 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e438029600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f84faffd700 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e438019e50 0x55e4380249b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f84fb7fe700 1 -- 172.21.15.33:0/1883794628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e438019e50 msgr2=0x55e4380249b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e438019e50 0x55e4380249b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f84fb7fe700 1 -- 172.21.15.33:0/1883794628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 msgr2=0x55e438029600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e438029600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f84fb7fe700 1 -- 172.21.15.33:0/1883794628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e437d96b10 con 0x55e4380194f0 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f84faffd700 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e438019e50 0x55e4380249b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.165+0000 7f84fbfff700 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e438029600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f84fb7fe700 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 0x55e438024470 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f84ec011010 tx=0x7f84ec0341a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b21689b70323a1e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f84daffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f84daffd700 1 -- 172.21.15.33:0/1883794628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec02c040 con 0x55e4380194f0 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f84daffd700 1 -- 172.21.15.33:0/1883794628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f84ec017070 con 0x55e4380194f0 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f850de9bb80 1 -- 172.21.15.33:0/1883794628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84ec015070 con 0x55e4380194f0 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f84daffd700 1 -- 172.21.15.33:0/1883794628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84ec0347d0 con 0x55e4380194f0 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f850de9bb80 10 client.15444 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f850de9bb80 1 -- 172.21.15.33:0/1883794628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55e437deec80 con 0x55e4380194f0 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f84daffd700 1 -- 172.21.15.33:0/1883794628 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84ec015070 con 0x55e4380194f0 2024-05-08T21:18:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f84daffd700 1 -- 172.21.15.33:0/1883794628 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84ec03dc40 con 0x55e4380194f0 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f850de9bb80 10 client.15444 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f850de9bb80 10 client.15444 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f850de9bb80 20 client.15444 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f850de9bb80 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e438088dd0 0x55e438085760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f850de9bb80 4 client.15444 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.166+0000 7f850de9bb80 1 -- 172.21.15.33:0/1883794628 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e437f61e70 con 0x55e438088dd0 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f84faffd700 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e438088dd0 0x55e438085760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f84daffd700 10 client.15444 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f84faffd700 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e438088dd0 0x55e438085760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2be1ed274baa3295 server_cookie=d3463d1a039560b8 in_seq=0 out_seq=0 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f84daffd700 1 -- 172.21.15.33:0/1883794628 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55e437f61e70 con 0x55e438088dd0 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f84daffd700 10 client.15444 handle_command_reply: tid=0 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f850de9bb80 1 client.15444 shutdown 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f850de9bb80 1 -- 172.21.15.33:0/1883794628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 msgr2=0x55e438024470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f850de9bb80 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 0x55e438024470 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f84ec011010 tx=0x7f84ec0341a0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f850de9bb80 1 -- 172.21.15.33:0/1883794628 shutdown_connections 2024-05-08T21:18:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f850de9bb80 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e43801a810 0x55e438029600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f850de9bb80 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4380194f0 0x55e438024470 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f850de9bb80 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e438088dd0 0x55e438085760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f850de9bb80 1 --2- 172.21.15.33:0/1883794628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e438019e50 0x55e4380249b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f84daffd700 0 client.15444 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:18:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.167+0000 7f850de9bb80 1 -- 172.21.15.33:0/1883794628 >> 172.21.15.33:0/1883794628 conn(0x55e437f771c0 msgr2=0x55e438021a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f850de9bb80 1 -- 172.21.15.33:0/1883794628 shutdown_connections 2024-05-08T21:18:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f850de9bb80 1 -- 172.21.15.33:0/1883794628 wait complete. 2024-05-08T21:18:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f850de9bb80 20 client.15444 trim_cache size 0 max 16384 2024-05-08T21:18:43.176 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:43.177 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:18:43.177 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 3, 2024-05-08T21:18:43.177 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24497, 2024-05-08T21:18:43.177 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-08T21:18:43.177 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-08T21:18:43.177 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-08T21:18:43.177 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 72.821854677999994, 2024-05-08T21:18:43.177 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-05-08T21:18:43.177 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-05-08T21:18:43.177 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-05-08T21:18:43.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 118.56584399099999 2024-05-08T21:18:43.177 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f8501ce1700 1 -- 172.21.15.33:0/854012042 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f84e80a11b0 msgr2=0x7f84e80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f8501ce1700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f84e80a11b0 0x7f84e80a3670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0fd780 tx=0x7f84ec00b410 comp rx=0 tx=0).stop 2024-05-08T21:18:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f8501ce1700 1 -- 172.21.15.33:0/854012042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84fc100f90 msgr2=0x7f84fc197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f8501ce1700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84fc100f90 0x7f84fc197240 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f84e4005910 tx=0x7f84e4005b00 comp rx=0 tx=0).stop 2024-05-08T21:18:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f8501ce1700 1 -- 172.21.15.33:0/854012042 shutdown_connections 2024-05-08T21:18:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f8501ce1700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84fc101930 0x7f84fc19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f8501ce1700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f84e80a11b0 0x7f84e80a3670 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f8501ce1700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84fc0ffda0 0x7f84fc196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f8501ce1700 1 --2- 172.21.15.33:0/854012042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84fc100f90 0x7f84fc197240 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f8501ce1700 1 -- 172.21.15.33:0/854012042 >> 172.21.15.33:0/854012042 conn(0x7f84fc0fb330 msgr2=0x7f84fc104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f8501ce1700 1 -- 172.21.15.33:0/854012042 shutdown_connections 2024-05-08T21:18:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.168+0000 7f8501ce1700 1 -- 172.21.15.33:0/854012042 wait complete. 2024-05-08T21:18:43.184 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-08T21:18:43.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.323+0000 7f74cfa93700 1 -- 172.21.15.33:0/1663510870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c8100030 msgr2=0x7f74c81004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.323+0000 7f74cfa93700 1 --2- 172.21.15.33:0/1663510870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c8100030 0x7f74c81004b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009990 tx=0x7f74b8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.323+0000 7f74cfa93700 1 -- 172.21.15.33:0/1663510870 shutdown_connections 2024-05-08T21:18:43.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.323+0000 7f74cfa93700 1 --2- 172.21.15.33:0/1663510870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c81009f0 0x7f74c8100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.323+0000 7f74cfa93700 1 --2- 172.21.15.33:0/1663510870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c80ff6d0 0x7f74c80ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.323+0000 7f74cfa93700 1 --2- 172.21.15.33:0/1663510870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c8100030 0x7f74c81004b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.323+0000 7f74cfa93700 1 -- 172.21.15.33:0/1663510870 >> 172.21.15.33:0/1663510870 conn(0x7f74c80fb290 msgr2=0x7f74c80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.323+0000 7f74cfa93700 1 -- 172.21.15.33:0/1663510870 shutdown_connections 2024-05-08T21:18:43.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.323+0000 7f74cfa93700 1 -- 172.21.15.33:0/1663510870 wait complete. 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.324+0000 7f74cfa93700 1 Processor -- start 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.324+0000 7f74cfa93700 1 -- start start 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.324+0000 7f74cfa93700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c80ff6d0 0x7f74c8198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.324+0000 7f74cfa93700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c8100030 0x7f74c81994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.324+0000 7f74cfa93700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c81009f0 0x7f74c81a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.324+0000 7f74cfa93700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81a0a90 con 0x7f74c8100030 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.324+0000 7f74cfa93700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81a0bd0 con 0x7f74c80ff6d0 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.324+0000 7f74cfa93700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81a0d60 con 0x7f74c81009f0 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cd02e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c8100030 0x7f74c81994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cd02e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c8100030 0x7f74c81994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54730/0 (socket says 172.21.15.33:54730) 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cd02e700 1 -- 172.21.15.33:0/789539761 learned_addr learned my addr 172.21.15.33:0/789539761 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cd82f700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c80ff6d0 0x7f74c8198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74ce030700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c81009f0 0x7f74c81a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cd02e700 1 -- 172.21.15.33:0/789539761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c81009f0 msgr2=0x7f74c81a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cd02e700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c81009f0 0x7f74c81a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cd02e700 1 -- 172.21.15.33:0/789539761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c80ff6d0 msgr2=0x7f74c8198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cd02e700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c80ff6d0 0x7f74c8198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cd02e700 1 -- 172.21.15.33:0/789539761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b80095f0 con 0x7f74c8100030 2024-05-08T21:18:43.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cd82f700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c80ff6d0 0x7f74c8198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:43.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cd02e700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c8100030 0x7f74c81994a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f74b800b2e0 tx=0x7f74b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74beffd700 1 -- 172.21.15.33:0/789539761 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b801d070 con 0x7f74c8100030 2024-05-08T21:18:43.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cfa93700 1 -- 172.21.15.33:0/789539761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c81a0fe0 con 0x7f74c8100030 2024-05-08T21:18:43.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74beffd700 1 -- 172.21.15.33:0/789539761 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74b800beb0 con 0x7f74c8100030 2024-05-08T21:18:43.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74beffd700 1 -- 172.21.15.33:0/789539761 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8021410 con 0x7f74c8100030 2024-05-08T21:18:43.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.325+0000 7f74cfa93700 1 -- 172.21.15.33:0/789539761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c81a13a0 con 0x7f74c8100030 2024-05-08T21:18:43.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.328+0000 7f74beffd700 1 -- 172.21.15.33:0/789539761 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f74b800f640 con 0x7f74c8100030 2024-05-08T21:18:43.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.328+0000 7f74beffd700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f74b40a11b0 0x7f74b40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.328+0000 7f74beffd700 1 -- 172.21.15.33:0/789539761 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f74b80ce160 con 0x7f74c8100030 2024-05-08T21:18:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.328+0000 7f74cd82f700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f74b40a11b0 0x7f74b40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.329+0000 7f74cd82f700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f74b40a11b0 0x7f74b40a3670 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f74c4005950 tx=0x7f74c40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.329+0000 7f74dbc4db80 1 Processor -- start 2024-05-08T21:18:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.329+0000 7f74dbc4db80 1 -- start start 2024-05-08T21:18:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74dbc4db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1a4f0 0x55d68cf1a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74dbc4db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf1b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74dbc4db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf1bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74dbc4db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68ce71100 con 0x55d68cf1ae50 2024-05-08T21:18:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74dbc4db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68ce6eac0 con 0x55d68cf1b810 2024-05-08T21:18:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74dbc4db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68ce6a3b0 con 0x55d68cf1a4f0 2024-05-08T21:18:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74cd02e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf1b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74cd02e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf1b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54740/0 (socket says 172.21.15.33:54740) 2024-05-08T21:18:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74cd02e700 1 -- 172.21.15.33:0/255441829 learned_addr learned my addr 172.21.15.33:0/255441829 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74cd82f700 1 --2- 172.21.15.33:0/255441829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1a4f0 0x55d68cf1a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74ce030700 1 --2- 172.21.15.33:0/255441829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf1bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74cd02e700 1 -- 172.21.15.33:0/255441829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1a4f0 msgr2=0x55d68cf1a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74cd02e700 1 --2- 172.21.15.33:0/255441829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1a4f0 0x55d68cf1a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74cd02e700 1 -- 172.21.15.33:0/255441829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 msgr2=0x55d68cf1bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74cd02e700 1 --2- 172.21.15.33:0/255441829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf1bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74cd02e700 1 -- 172.21.15.33:0/255441829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d68cceb360 con 0x55d68cf1ae50 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74ce030700 1 --2- 172.21.15.33:0/255441829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf1bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74cd02e700 1 --2- 172.21.15.33:0/255441829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf1b2d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f74b8006010 tx=0x7f74b809ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4148e873ae079573 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.330+0000 7f74b3fff700 1 -- 172.21.15.33:0/255441829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b800f460 con 0x55d68cf1ae50 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.331+0000 7f74b3fff700 1 -- 172.21.15.33:0/255441829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74b809b8d0 con 0x55d68cf1ae50 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.331+0000 7f74b3fff700 1 -- 172.21.15.33:0/255441829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80a1070 con 0x55d68cf1ae50 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.331+0000 7f74dbc4db80 1 -- 172.21.15.33:0/255441829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 msgr2=0x55d68cf1b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.331+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/255441829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf1b2d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f74b8006010 tx=0x7f74b809ad10 comp rx=0 tx=0).stop 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.331+0000 7f74dbc4db80 1 -- 172.21.15.33:0/255441829 shutdown_connections 2024-05-08T21:18:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.331+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/255441829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf1bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.331+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/255441829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1a4f0 0x55d68cf1a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.331+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/255441829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf1b2d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.331+0000 7f74dbc4db80 1 -- 172.21.15.33:0/255441829 >> 172.21.15.33:0/255441829 conn(0x55d68ce781c0 msgr2=0x55d68cf14d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.331+0000 7f74dbc4db80 1 -- 172.21.15.33:0/255441829 shutdown_connections 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.331+0000 7f74dbc4db80 1 -- 172.21.15.33:0/255441829 wait complete. 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.332+0000 7f74dbc4db80 1 -- start start 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.332+0000 7f74dbc4db80 1 Processor -- start 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.332+0000 7f74dbc4db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1a4f0 0x55d68cf239f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.332+0000 7f74dbc4db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf23f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.332+0000 7f74dbc4db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf2af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.332+0000 7f74dbc4db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68ce06510 con 0x55d68cf1ae50 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.332+0000 7f74dbc4db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68ce061b0 con 0x55d68cf1b810 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.332+0000 7f74dbc4db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68cced650 con 0x55d68cf1a4f0 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.332+0000 7f74dbc4db80 10 client.0 _ll_register_callbacks cb 0x55d68ce0fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:43.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74cd02e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf23f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74cd02e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf23f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54742/0 (socket says 172.21.15.33:54742) 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74cd02e700 1 -- 172.21.15.33:0/1606378470 learned_addr learned my addr 172.21.15.33:0/1606378470 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74ce030700 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf2af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74cd82f700 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1a4f0 0x55d68cf239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74cd02e700 1 -- 172.21.15.33:0/1606378470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1a4f0 msgr2=0x55d68cf239f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74cd02e700 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1a4f0 0x55d68cf239f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74cd02e700 1 -- 172.21.15.33:0/1606378470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 msgr2=0x55d68cf2af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74cd02e700 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf2af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74cd02e700 1 -- 172.21.15.33:0/1606378470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c81a13a0 con 0x55d68cf1ae50 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74cd82f700 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1a4f0 0x55d68cf239f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74cd02e700 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf23f30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f74b8097c40 tx=0x7f74b8005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f61ca161cdaeeb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74b17fa700 1 -- 172.21.15.33:0/1606378470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b800f460 con 0x55d68cf1ae50 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74b17fa700 1 -- 172.21.15.33:0/1606378470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f74b809b1d0 con 0x55d68cf1ae50 2024-05-08T21:18:43.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1606378470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d68cdc3220 con 0x55d68cf1ae50 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74b17fa700 1 -- 172.21.15.33:0/1606378470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74b8099570 con 0x55d68cf1ae50 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.333+0000 7f74dbc4db80 10 client.15456 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.334+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1606378470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d68cc97b10 con 0x55d68cf1ae50 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.334+0000 7f74b17fa700 1 -- 172.21.15.33:0/1606378470 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74b8080080 con 0x55d68cf1ae50 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.334+0000 7f74b17fa700 1 -- 172.21.15.33:0/1606378470 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74b80a3130 con 0x55d68cf1ae50 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.334+0000 7f74dbc4db80 10 client.15456 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.334+0000 7f74dbc4db80 10 client.15456 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.334+0000 7f74dbc4db80 20 client.15456 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.334+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55d68cf2e7e0 0x55d68cf33a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.334+0000 7f74dbc4db80 4 client.15456 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.334+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1606378470 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d68cee97c0 con 0x55d68cf2e7e0 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.334+0000 7f74cd82f700 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55d68cf2e7e0 0x55d68cf33a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.335+0000 7f74b17fa700 10 client.15456 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:18:43.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.335+0000 7f74cd82f700 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55d68cf2e7e0 0x55d68cf33a80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c40fac2931a41873 server_cookie=544296f26e7f16a0 in_seq=0 out_seq=0 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.335+0000 7f74b17fa700 1 -- 172.21.15.33:0/1606378470 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d68cee97c0 con 0x55d68cf2e7e0 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.335+0000 7f74b17fa700 10 client.15456 handle_command_reply: tid=0 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.335+0000 7f74dbc4db80 1 client.15456 shutdown 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.335+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1606378470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 msgr2=0x55d68cf23f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.335+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf23f30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f74b8097c40 tx=0x7f74b8005ac0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.336+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1606378470 shutdown_connections 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.336+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55d68cf2e7e0 0x55d68cf33a80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.336+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf2af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.336+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1a4f0 0x55d68cf239f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.336+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/1606378470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf23f30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.336+0000 7f74b17fa700 0 client.15456 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.336+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1606378470 >> 172.21.15.33:0/1606378470 conn(0x55d68ce781c0 msgr2=0x55d68cf14550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.336+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1606378470 shutdown_connections 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.336+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1606378470 wait complete. 2024-05-08T21:18:43.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.336+0000 7f74dbc4db80 20 client.15456 trim_cache size 0 max 16384 2024-05-08T21:18:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74dbc4db80 1 Processor -- start 2024-05-08T21:18:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74dbc4db80 1 -- start start 2024-05-08T21:18:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74dbc4db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1a4f0 0x55d68cf22f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74dbc4db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf2af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74dbc4db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1b810 0x55d68cf111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74dbc4db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68ce71100 con 0x55d68cf1ae50 2024-05-08T21:18:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74dbc4db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68cf30030 con 0x55d68cf1a4f0 2024-05-08T21:18:43.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74dbc4db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68cee97c0 con 0x55d68cf1b810 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74cd02e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf2af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74cd02e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf2af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54754/0 (socket says 172.21.15.33:54754) 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74cd02e700 1 -- 172.21.15.33:0/1745990132 learned_addr learned my addr 172.21.15.33:0/1745990132 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74ce030700 1 --2- 172.21.15.33:0/1745990132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1b810 0x55d68cf111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74cd82f700 1 --2- 172.21.15.33:0/1745990132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1a4f0 0x55d68cf22f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74cd02e700 1 -- 172.21.15.33:0/1745990132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1b810 msgr2=0x55d68cf111c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74cd02e700 1 --2- 172.21.15.33:0/1745990132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1b810 0x55d68cf111c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74cd02e700 1 -- 172.21.15.33:0/1745990132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1a4f0 msgr2=0x55d68cf22f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74cd02e700 1 --2- 172.21.15.33:0/1745990132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1a4f0 0x55d68cf22f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.357+0000 7f74cd02e700 1 -- 172.21.15.33:0/1745990132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d68ce30510 con 0x55d68cf1ae50 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.358+0000 7f74cd02e700 1 --2- 172.21.15.33:0/1745990132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf2af30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f74b809afc0 tx=0x7f74b8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8983ae42a23ebf52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.358+0000 7f74bcda4700 1 -- 172.21.15.33:0/1745990132 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b809ad10 con 0x55d68cf1ae50 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.358+0000 7f74bcda4700 1 -- 172.21.15.33:0/1745990132 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d68cdc3220 con 0x55d68cf1ae50 2024-05-08T21:18:43.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.358+0000 7f74bcda4700 1 -- 172.21.15.33:0/1745990132 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80a35c0 con 0x55d68cf1ae50 2024-05-08T21:18:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.359+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1745990132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 msgr2=0x55d68cf2af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.359+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/1745990132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf2af30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f74b809afc0 tx=0x7f74b8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.359+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1745990132 shutdown_connections 2024-05-08T21:18:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.359+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/1745990132 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1b810 0x55d68cf111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.359+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/1745990132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1a4f0 0x55d68cf22f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.359+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/1745990132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1ae50 0x55d68cf2af30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.359+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1745990132 >> 172.21.15.33:0/1745990132 conn(0x55d68ce781c0 msgr2=0x55d68cf227b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.359+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1745990132 shutdown_connections 2024-05-08T21:18:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.359+0000 7f74dbc4db80 1 -- 172.21.15.33:0/1745990132 wait complete. 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.359+0000 7f74dbc4db80 1 -- start start 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.359+0000 7f74dbc4db80 1 Processor -- start 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74dbc4db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1a4f0 0x55d68cf27160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74dbc4db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1ae50 0x55d68cf296b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74dbc4db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf29ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74dbc4db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68cf2a420 con 0x55d68cf1a4f0 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74dbc4db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68cf2a590 con 0x55d68cf1b810 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74dbc4db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68cf2a750 con 0x55d68cf1ae50 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74dbc4db80 10 client.0 _ll_register_callbacks cb 0x55d68ce0fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74cd82f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1a4f0 0x55d68cf27160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74cd82f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1a4f0 0x55d68cf27160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54758/0 (socket says 172.21.15.33:54758) 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74cd82f700 1 -- 172.21.15.33:0/3566399294 learned_addr learned my addr 172.21.15.33:0/3566399294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74ce030700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf29ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74cd02e700 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1ae50 0x55d68cf296b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74cd02e700 1 -- 172.21.15.33:0/3566399294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 msgr2=0x55d68cf29ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74cd02e700 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf29ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74cd02e700 1 -- 172.21.15.33:0/3566399294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1a4f0 msgr2=0x55d68cf27160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74cd02e700 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1a4f0 0x55d68cf27160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74cd02e700 1 -- 172.21.15.33:0/3566399294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d68cc97b10 con 0x55d68cf1ae50 2024-05-08T21:18:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74cd82f700 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1a4f0 0x55d68cf27160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74cd02e700 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1ae50 0x55d68cf296b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f74b8097c40 tx=0x7f74b80a3360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed07edff93184fa0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.360+0000 7f74b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:18:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74b17fa700 1 -- 172.21.15.33:0/3566399294 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80b94d0 con 0x55d68cf1ae50 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74dbc4db80 1 -- 172.21.15.33:0/3566399294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74b809b8d0 con 0x55d68cf1ae50 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74b17fa700 1 -- 172.21.15.33:0/3566399294 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f74b80a35c0 con 0x55d68cf1ae50 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74b17fa700 1 -- 172.21.15.33:0/3566399294 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74b809ad10 con 0x55d68cf1ae50 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74dbc4db80 10 client.25354 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74dbc4db80 1 -- 172.21.15.33:0/3566399294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d68ccefc80 con 0x55d68cf1ae50 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74b17fa700 1 -- 172.21.15.33:0/3566399294 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74b809b8d0 con 0x55d68cf1ae50 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74b17fa700 1 -- 172.21.15.33:0/3566399294 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74b8096fc0 con 0x55d68cf1ae50 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74dbc4db80 10 client.25354 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74dbc4db80 10 client.25354 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74dbc4db80 20 client.25354 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55d68cf88ea0 0x55d68cf85be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74dbc4db80 4 client.25354 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.361+0000 7f74dbc4db80 1 -- 172.21.15.33:0/3566399294 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d68ce62e70 con 0x55d68cf88ea0 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.362+0000 7f74cd82f700 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55d68cf88ea0 0x55d68cf85be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.362+0000 7f74b17fa700 10 client.25354 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.362+0000 7f74cd82f700 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55d68cf88ea0 0x55d68cf85be0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=598f87dc56047db4 server_cookie=c250956e67c5d39a in_seq=0 out_seq=0 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.362+0000 7f74b17fa700 1 -- 172.21.15.33:0/3566399294 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55d68ce62e70 con 0x55d68cf88ea0 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.362+0000 7f74b17fa700 10 client.25354 handle_command_reply: tid=0 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.362+0000 7f74dbc4db80 1 client.25354 shutdown 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.362+0000 7f74dbc4db80 1 -- 172.21.15.33:0/3566399294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1ae50 msgr2=0x55d68cf296b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.362+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1ae50 0x55d68cf296b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f74b8097c40 tx=0x7f74b80a3360 comp rx=0 tx=0).stop 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.362+0000 7f74dbc4db80 1 -- 172.21.15.33:0/3566399294 shutdown_connections 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.362+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55d68cf88ea0 0x55d68cf85be0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68cf1b810 0x55d68cf29ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d68cf1a4f0 0x55d68cf27160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74dbc4db80 1 --2- 172.21.15.33:0/3566399294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d68cf1ae50 0x55d68cf296b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74b17fa700 0 client.25354 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74dbc4db80 1 -- 172.21.15.33:0/3566399294 >> 172.21.15.33:0/3566399294 conn(0x55d68ce781c0 msgr2=0x55d68cf227b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74dbc4db80 1 -- 172.21.15.33:0/3566399294 shutdown_connections 2024-05-08T21:18:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74dbc4db80 1 -- 172.21.15.33:0/3566399294 wait complete. 2024-05-08T21:18:43.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74dbc4db80 20 client.25354 trim_cache size 0 max 16384 2024-05-08T21:18:43.371 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:43.371 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:18:43.371 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-05-08T21:18:43.371 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24499, 2024-05-08T21:18:43.371 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-08T21:18:43.371 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-08T21:18:43.371 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-08T21:18:43.371 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 74.031509454000002, 2024-05-08T21:18:43.371 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-05-08T21:18:43.371 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-05-08T21:18:43.372 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-05-08T21:18:43.372 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 115.279999139 2024-05-08T21:18:43.372 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74cfa93700 1 -- 172.21.15.33:0/789539761 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f74b40a11b0 msgr2=0x7f74b40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74cfa93700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f74b40a11b0 0x7f74b40a3670 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f74c4005950 tx=0x7f74c40058e0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74cfa93700 1 -- 172.21.15.33:0/789539761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c8100030 msgr2=0x7f74c81994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74cfa93700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c8100030 0x7f74c81994a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f74b800b2e0 tx=0x7f74b8004730 comp rx=0 tx=0).stop 2024-05-08T21:18:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74cfa93700 1 -- 172.21.15.33:0/789539761 shutdown_connections 2024-05-08T21:18:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74cfa93700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74c81009f0 0x7f74c81a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74cfa93700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f74b40a11b0 0x7f74b40a3670 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74cfa93700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c80ff6d0 0x7f74c8198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74cfa93700 1 --2- 172.21.15.33:0/789539761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c8100030 0x7f74c81994a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74cfa93700 1 -- 172.21.15.33:0/789539761 >> 172.21.15.33:0/789539761 conn(0x7f74c80fb290 msgr2=0x7f74c80fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74cfa93700 1 -- 172.21.15.33:0/789539761 shutdown_connections 2024-05-08T21:18:43.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.363+0000 7f74cfa93700 1 -- 172.21.15.33:0/789539761 wait complete. 2024-05-08T21:18:43.379 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-05-08T21:18:43.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:43 smithi033 ceph-mon[42528]: pgmap v280: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:18:43.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:43 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2295732255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:43.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:43 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2464078900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.517+0000 7f0fad446700 1 -- 172.21.15.33:0/3053442066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fa81021e0 msgr2=0x7f0fa8102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.517+0000 7f0fad446700 1 --2- 172.21.15.33:0/3053442066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fa81021e0 0x7f0fa8102640 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009990 tx=0x7f0f9c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.517+0000 7f0fad446700 1 -- 172.21.15.33:0/3053442066 shutdown_connections 2024-05-08T21:18:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.517+0000 7f0fad446700 1 --2- 172.21.15.33:0/3053442066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8102b80 0x7f0fa8109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.517+0000 7f0fad446700 1 --2- 172.21.15.33:0/3053442066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa80ff170 0x7f0fa80ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.517+0000 7f0fad446700 1 --2- 172.21.15.33:0/3053442066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fa81021e0 0x7f0fa8102640 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.517+0000 7f0fad446700 1 -- 172.21.15.33:0/3053442066 >> 172.21.15.33:0/3053442066 conn(0x7f0fa80faa60 msgr2=0x7f0fa80fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.517+0000 7f0fad446700 1 -- 172.21.15.33:0/3053442066 shutdown_connections 2024-05-08T21:18:43.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.517+0000 7f0fad446700 1 -- 172.21.15.33:0/3053442066 wait complete. 2024-05-08T21:18:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fad446700 1 Processor -- start 2024-05-08T21:18:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fad446700 1 -- start start 2024-05-08T21:18:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fad446700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fa80ff170 0x7f0fa8196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fad446700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa81021e0 0x7f0fa8197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fad446700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8102b80 0x7f0fa819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fad446700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa819e830 con 0x7f0fa80ff170 2024-05-08T21:18:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fad446700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa819e9a0 con 0x7f0fa8102b80 2024-05-08T21:18:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fad446700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa819eb60 con 0x7f0fa81021e0 2024-05-08T21:18:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fa6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fa80ff170 0x7f0fa8196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fa6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fa80ff170 0x7f0fa8196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54780/0 (socket says 172.21.15.33:54780) 2024-05-08T21:18:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fa6ffd700 1 -- 172.21.15.33:0/3805640323 learned_addr learned my addr 172.21.15.33:0/3805640323 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fa67fc700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa81021e0 0x7f0fa8197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.518+0000 7f0fa77fe700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8102b80 0x7f0fa819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.519+0000 7f0fa6ffd700 1 -- 172.21.15.33:0/3805640323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa81021e0 msgr2=0x7f0fa8197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.519+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa81021e0 0x7f0fa8197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.519+0000 7f0fa6ffd700 1 -- 172.21.15.33:0/3805640323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8102b80 msgr2=0x7f0fa819e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.519+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8102b80 0x7f0fa819e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.519+0000 7f0fa6ffd700 1 -- 172.21.15.33:0/3805640323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9c0095f0 con 0x7f0fa80ff170 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.519+0000 7f0fa77fe700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8102b80 0x7f0fa819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.519+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fa80ff170 0x7f0fa8196d40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800d980 tx=0x7f0f9800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.519+0000 7f0f8ffff700 1 -- 172.21.15.33:0/3805640323 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9800de60 con 0x7f0fa80ff170 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.519+0000 7f0fad446700 1 -- 172.21.15.33:0/3805640323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa819ee40 con 0x7f0fa80ff170 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.519+0000 7f0f8ffff700 1 -- 172.21.15.33:0/3805640323 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f980044b0 con 0x7f0fa80ff170 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.519+0000 7f0f8ffff700 1 -- 172.21.15.33:0/3805640323 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98003b40 con 0x7f0fa80ff170 2024-05-08T21:18:43.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.519+0000 7f0fad446700 1 -- 172.21.15.33:0/3805640323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa819f270 con 0x7f0fa80ff170 2024-05-08T21:18:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.522+0000 7f0f8ffff700 1 -- 172.21.15.33:0/3805640323 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f0f98003ca0 con 0x7f0fa80ff170 2024-05-08T21:18:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.522+0000 7f0f8ffff700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0f900a11b0 0x7f0f900a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.522+0000 7f0fa67fc700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0f900a11b0 0x7f0f900a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.522+0000 7f0f8ffff700 1 -- 172.21.15.33:0/3805640323 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0f980cca80 con 0x7f0fa80ff170 2024-05-08T21:18:43.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.523+0000 7f0fa67fc700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0f900a11b0 0x7f0f900a3670 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c003780 tx=0x7f0f9c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.523+0000 7f0fb9600b80 1 Processor -- start 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.523+0000 7f0fb9600b80 1 -- start start 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.523+0000 7f0fb9600b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddc4f0 0x55ff4eddc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.523+0000 7f0fb9600b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddce50 0x55ff4eddd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.523+0000 7f0fb9600b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddd810 0x55ff4edddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.523+0000 7f0fb9600b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4ed33100 con 0x55ff4eddce50 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.523+0000 7f0fb9600b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4ed30ac0 con 0x55ff4eddc4f0 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.523+0000 7f0fb9600b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4ed2c3b0 con 0x55ff4eddd810 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.523+0000 7f0fa67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddce50 0x55ff4eddd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.523+0000 7f0fa67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddce50 0x55ff4eddd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54788/0 (socket says 172.21.15.33:54788) 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.523+0000 7f0fa67fc700 1 -- 172.21.15.33:0/266566846 learned_addr learned my addr 172.21.15.33:0/266566846 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.524+0000 7f0fa77fe700 1 --2- 172.21.15.33:0/266566846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddd810 0x55ff4edddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.524+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/266566846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddc4f0 0x55ff4eddc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.524+0000 7f0fa67fc700 1 -- 172.21.15.33:0/266566846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddd810 msgr2=0x55ff4edddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.524+0000 7f0fa67fc700 1 --2- 172.21.15.33:0/266566846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddd810 0x55ff4edddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.524+0000 7f0fa67fc700 1 -- 172.21.15.33:0/266566846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddc4f0 msgr2=0x55ff4eddc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.524+0000 7f0fa67fc700 1 --2- 172.21.15.33:0/266566846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddc4f0 0x55ff4eddc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.524+0000 7f0fa67fc700 1 -- 172.21.15.33:0/266566846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff4ebad360 con 0x55ff4eddce50 2024-05-08T21:18:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.524+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/266566846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddc4f0 0x55ff4eddc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.524+0000 7f0fa67fc700 1 --2- 172.21.15.33:0/266566846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddce50 0x55ff4eddd2d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c015000 tx=0x7f0f9c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c90ea4e69cb4bbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.524+0000 7f0f8d7fa700 1 -- 172.21.15.33:0/266566846 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c02b070 con 0x55ff4eddce50 2024-05-08T21:18:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.524+0000 7f0f8d7fa700 1 -- 172.21.15.33:0/266566846 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0f9c01e040 con 0x55ff4eddce50 2024-05-08T21:18:43.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.524+0000 7f0f8d7fa700 1 -- 172.21.15.33:0/266566846 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c004b40 con 0x55ff4eddce50 2024-05-08T21:18:43.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.525+0000 7f0fb9600b80 1 -- 172.21.15.33:0/266566846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddce50 msgr2=0x55ff4eddd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.525+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/266566846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddce50 0x55ff4eddd2d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c015000 tx=0x7f0f9c01f000 comp rx=0 tx=0).stop 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.525+0000 7f0fb9600b80 1 -- 172.21.15.33:0/266566846 shutdown_connections 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.525+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/266566846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddd810 0x55ff4edddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.525+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/266566846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddc4f0 0x55ff4eddc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.525+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/266566846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddce50 0x55ff4eddd2d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.525+0000 7f0fb9600b80 1 -- 172.21.15.33:0/266566846 >> 172.21.15.33:0/266566846 conn(0x55ff4ed3a1c0 msgr2=0x55ff4edd6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.525+0000 7f0fb9600b80 1 -- 172.21.15.33:0/266566846 shutdown_connections 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.525+0000 7f0fb9600b80 1 -- 172.21.15.33:0/266566846 wait complete. 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.525+0000 7f0fb9600b80 1 -- start start 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fb9600b80 1 Processor -- start 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fb9600b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddc4f0 0x55ff4ede5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fb9600b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 0x55ff4ede61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fb9600b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 0x55ff4eded210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fb9600b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4ecc8510 con 0x55ff4eddd810 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fb9600b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4ecc81b0 con 0x55ff4eddce50 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fb9600b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4ebaf650 con 0x55ff4eddc4f0 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fb9600b80 10 client.0 _ll_register_callbacks cb 0x55ff4ecd1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fa77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 0x55ff4eded210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fa77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 0x55ff4eded210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54790/0 (socket says 172.21.15.33:54790) 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fa77fe700 1 -- 172.21.15.33:0/3146619671 learned_addr learned my addr 172.21.15.33:0/3146619671 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddc4f0 0x55ff4ede5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fa67fc700 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 0x55ff4ede61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fa77fe700 1 -- 172.21.15.33:0/3146619671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddc4f0 msgr2=0x55ff4ede5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fa77fe700 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddc4f0 0x55ff4ede5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fa77fe700 1 -- 172.21.15.33:0/3146619671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 msgr2=0x55ff4ede61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fa77fe700 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 0x55ff4ede61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fa77fe700 1 -- 172.21.15.33:0/3146619671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa8109c20 con 0x55ff4eddd810 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0fa77fe700 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 0x55ff4eded210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0007b90 tx=0x7f0fa0007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a0aac623d774ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.526+0000 7f0f8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0f8aff5700 1 -- 172.21.15.33:0/3146619671 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0012c60 con 0x55ff4eddd810 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0fb9600b80 1 -- 172.21.15.33:0/3146619671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff4ec85220 con 0x55ff4eddd810 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0f8aff5700 1 -- 172.21.15.33:0/3146619671 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0fa0012dc0 con 0x55ff4eddd810 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0f8aff5700 1 -- 172.21.15.33:0/3146619671 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fa0016420 con 0x55ff4eddd810 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0fb9600b80 10 client.15474 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0fb9600b80 1 -- 172.21.15.33:0/3146619671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ff4eb59b10 con 0x55ff4eddd810 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0f8aff5700 1 -- 172.21.15.33:0/3146619671 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ff4ec85220 con 0x55ff4eddd810 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0f8aff5700 1 -- 172.21.15.33:0/3146619671 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0fa0014040 con 0x55ff4eddd810 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0fb9600b80 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0fb9600b80 10 client.15474 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0fb9600b80 20 client.15474 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55ff4edf03c0 0x55ff4edf61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0fb9600b80 4 client.15474 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:18:43.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.527+0000 7f0fb9600b80 1 -- 172.21.15.33:0/3146619671 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff4edab7c0 con 0x55ff4edf03c0 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.528+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55ff4edf03c0 0x55ff4edf61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.528+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55ff4edf03c0 0x55ff4edf61e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81db3df5ba16da35 server_cookie=5fc30c03fd88edfb in_seq=0 out_seq=0 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.528+0000 7f0f8aff5700 10 client.15474 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0f8aff5700 1 -- 172.21.15.33:0/3146619671 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff4edab7c0 con 0x55ff4edf03c0 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0f8aff5700 10 client.15474 handle_command_reply: tid=0 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0fb9600b80 1 client.15474 shutdown 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0fb9600b80 1 -- 172.21.15.33:0/3146619671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 msgr2=0x55ff4eded210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 0x55ff4eded210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0007b90 tx=0x7f0fa0007ea0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0fb9600b80 1 -- 172.21.15.33:0/3146619671 shutdown_connections 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55ff4edf03c0 0x55ff4edf61e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 0x55ff4eded210 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddc4f0 0x55ff4ede5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0f8aff5700 0 client.15474 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/3146619671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 0x55ff4ede61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0fb9600b80 1 -- 172.21.15.33:0/3146619671 >> 172.21.15.33:0/3146619671 conn(0x55ff4ed3a1c0 msgr2=0x55ff4edd6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0fb9600b80 1 -- 172.21.15.33:0/3146619671 shutdown_connections 2024-05-08T21:18:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0fb9600b80 1 -- 172.21.15.33:0/3146619671 wait complete. 2024-05-08T21:18:43.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.529+0000 7f0fb9600b80 20 client.15474 trim_cache size 0 max 16384 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.548+0000 7f0fb9600b80 1 Processor -- start 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.548+0000 7f0fb9600b80 1 -- start start 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.548+0000 7f0fb9600b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddc4f0 0x55ff4eddc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.548+0000 7f0fb9600b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 0x55ff4eded130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.548+0000 7f0fb9600b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 0x55ff4ede51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.548+0000 7f0fb9600b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4ed29010 con 0x55ff4eddd810 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.548+0000 7f0fb9600b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4eccb5a0 con 0x55ff4eddce50 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.548+0000 7f0fb9600b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4edeefa0 con 0x55ff4eddc4f0 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0fa77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 0x55ff4ede51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0fa77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 0x55ff4ede51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54798/0 (socket says 172.21.15.33:54798) 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0fa77fe700 1 -- 172.21.15.33:0/90080922 learned_addr learned my addr 172.21.15.33:0/90080922 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0fa67fc700 1 --2- 172.21.15.33:0/90080922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 0x55ff4eded130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/90080922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddc4f0 0x55ff4eddc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0fa77fe700 1 -- 172.21.15.33:0/90080922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddc4f0 msgr2=0x55ff4eddc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0fa77fe700 1 --2- 172.21.15.33:0/90080922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddc4f0 0x55ff4eddc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0fa77fe700 1 -- 172.21.15.33:0/90080922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 msgr2=0x55ff4eded130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0fa77fe700 1 --2- 172.21.15.33:0/90080922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 0x55ff4eded130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0fa77fe700 1 -- 172.21.15.33:0/90080922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff4ecf2510 con 0x55ff4eddd810 2024-05-08T21:18:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/90080922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddc4f0 0x55ff4eddc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0fa77fe700 1 --2- 172.21.15.33:0/90080922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 0x55ff4ede51a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009a70 tx=0x7f0fa0009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0deca27c1e4f5f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0f8dffb700 1 -- 172.21.15.33:0/90080922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0015ca0 con 0x55ff4eddd810 2024-05-08T21:18:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0f8dffb700 1 -- 172.21.15.33:0/90080922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fa000f040 con 0x55ff4eddd810 2024-05-08T21:18:43.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.549+0000 7f0f8dffb700 1 -- 172.21.15.33:0/90080922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0012e80 con 0x55ff4eddd810 2024-05-08T21:18:43.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.550+0000 7f0fb9600b80 1 -- 172.21.15.33:0/90080922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 msgr2=0x55ff4ede51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.550+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/90080922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 0x55ff4ede51a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009a70 tx=0x7f0fa0009aa0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.550+0000 7f0fb9600b80 1 -- 172.21.15.33:0/90080922 shutdown_connections 2024-05-08T21:18:43.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.550+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/90080922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddd810 0x55ff4ede51a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.550+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/90080922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddc4f0 0x55ff4eddc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.550+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/90080922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 0x55ff4eded130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.550+0000 7f0fb9600b80 1 -- 172.21.15.33:0/90080922 >> 172.21.15.33:0/90080922 conn(0x55ff4ed3a1c0 msgr2=0x55ff4edddbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.550+0000 7f0fb9600b80 1 -- 172.21.15.33:0/90080922 shutdown_connections 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.550+0000 7f0fb9600b80 1 -- 172.21.15.33:0/90080922 wait complete. 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.550+0000 7f0fb9600b80 1 -- start start 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.550+0000 7f0fb9600b80 1 Processor -- start 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fb9600b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddc4f0 0x55ff4edd6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fb9600b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 0x55ff4edec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fb9600b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddd810 0x55ff4edec6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fb9600b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4edecdc0 con 0x55ff4eddc4f0 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fb9600b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4edecf30 con 0x55ff4eddce50 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fb9600b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff4eded0f0 con 0x55ff4eddd810 2024-05-08T21:18:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fb9600b80 10 client.0 _ll_register_callbacks cb 0x55ff4ecd1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fa6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddc4f0 0x55ff4edd6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fa6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddc4f0 0x55ff4edd6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54814/0 (socket says 172.21.15.33:54814) 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fa6ffd700 1 -- 172.21.15.33:0/673040814 learned_addr learned my addr 172.21.15.33:0/673040814 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fa67fc700 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 0x55ff4edec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fa77fe700 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddd810 0x55ff4edec6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fa6ffd700 1 -- 172.21.15.33:0/673040814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddd810 msgr2=0x55ff4edec6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddd810 0x55ff4edec6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fa6ffd700 1 -- 172.21.15.33:0/673040814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 msgr2=0x55ff4edec170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 0x55ff4edec170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fa6ffd700 1 -- 172.21.15.33:0/673040814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff4eb59b10 con 0x55ff4eddc4f0 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fa6ffd700 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddc4f0 0x55ff4edd6660 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0f98095f60 tx=0x7f0f98005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2535e4ab55d2bbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0f8aff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0f8aff5700 1 -- 172.21.15.33:0/673040814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f980a1ee0 con 0x55ff4eddc4f0 2024-05-08T21:18:43.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0f8aff5700 1 -- 172.21.15.33:0/673040814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0f980b1030 con 0x55ff4eddc4f0 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fb9600b80 1 -- 172.21.15.33:0/673040814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff4ede4ca0 con 0x55ff4eddc4f0 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0f8aff5700 1 -- 172.21.15.33:0/673040814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f980b2290 con 0x55ff4eddc4f0 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.551+0000 7f0fb9600b80 10 client.15480 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.552+0000 7f0fb9600b80 1 -- 172.21.15.33:0/673040814 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ff4ebb1c80 con 0x55ff4eddc4f0 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.552+0000 7f0f8aff5700 1 -- 172.21.15.33:0/673040814 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ff4ede4ca0 con 0x55ff4eddc4f0 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.552+0000 7f0f8aff5700 1 -- 172.21.15.33:0/673040814 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0f98098f50 con 0x55ff4eddc4f0 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.552+0000 7f0fb9600b80 10 client.15480 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.552+0000 7f0fb9600b80 10 client.15480 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.552+0000 7f0fb9600b80 20 client.15480 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.552+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55ff4ee4c570 0x55ff4ee48e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.552+0000 7f0fb9600b80 4 client.15480 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.552+0000 7f0fb9600b80 1 -- 172.21.15.33:0/673040814 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f0f9c01e040 con 0x55ff4ee4c570 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.552+0000 7f0fa67fc700 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55ff4ee4c570 0x55ff4ee48e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fa67fc700 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55ff4ee4c570 0x55ff4ee48e60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=493cd5484cbad088 server_cookie=590eec43e138799b in_seq=0 out_seq=0 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0f8aff5700 10 client.15480 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0f8aff5700 1 -- 172.21.15.33:0/673040814 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x7f0f9c01e040 con 0x55ff4ee4c570 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0f8aff5700 10 client.15480 handle_command_reply: tid=0 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fb9600b80 1 client.15480 shutdown 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fb9600b80 1 -- 172.21.15.33:0/673040814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddc4f0 msgr2=0x55ff4edd6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddc4f0 0x55ff4edd6660 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0f98095f60 tx=0x7f0f98005da0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fb9600b80 1 -- 172.21.15.33:0/673040814 shutdown_connections 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55ff4ee4c570 0x55ff4ee48e60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff4eddd810 0x55ff4edec6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff4eddc4f0 0x55ff4edd6660 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0f8aff5700 0 client.15480 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fb9600b80 1 --2- 172.21.15.33:0/673040814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff4eddce50 0x55ff4edec170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fb9600b80 1 -- 172.21.15.33:0/673040814 >> 172.21.15.33:0/673040814 conn(0x55ff4ed3a1c0 msgr2=0x55ff4ede06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fb9600b80 1 -- 172.21.15.33:0/673040814 shutdown_connections 2024-05-08T21:18:43.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fb9600b80 1 -- 172.21.15.33:0/673040814 wait complete. 2024-05-08T21:18:43.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.553+0000 7f0fb9600b80 20 client.15480 trim_cache size 0 max 16384 2024-05-08T21:18:43.562 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:43.562 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:18:43.562 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-05-08T21:18:43.562 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24521, 2024-05-08T21:18:43.562 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-08T21:18:43.562 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-08T21:18:43.562 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-08T21:18:43.562 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 75.229473655000007, 2024-05-08T21:18:43.562 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-05-08T21:18:43.562 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-05-08T21:18:43.563 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-05-08T21:18:43.563 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 113.68540335 2024-05-08T21:18:43.563 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:43.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.554+0000 7f0fad446700 1 -- 172.21.15.33:0/3805640323 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0f900a11b0 msgr2=0x7f0f900a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.554+0000 7f0fad446700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0f900a11b0 0x7f0f900a3670 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c003780 tx=0x7f0f9c005fd0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.554+0000 7f0fad446700 1 -- 172.21.15.33:0/3805640323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fa80ff170 msgr2=0x7f0fa8196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.554+0000 7f0fad446700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fa80ff170 0x7f0fa8196d40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800d980 tx=0x7f0f9800dd40 comp rx=0 tx=0).stop 2024-05-08T21:18:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.554+0000 7f0fad446700 1 -- 172.21.15.33:0/3805640323 shutdown_connections 2024-05-08T21:18:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.554+0000 7f0fad446700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0f900a11b0 0x7f0f900a3670 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.554+0000 7f0fad446700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fa8102b80 0x7f0fa819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.554+0000 7f0fad446700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fa80ff170 0x7f0fa8196d40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.554+0000 7f0fad446700 1 --2- 172.21.15.33:0/3805640323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa81021e0 0x7f0fa8197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.554+0000 7f0fad446700 1 -- 172.21.15.33:0/3805640323 >> 172.21.15.33:0/3805640323 conn(0x7f0fa80faa60 msgr2=0x7f0fa8105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.554+0000 7f0fad446700 1 -- 172.21.15.33:0/3805640323 shutdown_connections 2024-05-08T21:18:43.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.554+0000 7f0fad446700 1 -- 172.21.15.33:0/3805640323 wait complete. 2024-05-08T21:18:43.570 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-05-08T21:18:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.706+0000 7f01f9d77700 1 -- 172.21.15.33:0/1902861953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4101980 msgr2=0x7f01f4103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.706+0000 7f01f9d77700 1 --2- 172.21.15.33:0/1902861953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4101980 0x7f01f4103d70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009990 tx=0x7f01e4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.706+0000 7f01f9d77700 1 -- 172.21.15.33:0/1902861953 shutdown_connections 2024-05-08T21:18:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.706+0000 7f01f9d77700 1 --2- 172.21.15.33:0/1902861953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f4106be0 0x7f01f4108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.706+0000 7f01f9d77700 1 --2- 172.21.15.33:0/1902861953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4101980 0x7f01f4103d70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.706+0000 7f01f9d77700 1 --2- 172.21.15.33:0/1902861953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f41042b0 0x7f01f41066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.706+0000 7f01f9d77700 1 -- 172.21.15.33:0/1902861953 >> 172.21.15.33:0/1902861953 conn(0x7f01f40fb290 msgr2=0x7f01f40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.706+0000 7f01f9d77700 1 -- 172.21.15.33:0/1902861953 shutdown_connections 2024-05-08T21:18:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.706+0000 7f01f9d77700 1 -- 172.21.15.33:0/1902861953 wait complete. 2024-05-08T21:18:43.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.707+0000 7f01f9d77700 1 Processor -- start 2024-05-08T21:18:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.707+0000 7f01f9d77700 1 -- start start 2024-05-08T21:18:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.707+0000 7f01f9d77700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4101980 0x7f01f4196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.707+0000 7f01f9d77700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f41042b0 0x7f01f4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.707+0000 7f01f9d77700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f4106be0 0x7f01f419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.707+0000 7f01f9d77700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f419e7b0 con 0x7f01f4101980 2024-05-08T21:18:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.707+0000 7f01f9d77700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f419e920 con 0x7f01f41042b0 2024-05-08T21:18:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.707+0000 7f01f9d77700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f419eae0 con 0x7f01f4106be0 2024-05-08T21:18:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.707+0000 7f01f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4101980 0x7f01f4196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.707+0000 7f01f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4101980 0x7f01f4196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54830/0 (socket says 172.21.15.33:54830) 2024-05-08T21:18:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.707+0000 7f01f37fe700 1 -- 172.21.15.33:0/4048063438 learned_addr learned my addr 172.21.15.33:0/4048063438 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.707+0000 7f01f3fff700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f4106be0 0x7f01f419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f2ffd700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f41042b0 0x7f01f4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f37fe700 1 -- 172.21.15.33:0/4048063438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f4106be0 msgr2=0x7f01f419e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f37fe700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f4106be0 0x7f01f419e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f37fe700 1 -- 172.21.15.33:0/4048063438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f41042b0 msgr2=0x7f01f4197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f37fe700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f41042b0 0x7f01f4197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f37fe700 1 -- 172.21.15.33:0/4048063438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e40095f0 con 0x7f01f4101980 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f3fff700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f4106be0 0x7f01f419e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f37fe700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4101980 0x7f01f4196cc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f01e4004710 tx=0x7f01e4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f0ff9700 1 -- 172.21.15.33:0/4048063438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e401d070 con 0x7f01f4101980 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f0ff9700 1 -- 172.21.15.33:0/4048063438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01e400be70 con 0x7f01f4101980 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f9d77700 1 -- 172.21.15.33:0/4048063438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f419ed60 con 0x7f01f4101980 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f0ff9700 1 -- 172.21.15.33:0/4048063438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4021410 con 0x7f01f4101980 2024-05-08T21:18:43.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.708+0000 7f01f9d77700 1 -- 172.21.15.33:0/4048063438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f419f190 con 0x7f01f4101980 2024-05-08T21:18:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.710+0000 7f01f0ff9700 1 -- 172.21.15.33:0/4048063438 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f01e400f680 con 0x7f01f4101980 2024-05-08T21:18:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.711+0000 7f01f0ff9700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f01dc0a1230 0x7f01dc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.711+0000 7f01f0ff9700 1 -- 172.21.15.33:0/4048063438 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f01e40cdf50 con 0x7f01f4101980 2024-05-08T21:18:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.711+0000 7f01f2ffd700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f01dc0a1230 0x7f01dc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f01f2ffd700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f01dc0a1230 0x7f01dc0a36f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f01e8005950 tx=0x7f01e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f0205f31b80 1 Processor -- start 2024-05-08T21:18:43.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f0205f31b80 1 -- start start 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f0205f31b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 0x5620a1cdc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f0205f31b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdce50 0x5620a1cdd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f0205f31b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdd810 0x5620a1cddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f0205f31b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a1c33100 con 0x5620a1cdc4f0 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f0205f31b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a1c30ac0 con 0x5620a1cdd810 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f0205f31b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a1c2c3b0 con 0x5620a1cdce50 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f01f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 0x5620a1cdc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f01f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 0x5620a1cdc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54832/0 (socket says 172.21.15.33:54832) 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f01f37fe700 1 -- 172.21.15.33:0/1346628239 learned_addr learned my addr 172.21.15.33:0/1346628239 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.712+0000 7f01f2ffd700 1 --2- 172.21.15.33:0/1346628239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdce50 0x5620a1cdd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.713+0000 7f01f3fff700 1 --2- 172.21.15.33:0/1346628239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdd810 0x5620a1cddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.713+0000 7f01f37fe700 1 -- 172.21.15.33:0/1346628239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdce50 msgr2=0x5620a1cdd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.713+0000 7f01f37fe700 1 --2- 172.21.15.33:0/1346628239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdce50 0x5620a1cdd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.713+0000 7f01f37fe700 1 -- 172.21.15.33:0/1346628239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdd810 msgr2=0x5620a1cddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.713+0000 7f01f37fe700 1 --2- 172.21.15.33:0/1346628239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdd810 0x5620a1cddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.713+0000 7f01f37fe700 1 -- 172.21.15.33:0/1346628239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620a1aad360 con 0x5620a1cdc4f0 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.713+0000 7f01f37fe700 1 --2- 172.21.15.33:0/1346628239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 0x5620a1cdc910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f01e40a2e80 tx=0x7f01e409ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf39e11084e9e552 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.713+0000 7f01d9ffb700 1 -- 172.21.15.33:0/1346628239 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e409afc0 con 0x5620a1cdc4f0 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.713+0000 7f01d9ffb700 1 -- 172.21.15.33:0/1346628239 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01e409edf0 con 0x5620a1cdc4f0 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.713+0000 7f01d9ffb700 1 -- 172.21.15.33:0/1346628239 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40cb840 con 0x5620a1cdc4f0 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.714+0000 7f0205f31b80 1 -- 172.21.15.33:0/1346628239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 msgr2=0x5620a1cdc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.714+0000 7f0205f31b80 1 --2- 172.21.15.33:0/1346628239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 0x5620a1cdc910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f01e40a2e80 tx=0x7f01e409ab80 comp rx=0 tx=0).stop 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.714+0000 7f0205f31b80 1 -- 172.21.15.33:0/1346628239 shutdown_connections 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.714+0000 7f0205f31b80 1 --2- 172.21.15.33:0/1346628239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdd810 0x5620a1cddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.714+0000 7f0205f31b80 1 --2- 172.21.15.33:0/1346628239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 0x5620a1cdc910 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.714+0000 7f0205f31b80 1 --2- 172.21.15.33:0/1346628239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdce50 0x5620a1cdd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.714+0000 7f0205f31b80 1 -- 172.21.15.33:0/1346628239 >> 172.21.15.33:0/1346628239 conn(0x5620a1c3a1c0 msgr2=0x5620a1cd6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.714+0000 7f0205f31b80 1 -- 172.21.15.33:0/1346628239 shutdown_connections 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.714+0000 7f0205f31b80 1 -- 172.21.15.33:0/1346628239 wait complete. 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.714+0000 7f0205f31b80 1 -- start start 2024-05-08T21:18:43.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.714+0000 7f0205f31b80 1 Processor -- start 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f0205f31b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f0205f31b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdce50 0x5620a1ce5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f0205f31b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdd810 0x5620a1cecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f0205f31b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a1bc8510 con 0x5620a1cdc4f0 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f0205f31b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a1bc81b0 con 0x5620a1cdce50 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f0205f31b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a1aaf650 con 0x5620a1cdd810 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f0205f31b80 10 client.0 _ll_register_callbacks cb 0x5620a1bd1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f01f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f01f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54836/0 (socket says 172.21.15.33:54836) 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f01f37fe700 1 -- 172.21.15.33:0/203724617 learned_addr learned my addr 172.21.15.33:0/203724617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f01f3fff700 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdd810 0x5620a1cecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f01f2ffd700 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdce50 0x5620a1ce5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f01f37fe700 1 -- 172.21.15.33:0/203724617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdd810 msgr2=0x5620a1cecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f01f37fe700 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdd810 0x5620a1cecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f01f37fe700 1 -- 172.21.15.33:0/203724617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdce50 msgr2=0x5620a1ce5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f01f37fe700 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdce50 0x5620a1ce5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f01f37fe700 1 -- 172.21.15.33:0/203724617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f419f190 con 0x5620a1cdc4f0 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f01f37fe700 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce5a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f01e4097a30 tx=0x7f01e4097400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc4e9ee54a272ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.715+0000 7f01d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f01d2ffd700 1 -- 172.21.15.33:0/203724617 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40b92c0 con 0x5620a1cdc4f0 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f01d2ffd700 1 -- 172.21.15.33:0/203724617 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f01e400f460 con 0x5620a1cdc4f0 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f0205f31b80 1 -- 172.21.15.33:0/203724617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620a1b85220 con 0x5620a1cdc4f0 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f01d2ffd700 1 -- 172.21.15.33:0/203724617 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01e409a4b0 con 0x5620a1cdc4f0 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f0205f31b80 10 client.15492 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f0205f31b80 1 -- 172.21.15.33:0/203724617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5620a1a59b10 con 0x5620a1cdc4f0 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f01d2ffd700 1 -- 172.21.15.33:0/203724617 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5620a1b85220 con 0x5620a1cdc4f0 2024-05-08T21:18:43.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f01d2ffd700 1 -- 172.21.15.33:0/203724617 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01e40a2f20 con 0x5620a1cdc4f0 2024-05-08T21:18:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f0205f31b80 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f0205f31b80 10 client.15492 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T21:18:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f0205f31b80 20 client.15492 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f0205f31b80 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5620a1cf0700 0x5620a1cf59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f0205f31b80 4 client.15492 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:18:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.716+0000 7f0205f31b80 1 -- 172.21.15.33:0/203724617 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620a1cab7c0 con 0x5620a1cf0700 2024-05-08T21:18:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.717+0000 7f01f2ffd700 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5620a1cf0700 0x5620a1cf59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.717+0000 7f01f2ffd700 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5620a1cf0700 0x5620a1cf59c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ea5814ead07dd1 server_cookie=480bb581d1450c6e in_seq=0 out_seq=0 2024-05-08T21:18:43.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.717+0000 7f01d2ffd700 10 client.15492 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f01d2ffd700 1 -- 172.21.15.33:0/203724617 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5620a1cab7c0 con 0x5620a1cf0700 2024-05-08T21:18:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f01d2ffd700 10 client.15492 handle_command_reply: tid=0 2024-05-08T21:18:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f0205f31b80 1 client.15492 shutdown 2024-05-08T21:18:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f0205f31b80 1 -- 172.21.15.33:0/203724617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 msgr2=0x5620a1ce5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f0205f31b80 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce5a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f01e4097a30 tx=0x7f01e4097400 comp rx=0 tx=0).stop 2024-05-08T21:18:43.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f0205f31b80 1 -- 172.21.15.33:0/203724617 shutdown_connections 2024-05-08T21:18:43.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f0205f31b80 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5620a1cf0700 0x5620a1cf59c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f0205f31b80 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdd810 0x5620a1cecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f0205f31b80 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce5a20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f0205f31b80 1 --2- 172.21.15.33:0/203724617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdce50 0x5620a1ce5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f0205f31b80 1 -- 172.21.15.33:0/203724617 >> 172.21.15.33:0/203724617 conn(0x5620a1c3a1c0 msgr2=0x5620a1cd6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f01d2ffd700 0 client.15492 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:43.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f0205f31b80 1 -- 172.21.15.33:0/203724617 shutdown_connections 2024-05-08T21:18:43.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f0205f31b80 1 -- 172.21.15.33:0/203724617 wait complete. 2024-05-08T21:18:43.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.718+0000 7f0205f31b80 20 client.15492 trim_cache size 0 max 16384 2024-05-08T21:18:43.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.737+0000 7f0205f31b80 1 Processor -- start 2024-05-08T21:18:43.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.737+0000 7f0205f31b80 1 -- start start 2024-05-08T21:18:43.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f0205f31b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f0205f31b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdce50 0x5620a1cdc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f0205f31b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdd810 0x5620a1cd7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f0205f31b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a1cab7f0 con 0x5620a1cdd810 2024-05-08T21:18:43.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f0205f31b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a1b85250 con 0x5620a1cdce50 2024-05-08T21:18:43.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f0205f31b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a1c25150 con 0x5620a1cdc4f0 2024-05-08T21:18:43.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01f3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdd810 0x5620a1cd7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01f3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdd810 0x5620a1cd7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54850/0 (socket says 172.21.15.33:54850) 2024-05-08T21:18:43.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01f3fff700 1 -- 172.21.15.33:0/2339292890 learned_addr learned my addr 172.21.15.33:0/2339292890 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01f37fe700 1 --2- 172.21.15.33:0/2339292890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01f2ffd700 1 --2- 172.21.15.33:0/2339292890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdce50 0x5620a1cdc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01f3fff700 1 -- 172.21.15.33:0/2339292890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdc4f0 msgr2=0x5620a1ce6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01f3fff700 1 --2- 172.21.15.33:0/2339292890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01f3fff700 1 -- 172.21.15.33:0/2339292890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdce50 msgr2=0x5620a1cdc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01f3fff700 1 --2- 172.21.15.33:0/2339292890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdce50 0x5620a1cdc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01f3fff700 1 -- 172.21.15.33:0/2339292890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620a1bf2510 con 0x5620a1cdd810 2024-05-08T21:18:43.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01f3fff700 1 --2- 172.21.15.33:0/2339292890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdd810 0x5620a1cd7580 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f01ec003bc0 tx=0x7f01ec009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36046806de8261bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01da7fc700 1 -- 172.21.15.33:0/2339292890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0044b0 con 0x5620a1cdd810 2024-05-08T21:18:43.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01da7fc700 1 -- 172.21.15.33:0/2339292890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01ec00f040 con 0x5620a1cdd810 2024-05-08T21:18:43.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.738+0000 7f01da7fc700 1 -- 172.21.15.33:0/2339292890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00e4f0 con 0x5620a1cdd810 2024-05-08T21:18:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.739+0000 7f0205f31b80 1 -- 172.21.15.33:0/2339292890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdd810 msgr2=0x5620a1cd7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.739+0000 7f0205f31b80 1 --2- 172.21.15.33:0/2339292890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdd810 0x5620a1cd7580 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f01ec003bc0 tx=0x7f01ec009920 comp rx=0 tx=0).stop 2024-05-08T21:18:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.739+0000 7f0205f31b80 1 -- 172.21.15.33:0/2339292890 shutdown_connections 2024-05-08T21:18:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.739+0000 7f0205f31b80 1 --2- 172.21.15.33:0/2339292890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdd810 0x5620a1cd7580 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.739+0000 7f0205f31b80 1 --2- 172.21.15.33:0/2339292890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.739+0000 7f0205f31b80 1 --2- 172.21.15.33:0/2339292890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdce50 0x5620a1cdc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.739+0000 7f0205f31b80 1 -- 172.21.15.33:0/2339292890 >> 172.21.15.33:0/2339292890 conn(0x5620a1c3a1c0 msgr2=0x5620a1cddbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.739+0000 7f0205f31b80 1 -- 172.21.15.33:0/2339292890 shutdown_connections 2024-05-08T21:18:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.739+0000 7f0205f31b80 1 -- 172.21.15.33:0/2339292890 wait complete. 2024-05-08T21:18:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.739+0000 7f0205f31b80 1 -- start start 2024-05-08T21:18:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f0205f31b80 1 Processor -- start 2024-05-08T21:18:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f0205f31b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f0205f31b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdce50 0x5620a1ce7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f0205f31b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdd810 0x5620a1ce7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f0205f31b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a1ce8320 con 0x5620a1cdce50 2024-05-08T21:18:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f0205f31b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a1ce8490 con 0x5620a1cdd810 2024-05-08T21:18:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f0205f31b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620a1ce8650 con 0x5620a1cdc4f0 2024-05-08T21:18:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f0205f31b80 10 client.0 _ll_register_callbacks cb 0x5620a1bd1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f01f2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdce50 0x5620a1ce7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f01f2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdce50 0x5620a1ce7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54858/0 (socket says 172.21.15.33:54858) 2024-05-08T21:18:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f01f2ffd700 1 -- 172.21.15.33:0/2745613500 learned_addr learned my addr 172.21.15.33:0/2745613500 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f01f37fe700 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f01f2ffd700 1 -- 172.21.15.33:0/2745613500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdc4f0 msgr2=0x5620a1ce70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f01f2ffd700 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f01f2ffd700 1 -- 172.21.15.33:0/2745613500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdd810 msgr2=0x5620a1ce7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f01f3fff700 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdd810 0x5620a1ce7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f01f2ffd700 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdd810 0x5620a1ce7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.740+0000 7f01f2ffd700 1 -- 172.21.15.33:0/2745613500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec009370 con 0x5620a1cdce50 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.741+0000 7f01f2ffd700 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdce50 0x5620a1ce7620 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f01e8005d90 tx=0x7f01e8013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9c16baa2c2c2bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.741+0000 7f01d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.741+0000 7f01d2ffd700 1 -- 172.21.15.33:0/2745613500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8026040 con 0x5620a1cdce50 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.741+0000 7f01d2ffd700 1 -- 172.21.15.33:0/2745613500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f01e8017070 con 0x5620a1cdce50 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.741+0000 7f0205f31b80 1 -- 172.21.15.33:0/2745613500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01e409edf0 con 0x5620a1cdce50 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.741+0000 7f01d2ffd700 1 -- 172.21.15.33:0/2745613500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01e8026b00 con 0x5620a1cdce50 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.741+0000 7f0205f31b80 10 client.15498 fetch_fsmap learned FSMap version 28 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.741+0000 7f0205f31b80 1 -- 172.21.15.33:0/2745613500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5620a1ab1c80 con 0x5620a1cdce50 2024-05-08T21:18:43.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.741+0000 7f01d2ffd700 1 -- 172.21.15.33:0/2745613500 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01e409edf0 con 0x5620a1cdce50 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.741+0000 7f01d2ffd700 1 -- 172.21.15.33:0/2745613500 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01e801e4b0 con 0x5620a1cdce50 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.741+0000 7f0205f31b80 10 client.15498 fetch_fsmap finished waiting for FSMap version 28 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.742+0000 7f0205f31b80 10 client.15498 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.742+0000 7f0205f31b80 20 client.15498 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.742+0000 7f0205f31b80 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5620a1d4fd30 0x5620a1d4c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.742+0000 7f0205f31b80 4 client.15498 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.742+0000 7f0205f31b80 1 -- 172.21.15.33:0/2745613500 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5620a1c24e70 con 0x5620a1d4fd30 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.742+0000 7f01f37fe700 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5620a1d4fd30 0x5620a1d4c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.742+0000 7f01f37fe700 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5620a1d4fd30 0x5620a1d4c640 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2900d22645c6de57 server_cookie=8740e1764c22e463 in_seq=0 out_seq=0 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.742+0000 7f01d2ffd700 10 client.15498 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.742+0000 7f01d2ffd700 1 -- 172.21.15.33:0/2745613500 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5620a1c24e70 con 0x5620a1d4fd30 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.742+0000 7f01d2ffd700 10 client.15498 handle_command_reply: tid=0 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.742+0000 7f0205f31b80 1 client.15498 shutdown 2024-05-08T21:18:43.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f0205f31b80 1 -- 172.21.15.33:0/2745613500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdce50 msgr2=0x5620a1ce7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f0205f31b80 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdce50 0x5620a1ce7620 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f01e8005d90 tx=0x7f01e8013070 comp rx=0 tx=0).stop 2024-05-08T21:18:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f0205f31b80 1 -- 172.21.15.33:0/2745613500 shutdown_connections 2024-05-08T21:18:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f0205f31b80 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5620a1d4fd30 0x5620a1d4c640 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f0205f31b80 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620a1cdd810 0x5620a1ce7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f0205f31b80 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5620a1cdc4f0 0x5620a1ce70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f01d2ffd700 0 client.15498 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f0205f31b80 1 --2- 172.21.15.33:0/2745613500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620a1cdce50 0x5620a1ce7620 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f0205f31b80 1 -- 172.21.15.33:0/2745613500 >> 172.21.15.33:0/2745613500 conn(0x5620a1c3a1c0 msgr2=0x5620a1ce4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f0205f31b80 1 -- 172.21.15.33:0/2745613500 shutdown_connections 2024-05-08T21:18:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f0205f31b80 1 -- 172.21.15.33:0/2745613500 wait complete. 2024-05-08T21:18:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f0205f31b80 20 client.15498 trim_cache size 0 max 16384 2024-05-08T21:18:43.751 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:43.752 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "53ae3a5a-0d7f-11ef-bc97-c7b262605968", 2024-05-08T21:18:43.752 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-05-08T21:18:43.752 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24563, 2024-05-08T21:18:43.752 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-05-08T21:18:43.752 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-05-08T21:18:43.752 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-05-08T21:18:43.752 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 100.520474948, 2024-05-08T21:18:43.752 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-05-08T21:18:43.752 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-05-08T21:18:43.752 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-05-08T21:18:43.752 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 106.343303722 2024-05-08T21:18:43.752 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:43.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f01f9d77700 1 -- 172.21.15.33:0/4048063438 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f01dc0a1230 msgr2=0x7f01dc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f01f9d77700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f01dc0a1230 0x7f01dc0a36f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f01e8005950 tx=0x7f01e800b410 comp rx=0 tx=0).stop 2024-05-08T21:18:43.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f01f9d77700 1 -- 172.21.15.33:0/4048063438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4101980 msgr2=0x7f01f4196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:43.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f01f9d77700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4101980 0x7f01f4196cc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f01e4004710 tx=0x7f01e4009580 comp rx=0 tx=0).stop 2024-05-08T21:18:43.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f01f9d77700 1 -- 172.21.15.33:0/4048063438 shutdown_connections 2024-05-08T21:18:43.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f01f9d77700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f4106be0 0x7f01f419e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f01f9d77700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01f4101980 0x7f01f4196cc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f01f9d77700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f01dc0a1230 0x7f01dc0a36f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f01f9d77700 1 --2- 172.21.15.33:0/4048063438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f41042b0 0x7f01f4197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:43.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.743+0000 7f01f9d77700 1 -- 172.21.15.33:0/4048063438 >> 172.21.15.33:0/4048063438 conn(0x7f01f40fb290 msgr2=0x7f01f4102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:43.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.744+0000 7f01f9d77700 1 -- 172.21.15.33:0/4048063438 shutdown_connections 2024-05-08T21:18:43.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:43.744+0000 7f01f9d77700 1 -- 172.21.15.33:0/4048063438 wait complete. 2024-05-08T21:18:43.759 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T21:18:43.769 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T21:18:43.769 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T21:18:43.769 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T21:18:43.769 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-08T21:18:43.787 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-08T21:18:43.787 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-08T21:18:43.787 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-05-08T21:18:43.787 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T21:18:43.787 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T21:18:43.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:43.787 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-08T21:18:43.807 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-08T21:18:43.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:43.808 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T21:18:43.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:43 smithi071 ceph-mon[47074]: pgmap v280: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:18:43.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:43 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2295732255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:43.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:43 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2464078900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:43.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:43 smithi082 ceph-mon[47512]: pgmap v280: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:18:43.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:43 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2295732255' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:43.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:43 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2464078900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:43.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:43.894 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-05-08T21:18:43.976 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-08T21:18:43.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T21:18:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:18:45.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.288+0000 7f5f35a1e700 1 -- 172.21.15.33:0/3529977898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f301051c0 msgr2=0x7f5f3010cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:45.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.288+0000 7f5f35a1e700 1 --2- 172.21.15.33:0/3529977898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f301051c0 0x7f5f3010cd90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009f90 tx=0x7f5f2800b330 comp rx=0 tx=0).stop 2024-05-08T21:18:45.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.288+0000 7f5f35a1e700 1 -- 172.21.15.33:0/3529977898 shutdown_connections 2024-05-08T21:18:45.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.288+0000 7f5f35a1e700 1 --2- 172.21.15.33:0/3529977898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f301051c0 0x7f5f3010cd90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:45.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.288+0000 7f5f35a1e700 1 --2- 172.21.15.33:0/3529977898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f301000e0 0x7f5f301004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:45.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.288+0000 7f5f35a1e700 1 --2- 172.21.15.33:0/3529977898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f30100a90 0x7f5f30104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:45.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.288+0000 7f5f35a1e700 1 -- 172.21.15.33:0/3529977898 >> 172.21.15.33:0/3529977898 conn(0x7f5f30076f40 msgr2=0x7f5f30077350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:45.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.288+0000 7f5f35a1e700 1 -- 172.21.15.33:0/3529977898 shutdown_connections 2024-05-08T21:18:45.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.288+0000 7f5f35a1e700 1 -- 172.21.15.33:0/3529977898 wait complete. 2024-05-08T21:18:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.289+0000 7f5f35a1e700 1 Processor -- start 2024-05-08T21:18:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.289+0000 7f5f35a1e700 1 -- start start 2024-05-08T21:18:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.289+0000 7f5f35a1e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f301000e0 0x7f5f301973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.289+0000 7f5f35a1e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f30100a90 0x7f5f30197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:45.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.289+0000 7f5f35a1e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f301051c0 0x7f5f3019bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.289+0000 7f5f35a1e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3019c2f0 con 0x7f5f301000e0 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.289+0000 7f5f35a1e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3019c460 con 0x7f5f30100a90 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.289+0000 7f5f35a1e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3019c620 con 0x7f5f301051c0 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.289+0000 7f5f2effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f301000e0 0x7f5f301973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.289+0000 7f5f2effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f301000e0 0x7f5f301973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54878/0 (socket says 172.21.15.33:54878) 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.289+0000 7f5f2effd700 1 -- 172.21.15.33:0/4111987872 learned_addr learned my addr 172.21.15.33:0/4111987872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.289+0000 7f5f2e7fc700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f30100a90 0x7f5f30197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f2f7fe700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f301051c0 0x7f5f3019bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f2effd700 1 -- 172.21.15.33:0/4111987872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f301051c0 msgr2=0x7f5f3019bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f2effd700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f301051c0 0x7f5f3019bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f2effd700 1 -- 172.21.15.33:0/4111987872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f30100a90 msgr2=0x7f5f30197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f2effd700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f30100a90 0x7f5f30197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f2effd700 1 -- 172.21.15.33:0/4111987872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f28009c40 con 0x7f5f301000e0 2024-05-08T21:18:45.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f2f7fe700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f301051c0 0x7f5f3019bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:18:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f2effd700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f301000e0 0x7f5f301973f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000e9a0 tx=0x7f5f2000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f34a1c700 1 -- 172.21.15.33:0/4111987872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f20009790 con 0x7f5f301000e0 2024-05-08T21:18:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f35a1e700 1 -- 172.21.15.33:0/4111987872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3019c900 con 0x7f5f301000e0 2024-05-08T21:18:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f34a1c700 1 -- 172.21.15.33:0/4111987872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5f2000cbd0 con 0x7f5f301000e0 2024-05-08T21:18:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f34a1c700 1 -- 172.21.15.33:0/4111987872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000eee0 con 0x7f5f301000e0 2024-05-08T21:18:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.290+0000 7f5f35a1e700 1 -- 172.21.15.33:0/4111987872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f301a4160 con 0x7f5f301000e0 2024-05-08T21:18:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.291+0000 7f5f35a1e700 1 -- 172.21.15.33:0/4111987872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f301094f0 con 0x7f5f301000e0 2024-05-08T21:18:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.292+0000 7f5f34a1c700 1 -- 172.21.15.33:0/4111987872 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f5f2000cd40 con 0x7f5f301000e0 2024-05-08T21:18:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.293+0000 7f5f34a1c700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5f180a11b0 0x7f5f180a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.293+0000 7f5f2e7fc700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5f180a11b0 0x7f5f180a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.294+0000 7f5f34a1c700 1 -- 172.21.15.33:0/4111987872 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5f20014070 con 0x7f5f301000e0 2024-05-08T21:18:45.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.294+0000 7f5f2e7fc700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5f180a11b0 0x7f5f180a3670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5f30198950 tx=0x7f5f24009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:45.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.298+0000 7f5f34a1c700 1 -- 172.21.15.33:0/4111987872 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5f200964e0 con 0x7f5f301000e0 2024-05-08T21:18:45.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:45 smithi033 ceph-mon[42528]: pgmap v281: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.559+0000 7f5f35a1e700 1 -- 172.21.15.33:0/4111987872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5f301015c0 con 0x7f5f301000e0 2024-05-08T21:18:45.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.560+0000 7f5f34a1c700 1 -- 172.21.15.33:0/4111987872 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5f20026070 con 0x7f5f301000e0 2024-05-08T21:18:45.562 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T21:18:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.562+0000 7f5f35a1e700 1 -- 172.21.15.33:0/4111987872 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5f180a11b0 msgr2=0x7f5f180a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.562+0000 7f5f35a1e700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5f180a11b0 0x7f5f180a3670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5f30198950 tx=0x7f5f24009310 comp rx=0 tx=0).stop 2024-05-08T21:18:45.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.562+0000 7f5f35a1e700 1 -- 172.21.15.33:0/4111987872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f301000e0 msgr2=0x7f5f301973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.562+0000 7f5f35a1e700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f301000e0 0x7f5f301973f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000e9a0 tx=0x7f5f2000ed60 comp rx=0 tx=0).stop 2024-05-08T21:18:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.562+0000 7f5f35a1e700 1 -- 172.21.15.33:0/4111987872 shutdown_connections 2024-05-08T21:18:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.562+0000 7f5f35a1e700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f301051c0 0x7f5f3019bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.562+0000 7f5f35a1e700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f301000e0 0x7f5f301973f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.562+0000 7f5f35a1e700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5f180a11b0 0x7f5f180a3670 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.562+0000 7f5f35a1e700 1 --2- 172.21.15.33:0/4111987872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f30100a90 0x7f5f30197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.562+0000 7f5f35a1e700 1 -- 172.21.15.33:0/4111987872 >> 172.21.15.33:0/4111987872 conn(0x7f5f30076f40 msgr2=0x7f5f300699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.562+0000 7f5f35a1e700 1 -- 172.21.15.33:0/4111987872 shutdown_connections 2024-05-08T21:18:45.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:45.562+0000 7f5f35a1e700 1 -- 172.21.15.33:0/4111987872 wait complete. 2024-05-08T21:18:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:45 smithi071 ceph-mon[47074]: pgmap v281: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:45.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:45 smithi082 ceph-mon[47512]: pgmap v281: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:45.963 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T21:18:45.963 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T21:18:45.963 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-05-08T21:18:45.963 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T21:18:45.963 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T21:18:45.963 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T21:18:45.963 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T21:18:45.963 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T21:18:45.964 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T21:18:45.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:45.964 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-05-08T21:18:45.985 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T21:18:45.985 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T21:18:45.985 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T21:18:45.986 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 41695sec preferred_lft 41695sec 2024-05-08T21:18:45.987 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-05-08T21:18:45.987 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-05-08T21:18:45.987 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T21:18:45.987 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-05-08T21:18:45.987 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-05-08T21:18:45.987 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T21:18:45.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:45.987 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T21:18:45.987 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T21:18:45.987 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-05-08T21:18:45.987 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-05-08T21:18:45.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-05-08T21:18:45.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T21:18:45.988 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T21:18:46.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:46.129 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T21:18:46.162 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-05-08T21:18:46.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:46.166 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-05-08T21:18:46.233 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-05-08T21:18:46.233 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T21:18:46.233 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T21:18:46.233 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T21:18:46.233 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T21:18:46.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:46.234 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T21:18:46.234 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T21:18:46.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T21:18:46.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T21:18:46.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T21:18:46.234 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T21:18:46.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:46.359 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-08T21:18:46.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:46.380 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-05-08T21:18:46.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:46.443 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T21:18:46.443 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T21:18:46.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T21:18:46.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T21:18:46.443 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T21:18:46.554 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T21:18:46.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:46.555 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T21:18:46.555 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T21:18:46.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T21:18:46.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T21:18:46.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T21:18:46.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T21:18:46.555 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T21:18:46.555 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T21:18:46.602 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:46 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4111987872' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T21:18:46.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:46.763 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-05-08T21:18:46.763 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-05-08T21:18:46.763 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-05-08T21:18:46.763 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T21:18:46.763 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-05-08T21:18:46.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:46 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/4111987872' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T21:18:46.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:46 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/4111987872' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T21:18:46.873 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-05-08T21:18:46.873 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T21:18:46.873 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T21:18:46.893 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T21:18:46.893 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T21:18:46.893 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T21:18:46.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-05-08T21:18:47.029 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T21:18:47.029 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T21:18:47.105 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T21:18:47.106 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T21:18:47.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:47.107 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-05-08T21:18:47.169 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T21:18:47.169 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/72e45633-b2f0-4430-9cea-2dd00820c0b7 --client_fs=cephfs) 2024-05-08T21:18:47.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-05-08T21:18:47.249 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-05-08T21:18:47.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-05-08T21:18:47.322 INFO:teuthology.orchestra.run.smithi033.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-05-08T21:18:47.323 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-05-08T21:18:47.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:47.324 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-05-08T21:18:47.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:2024-05-08T21:18:47.322+0000 7f9cfd8bd5c0 -1 init, newargv = 0x557c7431d9f0 newargc=15 2024-05-08T21:18:47.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[108675]: starting ceph client 2024-05-08T21:18:47.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[108675]: starting fuse 2024-05-08T21:18:47.369 INFO:teuthology.orchestra.run.smithi033.stdout:124 2024-05-08T21:18:47.369 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2024-05-08T21:18:47.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-05-08T21:18:47.369 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-05-08T21:18:47.369 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-05-08T21:18:47.369 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-05-08T21:18:47.370 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T21:18:47.371 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T21:18:47.371 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T21:18:47.371 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-05-08T21:18:47.371 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-05-08T21:18:47.371 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-05-08T21:18:47.371 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T21:18:47.371 DEBUG:teuthology.orchestra.run.smithi033:> 2024-05-08T21:18:47.371 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.0")) 2024-05-08T21:18:47.371 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-05-08T21:18:47.510 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.0.108675.asok 2024-05-08T21:18:47.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:47 smithi033 ceph-mon[42528]: pgmap v282: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:47.691 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.108675.asok 2024-05-08T21:18:47.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:47.691 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.108675.asok status 2024-05-08T21:18:47.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:47 smithi071 ceph-mon[47074]: pgmap v282: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:47.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:47 smithi082 ceph-mon[47512]: pgmap v282: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:18:47.871 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "d21e4f961318b56f0d915caa2490c3711bf13c34", 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 18.2.2-1322-gd21e4f96 (d21e4f961318b56f0d915caa2490c3711bf13c34) reef (stable)", 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "0", 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "108675", 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/volumes/qa/sv_0/72e45633-b2f0-4430-9cea-2dd00820c0b7" 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 15510, 2024-05-08T21:18:47.872 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 15510 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 960733661 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 960733661 2024-05-08T21:18:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:18:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.15510 192.168.0.1:0/960733661", 2024-05-08T21:18:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/960733661", 2024-05-08T21:18:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 5, 2024-05-08T21:18:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 28, 2024-05-08T21:18:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 87, 2024-05-08T21:18:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 85, 2024-05-08T21:18:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-05-08T21:18:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-05-08T21:18:47.874 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:47.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:47.883 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-05-08T21:18:47.883 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-08T21:18:47.902 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-05-08T21:18:47.903 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-08T21:18:47.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:18:47.903 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T21:18:47.978 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T21:18:47.988 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T21:18:47.988 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T21:18:47.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-08T21:18:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.161+0000 7fc2cddc3700 1 -- 172.21.15.33:0/1871396371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2c8101920 msgr2=0x7fc2c8103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.161+0000 7fc2cddc3700 1 --2- 172.21.15.33:0/1871396371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2c8101920 0x7fc2c8103d10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009990 tx=0x7fc2b8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.161+0000 7fc2cddc3700 1 -- 172.21.15.33:0/1871396371 shutdown_connections 2024-05-08T21:18:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.161+0000 7fc2cddc3700 1 --2- 172.21.15.33:0/1871396371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2c8106b80 0x7fc2c8108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.161+0000 7fc2cddc3700 1 --2- 172.21.15.33:0/1871396371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2c8101920 0x7fc2c8103d10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.161+0000 7fc2cddc3700 1 --2- 172.21.15.33:0/1871396371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2c8104250 0x7fc2c8106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.161+0000 7fc2cddc3700 1 -- 172.21.15.33:0/1871396371 >> 172.21.15.33:0/1871396371 conn(0x7fc2c80fb310 msgr2=0x7fc2c80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.162+0000 7fc2cddc3700 1 -- 172.21.15.33:0/1871396371 shutdown_connections 2024-05-08T21:18:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.162+0000 7fc2cddc3700 1 -- 172.21.15.33:0/1871396371 wait complete. 2024-05-08T21:18:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.162+0000 7fc2cddc3700 1 Processor -- start 2024-05-08T21:18:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.162+0000 7fc2cddc3700 1 -- start start 2024-05-08T21:18:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2cddc3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2c8101920 0x7fc2c8196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2cddc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2c8104250 0x7fc2c8197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2cddc3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2c8106b80 0x7fc2c819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2cddc3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c819e7c0 con 0x7fc2c8104250 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2cddc3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c819e930 con 0x7fc2c8106b80 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2cddc3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c819eaf0 con 0x7fc2c8101920 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2c6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2c8104250 0x7fc2c8197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2c6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2c8104250 0x7fc2c8197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54902/0 (socket says 172.21.15.33:54902) 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2c6ffd700 1 -- 172.21.15.33:0/2905245354 learned_addr learned my addr 172.21.15.33:0/2905245354 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2c7fff700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2c8106b80 0x7fc2c819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2c77fe700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2c8101920 0x7fc2c8196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2c6ffd700 1 -- 172.21.15.33:0/2905245354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2c8101920 msgr2=0x7fc2c8196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2c6ffd700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2c8101920 0x7fc2c8196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2c6ffd700 1 -- 172.21.15.33:0/2905245354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2c8106b80 msgr2=0x7fc2c819e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2c6ffd700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2c8106b80 0x7fc2c819e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2c6ffd700 1 -- 172.21.15.33:0/2905245354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b80095f0 con 0x7fc2c8104250 2024-05-08T21:18:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2c77fe700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2c8101920 0x7fc2c8196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.163+0000 7fc2c6ffd700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2c8104250 0x7fc2c8197210 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00e9a0 tx=0x7fc2bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.164+0000 7fc2c4ff9700 1 -- 172.21.15.33:0/2905245354 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc009790 con 0x7fc2c8104250 2024-05-08T21:18:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.164+0000 7fc2c4ff9700 1 -- 172.21.15.33:0/2905245354 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2bc00cbd0 con 0x7fc2c8104250 2024-05-08T21:18:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.164+0000 7fc2cddc3700 1 -- 172.21.15.33:0/2905245354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c819edd0 con 0x7fc2c8104250 2024-05-08T21:18:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.164+0000 7fc2c4ff9700 1 -- 172.21.15.33:0/2905245354 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc018680 con 0x7fc2c8104250 2024-05-08T21:18:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.164+0000 7fc2cddc3700 1 -- 172.21.15.33:0/2905245354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c819f220 con 0x7fc2c8104250 2024-05-08T21:18:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.165+0000 7fc2cddc3700 1 -- 172.21.15.33:0/2905245354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c80fce90 con 0x7fc2c8104250 2024-05-08T21:18:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.166+0000 7fc2c4ff9700 1 -- 172.21.15.33:0/2905245354 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc2bc00cd40 con 0x7fc2c8104250 2024-05-08T21:18:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.166+0000 7fc2c4ff9700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2b00a11a0 0x7fc2b00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.167+0000 7fc2c4ff9700 1 -- 172.21.15.33:0/2905245354 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc2bc014070 con 0x7fc2c8104250 2024-05-08T21:18:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.167+0000 7fc2c77fe700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2b00a11a0 0x7fc2b00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.167+0000 7fc2c77fe700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2b00a11a0 0x7fc2b00a3660 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8005910 tx=0x7fc2b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.172+0000 7fc2c4ff9700 1 -- 172.21.15.33:0/2905245354 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc2bc052d60 con 0x7fc2c8104250 2024-05-08T21:18:48.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:48.438+0000 7fc2cddc3700 1 -- 172.21.15.33:0/2905245354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) v1 -- 0x7fc2c804ec60 con 0x7fc2c8104250 2024-05-08T21:18:49.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.380+0000 7fc2c4ff9700 1 -- 172.21.15.33:0/2905245354 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) v1 ==== 117+0+0 (secure 0 0 0) 0x7fc2bc096310 con 0x7fc2c8104250 2024-05-08T21:18:49.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.383+0000 7fc2cddc3700 1 -- 172.21.15.33:0/2905245354 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2b00a11a0 msgr2=0x7fc2b00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:49.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.383+0000 7fc2cddc3700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2b00a11a0 0x7fc2b00a3660 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8005910 tx=0x7fc2b8005fb0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.383+0000 7fc2cddc3700 1 -- 172.21.15.33:0/2905245354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2c8104250 msgr2=0x7fc2c8197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.383+0000 7fc2cddc3700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2c8104250 0x7fc2c8197210 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00e9a0 tx=0x7fc2bc00ecb0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.383+0000 7fc2cddc3700 1 -- 172.21.15.33:0/2905245354 shutdown_connections 2024-05-08T21:18:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.383+0000 7fc2cddc3700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2b00a11a0 0x7fc2b00a3660 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.383+0000 7fc2cddc3700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2c8106b80 0x7fc2c819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.383+0000 7fc2cddc3700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2c8101920 0x7fc2c8196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.383+0000 7fc2cddc3700 1 --2- 172.21.15.33:0/2905245354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2c8104250 0x7fc2c8197210 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.383+0000 7fc2cddc3700 1 -- 172.21.15.33:0/2905245354 >> 172.21.15.33:0/2905245354 conn(0x7fc2c80fb310 msgr2=0x7fc2c8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.383+0000 7fc2cddc3700 1 -- 172.21.15.33:0/2905245354 shutdown_connections 2024-05-08T21:18:49.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.383+0000 7fc2cddc3700 1 -- 172.21.15.33:0/2905245354 wait complete. 2024-05-08T21:18:49.401 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-08T21:18:49.413 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-08T21:18:49.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:18:49.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.566+0000 7ff113039700 1 -- 172.21.15.33:0/776573175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff10c0ffd80 msgr2=0x7ff10c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:49.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.566+0000 7ff113039700 1 --2- 172.21.15.33:0/776573175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff10c0ffd80 0x7ff10c1001a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff1000099c0 tx=0x7ff100009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.566+0000 7ff113039700 1 -- 172.21.15.33:0/776573175 shutdown_connections 2024-05-08T21:18:49.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.566+0000 7ff113039700 1 --2- 172.21.15.33:0/776573175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff10c101910 0x7ff10c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.566+0000 7ff113039700 1 --2- 172.21.15.33:0/776573175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff10c0ffd80 0x7ff10c1001a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.566+0000 7ff113039700 1 --2- 172.21.15.33:0/776573175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff10c100f70 0x7ff10c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.566+0000 7ff113039700 1 -- 172.21.15.33:0/776573175 >> 172.21.15.33:0/776573175 conn(0x7ff10c0fb310 msgr2=0x7ff10c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.566+0000 7ff113039700 1 -- 172.21.15.33:0/776573175 shutdown_connections 2024-05-08T21:18:49.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.566+0000 7ff113039700 1 -- 172.21.15.33:0/776573175 wait complete. 2024-05-08T21:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.566+0000 7ff113039700 1 Processor -- start 2024-05-08T21:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.566+0000 7ff113039700 1 -- start start 2024-05-08T21:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff113039700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff10c0ffd80 0x7ff10c103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff113039700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff10c100f70 0x7ff10c102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff113039700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff10c101910 0x7ff10c102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff113039700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c103030 con 0x7ff10c100f70 2024-05-08T21:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff113039700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c1031a0 con 0x7ff10c0ffd80 2024-05-08T21:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff113039700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c1a0d10 con 0x7ff10c101910 2024-05-08T21:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff10bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff10c100f70 0x7ff10c102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff110dd5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff10c0ffd80 0x7ff10c103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:49.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff10bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff10c100f70 0x7ff10c102350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54928/0 (socket says 172.21.15.33:54928) 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff10bfff700 1 -- 172.21.15.33:0/1797613376 learned_addr learned my addr 172.21.15.33:0/1797613376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff1115d6700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff10c101910 0x7ff10c102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff10bfff700 1 -- 172.21.15.33:0/1797613376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff10c101910 msgr2=0x7ff10c102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff10bfff700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff10c101910 0x7ff10c102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff10bfff700 1 -- 172.21.15.33:0/1797613376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff10c0ffd80 msgr2=0x7ff10c103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff10bfff700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff10c0ffd80 0x7ff10c103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff10bfff700 1 -- 172.21.15.33:0/1797613376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1000095f0 con 0x7ff10c100f70 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff10bfff700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff10c100f70 0x7ff10c102350 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800e9a0 tx=0x7ff0f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.567+0000 7ff109ffb700 1 -- 172.21.15.33:0/1797613376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8009790 con 0x7ff10c100f70 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.568+0000 7ff109ffb700 1 -- 172.21.15.33:0/1797613376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0f800ce40 con 0x7ff10c100f70 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.568+0000 7ff113039700 1 -- 172.21.15.33:0/1797613376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10c1a0fa0 con 0x7ff10c100f70 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.568+0000 7ff109ffb700 1 -- 172.21.15.33:0/1797613376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8018420 con 0x7ff10c100f70 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.568+0000 7ff113039700 1 -- 172.21.15.33:0/1797613376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10c1a1470 con 0x7ff10c100f70 2024-05-08T21:18:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.569+0000 7ff113039700 1 -- 172.21.15.33:0/1797613376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10c067d10 con 0x7ff10c100f70 2024-05-08T21:18:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.570+0000 7ff109ffb700 1 -- 172.21.15.33:0/1797613376 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7ff0f8010430 con 0x7ff10c100f70 2024-05-08T21:18:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.571+0000 7ff109ffb700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff0fc0a1230 0x7ff0fc0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.571+0000 7ff110dd5700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff0fc0a1230 0x7ff0fc0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.571+0000 7ff109ffb700 1 -- 172.21.15.33:0/1797613376 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff0f8014070 con 0x7ff10c100f70 2024-05-08T21:18:49.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.572+0000 7ff110dd5700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff0fc0a1230 0x7ff0fc0a36f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff100000c00 tx=0x7ff100003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:49.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.575+0000 7ff109ffb700 1 -- 172.21.15.33:0/1797613376 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff0f80965d0 con 0x7ff10c100f70 2024-05-08T21:18:49.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:49 smithi033 ceph-mon[42528]: pgmap v283: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:18:49.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:49 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2905245354' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T21:18:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.798+0000 7ff113039700 1 -- 172.21.15.33:0/1797613376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff10c10b440 con 0x7ff10c100f70 2024-05-08T21:18:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.800+0000 7ff109ffb700 1 -- 172.21.15.33:0/1797613376 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24374 (secure 0 0 0) 0x7ff0f8026090 con 0x7ff10c100f70 2024-05-08T21:18:49.806 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:49.806 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:57.410336+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:18:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.802+0000 7ff113039700 1 -- 172.21.15.33:0/1797613376 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff0fc0a1230 msgr2=0x7ff0fc0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.802+0000 7ff113039700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff0fc0a1230 0x7ff0fc0a36f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff100000c00 tx=0x7ff100003680 comp rx=0 tx=0).stop 2024-05-08T21:18:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.802+0000 7ff113039700 1 -- 172.21.15.33:0/1797613376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff10c100f70 msgr2=0x7ff10c102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.802+0000 7ff113039700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff10c100f70 0x7ff10c102350 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800e9a0 tx=0x7ff0f800ecb0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.803+0000 7ff113039700 1 -- 172.21.15.33:0/1797613376 shutdown_connections 2024-05-08T21:18:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.803+0000 7ff113039700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff10c101910 0x7ff10c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.803+0000 7ff113039700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff0fc0a1230 0x7ff0fc0a36f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.803+0000 7ff113039700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff10c0ffd80 0x7ff10c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.803+0000 7ff113039700 1 --2- 172.21.15.33:0/1797613376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff10c100f70 0x7ff10c102350 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.803+0000 7ff113039700 1 -- 172.21.15.33:0/1797613376 >> 172.21.15.33:0/1797613376 conn(0x7ff10c0fb310 msgr2=0x7ff10c068190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.803+0000 7ff113039700 1 -- 172.21.15.33:0/1797613376 shutdown_connections 2024-05-08T21:18:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.803+0000 7ff113039700 1 -- 172.21.15.33:0/1797613376 wait complete. 2024-05-08T21:18:49.819 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:17:01.282803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:17:02.555552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T21:17:05.686277+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T21:18:49.820 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:18:49.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:49 smithi071 ceph-mon[47074]: pgmap v283: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:18:49.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:49 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2905245354' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T21:18:49.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:49 smithi082 ceph-mon[47512]: pgmap v283: 129 pgs: 129 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:18:49.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:49 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2905245354' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-08T21:18:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.959+0000 7fd2dd25c700 1 -- 172.21.15.33:0/1455904396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2d8106be0 msgr2=0x7fd2d8108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.959+0000 7fd2dd25c700 1 --2- 172.21.15.33:0/1455904396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2d8106be0 0x7fd2d8108ff0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000b0a0 tx=0x7fd2d000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.959+0000 7fd2dd25c700 1 -- 172.21.15.33:0/1455904396 shutdown_connections 2024-05-08T21:18:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.959+0000 7fd2dd25c700 1 --2- 172.21.15.33:0/1455904396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2d8106be0 0x7fd2d8108ff0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.959+0000 7fd2dd25c700 1 --2- 172.21.15.33:0/1455904396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8101980 0x7fd2d8103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.959+0000 7fd2dd25c700 1 --2- 172.21.15.33:0/1455904396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d81042b0 0x7fd2d81066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.959+0000 7fd2dd25c700 1 -- 172.21.15.33:0/1455904396 >> 172.21.15.33:0/1455904396 conn(0x7fd2d80fb2d0 msgr2=0x7fd2d80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.959+0000 7fd2dd25c700 1 -- 172.21.15.33:0/1455904396 shutdown_connections 2024-05-08T21:18:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.959+0000 7fd2dd25c700 1 -- 172.21.15.33:0/1455904396 wait complete. 2024-05-08T21:18:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.960+0000 7fd2dd25c700 1 Processor -- start 2024-05-08T21:18:49.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2dd25c700 1 -- start start 2024-05-08T21:18:49.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2dd25c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d8101980 0x7fd2d8198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:49.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2dd25c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2d81042b0 0x7fd2d8199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:49.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2dd25c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8106be0 0x7fd2d81a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:49.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2dd25c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81a0b10 con 0x7fd2d81042b0 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2dd25c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81a0c50 con 0x7fd2d8106be0 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2dd25c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81a0de0 con 0x7fd2d8101980 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2d81042b0 0x7fd2d8199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2ce59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2d81042b0 0x7fd2d8199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54936/0 (socket says 172.21.15.33:54936) 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2ce59c700 1 -- 172.21.15.33:0/3576592055 learned_addr learned my addr 172.21.15.33:0/3576592055 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2d6d9d700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d8101980 0x7fd2d8198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2d759e700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8106be0 0x7fd2d81a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2ce59c700 1 -- 172.21.15.33:0/3576592055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d8101980 msgr2=0x7fd2d8198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2ce59c700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d8101980 0x7fd2d8198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2ce59c700 1 -- 172.21.15.33:0/3576592055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8106be0 msgr2=0x7fd2d81a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2ce59c700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8106be0 0x7fd2d81a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2ce59c700 1 -- 172.21.15.33:0/3576592055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d0009d00 con 0x7fd2d81042b0 2024-05-08T21:18:49.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2d6d9d700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d8101980 0x7fd2d8198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:49.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.961+0000 7fd2ce59c700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2d81042b0 0x7fd2d8199520 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000e9a0 tx=0x7fd2c000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:49.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.962+0000 7fd2d4d99700 1 -- 172.21.15.33:0/3576592055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0009790 con 0x7fd2d81042b0 2024-05-08T21:18:49.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.962+0000 7fd2d4d99700 1 -- 172.21.15.33:0/3576592055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2c000cbd0 con 0x7fd2d81042b0 2024-05-08T21:18:49.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.962+0000 7fd2dd25c700 1 -- 172.21.15.33:0/3576592055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d81a10c0 con 0x7fd2d81042b0 2024-05-08T21:18:49.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.962+0000 7fd2d4d99700 1 -- 172.21.15.33:0/3576592055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0018680 con 0x7fd2d81042b0 2024-05-08T21:18:49.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.962+0000 7fd2dd25c700 1 -- 172.21.15.33:0/3576592055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d81a1620 con 0x7fd2d81042b0 2024-05-08T21:18:49.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.963+0000 7fd2dd25c700 1 -- 172.21.15.33:0/3576592055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d8108760 con 0x7fd2d81042b0 2024-05-08T21:18:49.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.964+0000 7fd2d4d99700 1 -- 172.21.15.33:0/3576592055 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd2c000cd40 con 0x7fd2d81042b0 2024-05-08T21:18:49.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.965+0000 7fd2d4d99700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd2c40a1230 0x7fd2c40a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:49.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.965+0000 7fd2d6d9d700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd2c40a1230 0x7fd2c40a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:49.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.965+0000 7fd2d4d99700 1 -- 172.21.15.33:0/3576592055 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd2c0014070 con 0x7fd2d81042b0 2024-05-08T21:18:49.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.965+0000 7fd2d6d9d700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd2c40a1230 0x7fd2c40a36f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009cb0 tx=0x7fd2c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:49.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:49.969+0000 7fd2d4d99700 1 -- 172.21.15.33:0/3576592055 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd2c0096530 con 0x7fd2d81042b0 2024-05-08T21:18:50.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.188+0000 7fd2dd25c700 1 -- 172.21.15.33:0/3576592055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd2d810c4e0 con 0x7fd2d81042b0 2024-05-08T21:18:50.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.188+0000 7fd2d4d99700 1 -- 172.21.15.33:0/3576592055 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd2c00260a0 con 0x7fd2d81042b0 2024-05-08T21:18:50.191 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:18:50.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.191+0000 7fd2dd25c700 1 -- 172.21.15.33:0/3576592055 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd2c40a1230 msgr2=0x7fd2c40a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:50.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.191+0000 7fd2dd25c700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd2c40a1230 0x7fd2c40a36f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009cb0 tx=0x7fd2c8009260 comp rx=0 tx=0).stop 2024-05-08T21:18:50.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.191+0000 7fd2dd25c700 1 -- 172.21.15.33:0/3576592055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2d81042b0 msgr2=0x7fd2d8199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:50.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.191+0000 7fd2dd25c700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2d81042b0 0x7fd2d8199520 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000e9a0 tx=0x7fd2c000ed60 comp rx=0 tx=0).stop 2024-05-08T21:18:50.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.191+0000 7fd2dd25c700 1 -- 172.21.15.33:0/3576592055 shutdown_connections 2024-05-08T21:18:50.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.191+0000 7fd2dd25c700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd2c40a1230 0x7fd2c40a36f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.191+0000 7fd2dd25c700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d8106be0 0x7fd2d81a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.191+0000 7fd2dd25c700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d8101980 0x7fd2d8198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.191+0000 7fd2dd25c700 1 --2- 172.21.15.33:0/3576592055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2d81042b0 0x7fd2d8199520 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.191+0000 7fd2dd25c700 1 -- 172.21.15.33:0/3576592055 >> 172.21.15.33:0/3576592055 conn(0x7fd2d80fb2d0 msgr2=0x7fd2d8101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:50.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.191+0000 7fd2dd25c700 1 -- 172.21.15.33:0/3576592055 shutdown_connections 2024-05-08T21:18:50.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.191+0000 7fd2dd25c700 1 -- 172.21.15.33:0/3576592055 wait complete. 2024-05-08T21:18:50.207 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:18:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.346+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/2774345945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcd0101920 msgr2=0x7fdcd01081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.346+0000 7fdcd7a3d700 1 --2- 172.21.15.33:0/2774345945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcd0101920 0x7fdcd01081d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800b0d0 tx=0x7fdcc800b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.346+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/2774345945 shutdown_connections 2024-05-08T21:18:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.346+0000 7fdcd7a3d700 1 --2- 172.21.15.33:0/2774345945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcd0101920 0x7fdcd01081d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.346+0000 7fdcd7a3d700 1 --2- 172.21.15.33:0/2774345945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd00ffd90 0x7fdcd01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.346+0000 7fdcd7a3d700 1 --2- 172.21.15.33:0/2774345945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd0100f80 0x7fdcd01013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.346+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/2774345945 >> 172.21.15.33:0/2774345945 conn(0x7fdcd00fb300 msgr2=0x7fdcd00fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.346+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/2774345945 shutdown_connections 2024-05-08T21:18:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.346+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/2774345945 wait complete. 2024-05-08T21:18:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.347+0000 7fdcd7a3d700 1 Processor -- start 2024-05-08T21:18:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.347+0000 7fdcd7a3d700 1 -- start start 2024-05-08T21:18:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.347+0000 7fdcd7a3d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd00ffd90 0x7fdcd0196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.347+0000 7fdcd7a3d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd0100f80 0x7fdcd0197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.347+0000 7fdcd7a3d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcd0101920 0x7fdcd019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.347+0000 7fdcd7a3d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd019e8f0 con 0x7fdcd0101920 2024-05-08T21:18:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.347+0000 7fdcd7a3d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd019ea60 con 0x7fdcd0100f80 2024-05-08T21:18:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.347+0000 7fdcd7a3d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd019ec20 con 0x7fdcd00ffd90 2024-05-08T21:18:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.347+0000 7fdcd5fda700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcd0101920 0x7fdcd019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.347+0000 7fdcd5fda700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcd0101920 0x7fdcd019e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54964/0 (socket says 172.21.15.33:54964) 2024-05-08T21:18:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.347+0000 7fdcd5fda700 1 -- 172.21.15.33:0/779475751 learned_addr learned my addr 172.21.15.33:0/779475751 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.347+0000 7fdcd4fd8700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd0100f80 0x7fdcd0197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.348+0000 7fdcd57d9700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd00ffd90 0x7fdcd0196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.348+0000 7fdcd5fda700 1 -- 172.21.15.33:0/779475751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd00ffd90 msgr2=0x7fdcd0196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.348+0000 7fdcd5fda700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd00ffd90 0x7fdcd0196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.348+0000 7fdcd5fda700 1 -- 172.21.15.33:0/779475751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd0100f80 msgr2=0x7fdcd0197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.348+0000 7fdcd5fda700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd0100f80 0x7fdcd0197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.348+0000 7fdcd5fda700 1 -- 172.21.15.33:0/779475751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc0009520 con 0x7fdcd0101920 2024-05-08T21:18:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.348+0000 7fdcd5fda700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcd0101920 0x7fdcd019e320 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8007af0 tx=0x7fdcc8007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.348+0000 7fdcc67fc700 1 -- 172.21.15.33:0/779475751 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc800e040 con 0x7fdcd0101920 2024-05-08T21:18:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.348+0000 7fdcc67fc700 1 -- 172.21.15.33:0/779475751 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdcc801d900 con 0x7fdcd0101920 2024-05-08T21:18:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.348+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/779475751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc8009d00 con 0x7fdcd0101920 2024-05-08T21:18:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.348+0000 7fdcc67fc700 1 -- 172.21.15.33:0/779475751 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc801b890 con 0x7fdcd0101920 2024-05-08T21:18:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.348+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/779475751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd019f190 con 0x7fdcd0101920 2024-05-08T21:18:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.349+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/779475751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd0067480 con 0x7fdcd0101920 2024-05-08T21:18:50.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.351+0000 7fdcc67fc700 1 -- 172.21.15.33:0/779475751 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fdcc801da70 con 0x7fdcd0101920 2024-05-08T21:18:50.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.352+0000 7fdcc67fc700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdcbc0a11b0 0x7fdcbc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:50.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.352+0000 7fdcd57d9700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdcbc0a11b0 0x7fdcbc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:50.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.352+0000 7fdcc67fc700 1 -- 172.21.15.33:0/779475751 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdcc80cec60 con 0x7fdcd0101920 2024-05-08T21:18:50.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.352+0000 7fdcd57d9700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdcbc0a11b0 0x7fdcbc0a3670 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0198280 tx=0x7fdccc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:50.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.356+0000 7fdcc67fc700 1 -- 172.21.15.33:0/779475751 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fdcc8098010 con 0x7fdcd0101920 2024-05-08T21:18:50.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:50 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2905245354' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T21:18:50.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:50 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:18:50.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:50 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1797613376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:50.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:50 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3576592055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:50.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.577+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/779475751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdcd010b620 con 0x7fdcd0101920 2024-05-08T21:18:50.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.577+0000 7fdcc67fc700 1 -- 172.21.15.33:0/779475751 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdcc802e380 con 0x7fdcd0101920 2024-05-08T21:18:50.580 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:50.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.580+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/779475751 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdcbc0a11b0 msgr2=0x7fdcbc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:50.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.580+0000 7fdcd7a3d700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdcbc0a11b0 0x7fdcbc0a3670 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0198280 tx=0x7fdccc009260 comp rx=0 tx=0).stop 2024-05-08T21:18:50.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.580+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/779475751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcd0101920 msgr2=0x7fdcd019e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:50.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.580+0000 7fdcd7a3d700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcd0101920 0x7fdcd019e320 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8007af0 tx=0x7fdcc8007870 comp rx=0 tx=0).stop 2024-05-08T21:18:50.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.580+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/779475751 shutdown_connections 2024-05-08T21:18:50.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.580+0000 7fdcd7a3d700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcd0101920 0x7fdcd019e320 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.580+0000 7fdcd7a3d700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcd00ffd90 0x7fdcd0196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.580+0000 7fdcd7a3d700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdcbc0a11b0 0x7fdcbc0a3670 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.580+0000 7fdcd7a3d700 1 --2- 172.21.15.33:0/779475751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcd0100f80 0x7fdcd0197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.580+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/779475751 >> 172.21.15.33:0/779475751 conn(0x7fdcd00fb300 msgr2=0x7fdcd0069060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:50.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.580+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/779475751 shutdown_connections 2024-05-08T21:18:50.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.580+0000 7fdcd7a3d700 1 -- 172.21.15.33:0/779475751 wait complete. 2024-05-08T21:18:50.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:18:50.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.736+0000 7f54c192b700 1 -- 172.21.15.33:0/2287383430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54bc102990 msgr2=0x7f54bc1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:50.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.736+0000 7f54c192b700 1 --2- 172.21.15.33:0/2287383430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54bc102990 0x7f54bc1092b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f54b400b0a0 tx=0x7f54b400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.736+0000 7f54c192b700 1 -- 172.21.15.33:0/2287383430 shutdown_connections 2024-05-08T21:18:50.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.736+0000 7f54c192b700 1 --2- 172.21.15.33:0/2287383430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54bc102990 0x7f54bc1092b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.736+0000 7f54c192b700 1 --2- 172.21.15.33:0/2287383430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc100e80 0x7f54bc1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.736+0000 7f54c192b700 1 --2- 172.21.15.33:0/2287383430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54bc101fd0 0x7f54bc102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.736+0000 7f54c192b700 1 -- 172.21.15.33:0/2287383430 >> 172.21.15.33:0/2287383430 conn(0x7f54bc0fc3f0 msgr2=0x7f54bc0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.736+0000 7f54c192b700 1 -- 172.21.15.33:0/2287383430 shutdown_connections 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.736+0000 7f54c192b700 1 -- 172.21.15.33:0/2287383430 wait complete. 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.737+0000 7f54c192b700 1 Processor -- start 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.737+0000 7f54c192b700 1 -- start start 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.737+0000 7f54c192b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54bc100e80 0x7f54bc196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.737+0000 7f54c192b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc101fd0 0x7f54bc197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.737+0000 7f54c192b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54bc102990 0x7f54bc19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.737+0000 7f54c192b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc19e960 con 0x7f54bc102990 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.737+0000 7f54c192b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc19eaa0 con 0x7f54bc101fd0 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.737+0000 7f54c192b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc19ec30 con 0x7f54bc100e80 2024-05-08T21:18:50.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54bc102990 0x7f54bc19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54ba7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc101fd0 0x7f54bc197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54bc102990 0x7f54bc19e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54034/0 (socket says 172.21.15.33:54034) 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54ba7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc101fd0 0x7f54bc197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:47906/0 (socket says 172.21.15.33:47906) 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54bb7fe700 1 -- 172.21.15.33:0/113123028 learned_addr learned my addr 172.21.15.33:0/113123028 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54baffd700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54bc100e80 0x7f54bc196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54bb7fe700 1 -- 172.21.15.33:0/113123028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54bc100e80 msgr2=0x7f54bc196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54bb7fe700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54bc100e80 0x7f54bc196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54bb7fe700 1 -- 172.21.15.33:0/113123028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc101fd0 msgr2=0x7f54bc197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54bb7fe700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc101fd0 0x7f54bc197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54bb7fe700 1 -- 172.21.15.33:0/113123028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4009d00 con 0x7f54bc102990 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54ba7fc700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc101fd0 0x7f54bc197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54baffd700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54bc100e80 0x7f54bc196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54bb7fe700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54bc102990 0x7f54bc19e390 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f54b4007950 tx=0x7f54b4007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54c0929700 1 -- 172.21.15.33:0/113123028 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b400e040 con 0x7f54bc102990 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54c0929700 1 -- 172.21.15.33:0/113123028 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54b4004c50 con 0x7f54bc102990 2024-05-08T21:18:50.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54c192b700 1 -- 172.21.15.33:0/113123028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54bc19eeb0 con 0x7f54bc102990 2024-05-08T21:18:50.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54c0929700 1 -- 172.21.15.33:0/113123028 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4005040 con 0x7f54bc102990 2024-05-08T21:18:50.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.738+0000 7f54c192b700 1 -- 172.21.15.33:0/113123028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54bc19f2d0 con 0x7f54bc102990 2024-05-08T21:18:50.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.740+0000 7f54c192b700 1 -- 172.21.15.33:0/113123028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54bc108240 con 0x7f54bc102990 2024-05-08T21:18:50.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.741+0000 7f54c0929700 1 -- 172.21.15.33:0/113123028 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f54b401d470 con 0x7f54bc102990 2024-05-08T21:18:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.741+0000 7f54c0929700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f54a40a11b0 0x7f54a40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.741+0000 7f54c0929700 1 -- 172.21.15.33:0/113123028 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f54b40ce250 con 0x7f54bc102990 2024-05-08T21:18:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.741+0000 7f54baffd700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f54a40a11b0 0x7f54a40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:50.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.742+0000 7f54baffd700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f54a40a11b0 0x7f54a40a3670 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f54ac011490 tx=0x7f54ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:50.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.746+0000 7f54c0929700 1 -- 172.21.15.33:0/113123028 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f54b4097600 con 0x7f54bc102990 2024-05-08T21:18:50.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:50 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2905245354' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T21:18:50.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:50 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:18:50.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:50 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1797613376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:50.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:50 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3576592055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:50.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:50 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2905245354' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-08T21:18:50.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:50 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:18:50.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:50 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1797613376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:50.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:50 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3576592055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.966+0000 7f54c192b700 1 -- 172.21.15.33:0/113123028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54bc10c580 con 0x7f54bc102990 2024-05-08T21:18:50.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.966+0000 7f54c0929700 1 -- 172.21.15.33:0/113123028 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f54b409ac20 con 0x7f54bc102990 2024-05-08T21:18:50.969 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T21:18:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.968+0000 7f54c192b700 1 -- 172.21.15.33:0/113123028 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f54a40a11b0 msgr2=0x7f54a40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.968+0000 7f54c192b700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f54a40a11b0 0x7f54a40a3670 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f54ac011490 tx=0x7f54ac009260 comp rx=0 tx=0).stop 2024-05-08T21:18:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.968+0000 7f54c192b700 1 -- 172.21.15.33:0/113123028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54bc102990 msgr2=0x7f54bc19e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.968+0000 7f54c192b700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54bc102990 0x7f54bc19e390 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f54b4007950 tx=0x7f54b4007870 comp rx=0 tx=0).stop 2024-05-08T21:18:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.968+0000 7f54c192b700 1 -- 172.21.15.33:0/113123028 shutdown_connections 2024-05-08T21:18:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.968+0000 7f54c192b700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54bc102990 0x7f54bc19e390 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.968+0000 7f54c192b700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54bc100e80 0x7f54bc196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.968+0000 7f54c192b700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f54a40a11b0 0x7f54a40a3670 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.968+0000 7f54c192b700 1 --2- 172.21.15.33:0/113123028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54bc101fd0 0x7f54bc197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.968+0000 7f54c192b700 1 -- 172.21.15.33:0/113123028 >> 172.21.15.33:0/113123028 conn(0x7f54bc0fc3f0 msgr2=0x7f54bc105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.969+0000 7f54c192b700 1 -- 172.21.15.33:0/113123028 shutdown_connections 2024-05-08T21:18:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:50.969+0000 7f54c192b700 1 -- 172.21.15.33:0/113123028 wait complete. 2024-05-08T21:18:50.985 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T21:18:51.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.124+0000 7fec963b2700 1 -- 172.21.15.33:0/2748927498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec901018f0 msgr2=0x7fec901081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.124+0000 7fec963b2700 1 --2- 172.21.15.33:0/2748927498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec901018f0 0x7fec901081a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fec8800b0d0 tx=0x7fec8800b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.124+0000 7fec963b2700 1 -- 172.21.15.33:0/2748927498 shutdown_connections 2024-05-08T21:18:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.124+0000 7fec963b2700 1 --2- 172.21.15.33:0/2748927498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec901018f0 0x7fec901081a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.124+0000 7fec963b2700 1 --2- 172.21.15.33:0/2748927498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec900ffd60 0x7fec90100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.124+0000 7fec963b2700 1 --2- 172.21.15.33:0/2748927498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec90100f50 0x7fec901013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.124+0000 7fec963b2700 1 -- 172.21.15.33:0/2748927498 >> 172.21.15.33:0/2748927498 conn(0x7fec900fb290 msgr2=0x7fec900fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.124+0000 7fec963b2700 1 -- 172.21.15.33:0/2748927498 shutdown_connections 2024-05-08T21:18:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.124+0000 7fec963b2700 1 -- 172.21.15.33:0/2748927498 wait complete. 2024-05-08T21:18:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.124+0000 7fec963b2700 1 Processor -- start 2024-05-08T21:18:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec963b2700 1 -- start start 2024-05-08T21:18:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec963b2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec900ffd60 0x7fec90198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:51.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec963b2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec90100f50 0x7fec90199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec963b2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec901018f0 0x7fec901a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec963b2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec901a0b20 con 0x7fec901018f0 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec963b2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec901a0c60 con 0x7fec90100f50 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec963b2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec901a0df0 con 0x7fec900ffd60 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec9494f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec901018f0 0x7fec901a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec9494f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec901018f0 0x7fec901a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54046/0 (socket says 172.21.15.33:54046) 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec9494f700 1 -- 172.21.15.33:0/1610210506 learned_addr learned my addr 172.21.15.33:0/1610210506 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec8ffff700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec900ffd60 0x7fec90198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec877fe700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec90100f50 0x7fec90199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec9494f700 1 -- 172.21.15.33:0/1610210506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec900ffd60 msgr2=0x7fec90198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec9494f700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec900ffd60 0x7fec90198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec9494f700 1 -- 172.21.15.33:0/1610210506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec90100f50 msgr2=0x7fec90199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec9494f700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec90100f50 0x7fec90199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec9494f700 1 -- 172.21.15.33:0/1610210506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec88009d00 con 0x7fec901018f0 2024-05-08T21:18:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec877fe700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec90100f50 0x7fec90199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.125+0000 7fec9494f700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec901018f0 0x7fec901a0550 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fec880079c0 tx=0x7fec8800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.126+0000 7fec8dffb700 1 -- 172.21.15.33:0/1610210506 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8800e040 con 0x7fec901018f0 2024-05-08T21:18:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.126+0000 7fec963b2700 1 -- 172.21.15.33:0/1610210506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec901a1070 con 0x7fec901018f0 2024-05-08T21:18:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.126+0000 7fec8dffb700 1 -- 172.21.15.33:0/1610210506 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec88004e60 con 0x7fec901018f0 2024-05-08T21:18:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.126+0000 7fec8dffb700 1 -- 172.21.15.33:0/1610210506 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8801b6b0 con 0x7fec901018f0 2024-05-08T21:18:51.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.126+0000 7fec963b2700 1 -- 172.21.15.33:0/1610210506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec901a1540 con 0x7fec901018f0 2024-05-08T21:18:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.127+0000 7fec963b2700 1 -- 172.21.15.33:0/1610210506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec90067d10 con 0x7fec901018f0 2024-05-08T21:18:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.128+0000 7fec8dffb700 1 -- 172.21.15.33:0/1610210506 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fec8801d9a0 con 0x7fec901018f0 2024-05-08T21:18:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.129+0000 7fec8dffb700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fec7c0a11b0 0x7fec7c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.129+0000 7fec8dffb700 1 -- 172.21.15.33:0/1610210506 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fec880ce690 con 0x7fec901018f0 2024-05-08T21:18:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.129+0000 7fec8ffff700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fec7c0a11b0 0x7fec7c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:51.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.129+0000 7fec8ffff700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fec7c0a11b0 0x7fec7c0a3670 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fec90100db0 tx=0x7fec80007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:51.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.134+0000 7fec8dffb700 1 -- 172.21.15.33:0/1610210506 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fec88097a40 con 0x7fec901018f0 2024-05-08T21:18:51.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.353+0000 7fec963b2700 1 -- 172.21.15.33:0/1610210506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fec9010b430 con 0x7fec901018f0 2024-05-08T21:18:51.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.353+0000 7fec8dffb700 1 -- 172.21.15.33:0/1610210506 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fec8809b060 con 0x7fec901018f0 2024-05-08T21:18:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:51.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.356+0000 7fec963b2700 1 -- 172.21.15.33:0/1610210506 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fec7c0a11b0 msgr2=0x7fec7c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.356+0000 7fec963b2700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fec7c0a11b0 0x7fec7c0a3670 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fec90100db0 tx=0x7fec80007400 comp rx=0 tx=0).stop 2024-05-08T21:18:51.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.356+0000 7fec963b2700 1 -- 172.21.15.33:0/1610210506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec901018f0 msgr2=0x7fec901a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.356+0000 7fec963b2700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec901018f0 0x7fec901a0550 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fec880079c0 tx=0x7fec8800ccc0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.356+0000 7fec963b2700 1 -- 172.21.15.33:0/1610210506 shutdown_connections 2024-05-08T21:18:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.356+0000 7fec963b2700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec901018f0 0x7fec901a0550 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.356+0000 7fec963b2700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec900ffd60 0x7fec90198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.356+0000 7fec963b2700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fec7c0a11b0 0x7fec7c0a3670 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.356+0000 7fec963b2700 1 --2- 172.21.15.33:0/1610210506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec90100f50 0x7fec90199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.356+0000 7fec963b2700 1 -- 172.21.15.33:0/1610210506 >> 172.21.15.33:0/1610210506 conn(0x7fec900fb290 msgr2=0x7fec90068160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.356+0000 7fec963b2700 1 -- 172.21.15.33:0/1610210506 shutdown_connections 2024-05-08T21:18:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.356+0000 7fec963b2700 1 -- 172.21.15.33:0/1610210506 wait complete. 2024-05-08T21:18:51.372 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:18:51.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.509+0000 7fc9feeb2700 1 -- 172.21.15.33:0/278252702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f80ffd80 msgr2=0x7fc9f81001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.509+0000 7fc9feeb2700 1 --2- 172.21.15.33:0/278252702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f80ffd80 0x7fc9f81001a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0099c0 tx=0x7fc9ec009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.509+0000 7fc9feeb2700 1 -- 172.21.15.33:0/278252702 shutdown_connections 2024-05-08T21:18:51.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.509+0000 7fc9feeb2700 1 --2- 172.21.15.33:0/278252702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f8101910 0x7fc9f81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.509+0000 7fc9feeb2700 1 --2- 172.21.15.33:0/278252702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f80ffd80 0x7fc9f81001a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.509+0000 7fc9feeb2700 1 --2- 172.21.15.33:0/278252702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9f8100f70 0x7fc9f81013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.509+0000 7fc9feeb2700 1 -- 172.21.15.33:0/278252702 >> 172.21.15.33:0/278252702 conn(0x7fc9f80fb310 msgr2=0x7fc9f80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:51.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.509+0000 7fc9feeb2700 1 -- 172.21.15.33:0/278252702 shutdown_connections 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.509+0000 7fc9feeb2700 1 -- 172.21.15.33:0/278252702 wait complete. 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.509+0000 7fc9feeb2700 1 Processor -- start 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.509+0000 7fc9feeb2700 1 -- start start 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9feeb2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f80ffd80 0x7fc9f8196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9feeb2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9f8100f70 0x7fc9f8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9feeb2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8101910 0x7fc9f819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9feeb2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f819e8e0 con 0x7fc9f8101910 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9feeb2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f819ea20 con 0x7fc9f8100f70 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9feeb2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f819ebb0 con 0x7fc9f80ffd80 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9fd44f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8101910 0x7fc9f819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9fd44f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8101910 0x7fc9f819e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54068/0 (socket says 172.21.15.33:54068) 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9fcc4e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f80ffd80 0x7fc9f8196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9fd44f700 1 -- 172.21.15.33:0/763436918 learned_addr learned my addr 172.21.15.33:0/763436918 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:51.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9f7fff700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9f8100f70 0x7fc9f8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:51.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9fd44f700 1 -- 172.21.15.33:0/763436918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f80ffd80 msgr2=0x7fc9f8196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9fd44f700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f80ffd80 0x7fc9f8196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9fd44f700 1 -- 172.21.15.33:0/763436918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9f8100f70 msgr2=0x7fc9f8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9fd44f700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9f8100f70 0x7fc9f8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9fd44f700 1 -- 172.21.15.33:0/763436918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec0095f0 con 0x7fc9f8101910 2024-05-08T21:18:51.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9fd44f700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8101910 0x7fc9f819e310 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0007ee0 tx=0x7fc9f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:51.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.510+0000 7fc9f5ffb700 1 -- 172.21.15.33:0/763436918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0017070 con 0x7fc9f8101910 2024-05-08T21:18:51.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.511+0000 7fc9f5ffb700 1 -- 172.21.15.33:0/763436918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9f000f040 con 0x7fc9f8101910 2024-05-08T21:18:51.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.511+0000 7fc9feeb2700 1 -- 172.21.15.33:0/763436918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f819ee90 con 0x7fc9f8101910 2024-05-08T21:18:51.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.511+0000 7fc9f5ffb700 1 -- 172.21.15.33:0/763436918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0005570 con 0x7fc9f8101910 2024-05-08T21:18:51.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.511+0000 7fc9feeb2700 1 -- 172.21.15.33:0/763436918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f819f2c0 con 0x7fc9f8101910 2024-05-08T21:18:51.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.512+0000 7fc9feeb2700 1 -- 172.21.15.33:0/763436918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f8067d10 con 0x7fc9f8101910 2024-05-08T21:18:51.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.513+0000 7fc9f5ffb700 1 -- 172.21.15.33:0/763436918 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc9f000db20 con 0x7fc9f8101910 2024-05-08T21:18:51.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.514+0000 7fc9f5ffb700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9e80a11b0 0x7fc9e80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:51.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.514+0000 7fc9fcc4e700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9e80a11b0 0x7fc9e80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:51.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.514+0000 7fc9f5ffb700 1 -- 172.21.15.33:0/763436918 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc9f00cc580 con 0x7fc9f8101910 2024-05-08T21:18:51.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.514+0000 7fc9fcc4e700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9e80a11b0 0x7fc9e80a3670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec006010 tx=0x7fc9ec003740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:51.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.519+0000 7fc9f5ffb700 1 -- 172.21.15.33:0/763436918 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc9f0095840 con 0x7fc9f8101910 2024-05-08T21:18:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:51 smithi033 ceph-mon[42528]: pgmap v284: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-08T21:18:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:51 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/779475751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:51 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/113123028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:51 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1610210506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.737+0000 7fc9feeb2700 1 -- 172.21.15.33:0/763436918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9f810b490 con 0x7fc9f8101910 2024-05-08T21:18:51.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.739+0000 7fc9f5ffb700 1 -- 172.21.15.33:0/763436918 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24374 (secure 0 0 0) 0x7fc9f0098e60 con 0x7fc9f8101910 2024-05-08T21:18:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:51.742 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:57.410336+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:18:51.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.741+0000 7fc9feeb2700 1 -- 172.21.15.33:0/763436918 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9e80a11b0 msgr2=0x7fc9e80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.741+0000 7fc9feeb2700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9e80a11b0 0x7fc9e80a3670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec006010 tx=0x7fc9ec003740 comp rx=0 tx=0).stop 2024-05-08T21:18:51.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.741+0000 7fc9feeb2700 1 -- 172.21.15.33:0/763436918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8101910 msgr2=0x7fc9f819e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.741+0000 7fc9feeb2700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8101910 0x7fc9f819e310 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0007ee0 tx=0x7fc9f000d3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.741+0000 7fc9feeb2700 1 -- 172.21.15.33:0/763436918 shutdown_connections 2024-05-08T21:18:51.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.741+0000 7fc9feeb2700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8101910 0x7fc9f819e310 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.741+0000 7fc9feeb2700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f80ffd80 0x7fc9f8196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.741+0000 7fc9feeb2700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc9e80a11b0 0x7fc9e80a3670 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.741+0000 7fc9feeb2700 1 --2- 172.21.15.33:0/763436918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9f8100f70 0x7fc9f8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.741+0000 7fc9feeb2700 1 -- 172.21.15.33:0/763436918 >> 172.21.15.33:0/763436918 conn(0x7fc9f80fb310 msgr2=0x7fc9f8068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:51.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.742+0000 7fc9feeb2700 1 -- 172.21.15.33:0/763436918 shutdown_connections 2024-05-08T21:18:51.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.742+0000 7fc9feeb2700 1 -- 172.21.15.33:0/763436918 wait complete. 2024-05-08T21:18:51.758 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:17:01.282803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:17:02.555552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T21:17:05.686277+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T21:18:51.758 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:18:51.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:51 smithi071 ceph-mon[47074]: pgmap v284: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-08T21:18:51.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:51 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/779475751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:51.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:51 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/113123028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:51.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:51 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1610210506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:51.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:51 smithi082 ceph-mon[47512]: pgmap v284: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-08T21:18:51.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:51 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/779475751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:51.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:51 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/113123028' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:51.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:51 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1610210506' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.896+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/2400883562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a38104210 msgr2=0x7f3a38106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.896+0000 7f3a3fcf8700 1 --2- 172.21.15.33:0/2400883562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a38104210 0x7f3a38106600 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009990 tx=0x7f3a28009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.896+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/2400883562 shutdown_connections 2024-05-08T21:18:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.896+0000 7f3a3fcf8700 1 --2- 172.21.15.33:0/2400883562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a38106b40 0x7f3a38108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.896+0000 7f3a3fcf8700 1 --2- 172.21.15.33:0/2400883562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a381018e0 0x7f3a38103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.896+0000 7f3a3fcf8700 1 --2- 172.21.15.33:0/2400883562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a38104210 0x7f3a38106600 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.896+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/2400883562 >> 172.21.15.33:0/2400883562 conn(0x7f3a380fb290 msgr2=0x7f3a380fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.896+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/2400883562 shutdown_connections 2024-05-08T21:18:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.896+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/2400883562 wait complete. 2024-05-08T21:18:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.897+0000 7f3a3fcf8700 1 Processor -- start 2024-05-08T21:18:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.897+0000 7f3a3fcf8700 1 -- start start 2024-05-08T21:18:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.897+0000 7f3a3fcf8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a381018e0 0x7f3a38196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.897+0000 7f3a3fcf8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a38104210 0x7f3a38197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.897+0000 7f3a3fcf8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a38106b40 0x7f3a3819e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.897+0000 7f3a3fcf8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3819e820 con 0x7f3a381018e0 2024-05-08T21:18:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.897+0000 7f3a3fcf8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3819e990 con 0x7f3a38104210 2024-05-08T21:18:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.897+0000 7f3a3fcf8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3819eb50 con 0x7f3a38106b40 2024-05-08T21:18:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.897+0000 7f3a3da94700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a381018e0 0x7f3a38196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.897+0000 7f3a3da94700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a381018e0 0x7f3a38196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54072/0 (socket says 172.21.15.33:54072) 2024-05-08T21:18:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.897+0000 7f3a3da94700 1 -- 172.21.15.33:0/4046153622 learned_addr learned my addr 172.21.15.33:0/4046153622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a3e295700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a38106b40 0x7f3a3819e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a3d293700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a38104210 0x7f3a38197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a3da94700 1 -- 172.21.15.33:0/4046153622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a38106b40 msgr2=0x7f3a3819e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a3da94700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a38106b40 0x7f3a3819e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a3da94700 1 -- 172.21.15.33:0/4046153622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a38104210 msgr2=0x7f3a38197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a3da94700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a38104210 0x7f3a38197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a3da94700 1 -- 172.21.15.33:0/4046153622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a280095f0 con 0x7f3a381018e0 2024-05-08T21:18:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a3d293700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a38104210 0x7f3a38197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:51.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a3da94700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a381018e0 0x7f3a38196d00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400e9a0 tx=0x7f3a3400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:51.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a2effd700 1 -- 172.21.15.33:0/4046153622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34009790 con 0x7f3a381018e0 2024-05-08T21:18:51.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a2effd700 1 -- 172.21.15.33:0/4046153622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a3400cbd0 con 0x7f3a381018e0 2024-05-08T21:18:51.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/4046153622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3819ee30 con 0x7f3a381018e0 2024-05-08T21:18:51.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a2effd700 1 -- 172.21.15.33:0/4046153622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a34018680 con 0x7f3a381018e0 2024-05-08T21:18:51.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.898+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/4046153622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3819f250 con 0x7f3a381018e0 2024-05-08T21:18:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.899+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/4046153622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a380fce10 con 0x7f3a381018e0 2024-05-08T21:18:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.901+0000 7f3a2effd700 1 -- 172.21.15.33:0/4046153622 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3a3400cd40 con 0x7f3a381018e0 2024-05-08T21:18:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.902+0000 7f3a2effd700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3a240a11b0 0x7f3a240a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.902+0000 7f3a3d293700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3a240a11b0 0x7f3a240a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.902+0000 7f3a2effd700 1 -- 172.21.15.33:0/4046153622 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3a34014070 con 0x7f3a381018e0 2024-05-08T21:18:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.902+0000 7f3a3d293700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3a240a11b0 0x7f3a240a3670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3a381981b0 tx=0x7f3a28005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:51.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:51.906+0000 7f3a2effd700 1 -- 172.21.15.33:0/4046153622 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3a340966d0 con 0x7f3a381018e0 2024-05-08T21:18:52.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.120+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/4046153622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a3802cda0 con 0x7f3a381018e0 2024-05-08T21:18:52.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.120+0000 7f3a2effd700 1 -- 172.21.15.33:0/4046153622 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a34099cf0 con 0x7f3a381018e0 2024-05-08T21:18:52.123 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:18:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.122+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/4046153622 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3a240a11b0 msgr2=0x7f3a240a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.122+0000 7f3a3fcf8700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3a240a11b0 0x7f3a240a3670 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3a381981b0 tx=0x7f3a28005fb0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.122+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/4046153622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a381018e0 msgr2=0x7f3a38196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.122+0000 7f3a3fcf8700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a381018e0 0x7f3a38196d00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400e9a0 tx=0x7f3a3400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.122+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/4046153622 shutdown_connections 2024-05-08T21:18:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.122+0000 7f3a3fcf8700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a38106b40 0x7f3a3819e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.122+0000 7f3a3fcf8700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a381018e0 0x7f3a38196d00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.122+0000 7f3a3fcf8700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3a240a11b0 0x7f3a240a3670 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.122+0000 7f3a3fcf8700 1 --2- 172.21.15.33:0/4046153622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a38104210 0x7f3a38197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.122+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/4046153622 >> 172.21.15.33:0/4046153622 conn(0x7f3a380fb290 msgr2=0x7f3a381023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.122+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/4046153622 shutdown_connections 2024-05-08T21:18:52.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.122+0000 7f3a3fcf8700 1 -- 172.21.15.33:0/4046153622 wait complete. 2024-05-08T21:18:52.139 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:18:52.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.276+0000 7f33496dd700 1 -- 172.21.15.33:0/2107580386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3344100f70 msgr2=0x7f33441013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:52.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.276+0000 7f33496dd700 1 --2- 172.21.15.33:0/2107580386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3344100f70 0x7f33441013d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f33380099c0 tx=0x7f3338009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.276+0000 7f33496dd700 1 -- 172.21.15.33:0/2107580386 shutdown_connections 2024-05-08T21:18:52.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.276+0000 7f33496dd700 1 --2- 172.21.15.33:0/2107580386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3344101910 0x7f33441081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.276+0000 7f33496dd700 1 --2- 172.21.15.33:0/2107580386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33440ffd80 0x7f33441001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.276+0000 7f33496dd700 1 --2- 172.21.15.33:0/2107580386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3344100f70 0x7f33441013d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.276+0000 7f33496dd700 1 -- 172.21.15.33:0/2107580386 >> 172.21.15.33:0/2107580386 conn(0x7f33440fb310 msgr2=0x7f33440fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:52.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.276+0000 7f33496dd700 1 -- 172.21.15.33:0/2107580386 shutdown_connections 2024-05-08T21:18:52.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.276+0000 7f33496dd700 1 -- 172.21.15.33:0/2107580386 wait complete. 2024-05-08T21:18:52.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.277+0000 7f33496dd700 1 Processor -- start 2024-05-08T21:18:52.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.277+0000 7f33496dd700 1 -- start start 2024-05-08T21:18:52.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.277+0000 7f33496dd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33440ffd80 0x7f3344103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:52.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.277+0000 7f33496dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3344100f70 0x7f3344102590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.277+0000 7f33496dd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3344101910 0x7f3344102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.277+0000 7f33496dd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3344103270 con 0x7f3344100f70 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33496dd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33441033e0 con 0x7f33440ffd80 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33496dd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33441a0db0 con 0x7f3344101910 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33427fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3344100f70 0x7f3344102590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33427fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3344100f70 0x7f3344102590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54092/0 (socket says 172.21.15.33:54092) 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33427fc700 1 -- 172.21.15.33:0/2371362809 learned_addr learned my addr 172.21.15.33:0/2371362809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f3342ffd700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33440ffd80 0x7f3344103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33437fe700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3344101910 0x7f3344102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33427fc700 1 -- 172.21.15.33:0/2371362809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3344101910 msgr2=0x7f3344102ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33427fc700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3344101910 0x7f3344102ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33427fc700 1 -- 172.21.15.33:0/2371362809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33440ffd80 msgr2=0x7f3344103f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33427fc700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33440ffd80 0x7f3344103f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33427fc700 1 -- 172.21.15.33:0/2371362809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c009c40 con 0x7f3344100f70 2024-05-08T21:18:52.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33437fe700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3344101910 0x7f3344102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:52.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33427fc700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3344100f70 0x7f3344102590 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3338000c00 tx=0x7f333800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:52.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f332bfff700 1 -- 172.21.15.33:0/2371362809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333801d070 con 0x7f3344100f70 2024-05-08T21:18:52.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33496dd700 1 -- 172.21.15.33:0/2371362809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33380095f0 con 0x7f3344100f70 2024-05-08T21:18:52.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f332bfff700 1 -- 172.21.15.33:0/2371362809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f33380054e0 con 0x7f3344100f70 2024-05-08T21:18:52.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f332bfff700 1 -- 172.21.15.33:0/2371362809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3338004b30 con 0x7f3344100f70 2024-05-08T21:18:52.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.278+0000 7f33496dd700 1 -- 172.21.15.33:0/2371362809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33441a11c0 con 0x7f3344100f70 2024-05-08T21:18:52.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.279+0000 7f33496dd700 1 -- 172.21.15.33:0/2371362809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33440681d0 con 0x7f3344100f70 2024-05-08T21:18:52.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.281+0000 7f332bfff700 1 -- 172.21.15.33:0/2371362809 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3338004c90 con 0x7f3344100f70 2024-05-08T21:18:52.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.281+0000 7f332bfff700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f332c0a11b0 0x7f332c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:52.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.281+0000 7f3342ffd700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f332c0a11b0 0x7f332c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:52.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.281+0000 7f332bfff700 1 -- 172.21.15.33:0/2371362809 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f33380cdda0 con 0x7f3344100f70 2024-05-08T21:18:52.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.282+0000 7f3342ffd700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f332c0a11b0 0x7f332c0a3670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3344100dd0 tx=0x7f3334009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:52.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.285+0000 7f332bfff700 1 -- 172.21.15.33:0/2371362809 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3338097150 con 0x7f3344100f70 2024-05-08T21:18:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.507+0000 7f33496dd700 1 -- 172.21.15.33:0/2371362809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33440681d0 con 0x7f3344100f70 2024-05-08T21:18:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.507+0000 7f332bfff700 1 -- 172.21.15.33:0/2371362809 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f33440681d0 con 0x7f3344100f70 2024-05-08T21:18:52.510 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.510+0000 7f33496dd700 1 -- 172.21.15.33:0/2371362809 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f332c0a11b0 msgr2=0x7f332c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.510+0000 7f33496dd700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f332c0a11b0 0x7f332c0a3670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3344100dd0 tx=0x7f3334009260 comp rx=0 tx=0).stop 2024-05-08T21:18:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.510+0000 7f33496dd700 1 -- 172.21.15.33:0/2371362809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3344100f70 msgr2=0x7f3344102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.510+0000 7f33496dd700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3344100f70 0x7f3344102590 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3338000c00 tx=0x7f333800f690 comp rx=0 tx=0).stop 2024-05-08T21:18:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.510+0000 7f33496dd700 1 -- 172.21.15.33:0/2371362809 shutdown_connections 2024-05-08T21:18:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.510+0000 7f33496dd700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3344101910 0x7f3344102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.510+0000 7f33496dd700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f332c0a11b0 0x7f332c0a3670 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.510+0000 7f33496dd700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33440ffd80 0x7f3344103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.510+0000 7f33496dd700 1 --2- 172.21.15.33:0/2371362809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3344100f70 0x7f3344102590 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.510+0000 7f33496dd700 1 -- 172.21.15.33:0/2371362809 >> 172.21.15.33:0/2371362809 conn(0x7f33440fb310 msgr2=0x7f33440694d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.510+0000 7f33496dd700 1 -- 172.21.15.33:0/2371362809 shutdown_connections 2024-05-08T21:18:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.510+0000 7f33496dd700 1 -- 172.21.15.33:0/2371362809 wait complete. 2024-05-08T21:18:52.526 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:18:52.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.664+0000 7fad7a646700 1 -- 172.21.15.33:0/416360606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad74101910 msgr2=0x7fad74103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:52.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.664+0000 7fad7a646700 1 --2- 172.21.15.33:0/416360606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad74101910 0x7fad74103d00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fad64009990 tx=0x7fad64009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.664+0000 7fad7a646700 1 -- 172.21.15.33:0/416360606 shutdown_connections 2024-05-08T21:18:52.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.664+0000 7fad7a646700 1 --2- 172.21.15.33:0/416360606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74106b70 0x7fad74108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.664+0000 7fad7a646700 1 --2- 172.21.15.33:0/416360606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad74101910 0x7fad74103d00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.664+0000 7fad7a646700 1 --2- 172.21.15.33:0/416360606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad74104240 0x7fad74106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.664+0000 7fad7a646700 1 -- 172.21.15.33:0/416360606 >> 172.21.15.33:0/416360606 conn(0x7fad740fb300 msgr2=0x7fad740fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:52.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.664+0000 7fad7a646700 1 -- 172.21.15.33:0/416360606 shutdown_connections 2024-05-08T21:18:52.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.664+0000 7fad7a646700 1 -- 172.21.15.33:0/416360606 wait complete. 2024-05-08T21:18:52.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.665+0000 7fad7a646700 1 Processor -- start 2024-05-08T21:18:52.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.665+0000 7fad7a646700 1 -- start start 2024-05-08T21:18:52.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.665+0000 7fad7a646700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad74101910 0x7fad74196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:52.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.665+0000 7fad7a646700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74104240 0x7fad74197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:52.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.665+0000 7fad7a646700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad74106b70 0x7fad7419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:52.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.665+0000 7fad7a646700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7419e7b0 con 0x7fad74106b70 2024-05-08T21:18:52.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.665+0000 7fad7a646700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7419e920 con 0x7fad74104240 2024-05-08T21:18:52.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.665+0000 7fad7a646700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7419eae0 con 0x7fad74101910 2024-05-08T21:18:52.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.665+0000 7fad78be3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad74106b70 0x7fad7419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:52.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.665+0000 7fad78be3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad74106b70 0x7fad7419e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54114/0 (socket says 172.21.15.33:54114) 2024-05-08T21:18:52.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.665+0000 7fad78be3700 1 -- 172.21.15.33:0/2197047790 learned_addr learned my addr 172.21.15.33:0/2197047790 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad73fff700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad74101910 0x7fad74196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad737fe700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74104240 0x7fad74197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad78be3700 1 -- 172.21.15.33:0/2197047790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad74101910 msgr2=0x7fad74196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad78be3700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad74101910 0x7fad74196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad78be3700 1 -- 172.21.15.33:0/2197047790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74104240 msgr2=0x7fad74197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad78be3700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74104240 0x7fad74197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad78be3700 1 -- 172.21.15.33:0/2197047790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad640095f0 con 0x7fad74106b70 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad78be3700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad74106b70 0x7fad7419e1e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fad6c007ee0 tx=0x7fad6c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad717fa700 1 -- 172.21.15.33:0/2197047790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c017070 con 0x7fad74106b70 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad717fa700 1 -- 172.21.15.33:0/2197047790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad6c00f040 con 0x7fad74106b70 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad7a646700 1 -- 172.21.15.33:0/2197047790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7419edc0 con 0x7fad74106b70 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad717fa700 1 -- 172.21.15.33:0/2197047790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c005570 con 0x7fad74106b70 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.666+0000 7fad7a646700 1 -- 172.21.15.33:0/2197047790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7419f1c0 con 0x7fad74106b70 2024-05-08T21:18:52.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.667+0000 7fad7a646700 1 -- 172.21.15.33:0/2197047790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad740fce80 con 0x7fad74106b70 2024-05-08T21:18:52.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.668+0000 7fad717fa700 1 -- 172.21.15.33:0/2197047790 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fad6c00db20 con 0x7fad74106b70 2024-05-08T21:18:52.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.669+0000 7fad717fa700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad5c0a11b0 0x7fad5c0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:52.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.669+0000 7fad717fa700 1 -- 172.21.15.33:0/2197047790 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fad6c0cc480 con 0x7fad74106b70 2024-05-08T21:18:52.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.669+0000 7fad73fff700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad5c0a11b0 0x7fad5c0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:52.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.670+0000 7fad73fff700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad5c0a11b0 0x7fad5c0a3670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fad64000c00 tx=0x7fad64005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:52.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.674+0000 7fad717fa700 1 -- 172.21.15.33:0/2197047790 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fad6c0957a0 con 0x7fad74106b70 2024-05-08T21:18:52.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:52 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/763436918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:52.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:52 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4046153622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:52.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:52 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/763436918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:52.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:52 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/4046153622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:52.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:52 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/763436918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:52.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:52 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/4046153622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:52.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.893+0000 7fad7a646700 1 -- 172.21.15.33:0/2197047790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fad7410c6b0 con 0x7fad74106b70 2024-05-08T21:18:52.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.894+0000 7fad717fa700 1 -- 172.21.15.33:0/2197047790 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fad6c098dc0 con 0x7fad74106b70 2024-05-08T21:18:52.896 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T21:18:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.896+0000 7fad7a646700 1 -- 172.21.15.33:0/2197047790 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad5c0a11b0 msgr2=0x7fad5c0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.896+0000 7fad7a646700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad5c0a11b0 0x7fad5c0a3670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fad64000c00 tx=0x7fad64005fb0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.896+0000 7fad7a646700 1 -- 172.21.15.33:0/2197047790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad74106b70 msgr2=0x7fad7419e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.896+0000 7fad7a646700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad74106b70 0x7fad7419e1e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fad6c007ee0 tx=0x7fad6c00d3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.896+0000 7fad7a646700 1 -- 172.21.15.33:0/2197047790 shutdown_connections 2024-05-08T21:18:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.896+0000 7fad7a646700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad74106b70 0x7fad7419e1e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.896+0000 7fad7a646700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad74101910 0x7fad74196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.896+0000 7fad7a646700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad5c0a11b0 0x7fad5c0a3670 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.896+0000 7fad7a646700 1 --2- 172.21.15.33:0/2197047790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad74104240 0x7fad74197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.896+0000 7fad7a646700 1 -- 172.21.15.33:0/2197047790 >> 172.21.15.33:0/2197047790 conn(0x7fad740fb300 msgr2=0x7fad74103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.896+0000 7fad7a646700 1 -- 172.21.15.33:0/2197047790 shutdown_connections 2024-05-08T21:18:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:52.896+0000 7fad7a646700 1 -- 172.21.15.33:0/2197047790 wait complete. 2024-05-08T21:18:52.912 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T21:18:53.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.050+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2306588377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c100f50 msgr2=0x7fcb1c1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.050+0000 7fcb21fc8700 1 --2- 172.21.15.33:0/2306588377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c100f50 0x7fcb1c1013b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcb040099c0 tx=0x7fcb04009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.050+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2306588377 shutdown_connections 2024-05-08T21:18:53.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.050+0000 7fcb21fc8700 1 --2- 172.21.15.33:0/2306588377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb1c1018f0 0x7fcb1c1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.050+0000 7fcb21fc8700 1 --2- 172.21.15.33:0/2306588377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb1c0ffd60 0x7fcb1c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.050+0000 7fcb21fc8700 1 --2- 172.21.15.33:0/2306588377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c100f50 0x7fcb1c1013b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.050+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2306588377 >> 172.21.15.33:0/2306588377 conn(0x7fcb1c0fb290 msgr2=0x7fcb1c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:53.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.050+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2306588377 shutdown_connections 2024-05-08T21:18:53.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.050+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2306588377 wait complete. 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.051+0000 7fcb21fc8700 1 Processor -- start 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.051+0000 7fcb21fc8700 1 -- start start 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.051+0000 7fcb21fc8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb1c0ffd60 0x7fcb1c196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.051+0000 7fcb21fc8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c100f50 0x7fcb1c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.051+0000 7fcb21fc8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb1c1018f0 0x7fcb1c19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.051+0000 7fcb21fc8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c19e840 con 0x7fcb1c100f50 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb21fc8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c19e9b0 con 0x7fcb1c1018f0 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb21fc8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c19eb70 con 0x7fcb1c0ffd60 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb12ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c100f50 0x7fcb1c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb12ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c100f50 0x7fcb1c197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54132/0 (socket says 172.21.15.33:54132) 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb12ffd700 1 -- 172.21.15.33:0/2664537230 learned_addr learned my addr 172.21.15.33:0/2664537230 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb1bfff700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb1c1018f0 0x7fcb1c19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:53.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb1b7fe700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb1c0ffd60 0x7fcb1c196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb12ffd700 1 -- 172.21.15.33:0/2664537230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb1c0ffd60 msgr2=0x7fcb1c196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb12ffd700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb1c0ffd60 0x7fcb1c196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb12ffd700 1 -- 172.21.15.33:0/2664537230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb1c1018f0 msgr2=0x7fcb1c19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb12ffd700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb1c1018f0 0x7fcb1c19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb12ffd700 1 -- 172.21.15.33:0/2664537230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040095f0 con 0x7fcb1c100f50 2024-05-08T21:18:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb12ffd700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c100f50 0x7fcb1c197290 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcb04000c00 tx=0x7fcb0400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb197fa700 1 -- 172.21.15.33:0/2664537230 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0401d070 con 0x7fcb1c100f50 2024-05-08T21:18:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2664537230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1c19edf0 con 0x7fcb1c100f50 2024-05-08T21:18:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb197fa700 1 -- 172.21.15.33:0/2664537230 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb04005030 con 0x7fcb1c100f50 2024-05-08T21:18:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb197fa700 1 -- 172.21.15.33:0/2664537230 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04004b10 con 0x7fcb1c100f50 2024-05-08T21:18:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.052+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2664537230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c19f1c0 con 0x7fcb1c100f50 2024-05-08T21:18:53.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.054+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2664537230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1c107320 con 0x7fcb1c100f50 2024-05-08T21:18:53.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.055+0000 7fcb197fa700 1 -- 172.21.15.33:0/2664537230 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fcb040051a0 con 0x7fcb1c100f50 2024-05-08T21:18:53.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.056+0000 7fcb197fa700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcb080a11b0 0x7fcb080a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:53.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.056+0000 7fcb1b7fe700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcb080a11b0 0x7fcb080a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:53.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.056+0000 7fcb197fa700 1 -- 172.21.15.33:0/2664537230 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcb040cde40 con 0x7fcb1c100f50 2024-05-08T21:18:53.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.056+0000 7fcb1b7fe700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcb080a11b0 0x7fcb080a3670 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c1981d0 tx=0x7fcb0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:53.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.060+0000 7fcb197fa700 1 -- 172.21.15.33:0/2664537230 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcb04097310 con 0x7fcb1c100f50 2024-05-08T21:18:53.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.279+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2664537230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcb1c10c710 con 0x7fcb1c100f50 2024-05-08T21:18:53.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.279+0000 7fcb197fa700 1 -- 172.21.15.33:0/2664537230 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcb04026020 con 0x7fcb1c100f50 2024-05-08T21:18:53.282 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:53.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.282+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2664537230 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcb080a11b0 msgr2=0x7fcb080a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.282+0000 7fcb21fc8700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcb080a11b0 0x7fcb080a3670 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c1981d0 tx=0x7fcb0c009260 comp rx=0 tx=0).stop 2024-05-08T21:18:53.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.282+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2664537230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c100f50 msgr2=0x7fcb1c197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.282+0000 7fcb21fc8700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c100f50 0x7fcb1c197290 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcb04000c00 tx=0x7fcb0400f690 comp rx=0 tx=0).stop 2024-05-08T21:18:53.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.282+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2664537230 shutdown_connections 2024-05-08T21:18:53.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.282+0000 7fcb21fc8700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcb080a11b0 0x7fcb080a3670 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.282+0000 7fcb21fc8700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb1c1018f0 0x7fcb1c19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.282+0000 7fcb21fc8700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb1c0ffd60 0x7fcb1c196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.282+0000 7fcb21fc8700 1 --2- 172.21.15.33:0/2664537230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c100f50 0x7fcb1c197290 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.282+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2664537230 >> 172.21.15.33:0/2664537230 conn(0x7fcb1c0fb290 msgr2=0x7fcb1c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:53.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.282+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2664537230 shutdown_connections 2024-05-08T21:18:53.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.282+0000 7fcb21fc8700 1 -- 172.21.15.33:0/2664537230 wait complete. 2024-05-08T21:18:53.298 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-08T21:18:53.298 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:18:53.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.434+0000 7f2d520fd700 1 -- 172.21.15.33:0/428273146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d4c0ffd80 msgr2=0x7f2d4c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.434+0000 7f2d520fd700 1 --2- 172.21.15.33:0/428273146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d4c0ffd80 0x7f2d4c1001a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009920 tx=0x7f2d3c009c30 comp rx=0 tx=0).stop 2024-05-08T21:18:53.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.434+0000 7f2d520fd700 1 -- 172.21.15.33:0/428273146 shutdown_connections 2024-05-08T21:18:53.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.434+0000 7f2d520fd700 1 --2- 172.21.15.33:0/428273146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d4c101910 0x7f2d4c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.434+0000 7f2d520fd700 1 --2- 172.21.15.33:0/428273146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d4c0ffd80 0x7f2d4c1001a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.434+0000 7f2d520fd700 1 --2- 172.21.15.33:0/428273146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c100f70 0x7f2d4c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.434+0000 7f2d520fd700 1 -- 172.21.15.33:0/428273146 >> 172.21.15.33:0/428273146 conn(0x7f2d4c0fb310 msgr2=0x7f2d4c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:53.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.434+0000 7f2d520fd700 1 -- 172.21.15.33:0/428273146 shutdown_connections 2024-05-08T21:18:53.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.434+0000 7f2d520fd700 1 -- 172.21.15.33:0/428273146 wait complete. 2024-05-08T21:18:53.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.434+0000 7f2d520fd700 1 Processor -- start 2024-05-08T21:18:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d520fd700 1 -- start start 2024-05-08T21:18:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d520fd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d4c0ffd80 0x7f2d4c103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d520fd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d4c100f70 0x7f2d4c102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d520fd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c101910 0x7f2d4c102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d520fd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c103030 con 0x7f2d4c101910 2024-05-08T21:18:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d520fd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c1031a0 con 0x7f2d4c0ffd80 2024-05-08T21:18:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d520fd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c1a0e90 con 0x7f2d4c100f70 2024-05-08T21:18:53.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d4bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c101910 0x7f2d4c102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d4affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d4c100f70 0x7f2d4c102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d4bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c101910 0x7f2d4c102890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54156/0 (socket says 172.21.15.33:54156) 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d4b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d4c0ffd80 0x7f2d4c103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d4bfff700 1 -- 172.21.15.33:0/1391443389 learned_addr learned my addr 172.21.15.33:0/1391443389 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d4affd700 1 -- 172.21.15.33:0/1391443389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d4c0ffd80 msgr2=0x7f2d4c103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d4affd700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d4c0ffd80 0x7f2d4c103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d4affd700 1 -- 172.21.15.33:0/1391443389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c101910 msgr2=0x7f2d4c102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d4affd700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c101910 0x7f2d4c102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d4affd700 1 -- 172.21.15.33:0/1391443389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d400095f0 con 0x7f2d4c100f70 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.435+0000 7f2d4bfff700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c101910 0x7f2d4c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.436+0000 7f2d4affd700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d4c100f70 0x7f2d4c102350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000eb90 tx=0x7f2d4000eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.436+0000 7f2d48ff9700 1 -- 172.21.15.33:0/1391443389 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40009970 con 0x7f2d4c100f70 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.436+0000 7f2d48ff9700 1 -- 172.21.15.33:0/1391443389 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d400044b0 con 0x7f2d4c100f70 2024-05-08T21:18:53.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.436+0000 7f2d520fd700 1 -- 172.21.15.33:0/1391443389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3c009520 con 0x7f2d4c100f70 2024-05-08T21:18:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.436+0000 7f2d48ff9700 1 -- 172.21.15.33:0/1391443389 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40003d70 con 0x7f2d4c100f70 2024-05-08T21:18:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.436+0000 7f2d520fd700 1 -- 172.21.15.33:0/1391443389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c1a1440 con 0x7f2d4c100f70 2024-05-08T21:18:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.437+0000 7f2d520fd700 1 -- 172.21.15.33:0/1391443389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4c067d10 con 0x7f2d4c100f70 2024-05-08T21:18:53.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.438+0000 7f2d48ff9700 1 -- 172.21.15.33:0/1391443389 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f2d40004000 con 0x7f2d4c100f70 2024-05-08T21:18:53.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.439+0000 7f2d48ff9700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2d340a11b0 0x7f2d340a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:53.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.439+0000 7f2d4b7fe700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2d340a11b0 0x7f2d340a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:53.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.439+0000 7f2d48ff9700 1 -- 172.21.15.33:0/1391443389 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2d40014070 con 0x7f2d4c100f70 2024-05-08T21:18:53.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.439+0000 7f2d4b7fe700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2d340a11b0 0x7f2d340a3670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0098f0 tx=0x7f2d3c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:53.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.443+0000 7f2d48ff9700 1 -- 172.21.15.33:0/1391443389 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2d40096990 con 0x7f2d4c100f70 2024-05-08T21:18:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:53 smithi033 ceph-mon[42528]: pgmap v285: 129 pgs: 129 active+clean; 646 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-08T21:18:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:53 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2371362809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:53.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:53 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2197047790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:53.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:53 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2664537230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:53.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.708+0000 7f2d520fd700 1 -- 172.21.15.33:0/1391443389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d4c10b490 con 0x7f2d4c100f70 2024-05-08T21:18:53.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.709+0000 7f2d48ff9700 1 -- 172.21.15.33:0/1391443389 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10899 (secure 0 0 0) 0x7f2d40099fb0 con 0x7f2d4c100f70 2024-05-08T21:18:53.712 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:53.712 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:18:49.368057+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:18:53.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.712+0000 7f2d520fd700 1 -- 172.21.15.33:0/1391443389 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2d340a11b0 msgr2=0x7f2d340a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.712+0000 7f2d520fd700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2d340a11b0 0x7f2d340a3670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0098f0 tx=0x7f2d3c019040 comp rx=0 tx=0).stop 2024-05-08T21:18:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.712+0000 7f2d520fd700 1 -- 172.21.15.33:0/1391443389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d4c100f70 msgr2=0x7f2d4c102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.712+0000 7f2d520fd700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d4c100f70 0x7f2d4c102350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000eb90 tx=0x7f2d4000eea0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.712+0000 7f2d520fd700 1 -- 172.21.15.33:0/1391443389 shutdown_connections 2024-05-08T21:18:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.712+0000 7f2d520fd700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c101910 0x7f2d4c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.712+0000 7f2d520fd700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2d340a11b0 0x7f2d340a3670 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.712+0000 7f2d520fd700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d4c0ffd80 0x7f2d4c103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.712+0000 7f2d520fd700 1 --2- 172.21.15.33:0/1391443389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d4c100f70 0x7f2d4c102350 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.712+0000 7f2d520fd700 1 -- 172.21.15.33:0/1391443389 >> 172.21.15.33:0/1391443389 conn(0x7f2d4c0fb310 msgr2=0x7f2d4c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.712+0000 7f2d520fd700 1 -- 172.21.15.33:0/1391443389 shutdown_connections 2024-05-08T21:18:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.712+0000 7f2d520fd700 1 -- 172.21.15.33:0/1391443389 wait complete. 2024-05-08T21:18:53.715 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-05-08T21:18:53.728 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-08T21:18:53.728 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-08T21:18:53.729 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T21:18:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:53 smithi071 ceph-mon[47074]: pgmap v285: 129 pgs: 129 active+clean; 646 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-08T21:18:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:53 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2371362809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:53.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:53 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2197047790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:53.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:53 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2664537230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:53 smithi082 ceph-mon[47512]: pgmap v285: 129 pgs: 129 active+clean; 646 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-08T21:18:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:53 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2371362809' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:53 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2197047790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:53 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2664537230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:53.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.864+0000 7fb57bf8a700 1 -- 172.21.15.33:0/1410682099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574101910 msgr2=0x7fb5741081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.864+0000 7fb57bf8a700 1 --2- 172.21.15.33:0/1410682099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574101910 0x7fb5741081c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00b0d0 tx=0x7fb56c00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.864+0000 7fb57bf8a700 1 -- 172.21.15.33:0/1410682099 shutdown_connections 2024-05-08T21:18:53.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.864+0000 7fb57bf8a700 1 --2- 172.21.15.33:0/1410682099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574101910 0x7fb5741081c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.864+0000 7fb57bf8a700 1 --2- 172.21.15.33:0/1410682099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5740ffd80 0x7fb5741001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.864+0000 7fb57bf8a700 1 --2- 172.21.15.33:0/1410682099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb574100f70 0x7fb5741013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.864+0000 7fb57bf8a700 1 -- 172.21.15.33:0/1410682099 >> 172.21.15.33:0/1410682099 conn(0x7fb5740fb310 msgr2=0x7fb5740fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:53.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.864+0000 7fb57bf8a700 1 -- 172.21.15.33:0/1410682099 shutdown_connections 2024-05-08T21:18:53.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.864+0000 7fb57bf8a700 1 -- 172.21.15.33:0/1410682099 wait complete. 2024-05-08T21:18:53.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.864+0000 7fb57bf8a700 1 Processor -- start 2024-05-08T21:18:53.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57bf8a700 1 -- start start 2024-05-08T21:18:53.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57bf8a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5740ffd80 0x7fb574196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:53.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57bf8a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb574100f70 0x7fb5741973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:53.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57bf8a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574101910 0x7fb57419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:53.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57bf8a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57419e940 con 0x7fb574101910 2024-05-08T21:18:53.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57bf8a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57419ea80 con 0x7fb5740ffd80 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57bf8a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57419ec10 con 0x7fb574100f70 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57a527700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574101910 0x7fb57419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57a527700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574101910 0x7fb57419e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54162/0 (socket says 172.21.15.33:54162) 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57a527700 1 -- 172.21.15.33:0/3046098440 learned_addr learned my addr 172.21.15.33:0/3046098440 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb579525700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb574100f70 0x7fb5741973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb579d26700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5740ffd80 0x7fb574196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57a527700 1 -- 172.21.15.33:0/3046098440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb574100f70 msgr2=0x7fb5741973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57a527700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb574100f70 0x7fb5741973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57a527700 1 -- 172.21.15.33:0/3046098440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5740ffd80 msgr2=0x7fb574196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57a527700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5740ffd80 0x7fb574196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57a527700 1 -- 172.21.15.33:0/3046098440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c009d00 con 0x7fb574101910 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb579d26700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5740ffd80 0x7fb574196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.865+0000 7fb57a527700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574101910 0x7fb57419e370 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb56c007af0 tx=0x7fb56c007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.866+0000 7fb56affd700 1 -- 172.21.15.33:0/3046098440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c00e040 con 0x7fb574101910 2024-05-08T21:18:53.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.866+0000 7fb56affd700 1 -- 172.21.15.33:0/3046098440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb56c00be30 con 0x7fb574101910 2024-05-08T21:18:53.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.866+0000 7fb57bf8a700 1 -- 172.21.15.33:0/3046098440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57419ee90 con 0x7fb574101910 2024-05-08T21:18:53.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.866+0000 7fb56affd700 1 -- 172.21.15.33:0/3046098440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c01b930 con 0x7fb574101910 2024-05-08T21:18:53.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.866+0000 7fb57bf8a700 1 -- 172.21.15.33:0/3046098440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57419f2b0 con 0x7fb574101910 2024-05-08T21:18:53.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.867+0000 7fb57bf8a700 1 -- 172.21.15.33:0/3046098440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb574067d10 con 0x7fb574101910 2024-05-08T21:18:53.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.868+0000 7fb56affd700 1 -- 172.21.15.33:0/3046098440 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fb56c01ba90 con 0x7fb574101910 2024-05-08T21:18:53.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.868+0000 7fb56affd700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5600a11b0 0x7fb5600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.869+0000 7fb579d26700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5600a11b0 0x7fb5600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.869+0000 7fb56affd700 1 -- 172.21.15.33:0/3046098440 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb56c0ce5b0 con 0x7fb574101910 2024-05-08T21:18:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.869+0000 7fb579d26700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5600a11b0 0x7fb5600a3670 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb574100dd0 tx=0x7fb570009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:53.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:53.873+0000 7fb56affd700 1 -- 172.21.15.33:0/3046098440 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fb56c097960 con 0x7fb574101910 2024-05-08T21:18:54.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.087+0000 7fb57bf8a700 1 -- 172.21.15.33:0/3046098440 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5740688e0 con 0x7fb5600a11b0 2024-05-08T21:18:54.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.101+0000 7fb56affd700 1 -- 172.21.15.33:0/3046098440 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427327 (secure 0 0 0) 0x7fb5740688e0 con 0x7fb5600a11b0 2024-05-08T21:18:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:54.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.105+0000 7fb57bf8a700 1 -- 172.21.15.33:0/3046098440 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5600a11b0 msgr2=0x7fb5600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:54.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.105+0000 7fb57bf8a700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5600a11b0 0x7fb5600a3670 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb574100dd0 tx=0x7fb570009260 comp rx=0 tx=0).stop 2024-05-08T21:18:54.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.105+0000 7fb57bf8a700 1 -- 172.21.15.33:0/3046098440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574101910 msgr2=0x7fb57419e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:54.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.105+0000 7fb57bf8a700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574101910 0x7fb57419e370 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb56c007af0 tx=0x7fb56c007870 comp rx=0 tx=0).stop 2024-05-08T21:18:54.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.105+0000 7fb57bf8a700 1 -- 172.21.15.33:0/3046098440 shutdown_connections 2024-05-08T21:18:54.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.105+0000 7fb57bf8a700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574101910 0x7fb57419e370 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.105+0000 7fb57bf8a700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5600a11b0 0x7fb5600a3670 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.105+0000 7fb57bf8a700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5740ffd80 0x7fb574196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.105+0000 7fb57bf8a700 1 --2- 172.21.15.33:0/3046098440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb574100f70 0x7fb5741973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.105+0000 7fb57bf8a700 1 -- 172.21.15.33:0/3046098440 >> 172.21.15.33:0/3046098440 conn(0x7fb5740fb310 msgr2=0x7fb5740681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:54.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.105+0000 7fb57bf8a700 1 -- 172.21.15.33:0/3046098440 shutdown_connections 2024-05-08T21:18:54.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.105+0000 7fb57bf8a700 1 -- 172.21.15.33:0/3046098440 wait complete. 2024-05-08T21:18:54.113 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T21:18:54.121 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-05-08T21:18:52.144621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661994,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":949,"num_read_kb":597,"num_write":434,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1181982,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":359,"ondisk_log_size":359,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374776,"kb_used_data":13696,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124567048,"statfs":{"total":1151940427776,"available":1151556657152,"internally_reserved":0,"allocated":14024704,"data_stored":7753078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2480,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":70,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003354"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418982+0000","last_change":"2024-05-08T21:17:03.243903+0000","last_active":"2024-05-08T21:17:56.418982+0000","last_peered":"2024-05-08T21:17:56.418982+0000","last_clean":"2024-05-08T21:17:56.418982+0000","last_became_active":"2024-05-08T21:17:03.240821+0000","last_became_peered":"2024-05-08T21:17:03.240821+0000","last_unstale":"2024-05-08T21:17:56.418982+0000","last_undegraded":"2024-05-08T21:17:56.418982+0000","last_fullsized":"2024-05-08T21:17:56.418982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:01.878614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423327+0000","last_change":"2024-05-08T21:17:03.243119+0000","last_active":"2024-05-08T21:17:56.423327+0000","last_peered":"2024-05-08T21:17:56.423327+0000","last_clean":"2024-05-08T21:17:56.423327+0000","last_became_active":"2024-05-08T21:17:03.242772+0000","last_became_peered":"2024-05-08T21:17:03.242772+0000","last_unstale":"2024-05-08T21:17:56.423327+0000","last_undegraded":"2024-05-08T21:17:56.423327+0000","last_fullsized":"2024-05-08T21:17:56.423327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:20.550646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423287+0000","last_change":"2024-05-08T21:17:03.237843+0000","last_active":"2024-05-08T21:17:56.423287+0000","last_peered":"2024-05-08T21:17:56.423287+0000","last_clean":"2024-05-08T21:17:56.423287+0000","last_became_active":"2024-05-08T21:17:03.237567+0000","last_became_peered":"2024-05-08T21:17:03.237567+0000","last_unstale":"2024-05-08T21:17:56.423287+0000","last_undegraded":"2024-05-08T21:17:56.423287+0000","last_fullsized":"2024-05-08T21:17:56.423287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:23.856461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.430448+0000","last_change":"2024-05-08T21:17:57.430448+0000","last_active":"2024-05-08T21:17:57.430448+0000","last_peered":"2024-05-08T21:17:57.430448+0000","last_clean":"2024-05-08T21:17:57.430448+0000","last_became_active":"2024-05-08T21:17:57.430195+0000","last_became_peered":"2024-05-08T21:17:57.430195+0000","last_unstale":"2024-05-08T21:17:57.430448+0000","last_undegraded":"2024-05-08T21:17:57.430448+0000","last_fullsized":"2024-05-08T21:17:57.430448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:41.536610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416067+0000","last_change":"2024-05-08T21:17:03.252246+0000","last_active":"2024-05-08T21:17:56.416067+0000","last_peered":"2024-05-08T21:17:56.416067+0000","last_clean":"2024-05-08T21:17:56.416067+0000","last_became_active":"2024-05-08T21:17:03.251898+0000","last_became_peered":"2024-05-08T21:17:03.251898+0000","last_unstale":"2024-05-08T21:17:56.416067+0000","last_undegraded":"2024-05-08T21:17:56.416067+0000","last_fullsized":"2024-05-08T21:17:56.416067+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:08.996033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417704+0000","last_change":"2024-05-08T21:17:03.238422+0000","last_active":"2024-05-08T21:17:56.417704+0000","last_peered":"2024-05-08T21:17:56.417704+0000","last_clean":"2024-05-08T21:17:56.417704+0000","last_became_active":"2024-05-08T21:17:03.238096+0000","last_became_peered":"2024-05-08T21:17:03.238096+0000","last_unstale":"2024-05-08T21:17:56.417704+0000","last_undegraded":"2024-05-08T21:17:56.417704+0000","last_fullsized":"2024-05-08T21:17:56.417704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:54.635089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419866+0000","last_change":"2024-05-08T21:17:03.244315+0000","last_active":"2024-05-08T21:17:56.419866+0000","last_peered":"2024-05-08T21:17:56.419866+0000","last_clean":"2024-05-08T21:17:56.419866+0000","last_became_active":"2024-05-08T21:17:03.243939+0000","last_became_peered":"2024-05-08T21:17:03.243939+0000","last_unstale":"2024-05-08T21:17:56.419866+0000","last_undegraded":"2024-05-08T21:17:56.419866+0000","last_fullsized":"2024-05-08T21:17:56.419866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:24.371495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415520+0000","last_change":"2024-05-08T21:17:03.240144+0000","last_active":"2024-05-08T21:17:56.415520+0000","last_peered":"2024-05-08T21:17:56.415520+0000","last_clean":"2024-05-08T21:17:56.415520+0000","last_became_active":"2024-05-08T21:17:03.239493+0000","last_became_peered":"2024-05-08T21:17:03.239493+0000","last_unstale":"2024-05-08T21:17:56.415520+0000","last_undegraded":"2024-05-08T21:17:56.415520+0000","last_fullsized":"2024-05-08T21:17:56.415520+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:21.374357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424317+0000","last_change":"2024-05-08T21:17:03.256706+0000","last_active":"2024-05-08T21:17:56.424317+0000","last_peered":"2024-05-08T21:17:56.424317+0000","last_clean":"2024-05-08T21:17:56.424317+0000","last_became_active":"2024-05-08T21:17:03.255345+0000","last_became_peered":"2024-05-08T21:17:03.255345+0000","last_unstale":"2024-05-08T21:17:56.424317+0000","last_undegraded":"2024-05-08T21:17:56.424317+0000","last_fullsized":"2024-05-08T21:17:56.424317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:09:50.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.430334+0000","last_change":"2024-05-08T21:17:57.430334+0000","last_active":"2024-05-08T21:17:57.430334+0000","last_peered":"2024-05-08T21:17:57.430334+0000","last_clean":"2024-05-08T21:17:57.430334+0000","last_became_active":"2024-05-08T21:17:57.429924+0000","last_became_peered":"2024-05-08T21:17:57.429924+0000","last_unstale":"2024-05-08T21:17:57.430334+0000","last_undegraded":"2024-05-08T21:17:57.430334+0000","last_fullsized":"2024-05-08T21:17:57.430334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.901755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419810+0000","last_change":"2024-05-08T21:17:03.253996+0000","last_active":"2024-05-08T21:17:56.419810+0000","last_peered":"2024-05-08T21:17:56.419810+0000","last_clean":"2024-05-08T21:17:56.419810+0000","last_became_active":"2024-05-08T21:17:03.253686+0000","last_became_peered":"2024-05-08T21:17:03.253686+0000","last_unstale":"2024-05-08T21:17:56.419810+0000","last_undegraded":"2024-05-08T21:17:56.419810+0000","last_fullsized":"2024-05-08T21:17:56.419810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:17.995692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423092+0000","last_change":"2024-05-08T21:17:03.254357+0000","last_active":"2024-05-08T21:17:56.423092+0000","last_peered":"2024-05-08T21:17:56.423092+0000","last_clean":"2024-05-08T21:17:56.423092+0000","last_became_active":"2024-05-08T21:17:03.254013+0000","last_became_peered":"2024-05-08T21:17:03.254013+0000","last_unstale":"2024-05-08T21:17:56.423092+0000","last_undegraded":"2024-05-08T21:17:56.423092+0000","last_fullsized":"2024-05-08T21:17:56.423092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:21.480612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.413995+0000","last_change":"2024-05-08T21:17:03.244190+0000","last_active":"2024-05-08T21:17:56.413995+0000","last_peered":"2024-05-08T21:17:56.413995+0000","last_clean":"2024-05-08T21:17:56.413995+0000","last_became_active":"2024-05-08T21:17:03.243989+0000","last_became_peered":"2024-05-08T21:17:03.243989+0000","last_unstale":"2024-05-08T21:17:56.413995+0000","last_undegraded":"2024-05-08T21:17:56.413995+0000","last_fullsized":"2024-05-08T21:17:56.413995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:45.059343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422938+0000","last_change":"2024-05-08T21:17:03.242996+0000","last_active":"2024-05-08T21:17:56.422938+0000","last_peered":"2024-05-08T21:17:56.422938+0000","last_clean":"2024-05-08T21:17:56.422938+0000","last_became_active":"2024-05-08T21:17:03.242648+0000","last_became_peered":"2024-05-08T21:17:03.242648+0000","last_unstale":"2024-05-08T21:17:56.422938+0000","last_undegraded":"2024-05-08T21:17:56.422938+0000","last_fullsized":"2024-05-08T21:17:56.422938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:47:19.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422908+0000","last_change":"2024-05-08T21:17:03.244970+0000","last_active":"2024-05-08T21:17:56.422908+0000","last_peered":"2024-05-08T21:17:56.422908+0000","last_clean":"2024-05-08T21:17:56.422908+0000","last_became_active":"2024-05-08T21:17:03.244604+0000","last_became_peered":"2024-05-08T21:17:03.244604+0000","last_unstale":"2024-05-08T21:17:56.422908+0000","last_undegraded":"2024-05-08T21:17:56.422908+0000","last_fullsized":"2024-05-08T21:17:56.422908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:32.484155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422727+0000","last_change":"2024-05-08T21:17:03.254628+0000","last_active":"2024-05-08T21:17:56.422727+0000","last_peered":"2024-05-08T21:17:56.422727+0000","last_clean":"2024-05-08T21:17:56.422727+0000","last_became_active":"2024-05-08T21:17:03.254305+0000","last_became_peered":"2024-05-08T21:17:03.254305+0000","last_unstale":"2024-05-08T21:17:56.422727+0000","last_undegraded":"2024-05-08T21:17:56.422727+0000","last_fullsized":"2024-05-08T21:17:56.422727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:52.727821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418732+0000","last_change":"2024-05-08T21:17:03.243406+0000","last_active":"2024-05-08T21:17:56.418732+0000","last_peered":"2024-05-08T21:17:56.418732+0000","last_clean":"2024-05-08T21:17:56.418732+0000","last_became_active":"2024-05-08T21:17:03.240379+0000","last_became_peered":"2024-05-08T21:17:03.240379+0000","last_unstale":"2024-05-08T21:17:56.418732+0000","last_undegraded":"2024-05-08T21:17:56.418732+0000","last_fullsized":"2024-05-08T21:17:56.418732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:49.897459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420035+0000","last_change":"2024-05-08T21:17:03.239664+0000","last_active":"2024-05-08T21:17:56.420035+0000","last_peered":"2024-05-08T21:17:56.420035+0000","last_clean":"2024-05-08T21:17:56.420035+0000","last_became_active":"2024-05-08T21:17:03.239347+0000","last_became_peered":"2024-05-08T21:17:03.239347+0000","last_unstale":"2024-05-08T21:17:56.420035+0000","last_undegraded":"2024-05-08T21:17:56.420035+0000","last_fullsized":"2024-05-08T21:17:56.420035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:06:37.557835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423543+0000","last_change":"2024-05-08T21:17:03.249424+0000","last_active":"2024-05-08T21:17:56.423543+0000","last_peered":"2024-05-08T21:17:56.423543+0000","last_clean":"2024-05-08T21:17:56.423543+0000","last_became_active":"2024-05-08T21:17:03.248971+0000","last_became_peered":"2024-05-08T21:17:03.248971+0000","last_unstale":"2024-05-08T21:17:56.423543+0000","last_undegraded":"2024-05-08T21:17:56.423543+0000","last_fullsized":"2024-05-08T21:17:56.423543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:08.096762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422849+0000","last_change":"2024-05-08T21:17:07.272414+0000","last_active":"2024-05-08T21:17:56.422849+0000","last_peered":"2024-05-08T21:17:56.422849+0000","last_clean":"2024-05-08T21:17:56.422849+0000","last_became_active":"2024-05-08T21:17:07.271660+0000","last_became_peered":"2024-05-08T21:17:07.271660+0000","last_unstale":"2024-05-08T21:17:56.422849+0000","last_undegraded":"2024-05-08T21:17:56.422849+0000","last_fullsized":"2024-05-08T21:17:56.422849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:15.697910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":133,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:47.545496+0000","last_change":"2024-05-08T21:17:57.483017+0000","last_active":"2024-05-08T21:18:47.545496+0000","last_peered":"2024-05-08T21:18:47.545496+0000","last_clean":"2024-05-08T21:18:47.545496+0000","last_became_active":"2024-05-08T21:17:57.473420+0000","last_became_peered":"2024-05-08T21:17:57.473420+0000","last_unstale":"2024-05-08T21:18:47.545496+0000","last_undegraded":"2024-05-08T21:18:47.545496+0000","last_fullsized":"2024-05-08T21:18:47.545496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:42.727117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417325+0000","last_change":"2024-05-08T21:17:03.240780+0000","last_active":"2024-05-08T21:17:56.417325+0000","last_peered":"2024-05-08T21:17:56.417325+0000","last_clean":"2024-05-08T21:17:56.417325+0000","last_became_active":"2024-05-08T21:17:03.240472+0000","last_became_peered":"2024-05-08T21:17:03.240472+0000","last_unstale":"2024-05-08T21:17:56.417325+0000","last_undegraded":"2024-05-08T21:17:56.417325+0000","last_fullsized":"2024-05-08T21:17:56.417325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.089760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.421952+0000","last_change":"2024-05-08T21:17:07.272308+0000","last_active":"2024-05-08T21:17:56.421952+0000","last_peered":"2024-05-08T21:17:56.421952+0000","last_clean":"2024-05-08T21:17:56.421952+0000","last_became_active":"2024-05-08T21:17:07.271869+0000","last_became_peered":"2024-05-08T21:17:07.271869+0000","last_unstale":"2024-05-08T21:17:56.421952+0000","last_undegraded":"2024-05-08T21:17:56.421952+0000","last_fullsized":"2024-05-08T21:17:56.421952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:31.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417727+0000","last_change":"2024-05-08T21:17:02.736446+0000","last_active":"2024-05-08T21:17:56.417727+0000","last_peered":"2024-05-08T21:17:56.417727+0000","last_clean":"2024-05-08T21:17:56.417727+0000","last_became_active":"2024-05-08T21:17:02.736187+0000","last_became_peered":"2024-05-08T21:17:02.736187+0000","last_unstale":"2024-05-08T21:17:56.417727+0000","last_undegraded":"2024-05-08T21:17:56.417727+0000","last_fullsized":"2024-05-08T21:17:56.417727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:29.302558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423206+0000","last_change":"2024-05-08T21:17:03.251896+0000","last_active":"2024-05-08T21:17:56.423206+0000","last_peered":"2024-05-08T21:17:56.423206+0000","last_clean":"2024-05-08T21:17:56.423206+0000","last_became_active":"2024-05-08T21:17:03.251495+0000","last_became_peered":"2024-05-08T21:17:03.251495+0000","last_unstale":"2024-05-08T21:17:56.423206+0000","last_undegraded":"2024-05-08T21:17:56.423206+0000","last_fullsized":"2024-05-08T21:17:56.423206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:50.774848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422223+0000","last_change":"2024-05-08T21:17:07.272150+0000","last_active":"2024-05-08T21:17:56.422223+0000","last_peered":"2024-05-08T21:17:56.422223+0000","last_clean":"2024-05-08T21:17:56.422223+0000","last_became_active":"2024-05-08T21:17:07.271727+0000","last_became_peered":"2024-05-08T21:17:07.271727+0000","last_unstale":"2024-05-08T21:17:56.422223+0000","last_undegraded":"2024-05-08T21:17:56.422223+0000","last_fullsized":"2024-05-08T21:17:56.422223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.058277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416860+0000","last_change":"2024-05-08T21:17:02.733611+0000","last_active":"2024-05-08T21:17:56.416860+0000","last_peered":"2024-05-08T21:17:56.416860+0000","last_clean":"2024-05-08T21:17:56.416860+0000","last_became_active":"2024-05-08T21:17:02.733306+0000","last_became_peered":"2024-05-08T21:17:02.733306+0000","last_unstale":"2024-05-08T21:17:56.416860+0000","last_undegraded":"2024-05-08T21:17:56.416860+0000","last_fullsized":"2024-05-08T21:17:56.416860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:33.304753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423294+0000","last_change":"2024-05-08T21:17:03.248480+0000","last_active":"2024-05-08T21:17:56.423294+0000","last_peered":"2024-05-08T21:17:56.423294+0000","last_clean":"2024-05-08T21:17:56.423294+0000","last_became_active":"2024-05-08T21:17:03.248126+0000","last_became_peered":"2024-05-08T21:17:03.248126+0000","last_unstale":"2024-05-08T21:17:56.423294+0000","last_undegraded":"2024-05-08T21:17:56.423294+0000","last_fullsized":"2024-05-08T21:17:56.423294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:08.785761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422472+0000","last_change":"2024-05-08T21:17:07.272344+0000","last_active":"2024-05-08T21:17:56.422472+0000","last_peered":"2024-05-08T21:17:56.422472+0000","last_clean":"2024-05-08T21:17:56.422472+0000","last_became_active":"2024-05-08T21:17:07.271629+0000","last_became_peered":"2024-05-08T21:17:07.271629+0000","last_unstale":"2024-05-08T21:17:56.422472+0000","last_undegraded":"2024-05-08T21:17:56.422472+0000","last_fullsized":"2024-05-08T21:17:56.422472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:15.052748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416295+0000","last_change":"2024-05-08T21:17:02.730364+0000","last_active":"2024-05-08T21:17:56.416295+0000","last_peered":"2024-05-08T21:17:56.416295+0000","last_clean":"2024-05-08T21:17:56.416295+0000","last_became_active":"2024-05-08T21:17:02.728854+0000","last_became_peered":"2024-05-08T21:17:02.728854+0000","last_unstale":"2024-05-08T21:17:56.416295+0000","last_undegraded":"2024-05-08T21:17:56.416295+0000","last_fullsized":"2024-05-08T21:17:56.416295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:09.552016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419025+0000","last_change":"2024-05-08T21:17:03.252713+0000","last_active":"2024-05-08T21:17:56.419025+0000","last_peered":"2024-05-08T21:17:56.419025+0000","last_clean":"2024-05-08T21:17:56.419025+0000","last_became_active":"2024-05-08T21:17:03.252388+0000","last_became_peered":"2024-05-08T21:17:03.252388+0000","last_unstale":"2024-05-08T21:17:56.419025+0000","last_undegraded":"2024-05-08T21:17:56.419025+0000","last_fullsized":"2024-05-08T21:17:56.419025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:09.114064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423183+0000","last_change":"2024-05-08T21:17:07.261112+0000","last_active":"2024-05-08T21:17:56.423183+0000","last_peered":"2024-05-08T21:17:56.423183+0000","last_clean":"2024-05-08T21:17:56.423183+0000","last_became_active":"2024-05-08T21:17:07.260713+0000","last_became_peered":"2024-05-08T21:17:07.260713+0000","last_unstale":"2024-05-08T21:17:56.423183+0000","last_undegraded":"2024-05-08T21:17:56.423183+0000","last_fullsized":"2024-05-08T21:17:56.423183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:37.262113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":122,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:49.510444+0000","last_change":"2024-05-08T21:17:57.436812+0000","last_active":"2024-05-08T21:18:49.510444+0000","last_peered":"2024-05-08T21:18:49.510444+0000","last_clean":"2024-05-08T21:18:49.510444+0000","last_became_active":"2024-05-08T21:17:57.432431+0000","last_became_peered":"2024-05-08T21:17:57.432431+0000","last_unstale":"2024-05-08T21:18:49.510444+0000","last_undegraded":"2024-05-08T21:18:49.510444+0000","last_fullsized":"2024-05-08T21:18:49.510444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:46.062273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1070,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":574,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417176+0000","last_change":"2024-05-08T21:17:03.251245+0000","last_active":"2024-05-08T21:17:56.417176+0000","last_peered":"2024-05-08T21:17:56.417176+0000","last_clean":"2024-05-08T21:17:56.417176+0000","last_became_active":"2024-05-08T21:17:03.250296+0000","last_became_peered":"2024-05-08T21:17:03.250296+0000","last_unstale":"2024-05-08T21:17:56.417176+0000","last_undegraded":"2024-05-08T21:17:56.417176+0000","last_fullsized":"2024-05-08T21:17:56.417176+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:28.391927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423437+0000","last_change":"2024-05-08T21:17:07.266752+0000","last_active":"2024-05-08T21:17:56.423437+0000","last_peered":"2024-05-08T21:17:56.423437+0000","last_clean":"2024-05-08T21:17:56.423437+0000","last_became_active":"2024-05-08T21:17:07.266367+0000","last_became_peered":"2024-05-08T21:17:07.266367+0000","last_unstale":"2024-05-08T21:17:56.423437+0000","last_undegraded":"2024-05-08T21:17:56.423437+0000","last_fullsized":"2024-05-08T21:17:56.423437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:49.874073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417579+0000","last_change":"2024-05-08T21:17:02.728113+0000","last_active":"2024-05-08T21:17:56.417579+0000","last_peered":"2024-05-08T21:17:56.417579+0000","last_clean":"2024-05-08T21:17:56.417579+0000","last_became_active":"2024-05-08T21:17:02.727754+0000","last_became_peered":"2024-05-08T21:17:02.727754+0000","last_unstale":"2024-05-08T21:17:56.417579+0000","last_undegraded":"2024-05-08T21:17:56.417579+0000","last_fullsized":"2024-05-08T21:17:56.417579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:53.789218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418802+0000","last_change":"2024-05-08T21:17:03.245482+0000","last_active":"2024-05-08T21:17:56.418802+0000","last_peered":"2024-05-08T21:17:56.418802+0000","last_clean":"2024-05-08T21:17:56.418802+0000","last_became_active":"2024-05-08T21:17:03.245144+0000","last_became_peered":"2024-05-08T21:17:03.245144+0000","last_unstale":"2024-05-08T21:17:56.418802+0000","last_undegraded":"2024-05-08T21:17:56.418802+0000","last_fullsized":"2024-05-08T21:17:56.418802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:04.051234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418636+0000","last_change":"2024-05-08T21:17:07.268505+0000","last_active":"2024-05-08T21:17:56.418636+0000","last_peered":"2024-05-08T21:17:56.418636+0000","last_clean":"2024-05-08T21:17:56.418636+0000","last_became_active":"2024-05-08T21:17:07.268157+0000","last_became_peered":"2024-05-08T21:17:07.268157+0000","last_unstale":"2024-05-08T21:17:56.418636+0000","last_undegraded":"2024-05-08T21:17:56.418636+0000","last_fullsized":"2024-05-08T21:17:56.418636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:55.220521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412606+0000","last_change":"2024-05-08T21:17:02.727264+0000","last_active":"2024-05-08T21:17:56.412606+0000","last_peered":"2024-05-08T21:17:56.412606+0000","last_clean":"2024-05-08T21:17:56.412606+0000","last_became_active":"2024-05-08T21:17:02.726922+0000","last_became_peered":"2024-05-08T21:17:02.726922+0000","last_unstale":"2024-05-08T21:17:56.412606+0000","last_undegraded":"2024-05-08T21:17:56.412606+0000","last_fullsized":"2024-05-08T21:17:56.412606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:27.410231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415635+0000","last_change":"2024-05-08T21:17:03.233910+0000","last_active":"2024-05-08T21:17:56.415635+0000","last_peered":"2024-05-08T21:17:56.415635+0000","last_clean":"2024-05-08T21:17:56.415635+0000","last_became_active":"2024-05-08T21:17:03.233562+0000","last_became_peered":"2024-05-08T21:17:03.233562+0000","last_unstale":"2024-05-08T21:17:56.415635+0000","last_undegraded":"2024-05-08T21:17:56.415635+0000","last_fullsized":"2024-05-08T21:17:56.415635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:40.309400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423244+0000","last_change":"2024-05-08T21:17:07.264992+0000","last_active":"2024-05-08T21:17:56.423244+0000","last_peered":"2024-05-08T21:17:56.423244+0000","last_clean":"2024-05-08T21:17:56.423244+0000","last_became_active":"2024-05-08T21:17:07.264498+0000","last_became_peered":"2024-05-08T21:17:07.264498+0000","last_unstale":"2024-05-08T21:17:56.423244+0000","last_undegraded":"2024-05-08T21:17:56.423244+0000","last_fullsized":"2024-05-08T21:17:56.423244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:23:26.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418593+0000","last_change":"2024-05-08T21:17:02.725531+0000","last_active":"2024-05-08T21:17:56.418593+0000","last_peered":"2024-05-08T21:17:56.418593+0000","last_clean":"2024-05-08T21:17:56.418593+0000","last_became_active":"2024-05-08T21:17:02.725235+0000","last_became_peered":"2024-05-08T21:17:02.725235+0000","last_unstale":"2024-05-08T21:17:56.418593+0000","last_undegraded":"2024-05-08T21:17:56.418593+0000","last_fullsized":"2024-05-08T21:17:56.418593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:27.265590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419530+0000","last_change":"2024-05-08T21:17:03.236393+0000","last_active":"2024-05-08T21:17:56.419530+0000","last_peered":"2024-05-08T21:17:56.419530+0000","last_clean":"2024-05-08T21:17:56.419530+0000","last_became_active":"2024-05-08T21:17:03.236092+0000","last_became_peered":"2024-05-08T21:17:03.236092+0000","last_unstale":"2024-05-08T21:17:56.419530+0000","last_undegraded":"2024-05-08T21:17:56.419530+0000","last_fullsized":"2024-05-08T21:17:56.419530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:58.232669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417799+0000","last_change":"2024-05-08T21:17:07.271374+0000","last_active":"2024-05-08T21:17:56.417799+0000","last_peered":"2024-05-08T21:17:56.417799+0000","last_clean":"2024-05-08T21:17:56.417799+0000","last_became_active":"2024-05-08T21:17:07.270778+0000","last_became_peered":"2024-05-08T21:17:07.270778+0000","last_unstale":"2024-05-08T21:17:56.417799+0000","last_undegraded":"2024-05-08T21:17:56.417799+0000","last_fullsized":"2024-05-08T21:17:56.417799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:50.212213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'6","reported_seq":184,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:50.447209+0000","last_change":"2024-05-08T21:17:02.725624+0000","last_active":"2024-05-08T21:18:50.447209+0000","last_peered":"2024-05-08T21:18:50.447209+0000","last_clean":"2024-05-08T21:18:50.447209+0000","last_became_active":"2024-05-08T21:17:02.725367+0000","last_became_peered":"2024-05-08T21:17:02.725367+0000","last_unstale":"2024-05-08T21:18:50.447209+0000","last_undegraded":"2024-05-08T21:18:50.447209+0000","last_fullsized":"2024-05-08T21:18:50.447209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:19.573931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":160,"num_read_kb":160,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423993+0000","last_change":"2024-05-08T21:17:03.248729+0000","last_active":"2024-05-08T21:17:56.423993+0000","last_peered":"2024-05-08T21:17:56.423993+0000","last_clean":"2024-05-08T21:17:56.423993+0000","last_became_active":"2024-05-08T21:17:03.247899+0000","last_became_peered":"2024-05-08T21:17:03.247899+0000","last_unstale":"2024-05-08T21:17:56.423993+0000","last_undegraded":"2024-05-08T21:17:56.423993+0000","last_fullsized":"2024-05-08T21:17:56.423993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:27:57.012249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419291+0000","last_change":"2024-05-08T21:17:07.261257+0000","last_active":"2024-05-08T21:17:56.419291+0000","last_peered":"2024-05-08T21:17:56.419291+0000","last_clean":"2024-05-08T21:17:56.419291+0000","last_became_active":"2024-05-08T21:17:07.260785+0000","last_became_peered":"2024-05-08T21:17:07.260785+0000","last_unstale":"2024-05-08T21:17:56.419291+0000","last_undegraded":"2024-05-08T21:17:56.419291+0000","last_fullsized":"2024-05-08T21:17:56.419291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:58.469134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419901+0000","last_change":"2024-05-08T21:17:02.729569+0000","last_active":"2024-05-08T21:17:56.419901+0000","last_peered":"2024-05-08T21:17:56.419901+0000","last_clean":"2024-05-08T21:17:56.419901+0000","last_became_active":"2024-05-08T21:17:02.729096+0000","last_became_peered":"2024-05-08T21:17:02.729096+0000","last_unstale":"2024-05-08T21:17:56.419901+0000","last_undegraded":"2024-05-08T21:17:56.419901+0000","last_fullsized":"2024-05-08T21:17:56.419901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:30.668169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423406+0000","last_change":"2024-05-08T21:17:03.246387+0000","last_active":"2024-05-08T21:17:56.423406+0000","last_peered":"2024-05-08T21:17:56.423406+0000","last_clean":"2024-05-08T21:17:56.423406+0000","last_became_active":"2024-05-08T21:17:03.244658+0000","last_became_peered":"2024-05-08T21:17:03.244658+0000","last_unstale":"2024-05-08T21:17:56.423406+0000","last_undegraded":"2024-05-08T21:17:56.423406+0000","last_fullsized":"2024-05-08T21:17:56.423406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:02.602080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.473101+0000","last_change":"2024-05-08T21:17:57.473101+0000","last_active":"2024-05-08T21:17:57.473101+0000","last_peered":"2024-05-08T21:17:57.473101+0000","last_clean":"2024-05-08T21:17:57.473101+0000","last_became_active":"2024-05-08T21:17:57.472563+0000","last_became_peered":"2024-05-08T21:17:57.472563+0000","last_unstale":"2024-05-08T21:17:57.473101+0000","last_undegraded":"2024-05-08T21:17:57.473101+0000","last_fullsized":"2024-05-08T21:17:57.473101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:23.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418913+0000","last_change":"2024-05-08T21:17:02.732738+0000","last_active":"2024-05-08T21:17:56.418913+0000","last_peered":"2024-05-08T21:17:56.418913+0000","last_clean":"2024-05-08T21:17:56.418913+0000","last_became_active":"2024-05-08T21:17:02.731733+0000","last_became_peered":"2024-05-08T21:17:02.731733+0000","last_unstale":"2024-05-08T21:17:56.418913+0000","last_undegraded":"2024-05-08T21:17:56.418913+0000","last_fullsized":"2024-05-08T21:17:56.418913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:24.201171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423479+0000","last_change":"2024-05-08T21:17:03.254405+0000","last_active":"2024-05-08T21:17:56.423479+0000","last_peered":"2024-05-08T21:17:56.423479+0000","last_clean":"2024-05-08T21:17:56.423479+0000","last_became_active":"2024-05-08T21:17:03.252137+0000","last_became_peered":"2024-05-08T21:17:03.252137+0000","last_unstale":"2024-05-08T21:17:56.423479+0000","last_undegraded":"2024-05-08T21:17:56.423479+0000","last_fullsized":"2024-05-08T21:17:56.423479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:15.472225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.431574+0000","last_change":"2024-05-08T21:17:57.431091+0000","last_active":"2024-05-08T21:17:57.431574+0000","last_peered":"2024-05-08T21:17:57.431574+0000","last_clean":"2024-05-08T21:17:57.431574+0000","last_became_active":"2024-05-08T21:17:57.430668+0000","last_became_peered":"2024-05-08T21:17:57.430668+0000","last_unstale":"2024-05-08T21:17:57.431574+0000","last_undegraded":"2024-05-08T21:17:57.431574+0000","last_fullsized":"2024-05-08T21:17:57.431574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:33.208801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"87'38","reported_seq":225,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:50.511512+0000","last_change":"2024-05-08T21:17:02.723975+0000","last_active":"2024-05-08T21:18:50.511512+0000","last_peered":"2024-05-08T21:18:50.511512+0000","last_clean":"2024-05-08T21:18:50.511512+0000","last_became_active":"2024-05-08T21:17:02.723631+0000","last_became_peered":"2024-05-08T21:17:02.723631+0000","last_unstale":"2024-05-08T21:18:50.511512+0000","last_undegraded":"2024-05-08T21:18:50.511512+0000","last_fullsized":"2024-05-08T21:18:50.511512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:17.676007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62785,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":169,"num_read_kb":145,"num_write":40,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416534+0000","last_change":"2024-05-08T21:17:03.251570+0000","last_active":"2024-05-08T21:17:56.416534+0000","last_peered":"2024-05-08T21:17:56.416534+0000","last_clean":"2024-05-08T21:17:56.416534+0000","last_became_active":"2024-05-08T21:17:03.251134+0000","last_became_peered":"2024-05-08T21:17:03.251134+0000","last_unstale":"2024-05-08T21:17:56.416534+0000","last_undegraded":"2024-05-08T21:17:56.416534+0000","last_fullsized":"2024-05-08T21:17:56.416534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:28:09.339958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419635+0000","last_change":"2024-05-08T21:17:07.272051+0000","last_active":"2024-05-08T21:17:56.419635+0000","last_peered":"2024-05-08T21:17:56.419635+0000","last_clean":"2024-05-08T21:17:56.419635+0000","last_became_active":"2024-05-08T21:17:07.271616+0000","last_became_peered":"2024-05-08T21:17:07.271616+0000","last_unstale":"2024-05-08T21:17:56.419635+0000","last_undegraded":"2024-05-08T21:17:56.419635+0000","last_fullsized":"2024-05-08T21:17:56.419635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.364731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423259+0000","last_change":"2024-05-08T21:17:02.736000+0000","last_active":"2024-05-08T21:17:56.423259+0000","last_peered":"2024-05-08T21:17:56.423259+0000","last_clean":"2024-05-08T21:17:56.423259+0000","last_became_active":"2024-05-08T21:17:02.735629+0000","last_became_peered":"2024-05-08T21:17:02.735629+0000","last_unstale":"2024-05-08T21:17:56.423259+0000","last_undegraded":"2024-05-08T21:17:56.423259+0000","last_fullsized":"2024-05-08T21:17:56.423259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:21.842135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412729+0000","last_change":"2024-05-08T21:17:03.237144+0000","last_active":"2024-05-08T21:17:56.412729+0000","last_peered":"2024-05-08T21:17:56.412729+0000","last_clean":"2024-05-08T21:17:56.412729+0000","last_became_active":"2024-05-08T21:17:03.236792+0000","last_became_peered":"2024-05-08T21:17:03.236792+0000","last_unstale":"2024-05-08T21:17:56.412729+0000","last_undegraded":"2024-05-08T21:17:56.412729+0000","last_fullsized":"2024-05-08T21:17:56.412729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:04.520986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.627656+0000","last_change":"2024-05-08T21:17:57.627656+0000","last_active":"2024-05-08T21:17:57.627656+0000","last_peered":"2024-05-08T21:17:57.627656+0000","last_clean":"2024-05-08T21:17:57.627656+0000","last_became_active":"2024-05-08T21:17:57.627108+0000","last_became_peered":"2024-05-08T21:17:57.627108+0000","last_unstale":"2024-05-08T21:17:57.627656+0000","last_undegraded":"2024-05-08T21:17:57.627656+0000","last_fullsized":"2024-05-08T21:17:57.627656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:24.025145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419802+0000","last_change":"2024-05-08T21:17:02.727748+0000","last_active":"2024-05-08T21:17:56.419802+0000","last_peered":"2024-05-08T21:17:56.419802+0000","last_clean":"2024-05-08T21:17:56.419802+0000","last_became_active":"2024-05-08T21:17:02.727444+0000","last_became_peered":"2024-05-08T21:17:02.727444+0000","last_unstale":"2024-05-08T21:17:56.419802+0000","last_undegraded":"2024-05-08T21:17:56.419802+0000","last_fullsized":"2024-05-08T21:17:56.419802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:47.939235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420245+0000","last_change":"2024-05-08T21:17:03.247185+0000","last_active":"2024-05-08T21:17:56.420245+0000","last_peered":"2024-05-08T21:17:56.420245+0000","last_clean":"2024-05-08T21:17:56.420245+0000","last_became_active":"2024-05-08T21:17:03.246880+0000","last_became_peered":"2024-05-08T21:17:03.246880+0000","last_unstale":"2024-05-08T21:17:56.420245+0000","last_undegraded":"2024-05-08T21:17:56.420245+0000","last_fullsized":"2024-05-08T21:17:56.420245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:38.978353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420018+0000","last_change":"2024-05-08T21:17:07.273710+0000","last_active":"2024-05-08T21:17:56.420018+0000","last_peered":"2024-05-08T21:17:56.420018+0000","last_clean":"2024-05-08T21:17:56.420018+0000","last_became_active":"2024-05-08T21:17:07.273119+0000","last_became_peered":"2024-05-08T21:17:07.273119+0000","last_unstale":"2024-05-08T21:17:56.420018+0000","last_undegraded":"2024-05-08T21:17:56.420018+0000","last_fullsized":"2024-05-08T21:17:56.420018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:30.589440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":104,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:47.422631+0000","last_change":"2024-05-08T21:17:02.725508+0000","last_active":"2024-05-08T21:18:47.422631+0000","last_peered":"2024-05-08T21:18:47.422631+0000","last_clean":"2024-05-08T21:18:47.422631+0000","last_became_active":"2024-05-08T21:17:02.725107+0000","last_became_peered":"2024-05-08T21:17:02.725107+0000","last_unstale":"2024-05-08T21:18:47.422631+0000","last_undegraded":"2024-05-08T21:18:47.422631+0000","last_fullsized":"2024-05-08T21:18:47.422631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:13.503702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1036,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":81,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420302+0000","last_change":"2024-05-08T21:17:03.245372+0000","last_active":"2024-05-08T21:17:56.420302+0000","last_peered":"2024-05-08T21:17:56.420302+0000","last_clean":"2024-05-08T21:17:56.420302+0000","last_became_active":"2024-05-08T21:17:03.245054+0000","last_became_peered":"2024-05-08T21:17:03.245054+0000","last_unstale":"2024-05-08T21:17:56.420302+0000","last_undegraded":"2024-05-08T21:17:56.420302+0000","last_fullsized":"2024-05-08T21:17:56.420302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:30:25.900846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414821+0000","last_change":"2024-05-08T21:17:07.272081+0000","last_active":"2024-05-08T21:17:56.414821+0000","last_peered":"2024-05-08T21:17:56.414821+0000","last_clean":"2024-05-08T21:17:56.414821+0000","last_became_active":"2024-05-08T21:17:07.271560+0000","last_became_peered":"2024-05-08T21:17:07.271560+0000","last_unstale":"2024-05-08T21:17:56.414821+0000","last_undegraded":"2024-05-08T21:17:56.414821+0000","last_fullsized":"2024-05-08T21:17:56.414821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:00.490798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416027+0000","last_change":"2024-05-08T21:17:02.734420+0000","last_active":"2024-05-08T21:17:56.416027+0000","last_peered":"2024-05-08T21:17:56.416027+0000","last_clean":"2024-05-08T21:17:56.416027+0000","last_became_active":"2024-05-08T21:17:02.734071+0000","last_became_peered":"2024-05-08T21:17:02.734071+0000","last_unstale":"2024-05-08T21:17:56.416027+0000","last_undegraded":"2024-05-08T21:17:56.416027+0000","last_fullsized":"2024-05-08T21:17:56.416027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:28.537464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419127+0000","last_change":"2024-05-08T21:17:03.250706+0000","last_active":"2024-05-08T21:17:56.419127+0000","last_peered":"2024-05-08T21:17:56.419127+0000","last_clean":"2024-05-08T21:17:56.419127+0000","last_became_active":"2024-05-08T21:17:03.250408+0000","last_became_peered":"2024-05-08T21:17:03.250408+0000","last_unstale":"2024-05-08T21:17:56.419127+0000","last_undegraded":"2024-05-08T21:17:56.419127+0000","last_fullsized":"2024-05-08T21:17:56.419127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:43.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418530+0000","last_change":"2024-05-08T21:17:07.272066+0000","last_active":"2024-05-08T21:17:56.418530+0000","last_peered":"2024-05-08T21:17:56.418530+0000","last_clean":"2024-05-08T21:17:56.418530+0000","last_became_active":"2024-05-08T21:17:07.271628+0000","last_became_peered":"2024-05-08T21:17:07.271628+0000","last_unstale":"2024-05-08T21:17:56.418530+0000","last_undegraded":"2024-05-08T21:17:56.418530+0000","last_fullsized":"2024-05-08T21:17:56.418530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:27.731995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.424263+0000","last_change":"2024-05-08T21:17:02.728464+0000","last_active":"2024-05-08T21:17:57.424263+0000","last_peered":"2024-05-08T21:17:57.424263+0000","last_clean":"2024-05-08T21:17:57.424263+0000","last_became_active":"2024-05-08T21:17:02.727914+0000","last_became_peered":"2024-05-08T21:17:02.727914+0000","last_unstale":"2024-05-08T21:17:57.424263+0000","last_undegraded":"2024-05-08T21:17:57.424263+0000","last_fullsized":"2024-05-08T21:17:57.424263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:55.948661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418448+0000","last_change":"2024-05-08T21:17:03.245578+0000","last_active":"2024-05-08T21:17:56.418448+0000","last_peered":"2024-05-08T21:17:56.418448+0000","last_clean":"2024-05-08T21:17:56.418448+0000","last_became_active":"2024-05-08T21:17:03.245256+0000","last_became_peered":"2024-05-08T21:17:03.245256+0000","last_unstale":"2024-05-08T21:17:56.418448+0000","last_undegraded":"2024-05-08T21:17:56.418448+0000","last_fullsized":"2024-05-08T21:17:56.418448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417048+0000","last_change":"2024-05-08T21:17:07.439525+0000","last_active":"2024-05-08T21:17:56.417048+0000","last_peered":"2024-05-08T21:17:56.417048+0000","last_clean":"2024-05-08T21:17:56.417048+0000","last_became_active":"2024-05-08T21:17:07.438987+0000","last_became_peered":"2024-05-08T21:17:07.438987+0000","last_unstale":"2024-05-08T21:17:56.417048+0000","last_undegraded":"2024-05-08T21:17:56.417048+0000","last_fullsized":"2024-05-08T21:17:56.417048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:57:45.373981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.432999+0000","last_change":"2024-05-08T21:17:57.432999+0000","last_active":"2024-05-08T21:17:57.432999+0000","last_peered":"2024-05-08T21:17:57.432999+0000","last_clean":"2024-05-08T21:17:57.432999+0000","last_became_active":"2024-05-08T21:17:57.432567+0000","last_became_peered":"2024-05-08T21:17:57.432567+0000","last_unstale":"2024-05-08T21:17:57.432999+0000","last_undegraded":"2024-05-08T21:17:57.432999+0000","last_fullsized":"2024-05-08T21:17:57.432999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:33:00.188046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417907+0000","last_change":"2024-05-08T21:17:03.235556+0000","last_active":"2024-05-08T21:17:56.417907+0000","last_peered":"2024-05-08T21:17:56.417907+0000","last_clean":"2024-05-08T21:17:56.417907+0000","last_became_active":"2024-05-08T21:17:03.235214+0000","last_became_peered":"2024-05-08T21:17:03.235214+0000","last_unstale":"2024-05-08T21:17:56.417907+0000","last_undegraded":"2024-05-08T21:17:56.417907+0000","last_fullsized":"2024-05-08T21:17:56.417907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:45.042596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414845+0000","last_change":"2024-05-08T21:17:07.266080+0000","last_active":"2024-05-08T21:17:56.414845+0000","last_peered":"2024-05-08T21:17:56.414845+0000","last_clean":"2024-05-08T21:17:56.414845+0000","last_became_active":"2024-05-08T21:17:07.265605+0000","last_became_peered":"2024-05-08T21:17:07.265605+0000","last_unstale":"2024-05-08T21:17:56.414845+0000","last_undegraded":"2024-05-08T21:17:56.414845+0000","last_fullsized":"2024-05-08T21:17:56.414845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:49:48.387170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:49.549637+0000","last_change":"2024-05-08T21:17:02.728743+0000","last_active":"2024-05-08T21:18:49.549637+0000","last_peered":"2024-05-08T21:18:49.549637+0000","last_clean":"2024-05-08T21:18:49.549637+0000","last_became_active":"2024-05-08T21:17:02.728330+0000","last_became_peered":"2024-05-08T21:17:02.728330+0000","last_unstale":"2024-05-08T21:18:49.549637+0000","last_undegraded":"2024-05-08T21:18:49.549637+0000","last_fullsized":"2024-05-08T21:18:49.549637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:43.865117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1036,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417780+0000","last_change":"2024-05-08T21:17:03.245672+0000","last_active":"2024-05-08T21:17:56.417780+0000","last_peered":"2024-05-08T21:17:56.417780+0000","last_clean":"2024-05-08T21:17:56.417780+0000","last_became_active":"2024-05-08T21:17:03.245318+0000","last_became_peered":"2024-05-08T21:17:03.245318+0000","last_unstale":"2024-05-08T21:17:56.417780+0000","last_undegraded":"2024-05-08T21:17:56.417780+0000","last_fullsized":"2024-05-08T21:17:56.417780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:43.030011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414114+0000","last_change":"2024-05-08T21:17:07.439289+0000","last_active":"2024-05-08T21:17:56.414114+0000","last_peered":"2024-05-08T21:17:56.414114+0000","last_clean":"2024-05-08T21:17:56.414114+0000","last_became_active":"2024-05-08T21:17:07.438874+0000","last_became_peered":"2024-05-08T21:17:07.438874+0000","last_unstale":"2024-05-08T21:17:56.414114+0000","last_undegraded":"2024-05-08T21:17:56.414114+0000","last_fullsized":"2024-05-08T21:17:56.414114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:14.820087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419309+0000","last_change":"2024-05-08T21:17:03.247081+0000","last_active":"2024-05-08T21:17:56.419309+0000","last_peered":"2024-05-08T21:17:56.419309+0000","last_clean":"2024-05-08T21:17:56.419309+0000","last_became_active":"2024-05-08T21:17:03.246418+0000","last_became_peered":"2024-05-08T21:17:03.246418+0000","last_unstale":"2024-05-08T21:17:56.419309+0000","last_undegraded":"2024-05-08T21:17:56.419309+0000","last_fullsized":"2024-05-08T21:17:56.419309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:03.993770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":103,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:47.421770+0000","last_change":"2024-05-08T21:17:02.730764+0000","last_active":"2024-05-08T21:18:47.421770+0000","last_peered":"2024-05-08T21:18:47.421770+0000","last_clean":"2024-05-08T21:18:47.421770+0000","last_became_active":"2024-05-08T21:17:02.730494+0000","last_became_peered":"2024-05-08T21:17:02.730494+0000","last_unstale":"2024-05-08T21:18:47.421770+0000","last_undegraded":"2024-05-08T21:18:47.421770+0000","last_fullsized":"2024-05-08T21:18:47.421770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:50.070368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419669+0000","last_change":"2024-05-08T21:17:07.266424+0000","last_active":"2024-05-08T21:17:56.419669+0000","last_peered":"2024-05-08T21:17:56.419669+0000","last_clean":"2024-05-08T21:17:56.419669+0000","last_became_active":"2024-05-08T21:17:07.266144+0000","last_became_peered":"2024-05-08T21:17:07.266144+0000","last_unstale":"2024-05-08T21:17:56.419669+0000","last_undegraded":"2024-05-08T21:17:56.419669+0000","last_fullsized":"2024-05-08T21:17:56.419669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:22.285749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417254+0000","last_change":"2024-05-08T21:17:03.250461+0000","last_active":"2024-05-08T21:17:56.417254+0000","last_peered":"2024-05-08T21:17:56.417254+0000","last_clean":"2024-05-08T21:17:56.417254+0000","last_became_active":"2024-05-08T21:17:03.249974+0000","last_became_peered":"2024-05-08T21:17:03.249974+0000","last_unstale":"2024-05-08T21:17:56.417254+0000","last_undegraded":"2024-05-08T21:17:56.417254+0000","last_fullsized":"2024-05-08T21:17:56.417254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:25:58.759668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422681+0000","last_change":"2024-05-08T21:17:02.725648+0000","last_active":"2024-05-08T21:17:56.422681+0000","last_peered":"2024-05-08T21:17:56.422681+0000","last_clean":"2024-05-08T21:17:56.422681+0000","last_became_active":"2024-05-08T21:17:02.725297+0000","last_became_peered":"2024-05-08T21:17:02.725297+0000","last_unstale":"2024-05-08T21:17:56.422681+0000","last_undegraded":"2024-05-08T21:17:56.422681+0000","last_fullsized":"2024-05-08T21:17:56.422681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:41.462830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":328,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420115+0000","last_change":"2024-05-08T21:16:15.194478+0000","last_active":"2024-05-08T21:17:56.420115+0000","last_peered":"2024-05-08T21:17:56.420115+0000","last_clean":"2024-05-08T21:17:56.420115+0000","last_became_active":"2024-05-08T21:16:15.193964+0000","last_became_peered":"2024-05-08T21:16:15.193964+0000","last_unstale":"2024-05-08T21:17:56.420115+0000","last_undegraded":"2024-05-08T21:17:56.420115+0000","last_fullsized":"2024-05-08T21:17:56.420115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_clean_scrub_stamp":"2024-05-08T21:13:48.584026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:28.091200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.628348+0000","last_change":"2024-05-08T21:17:57.627889+0000","last_active":"2024-05-08T21:17:57.628348+0000","last_peered":"2024-05-08T21:17:57.628348+0000","last_clean":"2024-05-08T21:17:57.628348+0000","last_became_active":"2024-05-08T21:17:57.627502+0000","last_became_peered":"2024-05-08T21:17:57.627502+0000","last_unstale":"2024-05-08T21:17:57.628348+0000","last_undegraded":"2024-05-08T21:17:57.628348+0000","last_fullsized":"2024-05-08T21:17:57.628348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:10.025347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412481+0000","last_change":"2024-05-08T21:17:03.246034+0000","last_active":"2024-05-08T21:17:56.412481+0000","last_peered":"2024-05-08T21:17:56.412481+0000","last_clean":"2024-05-08T21:17:56.412481+0000","last_became_active":"2024-05-08T21:17:03.245620+0000","last_became_peered":"2024-05-08T21:17:03.245620+0000","last_unstale":"2024-05-08T21:17:56.412481+0000","last_undegraded":"2024-05-08T21:17:56.412481+0000","last_fullsized":"2024-05-08T21:17:56.412481+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:02.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420424+0000","last_change":"2024-05-08T21:17:02.731260+0000","last_active":"2024-05-08T21:17:56.420424+0000","last_peered":"2024-05-08T21:17:56.420424+0000","last_clean":"2024-05-08T21:17:56.420424+0000","last_became_active":"2024-05-08T21:17:02.730879+0000","last_became_peered":"2024-05-08T21:17:02.730879+0000","last_unstale":"2024-05-08T21:17:56.420424+0000","last_undegraded":"2024-05-08T21:17:56.420424+0000","last_fullsized":"2024-05-08T21:17:56.420424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:59.030674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417621+0000","last_change":"2024-05-08T21:17:07.269791+0000","last_active":"2024-05-08T21:17:56.417621+0000","last_peered":"2024-05-08T21:17:56.417621+0000","last_clean":"2024-05-08T21:17:56.417621+0000","last_became_active":"2024-05-08T21:17:07.269345+0000","last_became_peered":"2024-05-08T21:17:07.269345+0000","last_unstale":"2024-05-08T21:17:56.417621+0000","last_undegraded":"2024-05-08T21:17:56.417621+0000","last_fullsized":"2024-05-08T21:17:56.417621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:42.866864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416940+0000","last_change":"2024-05-08T21:17:02.735423+0000","last_active":"2024-05-08T21:17:56.416940+0000","last_peered":"2024-05-08T21:17:56.416940+0000","last_clean":"2024-05-08T21:17:56.416940+0000","last_became_active":"2024-05-08T21:17:02.735158+0000","last_became_peered":"2024-05-08T21:17:02.735158+0000","last_unstale":"2024-05-08T21:17:56.416940+0000","last_undegraded":"2024-05-08T21:17:56.416940+0000","last_fullsized":"2024-05-08T21:17:56.416940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:19.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415012+0000","last_change":"2024-05-08T21:17:03.239316+0000","last_active":"2024-05-08T21:17:56.415012+0000","last_peered":"2024-05-08T21:17:56.415012+0000","last_clean":"2024-05-08T21:17:56.415012+0000","last_became_active":"2024-05-08T21:17:03.239018+0000","last_became_peered":"2024-05-08T21:17:03.239018+0000","last_unstale":"2024-05-08T21:17:56.415012+0000","last_undegraded":"2024-05-08T21:17:56.415012+0000","last_fullsized":"2024-05-08T21:17:56.415012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:36.952155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415569+0000","last_change":"2024-05-08T21:17:07.439338+0000","last_active":"2024-05-08T21:17:56.415569+0000","last_peered":"2024-05-08T21:17:56.415569+0000","last_clean":"2024-05-08T21:17:56.415569+0000","last_became_active":"2024-05-08T21:17:07.438703+0000","last_became_peered":"2024-05-08T21:17:07.438703+0000","last_unstale":"2024-05-08T21:17:56.415569+0000","last_undegraded":"2024-05-08T21:17:56.415569+0000","last_fullsized":"2024-05-08T21:17:56.415569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:50.997138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420144+0000","last_change":"2024-05-08T21:17:02.729012+0000","last_active":"2024-05-08T21:17:56.420144+0000","last_peered":"2024-05-08T21:17:56.420144+0000","last_clean":"2024-05-08T21:17:56.420144+0000","last_became_active":"2024-05-08T21:17:02.728606+0000","last_became_peered":"2024-05-08T21:17:02.728606+0000","last_unstale":"2024-05-08T21:17:56.420144+0000","last_undegraded":"2024-05-08T21:17:56.420144+0000","last_fullsized":"2024-05-08T21:17:56.420144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.140609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416523+0000","last_change":"2024-05-08T21:17:03.234712+0000","last_active":"2024-05-08T21:17:56.416523+0000","last_peered":"2024-05-08T21:17:56.416523+0000","last_clean":"2024-05-08T21:17:56.416523+0000","last_became_active":"2024-05-08T21:17:03.234404+0000","last_became_peered":"2024-05-08T21:17:03.234404+0000","last_unstale":"2024-05-08T21:17:56.416523+0000","last_undegraded":"2024-05-08T21:17:56.416523+0000","last_fullsized":"2024-05-08T21:17:56.416523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:41.479470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422480+0000","last_change":"2024-05-08T21:17:07.259120+0000","last_active":"2024-05-08T21:17:56.422480+0000","last_peered":"2024-05-08T21:17:56.422480+0000","last_clean":"2024-05-08T21:17:56.422480+0000","last_became_active":"2024-05-08T21:17:07.258696+0000","last_became_peered":"2024-05-08T21:17:07.258696+0000","last_unstale":"2024-05-08T21:17:56.422480+0000","last_undegraded":"2024-05-08T21:17:56.422480+0000","last_fullsized":"2024-05-08T21:17:56.422480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:26.950767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423353+0000","last_change":"2024-05-08T21:17:02.727795+0000","last_active":"2024-05-08T21:17:56.423353+0000","last_peered":"2024-05-08T21:17:56.423353+0000","last_clean":"2024-05-08T21:17:56.423353+0000","last_became_active":"2024-05-08T21:17:02.727553+0000","last_became_peered":"2024-05-08T21:17:02.727553+0000","last_unstale":"2024-05-08T21:17:56.423353+0000","last_undegraded":"2024-05-08T21:17:56.423353+0000","last_fullsized":"2024-05-08T21:17:56.423353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:51.540221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423413+0000","last_change":"2024-05-08T21:17:03.247270+0000","last_active":"2024-05-08T21:17:56.423413+0000","last_peered":"2024-05-08T21:17:56.423413+0000","last_clean":"2024-05-08T21:17:56.423413+0000","last_became_active":"2024-05-08T21:17:03.246853+0000","last_became_peered":"2024-05-08T21:17:03.246853+0000","last_unstale":"2024-05-08T21:17:56.423413+0000","last_undegraded":"2024-05-08T21:17:56.423413+0000","last_fullsized":"2024-05-08T21:17:56.423413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:46.224088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422880+0000","last_change":"2024-05-08T21:17:07.439920+0000","last_active":"2024-05-08T21:17:56.422880+0000","last_peered":"2024-05-08T21:17:56.422880+0000","last_clean":"2024-05-08T21:17:56.422880+0000","last_became_active":"2024-05-08T21:17:07.439459+0000","last_became_peered":"2024-05-08T21:17:07.439459+0000","last_unstale":"2024-05-08T21:17:56.422880+0000","last_undegraded":"2024-05-08T21:17:56.422880+0000","last_fullsized":"2024-05-08T21:17:56.422880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:03.008930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422835+0000","last_change":"2024-05-08T21:17:02.731889+0000","last_active":"2024-05-08T21:17:56.422835+0000","last_peered":"2024-05-08T21:17:56.422835+0000","last_clean":"2024-05-08T21:17:56.422835+0000","last_became_active":"2024-05-08T21:17:02.731554+0000","last_became_peered":"2024-05-08T21:17:02.731554+0000","last_unstale":"2024-05-08T21:17:56.422835+0000","last_undegraded":"2024-05-08T21:17:56.422835+0000","last_fullsized":"2024-05-08T21:17:56.422835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:25.223003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417051+0000","last_change":"2024-05-08T21:17:03.252672+0000","last_active":"2024-05-08T21:17:56.417051+0000","last_peered":"2024-05-08T21:17:56.417051+0000","last_clean":"2024-05-08T21:17:56.417051+0000","last_became_active":"2024-05-08T21:17:03.251684+0000","last_became_peered":"2024-05-08T21:17:03.251684+0000","last_unstale":"2024-05-08T21:17:56.417051+0000","last_undegraded":"2024-05-08T21:17:56.417051+0000","last_fullsized":"2024-05-08T21:17:56.417051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:06.460419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420488+0000","last_change":"2024-05-08T21:17:07.273478+0000","last_active":"2024-05-08T21:17:56.420488+0000","last_peered":"2024-05-08T21:17:56.420488+0000","last_clean":"2024-05-08T21:17:56.420488+0000","last_became_active":"2024-05-08T21:17:07.272525+0000","last_became_peered":"2024-05-08T21:17:07.272525+0000","last_unstale":"2024-05-08T21:17:56.420488+0000","last_undegraded":"2024-05-08T21:17:56.420488+0000","last_fullsized":"2024-05-08T21:17:56.420488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:51:05.107387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417904+0000","last_change":"2024-05-08T21:17:02.728858+0000","last_active":"2024-05-08T21:17:56.417904+0000","last_peered":"2024-05-08T21:17:56.417904+0000","last_clean":"2024-05-08T21:17:56.417904+0000","last_became_active":"2024-05-08T21:17:02.728516+0000","last_became_peered":"2024-05-08T21:17:02.728516+0000","last_unstale":"2024-05-08T21:17:56.417904+0000","last_undegraded":"2024-05-08T21:17:56.417904+0000","last_fullsized":"2024-05-08T21:17:56.417904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:40.263983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418555+0000","last_change":"2024-05-08T21:17:03.239354+0000","last_active":"2024-05-08T21:17:56.418555+0000","last_peered":"2024-05-08T21:17:56.418555+0000","last_clean":"2024-05-08T21:17:56.418555+0000","last_became_active":"2024-05-08T21:17:03.239020+0000","last_became_peered":"2024-05-08T21:17:03.239020+0000","last_unstale":"2024-05-08T21:17:56.418555+0000","last_undegraded":"2024-05-08T21:17:56.418555+0000","last_fullsized":"2024-05-08T21:17:56.418555+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:02:43.362016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422887+0000","last_change":"2024-05-08T21:17:07.262401+0000","last_active":"2024-05-08T21:17:56.422887+0000","last_peered":"2024-05-08T21:17:56.422887+0000","last_clean":"2024-05-08T21:17:56.422887+0000","last_became_active":"2024-05-08T21:17:07.261946+0000","last_became_peered":"2024-05-08T21:17:07.261946+0000","last_unstale":"2024-05-08T21:17:56.422887+0000","last_undegraded":"2024-05-08T21:17:56.422887+0000","last_fullsized":"2024-05-08T21:17:56.422887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:56.565604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.438045+0000","last_change":"2024-05-08T21:17:57.437573+0000","last_active":"2024-05-08T21:17:57.438045+0000","last_peered":"2024-05-08T21:17:57.438045+0000","last_clean":"2024-05-08T21:17:57.438045+0000","last_became_active":"2024-05-08T21:17:57.432315+0000","last_became_peered":"2024-05-08T21:17:57.432315+0000","last_unstale":"2024-05-08T21:17:57.438045+0000","last_undegraded":"2024-05-08T21:17:57.438045+0000","last_fullsized":"2024-05-08T21:17:57.438045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:56:08.847290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417312+0000","last_change":"2024-05-08T21:17:03.250382+0000","last_active":"2024-05-08T21:17:56.417312+0000","last_peered":"2024-05-08T21:17:56.417312+0000","last_clean":"2024-05-08T21:17:56.417312+0000","last_became_active":"2024-05-08T21:17:03.250031+0000","last_became_peered":"2024-05-08T21:17:03.250031+0000","last_unstale":"2024-05-08T21:17:56.417312+0000","last_undegraded":"2024-05-08T21:17:56.417312+0000","last_fullsized":"2024-05-08T21:17:56.417312+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:59.961012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422877+0000","last_change":"2024-05-08T21:17:07.439341+0000","last_active":"2024-05-08T21:17:56.422877+0000","last_peered":"2024-05-08T21:17:56.422877+0000","last_clean":"2024-05-08T21:17:56.422877+0000","last_became_active":"2024-05-08T21:17:07.438896+0000","last_became_peered":"2024-05-08T21:17:07.438896+0000","last_unstale":"2024-05-08T21:17:56.422877+0000","last_undegraded":"2024-05-08T21:17:56.422877+0000","last_fullsized":"2024-05-08T21:17:56.422877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:55.180686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416194+0000","last_change":"2024-05-08T21:17:02.732967+0000","last_active":"2024-05-08T21:17:56.416194+0000","last_peered":"2024-05-08T21:17:56.416194+0000","last_clean":"2024-05-08T21:17:56.416194+0000","last_became_active":"2024-05-08T21:17:02.732616+0000","last_became_peered":"2024-05-08T21:17:02.732616+0000","last_unstale":"2024-05-08T21:17:56.416194+0000","last_undegraded":"2024-05-08T21:17:56.416194+0000","last_fullsized":"2024-05-08T21:17:56.416194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:50:26.116841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420367+0000","last_change":"2024-05-08T21:17:03.241974+0000","last_active":"2024-05-08T21:17:56.420367+0000","last_peered":"2024-05-08T21:17:56.420367+0000","last_clean":"2024-05-08T21:17:56.420367+0000","last_became_active":"2024-05-08T21:17:03.241627+0000","last_became_peered":"2024-05-08T21:17:03.241627+0000","last_unstale":"2024-05-08T21:17:56.420367+0000","last_undegraded":"2024-05-08T21:17:56.420367+0000","last_fullsized":"2024-05-08T21:17:56.420367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:57.351078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424107+0000","last_change":"2024-05-08T21:17:07.259593+0000","last_active":"2024-05-08T21:17:56.424107+0000","last_peered":"2024-05-08T21:17:56.424107+0000","last_clean":"2024-05-08T21:17:56.424107+0000","last_became_active":"2024-05-08T21:17:07.258362+0000","last_became_peered":"2024-05-08T21:17:07.258362+0000","last_unstale":"2024-05-08T21:17:56.424107+0000","last_undegraded":"2024-05-08T21:17:56.424107+0000","last_fullsized":"2024-05-08T21:17:56.424107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:07.877264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'7","reported_seq":109,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:50.448028+0000","last_change":"2024-05-08T21:17:02.731685+0000","last_active":"2024-05-08T21:18:50.448028+0000","last_peered":"2024-05-08T21:18:50.448028+0000","last_clean":"2024-05-08T21:18:50.448028+0000","last_became_active":"2024-05-08T21:17:02.731293+0000","last_became_peered":"2024-05-08T21:17:02.731293+0000","last_unstale":"2024-05-08T21:18:50.448028+0000","last_undegraded":"2024-05-08T21:18:50.448028+0000","last_fullsized":"2024-05-08T21:18:50.448028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:36.532962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2011,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419974+0000","last_change":"2024-05-08T21:17:03.250090+0000","last_active":"2024-05-08T21:17:56.419974+0000","last_peered":"2024-05-08T21:17:56.419974+0000","last_clean":"2024-05-08T21:17:56.419974+0000","last_became_active":"2024-05-08T21:17:03.249744+0000","last_became_peered":"2024-05-08T21:17:03.249744+0000","last_unstale":"2024-05-08T21:17:56.419974+0000","last_undegraded":"2024-05-08T21:17:56.419974+0000","last_fullsized":"2024-05-08T21:17:56.419974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:43.155040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422926+0000","last_change":"2024-05-08T21:17:07.264101+0000","last_active":"2024-05-08T21:17:56.422926+0000","last_peered":"2024-05-08T21:17:56.422926+0000","last_clean":"2024-05-08T21:17:56.422926+0000","last_became_active":"2024-05-08T21:17:07.263720+0000","last_became_peered":"2024-05-08T21:17:07.263720+0000","last_unstale":"2024-05-08T21:17:56.422926+0000","last_undegraded":"2024-05-08T21:17:56.422926+0000","last_fullsized":"2024-05-08T21:17:56.422926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:16.943044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419108+0000","last_change":"2024-05-08T21:17:02.726763+0000","last_active":"2024-05-08T21:17:56.419108+0000","last_peered":"2024-05-08T21:17:56.419108+0000","last_clean":"2024-05-08T21:17:56.419108+0000","last_became_active":"2024-05-08T21:17:02.726343+0000","last_became_peered":"2024-05-08T21:17:02.726343+0000","last_unstale":"2024-05-08T21:17:56.419108+0000","last_undegraded":"2024-05-08T21:17:56.419108+0000","last_fullsized":"2024-05-08T21:17:56.419108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:03.504667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420530+0000","last_change":"2024-05-08T21:17:03.244468+0000","last_active":"2024-05-08T21:17:56.420530+0000","last_peered":"2024-05-08T21:17:56.420530+0000","last_clean":"2024-05-08T21:17:56.420530+0000","last_became_active":"2024-05-08T21:17:03.244132+0000","last_became_peered":"2024-05-08T21:17:03.244132+0000","last_unstale":"2024-05-08T21:17:56.420530+0000","last_undegraded":"2024-05-08T21:17:56.420530+0000","last_fullsized":"2024-05-08T21:17:56.420530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:17.423657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418261+0000","last_change":"2024-05-08T21:17:07.260981+0000","last_active":"2024-05-08T21:17:56.418261+0000","last_peered":"2024-05-08T21:17:56.418261+0000","last_clean":"2024-05-08T21:17:56.418261+0000","last_became_active":"2024-05-08T21:17:07.260582+0000","last_became_peered":"2024-05-08T21:17:07.260582+0000","last_unstale":"2024-05-08T21:17:56.418261+0000","last_undegraded":"2024-05-08T21:17:56.418261+0000","last_fullsized":"2024-05-08T21:17:56.418261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:25.319411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414755+0000","last_change":"2024-05-08T21:17:02.732531+0000","last_active":"2024-05-08T21:17:56.414755+0000","last_peered":"2024-05-08T21:17:56.414755+0000","last_clean":"2024-05-08T21:17:56.414755+0000","last_became_active":"2024-05-08T21:17:02.732121+0000","last_became_peered":"2024-05-08T21:17:02.732121+0000","last_unstale":"2024-05-08T21:17:56.414755+0000","last_undegraded":"2024-05-08T21:17:56.414755+0000","last_fullsized":"2024-05-08T21:17:56.414755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:25.482458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420635+0000","last_change":"2024-05-08T21:17:03.249676+0000","last_active":"2024-05-08T21:17:56.420635+0000","last_peered":"2024-05-08T21:17:56.420635+0000","last_clean":"2024-05-08T21:17:56.420635+0000","last_became_active":"2024-05-08T21:17:03.249120+0000","last_became_peered":"2024-05-08T21:17:03.249120+0000","last_unstale":"2024-05-08T21:17:56.420635+0000","last_undegraded":"2024-05-08T21:17:56.420635+0000","last_fullsized":"2024-05-08T21:17:56.420635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:58:40.678800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418654+0000","last_change":"2024-05-08T21:17:03.245585+0000","last_active":"2024-05-08T21:17:56.418654+0000","last_peered":"2024-05-08T21:17:56.418654+0000","last_clean":"2024-05-08T21:17:56.418654+0000","last_became_active":"2024-05-08T21:17:03.245274+0000","last_became_peered":"2024-05-08T21:17:03.245274+0000","last_unstale":"2024-05-08T21:17:56.418654+0000","last_undegraded":"2024-05-08T21:17:56.418654+0000","last_fullsized":"2024-05-08T21:17:56.418654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:32.003962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423037+0000","last_change":"2024-05-08T21:17:03.246622+0000","last_active":"2024-05-08T21:17:56.423037+0000","last_peered":"2024-05-08T21:17:56.423037+0000","last_clean":"2024-05-08T21:17:56.423037+0000","last_became_active":"2024-05-08T21:17:03.246309+0000","last_became_peered":"2024-05-08T21:17:03.246309+0000","last_unstale":"2024-05-08T21:17:56.423037+0000","last_undegraded":"2024-05-08T21:17:56.423037+0000","last_fullsized":"2024-05-08T21:17:56.423037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:32.439389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422776+0000","last_change":"2024-05-08T21:17:03.243460+0000","last_active":"2024-05-08T21:17:56.422776+0000","last_peered":"2024-05-08T21:17:56.422776+0000","last_clean":"2024-05-08T21:17:56.422776+0000","last_became_active":"2024-05-08T21:17:03.243097+0000","last_became_peered":"2024-05-08T21:17:03.243097+0000","last_unstale":"2024-05-08T21:17:56.422776+0000","last_undegraded":"2024-05-08T21:17:56.422776+0000","last_fullsized":"2024-05-08T21:17:56.422776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:52:14.492180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423038+0000","last_change":"2024-05-08T21:17:03.245077+0000","last_active":"2024-05-08T21:17:56.423038+0000","last_peered":"2024-05-08T21:17:56.423038+0000","last_clean":"2024-05-08T21:17:56.423038+0000","last_became_active":"2024-05-08T21:17:03.244719+0000","last_became_peered":"2024-05-08T21:17:03.244719+0000","last_unstale":"2024-05-08T21:17:56.423038+0000","last_undegraded":"2024-05-08T21:17:56.423038+0000","last_fullsized":"2024-05-08T21:17:56.423038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.388318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415101+0000","last_change":"2024-05-08T21:17:03.245261+0000","last_active":"2024-05-08T21:17:56.415101+0000","last_peered":"2024-05-08T21:17:56.415101+0000","last_clean":"2024-05-08T21:17:56.415101+0000","last_became_active":"2024-05-08T21:17:03.244892+0000","last_became_peered":"2024-05-08T21:17:03.244892+0000","last_unstale":"2024-05-08T21:17:56.415101+0000","last_undegraded":"2024-05-08T21:17:56.415101+0000","last_fullsized":"2024-05-08T21:17:56.415101+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:21.075634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418251+0000","last_change":"2024-05-08T21:17:03.246250+0000","last_active":"2024-05-08T21:17:56.418251+0000","last_peered":"2024-05-08T21:17:56.418251+0000","last_clean":"2024-05-08T21:17:56.418251+0000","last_became_active":"2024-05-08T21:17:03.245807+0000","last_became_peered":"2024-05-08T21:17:03.245807+0000","last_unstale":"2024-05-08T21:17:56.418251+0000","last_undegraded":"2024-05-08T21:17:56.418251+0000","last_fullsized":"2024-05-08T21:17:56.418251+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:20.618260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423477+0000","last_change":"2024-05-08T21:17:03.254559+0000","last_active":"2024-05-08T21:17:56.423477+0000","last_peered":"2024-05-08T21:17:56.423477+0000","last_clean":"2024-05-08T21:17:56.423477+0000","last_became_active":"2024-05-08T21:17:03.254231+0000","last_became_peered":"2024-05-08T21:17:03.254231+0000","last_unstale":"2024-05-08T21:17:56.423477+0000","last_undegraded":"2024-05-08T21:17:56.423477+0000","last_fullsized":"2024-05-08T21:17:56.423477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:47.996927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418065+0000","last_change":"2024-05-08T21:17:03.242479+0000","last_active":"2024-05-08T21:17:56.418065+0000","last_peered":"2024-05-08T21:17:56.418065+0000","last_clean":"2024-05-08T21:17:56.418065+0000","last_became_active":"2024-05-08T21:17:03.241397+0000","last_became_peered":"2024-05-08T21:17:03.241397+0000","last_unstale":"2024-05-08T21:17:56.418065+0000","last_undegraded":"2024-05-08T21:17:56.418065+0000","last_fullsized":"2024-05-08T21:17:56.418065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:39.029139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419220+0000","last_change":"2024-05-08T21:17:03.244208+0000","last_active":"2024-05-08T21:17:56.419220+0000","last_peered":"2024-05-08T21:17:56.419220+0000","last_clean":"2024-05-08T21:17:56.419220+0000","last_became_active":"2024-05-08T21:17:03.243828+0000","last_became_peered":"2024-05-08T21:17:03.243828+0000","last_unstale":"2024-05-08T21:17:56.419220+0000","last_undegraded":"2024-05-08T21:17:56.419220+0000","last_fullsized":"2024-05-08T21:17:56.419220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:39.936750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418012+0000","last_change":"2024-05-08T21:17:03.240135+0000","last_active":"2024-05-08T21:17:56.418012+0000","last_peered":"2024-05-08T21:17:56.418012+0000","last_clean":"2024-05-08T21:17:56.418012+0000","last_became_active":"2024-05-08T21:17:03.239757+0000","last_became_peered":"2024-05-08T21:17:03.239757+0000","last_unstale":"2024-05-08T21:17:56.418012+0000","last_undegraded":"2024-05-08T21:17:56.418012+0000","last_fullsized":"2024-05-08T21:17:56.418012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:46.066111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424382+0000","last_change":"2024-05-08T21:17:03.256254+0000","last_active":"2024-05-08T21:17:56.424382+0000","last_peered":"2024-05-08T21:17:56.424382+0000","last_clean":"2024-05-08T21:17:56.424382+0000","last_became_active":"2024-05-08T21:17:03.255310+0000","last_became_peered":"2024-05-08T21:17:03.255310+0000","last_unstale":"2024-05-08T21:17:56.424382+0000","last_undegraded":"2024-05-08T21:17:56.424382+0000","last_fullsized":"2024-05-08T21:17:56.424382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:59.571958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412843+0000","last_change":"2024-05-08T21:17:03.241901+0000","last_active":"2024-05-08T21:17:56.412843+0000","last_peered":"2024-05-08T21:17:56.412843+0000","last_clean":"2024-05-08T21:17:56.412843+0000","last_became_active":"2024-05-08T21:17:03.241402+0000","last_became_peered":"2024-05-08T21:17:03.241402+0000","last_unstale":"2024-05-08T21:17:56.412843+0000","last_undegraded":"2024-05-08T21:17:56.412843+0000","last_fullsized":"2024-05-08T21:17:56.412843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:59.202987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417882+0000","last_change":"2024-05-08T21:17:03.243050+0000","last_active":"2024-05-08T21:17:56.417882+0000","last_peered":"2024-05-08T21:17:56.417882+0000","last_clean":"2024-05-08T21:17:56.417882+0000","last_became_active":"2024-05-08T21:17:03.240676+0000","last_became_peered":"2024-05-08T21:17:03.240676+0000","last_unstale":"2024-05-08T21:17:56.417882+0000","last_undegraded":"2024-05-08T21:17:56.417882+0000","last_fullsized":"2024-05-08T21:17:56.417882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:19:53.981559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71392,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":823,"num_read_kb":488,"num_write":251,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1246,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514528,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":484195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76800000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65800000000000003,"15min":0.66300000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83699999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67100000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.498,"15min":0.504},"min":{"1min":0.34699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69499999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.54700000000000004,"15min":0.55500000000000005},"min":{"1min":0.32700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.64000000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":4,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88900000000000001,"5min":1.034,"15min":1.034},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.748,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68100000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68999999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80100000000000005}]},{"osd":6,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.625,"15min":0.63900000000000001},"min":{"1min":0.43099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.82899999999999996,"5min":1.085,"15min":1.085},"last":0.78200000000000003}]},{"osd":7,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.56000000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92400000000000004,"5min":1.228,"15min":1.228},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67700000000000005,"15min":0.67800000000000005},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79900000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77000000000000002}]},{"osd":8,"last update":"Wed May 8 21:18:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.84699999999999998}]},{"osd":9,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77300000000000002,"15min":0.77200000000000002},"min":{"1min":0.66100000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93999999999999995,"5min":1.113,"15min":1.113},"last":0.88},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.80700000000000005},"min":{"1min":0.65600000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.95499999999999996,"5min":1.27,"15min":1.27},"last":0.88900000000000001}]},{"osd":10,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.747,"15min":0.75600000000000001},"min":{"1min":0.57099999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.84199999999999997,"5min":1.419,"15min":1.419},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.64500000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90400000000000003,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.83099999999999996}]}]},{"osd":10,"up_from":69,"seq":296352743459,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36848,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708304,"statfs":{"total":95995035648,"available":95957303296,"internally_reserved":0,"allocated":1679360,"data_stored":1133072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.71299999999999997}]},{"osd":1,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42899999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65100000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46300000000000002,"15min":0.46200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68100000000000005,"5min":0.748,"15min":0.748},"last":0.42399999999999999}]},{"osd":2,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.373,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58399999999999996,"15min":0.58299999999999996},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81599999999999995,"5min":0.88,"15min":0.88},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.626,"15min":0.627},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61299999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85599999999999998,"5min":1.248,"15min":1.248},"last":0.46100000000000002}]},{"osd":5,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.47599999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97199999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.45100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91400000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57999999999999996}]},{"osd":6,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95999999999999996,"5min":1.014,"15min":1.014},"last":0.627},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69499999999999995,"15min":0.69699999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94699999999999995,"5min":1.048,"15min":1.048},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":8,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46300000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.745,"15min":0.751},"min":{"1min":0.47799999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90700000000000003,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.753,"15min":0.755},"min":{"1min":0.56799999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0229999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77300000000000002,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.93000000000000005,"5min":1.419,"15min":1.419},"last":0.73299999999999998}]},{"osd":11,"last update":"Wed May 8 21:18:33 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.81999999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.749,"15min":0.753},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.876,"5min":0.995,"15min":0.995},"last":0.74399999999999999}]}]},{"osd":9,"up_from":64,"seq":274877906982,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":487380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.48099999999999998,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65000000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.5,"15min":0.50800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63700000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.34799999999999998}]},{"osd":1,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.84399999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81100000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76500000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63400000000000001,"15min":0.64300000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":1.048,"15min":1.048},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.496,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79200000000000004}]},{"osd":5,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.77800000000000002,"15min":0.79500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92300000000000004,"5min":1.234,"15min":1.234},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.056,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.875},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.47099999999999997}]},{"osd":7,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.81599999999999995,"15min":0.83099999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96099999999999997,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.84199999999999997,"15min":0.86399999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.67500000000000004}]},{"osd":8,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.80100000000000005,"15min":0.82299999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93600000000000005,"5min":1.101,"15min":1.101},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.88600000000000001,"15min":0.90700000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.004,"5min":1.589,"15min":1.589},"last":0.82599999999999996}]},{"osd":10,"last update":"Wed May 8 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.84399999999999997,"15min":0.85599999999999998},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0409999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.745},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.875,"15min":0.88700000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1140000000000001,"5min":1.153,"15min":1.153},"last":0.625}]},{"osd":11,"last update":"Wed May 8 21:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.77100000000000002}]}]},{"osd":8,"up_from":56,"seq":240518168619,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":482659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.42699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.501}]},{"osd":1,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59699999999999998,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.52800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63900000000000001}]},{"osd":2,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.47899999999999998,"15min":0.47199999999999998},"min":{"1min":0.35999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.75600000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.47899999999999998,"15min":0.45000000000000001},"min":{"1min":0.375,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47999999999999998}]},{"osd":3,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57099999999999995,"15min":0.55800000000000005},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89200000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.62,"15min":0.62},"min":{"1min":0.501,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84599999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59799999999999998}]},{"osd":4,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.49399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85899999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.69099999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54400000000000004,"15min":0.51400000000000001},"min":{"1min":0.41299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.46999999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66400000000000003,"15min":0.64900000000000002},"min":{"1min":0.51700000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.70399999999999996,"15min":0.67900000000000005},"min":{"1min":0.66700000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.67500000000000004,"15min":0.64300000000000002},"min":{"1min":0.53600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.71699999999999997,"15min":0.70499999999999996},"min":{"1min":0.55800000000000005,"5min":0.373,"15min":0.373},"max":{"1min":0.95799999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76400000000000001}]},{"osd":9,"last update":"Wed May 8 21:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70199999999999996,"15min":0.68899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.745,"15min":0.73499999999999999},"min":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.96999999999999997,"5min":1.04,"15min":1.04},"last":0.79300000000000004}]},{"osd":10,"last update":"Wed May 8 21:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.60099999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.61199999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94899999999999995,"5min":1.008,"15min":1.008},"last":0.77200000000000002}]},{"osd":11,"last update":"Wed May 8 21:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.752},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.83299999999999996}]}]},{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1080508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41099999999999998,"15min":0.40000000000000002},"min":{"1min":0.32400000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.54400000000000004,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.40200000000000002},"min":{"1min":0.317,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61699999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.496}]},{"osd":2,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58799999999999997,"15min":0.57299999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56499999999999995,"15min":0.54600000000000004},"min":{"1min":0.46999999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.89900000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52800000000000002}]},{"osd":3,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71499999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72299999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.36599999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.379,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61799999999999999,"5min":0.745,"15min":0.745},"last":0.438},{"interface":"front","average":{"1min":0.499,"5min":0.48799999999999999,"15min":0.49099999999999999},"min":{"1min":0.35599999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71099999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.38500000000000001}]},{"osd":5,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.45000000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.76500000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.622,"15min":0.60099999999999998},"min":{"1min":0.51700000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.873,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.65100000000000002,"15min":0.65600000000000003},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83999999999999997,"5min":0.87,"15min":0.87},"last":0.41799999999999998}]},{"osd":8,"last update":"Wed May 8 21:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.66900000000000004,"15min":0.65500000000000003},"min":{"1min":0.54400000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88500000000000001,"5min":1,"15min":1},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.67200000000000004,"15min":0.65200000000000002},"min":{"1min":0.57399999999999995,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95399999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67200000000000004}]},{"osd":9,"last update":"Wed May 8 21:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67200000000000004,"15min":0.67600000000000005},"min":{"1min":0.47799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.81999999999999995,"5min":1.008,"15min":1.008},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.86499999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65600000000000003}]},{"osd":10,"last update":"Wed May 8 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.57199999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.66300000000000003,"15min":0.65000000000000002},"min":{"1min":0.625,"5min":0.437,"15min":0.437},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.70699999999999996}]},{"osd":11,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.622}]}]},{"osd":6,"up_from":44,"seq":188978561074,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":479236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.438},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.46200000000000002}]},{"osd":1,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.502,"5min":0.434,"15min":0.434},"max":{"1min":1,"5min":1,"15min":1},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.50900000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.67800000000000005}]},{"osd":3,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.56200000000000006}]},{"osd":4,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.45700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.82799999999999996}]},{"osd":5,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.56000000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.72399999999999998,"15min":0.71199999999999997},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.70099999999999996}]},{"osd":7,"last update":"Wed May 8 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.76400000000000001,"15min":0.76100000000000001},"min":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.83799999999999997}]},{"osd":8,"last update":"Wed May 8 21:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.59599999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.66400000000000003}]},{"osd":9,"last update":"Wed May 8 21:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.80400000000000005,"15min":0.79300000000000004},"min":{"1min":0.54800000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.745},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.76700000000000002},"min":{"1min":0.625,"5min":0.498,"15min":0.498},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.755}]},{"osd":10,"last update":"Wed May 8 21:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.76500000000000001,"15min":0.754},"min":{"1min":0.65000000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.81299999999999994,"15min":0.79900000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.89500000000000002}]},{"osd":11,"last update":"Wed May 8 21:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.85299999999999998}]}]},{"osd":1,"up_from":12,"seq":51539607619,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":543723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56699999999999995,"15min":0.53300000000000003},"min":{"1min":0.44700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.745,"5min":1.548,"15min":1.548},"last":0.84999999999999998}]},{"osd":2,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.54000000000000004,"15min":0.50700000000000001},"min":{"1min":0.41699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.52900000000000003,"15min":0.496},"min":{"1min":0.40600000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80900000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.77300000000000002}]},{"osd":3,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.51100000000000001},"min":{"1min":0.42799999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.57999999999999996,"15min":0.52500000000000002},"min":{"1min":0.432,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85999999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.73399999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45000000000000001,"15min":0.41599999999999998},"min":{"1min":0.35799999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.63200000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.42499999999999999},"min":{"1min":0.41899999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.66300000000000003,"5min":0.753,"15min":0.753},"last":0.60599999999999998}]},{"osd":5,"last update":"Wed May 8 21:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.877,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.54400000000000004},"min":{"1min":0.495,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88300000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.82299999999999995}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64300000000000002,"15min":0.61199999999999999},"min":{"1min":0.502,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89100000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.92000000000000004}]},{"osd":7,"last update":"Wed May 8 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.40899999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84399999999999997,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.55300000000000005,"15min":0.51600000000000001},"min":{"1min":0.443,"5min":0.308,"15min":0.308},"max":{"1min":0.94499999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78900000000000003}]},{"osd":8,"last update":"Wed May 8 21:18:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.67100000000000004,"15min":0.63900000000000001},"min":{"1min":0.48499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.99099999999999999,"5min":1.032,"15min":1.032},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72099999999999997,"15min":0.70299999999999996},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.607,"15min":1.607},"last":0.80400000000000005}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70099999999999996,"15min":0.69099999999999995},"min":{"1min":0.54300000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73299999999999998,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.877}]},{"osd":10,"last update":"Wed May 8 21:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.752,"15min":0.747},"min":{"1min":0.54800000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95499999999999996,"5min":1.004,"15min":1.004},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.71799999999999997,"15min":0.70799999999999996},"min":{"1min":0.63300000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.89000000000000001}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.90900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":1011712,"data_stored":480589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 21:18:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.41599999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.84299999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.627},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.46400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65900000000000003}]},{"osd":2,"last update":"Wed May 8 21:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.748,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.80200000000000005}]},{"osd":3,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68999999999999995}]},{"osd":4,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85199999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.627,"15min":0.58999999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.82699999999999996}]},{"osd":5,"last update":"Wed May 8 21:17:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.621,"15min":0.60799999999999998},"min":{"1min":0.45400000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86399999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60699999999999998,"15min":0.60199999999999998},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.76400000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.84799999999999998}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.42999999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89700000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86499999999999999}]},{"osd":7,"last update":"Wed May 8 21:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":1.0529999999999999}]},{"osd":8,"last update":"Wed May 8 21:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.54000000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.63100000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.91000000000000003}]},{"osd":9,"last update":"Wed May 8 21:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77400000000000002},"min":{"1min":0.65100000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97799999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":1.042}]},{"osd":10,"last update":"Wed May 8 21:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81799999999999995,"15min":0.81299999999999994},"min":{"1min":0.68500000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.86199999999999999,"15min":0.85199999999999998},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":1.133}]},{"osd":11,"last update":"Wed May 8 21:18:34 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.71899999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.71599999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":1.0109999999999999}]}]},{"osd":2,"up_from":17,"seq":73014444097,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1069056,"data_stored":543042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.56799999999999995,"15min":0.52100000000000002},"min":{"1min":0.32600000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":6.7270000000000003,"5min":6.7270000000000003,"15min":6.7270000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.51200000000000001},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47999999999999998}]},{"osd":1,"last update":"Wed May 8 21:17:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.44500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.42899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70599999999999996}]},{"osd":3,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.437,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.45900000000000002,"15min":0.41799999999999998},"min":{"1min":0.36499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.45900000000000002,"15min":0.42299999999999999},"min":{"1min":0.30299999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.753,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58399999999999996}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.55400000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.212,"5min":0.71199999999999997,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":7.7779999999999996,"5min":7.7779999999999996,"15min":7.7779999999999996},"last":0.754}]},{"osd":7,"last update":"Wed May 8 21:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.83799999999999997}]},{"osd":8,"last update":"Wed May 8 21:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.39000000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95199999999999996,"5min":0.995,"15min":0.995},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59399999999999997,"15min":0.56799999999999995},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71999999999999997}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.67700000000000005,"15min":0.65800000000000003},"min":{"1min":0.66800000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.56799999999999995,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.82399999999999995}]},{"osd":10,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67800000000000005,"15min":0.66300000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.753,"15min":0.73499999999999999},"min":{"1min":0.629,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.89200000000000002}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.871},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.81299999999999994}]}]},{"osd":3,"up_from":24,"seq":103079215165,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":999424,"data_stored":478496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.40400000000000003,"15min":0.36799999999999999},"min":{"1min":0.29899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42099999999999999,"15min":0.39100000000000001},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76500000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54200000000000004,"15min":0.51600000000000001},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80700000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.52100000000000002},"min":{"1min":0.42299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.628}]},{"osd":4,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.54500000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.56599999999999995},"min":{"1min":0.48799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed May 8 21:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59999999999999998,"15min":0.56699999999999995},"min":{"1min":0.48699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78700000000000003,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96799999999999997,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Wed May 8 21:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.504,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.73699999999999999}]},{"osd":7,"last update":"Wed May 8 21:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.53300000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.58999999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.78200000000000003}]},{"osd":8,"last update":"Wed May 8 21:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79800000000000004}]},{"osd":9,"last update":"Wed May 8 21:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.85599999999999998}]},{"osd":10,"last update":"Wed May 8 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77300000000000002,"15min":0.77000000000000002},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80900000000000005,"15min":0.81100000000000005},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.872}]},{"osd":11,"last update":"Wed May 8 21:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.91600000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986232,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":484380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.36599999999999999},"min":{"1min":0.32100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.40200000000000002,"15min":0.39300000000000002},"min":{"1min":0.34599999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59399999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.45100000000000001}]},{"osd":2,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82599999999999996,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.53600000000000003},"min":{"1min":0.48099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.60299999999999998}]},{"osd":3,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56999999999999995,"15min":0.53800000000000003},"min":{"1min":0.47599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73899999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.52500000000000002},"min":{"1min":0.41499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.48299999999999998}]},{"osd":5,"last update":"Wed May 8 21:17:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.376,"5min":0.371,"15min":0.371},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.57599999999999996,"15min":0.53800000000000003},"min":{"1min":0.54600000000000004,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.75700000000000001}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59999999999999998,"15min":0.57999999999999996},"min":{"1min":0.51200000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82399999999999995,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Wed May 8 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52200000000000002,"15min":0.48799999999999999},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.81599999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56100000000000005,"15min":0.52100000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.91100000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.63300000000000001}]},{"osd":8,"last update":"Wed May 8 21:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66400000000000003,"15min":0.64400000000000002},"min":{"1min":0.59499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.83099999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.746},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.69999999999999996},"min":{"1min":0.59499999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.82499999999999996}]},{"osd":9,"last update":"Wed May 8 21:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.747,"15min":0.747},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.876,"5min":1.347,"15min":1.347},"last":0.77200000000000002}]},{"osd":10,"last update":"Wed May 8 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69399999999999995,"15min":0.69699999999999995},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70599999999999996,"15min":0.71499999999999997},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":1.323,"15min":1.323},"last":0.67800000000000005}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.80700000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36784,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708368,"statfs":{"total":95995035648,"available":95957368832,"internally_reserved":0,"allocated":1613824,"data_stored":1075798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.38100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77100000000000002,"5min":0.877,"15min":0.877},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.374,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.442}]},{"osd":1,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.434,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73699999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.751,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.72099999999999997}]},{"osd":2,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40699999999999997,"15min":0.40000000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61799999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.40699999999999997,"15min":0.40799999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61599999999999999,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.45700000000000002}]},{"osd":3,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.54800000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89800000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.373},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.42099999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.45300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81699999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed May 8 21:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.75700000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.623,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93300000000000005,"5min":1.01,"15min":1.01},"last":0.502}]},{"osd":7,"last update":"Wed May 8 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.623,"15min":0.623},"min":{"1min":0.42999999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82699999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88900000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66700000000000004}]},{"osd":8,"last update":"Wed May 8 21:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73499999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48699999999999999}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.55100000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88900000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52200000000000002}]},{"osd":10,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90700000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.51900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T21:18:54.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T21:18:54.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.300+0000 7f956be66700 1 -- 172.21.15.33:0/719117257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9564100e80 msgr2=0x7f95641012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:54.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.300+0000 7f956be66700 1 --2- 172.21.15.33:0/719117257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9564100e80 0x7f95641012a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9560009990 tx=0x7f9560009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.300+0000 7f956be66700 1 -- 172.21.15.33:0/719117257 shutdown_connections 2024-05-08T21:18:54.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.300+0000 7f956be66700 1 --2- 172.21.15.33:0/719117257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9564102990 0x7f95641092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.300+0000 7f956be66700 1 --2- 172.21.15.33:0/719117257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9564100e80 0x7f95641012a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.300+0000 7f956be66700 1 --2- 172.21.15.33:0/719117257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9564101fd0 0x7f9564102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.300+0000 7f956be66700 1 -- 172.21.15.33:0/719117257 >> 172.21.15.33:0/719117257 conn(0x7f95640fc410 msgr2=0x7f95640fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:54.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.300+0000 7f956be66700 1 -- 172.21.15.33:0/719117257 shutdown_connections 2024-05-08T21:18:54.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.300+0000 7f956be66700 1 -- 172.21.15.33:0/719117257 wait complete. 2024-05-08T21:18:54.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.301+0000 7f956be66700 1 Processor -- start 2024-05-08T21:18:54.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.301+0000 7f956be66700 1 -- start start 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.301+0000 7f956be66700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9564100e80 0x7f9564074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.301+0000 7f956be66700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9564101fd0 0x7f9564073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.301+0000 7f956be66700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9564102990 0x7f95640738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.301+0000 7f956be66700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9564073fd0 con 0x7f9564102990 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.301+0000 7f956be66700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9564074140 con 0x7f9564101fd0 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.301+0000 7f956be66700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95641a0dd0 con 0x7f9564100e80 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f956a403700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9564102990 0x7f95640738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f956a403700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9564102990 0x7f95640738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54186/0 (socket says 172.21.15.33:54186) 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f956a403700 1 -- 172.21.15.33:0/3701125194 learned_addr learned my addr 172.21.15.33:0/3701125194 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f9569401700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9564101fd0 0x7f9564073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f9569c02700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9564100e80 0x7f9564074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f956a403700 1 -- 172.21.15.33:0/3701125194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9564100e80 msgr2=0x7f9564074cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f956a403700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9564100e80 0x7f9564074cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f956a403700 1 -- 172.21.15.33:0/3701125194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9564101fd0 msgr2=0x7f9564073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f956a403700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9564101fd0 0x7f9564073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f956a403700 1 -- 172.21.15.33:0/3701125194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600095f0 con 0x7f9564102990 2024-05-08T21:18:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f956a403700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9564102990 0x7f95640738d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f955c00ef30 tx=0x7f955c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f955affd700 1 -- 172.21.15.33:0/3701125194 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c018070 con 0x7f9564102990 2024-05-08T21:18:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f955affd700 1 -- 172.21.15.33:0/3701125194 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f955c00f040 con 0x7f9564102990 2024-05-08T21:18:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f956be66700 1 -- 172.21.15.33:0/3701125194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95641a0fd0 con 0x7f9564102990 2024-05-08T21:18:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.302+0000 7f955affd700 1 -- 172.21.15.33:0/3701125194 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c00cbc0 con 0x7f9564102990 2024-05-08T21:18:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.303+0000 7f956be66700 1 -- 172.21.15.33:0/3701125194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95641a1400 con 0x7f9564102990 2024-05-08T21:18:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.304+0000 7f956be66700 1 -- 172.21.15.33:0/3701125194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9564108430 con 0x7f9564102990 2024-05-08T21:18:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.305+0000 7f955affd700 1 -- 172.21.15.33:0/3701125194 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f955c007500 con 0x7f9564102990 2024-05-08T21:18:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.305+0000 7f955affd700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95500a11b0 0x7f95500a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.305+0000 7f955affd700 1 -- 172.21.15.33:0/3701125194 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f955c0cc830 con 0x7f9564102990 2024-05-08T21:18:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.305+0000 7f9569c02700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95500a11b0 0x7f95500a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.306+0000 7f9569c02700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95500a11b0 0x7f95500a3670 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9560006010 tx=0x7f956000b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.311+0000 7f955affd700 1 -- 172.21.15.33:0/3701125194 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f955c095be0 con 0x7f9564102990 2024-05-08T21:18:54.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.533+0000 7f956be66700 1 -- 172.21.15.33:0/3701125194 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f956402ceb0 con 0x7f95500a11b0 2024-05-08T21:18:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.546+0000 7f955affd700 1 -- 172.21.15.33:0/3701125194 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427361 (secure 0 0 0) 0x7f956402ceb0 con 0x7f95500a11b0 2024-05-08T21:18:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:54.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.550+0000 7f956be66700 1 -- 172.21.15.33:0/3701125194 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95500a11b0 msgr2=0x7f95500a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:54.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.550+0000 7f956be66700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95500a11b0 0x7f95500a3670 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9560006010 tx=0x7f956000b5d0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.550+0000 7f956be66700 1 -- 172.21.15.33:0/3701125194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9564102990 msgr2=0x7f95640738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:54.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.550+0000 7f956be66700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9564102990 0x7f95640738d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f955c00ef30 tx=0x7f955c00c3a0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.550+0000 7f956be66700 1 -- 172.21.15.33:0/3701125194 shutdown_connections 2024-05-08T21:18:54.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.550+0000 7f956be66700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9564102990 0x7f95640738d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.550+0000 7f956be66700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9564100e80 0x7f9564074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.550+0000 7f956be66700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95500a11b0 0x7f95500a3670 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.550+0000 7f956be66700 1 --2- 172.21.15.33:0/3701125194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9564101fd0 0x7f9564073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.550+0000 7f956be66700 1 -- 172.21.15.33:0/3701125194 >> 172.21.15.33:0/3701125194 conn(0x7f95640fc410 msgr2=0x7f9564105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.550+0000 7f956be66700 1 -- 172.21.15.33:0/3701125194 shutdown_connections 2024-05-08T21:18:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.550+0000 7f956be66700 1 -- 172.21.15.33:0/3701125194 wait complete. 2024-05-08T21:18:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T21:18:54.567 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-05-08T21:18:54.145246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661994,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":959,"num_read_kb":602,"num_write":434,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1181982,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":359,"ondisk_log_size":359,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374768,"kb_used_data":13696,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124567056,"statfs":{"total":1151940427776,"available":1151556665344,"internally_reserved":0,"allocated":14024704,"data_stored":7753574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2480,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":60,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003270"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418982+0000","last_change":"2024-05-08T21:17:03.243903+0000","last_active":"2024-05-08T21:17:56.418982+0000","last_peered":"2024-05-08T21:17:56.418982+0000","last_clean":"2024-05-08T21:17:56.418982+0000","last_became_active":"2024-05-08T21:17:03.240821+0000","last_became_peered":"2024-05-08T21:17:03.240821+0000","last_unstale":"2024-05-08T21:17:56.418982+0000","last_undegraded":"2024-05-08T21:17:56.418982+0000","last_fullsized":"2024-05-08T21:17:56.418982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:01.878614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423327+0000","last_change":"2024-05-08T21:17:03.243119+0000","last_active":"2024-05-08T21:17:56.423327+0000","last_peered":"2024-05-08T21:17:56.423327+0000","last_clean":"2024-05-08T21:17:56.423327+0000","last_became_active":"2024-05-08T21:17:03.242772+0000","last_became_peered":"2024-05-08T21:17:03.242772+0000","last_unstale":"2024-05-08T21:17:56.423327+0000","last_undegraded":"2024-05-08T21:17:56.423327+0000","last_fullsized":"2024-05-08T21:17:56.423327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:20.550646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423287+0000","last_change":"2024-05-08T21:17:03.237843+0000","last_active":"2024-05-08T21:17:56.423287+0000","last_peered":"2024-05-08T21:17:56.423287+0000","last_clean":"2024-05-08T21:17:56.423287+0000","last_became_active":"2024-05-08T21:17:03.237567+0000","last_became_peered":"2024-05-08T21:17:03.237567+0000","last_unstale":"2024-05-08T21:17:56.423287+0000","last_undegraded":"2024-05-08T21:17:56.423287+0000","last_fullsized":"2024-05-08T21:17:56.423287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:23.856461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.430448+0000","last_change":"2024-05-08T21:17:57.430448+0000","last_active":"2024-05-08T21:17:57.430448+0000","last_peered":"2024-05-08T21:17:57.430448+0000","last_clean":"2024-05-08T21:17:57.430448+0000","last_became_active":"2024-05-08T21:17:57.430195+0000","last_became_peered":"2024-05-08T21:17:57.430195+0000","last_unstale":"2024-05-08T21:17:57.430448+0000","last_undegraded":"2024-05-08T21:17:57.430448+0000","last_fullsized":"2024-05-08T21:17:57.430448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:41.536610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416067+0000","last_change":"2024-05-08T21:17:03.252246+0000","last_active":"2024-05-08T21:17:56.416067+0000","last_peered":"2024-05-08T21:17:56.416067+0000","last_clean":"2024-05-08T21:17:56.416067+0000","last_became_active":"2024-05-08T21:17:03.251898+0000","last_became_peered":"2024-05-08T21:17:03.251898+0000","last_unstale":"2024-05-08T21:17:56.416067+0000","last_undegraded":"2024-05-08T21:17:56.416067+0000","last_fullsized":"2024-05-08T21:17:56.416067+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:08.996033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417704+0000","last_change":"2024-05-08T21:17:03.238422+0000","last_active":"2024-05-08T21:17:56.417704+0000","last_peered":"2024-05-08T21:17:56.417704+0000","last_clean":"2024-05-08T21:17:56.417704+0000","last_became_active":"2024-05-08T21:17:03.238096+0000","last_became_peered":"2024-05-08T21:17:03.238096+0000","last_unstale":"2024-05-08T21:17:56.417704+0000","last_undegraded":"2024-05-08T21:17:56.417704+0000","last_fullsized":"2024-05-08T21:17:56.417704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:54.635089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419866+0000","last_change":"2024-05-08T21:17:03.244315+0000","last_active":"2024-05-08T21:17:56.419866+0000","last_peered":"2024-05-08T21:17:56.419866+0000","last_clean":"2024-05-08T21:17:56.419866+0000","last_became_active":"2024-05-08T21:17:03.243939+0000","last_became_peered":"2024-05-08T21:17:03.243939+0000","last_unstale":"2024-05-08T21:17:56.419866+0000","last_undegraded":"2024-05-08T21:17:56.419866+0000","last_fullsized":"2024-05-08T21:17:56.419866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:24.371495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415520+0000","last_change":"2024-05-08T21:17:03.240144+0000","last_active":"2024-05-08T21:17:56.415520+0000","last_peered":"2024-05-08T21:17:56.415520+0000","last_clean":"2024-05-08T21:17:56.415520+0000","last_became_active":"2024-05-08T21:17:03.239493+0000","last_became_peered":"2024-05-08T21:17:03.239493+0000","last_unstale":"2024-05-08T21:17:56.415520+0000","last_undegraded":"2024-05-08T21:17:56.415520+0000","last_fullsized":"2024-05-08T21:17:56.415520+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:21.374357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424317+0000","last_change":"2024-05-08T21:17:03.256706+0000","last_active":"2024-05-08T21:17:56.424317+0000","last_peered":"2024-05-08T21:17:56.424317+0000","last_clean":"2024-05-08T21:17:56.424317+0000","last_became_active":"2024-05-08T21:17:03.255345+0000","last_became_peered":"2024-05-08T21:17:03.255345+0000","last_unstale":"2024-05-08T21:17:56.424317+0000","last_undegraded":"2024-05-08T21:17:56.424317+0000","last_fullsized":"2024-05-08T21:17:56.424317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:09:50.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.430334+0000","last_change":"2024-05-08T21:17:57.430334+0000","last_active":"2024-05-08T21:17:57.430334+0000","last_peered":"2024-05-08T21:17:57.430334+0000","last_clean":"2024-05-08T21:17:57.430334+0000","last_became_active":"2024-05-08T21:17:57.429924+0000","last_became_peered":"2024-05-08T21:17:57.429924+0000","last_unstale":"2024-05-08T21:17:57.430334+0000","last_undegraded":"2024-05-08T21:17:57.430334+0000","last_fullsized":"2024-05-08T21:17:57.430334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.901755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419810+0000","last_change":"2024-05-08T21:17:03.253996+0000","last_active":"2024-05-08T21:17:56.419810+0000","last_peered":"2024-05-08T21:17:56.419810+0000","last_clean":"2024-05-08T21:17:56.419810+0000","last_became_active":"2024-05-08T21:17:03.253686+0000","last_became_peered":"2024-05-08T21:17:03.253686+0000","last_unstale":"2024-05-08T21:17:56.419810+0000","last_undegraded":"2024-05-08T21:17:56.419810+0000","last_fullsized":"2024-05-08T21:17:56.419810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:17.995692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423092+0000","last_change":"2024-05-08T21:17:03.254357+0000","last_active":"2024-05-08T21:17:56.423092+0000","last_peered":"2024-05-08T21:17:56.423092+0000","last_clean":"2024-05-08T21:17:56.423092+0000","last_became_active":"2024-05-08T21:17:03.254013+0000","last_became_peered":"2024-05-08T21:17:03.254013+0000","last_unstale":"2024-05-08T21:17:56.423092+0000","last_undegraded":"2024-05-08T21:17:56.423092+0000","last_fullsized":"2024-05-08T21:17:56.423092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:21.480612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.413995+0000","last_change":"2024-05-08T21:17:03.244190+0000","last_active":"2024-05-08T21:17:56.413995+0000","last_peered":"2024-05-08T21:17:56.413995+0000","last_clean":"2024-05-08T21:17:56.413995+0000","last_became_active":"2024-05-08T21:17:03.243989+0000","last_became_peered":"2024-05-08T21:17:03.243989+0000","last_unstale":"2024-05-08T21:17:56.413995+0000","last_undegraded":"2024-05-08T21:17:56.413995+0000","last_fullsized":"2024-05-08T21:17:56.413995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:45.059343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422938+0000","last_change":"2024-05-08T21:17:03.242996+0000","last_active":"2024-05-08T21:17:56.422938+0000","last_peered":"2024-05-08T21:17:56.422938+0000","last_clean":"2024-05-08T21:17:56.422938+0000","last_became_active":"2024-05-08T21:17:03.242648+0000","last_became_peered":"2024-05-08T21:17:03.242648+0000","last_unstale":"2024-05-08T21:17:56.422938+0000","last_undegraded":"2024-05-08T21:17:56.422938+0000","last_fullsized":"2024-05-08T21:17:56.422938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:47:19.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422908+0000","last_change":"2024-05-08T21:17:03.244970+0000","last_active":"2024-05-08T21:17:56.422908+0000","last_peered":"2024-05-08T21:17:56.422908+0000","last_clean":"2024-05-08T21:17:56.422908+0000","last_became_active":"2024-05-08T21:17:03.244604+0000","last_became_peered":"2024-05-08T21:17:03.244604+0000","last_unstale":"2024-05-08T21:17:56.422908+0000","last_undegraded":"2024-05-08T21:17:56.422908+0000","last_fullsized":"2024-05-08T21:17:56.422908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:32.484155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422727+0000","last_change":"2024-05-08T21:17:03.254628+0000","last_active":"2024-05-08T21:17:56.422727+0000","last_peered":"2024-05-08T21:17:56.422727+0000","last_clean":"2024-05-08T21:17:56.422727+0000","last_became_active":"2024-05-08T21:17:03.254305+0000","last_became_peered":"2024-05-08T21:17:03.254305+0000","last_unstale":"2024-05-08T21:17:56.422727+0000","last_undegraded":"2024-05-08T21:17:56.422727+0000","last_fullsized":"2024-05-08T21:17:56.422727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:52.727821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418732+0000","last_change":"2024-05-08T21:17:03.243406+0000","last_active":"2024-05-08T21:17:56.418732+0000","last_peered":"2024-05-08T21:17:56.418732+0000","last_clean":"2024-05-08T21:17:56.418732+0000","last_became_active":"2024-05-08T21:17:03.240379+0000","last_became_peered":"2024-05-08T21:17:03.240379+0000","last_unstale":"2024-05-08T21:17:56.418732+0000","last_undegraded":"2024-05-08T21:17:56.418732+0000","last_fullsized":"2024-05-08T21:17:56.418732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:49.897459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420035+0000","last_change":"2024-05-08T21:17:03.239664+0000","last_active":"2024-05-08T21:17:56.420035+0000","last_peered":"2024-05-08T21:17:56.420035+0000","last_clean":"2024-05-08T21:17:56.420035+0000","last_became_active":"2024-05-08T21:17:03.239347+0000","last_became_peered":"2024-05-08T21:17:03.239347+0000","last_unstale":"2024-05-08T21:17:56.420035+0000","last_undegraded":"2024-05-08T21:17:56.420035+0000","last_fullsized":"2024-05-08T21:17:56.420035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:06:37.557835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423543+0000","last_change":"2024-05-08T21:17:03.249424+0000","last_active":"2024-05-08T21:17:56.423543+0000","last_peered":"2024-05-08T21:17:56.423543+0000","last_clean":"2024-05-08T21:17:56.423543+0000","last_became_active":"2024-05-08T21:17:03.248971+0000","last_became_peered":"2024-05-08T21:17:03.248971+0000","last_unstale":"2024-05-08T21:17:56.423543+0000","last_undegraded":"2024-05-08T21:17:56.423543+0000","last_fullsized":"2024-05-08T21:17:56.423543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:08.096762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422849+0000","last_change":"2024-05-08T21:17:07.272414+0000","last_active":"2024-05-08T21:17:56.422849+0000","last_peered":"2024-05-08T21:17:56.422849+0000","last_clean":"2024-05-08T21:17:56.422849+0000","last_became_active":"2024-05-08T21:17:07.271660+0000","last_became_peered":"2024-05-08T21:17:07.271660+0000","last_unstale":"2024-05-08T21:17:56.422849+0000","last_undegraded":"2024-05-08T21:17:56.422849+0000","last_fullsized":"2024-05-08T21:17:56.422849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:15.697910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":133,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:47.545496+0000","last_change":"2024-05-08T21:17:57.483017+0000","last_active":"2024-05-08T21:18:47.545496+0000","last_peered":"2024-05-08T21:18:47.545496+0000","last_clean":"2024-05-08T21:18:47.545496+0000","last_became_active":"2024-05-08T21:17:57.473420+0000","last_became_peered":"2024-05-08T21:17:57.473420+0000","last_unstale":"2024-05-08T21:18:47.545496+0000","last_undegraded":"2024-05-08T21:18:47.545496+0000","last_fullsized":"2024-05-08T21:18:47.545496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:42.727117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417325+0000","last_change":"2024-05-08T21:17:03.240780+0000","last_active":"2024-05-08T21:17:56.417325+0000","last_peered":"2024-05-08T21:17:56.417325+0000","last_clean":"2024-05-08T21:17:56.417325+0000","last_became_active":"2024-05-08T21:17:03.240472+0000","last_became_peered":"2024-05-08T21:17:03.240472+0000","last_unstale":"2024-05-08T21:17:56.417325+0000","last_undegraded":"2024-05-08T21:17:56.417325+0000","last_fullsized":"2024-05-08T21:17:56.417325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.089760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.421952+0000","last_change":"2024-05-08T21:17:07.272308+0000","last_active":"2024-05-08T21:17:56.421952+0000","last_peered":"2024-05-08T21:17:56.421952+0000","last_clean":"2024-05-08T21:17:56.421952+0000","last_became_active":"2024-05-08T21:17:07.271869+0000","last_became_peered":"2024-05-08T21:17:07.271869+0000","last_unstale":"2024-05-08T21:17:56.421952+0000","last_undegraded":"2024-05-08T21:17:56.421952+0000","last_fullsized":"2024-05-08T21:17:56.421952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:31.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417727+0000","last_change":"2024-05-08T21:17:02.736446+0000","last_active":"2024-05-08T21:17:56.417727+0000","last_peered":"2024-05-08T21:17:56.417727+0000","last_clean":"2024-05-08T21:17:56.417727+0000","last_became_active":"2024-05-08T21:17:02.736187+0000","last_became_peered":"2024-05-08T21:17:02.736187+0000","last_unstale":"2024-05-08T21:17:56.417727+0000","last_undegraded":"2024-05-08T21:17:56.417727+0000","last_fullsized":"2024-05-08T21:17:56.417727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:29.302558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423206+0000","last_change":"2024-05-08T21:17:03.251896+0000","last_active":"2024-05-08T21:17:56.423206+0000","last_peered":"2024-05-08T21:17:56.423206+0000","last_clean":"2024-05-08T21:17:56.423206+0000","last_became_active":"2024-05-08T21:17:03.251495+0000","last_became_peered":"2024-05-08T21:17:03.251495+0000","last_unstale":"2024-05-08T21:17:56.423206+0000","last_undegraded":"2024-05-08T21:17:56.423206+0000","last_fullsized":"2024-05-08T21:17:56.423206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:50.774848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422223+0000","last_change":"2024-05-08T21:17:07.272150+0000","last_active":"2024-05-08T21:17:56.422223+0000","last_peered":"2024-05-08T21:17:56.422223+0000","last_clean":"2024-05-08T21:17:56.422223+0000","last_became_active":"2024-05-08T21:17:07.271727+0000","last_became_peered":"2024-05-08T21:17:07.271727+0000","last_unstale":"2024-05-08T21:17:56.422223+0000","last_undegraded":"2024-05-08T21:17:56.422223+0000","last_fullsized":"2024-05-08T21:17:56.422223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.058277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416860+0000","last_change":"2024-05-08T21:17:02.733611+0000","last_active":"2024-05-08T21:17:56.416860+0000","last_peered":"2024-05-08T21:17:56.416860+0000","last_clean":"2024-05-08T21:17:56.416860+0000","last_became_active":"2024-05-08T21:17:02.733306+0000","last_became_peered":"2024-05-08T21:17:02.733306+0000","last_unstale":"2024-05-08T21:17:56.416860+0000","last_undegraded":"2024-05-08T21:17:56.416860+0000","last_fullsized":"2024-05-08T21:17:56.416860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:33.304753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423294+0000","last_change":"2024-05-08T21:17:03.248480+0000","last_active":"2024-05-08T21:17:56.423294+0000","last_peered":"2024-05-08T21:17:56.423294+0000","last_clean":"2024-05-08T21:17:56.423294+0000","last_became_active":"2024-05-08T21:17:03.248126+0000","last_became_peered":"2024-05-08T21:17:03.248126+0000","last_unstale":"2024-05-08T21:17:56.423294+0000","last_undegraded":"2024-05-08T21:17:56.423294+0000","last_fullsized":"2024-05-08T21:17:56.423294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:08.785761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422472+0000","last_change":"2024-05-08T21:17:07.272344+0000","last_active":"2024-05-08T21:17:56.422472+0000","last_peered":"2024-05-08T21:17:56.422472+0000","last_clean":"2024-05-08T21:17:56.422472+0000","last_became_active":"2024-05-08T21:17:07.271629+0000","last_became_peered":"2024-05-08T21:17:07.271629+0000","last_unstale":"2024-05-08T21:17:56.422472+0000","last_undegraded":"2024-05-08T21:17:56.422472+0000","last_fullsized":"2024-05-08T21:17:56.422472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:15.052748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416295+0000","last_change":"2024-05-08T21:17:02.730364+0000","last_active":"2024-05-08T21:17:56.416295+0000","last_peered":"2024-05-08T21:17:56.416295+0000","last_clean":"2024-05-08T21:17:56.416295+0000","last_became_active":"2024-05-08T21:17:02.728854+0000","last_became_peered":"2024-05-08T21:17:02.728854+0000","last_unstale":"2024-05-08T21:17:56.416295+0000","last_undegraded":"2024-05-08T21:17:56.416295+0000","last_fullsized":"2024-05-08T21:17:56.416295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:09.552016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419025+0000","last_change":"2024-05-08T21:17:03.252713+0000","last_active":"2024-05-08T21:17:56.419025+0000","last_peered":"2024-05-08T21:17:56.419025+0000","last_clean":"2024-05-08T21:17:56.419025+0000","last_became_active":"2024-05-08T21:17:03.252388+0000","last_became_peered":"2024-05-08T21:17:03.252388+0000","last_unstale":"2024-05-08T21:17:56.419025+0000","last_undegraded":"2024-05-08T21:17:56.419025+0000","last_fullsized":"2024-05-08T21:17:56.419025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:09.114064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423183+0000","last_change":"2024-05-08T21:17:07.261112+0000","last_active":"2024-05-08T21:17:56.423183+0000","last_peered":"2024-05-08T21:17:56.423183+0000","last_clean":"2024-05-08T21:17:56.423183+0000","last_became_active":"2024-05-08T21:17:07.260713+0000","last_became_peered":"2024-05-08T21:17:07.260713+0000","last_unstale":"2024-05-08T21:17:56.423183+0000","last_undegraded":"2024-05-08T21:17:56.423183+0000","last_fullsized":"2024-05-08T21:17:56.423183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:37.262113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":122,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:49.510444+0000","last_change":"2024-05-08T21:17:57.436812+0000","last_active":"2024-05-08T21:18:49.510444+0000","last_peered":"2024-05-08T21:18:49.510444+0000","last_clean":"2024-05-08T21:18:49.510444+0000","last_became_active":"2024-05-08T21:17:57.432431+0000","last_became_peered":"2024-05-08T21:17:57.432431+0000","last_unstale":"2024-05-08T21:18:49.510444+0000","last_undegraded":"2024-05-08T21:18:49.510444+0000","last_fullsized":"2024-05-08T21:18:49.510444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:46.062273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1070,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":574,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417176+0000","last_change":"2024-05-08T21:17:03.251245+0000","last_active":"2024-05-08T21:17:56.417176+0000","last_peered":"2024-05-08T21:17:56.417176+0000","last_clean":"2024-05-08T21:17:56.417176+0000","last_became_active":"2024-05-08T21:17:03.250296+0000","last_became_peered":"2024-05-08T21:17:03.250296+0000","last_unstale":"2024-05-08T21:17:56.417176+0000","last_undegraded":"2024-05-08T21:17:56.417176+0000","last_fullsized":"2024-05-08T21:17:56.417176+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:28.391927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423437+0000","last_change":"2024-05-08T21:17:07.266752+0000","last_active":"2024-05-08T21:17:56.423437+0000","last_peered":"2024-05-08T21:17:56.423437+0000","last_clean":"2024-05-08T21:17:56.423437+0000","last_became_active":"2024-05-08T21:17:07.266367+0000","last_became_peered":"2024-05-08T21:17:07.266367+0000","last_unstale":"2024-05-08T21:17:56.423437+0000","last_undegraded":"2024-05-08T21:17:56.423437+0000","last_fullsized":"2024-05-08T21:17:56.423437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:49.874073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417579+0000","last_change":"2024-05-08T21:17:02.728113+0000","last_active":"2024-05-08T21:17:56.417579+0000","last_peered":"2024-05-08T21:17:56.417579+0000","last_clean":"2024-05-08T21:17:56.417579+0000","last_became_active":"2024-05-08T21:17:02.727754+0000","last_became_peered":"2024-05-08T21:17:02.727754+0000","last_unstale":"2024-05-08T21:17:56.417579+0000","last_undegraded":"2024-05-08T21:17:56.417579+0000","last_fullsized":"2024-05-08T21:17:56.417579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:53.789218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418802+0000","last_change":"2024-05-08T21:17:03.245482+0000","last_active":"2024-05-08T21:17:56.418802+0000","last_peered":"2024-05-08T21:17:56.418802+0000","last_clean":"2024-05-08T21:17:56.418802+0000","last_became_active":"2024-05-08T21:17:03.245144+0000","last_became_peered":"2024-05-08T21:17:03.245144+0000","last_unstale":"2024-05-08T21:17:56.418802+0000","last_undegraded":"2024-05-08T21:17:56.418802+0000","last_fullsized":"2024-05-08T21:17:56.418802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:04.051234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418636+0000","last_change":"2024-05-08T21:17:07.268505+0000","last_active":"2024-05-08T21:17:56.418636+0000","last_peered":"2024-05-08T21:17:56.418636+0000","last_clean":"2024-05-08T21:17:56.418636+0000","last_became_active":"2024-05-08T21:17:07.268157+0000","last_became_peered":"2024-05-08T21:17:07.268157+0000","last_unstale":"2024-05-08T21:17:56.418636+0000","last_undegraded":"2024-05-08T21:17:56.418636+0000","last_fullsized":"2024-05-08T21:17:56.418636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:55.220521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412606+0000","last_change":"2024-05-08T21:17:02.727264+0000","last_active":"2024-05-08T21:17:56.412606+0000","last_peered":"2024-05-08T21:17:56.412606+0000","last_clean":"2024-05-08T21:17:56.412606+0000","last_became_active":"2024-05-08T21:17:02.726922+0000","last_became_peered":"2024-05-08T21:17:02.726922+0000","last_unstale":"2024-05-08T21:17:56.412606+0000","last_undegraded":"2024-05-08T21:17:56.412606+0000","last_fullsized":"2024-05-08T21:17:56.412606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:27.410231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415635+0000","last_change":"2024-05-08T21:17:03.233910+0000","last_active":"2024-05-08T21:17:56.415635+0000","last_peered":"2024-05-08T21:17:56.415635+0000","last_clean":"2024-05-08T21:17:56.415635+0000","last_became_active":"2024-05-08T21:17:03.233562+0000","last_became_peered":"2024-05-08T21:17:03.233562+0000","last_unstale":"2024-05-08T21:17:56.415635+0000","last_undegraded":"2024-05-08T21:17:56.415635+0000","last_fullsized":"2024-05-08T21:17:56.415635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:40.309400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423244+0000","last_change":"2024-05-08T21:17:07.264992+0000","last_active":"2024-05-08T21:17:56.423244+0000","last_peered":"2024-05-08T21:17:56.423244+0000","last_clean":"2024-05-08T21:17:56.423244+0000","last_became_active":"2024-05-08T21:17:07.264498+0000","last_became_peered":"2024-05-08T21:17:07.264498+0000","last_unstale":"2024-05-08T21:17:56.423244+0000","last_undegraded":"2024-05-08T21:17:56.423244+0000","last_fullsized":"2024-05-08T21:17:56.423244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:23:26.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418593+0000","last_change":"2024-05-08T21:17:02.725531+0000","last_active":"2024-05-08T21:17:56.418593+0000","last_peered":"2024-05-08T21:17:56.418593+0000","last_clean":"2024-05-08T21:17:56.418593+0000","last_became_active":"2024-05-08T21:17:02.725235+0000","last_became_peered":"2024-05-08T21:17:02.725235+0000","last_unstale":"2024-05-08T21:17:56.418593+0000","last_undegraded":"2024-05-08T21:17:56.418593+0000","last_fullsized":"2024-05-08T21:17:56.418593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:27.265590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419530+0000","last_change":"2024-05-08T21:17:03.236393+0000","last_active":"2024-05-08T21:17:56.419530+0000","last_peered":"2024-05-08T21:17:56.419530+0000","last_clean":"2024-05-08T21:17:56.419530+0000","last_became_active":"2024-05-08T21:17:03.236092+0000","last_became_peered":"2024-05-08T21:17:03.236092+0000","last_unstale":"2024-05-08T21:17:56.419530+0000","last_undegraded":"2024-05-08T21:17:56.419530+0000","last_fullsized":"2024-05-08T21:17:56.419530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:58.232669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417799+0000","last_change":"2024-05-08T21:17:07.271374+0000","last_active":"2024-05-08T21:17:56.417799+0000","last_peered":"2024-05-08T21:17:56.417799+0000","last_clean":"2024-05-08T21:17:56.417799+0000","last_became_active":"2024-05-08T21:17:07.270778+0000","last_became_peered":"2024-05-08T21:17:07.270778+0000","last_unstale":"2024-05-08T21:17:56.417799+0000","last_undegraded":"2024-05-08T21:17:56.417799+0000","last_fullsized":"2024-05-08T21:17:56.417799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:50.212213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'6","reported_seq":184,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:50.447209+0000","last_change":"2024-05-08T21:17:02.725624+0000","last_active":"2024-05-08T21:18:50.447209+0000","last_peered":"2024-05-08T21:18:50.447209+0000","last_clean":"2024-05-08T21:18:50.447209+0000","last_became_active":"2024-05-08T21:17:02.725367+0000","last_became_peered":"2024-05-08T21:17:02.725367+0000","last_unstale":"2024-05-08T21:18:50.447209+0000","last_undegraded":"2024-05-08T21:18:50.447209+0000","last_fullsized":"2024-05-08T21:18:50.447209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:19.573931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":160,"num_read_kb":160,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423993+0000","last_change":"2024-05-08T21:17:03.248729+0000","last_active":"2024-05-08T21:17:56.423993+0000","last_peered":"2024-05-08T21:17:56.423993+0000","last_clean":"2024-05-08T21:17:56.423993+0000","last_became_active":"2024-05-08T21:17:03.247899+0000","last_became_peered":"2024-05-08T21:17:03.247899+0000","last_unstale":"2024-05-08T21:17:56.423993+0000","last_undegraded":"2024-05-08T21:17:56.423993+0000","last_fullsized":"2024-05-08T21:17:56.423993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:27:57.012249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419291+0000","last_change":"2024-05-08T21:17:07.261257+0000","last_active":"2024-05-08T21:17:56.419291+0000","last_peered":"2024-05-08T21:17:56.419291+0000","last_clean":"2024-05-08T21:17:56.419291+0000","last_became_active":"2024-05-08T21:17:07.260785+0000","last_became_peered":"2024-05-08T21:17:07.260785+0000","last_unstale":"2024-05-08T21:17:56.419291+0000","last_undegraded":"2024-05-08T21:17:56.419291+0000","last_fullsized":"2024-05-08T21:17:56.419291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:58.469134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419901+0000","last_change":"2024-05-08T21:17:02.729569+0000","last_active":"2024-05-08T21:17:56.419901+0000","last_peered":"2024-05-08T21:17:56.419901+0000","last_clean":"2024-05-08T21:17:56.419901+0000","last_became_active":"2024-05-08T21:17:02.729096+0000","last_became_peered":"2024-05-08T21:17:02.729096+0000","last_unstale":"2024-05-08T21:17:56.419901+0000","last_undegraded":"2024-05-08T21:17:56.419901+0000","last_fullsized":"2024-05-08T21:17:56.419901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:30.668169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423406+0000","last_change":"2024-05-08T21:17:03.246387+0000","last_active":"2024-05-08T21:17:56.423406+0000","last_peered":"2024-05-08T21:17:56.423406+0000","last_clean":"2024-05-08T21:17:56.423406+0000","last_became_active":"2024-05-08T21:17:03.244658+0000","last_became_peered":"2024-05-08T21:17:03.244658+0000","last_unstale":"2024-05-08T21:17:56.423406+0000","last_undegraded":"2024-05-08T21:17:56.423406+0000","last_fullsized":"2024-05-08T21:17:56.423406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:02.602080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.473101+0000","last_change":"2024-05-08T21:17:57.473101+0000","last_active":"2024-05-08T21:17:57.473101+0000","last_peered":"2024-05-08T21:17:57.473101+0000","last_clean":"2024-05-08T21:17:57.473101+0000","last_became_active":"2024-05-08T21:17:57.472563+0000","last_became_peered":"2024-05-08T21:17:57.472563+0000","last_unstale":"2024-05-08T21:17:57.473101+0000","last_undegraded":"2024-05-08T21:17:57.473101+0000","last_fullsized":"2024-05-08T21:17:57.473101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:23.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418913+0000","last_change":"2024-05-08T21:17:02.732738+0000","last_active":"2024-05-08T21:17:56.418913+0000","last_peered":"2024-05-08T21:17:56.418913+0000","last_clean":"2024-05-08T21:17:56.418913+0000","last_became_active":"2024-05-08T21:17:02.731733+0000","last_became_peered":"2024-05-08T21:17:02.731733+0000","last_unstale":"2024-05-08T21:17:56.418913+0000","last_undegraded":"2024-05-08T21:17:56.418913+0000","last_fullsized":"2024-05-08T21:17:56.418913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:24.201171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423479+0000","last_change":"2024-05-08T21:17:03.254405+0000","last_active":"2024-05-08T21:17:56.423479+0000","last_peered":"2024-05-08T21:17:56.423479+0000","last_clean":"2024-05-08T21:17:56.423479+0000","last_became_active":"2024-05-08T21:17:03.252137+0000","last_became_peered":"2024-05-08T21:17:03.252137+0000","last_unstale":"2024-05-08T21:17:56.423479+0000","last_undegraded":"2024-05-08T21:17:56.423479+0000","last_fullsized":"2024-05-08T21:17:56.423479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:15.472225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.431574+0000","last_change":"2024-05-08T21:17:57.431091+0000","last_active":"2024-05-08T21:17:57.431574+0000","last_peered":"2024-05-08T21:17:57.431574+0000","last_clean":"2024-05-08T21:17:57.431574+0000","last_became_active":"2024-05-08T21:17:57.430668+0000","last_became_peered":"2024-05-08T21:17:57.430668+0000","last_unstale":"2024-05-08T21:17:57.431574+0000","last_undegraded":"2024-05-08T21:17:57.431574+0000","last_fullsized":"2024-05-08T21:17:57.431574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:33.208801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"87'38","reported_seq":225,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:50.511512+0000","last_change":"2024-05-08T21:17:02.723975+0000","last_active":"2024-05-08T21:18:50.511512+0000","last_peered":"2024-05-08T21:18:50.511512+0000","last_clean":"2024-05-08T21:18:50.511512+0000","last_became_active":"2024-05-08T21:17:02.723631+0000","last_became_peered":"2024-05-08T21:17:02.723631+0000","last_unstale":"2024-05-08T21:18:50.511512+0000","last_undegraded":"2024-05-08T21:18:50.511512+0000","last_fullsized":"2024-05-08T21:18:50.511512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:17.676007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62785,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":169,"num_read_kb":145,"num_write":40,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416534+0000","last_change":"2024-05-08T21:17:03.251570+0000","last_active":"2024-05-08T21:17:56.416534+0000","last_peered":"2024-05-08T21:17:56.416534+0000","last_clean":"2024-05-08T21:17:56.416534+0000","last_became_active":"2024-05-08T21:17:03.251134+0000","last_became_peered":"2024-05-08T21:17:03.251134+0000","last_unstale":"2024-05-08T21:17:56.416534+0000","last_undegraded":"2024-05-08T21:17:56.416534+0000","last_fullsized":"2024-05-08T21:17:56.416534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:28:09.339958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419635+0000","last_change":"2024-05-08T21:17:07.272051+0000","last_active":"2024-05-08T21:17:56.419635+0000","last_peered":"2024-05-08T21:17:56.419635+0000","last_clean":"2024-05-08T21:17:56.419635+0000","last_became_active":"2024-05-08T21:17:07.271616+0000","last_became_peered":"2024-05-08T21:17:07.271616+0000","last_unstale":"2024-05-08T21:17:56.419635+0000","last_undegraded":"2024-05-08T21:17:56.419635+0000","last_fullsized":"2024-05-08T21:17:56.419635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.364731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423259+0000","last_change":"2024-05-08T21:17:02.736000+0000","last_active":"2024-05-08T21:17:56.423259+0000","last_peered":"2024-05-08T21:17:56.423259+0000","last_clean":"2024-05-08T21:17:56.423259+0000","last_became_active":"2024-05-08T21:17:02.735629+0000","last_became_peered":"2024-05-08T21:17:02.735629+0000","last_unstale":"2024-05-08T21:17:56.423259+0000","last_undegraded":"2024-05-08T21:17:56.423259+0000","last_fullsized":"2024-05-08T21:17:56.423259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:21.842135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412729+0000","last_change":"2024-05-08T21:17:03.237144+0000","last_active":"2024-05-08T21:17:56.412729+0000","last_peered":"2024-05-08T21:17:56.412729+0000","last_clean":"2024-05-08T21:17:56.412729+0000","last_became_active":"2024-05-08T21:17:03.236792+0000","last_became_peered":"2024-05-08T21:17:03.236792+0000","last_unstale":"2024-05-08T21:17:56.412729+0000","last_undegraded":"2024-05-08T21:17:56.412729+0000","last_fullsized":"2024-05-08T21:17:56.412729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:04.520986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.627656+0000","last_change":"2024-05-08T21:17:57.627656+0000","last_active":"2024-05-08T21:17:57.627656+0000","last_peered":"2024-05-08T21:17:57.627656+0000","last_clean":"2024-05-08T21:17:57.627656+0000","last_became_active":"2024-05-08T21:17:57.627108+0000","last_became_peered":"2024-05-08T21:17:57.627108+0000","last_unstale":"2024-05-08T21:17:57.627656+0000","last_undegraded":"2024-05-08T21:17:57.627656+0000","last_fullsized":"2024-05-08T21:17:57.627656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:24.025145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419802+0000","last_change":"2024-05-08T21:17:02.727748+0000","last_active":"2024-05-08T21:17:56.419802+0000","last_peered":"2024-05-08T21:17:56.419802+0000","last_clean":"2024-05-08T21:17:56.419802+0000","last_became_active":"2024-05-08T21:17:02.727444+0000","last_became_peered":"2024-05-08T21:17:02.727444+0000","last_unstale":"2024-05-08T21:17:56.419802+0000","last_undegraded":"2024-05-08T21:17:56.419802+0000","last_fullsized":"2024-05-08T21:17:56.419802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:47.939235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420245+0000","last_change":"2024-05-08T21:17:03.247185+0000","last_active":"2024-05-08T21:17:56.420245+0000","last_peered":"2024-05-08T21:17:56.420245+0000","last_clean":"2024-05-08T21:17:56.420245+0000","last_became_active":"2024-05-08T21:17:03.246880+0000","last_became_peered":"2024-05-08T21:17:03.246880+0000","last_unstale":"2024-05-08T21:17:56.420245+0000","last_undegraded":"2024-05-08T21:17:56.420245+0000","last_fullsized":"2024-05-08T21:17:56.420245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:38.978353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420018+0000","last_change":"2024-05-08T21:17:07.273710+0000","last_active":"2024-05-08T21:17:56.420018+0000","last_peered":"2024-05-08T21:17:56.420018+0000","last_clean":"2024-05-08T21:17:56.420018+0000","last_became_active":"2024-05-08T21:17:07.273119+0000","last_became_peered":"2024-05-08T21:17:07.273119+0000","last_unstale":"2024-05-08T21:17:56.420018+0000","last_undegraded":"2024-05-08T21:17:56.420018+0000","last_fullsized":"2024-05-08T21:17:56.420018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:30.589440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":109,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:51.429542+0000","last_change":"2024-05-08T21:17:02.725508+0000","last_active":"2024-05-08T21:18:51.429542+0000","last_peered":"2024-05-08T21:18:51.429542+0000","last_clean":"2024-05-08T21:18:51.429542+0000","last_became_active":"2024-05-08T21:17:02.725107+0000","last_became_peered":"2024-05-08T21:17:02.725107+0000","last_unstale":"2024-05-08T21:18:51.429542+0000","last_undegraded":"2024-05-08T21:18:51.429542+0000","last_fullsized":"2024-05-08T21:18:51.429542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:13.503702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1036,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420302+0000","last_change":"2024-05-08T21:17:03.245372+0000","last_active":"2024-05-08T21:17:56.420302+0000","last_peered":"2024-05-08T21:17:56.420302+0000","last_clean":"2024-05-08T21:17:56.420302+0000","last_became_active":"2024-05-08T21:17:03.245054+0000","last_became_peered":"2024-05-08T21:17:03.245054+0000","last_unstale":"2024-05-08T21:17:56.420302+0000","last_undegraded":"2024-05-08T21:17:56.420302+0000","last_fullsized":"2024-05-08T21:17:56.420302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:30:25.900846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414821+0000","last_change":"2024-05-08T21:17:07.272081+0000","last_active":"2024-05-08T21:17:56.414821+0000","last_peered":"2024-05-08T21:17:56.414821+0000","last_clean":"2024-05-08T21:17:56.414821+0000","last_became_active":"2024-05-08T21:17:07.271560+0000","last_became_peered":"2024-05-08T21:17:07.271560+0000","last_unstale":"2024-05-08T21:17:56.414821+0000","last_undegraded":"2024-05-08T21:17:56.414821+0000","last_fullsized":"2024-05-08T21:17:56.414821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:00.490798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416027+0000","last_change":"2024-05-08T21:17:02.734420+0000","last_active":"2024-05-08T21:17:56.416027+0000","last_peered":"2024-05-08T21:17:56.416027+0000","last_clean":"2024-05-08T21:17:56.416027+0000","last_became_active":"2024-05-08T21:17:02.734071+0000","last_became_peered":"2024-05-08T21:17:02.734071+0000","last_unstale":"2024-05-08T21:17:56.416027+0000","last_undegraded":"2024-05-08T21:17:56.416027+0000","last_fullsized":"2024-05-08T21:17:56.416027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:28.537464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419127+0000","last_change":"2024-05-08T21:17:03.250706+0000","last_active":"2024-05-08T21:17:56.419127+0000","last_peered":"2024-05-08T21:17:56.419127+0000","last_clean":"2024-05-08T21:17:56.419127+0000","last_became_active":"2024-05-08T21:17:03.250408+0000","last_became_peered":"2024-05-08T21:17:03.250408+0000","last_unstale":"2024-05-08T21:17:56.419127+0000","last_undegraded":"2024-05-08T21:17:56.419127+0000","last_fullsized":"2024-05-08T21:17:56.419127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:43.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418530+0000","last_change":"2024-05-08T21:17:07.272066+0000","last_active":"2024-05-08T21:17:56.418530+0000","last_peered":"2024-05-08T21:17:56.418530+0000","last_clean":"2024-05-08T21:17:56.418530+0000","last_became_active":"2024-05-08T21:17:07.271628+0000","last_became_peered":"2024-05-08T21:17:07.271628+0000","last_unstale":"2024-05-08T21:17:56.418530+0000","last_undegraded":"2024-05-08T21:17:56.418530+0000","last_fullsized":"2024-05-08T21:17:56.418530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:27.731995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.424263+0000","last_change":"2024-05-08T21:17:02.728464+0000","last_active":"2024-05-08T21:17:57.424263+0000","last_peered":"2024-05-08T21:17:57.424263+0000","last_clean":"2024-05-08T21:17:57.424263+0000","last_became_active":"2024-05-08T21:17:02.727914+0000","last_became_peered":"2024-05-08T21:17:02.727914+0000","last_unstale":"2024-05-08T21:17:57.424263+0000","last_undegraded":"2024-05-08T21:17:57.424263+0000","last_fullsized":"2024-05-08T21:17:57.424263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:55.948661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418448+0000","last_change":"2024-05-08T21:17:03.245578+0000","last_active":"2024-05-08T21:17:56.418448+0000","last_peered":"2024-05-08T21:17:56.418448+0000","last_clean":"2024-05-08T21:17:56.418448+0000","last_became_active":"2024-05-08T21:17:03.245256+0000","last_became_peered":"2024-05-08T21:17:03.245256+0000","last_unstale":"2024-05-08T21:17:56.418448+0000","last_undegraded":"2024-05-08T21:17:56.418448+0000","last_fullsized":"2024-05-08T21:17:56.418448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417048+0000","last_change":"2024-05-08T21:17:07.439525+0000","last_active":"2024-05-08T21:17:56.417048+0000","last_peered":"2024-05-08T21:17:56.417048+0000","last_clean":"2024-05-08T21:17:56.417048+0000","last_became_active":"2024-05-08T21:17:07.438987+0000","last_became_peered":"2024-05-08T21:17:07.438987+0000","last_unstale":"2024-05-08T21:17:56.417048+0000","last_undegraded":"2024-05-08T21:17:56.417048+0000","last_fullsized":"2024-05-08T21:17:56.417048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:57:45.373981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.432999+0000","last_change":"2024-05-08T21:17:57.432999+0000","last_active":"2024-05-08T21:17:57.432999+0000","last_peered":"2024-05-08T21:17:57.432999+0000","last_clean":"2024-05-08T21:17:57.432999+0000","last_became_active":"2024-05-08T21:17:57.432567+0000","last_became_peered":"2024-05-08T21:17:57.432567+0000","last_unstale":"2024-05-08T21:17:57.432999+0000","last_undegraded":"2024-05-08T21:17:57.432999+0000","last_fullsized":"2024-05-08T21:17:57.432999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:33:00.188046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417907+0000","last_change":"2024-05-08T21:17:03.235556+0000","last_active":"2024-05-08T21:17:56.417907+0000","last_peered":"2024-05-08T21:17:56.417907+0000","last_clean":"2024-05-08T21:17:56.417907+0000","last_became_active":"2024-05-08T21:17:03.235214+0000","last_became_peered":"2024-05-08T21:17:03.235214+0000","last_unstale":"2024-05-08T21:17:56.417907+0000","last_undegraded":"2024-05-08T21:17:56.417907+0000","last_fullsized":"2024-05-08T21:17:56.417907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:45.042596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414845+0000","last_change":"2024-05-08T21:17:07.266080+0000","last_active":"2024-05-08T21:17:56.414845+0000","last_peered":"2024-05-08T21:17:56.414845+0000","last_clean":"2024-05-08T21:17:56.414845+0000","last_became_active":"2024-05-08T21:17:07.265605+0000","last_became_peered":"2024-05-08T21:17:07.265605+0000","last_unstale":"2024-05-08T21:17:56.414845+0000","last_undegraded":"2024-05-08T21:17:56.414845+0000","last_fullsized":"2024-05-08T21:17:56.414845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:49:48.387170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:49.549637+0000","last_change":"2024-05-08T21:17:02.728743+0000","last_active":"2024-05-08T21:18:49.549637+0000","last_peered":"2024-05-08T21:18:49.549637+0000","last_clean":"2024-05-08T21:18:49.549637+0000","last_became_active":"2024-05-08T21:17:02.728330+0000","last_became_peered":"2024-05-08T21:17:02.728330+0000","last_unstale":"2024-05-08T21:18:49.549637+0000","last_undegraded":"2024-05-08T21:18:49.549637+0000","last_fullsized":"2024-05-08T21:18:49.549637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:43.865117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1036,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417780+0000","last_change":"2024-05-08T21:17:03.245672+0000","last_active":"2024-05-08T21:17:56.417780+0000","last_peered":"2024-05-08T21:17:56.417780+0000","last_clean":"2024-05-08T21:17:56.417780+0000","last_became_active":"2024-05-08T21:17:03.245318+0000","last_became_peered":"2024-05-08T21:17:03.245318+0000","last_unstale":"2024-05-08T21:17:56.417780+0000","last_undegraded":"2024-05-08T21:17:56.417780+0000","last_fullsized":"2024-05-08T21:17:56.417780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:43.030011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414114+0000","last_change":"2024-05-08T21:17:07.439289+0000","last_active":"2024-05-08T21:17:56.414114+0000","last_peered":"2024-05-08T21:17:56.414114+0000","last_clean":"2024-05-08T21:17:56.414114+0000","last_became_active":"2024-05-08T21:17:07.438874+0000","last_became_peered":"2024-05-08T21:17:07.438874+0000","last_unstale":"2024-05-08T21:17:56.414114+0000","last_undegraded":"2024-05-08T21:17:56.414114+0000","last_fullsized":"2024-05-08T21:17:56.414114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:14.820087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419309+0000","last_change":"2024-05-08T21:17:03.247081+0000","last_active":"2024-05-08T21:17:56.419309+0000","last_peered":"2024-05-08T21:17:56.419309+0000","last_clean":"2024-05-08T21:17:56.419309+0000","last_became_active":"2024-05-08T21:17:03.246418+0000","last_became_peered":"2024-05-08T21:17:03.246418+0000","last_unstale":"2024-05-08T21:17:56.419309+0000","last_undegraded":"2024-05-08T21:17:56.419309+0000","last_fullsized":"2024-05-08T21:17:56.419309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:03.993770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":108,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:52.430374+0000","last_change":"2024-05-08T21:17:02.730764+0000","last_active":"2024-05-08T21:18:52.430374+0000","last_peered":"2024-05-08T21:18:52.430374+0000","last_clean":"2024-05-08T21:18:52.430374+0000","last_became_active":"2024-05-08T21:17:02.730494+0000","last_became_peered":"2024-05-08T21:17:02.730494+0000","last_unstale":"2024-05-08T21:18:52.430374+0000","last_undegraded":"2024-05-08T21:18:52.430374+0000","last_fullsized":"2024-05-08T21:18:52.430374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:50.070368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419669+0000","last_change":"2024-05-08T21:17:07.266424+0000","last_active":"2024-05-08T21:17:56.419669+0000","last_peered":"2024-05-08T21:17:56.419669+0000","last_clean":"2024-05-08T21:17:56.419669+0000","last_became_active":"2024-05-08T21:17:07.266144+0000","last_became_peered":"2024-05-08T21:17:07.266144+0000","last_unstale":"2024-05-08T21:17:56.419669+0000","last_undegraded":"2024-05-08T21:17:56.419669+0000","last_fullsized":"2024-05-08T21:17:56.419669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:22.285749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417254+0000","last_change":"2024-05-08T21:17:03.250461+0000","last_active":"2024-05-08T21:17:56.417254+0000","last_peered":"2024-05-08T21:17:56.417254+0000","last_clean":"2024-05-08T21:17:56.417254+0000","last_became_active":"2024-05-08T21:17:03.249974+0000","last_became_peered":"2024-05-08T21:17:03.249974+0000","last_unstale":"2024-05-08T21:17:56.417254+0000","last_undegraded":"2024-05-08T21:17:56.417254+0000","last_fullsized":"2024-05-08T21:17:56.417254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:25:58.759668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422681+0000","last_change":"2024-05-08T21:17:02.725648+0000","last_active":"2024-05-08T21:17:56.422681+0000","last_peered":"2024-05-08T21:17:56.422681+0000","last_clean":"2024-05-08T21:17:56.422681+0000","last_became_active":"2024-05-08T21:17:02.725297+0000","last_became_peered":"2024-05-08T21:17:02.725297+0000","last_unstale":"2024-05-08T21:17:56.422681+0000","last_undegraded":"2024-05-08T21:17:56.422681+0000","last_fullsized":"2024-05-08T21:17:56.422681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:41.462830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":328,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420115+0000","last_change":"2024-05-08T21:16:15.194478+0000","last_active":"2024-05-08T21:17:56.420115+0000","last_peered":"2024-05-08T21:17:56.420115+0000","last_clean":"2024-05-08T21:17:56.420115+0000","last_became_active":"2024-05-08T21:16:15.193964+0000","last_became_peered":"2024-05-08T21:16:15.193964+0000","last_unstale":"2024-05-08T21:17:56.420115+0000","last_undegraded":"2024-05-08T21:17:56.420115+0000","last_fullsized":"2024-05-08T21:17:56.420115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_clean_scrub_stamp":"2024-05-08T21:13:48.584026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:28.091200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.628348+0000","last_change":"2024-05-08T21:17:57.627889+0000","last_active":"2024-05-08T21:17:57.628348+0000","last_peered":"2024-05-08T21:17:57.628348+0000","last_clean":"2024-05-08T21:17:57.628348+0000","last_became_active":"2024-05-08T21:17:57.627502+0000","last_became_peered":"2024-05-08T21:17:57.627502+0000","last_unstale":"2024-05-08T21:17:57.628348+0000","last_undegraded":"2024-05-08T21:17:57.628348+0000","last_fullsized":"2024-05-08T21:17:57.628348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:10.025347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412481+0000","last_change":"2024-05-08T21:17:03.246034+0000","last_active":"2024-05-08T21:17:56.412481+0000","last_peered":"2024-05-08T21:17:56.412481+0000","last_clean":"2024-05-08T21:17:56.412481+0000","last_became_active":"2024-05-08T21:17:03.245620+0000","last_became_peered":"2024-05-08T21:17:03.245620+0000","last_unstale":"2024-05-08T21:17:56.412481+0000","last_undegraded":"2024-05-08T21:17:56.412481+0000","last_fullsized":"2024-05-08T21:17:56.412481+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:02.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420424+0000","last_change":"2024-05-08T21:17:02.731260+0000","last_active":"2024-05-08T21:17:56.420424+0000","last_peered":"2024-05-08T21:17:56.420424+0000","last_clean":"2024-05-08T21:17:56.420424+0000","last_became_active":"2024-05-08T21:17:02.730879+0000","last_became_peered":"2024-05-08T21:17:02.730879+0000","last_unstale":"2024-05-08T21:17:56.420424+0000","last_undegraded":"2024-05-08T21:17:56.420424+0000","last_fullsized":"2024-05-08T21:17:56.420424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:59.030674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417621+0000","last_change":"2024-05-08T21:17:07.269791+0000","last_active":"2024-05-08T21:17:56.417621+0000","last_peered":"2024-05-08T21:17:56.417621+0000","last_clean":"2024-05-08T21:17:56.417621+0000","last_became_active":"2024-05-08T21:17:07.269345+0000","last_became_peered":"2024-05-08T21:17:07.269345+0000","last_unstale":"2024-05-08T21:17:56.417621+0000","last_undegraded":"2024-05-08T21:17:56.417621+0000","last_fullsized":"2024-05-08T21:17:56.417621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:42.866864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416940+0000","last_change":"2024-05-08T21:17:02.735423+0000","last_active":"2024-05-08T21:17:56.416940+0000","last_peered":"2024-05-08T21:17:56.416940+0000","last_clean":"2024-05-08T21:17:56.416940+0000","last_became_active":"2024-05-08T21:17:02.735158+0000","last_became_peered":"2024-05-08T21:17:02.735158+0000","last_unstale":"2024-05-08T21:17:56.416940+0000","last_undegraded":"2024-05-08T21:17:56.416940+0000","last_fullsized":"2024-05-08T21:17:56.416940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:19.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415012+0000","last_change":"2024-05-08T21:17:03.239316+0000","last_active":"2024-05-08T21:17:56.415012+0000","last_peered":"2024-05-08T21:17:56.415012+0000","last_clean":"2024-05-08T21:17:56.415012+0000","last_became_active":"2024-05-08T21:17:03.239018+0000","last_became_peered":"2024-05-08T21:17:03.239018+0000","last_unstale":"2024-05-08T21:17:56.415012+0000","last_undegraded":"2024-05-08T21:17:56.415012+0000","last_fullsized":"2024-05-08T21:17:56.415012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:36.952155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415569+0000","last_change":"2024-05-08T21:17:07.439338+0000","last_active":"2024-05-08T21:17:56.415569+0000","last_peered":"2024-05-08T21:17:56.415569+0000","last_clean":"2024-05-08T21:17:56.415569+0000","last_became_active":"2024-05-08T21:17:07.438703+0000","last_became_peered":"2024-05-08T21:17:07.438703+0000","last_unstale":"2024-05-08T21:17:56.415569+0000","last_undegraded":"2024-05-08T21:17:56.415569+0000","last_fullsized":"2024-05-08T21:17:56.415569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:50.997138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420144+0000","last_change":"2024-05-08T21:17:02.729012+0000","last_active":"2024-05-08T21:17:56.420144+0000","last_peered":"2024-05-08T21:17:56.420144+0000","last_clean":"2024-05-08T21:17:56.420144+0000","last_became_active":"2024-05-08T21:17:02.728606+0000","last_became_peered":"2024-05-08T21:17:02.728606+0000","last_unstale":"2024-05-08T21:17:56.420144+0000","last_undegraded":"2024-05-08T21:17:56.420144+0000","last_fullsized":"2024-05-08T21:17:56.420144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.140609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416523+0000","last_change":"2024-05-08T21:17:03.234712+0000","last_active":"2024-05-08T21:17:56.416523+0000","last_peered":"2024-05-08T21:17:56.416523+0000","last_clean":"2024-05-08T21:17:56.416523+0000","last_became_active":"2024-05-08T21:17:03.234404+0000","last_became_peered":"2024-05-08T21:17:03.234404+0000","last_unstale":"2024-05-08T21:17:56.416523+0000","last_undegraded":"2024-05-08T21:17:56.416523+0000","last_fullsized":"2024-05-08T21:17:56.416523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:41.479470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422480+0000","last_change":"2024-05-08T21:17:07.259120+0000","last_active":"2024-05-08T21:17:56.422480+0000","last_peered":"2024-05-08T21:17:56.422480+0000","last_clean":"2024-05-08T21:17:56.422480+0000","last_became_active":"2024-05-08T21:17:07.258696+0000","last_became_peered":"2024-05-08T21:17:07.258696+0000","last_unstale":"2024-05-08T21:17:56.422480+0000","last_undegraded":"2024-05-08T21:17:56.422480+0000","last_fullsized":"2024-05-08T21:17:56.422480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:26.950767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423353+0000","last_change":"2024-05-08T21:17:02.727795+0000","last_active":"2024-05-08T21:17:56.423353+0000","last_peered":"2024-05-08T21:17:56.423353+0000","last_clean":"2024-05-08T21:17:56.423353+0000","last_became_active":"2024-05-08T21:17:02.727553+0000","last_became_peered":"2024-05-08T21:17:02.727553+0000","last_unstale":"2024-05-08T21:17:56.423353+0000","last_undegraded":"2024-05-08T21:17:56.423353+0000","last_fullsized":"2024-05-08T21:17:56.423353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:51.540221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423413+0000","last_change":"2024-05-08T21:17:03.247270+0000","last_active":"2024-05-08T21:17:56.423413+0000","last_peered":"2024-05-08T21:17:56.423413+0000","last_clean":"2024-05-08T21:17:56.423413+0000","last_became_active":"2024-05-08T21:17:03.246853+0000","last_became_peered":"2024-05-08T21:17:03.246853+0000","last_unstale":"2024-05-08T21:17:56.423413+0000","last_undegraded":"2024-05-08T21:17:56.423413+0000","last_fullsized":"2024-05-08T21:17:56.423413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:46.224088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422880+0000","last_change":"2024-05-08T21:17:07.439920+0000","last_active":"2024-05-08T21:17:56.422880+0000","last_peered":"2024-05-08T21:17:56.422880+0000","last_clean":"2024-05-08T21:17:56.422880+0000","last_became_active":"2024-05-08T21:17:07.439459+0000","last_became_peered":"2024-05-08T21:17:07.439459+0000","last_unstale":"2024-05-08T21:17:56.422880+0000","last_undegraded":"2024-05-08T21:17:56.422880+0000","last_fullsized":"2024-05-08T21:17:56.422880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:03.008930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422835+0000","last_change":"2024-05-08T21:17:02.731889+0000","last_active":"2024-05-08T21:17:56.422835+0000","last_peered":"2024-05-08T21:17:56.422835+0000","last_clean":"2024-05-08T21:17:56.422835+0000","last_became_active":"2024-05-08T21:17:02.731554+0000","last_became_peered":"2024-05-08T21:17:02.731554+0000","last_unstale":"2024-05-08T21:17:56.422835+0000","last_undegraded":"2024-05-08T21:17:56.422835+0000","last_fullsized":"2024-05-08T21:17:56.422835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:25.223003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417051+0000","last_change":"2024-05-08T21:17:03.252672+0000","last_active":"2024-05-08T21:17:56.417051+0000","last_peered":"2024-05-08T21:17:56.417051+0000","last_clean":"2024-05-08T21:17:56.417051+0000","last_became_active":"2024-05-08T21:17:03.251684+0000","last_became_peered":"2024-05-08T21:17:03.251684+0000","last_unstale":"2024-05-08T21:17:56.417051+0000","last_undegraded":"2024-05-08T21:17:56.417051+0000","last_fullsized":"2024-05-08T21:17:56.417051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:06.460419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420488+0000","last_change":"2024-05-08T21:17:07.273478+0000","last_active":"2024-05-08T21:17:56.420488+0000","last_peered":"2024-05-08T21:17:56.420488+0000","last_clean":"2024-05-08T21:17:56.420488+0000","last_became_active":"2024-05-08T21:17:07.272525+0000","last_became_peered":"2024-05-08T21:17:07.272525+0000","last_unstale":"2024-05-08T21:17:56.420488+0000","last_undegraded":"2024-05-08T21:17:56.420488+0000","last_fullsized":"2024-05-08T21:17:56.420488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:51:05.107387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417904+0000","last_change":"2024-05-08T21:17:02.728858+0000","last_active":"2024-05-08T21:17:56.417904+0000","last_peered":"2024-05-08T21:17:56.417904+0000","last_clean":"2024-05-08T21:17:56.417904+0000","last_became_active":"2024-05-08T21:17:02.728516+0000","last_became_peered":"2024-05-08T21:17:02.728516+0000","last_unstale":"2024-05-08T21:17:56.417904+0000","last_undegraded":"2024-05-08T21:17:56.417904+0000","last_fullsized":"2024-05-08T21:17:56.417904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:40.263983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418555+0000","last_change":"2024-05-08T21:17:03.239354+0000","last_active":"2024-05-08T21:17:56.418555+0000","last_peered":"2024-05-08T21:17:56.418555+0000","last_clean":"2024-05-08T21:17:56.418555+0000","last_became_active":"2024-05-08T21:17:03.239020+0000","last_became_peered":"2024-05-08T21:17:03.239020+0000","last_unstale":"2024-05-08T21:17:56.418555+0000","last_undegraded":"2024-05-08T21:17:56.418555+0000","last_fullsized":"2024-05-08T21:17:56.418555+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:02:43.362016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422887+0000","last_change":"2024-05-08T21:17:07.262401+0000","last_active":"2024-05-08T21:17:56.422887+0000","last_peered":"2024-05-08T21:17:56.422887+0000","last_clean":"2024-05-08T21:17:56.422887+0000","last_became_active":"2024-05-08T21:17:07.261946+0000","last_became_peered":"2024-05-08T21:17:07.261946+0000","last_unstale":"2024-05-08T21:17:56.422887+0000","last_undegraded":"2024-05-08T21:17:56.422887+0000","last_fullsized":"2024-05-08T21:17:56.422887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:56.565604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.438045+0000","last_change":"2024-05-08T21:17:57.437573+0000","last_active":"2024-05-08T21:17:57.438045+0000","last_peered":"2024-05-08T21:17:57.438045+0000","last_clean":"2024-05-08T21:17:57.438045+0000","last_became_active":"2024-05-08T21:17:57.432315+0000","last_became_peered":"2024-05-08T21:17:57.432315+0000","last_unstale":"2024-05-08T21:17:57.438045+0000","last_undegraded":"2024-05-08T21:17:57.438045+0000","last_fullsized":"2024-05-08T21:17:57.438045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:56:08.847290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417312+0000","last_change":"2024-05-08T21:17:03.250382+0000","last_active":"2024-05-08T21:17:56.417312+0000","last_peered":"2024-05-08T21:17:56.417312+0000","last_clean":"2024-05-08T21:17:56.417312+0000","last_became_active":"2024-05-08T21:17:03.250031+0000","last_became_peered":"2024-05-08T21:17:03.250031+0000","last_unstale":"2024-05-08T21:17:56.417312+0000","last_undegraded":"2024-05-08T21:17:56.417312+0000","last_fullsized":"2024-05-08T21:17:56.417312+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:59.961012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422877+0000","last_change":"2024-05-08T21:17:07.439341+0000","last_active":"2024-05-08T21:17:56.422877+0000","last_peered":"2024-05-08T21:17:56.422877+0000","last_clean":"2024-05-08T21:17:56.422877+0000","last_became_active":"2024-05-08T21:17:07.438896+0000","last_became_peered":"2024-05-08T21:17:07.438896+0000","last_unstale":"2024-05-08T21:17:56.422877+0000","last_undegraded":"2024-05-08T21:17:56.422877+0000","last_fullsized":"2024-05-08T21:17:56.422877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:55.180686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416194+0000","last_change":"2024-05-08T21:17:02.732967+0000","last_active":"2024-05-08T21:17:56.416194+0000","last_peered":"2024-05-08T21:17:56.416194+0000","last_clean":"2024-05-08T21:17:56.416194+0000","last_became_active":"2024-05-08T21:17:02.732616+0000","last_became_peered":"2024-05-08T21:17:02.732616+0000","last_unstale":"2024-05-08T21:17:56.416194+0000","last_undegraded":"2024-05-08T21:17:56.416194+0000","last_fullsized":"2024-05-08T21:17:56.416194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:50:26.116841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420367+0000","last_change":"2024-05-08T21:17:03.241974+0000","last_active":"2024-05-08T21:17:56.420367+0000","last_peered":"2024-05-08T21:17:56.420367+0000","last_clean":"2024-05-08T21:17:56.420367+0000","last_became_active":"2024-05-08T21:17:03.241627+0000","last_became_peered":"2024-05-08T21:17:03.241627+0000","last_unstale":"2024-05-08T21:17:56.420367+0000","last_undegraded":"2024-05-08T21:17:56.420367+0000","last_fullsized":"2024-05-08T21:17:56.420367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:57.351078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424107+0000","last_change":"2024-05-08T21:17:07.259593+0000","last_active":"2024-05-08T21:17:56.424107+0000","last_peered":"2024-05-08T21:17:56.424107+0000","last_clean":"2024-05-08T21:17:56.424107+0000","last_became_active":"2024-05-08T21:17:07.258362+0000","last_became_peered":"2024-05-08T21:17:07.258362+0000","last_unstale":"2024-05-08T21:17:56.424107+0000","last_undegraded":"2024-05-08T21:17:56.424107+0000","last_fullsized":"2024-05-08T21:17:56.424107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:07.877264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'7","reported_seq":109,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:50.448028+0000","last_change":"2024-05-08T21:17:02.731685+0000","last_active":"2024-05-08T21:18:50.448028+0000","last_peered":"2024-05-08T21:18:50.448028+0000","last_clean":"2024-05-08T21:18:50.448028+0000","last_became_active":"2024-05-08T21:17:02.731293+0000","last_became_peered":"2024-05-08T21:17:02.731293+0000","last_unstale":"2024-05-08T21:18:50.448028+0000","last_undegraded":"2024-05-08T21:18:50.448028+0000","last_fullsized":"2024-05-08T21:18:50.448028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:36.532962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2011,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419974+0000","last_change":"2024-05-08T21:17:03.250090+0000","last_active":"2024-05-08T21:17:56.419974+0000","last_peered":"2024-05-08T21:17:56.419974+0000","last_clean":"2024-05-08T21:17:56.419974+0000","last_became_active":"2024-05-08T21:17:03.249744+0000","last_became_peered":"2024-05-08T21:17:03.249744+0000","last_unstale":"2024-05-08T21:17:56.419974+0000","last_undegraded":"2024-05-08T21:17:56.419974+0000","last_fullsized":"2024-05-08T21:17:56.419974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:43.155040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422926+0000","last_change":"2024-05-08T21:17:07.264101+0000","last_active":"2024-05-08T21:17:56.422926+0000","last_peered":"2024-05-08T21:17:56.422926+0000","last_clean":"2024-05-08T21:17:56.422926+0000","last_became_active":"2024-05-08T21:17:07.263720+0000","last_became_peered":"2024-05-08T21:17:07.263720+0000","last_unstale":"2024-05-08T21:17:56.422926+0000","last_undegraded":"2024-05-08T21:17:56.422926+0000","last_fullsized":"2024-05-08T21:17:56.422926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:16.943044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419108+0000","last_change":"2024-05-08T21:17:02.726763+0000","last_active":"2024-05-08T21:17:56.419108+0000","last_peered":"2024-05-08T21:17:56.419108+0000","last_clean":"2024-05-08T21:17:56.419108+0000","last_became_active":"2024-05-08T21:17:02.726343+0000","last_became_peered":"2024-05-08T21:17:02.726343+0000","last_unstale":"2024-05-08T21:17:56.419108+0000","last_undegraded":"2024-05-08T21:17:56.419108+0000","last_fullsized":"2024-05-08T21:17:56.419108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:03.504667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420530+0000","last_change":"2024-05-08T21:17:03.244468+0000","last_active":"2024-05-08T21:17:56.420530+0000","last_peered":"2024-05-08T21:17:56.420530+0000","last_clean":"2024-05-08T21:17:56.420530+0000","last_became_active":"2024-05-08T21:17:03.244132+0000","last_became_peered":"2024-05-08T21:17:03.244132+0000","last_unstale":"2024-05-08T21:17:56.420530+0000","last_undegraded":"2024-05-08T21:17:56.420530+0000","last_fullsized":"2024-05-08T21:17:56.420530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:17.423657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418261+0000","last_change":"2024-05-08T21:17:07.260981+0000","last_active":"2024-05-08T21:17:56.418261+0000","last_peered":"2024-05-08T21:17:56.418261+0000","last_clean":"2024-05-08T21:17:56.418261+0000","last_became_active":"2024-05-08T21:17:07.260582+0000","last_became_peered":"2024-05-08T21:17:07.260582+0000","last_unstale":"2024-05-08T21:17:56.418261+0000","last_undegraded":"2024-05-08T21:17:56.418261+0000","last_fullsized":"2024-05-08T21:17:56.418261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:25.319411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414755+0000","last_change":"2024-05-08T21:17:02.732531+0000","last_active":"2024-05-08T21:17:56.414755+0000","last_peered":"2024-05-08T21:17:56.414755+0000","last_clean":"2024-05-08T21:17:56.414755+0000","last_became_active":"2024-05-08T21:17:02.732121+0000","last_became_peered":"2024-05-08T21:17:02.732121+0000","last_unstale":"2024-05-08T21:17:56.414755+0000","last_undegraded":"2024-05-08T21:17:56.414755+0000","last_fullsized":"2024-05-08T21:17:56.414755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:25.482458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420635+0000","last_change":"2024-05-08T21:17:03.249676+0000","last_active":"2024-05-08T21:17:56.420635+0000","last_peered":"2024-05-08T21:17:56.420635+0000","last_clean":"2024-05-08T21:17:56.420635+0000","last_became_active":"2024-05-08T21:17:03.249120+0000","last_became_peered":"2024-05-08T21:17:03.249120+0000","last_unstale":"2024-05-08T21:17:56.420635+0000","last_undegraded":"2024-05-08T21:17:56.420635+0000","last_fullsized":"2024-05-08T21:17:56.420635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:58:40.678800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418654+0000","last_change":"2024-05-08T21:17:03.245585+0000","last_active":"2024-05-08T21:17:56.418654+0000","last_peered":"2024-05-08T21:17:56.418654+0000","last_clean":"2024-05-08T21:17:56.418654+0000","last_became_active":"2024-05-08T21:17:03.245274+0000","last_became_peered":"2024-05-08T21:17:03.245274+0000","last_unstale":"2024-05-08T21:17:56.418654+0000","last_undegraded":"2024-05-08T21:17:56.418654+0000","last_fullsized":"2024-05-08T21:17:56.418654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:32.003962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423037+0000","last_change":"2024-05-08T21:17:03.246622+0000","last_active":"2024-05-08T21:17:56.423037+0000","last_peered":"2024-05-08T21:17:56.423037+0000","last_clean":"2024-05-08T21:17:56.423037+0000","last_became_active":"2024-05-08T21:17:03.246309+0000","last_became_peered":"2024-05-08T21:17:03.246309+0000","last_unstale":"2024-05-08T21:17:56.423037+0000","last_undegraded":"2024-05-08T21:17:56.423037+0000","last_fullsized":"2024-05-08T21:17:56.423037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:32.439389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422776+0000","last_change":"2024-05-08T21:17:03.243460+0000","last_active":"2024-05-08T21:17:56.422776+0000","last_peered":"2024-05-08T21:17:56.422776+0000","last_clean":"2024-05-08T21:17:56.422776+0000","last_became_active":"2024-05-08T21:17:03.243097+0000","last_became_peered":"2024-05-08T21:17:03.243097+0000","last_unstale":"2024-05-08T21:17:56.422776+0000","last_undegraded":"2024-05-08T21:17:56.422776+0000","last_fullsized":"2024-05-08T21:17:56.422776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:52:14.492180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423038+0000","last_change":"2024-05-08T21:17:03.245077+0000","last_active":"2024-05-08T21:17:56.423038+0000","last_peered":"2024-05-08T21:17:56.423038+0000","last_clean":"2024-05-08T21:17:56.423038+0000","last_became_active":"2024-05-08T21:17:03.244719+0000","last_became_peered":"2024-05-08T21:17:03.244719+0000","last_unstale":"2024-05-08T21:17:56.423038+0000","last_undegraded":"2024-05-08T21:17:56.423038+0000","last_fullsized":"2024-05-08T21:17:56.423038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.388318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415101+0000","last_change":"2024-05-08T21:17:03.245261+0000","last_active":"2024-05-08T21:17:56.415101+0000","last_peered":"2024-05-08T21:17:56.415101+0000","last_clean":"2024-05-08T21:17:56.415101+0000","last_became_active":"2024-05-08T21:17:03.244892+0000","last_became_peered":"2024-05-08T21:17:03.244892+0000","last_unstale":"2024-05-08T21:17:56.415101+0000","last_undegraded":"2024-05-08T21:17:56.415101+0000","last_fullsized":"2024-05-08T21:17:56.415101+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:21.075634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418251+0000","last_change":"2024-05-08T21:17:03.246250+0000","last_active":"2024-05-08T21:17:56.418251+0000","last_peered":"2024-05-08T21:17:56.418251+0000","last_clean":"2024-05-08T21:17:56.418251+0000","last_became_active":"2024-05-08T21:17:03.245807+0000","last_became_peered":"2024-05-08T21:17:03.245807+0000","last_unstale":"2024-05-08T21:17:56.418251+0000","last_undegraded":"2024-05-08T21:17:56.418251+0000","last_fullsized":"2024-05-08T21:17:56.418251+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:20.618260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423477+0000","last_change":"2024-05-08T21:17:03.254559+0000","last_active":"2024-05-08T21:17:56.423477+0000","last_peered":"2024-05-08T21:17:56.423477+0000","last_clean":"2024-05-08T21:17:56.423477+0000","last_became_active":"2024-05-08T21:17:03.254231+0000","last_became_peered":"2024-05-08T21:17:03.254231+0000","last_unstale":"2024-05-08T21:17:56.423477+0000","last_undegraded":"2024-05-08T21:17:56.423477+0000","last_fullsized":"2024-05-08T21:17:56.423477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:47.996927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418065+0000","last_change":"2024-05-08T21:17:03.242479+0000","last_active":"2024-05-08T21:17:56.418065+0000","last_peered":"2024-05-08T21:17:56.418065+0000","last_clean":"2024-05-08T21:17:56.418065+0000","last_became_active":"2024-05-08T21:17:03.241397+0000","last_became_peered":"2024-05-08T21:17:03.241397+0000","last_unstale":"2024-05-08T21:17:56.418065+0000","last_undegraded":"2024-05-08T21:17:56.418065+0000","last_fullsized":"2024-05-08T21:17:56.418065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:39.029139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419220+0000","last_change":"2024-05-08T21:17:03.244208+0000","last_active":"2024-05-08T21:17:56.419220+0000","last_peered":"2024-05-08T21:17:56.419220+0000","last_clean":"2024-05-08T21:17:56.419220+0000","last_became_active":"2024-05-08T21:17:03.243828+0000","last_became_peered":"2024-05-08T21:17:03.243828+0000","last_unstale":"2024-05-08T21:17:56.419220+0000","last_undegraded":"2024-05-08T21:17:56.419220+0000","last_fullsized":"2024-05-08T21:17:56.419220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:39.936750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418012+0000","last_change":"2024-05-08T21:17:03.240135+0000","last_active":"2024-05-08T21:17:56.418012+0000","last_peered":"2024-05-08T21:17:56.418012+0000","last_clean":"2024-05-08T21:17:56.418012+0000","last_became_active":"2024-05-08T21:17:03.239757+0000","last_became_peered":"2024-05-08T21:17:03.239757+0000","last_unstale":"2024-05-08T21:17:56.418012+0000","last_undegraded":"2024-05-08T21:17:56.418012+0000","last_fullsized":"2024-05-08T21:17:56.418012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:46.066111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424382+0000","last_change":"2024-05-08T21:17:03.256254+0000","last_active":"2024-05-08T21:17:56.424382+0000","last_peered":"2024-05-08T21:17:56.424382+0000","last_clean":"2024-05-08T21:17:56.424382+0000","last_became_active":"2024-05-08T21:17:03.255310+0000","last_became_peered":"2024-05-08T21:17:03.255310+0000","last_unstale":"2024-05-08T21:17:56.424382+0000","last_undegraded":"2024-05-08T21:17:56.424382+0000","last_fullsized":"2024-05-08T21:17:56.424382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:59.571958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412843+0000","last_change":"2024-05-08T21:17:03.241901+0000","last_active":"2024-05-08T21:17:56.412843+0000","last_peered":"2024-05-08T21:17:56.412843+0000","last_clean":"2024-05-08T21:17:56.412843+0000","last_became_active":"2024-05-08T21:17:03.241402+0000","last_became_peered":"2024-05-08T21:17:03.241402+0000","last_unstale":"2024-05-08T21:17:56.412843+0000","last_undegraded":"2024-05-08T21:17:56.412843+0000","last_fullsized":"2024-05-08T21:17:56.412843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:59.202987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417882+0000","last_change":"2024-05-08T21:17:03.243050+0000","last_active":"2024-05-08T21:17:56.417882+0000","last_peered":"2024-05-08T21:17:56.417882+0000","last_clean":"2024-05-08T21:17:56.417882+0000","last_became_active":"2024-05-08T21:17:03.240676+0000","last_became_peered":"2024-05-08T21:17:03.240676+0000","last_unstale":"2024-05-08T21:17:56.417882+0000","last_undegraded":"2024-05-08T21:17:56.417882+0000","last_fullsized":"2024-05-08T21:17:56.417882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:19:53.981559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71392,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":833,"num_read_kb":493,"num_write":251,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1246,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514528,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":484195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76800000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65800000000000003,"15min":0.66300000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83699999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67100000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.498,"15min":0.504},"min":{"1min":0.34699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69499999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.54700000000000004,"15min":0.55500000000000005},"min":{"1min":0.32700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.64000000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":4,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88900000000000001,"5min":1.034,"15min":1.034},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.748,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68100000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68999999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80100000000000005}]},{"osd":6,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.625,"15min":0.63900000000000001},"min":{"1min":0.43099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.82899999999999996,"5min":1.085,"15min":1.085},"last":0.78200000000000003}]},{"osd":7,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.56000000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92400000000000004,"5min":1.228,"15min":1.228},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67700000000000005,"15min":0.67800000000000005},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79900000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77000000000000002}]},{"osd":8,"last update":"Wed May 8 21:18:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.84699999999999998}]},{"osd":9,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77300000000000002,"15min":0.77200000000000002},"min":{"1min":0.66100000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93999999999999995,"5min":1.113,"15min":1.113},"last":0.88},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.80700000000000005},"min":{"1min":0.65600000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.95499999999999996,"5min":1.27,"15min":1.27},"last":0.88900000000000001}]},{"osd":10,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.747,"15min":0.75600000000000001},"min":{"1min":0.57099999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.84199999999999997,"5min":1.419,"15min":1.419},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.64500000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90400000000000003,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.83099999999999996}]}]},{"osd":10,"up_from":69,"seq":296352743459,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36848,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708304,"statfs":{"total":95995035648,"available":95957303296,"internally_reserved":0,"allocated":1679360,"data_stored":1133072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.71299999999999997}]},{"osd":1,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42899999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65100000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46300000000000002,"15min":0.46200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68100000000000005,"5min":0.748,"15min":0.748},"last":0.42399999999999999}]},{"osd":2,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.373,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58399999999999996,"15min":0.58299999999999996},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81599999999999995,"5min":0.88,"15min":0.88},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.626,"15min":0.627},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61299999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85599999999999998,"5min":1.248,"15min":1.248},"last":0.46100000000000002}]},{"osd":5,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.47599999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97199999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.45100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91400000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57999999999999996}]},{"osd":6,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95999999999999996,"5min":1.014,"15min":1.014},"last":0.627},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69499999999999995,"15min":0.69699999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94699999999999995,"5min":1.048,"15min":1.048},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":8,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46300000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.745,"15min":0.751},"min":{"1min":0.47799999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90700000000000003,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.753,"15min":0.755},"min":{"1min":0.56799999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0229999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77300000000000002,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.93000000000000005,"5min":1.419,"15min":1.419},"last":0.73299999999999998}]},{"osd":11,"last update":"Wed May 8 21:18:33 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.81999999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.749,"15min":0.753},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.876,"5min":0.995,"15min":0.995},"last":0.74399999999999999}]}]},{"osd":9,"up_from":64,"seq":274877906982,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":487380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.48099999999999998,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65000000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.5,"15min":0.50800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63700000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.34799999999999998}]},{"osd":1,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.84399999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81100000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76500000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63400000000000001,"15min":0.64300000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":1.048,"15min":1.048},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.496,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79200000000000004}]},{"osd":5,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.77800000000000002,"15min":0.79500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92300000000000004,"5min":1.234,"15min":1.234},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.056,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.875},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.47099999999999997}]},{"osd":7,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.81599999999999995,"15min":0.83099999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96099999999999997,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.84199999999999997,"15min":0.86399999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.67500000000000004}]},{"osd":8,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.80100000000000005,"15min":0.82299999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93600000000000005,"5min":1.101,"15min":1.101},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.88600000000000001,"15min":0.90700000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.004,"5min":1.589,"15min":1.589},"last":0.82599999999999996}]},{"osd":10,"last update":"Wed May 8 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.84399999999999997,"15min":0.85599999999999998},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0409999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.745},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.875,"15min":0.88700000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1140000000000001,"5min":1.153,"15min":1.153},"last":0.625}]},{"osd":11,"last update":"Wed May 8 21:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.77100000000000002}]}]},{"osd":8,"up_from":56,"seq":240518168619,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":482659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.42699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.501}]},{"osd":1,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59699999999999998,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.52800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63900000000000001}]},{"osd":2,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.47899999999999998,"15min":0.47199999999999998},"min":{"1min":0.35999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.75600000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.47899999999999998,"15min":0.45000000000000001},"min":{"1min":0.375,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47999999999999998}]},{"osd":3,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57099999999999995,"15min":0.55800000000000005},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89200000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.62,"15min":0.62},"min":{"1min":0.501,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84599999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59799999999999998}]},{"osd":4,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.49399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85899999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.69099999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54400000000000004,"15min":0.51400000000000001},"min":{"1min":0.41299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.46999999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66400000000000003,"15min":0.64900000000000002},"min":{"1min":0.51700000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.70399999999999996,"15min":0.67900000000000005},"min":{"1min":0.66700000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.67500000000000004,"15min":0.64300000000000002},"min":{"1min":0.53600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.71699999999999997,"15min":0.70499999999999996},"min":{"1min":0.55800000000000005,"5min":0.373,"15min":0.373},"max":{"1min":0.95799999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76400000000000001}]},{"osd":9,"last update":"Wed May 8 21:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70199999999999996,"15min":0.68899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.745,"15min":0.73499999999999999},"min":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.96999999999999997,"5min":1.04,"15min":1.04},"last":0.79300000000000004}]},{"osd":10,"last update":"Wed May 8 21:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.60099999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.61199999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94899999999999995,"5min":1.008,"15min":1.008},"last":0.77200000000000002}]},{"osd":11,"last update":"Wed May 8 21:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.752},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.83299999999999996}]}]},{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1080508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41099999999999998,"15min":0.40000000000000002},"min":{"1min":0.32400000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.54400000000000004,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.40200000000000002},"min":{"1min":0.317,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61699999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.496}]},{"osd":2,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58799999999999997,"15min":0.57299999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56499999999999995,"15min":0.54600000000000004},"min":{"1min":0.46999999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.89900000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52800000000000002}]},{"osd":3,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71499999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72299999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.36599999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.379,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61799999999999999,"5min":0.745,"15min":0.745},"last":0.438},{"interface":"front","average":{"1min":0.499,"5min":0.48799999999999999,"15min":0.49099999999999999},"min":{"1min":0.35599999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71099999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.38500000000000001}]},{"osd":5,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.45000000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.76500000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.622,"15min":0.60099999999999998},"min":{"1min":0.51700000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.873,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.65100000000000002,"15min":0.65600000000000003},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83999999999999997,"5min":0.87,"15min":0.87},"last":0.41799999999999998}]},{"osd":8,"last update":"Wed May 8 21:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.66900000000000004,"15min":0.65500000000000003},"min":{"1min":0.54400000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88500000000000001,"5min":1,"15min":1},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.67200000000000004,"15min":0.65200000000000002},"min":{"1min":0.57399999999999995,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95399999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67200000000000004}]},{"osd":9,"last update":"Wed May 8 21:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67200000000000004,"15min":0.67600000000000005},"min":{"1min":0.47799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.81999999999999995,"5min":1.008,"15min":1.008},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.86499999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65600000000000003}]},{"osd":10,"last update":"Wed May 8 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.57199999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.66300000000000003,"15min":0.65000000000000002},"min":{"1min":0.625,"5min":0.437,"15min":0.437},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.70699999999999996}]},{"osd":11,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.622}]}]},{"osd":6,"up_from":44,"seq":188978561075,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":479236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.502,"5min":0.434,"15min":0.434},"max":{"1min":1,"5min":1,"15min":1},"last":0.67800000000000005}]},{"osd":2,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.50900000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.61399999999999999}]},{"osd":3,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.59799999999999998}]},{"osd":4,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.45700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.71599999999999997}]},{"osd":5,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.56000000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.72399999999999998,"15min":0.71199999999999997},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.77300000000000002}]},{"osd":7,"last update":"Wed May 8 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.76400000000000001,"15min":0.76100000000000001},"min":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.85599999999999998}]},{"osd":8,"last update":"Wed May 8 21:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.75},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.59599999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.82399999999999995}]},{"osd":9,"last update":"Wed May 8 21:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.80400000000000005,"15min":0.79300000000000004},"min":{"1min":0.54800000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.76700000000000002},"min":{"1min":0.625,"5min":0.498,"15min":0.498},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.86699999999999999}]},{"osd":10,"last update":"Wed May 8 21:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.76500000000000001,"15min":0.754},"min":{"1min":0.65000000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.81299999999999994,"15min":0.79900000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.81100000000000005}]},{"osd":11,"last update":"Wed May 8 21:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.80300000000000005}]}]},{"osd":1,"up_from":12,"seq":51539607619,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":543723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56699999999999995,"15min":0.53300000000000003},"min":{"1min":0.44700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.745,"5min":1.548,"15min":1.548},"last":0.84999999999999998}]},{"osd":2,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.54000000000000004,"15min":0.50700000000000001},"min":{"1min":0.41699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.52900000000000003,"15min":0.496},"min":{"1min":0.40600000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80900000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.77300000000000002}]},{"osd":3,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.51100000000000001},"min":{"1min":0.42799999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.57999999999999996,"15min":0.52500000000000002},"min":{"1min":0.432,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85999999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.73399999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45000000000000001,"15min":0.41599999999999998},"min":{"1min":0.35799999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.63200000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.42499999999999999},"min":{"1min":0.41899999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.66300000000000003,"5min":0.753,"15min":0.753},"last":0.60599999999999998}]},{"osd":5,"last update":"Wed May 8 21:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.877,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.54400000000000004},"min":{"1min":0.495,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88300000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.82299999999999995}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64300000000000002,"15min":0.61199999999999999},"min":{"1min":0.502,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89100000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.92000000000000004}]},{"osd":7,"last update":"Wed May 8 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.40899999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84399999999999997,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.55300000000000005,"15min":0.51600000000000001},"min":{"1min":0.443,"5min":0.308,"15min":0.308},"max":{"1min":0.94499999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78900000000000003}]},{"osd":8,"last update":"Wed May 8 21:18:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.67100000000000004,"15min":0.63900000000000001},"min":{"1min":0.48499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.99099999999999999,"5min":1.032,"15min":1.032},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72099999999999997,"15min":0.70299999999999996},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.607,"15min":1.607},"last":0.80400000000000005}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70099999999999996,"15min":0.69099999999999995},"min":{"1min":0.54300000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73299999999999998,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.877}]},{"osd":10,"last update":"Wed May 8 21:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.752,"15min":0.747},"min":{"1min":0.54800000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95499999999999996,"5min":1.004,"15min":1.004},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.71799999999999997,"15min":0.70799999999999996},"min":{"1min":0.63300000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.89000000000000001}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.90900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":1011712,"data_stored":481085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 21:18:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.41599999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.84299999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.46400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed May 8 21:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.748,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67400000000000004}]},{"osd":3,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.439},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48899999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85199999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.627,"15min":0.58999999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69499999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.64700000000000002,"15min":0.61699999999999999},"min":{"1min":0.48799999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.624,"15min":0.60799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93600000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.751},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.42999999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89700000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.623}]},{"osd":7,"last update":"Wed May 8 21:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.85699999999999998}]},{"osd":8,"last update":"Wed May 8 21:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.54000000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.63100000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78400000000000003}]},{"osd":9,"last update":"Wed May 8 21:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77400000000000002},"min":{"1min":0.65100000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97799999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Wed May 8 21:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81799999999999995,"15min":0.81299999999999994},"min":{"1min":0.68500000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.86199999999999999,"15min":0.85199999999999998},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.82399999999999995}]},{"osd":11,"last update":"Wed May 8 21:18:34 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.71899999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.88},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.71599999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.83899999999999997}]}]},{"osd":2,"up_from":17,"seq":73014444097,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1069056,"data_stored":543042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.56799999999999995,"15min":0.52100000000000002},"min":{"1min":0.32600000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":6.7270000000000003,"5min":6.7270000000000003,"15min":6.7270000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.51200000000000001},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47999999999999998}]},{"osd":1,"last update":"Wed May 8 21:17:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.44500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.42899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70599999999999996}]},{"osd":3,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.437,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.45900000000000002,"15min":0.41799999999999998},"min":{"1min":0.36499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.45900000000000002,"15min":0.42299999999999999},"min":{"1min":0.30299999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.753,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58399999999999996}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.55400000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.212,"5min":0.71199999999999997,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":7.7779999999999996,"5min":7.7779999999999996,"15min":7.7779999999999996},"last":0.754}]},{"osd":7,"last update":"Wed May 8 21:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.83799999999999997}]},{"osd":8,"last update":"Wed May 8 21:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.39000000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95199999999999996,"5min":0.995,"15min":0.995},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59399999999999997,"15min":0.56799999999999995},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71999999999999997}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.67700000000000005,"15min":0.65800000000000003},"min":{"1min":0.66800000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.56799999999999995,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.82399999999999995}]},{"osd":10,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67800000000000005,"15min":0.66300000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.753,"15min":0.73499999999999999},"min":{"1min":0.629,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.89200000000000002}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.871},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.81299999999999994}]}]},{"osd":3,"up_from":24,"seq":103079215165,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":999424,"data_stored":478496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.40400000000000003,"15min":0.36799999999999999},"min":{"1min":0.29899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42099999999999999,"15min":0.39100000000000001},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76500000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54200000000000004,"15min":0.51600000000000001},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80700000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.52100000000000002},"min":{"1min":0.42299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.628}]},{"osd":4,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.54500000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.56599999999999995},"min":{"1min":0.48799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed May 8 21:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59999999999999998,"15min":0.56699999999999995},"min":{"1min":0.48699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78700000000000003,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96799999999999997,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Wed May 8 21:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.504,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.73699999999999999}]},{"osd":7,"last update":"Wed May 8 21:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.53300000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.58999999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.78200000000000003}]},{"osd":8,"last update":"Wed May 8 21:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79800000000000004}]},{"osd":9,"last update":"Wed May 8 21:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.85599999999999998}]},{"osd":10,"last update":"Wed May 8 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77300000000000002,"15min":0.77000000000000002},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80900000000000005,"15min":0.81100000000000005},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.872}]},{"osd":11,"last update":"Wed May 8 21:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.91600000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986233,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":484380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.36599999999999999},"min":{"1min":0.32100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.442},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.40200000000000002,"15min":0.39300000000000002},"min":{"1min":0.34599999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59399999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82599999999999996,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.53600000000000003},"min":{"1min":0.48099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71599999999999997}]},{"osd":3,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56999999999999995,"15min":0.53800000000000003},"min":{"1min":0.47599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73899999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.52500000000000002},"min":{"1min":0.41499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59499999999999997,"15min":0.56299999999999994},"min":{"1min":0.44500000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.79500000000000004,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.46000000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82399999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66900000000000004}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59999999999999998,"15min":0.57999999999999996},"min":{"1min":0.51200000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82399999999999995,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.48599999999999999}]},{"osd":7,"last update":"Wed May 8 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52200000000000002,"15min":0.48799999999999999},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.81599999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56100000000000005,"15min":0.52100000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.91100000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.59099999999999997}]},{"osd":8,"last update":"Wed May 8 21:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66400000000000003,"15min":0.64400000000000002},"min":{"1min":0.59499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.83099999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.69999999999999996},"min":{"1min":0.59499999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70399999999999996}]},{"osd":9,"last update":"Wed May 8 21:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.747,"15min":0.747},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.622},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.876,"5min":1.347,"15min":1.347},"last":0.61399999999999999}]},{"osd":10,"last update":"Wed May 8 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69399999999999995,"15min":0.69699999999999995},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70599999999999996,"15min":0.71499999999999997},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":1.323,"15min":1.323},"last":0.63400000000000001}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.78700000000000003}]}]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36784,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708368,"statfs":{"total":95995035648,"available":95957368832,"internally_reserved":0,"allocated":1613824,"data_stored":1075798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.38100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77100000000000002,"5min":0.877,"15min":0.877},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.374,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.442}]},{"osd":1,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.434,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73699999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.751,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.72099999999999997}]},{"osd":2,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40699999999999997,"15min":0.40000000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61799999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.40699999999999997,"15min":0.40799999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61599999999999999,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.45700000000000002}]},{"osd":3,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.54800000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89800000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.373},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.42099999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.45300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81699999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed May 8 21:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.75700000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.623,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93300000000000005,"5min":1.01,"15min":1.01},"last":0.502}]},{"osd":7,"last update":"Wed May 8 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.623,"15min":0.623},"min":{"1min":0.42999999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82699999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88900000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66700000000000004}]},{"osd":8,"last update":"Wed May 8 21:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73499999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48699999999999999}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.55100000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88900000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52200000000000002}]},{"osd":10,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90700000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.51900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T21:18:54.607 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-08T21:18:54.607 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T21:18:54.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:54 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1391443389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.747+0000 7f3c5fa64700 1 -- 172.21.15.33:0/520455015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c58100f60 msgr2=0x7f3c581013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.747+0000 7f3c5fa64700 1 --2- 172.21.15.33:0/520455015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c58100f60 0x7f3c581013c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3c480099c0 tx=0x7f3c48009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.747+0000 7f3c5fa64700 1 -- 172.21.15.33:0/520455015 shutdown_connections 2024-05-08T21:18:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.747+0000 7f3c5fa64700 1 --2- 172.21.15.33:0/520455015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c58101900 0x7f3c581081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.747+0000 7f3c5fa64700 1 --2- 172.21.15.33:0/520455015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c580ffd70 0x7f3c58100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.747+0000 7f3c5fa64700 1 --2- 172.21.15.33:0/520455015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c58100f60 0x7f3c581013c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.747+0000 7f3c5fa64700 1 -- 172.21.15.33:0/520455015 >> 172.21.15.33:0/520455015 conn(0x7f3c580fb2e0 msgr2=0x7f3c580fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.747+0000 7f3c5fa64700 1 -- 172.21.15.33:0/520455015 shutdown_connections 2024-05-08T21:18:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.748+0000 7f3c5fa64700 1 -- 172.21.15.33:0/520455015 wait complete. 2024-05-08T21:18:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.748+0000 7f3c5fa64700 1 Processor -- start 2024-05-08T21:18:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.748+0000 7f3c5fa64700 1 -- start start 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.748+0000 7f3c5fa64700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c580ffd70 0x7f3c58103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.748+0000 7f3c5fa64700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c58100f60 0x7f3c58102340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.748+0000 7f3c5fa64700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c58101900 0x7f3c58102880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.748+0000 7f3c5fa64700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58103020 con 0x7f3c580ffd70 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.748+0000 7f3c5fa64700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58103190 con 0x7f3c58101900 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.748+0000 7f3c5fa64700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c581a0d70 con 0x7f3c58100f60 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5d800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c580ffd70 0x7f3c58103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5d800700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c580ffd70 0x7f3c58103cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54212/0 (socket says 172.21.15.33:54212) 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5d800700 1 -- 172.21.15.33:0/3214027009 learned_addr learned my addr 172.21.15.33:0/3214027009 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5cfff700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c58100f60 0x7f3c58102340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5e001700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c58101900 0x7f3c58102880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5d800700 1 -- 172.21.15.33:0/3214027009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c58100f60 msgr2=0x7f3c58102340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5d800700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c58100f60 0x7f3c58102340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5d800700 1 -- 172.21.15.33:0/3214027009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c58101900 msgr2=0x7f3c58102880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5d800700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c58101900 0x7f3c58102880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5d800700 1 -- 172.21.15.33:0/3214027009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c480095f0 con 0x7f3c580ffd70 2024-05-08T21:18:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5d800700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c580ffd70 0x7f3c58103cb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400d980 tx=0x7f3c5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c4e7fc700 1 -- 172.21.15.33:0/3214027009 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5400f890 con 0x7f3c580ffd70 2024-05-08T21:18:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c4e7fc700 1 -- 172.21.15.33:0/3214027009 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c5400fe80 con 0x7f3c580ffd70 2024-05-08T21:18:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5fa64700 1 -- 172.21.15.33:0/3214027009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c581a0fa0 con 0x7f3c580ffd70 2024-05-08T21:18:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c4e7fc700 1 -- 172.21.15.33:0/3214027009 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54019360 con 0x7f3c580ffd70 2024-05-08T21:18:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.749+0000 7f3c5fa64700 1 -- 172.21.15.33:0/3214027009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c581a1420 con 0x7f3c580ffd70 2024-05-08T21:18:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.750+0000 7f3c5fa64700 1 -- 172.21.15.33:0/3214027009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c58068160 con 0x7f3c580ffd70 2024-05-08T21:18:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.751+0000 7f3c4e7fc700 1 -- 172.21.15.33:0/3214027009 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f3c5400f9f0 con 0x7f3c580ffd70 2024-05-08T21:18:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.752+0000 7f3c4e7fc700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3c440a11b0 0x7f3c440a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.752+0000 7f3c5cfff700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3c440a11b0 0x7f3c440a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.753+0000 7f3c4e7fc700 1 -- 172.21.15.33:0/3214027009 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3c540cc770 con 0x7f3c580ffd70 2024-05-08T21:18:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.753+0000 7f3c5cfff700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3c440a11b0 0x7f3c440a3670 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009990 tx=0x7f3c48005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.758+0000 7f3c4e7fc700 1 -- 172.21.15.33:0/3214027009 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3c54095a90 con 0x7f3c580ffd70 2024-05-08T21:18:54.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:54 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1391443389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:54.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:54 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1391443389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.971+0000 7f3c5fa64700 1 -- 172.21.15.33:0/3214027009 --> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c58069bf0 con 0x7f3c440a11b0 2024-05-08T21:18:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.984+0000 7f3c4e7fc700 1 -- 172.21.15.33:0/3214027009 <== mgr.14152 v2:172.21.15.33:6800/2288189532 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427361 (secure 0 0 0) 0x7f3c58069bf0 con 0x7f3c440a11b0 2024-05-08T21:18:54.992 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.989+0000 7f3c5fa64700 1 -- 172.21.15.33:0/3214027009 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3c440a11b0 msgr2=0x7f3c440a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.989+0000 7f3c5fa64700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3c440a11b0 0x7f3c440a3670 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009990 tx=0x7f3c48005e80 comp rx=0 tx=0).stop 2024-05-08T21:18:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.989+0000 7f3c5fa64700 1 -- 172.21.15.33:0/3214027009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c580ffd70 msgr2=0x7f3c58103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.989+0000 7f3c5fa64700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c580ffd70 0x7f3c58103cb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400d980 tx=0x7f3c5400dc90 comp rx=0 tx=0).stop 2024-05-08T21:18:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.989+0000 7f3c5fa64700 1 -- 172.21.15.33:0/3214027009 shutdown_connections 2024-05-08T21:18:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.989+0000 7f3c5fa64700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3c440a11b0 0x7f3c440a3670 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.989+0000 7f3c5fa64700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c58101900 0x7f3c58102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.989+0000 7f3c5fa64700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c580ffd70 0x7f3c58103cb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.989+0000 7f3c5fa64700 1 --2- 172.21.15.33:0/3214027009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c58100f60 0x7f3c58102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.989+0000 7f3c5fa64700 1 -- 172.21.15.33:0/3214027009 >> 172.21.15.33:0/3214027009 conn(0x7f3c580fb2e0 msgr2=0x7f3c580694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.989+0000 7f3c5fa64700 1 -- 172.21.15.33:0/3214027009 shutdown_connections 2024-05-08T21:18:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:54.989+0000 7f3c5fa64700 1 -- 172.21.15.33:0/3214027009 wait complete. 2024-05-08T21:18:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-05-08T21:18:55.005 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-05-08T21:18:54.145246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661994,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":959,"num_read_kb":602,"num_write":434,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1181982,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":359,"ondisk_log_size":359,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374768,"kb_used_data":13696,"kb_used_omap":18,"kb_used_meta":360941,"kb_avail":1124567056,"statfs":{"total":1151940427776,"available":1151556665344,"internally_reserved":0,"allocated":14024704,"data_stored":7753574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19130,"internal_metadata":369603910},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2480,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":60,"num_write":12,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003270"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418982+0000","last_change":"2024-05-08T21:17:03.243903+0000","last_active":"2024-05-08T21:17:56.418982+0000","last_peered":"2024-05-08T21:17:56.418982+0000","last_clean":"2024-05-08T21:17:56.418982+0000","last_became_active":"2024-05-08T21:17:03.240821+0000","last_became_peered":"2024-05-08T21:17:03.240821+0000","last_unstale":"2024-05-08T21:17:56.418982+0000","last_undegraded":"2024-05-08T21:17:56.418982+0000","last_fullsized":"2024-05-08T21:17:56.418982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:01.878614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423327+0000","last_change":"2024-05-08T21:17:03.243119+0000","last_active":"2024-05-08T21:17:56.423327+0000","last_peered":"2024-05-08T21:17:56.423327+0000","last_clean":"2024-05-08T21:17:56.423327+0000","last_became_active":"2024-05-08T21:17:03.242772+0000","last_became_peered":"2024-05-08T21:17:03.242772+0000","last_unstale":"2024-05-08T21:17:56.423327+0000","last_undegraded":"2024-05-08T21:17:56.423327+0000","last_fullsized":"2024-05-08T21:17:56.423327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:20.550646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423287+0000","last_change":"2024-05-08T21:17:03.237843+0000","last_active":"2024-05-08T21:17:56.423287+0000","last_peered":"2024-05-08T21:17:56.423287+0000","last_clean":"2024-05-08T21:17:56.423287+0000","last_became_active":"2024-05-08T21:17:03.237567+0000","last_became_peered":"2024-05-08T21:17:03.237567+0000","last_unstale":"2024-05-08T21:17:56.423287+0000","last_undegraded":"2024-05-08T21:17:56.423287+0000","last_fullsized":"2024-05-08T21:17:56.423287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:23.856461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.430448+0000","last_change":"2024-05-08T21:17:57.430448+0000","last_active":"2024-05-08T21:17:57.430448+0000","last_peered":"2024-05-08T21:17:57.430448+0000","last_clean":"2024-05-08T21:17:57.430448+0000","last_became_active":"2024-05-08T21:17:57.430195+0000","last_became_peered":"2024-05-08T21:17:57.430195+0000","last_unstale":"2024-05-08T21:17:57.430448+0000","last_undegraded":"2024-05-08T21:17:57.430448+0000","last_fullsized":"2024-05-08T21:17:57.430448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:41.536610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416067+0000","last_change":"2024-05-08T21:17:03.252246+0000","last_active":"2024-05-08T21:17:56.416067+0000","last_peered":"2024-05-08T21:17:56.416067+0000","last_clean":"2024-05-08T21:17:56.416067+0000","last_became_active":"2024-05-08T21:17:03.251898+0000","last_became_peered":"2024-05-08T21:17:03.251898+0000","last_unstale":"2024-05-08T21:17:56.416067+0000","last_undegraded":"2024-05-08T21:17:56.416067+0000","last_fullsized":"2024-05-08T21:17:56.416067+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:08.996033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417704+0000","last_change":"2024-05-08T21:17:03.238422+0000","last_active":"2024-05-08T21:17:56.417704+0000","last_peered":"2024-05-08T21:17:56.417704+0000","last_clean":"2024-05-08T21:17:56.417704+0000","last_became_active":"2024-05-08T21:17:03.238096+0000","last_became_peered":"2024-05-08T21:17:03.238096+0000","last_unstale":"2024-05-08T21:17:56.417704+0000","last_undegraded":"2024-05-08T21:17:56.417704+0000","last_fullsized":"2024-05-08T21:17:56.417704+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:59:54.635089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419866+0000","last_change":"2024-05-08T21:17:03.244315+0000","last_active":"2024-05-08T21:17:56.419866+0000","last_peered":"2024-05-08T21:17:56.419866+0000","last_clean":"2024-05-08T21:17:56.419866+0000","last_became_active":"2024-05-08T21:17:03.243939+0000","last_became_peered":"2024-05-08T21:17:03.243939+0000","last_unstale":"2024-05-08T21:17:56.419866+0000","last_undegraded":"2024-05-08T21:17:56.419866+0000","last_fullsized":"2024-05-08T21:17:56.419866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:24.371495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415520+0000","last_change":"2024-05-08T21:17:03.240144+0000","last_active":"2024-05-08T21:17:56.415520+0000","last_peered":"2024-05-08T21:17:56.415520+0000","last_clean":"2024-05-08T21:17:56.415520+0000","last_became_active":"2024-05-08T21:17:03.239493+0000","last_became_peered":"2024-05-08T21:17:03.239493+0000","last_unstale":"2024-05-08T21:17:56.415520+0000","last_undegraded":"2024-05-08T21:17:56.415520+0000","last_fullsized":"2024-05-08T21:17:56.415520+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:21.374357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424317+0000","last_change":"2024-05-08T21:17:03.256706+0000","last_active":"2024-05-08T21:17:56.424317+0000","last_peered":"2024-05-08T21:17:56.424317+0000","last_clean":"2024-05-08T21:17:56.424317+0000","last_became_active":"2024-05-08T21:17:03.255345+0000","last_became_peered":"2024-05-08T21:17:03.255345+0000","last_unstale":"2024-05-08T21:17:56.424317+0000","last_undegraded":"2024-05-08T21:17:56.424317+0000","last_fullsized":"2024-05-08T21:17:56.424317+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:09:50.082797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.430334+0000","last_change":"2024-05-08T21:17:57.430334+0000","last_active":"2024-05-08T21:17:57.430334+0000","last_peered":"2024-05-08T21:17:57.430334+0000","last_clean":"2024-05-08T21:17:57.430334+0000","last_became_active":"2024-05-08T21:17:57.429924+0000","last_became_peered":"2024-05-08T21:17:57.429924+0000","last_unstale":"2024-05-08T21:17:57.430334+0000","last_undegraded":"2024-05-08T21:17:57.430334+0000","last_fullsized":"2024-05-08T21:17:57.430334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:13:03.901755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419810+0000","last_change":"2024-05-08T21:17:03.253996+0000","last_active":"2024-05-08T21:17:56.419810+0000","last_peered":"2024-05-08T21:17:56.419810+0000","last_clean":"2024-05-08T21:17:56.419810+0000","last_became_active":"2024-05-08T21:17:03.253686+0000","last_became_peered":"2024-05-08T21:17:03.253686+0000","last_unstale":"2024-05-08T21:17:56.419810+0000","last_undegraded":"2024-05-08T21:17:56.419810+0000","last_fullsized":"2024-05-08T21:17:56.419810+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:31:17.995692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423092+0000","last_change":"2024-05-08T21:17:03.254357+0000","last_active":"2024-05-08T21:17:56.423092+0000","last_peered":"2024-05-08T21:17:56.423092+0000","last_clean":"2024-05-08T21:17:56.423092+0000","last_became_active":"2024-05-08T21:17:03.254013+0000","last_became_peered":"2024-05-08T21:17:03.254013+0000","last_unstale":"2024-05-08T21:17:56.423092+0000","last_undegraded":"2024-05-08T21:17:56.423092+0000","last_fullsized":"2024-05-08T21:17:56.423092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:21.480612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.413995+0000","last_change":"2024-05-08T21:17:03.244190+0000","last_active":"2024-05-08T21:17:56.413995+0000","last_peered":"2024-05-08T21:17:56.413995+0000","last_clean":"2024-05-08T21:17:56.413995+0000","last_became_active":"2024-05-08T21:17:03.243989+0000","last_became_peered":"2024-05-08T21:17:03.243989+0000","last_unstale":"2024-05-08T21:17:56.413995+0000","last_undegraded":"2024-05-08T21:17:56.413995+0000","last_fullsized":"2024-05-08T21:17:56.413995+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:34:45.059343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422938+0000","last_change":"2024-05-08T21:17:03.242996+0000","last_active":"2024-05-08T21:17:56.422938+0000","last_peered":"2024-05-08T21:17:56.422938+0000","last_clean":"2024-05-08T21:17:56.422938+0000","last_became_active":"2024-05-08T21:17:03.242648+0000","last_became_peered":"2024-05-08T21:17:03.242648+0000","last_unstale":"2024-05-08T21:17:56.422938+0000","last_undegraded":"2024-05-08T21:17:56.422938+0000","last_fullsized":"2024-05-08T21:17:56.422938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:47:19.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422908+0000","last_change":"2024-05-08T21:17:03.244970+0000","last_active":"2024-05-08T21:17:56.422908+0000","last_peered":"2024-05-08T21:17:56.422908+0000","last_clean":"2024-05-08T21:17:56.422908+0000","last_became_active":"2024-05-08T21:17:03.244604+0000","last_became_peered":"2024-05-08T21:17:03.244604+0000","last_unstale":"2024-05-08T21:17:56.422908+0000","last_undegraded":"2024-05-08T21:17:56.422908+0000","last_fullsized":"2024-05-08T21:17:56.422908+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:32.484155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422727+0000","last_change":"2024-05-08T21:17:03.254628+0000","last_active":"2024-05-08T21:17:56.422727+0000","last_peered":"2024-05-08T21:17:56.422727+0000","last_clean":"2024-05-08T21:17:56.422727+0000","last_became_active":"2024-05-08T21:17:03.254305+0000","last_became_peered":"2024-05-08T21:17:03.254305+0000","last_unstale":"2024-05-08T21:17:56.422727+0000","last_undegraded":"2024-05-08T21:17:56.422727+0000","last_fullsized":"2024-05-08T21:17:56.422727+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:00:52.727821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418732+0000","last_change":"2024-05-08T21:17:03.243406+0000","last_active":"2024-05-08T21:17:56.418732+0000","last_peered":"2024-05-08T21:17:56.418732+0000","last_clean":"2024-05-08T21:17:56.418732+0000","last_became_active":"2024-05-08T21:17:03.240379+0000","last_became_peered":"2024-05-08T21:17:03.240379+0000","last_unstale":"2024-05-08T21:17:56.418732+0000","last_undegraded":"2024-05-08T21:17:56.418732+0000","last_fullsized":"2024-05-08T21:17:56.418732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:49.897459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420035+0000","last_change":"2024-05-08T21:17:03.239664+0000","last_active":"2024-05-08T21:17:56.420035+0000","last_peered":"2024-05-08T21:17:56.420035+0000","last_clean":"2024-05-08T21:17:56.420035+0000","last_became_active":"2024-05-08T21:17:03.239347+0000","last_became_peered":"2024-05-08T21:17:03.239347+0000","last_unstale":"2024-05-08T21:17:56.420035+0000","last_undegraded":"2024-05-08T21:17:56.420035+0000","last_fullsized":"2024-05-08T21:17:56.420035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:06:37.557835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423543+0000","last_change":"2024-05-08T21:17:03.249424+0000","last_active":"2024-05-08T21:17:56.423543+0000","last_peered":"2024-05-08T21:17:56.423543+0000","last_clean":"2024-05-08T21:17:56.423543+0000","last_became_active":"2024-05-08T21:17:03.248971+0000","last_became_peered":"2024-05-08T21:17:03.248971+0000","last_unstale":"2024-05-08T21:17:56.423543+0000","last_undegraded":"2024-05-08T21:17:56.423543+0000","last_fullsized":"2024-05-08T21:17:56.423543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:04:08.096762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422849+0000","last_change":"2024-05-08T21:17:07.272414+0000","last_active":"2024-05-08T21:17:56.422849+0000","last_peered":"2024-05-08T21:17:56.422849+0000","last_clean":"2024-05-08T21:17:56.422849+0000","last_became_active":"2024-05-08T21:17:07.271660+0000","last_became_peered":"2024-05-08T21:17:07.271660+0000","last_unstale":"2024-05-08T21:17:56.422849+0000","last_undegraded":"2024-05-08T21:17:56.422849+0000","last_fullsized":"2024-05-08T21:17:56.422849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:50:15.697910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'14","reported_seq":133,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:47.545496+0000","last_change":"2024-05-08T21:17:57.483017+0000","last_active":"2024-05-08T21:18:47.545496+0000","last_peered":"2024-05-08T21:18:47.545496+0000","last_clean":"2024-05-08T21:18:47.545496+0000","last_became_active":"2024-05-08T21:17:57.473420+0000","last_became_peered":"2024-05-08T21:17:57.473420+0000","last_unstale":"2024-05-08T21:18:47.545496+0000","last_undegraded":"2024-05-08T21:18:47.545496+0000","last_fullsized":"2024-05-08T21:18:47.545496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:40:42.727117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417325+0000","last_change":"2024-05-08T21:17:03.240780+0000","last_active":"2024-05-08T21:17:56.417325+0000","last_peered":"2024-05-08T21:17:56.417325+0000","last_clean":"2024-05-08T21:17:56.417325+0000","last_became_active":"2024-05-08T21:17:03.240472+0000","last_became_peered":"2024-05-08T21:17:03.240472+0000","last_unstale":"2024-05-08T21:17:56.417325+0000","last_undegraded":"2024-05-08T21:17:56.417325+0000","last_fullsized":"2024-05-08T21:17:56.417325+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:09:00.089760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.421952+0000","last_change":"2024-05-08T21:17:07.272308+0000","last_active":"2024-05-08T21:17:56.421952+0000","last_peered":"2024-05-08T21:17:56.421952+0000","last_clean":"2024-05-08T21:17:56.421952+0000","last_became_active":"2024-05-08T21:17:07.271869+0000","last_became_peered":"2024-05-08T21:17:07.271869+0000","last_unstale":"2024-05-08T21:17:56.421952+0000","last_undegraded":"2024-05-08T21:17:56.421952+0000","last_fullsized":"2024-05-08T21:17:56.421952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:06:31.145404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417727+0000","last_change":"2024-05-08T21:17:02.736446+0000","last_active":"2024-05-08T21:17:56.417727+0000","last_peered":"2024-05-08T21:17:56.417727+0000","last_clean":"2024-05-08T21:17:56.417727+0000","last_became_active":"2024-05-08T21:17:02.736187+0000","last_became_peered":"2024-05-08T21:17:02.736187+0000","last_unstale":"2024-05-08T21:17:56.417727+0000","last_undegraded":"2024-05-08T21:17:56.417727+0000","last_fullsized":"2024-05-08T21:17:56.417727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:29.302558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423206+0000","last_change":"2024-05-08T21:17:03.251896+0000","last_active":"2024-05-08T21:17:56.423206+0000","last_peered":"2024-05-08T21:17:56.423206+0000","last_clean":"2024-05-08T21:17:56.423206+0000","last_became_active":"2024-05-08T21:17:03.251495+0000","last_became_peered":"2024-05-08T21:17:03.251495+0000","last_unstale":"2024-05-08T21:17:56.423206+0000","last_undegraded":"2024-05-08T21:17:56.423206+0000","last_fullsized":"2024-05-08T21:17:56.423206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:50.774848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422223+0000","last_change":"2024-05-08T21:17:07.272150+0000","last_active":"2024-05-08T21:17:56.422223+0000","last_peered":"2024-05-08T21:17:56.422223+0000","last_clean":"2024-05-08T21:17:56.422223+0000","last_became_active":"2024-05-08T21:17:07.271727+0000","last_became_peered":"2024-05-08T21:17:07.271727+0000","last_unstale":"2024-05-08T21:17:56.422223+0000","last_undegraded":"2024-05-08T21:17:56.422223+0000","last_fullsized":"2024-05-08T21:17:56.422223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:10:29.058277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416860+0000","last_change":"2024-05-08T21:17:02.733611+0000","last_active":"2024-05-08T21:17:56.416860+0000","last_peered":"2024-05-08T21:17:56.416860+0000","last_clean":"2024-05-08T21:17:56.416860+0000","last_became_active":"2024-05-08T21:17:02.733306+0000","last_became_peered":"2024-05-08T21:17:02.733306+0000","last_unstale":"2024-05-08T21:17:56.416860+0000","last_undegraded":"2024-05-08T21:17:56.416860+0000","last_fullsized":"2024-05-08T21:17:56.416860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:11:33.304753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423294+0000","last_change":"2024-05-08T21:17:03.248480+0000","last_active":"2024-05-08T21:17:56.423294+0000","last_peered":"2024-05-08T21:17:56.423294+0000","last_clean":"2024-05-08T21:17:56.423294+0000","last_became_active":"2024-05-08T21:17:03.248126+0000","last_became_peered":"2024-05-08T21:17:03.248126+0000","last_unstale":"2024-05-08T21:17:56.423294+0000","last_undegraded":"2024-05-08T21:17:56.423294+0000","last_fullsized":"2024-05-08T21:17:56.423294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:08.785761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422472+0000","last_change":"2024-05-08T21:17:07.272344+0000","last_active":"2024-05-08T21:17:56.422472+0000","last_peered":"2024-05-08T21:17:56.422472+0000","last_clean":"2024-05-08T21:17:56.422472+0000","last_became_active":"2024-05-08T21:17:07.271629+0000","last_became_peered":"2024-05-08T21:17:07.271629+0000","last_unstale":"2024-05-08T21:17:56.422472+0000","last_undegraded":"2024-05-08T21:17:56.422472+0000","last_fullsized":"2024-05-08T21:17:56.422472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:15.052748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416295+0000","last_change":"2024-05-08T21:17:02.730364+0000","last_active":"2024-05-08T21:17:56.416295+0000","last_peered":"2024-05-08T21:17:56.416295+0000","last_clean":"2024-05-08T21:17:56.416295+0000","last_became_active":"2024-05-08T21:17:02.728854+0000","last_became_peered":"2024-05-08T21:17:02.728854+0000","last_unstale":"2024-05-08T21:17:56.416295+0000","last_undegraded":"2024-05-08T21:17:56.416295+0000","last_fullsized":"2024-05-08T21:17:56.416295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:03:09.552016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419025+0000","last_change":"2024-05-08T21:17:03.252713+0000","last_active":"2024-05-08T21:17:56.419025+0000","last_peered":"2024-05-08T21:17:56.419025+0000","last_clean":"2024-05-08T21:17:56.419025+0000","last_became_active":"2024-05-08T21:17:03.252388+0000","last_became_peered":"2024-05-08T21:17:03.252388+0000","last_unstale":"2024-05-08T21:17:56.419025+0000","last_undegraded":"2024-05-08T21:17:56.419025+0000","last_fullsized":"2024-05-08T21:17:56.419025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:09.114064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423183+0000","last_change":"2024-05-08T21:17:07.261112+0000","last_active":"2024-05-08T21:17:56.423183+0000","last_peered":"2024-05-08T21:17:56.423183+0000","last_clean":"2024-05-08T21:17:56.423183+0000","last_became_active":"2024-05-08T21:17:07.260713+0000","last_became_peered":"2024-05-08T21:17:07.260713+0000","last_unstale":"2024-05-08T21:17:56.423183+0000","last_undegraded":"2024-05-08T21:17:56.423183+0000","last_fullsized":"2024-05-08T21:17:56.423183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:38:37.262113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":122,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:49.510444+0000","last_change":"2024-05-08T21:17:57.436812+0000","last_active":"2024-05-08T21:18:49.510444+0000","last_peered":"2024-05-08T21:18:49.510444+0000","last_clean":"2024-05-08T21:18:49.510444+0000","last_became_active":"2024-05-08T21:17:57.432431+0000","last_became_peered":"2024-05-08T21:17:57.432431+0000","last_unstale":"2024-05-08T21:18:49.510444+0000","last_undegraded":"2024-05-08T21:18:49.510444+0000","last_fullsized":"2024-05-08T21:18:49.510444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:02:46.062273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1070,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":574,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417176+0000","last_change":"2024-05-08T21:17:03.251245+0000","last_active":"2024-05-08T21:17:56.417176+0000","last_peered":"2024-05-08T21:17:56.417176+0000","last_clean":"2024-05-08T21:17:56.417176+0000","last_became_active":"2024-05-08T21:17:03.250296+0000","last_became_peered":"2024-05-08T21:17:03.250296+0000","last_unstale":"2024-05-08T21:17:56.417176+0000","last_undegraded":"2024-05-08T21:17:56.417176+0000","last_fullsized":"2024-05-08T21:17:56.417176+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:28.391927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423437+0000","last_change":"2024-05-08T21:17:07.266752+0000","last_active":"2024-05-08T21:17:56.423437+0000","last_peered":"2024-05-08T21:17:56.423437+0000","last_clean":"2024-05-08T21:17:56.423437+0000","last_became_active":"2024-05-08T21:17:07.266367+0000","last_became_peered":"2024-05-08T21:17:07.266367+0000","last_unstale":"2024-05-08T21:17:56.423437+0000","last_undegraded":"2024-05-08T21:17:56.423437+0000","last_fullsized":"2024-05-08T21:17:56.423437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:49.874073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417579+0000","last_change":"2024-05-08T21:17:02.728113+0000","last_active":"2024-05-08T21:17:56.417579+0000","last_peered":"2024-05-08T21:17:56.417579+0000","last_clean":"2024-05-08T21:17:56.417579+0000","last_became_active":"2024-05-08T21:17:02.727754+0000","last_became_peered":"2024-05-08T21:17:02.727754+0000","last_unstale":"2024-05-08T21:17:56.417579+0000","last_undegraded":"2024-05-08T21:17:56.417579+0000","last_fullsized":"2024-05-08T21:17:56.417579+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:01:53.789218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418802+0000","last_change":"2024-05-08T21:17:03.245482+0000","last_active":"2024-05-08T21:17:56.418802+0000","last_peered":"2024-05-08T21:17:56.418802+0000","last_clean":"2024-05-08T21:17:56.418802+0000","last_became_active":"2024-05-08T21:17:03.245144+0000","last_became_peered":"2024-05-08T21:17:03.245144+0000","last_unstale":"2024-05-08T21:17:56.418802+0000","last_undegraded":"2024-05-08T21:17:56.418802+0000","last_fullsized":"2024-05-08T21:17:56.418802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:00:04.051234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418636+0000","last_change":"2024-05-08T21:17:07.268505+0000","last_active":"2024-05-08T21:17:56.418636+0000","last_peered":"2024-05-08T21:17:56.418636+0000","last_clean":"2024-05-08T21:17:56.418636+0000","last_became_active":"2024-05-08T21:17:07.268157+0000","last_became_peered":"2024-05-08T21:17:07.268157+0000","last_unstale":"2024-05-08T21:17:56.418636+0000","last_undegraded":"2024-05-08T21:17:56.418636+0000","last_fullsized":"2024-05-08T21:17:56.418636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:55.220521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412606+0000","last_change":"2024-05-08T21:17:02.727264+0000","last_active":"2024-05-08T21:17:56.412606+0000","last_peered":"2024-05-08T21:17:56.412606+0000","last_clean":"2024-05-08T21:17:56.412606+0000","last_became_active":"2024-05-08T21:17:02.726922+0000","last_became_peered":"2024-05-08T21:17:02.726922+0000","last_unstale":"2024-05-08T21:17:56.412606+0000","last_undegraded":"2024-05-08T21:17:56.412606+0000","last_fullsized":"2024-05-08T21:17:56.412606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:27.410231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415635+0000","last_change":"2024-05-08T21:17:03.233910+0000","last_active":"2024-05-08T21:17:56.415635+0000","last_peered":"2024-05-08T21:17:56.415635+0000","last_clean":"2024-05-08T21:17:56.415635+0000","last_became_active":"2024-05-08T21:17:03.233562+0000","last_became_peered":"2024-05-08T21:17:03.233562+0000","last_unstale":"2024-05-08T21:17:56.415635+0000","last_undegraded":"2024-05-08T21:17:56.415635+0000","last_fullsized":"2024-05-08T21:17:56.415635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:40.309400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423244+0000","last_change":"2024-05-08T21:17:07.264992+0000","last_active":"2024-05-08T21:17:56.423244+0000","last_peered":"2024-05-08T21:17:56.423244+0000","last_clean":"2024-05-08T21:17:56.423244+0000","last_became_active":"2024-05-08T21:17:07.264498+0000","last_became_peered":"2024-05-08T21:17:07.264498+0000","last_unstale":"2024-05-08T21:17:56.423244+0000","last_undegraded":"2024-05-08T21:17:56.423244+0000","last_fullsized":"2024-05-08T21:17:56.423244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:23:26.498855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418593+0000","last_change":"2024-05-08T21:17:02.725531+0000","last_active":"2024-05-08T21:17:56.418593+0000","last_peered":"2024-05-08T21:17:56.418593+0000","last_clean":"2024-05-08T21:17:56.418593+0000","last_became_active":"2024-05-08T21:17:02.725235+0000","last_became_peered":"2024-05-08T21:17:02.725235+0000","last_unstale":"2024-05-08T21:17:56.418593+0000","last_undegraded":"2024-05-08T21:17:56.418593+0000","last_fullsized":"2024-05-08T21:17:56.418593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:16:27.265590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419530+0000","last_change":"2024-05-08T21:17:03.236393+0000","last_active":"2024-05-08T21:17:56.419530+0000","last_peered":"2024-05-08T21:17:56.419530+0000","last_clean":"2024-05-08T21:17:56.419530+0000","last_became_active":"2024-05-08T21:17:03.236092+0000","last_became_peered":"2024-05-08T21:17:03.236092+0000","last_unstale":"2024-05-08T21:17:56.419530+0000","last_undegraded":"2024-05-08T21:17:56.419530+0000","last_fullsized":"2024-05-08T21:17:56.419530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:08:58.232669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417799+0000","last_change":"2024-05-08T21:17:07.271374+0000","last_active":"2024-05-08T21:17:56.417799+0000","last_peered":"2024-05-08T21:17:56.417799+0000","last_clean":"2024-05-08T21:17:56.417799+0000","last_became_active":"2024-05-08T21:17:07.270778+0000","last_became_peered":"2024-05-08T21:17:07.270778+0000","last_unstale":"2024-05-08T21:17:56.417799+0000","last_undegraded":"2024-05-08T21:17:56.417799+0000","last_fullsized":"2024-05-08T21:17:56.417799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:50.212213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'6","reported_seq":184,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:50.447209+0000","last_change":"2024-05-08T21:17:02.725624+0000","last_active":"2024-05-08T21:18:50.447209+0000","last_peered":"2024-05-08T21:18:50.447209+0000","last_clean":"2024-05-08T21:18:50.447209+0000","last_became_active":"2024-05-08T21:17:02.725367+0000","last_became_peered":"2024-05-08T21:17:02.725367+0000","last_unstale":"2024-05-08T21:18:50.447209+0000","last_undegraded":"2024-05-08T21:18:50.447209+0000","last_fullsized":"2024-05-08T21:18:50.447209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:53:19.573931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":160,"num_read_kb":160,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423993+0000","last_change":"2024-05-08T21:17:03.248729+0000","last_active":"2024-05-08T21:17:56.423993+0000","last_peered":"2024-05-08T21:17:56.423993+0000","last_clean":"2024-05-08T21:17:56.423993+0000","last_became_active":"2024-05-08T21:17:03.247899+0000","last_became_peered":"2024-05-08T21:17:03.247899+0000","last_unstale":"2024-05-08T21:17:56.423993+0000","last_undegraded":"2024-05-08T21:17:56.423993+0000","last_fullsized":"2024-05-08T21:17:56.423993+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:27:57.012249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419291+0000","last_change":"2024-05-08T21:17:07.261257+0000","last_active":"2024-05-08T21:17:56.419291+0000","last_peered":"2024-05-08T21:17:56.419291+0000","last_clean":"2024-05-08T21:17:56.419291+0000","last_became_active":"2024-05-08T21:17:07.260785+0000","last_became_peered":"2024-05-08T21:17:07.260785+0000","last_unstale":"2024-05-08T21:17:56.419291+0000","last_undegraded":"2024-05-08T21:17:56.419291+0000","last_fullsized":"2024-05-08T21:17:56.419291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:17:58.469134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419901+0000","last_change":"2024-05-08T21:17:02.729569+0000","last_active":"2024-05-08T21:17:56.419901+0000","last_peered":"2024-05-08T21:17:56.419901+0000","last_clean":"2024-05-08T21:17:56.419901+0000","last_became_active":"2024-05-08T21:17:02.729096+0000","last_became_peered":"2024-05-08T21:17:02.729096+0000","last_unstale":"2024-05-08T21:17:56.419901+0000","last_undegraded":"2024-05-08T21:17:56.419901+0000","last_fullsized":"2024-05-08T21:17:56.419901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:30.668169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423406+0000","last_change":"2024-05-08T21:17:03.246387+0000","last_active":"2024-05-08T21:17:56.423406+0000","last_peered":"2024-05-08T21:17:56.423406+0000","last_clean":"2024-05-08T21:17:56.423406+0000","last_became_active":"2024-05-08T21:17:03.244658+0000","last_became_peered":"2024-05-08T21:17:03.244658+0000","last_unstale":"2024-05-08T21:17:56.423406+0000","last_undegraded":"2024-05-08T21:17:56.423406+0000","last_fullsized":"2024-05-08T21:17:56.423406+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:01:02.602080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.473101+0000","last_change":"2024-05-08T21:17:57.473101+0000","last_active":"2024-05-08T21:17:57.473101+0000","last_peered":"2024-05-08T21:17:57.473101+0000","last_clean":"2024-05-08T21:17:57.473101+0000","last_became_active":"2024-05-08T21:17:57.472563+0000","last_became_peered":"2024-05-08T21:17:57.472563+0000","last_unstale":"2024-05-08T21:17:57.473101+0000","last_undegraded":"2024-05-08T21:17:57.473101+0000","last_fullsized":"2024-05-08T21:17:57.473101+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:23.662588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418913+0000","last_change":"2024-05-08T21:17:02.732738+0000","last_active":"2024-05-08T21:17:56.418913+0000","last_peered":"2024-05-08T21:17:56.418913+0000","last_clean":"2024-05-08T21:17:56.418913+0000","last_became_active":"2024-05-08T21:17:02.731733+0000","last_became_peered":"2024-05-08T21:17:02.731733+0000","last_unstale":"2024-05-08T21:17:56.418913+0000","last_undegraded":"2024-05-08T21:17:56.418913+0000","last_fullsized":"2024-05-08T21:17:56.418913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:26:24.201171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423479+0000","last_change":"2024-05-08T21:17:03.254405+0000","last_active":"2024-05-08T21:17:56.423479+0000","last_peered":"2024-05-08T21:17:56.423479+0000","last_clean":"2024-05-08T21:17:56.423479+0000","last_became_active":"2024-05-08T21:17:03.252137+0000","last_became_peered":"2024-05-08T21:17:03.252137+0000","last_unstale":"2024-05-08T21:17:56.423479+0000","last_undegraded":"2024-05-08T21:17:56.423479+0000","last_fullsized":"2024-05-08T21:17:56.423479+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:15.472225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.431574+0000","last_change":"2024-05-08T21:17:57.431091+0000","last_active":"2024-05-08T21:17:57.431574+0000","last_peered":"2024-05-08T21:17:57.431574+0000","last_clean":"2024-05-08T21:17:57.431574+0000","last_became_active":"2024-05-08T21:17:57.430668+0000","last_became_peered":"2024-05-08T21:17:57.430668+0000","last_unstale":"2024-05-08T21:17:57.431574+0000","last_undegraded":"2024-05-08T21:17:57.431574+0000","last_fullsized":"2024-05-08T21:17:57.431574+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:09:33.208801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"87'38","reported_seq":225,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:50.511512+0000","last_change":"2024-05-08T21:17:02.723975+0000","last_active":"2024-05-08T21:18:50.511512+0000","last_peered":"2024-05-08T21:18:50.511512+0000","last_clean":"2024-05-08T21:18:50.511512+0000","last_became_active":"2024-05-08T21:17:02.723631+0000","last_became_peered":"2024-05-08T21:17:02.723631+0000","last_unstale":"2024-05-08T21:18:50.511512+0000","last_undegraded":"2024-05-08T21:18:50.511512+0000","last_fullsized":"2024-05-08T21:18:50.511512+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:17.676007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62785,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":169,"num_read_kb":145,"num_write":40,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416534+0000","last_change":"2024-05-08T21:17:03.251570+0000","last_active":"2024-05-08T21:17:56.416534+0000","last_peered":"2024-05-08T21:17:56.416534+0000","last_clean":"2024-05-08T21:17:56.416534+0000","last_became_active":"2024-05-08T21:17:03.251134+0000","last_became_peered":"2024-05-08T21:17:03.251134+0000","last_unstale":"2024-05-08T21:17:56.416534+0000","last_undegraded":"2024-05-08T21:17:56.416534+0000","last_fullsized":"2024-05-08T21:17:56.416534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:28:09.339958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419635+0000","last_change":"2024-05-08T21:17:07.272051+0000","last_active":"2024-05-08T21:17:56.419635+0000","last_peered":"2024-05-08T21:17:56.419635+0000","last_clean":"2024-05-08T21:17:56.419635+0000","last_became_active":"2024-05-08T21:17:07.271616+0000","last_became_peered":"2024-05-08T21:17:07.271616+0000","last_unstale":"2024-05-08T21:17:56.419635+0000","last_undegraded":"2024-05-08T21:17:56.419635+0000","last_fullsized":"2024-05-08T21:17:56.419635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:46:58.364731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423259+0000","last_change":"2024-05-08T21:17:02.736000+0000","last_active":"2024-05-08T21:17:56.423259+0000","last_peered":"2024-05-08T21:17:56.423259+0000","last_clean":"2024-05-08T21:17:56.423259+0000","last_became_active":"2024-05-08T21:17:02.735629+0000","last_became_peered":"2024-05-08T21:17:02.735629+0000","last_unstale":"2024-05-08T21:17:56.423259+0000","last_undegraded":"2024-05-08T21:17:56.423259+0000","last_fullsized":"2024-05-08T21:17:56.423259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:21.842135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412729+0000","last_change":"2024-05-08T21:17:03.237144+0000","last_active":"2024-05-08T21:17:56.412729+0000","last_peered":"2024-05-08T21:17:56.412729+0000","last_clean":"2024-05-08T21:17:56.412729+0000","last_became_active":"2024-05-08T21:17:03.236792+0000","last_became_peered":"2024-05-08T21:17:03.236792+0000","last_unstale":"2024-05-08T21:17:56.412729+0000","last_undegraded":"2024-05-08T21:17:56.412729+0000","last_fullsized":"2024-05-08T21:17:56.412729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:32:04.520986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.627656+0000","last_change":"2024-05-08T21:17:57.627656+0000","last_active":"2024-05-08T21:17:57.627656+0000","last_peered":"2024-05-08T21:17:57.627656+0000","last_clean":"2024-05-08T21:17:57.627656+0000","last_became_active":"2024-05-08T21:17:57.627108+0000","last_became_peered":"2024-05-08T21:17:57.627108+0000","last_unstale":"2024-05-08T21:17:57.627656+0000","last_undegraded":"2024-05-08T21:17:57.627656+0000","last_fullsized":"2024-05-08T21:17:57.627656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:24.025145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419802+0000","last_change":"2024-05-08T21:17:02.727748+0000","last_active":"2024-05-08T21:17:56.419802+0000","last_peered":"2024-05-08T21:17:56.419802+0000","last_clean":"2024-05-08T21:17:56.419802+0000","last_became_active":"2024-05-08T21:17:02.727444+0000","last_became_peered":"2024-05-08T21:17:02.727444+0000","last_unstale":"2024-05-08T21:17:56.419802+0000","last_undegraded":"2024-05-08T21:17:56.419802+0000","last_fullsized":"2024-05-08T21:17:56.419802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:47.939235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420245+0000","last_change":"2024-05-08T21:17:03.247185+0000","last_active":"2024-05-08T21:17:56.420245+0000","last_peered":"2024-05-08T21:17:56.420245+0000","last_clean":"2024-05-08T21:17:56.420245+0000","last_became_active":"2024-05-08T21:17:03.246880+0000","last_became_peered":"2024-05-08T21:17:03.246880+0000","last_unstale":"2024-05-08T21:17:56.420245+0000","last_undegraded":"2024-05-08T21:17:56.420245+0000","last_fullsized":"2024-05-08T21:17:56.420245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:38.978353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420018+0000","last_change":"2024-05-08T21:17:07.273710+0000","last_active":"2024-05-08T21:17:56.420018+0000","last_peered":"2024-05-08T21:17:56.420018+0000","last_clean":"2024-05-08T21:17:56.420018+0000","last_became_active":"2024-05-08T21:17:07.273119+0000","last_became_peered":"2024-05-08T21:17:07.273119+0000","last_unstale":"2024-05-08T21:17:56.420018+0000","last_undegraded":"2024-05-08T21:17:56.420018+0000","last_fullsized":"2024-05-08T21:17:56.420018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:15:30.589440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":109,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:51.429542+0000","last_change":"2024-05-08T21:17:02.725508+0000","last_active":"2024-05-08T21:18:51.429542+0000","last_peered":"2024-05-08T21:18:51.429542+0000","last_clean":"2024-05-08T21:18:51.429542+0000","last_became_active":"2024-05-08T21:17:02.725107+0000","last_became_peered":"2024-05-08T21:17:02.725107+0000","last_unstale":"2024-05-08T21:18:51.429542+0000","last_undegraded":"2024-05-08T21:18:51.429542+0000","last_fullsized":"2024-05-08T21:18:51.429542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:48:13.503702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1036,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420302+0000","last_change":"2024-05-08T21:17:03.245372+0000","last_active":"2024-05-08T21:17:56.420302+0000","last_peered":"2024-05-08T21:17:56.420302+0000","last_clean":"2024-05-08T21:17:56.420302+0000","last_became_active":"2024-05-08T21:17:03.245054+0000","last_became_peered":"2024-05-08T21:17:03.245054+0000","last_unstale":"2024-05-08T21:17:56.420302+0000","last_undegraded":"2024-05-08T21:17:56.420302+0000","last_fullsized":"2024-05-08T21:17:56.420302+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:30:25.900846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414821+0000","last_change":"2024-05-08T21:17:07.272081+0000","last_active":"2024-05-08T21:17:56.414821+0000","last_peered":"2024-05-08T21:17:56.414821+0000","last_clean":"2024-05-08T21:17:56.414821+0000","last_became_active":"2024-05-08T21:17:07.271560+0000","last_became_peered":"2024-05-08T21:17:07.271560+0000","last_unstale":"2024-05-08T21:17:56.414821+0000","last_undegraded":"2024-05-08T21:17:56.414821+0000","last_fullsized":"2024-05-08T21:17:56.414821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:45:00.490798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416027+0000","last_change":"2024-05-08T21:17:02.734420+0000","last_active":"2024-05-08T21:17:56.416027+0000","last_peered":"2024-05-08T21:17:56.416027+0000","last_clean":"2024-05-08T21:17:56.416027+0000","last_became_active":"2024-05-08T21:17:02.734071+0000","last_became_peered":"2024-05-08T21:17:02.734071+0000","last_unstale":"2024-05-08T21:17:56.416027+0000","last_undegraded":"2024-05-08T21:17:56.416027+0000","last_fullsized":"2024-05-08T21:17:56.416027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:26:28.537464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419127+0000","last_change":"2024-05-08T21:17:03.250706+0000","last_active":"2024-05-08T21:17:56.419127+0000","last_peered":"2024-05-08T21:17:56.419127+0000","last_clean":"2024-05-08T21:17:56.419127+0000","last_became_active":"2024-05-08T21:17:03.250408+0000","last_became_peered":"2024-05-08T21:17:03.250408+0000","last_unstale":"2024-05-08T21:17:56.419127+0000","last_undegraded":"2024-05-08T21:17:56.419127+0000","last_fullsized":"2024-05-08T21:17:56.419127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:25:43.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418530+0000","last_change":"2024-05-08T21:17:07.272066+0000","last_active":"2024-05-08T21:17:56.418530+0000","last_peered":"2024-05-08T21:17:56.418530+0000","last_clean":"2024-05-08T21:17:56.418530+0000","last_became_active":"2024-05-08T21:17:07.271628+0000","last_became_peered":"2024-05-08T21:17:07.271628+0000","last_unstale":"2024-05-08T21:17:56.418530+0000","last_undegraded":"2024-05-08T21:17:56.418530+0000","last_fullsized":"2024-05-08T21:17:56.418530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:27.731995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.424263+0000","last_change":"2024-05-08T21:17:02.728464+0000","last_active":"2024-05-08T21:17:57.424263+0000","last_peered":"2024-05-08T21:17:57.424263+0000","last_clean":"2024-05-08T21:17:57.424263+0000","last_became_active":"2024-05-08T21:17:02.727914+0000","last_became_peered":"2024-05-08T21:17:02.727914+0000","last_unstale":"2024-05-08T21:17:57.424263+0000","last_undegraded":"2024-05-08T21:17:57.424263+0000","last_fullsized":"2024-05-08T21:17:57.424263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:39:55.948661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418448+0000","last_change":"2024-05-08T21:17:03.245578+0000","last_active":"2024-05-08T21:17:56.418448+0000","last_peered":"2024-05-08T21:17:56.418448+0000","last_clean":"2024-05-08T21:17:56.418448+0000","last_became_active":"2024-05-08T21:17:03.245256+0000","last_became_peered":"2024-05-08T21:17:03.245256+0000","last_unstale":"2024-05-08T21:17:56.418448+0000","last_undegraded":"2024-05-08T21:17:56.418448+0000","last_fullsized":"2024-05-08T21:17:56.418448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:26.250395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417048+0000","last_change":"2024-05-08T21:17:07.439525+0000","last_active":"2024-05-08T21:17:56.417048+0000","last_peered":"2024-05-08T21:17:56.417048+0000","last_clean":"2024-05-08T21:17:56.417048+0000","last_became_active":"2024-05-08T21:17:07.438987+0000","last_became_peered":"2024-05-08T21:17:07.438987+0000","last_unstale":"2024-05-08T21:17:56.417048+0000","last_undegraded":"2024-05-08T21:17:56.417048+0000","last_fullsized":"2024-05-08T21:17:56.417048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:57:45.373981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.432999+0000","last_change":"2024-05-08T21:17:57.432999+0000","last_active":"2024-05-08T21:17:57.432999+0000","last_peered":"2024-05-08T21:17:57.432999+0000","last_clean":"2024-05-08T21:17:57.432999+0000","last_became_active":"2024-05-08T21:17:57.432567+0000","last_became_peered":"2024-05-08T21:17:57.432567+0000","last_unstale":"2024-05-08T21:17:57.432999+0000","last_undegraded":"2024-05-08T21:17:57.432999+0000","last_fullsized":"2024-05-08T21:17:57.432999+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:33:00.188046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417907+0000","last_change":"2024-05-08T21:17:03.235556+0000","last_active":"2024-05-08T21:17:56.417907+0000","last_peered":"2024-05-08T21:17:56.417907+0000","last_clean":"2024-05-08T21:17:56.417907+0000","last_became_active":"2024-05-08T21:17:03.235214+0000","last_became_peered":"2024-05-08T21:17:03.235214+0000","last_unstale":"2024-05-08T21:17:56.417907+0000","last_undegraded":"2024-05-08T21:17:56.417907+0000","last_fullsized":"2024-05-08T21:17:56.417907+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:02:45.042596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414845+0000","last_change":"2024-05-08T21:17:07.266080+0000","last_active":"2024-05-08T21:17:56.414845+0000","last_peered":"2024-05-08T21:17:56.414845+0000","last_clean":"2024-05-08T21:17:56.414845+0000","last_became_active":"2024-05-08T21:17:07.265605+0000","last_became_peered":"2024-05-08T21:17:07.265605+0000","last_unstale":"2024-05-08T21:17:56.414845+0000","last_undegraded":"2024-05-08T21:17:56.414845+0000","last_fullsized":"2024-05-08T21:17:56.414845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:49:48.387170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:49.549637+0000","last_change":"2024-05-08T21:17:02.728743+0000","last_active":"2024-05-08T21:18:49.549637+0000","last_peered":"2024-05-08T21:18:49.549637+0000","last_clean":"2024-05-08T21:18:49.549637+0000","last_became_active":"2024-05-08T21:17:02.728330+0000","last_became_peered":"2024-05-08T21:17:02.728330+0000","last_unstale":"2024-05-08T21:18:49.549637+0000","last_undegraded":"2024-05-08T21:18:49.549637+0000","last_fullsized":"2024-05-08T21:18:49.549637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:43.865117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1036,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417780+0000","last_change":"2024-05-08T21:17:03.245672+0000","last_active":"2024-05-08T21:17:56.417780+0000","last_peered":"2024-05-08T21:17:56.417780+0000","last_clean":"2024-05-08T21:17:56.417780+0000","last_became_active":"2024-05-08T21:17:03.245318+0000","last_became_peered":"2024-05-08T21:17:03.245318+0000","last_unstale":"2024-05-08T21:17:56.417780+0000","last_undegraded":"2024-05-08T21:17:56.417780+0000","last_fullsized":"2024-05-08T21:17:56.417780+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:48:43.030011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414114+0000","last_change":"2024-05-08T21:17:07.439289+0000","last_active":"2024-05-08T21:17:56.414114+0000","last_peered":"2024-05-08T21:17:56.414114+0000","last_clean":"2024-05-08T21:17:56.414114+0000","last_became_active":"2024-05-08T21:17:07.438874+0000","last_became_peered":"2024-05-08T21:17:07.438874+0000","last_unstale":"2024-05-08T21:17:56.414114+0000","last_undegraded":"2024-05-08T21:17:56.414114+0000","last_fullsized":"2024-05-08T21:17:56.414114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:14:14.820087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419309+0000","last_change":"2024-05-08T21:17:03.247081+0000","last_active":"2024-05-08T21:17:56.419309+0000","last_peered":"2024-05-08T21:17:56.419309+0000","last_clean":"2024-05-08T21:17:56.419309+0000","last_became_active":"2024-05-08T21:17:03.246418+0000","last_became_peered":"2024-05-08T21:17:03.246418+0000","last_unstale":"2024-05-08T21:17:56.419309+0000","last_undegraded":"2024-05-08T21:17:56.419309+0000","last_fullsized":"2024-05-08T21:17:56.419309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:03.993770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":108,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:52.430374+0000","last_change":"2024-05-08T21:17:02.730764+0000","last_active":"2024-05-08T21:18:52.430374+0000","last_peered":"2024-05-08T21:18:52.430374+0000","last_clean":"2024-05-08T21:18:52.430374+0000","last_became_active":"2024-05-08T21:17:02.730494+0000","last_became_peered":"2024-05-08T21:17:02.730494+0000","last_unstale":"2024-05-08T21:18:52.430374+0000","last_undegraded":"2024-05-08T21:18:52.430374+0000","last_fullsized":"2024-05-08T21:18:52.430374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:50.070368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419669+0000","last_change":"2024-05-08T21:17:07.266424+0000","last_active":"2024-05-08T21:17:56.419669+0000","last_peered":"2024-05-08T21:17:56.419669+0000","last_clean":"2024-05-08T21:17:56.419669+0000","last_became_active":"2024-05-08T21:17:07.266144+0000","last_became_peered":"2024-05-08T21:17:07.266144+0000","last_unstale":"2024-05-08T21:17:56.419669+0000","last_undegraded":"2024-05-08T21:17:56.419669+0000","last_fullsized":"2024-05-08T21:17:56.419669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:22.285749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417254+0000","last_change":"2024-05-08T21:17:03.250461+0000","last_active":"2024-05-08T21:17:56.417254+0000","last_peered":"2024-05-08T21:17:56.417254+0000","last_clean":"2024-05-08T21:17:56.417254+0000","last_became_active":"2024-05-08T21:17:03.249974+0000","last_became_peered":"2024-05-08T21:17:03.249974+0000","last_unstale":"2024-05-08T21:17:56.417254+0000","last_undegraded":"2024-05-08T21:17:56.417254+0000","last_fullsized":"2024-05-08T21:17:56.417254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:25:58.759668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422681+0000","last_change":"2024-05-08T21:17:02.725648+0000","last_active":"2024-05-08T21:17:56.422681+0000","last_peered":"2024-05-08T21:17:56.422681+0000","last_clean":"2024-05-08T21:17:56.422681+0000","last_became_active":"2024-05-08T21:17:02.725297+0000","last_became_peered":"2024-05-08T21:17:02.725297+0000","last_unstale":"2024-05-08T21:17:56.422681+0000","last_undegraded":"2024-05-08T21:17:56.422681+0000","last_fullsized":"2024-05-08T21:17:56.422681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:41.462830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":328,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420115+0000","last_change":"2024-05-08T21:16:15.194478+0000","last_active":"2024-05-08T21:17:56.420115+0000","last_peered":"2024-05-08T21:17:56.420115+0000","last_clean":"2024-05-08T21:17:56.420115+0000","last_became_active":"2024-05-08T21:16:15.193964+0000","last_became_peered":"2024-05-08T21:16:15.193964+0000","last_unstale":"2024-05-08T21:17:56.420115+0000","last_undegraded":"2024-05-08T21:17:56.420115+0000","last_fullsized":"2024-05-08T21:17:56.420115+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:13:48.584026+0000","last_clean_scrub_stamp":"2024-05-08T21:13:48.584026+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:28.091200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.628348+0000","last_change":"2024-05-08T21:17:57.627889+0000","last_active":"2024-05-08T21:17:57.628348+0000","last_peered":"2024-05-08T21:17:57.628348+0000","last_clean":"2024-05-08T21:17:57.628348+0000","last_became_active":"2024-05-08T21:17:57.627502+0000","last_became_peered":"2024-05-08T21:17:57.627502+0000","last_unstale":"2024-05-08T21:17:57.628348+0000","last_undegraded":"2024-05-08T21:17:57.628348+0000","last_fullsized":"2024-05-08T21:17:57.628348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:10.025347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6,10],"acting":[8,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412481+0000","last_change":"2024-05-08T21:17:03.246034+0000","last_active":"2024-05-08T21:17:56.412481+0000","last_peered":"2024-05-08T21:17:56.412481+0000","last_clean":"2024-05-08T21:17:56.412481+0000","last_became_active":"2024-05-08T21:17:03.245620+0000","last_became_peered":"2024-05-08T21:17:03.245620+0000","last_unstale":"2024-05-08T21:17:56.412481+0000","last_undegraded":"2024-05-08T21:17:56.412481+0000","last_fullsized":"2024-05-08T21:17:56.412481+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:02.748196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420424+0000","last_change":"2024-05-08T21:17:02.731260+0000","last_active":"2024-05-08T21:17:56.420424+0000","last_peered":"2024-05-08T21:17:56.420424+0000","last_clean":"2024-05-08T21:17:56.420424+0000","last_became_active":"2024-05-08T21:17:02.730879+0000","last_became_peered":"2024-05-08T21:17:02.730879+0000","last_unstale":"2024-05-08T21:17:56.420424+0000","last_undegraded":"2024-05-08T21:17:56.420424+0000","last_fullsized":"2024-05-08T21:17:56.420424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:59.030674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417621+0000","last_change":"2024-05-08T21:17:07.269791+0000","last_active":"2024-05-08T21:17:56.417621+0000","last_peered":"2024-05-08T21:17:56.417621+0000","last_clean":"2024-05-08T21:17:56.417621+0000","last_became_active":"2024-05-08T21:17:07.269345+0000","last_became_peered":"2024-05-08T21:17:07.269345+0000","last_unstale":"2024-05-08T21:17:56.417621+0000","last_undegraded":"2024-05-08T21:17:56.417621+0000","last_fullsized":"2024-05-08T21:17:56.417621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:42.866864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416940+0000","last_change":"2024-05-08T21:17:02.735423+0000","last_active":"2024-05-08T21:17:56.416940+0000","last_peered":"2024-05-08T21:17:56.416940+0000","last_clean":"2024-05-08T21:17:56.416940+0000","last_became_active":"2024-05-08T21:17:02.735158+0000","last_became_peered":"2024-05-08T21:17:02.735158+0000","last_unstale":"2024-05-08T21:17:56.416940+0000","last_undegraded":"2024-05-08T21:17:56.416940+0000","last_fullsized":"2024-05-08T21:17:56.416940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:19.861219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415012+0000","last_change":"2024-05-08T21:17:03.239316+0000","last_active":"2024-05-08T21:17:56.415012+0000","last_peered":"2024-05-08T21:17:56.415012+0000","last_clean":"2024-05-08T21:17:56.415012+0000","last_became_active":"2024-05-08T21:17:03.239018+0000","last_became_peered":"2024-05-08T21:17:03.239018+0000","last_unstale":"2024-05-08T21:17:56.415012+0000","last_undegraded":"2024-05-08T21:17:56.415012+0000","last_fullsized":"2024-05-08T21:17:56.415012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:38:36.952155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415569+0000","last_change":"2024-05-08T21:17:07.439338+0000","last_active":"2024-05-08T21:17:56.415569+0000","last_peered":"2024-05-08T21:17:56.415569+0000","last_clean":"2024-05-08T21:17:56.415569+0000","last_became_active":"2024-05-08T21:17:07.438703+0000","last_became_peered":"2024-05-08T21:17:07.438703+0000","last_unstale":"2024-05-08T21:17:56.415569+0000","last_undegraded":"2024-05-08T21:17:56.415569+0000","last_fullsized":"2024-05-08T21:17:56.415569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:50.997138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'10","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420144+0000","last_change":"2024-05-08T21:17:02.729012+0000","last_active":"2024-05-08T21:17:56.420144+0000","last_peered":"2024-05-08T21:17:56.420144+0000","last_clean":"2024-05-08T21:17:56.420144+0000","last_became_active":"2024-05-08T21:17:02.728606+0000","last_became_peered":"2024-05-08T21:17:02.728606+0000","last_unstale":"2024-05-08T21:17:56.420144+0000","last_undegraded":"2024-05-08T21:17:56.420144+0000","last_fullsized":"2024-05-08T21:17:56.420144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:34:35.140609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416523+0000","last_change":"2024-05-08T21:17:03.234712+0000","last_active":"2024-05-08T21:17:56.416523+0000","last_peered":"2024-05-08T21:17:56.416523+0000","last_clean":"2024-05-08T21:17:56.416523+0000","last_became_active":"2024-05-08T21:17:03.234404+0000","last_became_peered":"2024-05-08T21:17:03.234404+0000","last_unstale":"2024-05-08T21:17:56.416523+0000","last_undegraded":"2024-05-08T21:17:56.416523+0000","last_fullsized":"2024-05-08T21:17:56.416523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:41.479470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422480+0000","last_change":"2024-05-08T21:17:07.259120+0000","last_active":"2024-05-08T21:17:56.422480+0000","last_peered":"2024-05-08T21:17:56.422480+0000","last_clean":"2024-05-08T21:17:56.422480+0000","last_became_active":"2024-05-08T21:17:07.258696+0000","last_became_peered":"2024-05-08T21:17:07.258696+0000","last_unstale":"2024-05-08T21:17:56.422480+0000","last_undegraded":"2024-05-08T21:17:56.422480+0000","last_fullsized":"2024-05-08T21:17:56.422480+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:26.950767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423353+0000","last_change":"2024-05-08T21:17:02.727795+0000","last_active":"2024-05-08T21:17:56.423353+0000","last_peered":"2024-05-08T21:17:56.423353+0000","last_clean":"2024-05-08T21:17:56.423353+0000","last_became_active":"2024-05-08T21:17:02.727553+0000","last_became_peered":"2024-05-08T21:17:02.727553+0000","last_unstale":"2024-05-08T21:17:56.423353+0000","last_undegraded":"2024-05-08T21:17:56.423353+0000","last_fullsized":"2024-05-08T21:17:56.423353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:21:51.540221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423413+0000","last_change":"2024-05-08T21:17:03.247270+0000","last_active":"2024-05-08T21:17:56.423413+0000","last_peered":"2024-05-08T21:17:56.423413+0000","last_clean":"2024-05-08T21:17:56.423413+0000","last_became_active":"2024-05-08T21:17:03.246853+0000","last_became_peered":"2024-05-08T21:17:03.246853+0000","last_unstale":"2024-05-08T21:17:56.423413+0000","last_undegraded":"2024-05-08T21:17:56.423413+0000","last_fullsized":"2024-05-08T21:17:56.423413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:58:46.224088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422880+0000","last_change":"2024-05-08T21:17:07.439920+0000","last_active":"2024-05-08T21:17:56.422880+0000","last_peered":"2024-05-08T21:17:56.422880+0000","last_clean":"2024-05-08T21:17:56.422880+0000","last_became_active":"2024-05-08T21:17:07.439459+0000","last_became_peered":"2024-05-08T21:17:07.439459+0000","last_unstale":"2024-05-08T21:17:56.422880+0000","last_undegraded":"2024-05-08T21:17:56.422880+0000","last_fullsized":"2024-05-08T21:17:56.422880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:34:03.008930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422835+0000","last_change":"2024-05-08T21:17:02.731889+0000","last_active":"2024-05-08T21:17:56.422835+0000","last_peered":"2024-05-08T21:17:56.422835+0000","last_clean":"2024-05-08T21:17:56.422835+0000","last_became_active":"2024-05-08T21:17:02.731554+0000","last_became_peered":"2024-05-08T21:17:02.731554+0000","last_unstale":"2024-05-08T21:17:56.422835+0000","last_undegraded":"2024-05-08T21:17:56.422835+0000","last_fullsized":"2024-05-08T21:17:56.422835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:17:25.223003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417051+0000","last_change":"2024-05-08T21:17:03.252672+0000","last_active":"2024-05-08T21:17:56.417051+0000","last_peered":"2024-05-08T21:17:56.417051+0000","last_clean":"2024-05-08T21:17:56.417051+0000","last_became_active":"2024-05-08T21:17:03.251684+0000","last_became_peered":"2024-05-08T21:17:03.251684+0000","last_unstale":"2024-05-08T21:17:56.417051+0000","last_undegraded":"2024-05-08T21:17:56.417051+0000","last_fullsized":"2024-05-08T21:17:56.417051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:29:06.460419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420488+0000","last_change":"2024-05-08T21:17:07.273478+0000","last_active":"2024-05-08T21:17:56.420488+0000","last_peered":"2024-05-08T21:17:56.420488+0000","last_clean":"2024-05-08T21:17:56.420488+0000","last_became_active":"2024-05-08T21:17:07.272525+0000","last_became_peered":"2024-05-08T21:17:07.272525+0000","last_unstale":"2024-05-08T21:17:56.420488+0000","last_undegraded":"2024-05-08T21:17:56.420488+0000","last_fullsized":"2024-05-08T21:17:56.420488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:51:05.107387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417904+0000","last_change":"2024-05-08T21:17:02.728858+0000","last_active":"2024-05-08T21:17:56.417904+0000","last_peered":"2024-05-08T21:17:56.417904+0000","last_clean":"2024-05-08T21:17:56.417904+0000","last_became_active":"2024-05-08T21:17:02.728516+0000","last_became_peered":"2024-05-08T21:17:02.728516+0000","last_unstale":"2024-05-08T21:17:56.417904+0000","last_undegraded":"2024-05-08T21:17:56.417904+0000","last_fullsized":"2024-05-08T21:17:56.417904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:40.263983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418555+0000","last_change":"2024-05-08T21:17:03.239354+0000","last_active":"2024-05-08T21:17:56.418555+0000","last_peered":"2024-05-08T21:17:56.418555+0000","last_clean":"2024-05-08T21:17:56.418555+0000","last_became_active":"2024-05-08T21:17:03.239020+0000","last_became_peered":"2024-05-08T21:17:03.239020+0000","last_unstale":"2024-05-08T21:17:56.418555+0000","last_undegraded":"2024-05-08T21:17:56.418555+0000","last_fullsized":"2024-05-08T21:17:56.418555+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:02:43.362016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422887+0000","last_change":"2024-05-08T21:17:07.262401+0000","last_active":"2024-05-08T21:17:56.422887+0000","last_peered":"2024-05-08T21:17:56.422887+0000","last_clean":"2024-05-08T21:17:56.422887+0000","last_became_active":"2024-05-08T21:17:07.261946+0000","last_became_peered":"2024-05-08T21:17:07.261946+0000","last_unstale":"2024-05-08T21:17:56.422887+0000","last_undegraded":"2024-05-08T21:17:56.422887+0000","last_fullsized":"2024-05-08T21:17:56.422887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:29:56.565604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:17:57.438045+0000","last_change":"2024-05-08T21:17:57.437573+0000","last_active":"2024-05-08T21:17:57.438045+0000","last_peered":"2024-05-08T21:17:57.438045+0000","last_clean":"2024-05-08T21:17:57.438045+0000","last_became_active":"2024-05-08T21:17:57.432315+0000","last_became_peered":"2024-05-08T21:17:57.432315+0000","last_unstale":"2024-05-08T21:17:57.438045+0000","last_undegraded":"2024-05-08T21:17:57.438045+0000","last_fullsized":"2024-05-08T21:17:57.438045+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:56:08.847290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417312+0000","last_change":"2024-05-08T21:17:03.250382+0000","last_active":"2024-05-08T21:17:56.417312+0000","last_peered":"2024-05-08T21:17:56.417312+0000","last_clean":"2024-05-08T21:17:56.417312+0000","last_became_active":"2024-05-08T21:17:03.250031+0000","last_became_peered":"2024-05-08T21:17:03.250031+0000","last_unstale":"2024-05-08T21:17:56.417312+0000","last_undegraded":"2024-05-08T21:17:56.417312+0000","last_fullsized":"2024-05-08T21:17:56.417312+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:59.961012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422877+0000","last_change":"2024-05-08T21:17:07.439341+0000","last_active":"2024-05-08T21:17:56.422877+0000","last_peered":"2024-05-08T21:17:56.422877+0000","last_clean":"2024-05-08T21:17:56.422877+0000","last_became_active":"2024-05-08T21:17:07.438896+0000","last_became_peered":"2024-05-08T21:17:07.438896+0000","last_unstale":"2024-05-08T21:17:56.422877+0000","last_undegraded":"2024-05-08T21:17:56.422877+0000","last_fullsized":"2024-05-08T21:17:56.422877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:55.180686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.416194+0000","last_change":"2024-05-08T21:17:02.732967+0000","last_active":"2024-05-08T21:17:56.416194+0000","last_peered":"2024-05-08T21:17:56.416194+0000","last_clean":"2024-05-08T21:17:56.416194+0000","last_became_active":"2024-05-08T21:17:02.732616+0000","last_became_peered":"2024-05-08T21:17:02.732616+0000","last_unstale":"2024-05-08T21:17:56.416194+0000","last_undegraded":"2024-05-08T21:17:56.416194+0000","last_fullsized":"2024-05-08T21:17:56.416194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:50:26.116841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420367+0000","last_change":"2024-05-08T21:17:03.241974+0000","last_active":"2024-05-08T21:17:56.420367+0000","last_peered":"2024-05-08T21:17:56.420367+0000","last_clean":"2024-05-08T21:17:56.420367+0000","last_became_active":"2024-05-08T21:17:03.241627+0000","last_became_peered":"2024-05-08T21:17:03.241627+0000","last_unstale":"2024-05-08T21:17:56.420367+0000","last_undegraded":"2024-05-08T21:17:56.420367+0000","last_fullsized":"2024-05-08T21:17:56.420367+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:15:57.351078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424107+0000","last_change":"2024-05-08T21:17:07.259593+0000","last_active":"2024-05-08T21:17:56.424107+0000","last_peered":"2024-05-08T21:17:56.424107+0000","last_clean":"2024-05-08T21:17:56.424107+0000","last_became_active":"2024-05-08T21:17:07.258362+0000","last_became_peered":"2024-05-08T21:17:07.258362+0000","last_unstale":"2024-05-08T21:17:56.424107+0000","last_undegraded":"2024-05-08T21:17:56.424107+0000","last_fullsized":"2024-05-08T21:17:56.424107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:49:07.877264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'7","reported_seq":109,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T21:18:50.448028+0000","last_change":"2024-05-08T21:17:02.731685+0000","last_active":"2024-05-08T21:18:50.448028+0000","last_peered":"2024-05-08T21:18:50.448028+0000","last_clean":"2024-05-08T21:18:50.448028+0000","last_became_active":"2024-05-08T21:17:02.731293+0000","last_became_peered":"2024-05-08T21:17:02.731293+0000","last_unstale":"2024-05-08T21:18:50.448028+0000","last_undegraded":"2024-05-08T21:18:50.448028+0000","last_fullsized":"2024-05-08T21:18:50.448028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:36.532962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2011,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419974+0000","last_change":"2024-05-08T21:17:03.250090+0000","last_active":"2024-05-08T21:17:56.419974+0000","last_peered":"2024-05-08T21:17:56.419974+0000","last_clean":"2024-05-08T21:17:56.419974+0000","last_became_active":"2024-05-08T21:17:03.249744+0000","last_became_peered":"2024-05-08T21:17:03.249744+0000","last_unstale":"2024-05-08T21:17:56.419974+0000","last_undegraded":"2024-05-08T21:17:56.419974+0000","last_fullsized":"2024-05-08T21:17:56.419974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:26:43.155040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422926+0000","last_change":"2024-05-08T21:17:07.264101+0000","last_active":"2024-05-08T21:17:56.422926+0000","last_peered":"2024-05-08T21:17:56.422926+0000","last_clean":"2024-05-08T21:17:56.422926+0000","last_became_active":"2024-05-08T21:17:07.263720+0000","last_became_peered":"2024-05-08T21:17:07.263720+0000","last_unstale":"2024-05-08T21:17:56.422926+0000","last_undegraded":"2024-05-08T21:17:56.422926+0000","last_fullsized":"2024-05-08T21:17:56.422926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:35:16.943044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419108+0000","last_change":"2024-05-08T21:17:02.726763+0000","last_active":"2024-05-08T21:17:56.419108+0000","last_peered":"2024-05-08T21:17:56.419108+0000","last_clean":"2024-05-08T21:17:56.419108+0000","last_became_active":"2024-05-08T21:17:02.726343+0000","last_became_peered":"2024-05-08T21:17:02.726343+0000","last_unstale":"2024-05-08T21:17:56.419108+0000","last_undegraded":"2024-05-08T21:17:56.419108+0000","last_fullsized":"2024-05-08T21:17:56.419108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:13:03.504667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420530+0000","last_change":"2024-05-08T21:17:03.244468+0000","last_active":"2024-05-08T21:17:56.420530+0000","last_peered":"2024-05-08T21:17:56.420530+0000","last_clean":"2024-05-08T21:17:56.420530+0000","last_became_active":"2024-05-08T21:17:03.244132+0000","last_became_peered":"2024-05-08T21:17:03.244132+0000","last_unstale":"2024-05-08T21:17:56.420530+0000","last_undegraded":"2024-05-08T21:17:56.420530+0000","last_fullsized":"2024-05-08T21:17:56.420530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:16:17.423657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418261+0000","last_change":"2024-05-08T21:17:07.260981+0000","last_active":"2024-05-08T21:17:56.418261+0000","last_peered":"2024-05-08T21:17:56.418261+0000","last_clean":"2024-05-08T21:17:56.418261+0000","last_became_active":"2024-05-08T21:17:07.260582+0000","last_became_peered":"2024-05-08T21:17:07.260582+0000","last_unstale":"2024-05-08T21:17:56.418261+0000","last_undegraded":"2024-05-08T21:17:56.418261+0000","last_fullsized":"2024-05-08T21:17:56.418261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:06.230380+0000","last_clean_scrub_stamp":"2024-05-08T21:17:06.230380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:25.319411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.414755+0000","last_change":"2024-05-08T21:17:02.732531+0000","last_active":"2024-05-08T21:17:56.414755+0000","last_peered":"2024-05-08T21:17:56.414755+0000","last_clean":"2024-05-08T21:17:56.414755+0000","last_became_active":"2024-05-08T21:17:02.732121+0000","last_became_peered":"2024-05-08T21:17:02.732121+0000","last_unstale":"2024-05-08T21:17:56.414755+0000","last_undegraded":"2024-05-08T21:17:56.414755+0000","last_fullsized":"2024-05-08T21:17:56.414755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:01.697035+0000","last_clean_scrub_stamp":"2024-05-08T21:17:01.697035+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:27:25.482458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.420635+0000","last_change":"2024-05-08T21:17:03.249676+0000","last_active":"2024-05-08T21:17:56.420635+0000","last_peered":"2024-05-08T21:17:56.420635+0000","last_clean":"2024-05-08T21:17:56.420635+0000","last_became_active":"2024-05-08T21:17:03.249120+0000","last_became_peered":"2024-05-08T21:17:03.249120+0000","last_unstale":"2024-05-08T21:17:56.420635+0000","last_undegraded":"2024-05-08T21:17:56.420635+0000","last_fullsized":"2024-05-08T21:17:56.420635+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:58:40.678800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418654+0000","last_change":"2024-05-08T21:17:03.245585+0000","last_active":"2024-05-08T21:17:56.418654+0000","last_peered":"2024-05-08T21:17:56.418654+0000","last_clean":"2024-05-08T21:17:56.418654+0000","last_became_active":"2024-05-08T21:17:03.245274+0000","last_became_peered":"2024-05-08T21:17:03.245274+0000","last_unstale":"2024-05-08T21:17:56.418654+0000","last_undegraded":"2024-05-08T21:17:56.418654+0000","last_fullsized":"2024-05-08T21:17:56.418654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:44:32.003962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423037+0000","last_change":"2024-05-08T21:17:03.246622+0000","last_active":"2024-05-08T21:17:56.423037+0000","last_peered":"2024-05-08T21:17:56.423037+0000","last_clean":"2024-05-08T21:17:56.423037+0000","last_became_active":"2024-05-08T21:17:03.246309+0000","last_became_peered":"2024-05-08T21:17:03.246309+0000","last_unstale":"2024-05-08T21:17:56.423037+0000","last_undegraded":"2024-05-08T21:17:56.423037+0000","last_fullsized":"2024-05-08T21:17:56.423037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:32.439389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.422776+0000","last_change":"2024-05-08T21:17:03.243460+0000","last_active":"2024-05-08T21:17:56.422776+0000","last_peered":"2024-05-08T21:17:56.422776+0000","last_clean":"2024-05-08T21:17:56.422776+0000","last_became_active":"2024-05-08T21:17:03.243097+0000","last_became_peered":"2024-05-08T21:17:03.243097+0000","last_unstale":"2024-05-08T21:17:56.422776+0000","last_undegraded":"2024-05-08T21:17:56.422776+0000","last_fullsized":"2024-05-08T21:17:56.422776+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:52:14.492180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423038+0000","last_change":"2024-05-08T21:17:03.245077+0000","last_active":"2024-05-08T21:17:56.423038+0000","last_peered":"2024-05-08T21:17:56.423038+0000","last_clean":"2024-05-08T21:17:56.423038+0000","last_became_active":"2024-05-08T21:17:03.244719+0000","last_became_peered":"2024-05-08T21:17:03.244719+0000","last_unstale":"2024-05-08T21:17:56.423038+0000","last_undegraded":"2024-05-08T21:17:56.423038+0000","last_fullsized":"2024-05-08T21:17:56.423038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:26.388318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.415101+0000","last_change":"2024-05-08T21:17:03.245261+0000","last_active":"2024-05-08T21:17:56.415101+0000","last_peered":"2024-05-08T21:17:56.415101+0000","last_clean":"2024-05-08T21:17:56.415101+0000","last_became_active":"2024-05-08T21:17:03.244892+0000","last_became_peered":"2024-05-08T21:17:03.244892+0000","last_unstale":"2024-05-08T21:17:56.415101+0000","last_undegraded":"2024-05-08T21:17:56.415101+0000","last_fullsized":"2024-05-08T21:17:56.415101+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:21.075634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418251+0000","last_change":"2024-05-08T21:17:03.246250+0000","last_active":"2024-05-08T21:17:56.418251+0000","last_peered":"2024-05-08T21:17:56.418251+0000","last_clean":"2024-05-08T21:17:56.418251+0000","last_became_active":"2024-05-08T21:17:03.245807+0000","last_became_peered":"2024-05-08T21:17:03.245807+0000","last_unstale":"2024-05-08T21:17:56.418251+0000","last_undegraded":"2024-05-08T21:17:56.418251+0000","last_fullsized":"2024-05-08T21:17:56.418251+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:33:20.618260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.423477+0000","last_change":"2024-05-08T21:17:03.254559+0000","last_active":"2024-05-08T21:17:56.423477+0000","last_peered":"2024-05-08T21:17:56.423477+0000","last_clean":"2024-05-08T21:17:56.423477+0000","last_became_active":"2024-05-08T21:17:03.254231+0000","last_became_peered":"2024-05-08T21:17:03.254231+0000","last_unstale":"2024-05-08T21:17:56.423477+0000","last_undegraded":"2024-05-08T21:17:56.423477+0000","last_fullsized":"2024-05-08T21:17:56.423477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:47.996927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418065+0000","last_change":"2024-05-08T21:17:03.242479+0000","last_active":"2024-05-08T21:17:56.418065+0000","last_peered":"2024-05-08T21:17:56.418065+0000","last_clean":"2024-05-08T21:17:56.418065+0000","last_became_active":"2024-05-08T21:17:03.241397+0000","last_became_peered":"2024-05-08T21:17:03.241397+0000","last_unstale":"2024-05-08T21:17:56.418065+0000","last_undegraded":"2024-05-08T21:17:56.418065+0000","last_fullsized":"2024-05-08T21:17:56.418065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:58:39.029139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.419220+0000","last_change":"2024-05-08T21:17:03.244208+0000","last_active":"2024-05-08T21:17:56.419220+0000","last_peered":"2024-05-08T21:17:56.419220+0000","last_clean":"2024-05-08T21:17:56.419220+0000","last_became_active":"2024-05-08T21:17:03.243828+0000","last_became_peered":"2024-05-08T21:17:03.243828+0000","last_unstale":"2024-05-08T21:17:56.419220+0000","last_undegraded":"2024-05-08T21:17:56.419220+0000","last_fullsized":"2024-05-08T21:17:56.419220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:41:39.936750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.418012+0000","last_change":"2024-05-08T21:17:03.240135+0000","last_active":"2024-05-08T21:17:56.418012+0000","last_peered":"2024-05-08T21:17:56.418012+0000","last_clean":"2024-05-08T21:17:56.418012+0000","last_became_active":"2024-05-08T21:17:03.239757+0000","last_became_peered":"2024-05-08T21:17:03.239757+0000","last_unstale":"2024-05-08T21:17:56.418012+0000","last_undegraded":"2024-05-08T21:17:56.418012+0000","last_fullsized":"2024-05-08T21:17:56.418012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:30:46.066111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.424382+0000","last_change":"2024-05-08T21:17:03.256254+0000","last_active":"2024-05-08T21:17:56.424382+0000","last_peered":"2024-05-08T21:17:56.424382+0000","last_clean":"2024-05-08T21:17:56.424382+0000","last_became_active":"2024-05-08T21:17:03.255310+0000","last_became_peered":"2024-05-08T21:17:03.255310+0000","last_unstale":"2024-05-08T21:17:56.424382+0000","last_undegraded":"2024-05-08T21:17:56.424382+0000","last_fullsized":"2024-05-08T21:17:56.424382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:59.571958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.412843+0000","last_change":"2024-05-08T21:17:03.241901+0000","last_active":"2024-05-08T21:17:56.412843+0000","last_peered":"2024-05-08T21:17:56.412843+0000","last_clean":"2024-05-08T21:17:56.412843+0000","last_became_active":"2024-05-08T21:17:03.241402+0000","last_became_peered":"2024-05-08T21:17:03.241402+0000","last_unstale":"2024-05-08T21:17:56.412843+0000","last_undegraded":"2024-05-08T21:17:56.412843+0000","last_fullsized":"2024-05-08T21:17:56.412843+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:59.202987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-08T21:17:56.417882+0000","last_change":"2024-05-08T21:17:03.243050+0000","last_active":"2024-05-08T21:17:56.417882+0000","last_peered":"2024-05-08T21:17:56.417882+0000","last_clean":"2024-05-08T21:17:56.417882+0000","last_became_active":"2024-05-08T21:17:03.240676+0000","last_became_peered":"2024-05-08T21:17:03.240676+0000","last_unstale":"2024-05-08T21:17:56.417882+0000","last_undegraded":"2024-05-08T21:17:56.417882+0000","last_fullsized":"2024-05-08T21:17:56.417882+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T21:17:02.699671+0000","last_clean_scrub_stamp":"2024-05-08T21:17:02.699671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:19:53.981559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71392,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":833,"num_read_kb":493,"num_write":251,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1246,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":220,"ondisk_log_size":220,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514528,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":484195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.76800000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.65800000000000003,"15min":0.66300000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83699999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67100000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53600000000000003}]},{"osd":2,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.498,"15min":0.504},"min":{"1min":0.34699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69499999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.54700000000000004,"15min":0.55500000000000005},"min":{"1min":0.32700000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.64000000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":4,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88900000000000001,"5min":1.034,"15min":1.034},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.748,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56999999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.68100000000000005,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68999999999999995,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.80100000000000005}]},{"osd":6,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.625,"15min":0.63900000000000001},"min":{"1min":0.43099999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.82899999999999996,"5min":1.085,"15min":1.085},"last":0.78200000000000003}]},{"osd":7,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.56000000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.92400000000000004,"5min":1.228,"15min":1.228},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67700000000000005,"15min":0.67800000000000005},"min":{"1min":0.48599999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79900000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.77000000000000002}]},{"osd":8,"last update":"Wed May 8 21:18:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.84699999999999998}]},{"osd":9,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77300000000000002,"15min":0.77200000000000002},"min":{"1min":0.66100000000000003,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.93999999999999995,"5min":1.113,"15min":1.113},"last":0.88},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.80700000000000005},"min":{"1min":0.65600000000000003,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.95499999999999996,"5min":1.27,"15min":1.27},"last":0.88900000000000001}]},{"osd":10,"last update":"Wed May 8 21:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.747,"15min":0.75600000000000001},"min":{"1min":0.57099999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.84199999999999997,"5min":1.419,"15min":1.419},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.64500000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90400000000000003,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.83099999999999996}]}]},{"osd":10,"up_from":69,"seq":296352743459,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36848,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708304,"statfs":{"total":95995035648,"available":95957303296,"internally_reserved":0,"allocated":1679360,"data_stored":1133072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36043201},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.71299999999999997}]},{"osd":1,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.42999999999999999,"15min":0.42899999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65100000000000002,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46300000000000002,"15min":0.46200000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68100000000000005,"5min":0.748,"15min":0.748},"last":0.42399999999999999}]},{"osd":2,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.441,"5min":0.373,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.58399999999999996,"15min":0.58299999999999996},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81599999999999995,"5min":0.88,"15min":0.88},"last":0.47699999999999998}]},{"osd":3,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78500000000000003,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.626,"15min":0.627},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61299999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.85599999999999998,"5min":1.248,"15min":1.248},"last":0.46100000000000002}]},{"osd":5,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65900000000000003,"15min":0.65800000000000003},"min":{"1min":0.47599999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97199999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.45100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91400000000000003,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57999999999999996}]},{"osd":6,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95999999999999996,"5min":1.014,"15min":1.014},"last":0.627},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.69499999999999995,"15min":0.69699999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.94699999999999995,"5min":1.048,"15min":1.048},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":8,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46300000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.745,"15min":0.751},"min":{"1min":0.47799999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90700000000000003,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.68700000000000006}]},{"osd":9,"last update":"Wed May 8 21:18:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.753,"15min":0.755},"min":{"1min":0.56799999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0229999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77300000000000002,"15min":0.77600000000000002},"min":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.93000000000000005,"5min":1.419,"15min":1.419},"last":0.73299999999999998}]},{"osd":11,"last update":"Wed May 8 21:18:33 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.81999999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.749,"15min":0.753},"min":{"1min":0.55400000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.876,"5min":0.995,"15min":0.995},"last":0.74399999999999999}]}]},{"osd":9,"up_from":64,"seq":274877906982,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":487380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.48099999999999998,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65000000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.5,"15min":0.50800000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63700000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.34799999999999998}]},{"osd":1,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.84399999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.81100000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.48499999999999999}]},{"osd":2,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.76500000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.63400000000000001,"15min":0.64300000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65800000000000003}]},{"osd":3,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81699999999999995,"5min":1.048,"15min":1.048},"last":0.46000000000000002}]},{"osd":4,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.496,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79200000000000004}]},{"osd":5,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.77800000000000002,"15min":0.79500000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92300000000000004,"5min":1.234,"15min":1.234},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.056,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Wed May 8 21:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.875},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.47099999999999997}]},{"osd":7,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.81599999999999995,"15min":0.83099999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96099999999999997,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.84199999999999997,"15min":0.86399999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.67500000000000004}]},{"osd":8,"last update":"Wed May 8 21:18:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.80100000000000005,"15min":0.82299999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93600000000000005,"5min":1.101,"15min":1.101},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.88600000000000001,"15min":0.90700000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.004,"5min":1.589,"15min":1.589},"last":0.82599999999999996}]},{"osd":10,"last update":"Wed May 8 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.84399999999999997,"15min":0.85599999999999998},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0409999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.745},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.875,"15min":0.88700000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1140000000000001,"5min":1.153,"15min":1.153},"last":0.625}]},{"osd":11,"last update":"Wed May 8 21:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.77100000000000002}]}]},{"osd":8,"up_from":56,"seq":240518168619,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":482659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.42699999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.56100000000000005},"min":{"1min":0.39400000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.501}]},{"osd":1,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59699999999999998,"15min":0.57499999999999996},"min":{"1min":0.48199999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.60299999999999998,"15min":0.58299999999999996},"min":{"1min":0.52800000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.63900000000000001}]},{"osd":2,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.47899999999999998,"15min":0.47199999999999998},"min":{"1min":0.35999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.75600000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.47899999999999998,"15min":0.45000000000000001},"min":{"1min":0.375,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47999999999999998}]},{"osd":3,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57099999999999995,"15min":0.55800000000000005},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89200000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.62,"15min":0.62},"min":{"1min":0.501,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84599999999999997,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59799999999999998}]},{"osd":4,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.49399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85899999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.64300000000000002,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.69099999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54400000000000004,"15min":0.51400000000000001},"min":{"1min":0.41299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.57299999999999995,"15min":0.55300000000000005},"min":{"1min":0.46999999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.66400000000000003,"15min":0.64900000000000002},"min":{"1min":0.51700000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.70399999999999996,"15min":0.67900000000000005},"min":{"1min":0.66700000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed May 8 21:18:39 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.67500000000000004,"15min":0.64300000000000002},"min":{"1min":0.53600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.71699999999999997,"15min":0.70499999999999996},"min":{"1min":0.55800000000000005,"5min":0.373,"15min":0.373},"max":{"1min":0.95799999999999996,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.76400000000000001}]},{"osd":9,"last update":"Wed May 8 21:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70199999999999996,"15min":0.68899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.745,"15min":0.73499999999999999},"min":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.96999999999999997,"5min":1.04,"15min":1.04},"last":0.79300000000000004}]},{"osd":10,"last update":"Wed May 8 21:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.60099999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.61199999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94899999999999995,"5min":1.008,"15min":1.008},"last":0.77200000000000002}]},{"osd":11,"last update":"Wed May 8 21:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.752},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.83299999999999996}]}]},{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1597440,"data_stored":1080508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.41099999999999998,"15min":0.40000000000000002},"min":{"1min":0.32400000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.54400000000000004,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.40200000000000002},"min":{"1min":0.317,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.61699999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.496}]},{"osd":2,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58799999999999997,"15min":0.57299999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79800000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.56499999999999995,"15min":0.54600000000000004},"min":{"1min":0.46999999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.89900000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.52800000000000002}]},{"osd":3,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71499999999999997,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.59599999999999997,"15min":0.59199999999999997},"min":{"1min":0.40999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.72299999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.36599999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.379,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.61799999999999999,"5min":0.745,"15min":0.745},"last":0.438},{"interface":"front","average":{"1min":0.499,"5min":0.48799999999999999,"15min":0.49099999999999999},"min":{"1min":0.35599999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.71099999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.38500000000000001}]},{"osd":5,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60999999999999999,"15min":0.59799999999999998},"min":{"1min":0.45000000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.76500000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.622,"15min":0.60099999999999998},"min":{"1min":0.51700000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.873,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Wed May 8 21:18:24 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65000000000000002,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81200000000000006,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.65100000000000002,"15min":0.65600000000000003},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83999999999999997,"5min":0.87,"15min":0.87},"last":0.41799999999999998}]},{"osd":8,"last update":"Wed May 8 21:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.66900000000000004,"15min":0.65500000000000003},"min":{"1min":0.54400000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88500000000000001,"5min":1,"15min":1},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.67200000000000004,"15min":0.65200000000000002},"min":{"1min":0.57399999999999995,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95399999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.67200000000000004}]},{"osd":9,"last update":"Wed May 8 21:17:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67200000000000004,"15min":0.67600000000000005},"min":{"1min":0.47799999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.81999999999999995,"5min":1.008,"15min":1.008},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68999999999999995,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.86499999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65600000000000003}]},{"osd":10,"last update":"Wed May 8 21:18:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.57199999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.66300000000000003,"15min":0.65000000000000002},"min":{"1min":0.625,"5min":0.437,"15min":0.437},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.70699999999999996}]},{"osd":11,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.622}]}]},{"osd":6,"up_from":44,"seq":188978561075,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":479236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46700000000000003},"min":{"1min":0.34899999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.34100000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.40600000000000003,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.57799999999999996},"min":{"1min":0.502,"5min":0.434,"15min":0.434},"max":{"1min":1,"5min":1,"15min":1},"last":0.67800000000000005}]},{"osd":2,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.50900000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.61299999999999999,"15min":0.61399999999999999},"min":{"1min":0.441,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.61399999999999999}]},{"osd":3,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.33500000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.59799999999999998}]},{"osd":4,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.626,"15min":0.60699999999999998},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.45700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.71599999999999997}]},{"osd":5,"last update":"Wed May 8 21:18:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.56000000000000005,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.72399999999999998,"15min":0.71199999999999997},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.77300000000000002}]},{"osd":7,"last update":"Wed May 8 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.76400000000000001,"15min":0.76100000000000001},"min":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.85599999999999998}]},{"osd":8,"last update":"Wed May 8 21:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.75},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.59599999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.82399999999999995}]},{"osd":9,"last update":"Wed May 8 21:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.80400000000000005,"15min":0.79300000000000004},"min":{"1min":0.54800000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.77100000000000002,"15min":0.76700000000000002},"min":{"1min":0.625,"5min":0.498,"15min":0.498},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.86699999999999999}]},{"osd":10,"last update":"Wed May 8 21:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.76500000000000001,"15min":0.754},"min":{"1min":0.65000000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.81299999999999994,"15min":0.79900000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.81100000000000005}]},{"osd":11,"last update":"Wed May 8 21:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.80300000000000005}]}]},{"osd":1,"up_from":12,"seq":51539607619,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":543723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.54300000000000004,"15min":0.52200000000000002},"min":{"1min":0.39300000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.56699999999999995,"15min":0.53300000000000003},"min":{"1min":0.44700000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.745,"5min":1.548,"15min":1.548},"last":0.84999999999999998}]},{"osd":2,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.54000000000000004,"15min":0.50700000000000001},"min":{"1min":0.41699999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.52900000000000003,"15min":0.496},"min":{"1min":0.40600000000000003,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.80900000000000005,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.77300000000000002}]},{"osd":3,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.51100000000000001},"min":{"1min":0.42799999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75600000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.57999999999999996,"15min":0.52500000000000002},"min":{"1min":0.432,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85999999999999999,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.73399999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45000000000000001,"15min":0.41599999999999998},"min":{"1min":0.35799999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.63200000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.42499999999999999},"min":{"1min":0.41899999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.66300000000000003,"5min":0.753,"15min":0.753},"last":0.60599999999999998}]},{"osd":5,"last update":"Wed May 8 21:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.58199999999999996,"15min":0.55200000000000005},"min":{"1min":0.373,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.877,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.54400000000000004},"min":{"1min":0.495,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88300000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.82299999999999995}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.60999999999999999,"15min":0.58299999999999996},"min":{"1min":0.46000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64300000000000002,"15min":0.61199999999999999},"min":{"1min":0.502,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89100000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.92000000000000004}]},{"osd":7,"last update":"Wed May 8 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.55800000000000005},"min":{"1min":0.40899999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84399999999999997,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.55300000000000005,"15min":0.51600000000000001},"min":{"1min":0.443,"5min":0.308,"15min":0.308},"max":{"1min":0.94499999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78900000000000003}]},{"osd":8,"last update":"Wed May 8 21:18:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.67100000000000004,"15min":0.63900000000000001},"min":{"1min":0.48499999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.99099999999999999,"5min":1.032,"15min":1.032},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.72099999999999997,"15min":0.70299999999999996},"min":{"1min":0.497,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.94799999999999995,"5min":1.607,"15min":1.607},"last":0.80400000000000005}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.70099999999999996,"15min":0.69099999999999995},"min":{"1min":0.54300000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.73299999999999998,"15min":0.72299999999999998},"min":{"1min":0.54900000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.877}]},{"osd":10,"last update":"Wed May 8 21:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.752,"15min":0.747},"min":{"1min":0.54800000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95499999999999996,"5min":1.004,"15min":1.004},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.71799999999999997,"15min":0.70799999999999996},"min":{"1min":0.63300000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.89000000000000001}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.90900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":1011712,"data_stored":481085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed May 8 21:18:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.41599999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.84299999999999997,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.46400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.51700000000000002}]},{"osd":2,"last update":"Wed May 8 21:17:53 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59899999999999998,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.748,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67400000000000004}]},{"osd":3,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.439},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.48899999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61199999999999999,"15min":0.59699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85199999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.627,"15min":0.58999999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69499999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.64700000000000002,"15min":0.61699999999999999},"min":{"1min":0.48799999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.624,"15min":0.60799999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93600000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57599999999999996,"15min":0.56799999999999995},"min":{"1min":0.35099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.751},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.42999999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89700000000000002,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.623}]},{"osd":7,"last update":"Wed May 8 21:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.85699999999999998}]},{"osd":8,"last update":"Wed May 8 21:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.54000000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.63100000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78400000000000003}]},{"osd":9,"last update":"Wed May 8 21:17:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77800000000000002,"15min":0.77400000000000002},"min":{"1min":0.65100000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.97799999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Wed May 8 21:18:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81799999999999995,"15min":0.81299999999999994},"min":{"1min":0.68500000000000005,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.86199999999999999,"15min":0.85199999999999998},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.82399999999999995}]},{"osd":11,"last update":"Wed May 8 21:18:34 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.81000000000000005,"15min":0.80000000000000004},"min":{"1min":0.71899999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.88},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.81399999999999995,"15min":0.80600000000000005},"min":{"1min":0.71599999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.83899999999999997}]}]},{"osd":2,"up_from":17,"seq":73014444097,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1069056,"data_stored":543042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:17:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.56799999999999995,"15min":0.52100000000000002},"min":{"1min":0.32600000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":6.7270000000000003,"5min":6.7270000000000003,"15min":6.7270000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.51200000000000001},"min":{"1min":0.33500000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47999999999999998}]},{"osd":1,"last update":"Wed May 8 21:17:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.44500000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.54300000000000004,"15min":0.52900000000000003},"min":{"1min":0.42899999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70599999999999996}]},{"osd":3,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.41399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.437,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.45900000000000002,"15min":0.41799999999999998},"min":{"1min":0.36499999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.45900000000000002,"15min":0.42299999999999999},"min":{"1min":0.30299999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.753,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58399999999999996}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.58199999999999996,"15min":0.56200000000000006},"min":{"1min":0.55400000000000005,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.212,"5min":0.71199999999999997,"15min":0.628},"min":{"1min":0.48299999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":7.7779999999999996,"5min":7.7779999999999996,"15min":7.7779999999999996},"last":0.754}]},{"osd":7,"last update":"Wed May 8 21:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.83799999999999997}]},{"osd":8,"last update":"Wed May 8 21:18:37 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59499999999999997,"15min":0.56699999999999995},"min":{"1min":0.39000000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.95199999999999996,"5min":0.995,"15min":0.995},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59399999999999997,"15min":0.56799999999999995},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71999999999999997}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.67700000000000005,"15min":0.65800000000000003},"min":{"1min":0.66800000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.68700000000000006,"15min":0.67700000000000005},"min":{"1min":0.56799999999999995,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.82399999999999995}]},{"osd":10,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.67800000000000005,"15min":0.66300000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.753,"15min":0.73499999999999999},"min":{"1min":0.629,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0149999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.89200000000000002}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.871},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.81299999999999994}]}]},{"osd":3,"up_from":24,"seq":103079215165,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":999424,"data_stored":478496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.40400000000000003,"15min":0.36799999999999999},"min":{"1min":0.29899999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42099999999999999,"15min":0.39100000000000001},"min":{"1min":0.317,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76500000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54500000000000004}]},{"osd":1,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54200000000000004,"15min":0.51600000000000001},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80700000000000005,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.52100000000000002},"min":{"1min":0.42299999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58899999999999997,"15min":0.56200000000000006},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.628}]},{"osd":4,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.54500000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.56599999999999995},"min":{"1min":0.48799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67000000000000004}]},{"osd":5,"last update":"Wed May 8 21:18:47 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59999999999999998,"15min":0.56699999999999995},"min":{"1min":0.48699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78700000000000003,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96799999999999997,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.71399999999999997}]},{"osd":6,"last update":"Wed May 8 21:18:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61599999999999999,"15min":0.59999999999999998},"min":{"1min":0.504,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.73699999999999999}]},{"osd":7,"last update":"Wed May 8 21:18:25 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.73799999999999999,"15min":0.746},"min":{"1min":0.53300000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.58999999999999997,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.78200000000000003}]},{"osd":8,"last update":"Wed May 8 21:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.79800000000000004}]},{"osd":9,"last update":"Wed May 8 21:17:58 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.60599999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.85599999999999998}]},{"osd":10,"last update":"Wed May 8 21:18:21 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77300000000000002,"15min":0.77000000000000002},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80900000000000005,"15min":0.81100000000000005},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.872}]},{"osd":11,"last update":"Wed May 8 21:18:08 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.91600000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986233,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":484380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:16:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.503,"15min":0.497},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41599999999999998}]},{"osd":1,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.38300000000000001,"15min":0.36599999999999999},"min":{"1min":0.32100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.57199999999999995,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.442},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.40200000000000002,"15min":0.39300000000000002},"min":{"1min":0.34599999999999997,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.59399999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57699999999999996,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82599999999999996,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.53600000000000003},"min":{"1min":0.48099999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.71599999999999997}]},{"osd":3,"last update":"Wed May 8 21:18:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56999999999999995,"15min":0.53800000000000003},"min":{"1min":0.47599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.73899999999999999,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55000000000000004,"15min":0.52500000000000002},"min":{"1min":0.41499999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.59499999999999997,"15min":0.56299999999999994},"min":{"1min":0.44500000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.79500000000000004,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.60699999999999998,"15min":0.54900000000000004},"min":{"1min":0.46000000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82399999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66900000000000004}]},{"osd":6,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59999999999999998,"15min":0.57999999999999996},"min":{"1min":0.51200000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79000000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82399999999999995,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.48599999999999999}]},{"osd":7,"last update":"Wed May 8 21:18:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.52200000000000002,"15min":0.48799999999999999},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.81599999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56100000000000005,"15min":0.52100000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.91100000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.59099999999999997}]},{"osd":8,"last update":"Wed May 8 21:18:40 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.66400000000000003,"15min":0.64400000000000002},"min":{"1min":0.59499999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.83099999999999996,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.69999999999999996},"min":{"1min":0.59499999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.70399999999999996}]},{"osd":9,"last update":"Wed May 8 21:17:57 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.747,"15min":0.747},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.622},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.49099999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.876,"5min":1.347,"15min":1.347},"last":0.61399999999999999}]},{"osd":10,"last update":"Wed May 8 21:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69399999999999995,"15min":0.69699999999999995},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70599999999999996,"15min":0.71499999999999997},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":1.323,"15min":1.323},"last":0.63400000000000001}]},{"osd":11,"last update":"Wed May 8 21:18:04 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.78700000000000003}]}]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36784,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":93708368,"statfs":{"total":95995035648,"available":95957368832,"internally_reserved":0,"allocated":1613824,"data_stored":1075798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.52200000000000002},"min":{"1min":0.38100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.77100000000000002,"5min":0.877,"15min":0.877},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.374,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.442}]},{"osd":1,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56799999999999995,"15min":0.54800000000000004},"min":{"1min":0.434,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73699999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.751,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.72099999999999997}]},{"osd":2,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40699999999999997,"15min":0.40000000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.61799999999999999,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.40699999999999997,"15min":0.40799999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61599999999999999,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.45700000000000002}]},{"osd":3,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.54800000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89800000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.373},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55900000000000005,"15min":0.56100000000000005},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.747,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.42099999999999999}]},{"osd":4,"last update":"Wed May 8 21:18:48 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.45300000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81699999999999995,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.73799999999999999}]},{"osd":6,"last update":"Wed May 8 21:18:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.46200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.75700000000000001,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.623,"15min":0.624},"min":{"1min":0.47499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93300000000000005,"5min":1.01,"15min":1.01},"last":0.502}]},{"osd":7,"last update":"Wed May 8 21:18:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.623,"15min":0.623},"min":{"1min":0.42999999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.82699999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.46500000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88900000000000001,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66700000000000004}]},{"osd":8,"last update":"Wed May 8 21:18:38 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73499999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48699999999999999}]},{"osd":9,"last update":"Wed May 8 21:17:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66600000000000004,"15min":0.66200000000000003},"min":{"1min":0.55100000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.67900000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.88900000000000001,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52200000000000002}]},{"osd":10,"last update":"Wed May 8 21:18:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.90700000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.68100000000000005,"15min":0.66900000000000004},"min":{"1min":0.51900000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T21:18:55.013 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:18:55.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.183+0000 7f7de776c700 1 -- 172.21.15.33:0/2445003131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7de0101910 msgr2=0x7f7de01081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.183+0000 7f7de776c700 1 --2- 172.21.15.33:0/2445003131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7de0101910 0x7f7de01081c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800b0d0 tx=0x7f7dd800b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.183+0000 7f7de776c700 1 -- 172.21.15.33:0/2445003131 shutdown_connections 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.183+0000 7f7de776c700 1 --2- 172.21.15.33:0/2445003131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7de0101910 0x7f7de01081c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.183+0000 7f7de776c700 1 --2- 172.21.15.33:0/2445003131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de00ffd80 0x7f7de01001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.183+0000 7f7de776c700 1 --2- 172.21.15.33:0/2445003131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7de0100f70 0x7f7de01013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.183+0000 7f7de776c700 1 -- 172.21.15.33:0/2445003131 >> 172.21.15.33:0/2445003131 conn(0x7f7de00fb310 msgr2=0x7f7de00fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.183+0000 7f7de776c700 1 -- 172.21.15.33:0/2445003131 shutdown_connections 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.183+0000 7f7de776c700 1 -- 172.21.15.33:0/2445003131 wait complete. 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.183+0000 7f7de776c700 1 Processor -- start 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.183+0000 7f7de776c700 1 -- start start 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de776c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7de00ffd80 0x7f7de0068ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de776c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7de0100f70 0x7f7de019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de776c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de0101910 0x7f7de019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de776c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de019e8f0 con 0x7f7de00ffd80 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de776c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de019ea60 con 0x7f7de0100f70 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de776c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de019ec20 con 0x7f7de0101910 2024-05-08T21:18:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de5508700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7de00ffd80 0x7f7de0068ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de5508700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7de00ffd80 0x7f7de0068ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54242/0 (socket says 172.21.15.33:54242) 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de5508700 1 -- 172.21.15.33:0/120064500 learned_addr learned my addr 172.21.15.33:0/120064500 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de5d09700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de0101910 0x7f7de019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de4d07700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7de0100f70 0x7f7de019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de5508700 1 -- 172.21.15.33:0/120064500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de0101910 msgr2=0x7f7de019e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de5508700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de0101910 0x7f7de019e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de5508700 1 -- 172.21.15.33:0/120064500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7de0100f70 msgr2=0x7f7de019bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de5508700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7de0100f70 0x7f7de019bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de5508700 1 -- 172.21.15.33:0/120064500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8009d00 con 0x7f7de00ffd80 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de4d07700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7de0100f70 0x7f7de019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7de5508700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7de00ffd80 0x7f7de0068ce0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00e970 tx=0x7f7ddc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.184+0000 7f7dd67fc700 1 -- 172.21.15.33:0/120064500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc009790 con 0x7f7de00ffd80 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.185+0000 7f7dd67fc700 1 -- 172.21.15.33:0/120064500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ddc00eec0 con 0x7f7de00ffd80 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.185+0000 7f7de776c700 1 -- 172.21.15.33:0/120064500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de019ef00 con 0x7f7de00ffd80 2024-05-08T21:18:55.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.185+0000 7f7dd67fc700 1 -- 172.21.15.33:0/120064500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc003c10 con 0x7f7de00ffd80 2024-05-08T21:18:55.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.185+0000 7f7de776c700 1 -- 172.21.15.33:0/120064500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de019f330 con 0x7f7de00ffd80 2024-05-08T21:18:55.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.186+0000 7f7de776c700 1 -- 172.21.15.33:0/120064500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de0107250 con 0x7f7de00ffd80 2024-05-08T21:18:55.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.187+0000 7f7dd67fc700 1 -- 172.21.15.33:0/120064500 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f7ddc003d90 con 0x7f7de00ffd80 2024-05-08T21:18:55.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.188+0000 7f7dd67fc700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7dcc0a11b0 0x7f7dcc0a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:55.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.188+0000 7f7de4d07700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7dcc0a11b0 0x7f7dcc0a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:55.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.188+0000 7f7dd67fc700 1 -- 172.21.15.33:0/120064500 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7ddc014070 con 0x7f7de00ffd80 2024-05-08T21:18:55.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.188+0000 7f7de4d07700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7dcc0a11b0 0x7f7dcc0a3670 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0005950 tx=0x7f7dd00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:55.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.192+0000 7f7dd67fc700 1 -- 172.21.15.33:0/120064500 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7ddc096470 con 0x7f7de00ffd80 2024-05-08T21:18:55.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.417+0000 7f7de776c700 1 -- 172.21.15.33:0/120064500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7de0067d10 con 0x7f7de00ffd80 2024-05-08T21:18:55.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.419+0000 7f7dd67fc700 1 -- 172.21.15.33:0/120064500 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24374 (secure 0 0 0) 0x7f7ddc099a90 con 0x7f7de00ffd80 2024-05-08T21:18:55.421 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:55.421 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:17:57.410336+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:18:55.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.421+0000 7f7de776c700 1 -- 172.21.15.33:0/120064500 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7dcc0a11b0 msgr2=0x7f7dcc0a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.421+0000 7f7de776c700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7dcc0a11b0 0x7f7dcc0a3670 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0005950 tx=0x7f7dd00058e0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.421+0000 7f7de776c700 1 -- 172.21.15.33:0/120064500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7de00ffd80 msgr2=0x7f7de0068ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.421+0000 7f7de776c700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7de00ffd80 0x7f7de0068ce0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc00e970 tx=0x7f7ddc00ec80 comp rx=0 tx=0).stop 2024-05-08T21:18:55.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.421+0000 7f7de776c700 1 -- 172.21.15.33:0/120064500 shutdown_connections 2024-05-08T21:18:55.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.421+0000 7f7de776c700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de0101910 0x7f7de019e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.421+0000 7f7de776c700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7de00ffd80 0x7f7de0068ce0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.421+0000 7f7de776c700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7dcc0a11b0 0x7f7dcc0a3670 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.421+0000 7f7de776c700 1 --2- 172.21.15.33:0/120064500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7de0100f70 0x7f7de019bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.421+0000 7f7de776c700 1 -- 172.21.15.33:0/120064500 >> 172.21.15.33:0/120064500 conn(0x7f7de00fb310 msgr2=0x7f7de0104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:55.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.421+0000 7f7de776c700 1 -- 172.21.15.33:0/120064500 shutdown_connections 2024-05-08T21:18:55.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.421+0000 7f7de776c700 1 -- 172.21.15.33:0/120064500 wait complete. 2024-05-08T21:18:55.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:17:01.282803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:17:02.555552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T21:17:05.686277+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T21:18:55.438 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:18:55.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.578+0000 7f9f56a31700 1 -- 172.21.15.33:0/3100621315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f50102990 msgr2=0x7f9f501092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.578+0000 7f9f56a31700 1 --2- 172.21.15.33:0/3100621315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f50102990 0x7f9f501092b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800b0a0 tx=0x7f9f4800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.578+0000 7f9f56a31700 1 -- 172.21.15.33:0/3100621315 shutdown_connections 2024-05-08T21:18:55.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.578+0000 7f9f56a31700 1 --2- 172.21.15.33:0/3100621315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f50102990 0x7f9f501092b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.578+0000 7f9f56a31700 1 --2- 172.21.15.33:0/3100621315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f50100e80 0x7f9f501012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.578+0000 7f9f56a31700 1 --2- 172.21.15.33:0/3100621315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f50101fd0 0x7f9f50102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.578+0000 7f9f56a31700 1 -- 172.21.15.33:0/3100621315 >> 172.21.15.33:0/3100621315 conn(0x7f9f500fc3d0 msgr2=0x7f9f500fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:55.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.578+0000 7f9f56a31700 1 -- 172.21.15.33:0/3100621315 shutdown_connections 2024-05-08T21:18:55.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.578+0000 7f9f56a31700 1 -- 172.21.15.33:0/3100621315 wait complete. 2024-05-08T21:18:55.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.579+0000 7f9f56a31700 1 Processor -- start 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.579+0000 7f9f56a31700 1 -- start start 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.579+0000 7f9f56a31700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f50100e80 0x7f9f50196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.579+0000 7f9f56a31700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f50101fd0 0x7f9f50197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.579+0000 7f9f56a31700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f50102990 0x7f9f5019e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f56a31700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5019e970 con 0x7f9f50102990 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f56a31700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5019eab0 con 0x7f9f50100e80 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f56a31700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5019ec40 con 0x7f9f50101fd0 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f54fce700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f50102990 0x7f9f5019e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f54fce700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f50102990 0x7f9f5019e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54268/0 (socket says 172.21.15.33:54268) 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f54fce700 1 -- 172.21.15.33:0/888422728 learned_addr learned my addr 172.21.15.33:0/888422728 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f4ffff700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f50100e80 0x7f9f50196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f4f7fe700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f50101fd0 0x7f9f50197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:55.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f54fce700 1 -- 172.21.15.33:0/888422728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f50101fd0 msgr2=0x7f9f50197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f54fce700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f50101fd0 0x7f9f50197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f54fce700 1 -- 172.21.15.33:0/888422728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f50100e80 msgr2=0x7f9f50196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f54fce700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f50100e80 0x7f9f50196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f54fce700 1 -- 172.21.15.33:0/888422728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f48009d00 con 0x7f9f50102990 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f4f7fe700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f50101fd0 0x7f9f50197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f4ffff700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f50100e80 0x7f9f50196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f54fce700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f50102990 0x7f9f5019e3a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9f48007950 tx=0x7f9f48007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f4d7fa700 1 -- 172.21.15.33:0/888422728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4800e040 con 0x7f9f50102990 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f4d7fa700 1 -- 172.21.15.33:0/888422728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f48004e60 con 0x7f9f50102990 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f56a31700 1 -- 172.21.15.33:0/888422728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5019eec0 con 0x7f9f50102990 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f4d7fa700 1 -- 172.21.15.33:0/888422728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4801dea0 con 0x7f9f50102990 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.580+0000 7f9f56a31700 1 -- 172.21.15.33:0/888422728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5019f2e0 con 0x7f9f50102990 2024-05-08T21:18:55.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.581+0000 7f9f56a31700 1 -- 172.21.15.33:0/888422728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f50108270 con 0x7f9f50102990 2024-05-08T21:18:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.583+0000 7f9f4d7fa700 1 -- 172.21.15.33:0/888422728 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f9f4801d470 con 0x7f9f50102990 2024-05-08T21:18:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.583+0000 7f9f4d7fa700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9f380a11b0 0x7f9f380a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.583+0000 7f9f4d7fa700 1 -- 172.21.15.33:0/888422728 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9f480ce6b0 con 0x7f9f50102990 2024-05-08T21:18:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.583+0000 7f9f4ffff700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9f380a11b0 0x7f9f380a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:55.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.584+0000 7f9f4ffff700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9f380a11b0 0x7f9f380a3670 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009cb0 tx=0x7f9f40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.588+0000 7f9f4d7fa700 1 -- 172.21.15.33:0/888422728 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f9f48097a60 con 0x7f9f50102990 2024-05-08T21:18:55.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:55 smithi033 ceph-mon[42528]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:55.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:55 smithi033 ceph-mon[42528]: pgmap v286: 129 pgs: 129 active+clean; 646 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-08T21:18:55.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.805+0000 7f9f56a31700 1 -- 172.21.15.33:0/888422728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f5010c5b0 con 0x7f9f50102990 2024-05-08T21:18:55.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.806+0000 7f9f4d7fa700 1 -- 172.21.15.33:0/888422728 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f4809b080 con 0x7f9f50102990 2024-05-08T21:18:55.808 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:18:55.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.808+0000 7f9f56a31700 1 -- 172.21.15.33:0/888422728 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9f380a11b0 msgr2=0x7f9f380a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.808+0000 7f9f56a31700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9f380a11b0 0x7f9f380a3670 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009cb0 tx=0x7f9f40009260 comp rx=0 tx=0).stop 2024-05-08T21:18:55.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.808+0000 7f9f56a31700 1 -- 172.21.15.33:0/888422728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f50102990 msgr2=0x7f9f5019e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.808+0000 7f9f56a31700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f50102990 0x7f9f5019e3a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9f48007950 tx=0x7f9f48007870 comp rx=0 tx=0).stop 2024-05-08T21:18:55.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.808+0000 7f9f56a31700 1 -- 172.21.15.33:0/888422728 shutdown_connections 2024-05-08T21:18:55.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.808+0000 7f9f56a31700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f50102990 0x7f9f5019e3a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.808+0000 7f9f56a31700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9f380a11b0 0x7f9f380a3670 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.808+0000 7f9f56a31700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f50100e80 0x7f9f50196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.808+0000 7f9f56a31700 1 --2- 172.21.15.33:0/888422728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f50101fd0 0x7f9f50197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.808+0000 7f9f56a31700 1 -- 172.21.15.33:0/888422728 >> 172.21.15.33:0/888422728 conn(0x7f9f500fc3d0 msgr2=0x7f9f501053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:55.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.808+0000 7f9f56a31700 1 -- 172.21.15.33:0/888422728 shutdown_connections 2024-05-08T21:18:55.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.808+0000 7f9f56a31700 1 -- 172.21.15.33:0/888422728 wait complete. 2024-05-08T21:18:55.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:55 smithi071 ceph-mon[47074]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:55.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:55 smithi071 ceph-mon[47074]: pgmap v286: 129 pgs: 129 active+clean; 646 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-08T21:18:55.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:18:55.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:55 smithi082 ceph-mon[47512]: from='client.15588 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:55.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:55 smithi082 ceph-mon[47512]: pgmap v286: 129 pgs: 129 active+clean; 646 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 10 op/s 2024-05-08T21:18:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.962+0000 7f0311806700 1 -- 172.21.15.33:0/3970267504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f030c100f60 msgr2=0x7f030c1013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.962+0000 7f0311806700 1 --2- 172.21.15.33:0/3970267504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f030c100f60 0x7f030c1013c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f02f40099c0 tx=0x7f02f4009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.962+0000 7f0311806700 1 -- 172.21.15.33:0/3970267504 shutdown_connections 2024-05-08T21:18:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.962+0000 7f0311806700 1 --2- 172.21.15.33:0/3970267504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f030c101900 0x7f030c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.962+0000 7f0311806700 1 --2- 172.21.15.33:0/3970267504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f030c0ffd70 0x7f030c100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.962+0000 7f0311806700 1 --2- 172.21.15.33:0/3970267504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f030c100f60 0x7f030c1013c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.962+0000 7f0311806700 1 -- 172.21.15.33:0/3970267504 >> 172.21.15.33:0/3970267504 conn(0x7f030c0fb300 msgr2=0x7f030c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.962+0000 7f0311806700 1 -- 172.21.15.33:0/3970267504 shutdown_connections 2024-05-08T21:18:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.962+0000 7f0311806700 1 -- 172.21.15.33:0/3970267504 wait complete. 2024-05-08T21:18:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.963+0000 7f0311806700 1 Processor -- start 2024-05-08T21:18:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.963+0000 7f0311806700 1 -- start start 2024-05-08T21:18:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.963+0000 7f0311806700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f030c0ffd70 0x7f030c10a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.963+0000 7f0311806700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f030c100f60 0x7f030c108cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:55.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f0311806700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f030c101900 0x7f030c109200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f0311806700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c1099a0 con 0x7f030c100f60 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f0311806700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c109b10 con 0x7f030c101900 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f0311806700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c1a0d40 con 0x7f030c0ffd70 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f03027fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f030c100f60 0x7f030c108cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f03027fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f030c100f60 0x7f030c108cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54288/0 (socket says 172.21.15.33:54288) 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f03027fc700 1 -- 172.21.15.33:0/2204946967 learned_addr learned my addr 172.21.15.33:0/2204946967 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f030affd700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f030c0ffd70 0x7f030c10a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f030b7fe700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f030c101900 0x7f030c109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f03027fc700 1 -- 172.21.15.33:0/2204946967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f030c0ffd70 msgr2=0x7f030c10a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f03027fc700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f030c0ffd70 0x7f030c10a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f03027fc700 1 -- 172.21.15.33:0/2204946967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f030c101900 msgr2=0x7f030c109200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f03027fc700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f030c101900 0x7f030c109200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f03027fc700 1 -- 172.21.15.33:0/2204946967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f40095f0 con 0x7f030c100f60 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f030affd700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f030c0ffd70 0x7f030c10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f03027fc700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f030c100f60 0x7f030c108cc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f02f4000c00 tx=0x7f02f4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:55.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f0310804700 1 -- 172.21.15.33:0/2204946967 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f401d070 con 0x7f030c100f60 2024-05-08T21:18:55.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f0310804700 1 -- 172.21.15.33:0/2204946967 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02f400bd30 con 0x7f030c100f60 2024-05-08T21:18:55.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.964+0000 7f0311806700 1 -- 172.21.15.33:0/2204946967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030c1a0f70 con 0x7f030c100f60 2024-05-08T21:18:55.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.965+0000 7f0310804700 1 -- 172.21.15.33:0/2204946967 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4022c80 con 0x7f030c100f60 2024-05-08T21:18:55.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.965+0000 7f0311806700 1 -- 172.21.15.33:0/2204946967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030c1a1330 con 0x7f030c100f60 2024-05-08T21:18:55.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.966+0000 7f0311806700 1 -- 172.21.15.33:0/2204946967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030c0681d0 con 0x7f030c100f60 2024-05-08T21:18:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.967+0000 7f0310804700 1 -- 172.21.15.33:0/2204946967 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7f02f4003f50 con 0x7f030c100f60 2024-05-08T21:18:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.967+0000 7f0310804700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02f80a11b0 0x7f02f80a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.968+0000 7f030affd700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02f80a11b0 0x7f02f80a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.968+0000 7f0310804700 1 -- 172.21.15.33:0/2204946967 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f02f40ce2e0 con 0x7f030c100f60 2024-05-08T21:18:55.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.968+0000 7f030affd700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02f80a11b0 0x7f02f80a3670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f030c100dc0 tx=0x7f02fc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:55.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:55.973+0000 7f0310804700 1 -- 172.21.15.33:0/2204946967 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f030c0681d0 con 0x7f030c100f60 2024-05-08T21:18:56.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.219+0000 7f0311806700 1 -- 172.21.15.33:0/2204946967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f030c068160 con 0x7f030c100f60 2024-05-08T21:18:56.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.220+0000 7f0310804700 1 -- 172.21.15.33:0/2204946967 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f02f4097690 con 0x7f030c100f60 2024-05-08T21:18:56.222 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.222+0000 7f0311806700 1 -- 172.21.15.33:0/2204946967 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02f80a11b0 msgr2=0x7f02f80a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.222+0000 7f0311806700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02f80a11b0 0x7f02f80a3670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f030c100dc0 tx=0x7f02fc00b410 comp rx=0 tx=0).stop 2024-05-08T21:18:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.222+0000 7f0311806700 1 -- 172.21.15.33:0/2204946967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f030c100f60 msgr2=0x7f030c108cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.222+0000 7f0311806700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f030c100f60 0x7f030c108cc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f02f4000c00 tx=0x7f02f4005b00 comp rx=0 tx=0).stop 2024-05-08T21:18:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.222+0000 7f0311806700 1 -- 172.21.15.33:0/2204946967 shutdown_connections 2024-05-08T21:18:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.222+0000 7f0311806700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02f80a11b0 0x7f02f80a3670 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.222+0000 7f0311806700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f030c101900 0x7f030c109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.222+0000 7f0311806700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f030c0ffd70 0x7f030c10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.222+0000 7f0311806700 1 --2- 172.21.15.33:0/2204946967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f030c100f60 0x7f030c108cc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.222+0000 7f0311806700 1 -- 172.21.15.33:0/2204946967 >> 172.21.15.33:0/2204946967 conn(0x7f030c0fb300 msgr2=0x7f030c104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:56.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.222+0000 7f0311806700 1 -- 172.21.15.33:0/2204946967 shutdown_connections 2024-05-08T21:18:56.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.222+0000 7f0311806700 1 -- 172.21.15.33:0/2204946967 wait complete. 2024-05-08T21:18:56.239 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:18:56.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.378+0000 7fd4e2930700 1 -- 172.21.15.33:0/1359571254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4dc101910 msgr2=0x7fd4dc103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:56.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.378+0000 7fd4e2930700 1 --2- 172.21.15.33:0/1359571254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4dc101910 0x7fd4dc103d00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009990 tx=0x7fd4cc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.378+0000 7fd4e2930700 1 -- 172.21.15.33:0/1359571254 shutdown_connections 2024-05-08T21:18:56.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.378+0000 7fd4e2930700 1 --2- 172.21.15.33:0/1359571254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4dc106b70 0x7fd4dc108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.378+0000 7fd4e2930700 1 --2- 172.21.15.33:0/1359571254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4dc101910 0x7fd4dc103d00 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.378+0000 7fd4e2930700 1 --2- 172.21.15.33:0/1359571254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4dc104240 0x7fd4dc106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.378+0000 7fd4e2930700 1 -- 172.21.15.33:0/1359571254 >> 172.21.15.33:0/1359571254 conn(0x7fd4dc0fb300 msgr2=0x7fd4dc0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:56.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.378+0000 7fd4e2930700 1 -- 172.21.15.33:0/1359571254 shutdown_connections 2024-05-08T21:18:56.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.378+0000 7fd4e2930700 1 -- 172.21.15.33:0/1359571254 wait complete. 2024-05-08T21:18:56.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.378+0000 7fd4e2930700 1 Processor -- start 2024-05-08T21:18:56.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.379+0000 7fd4e2930700 1 -- start start 2024-05-08T21:18:56.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.379+0000 7fd4e2930700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4dc101910 0x7fd4dc196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:56.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.379+0000 7fd4e2930700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4dc104240 0x7fd4dc197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:56.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.379+0000 7fd4e2930700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4dc106b70 0x7fd4dc19e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:56.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.379+0000 7fd4e2930700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc19e7b0 con 0x7fd4dc106b70 2024-05-08T21:18:56.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.379+0000 7fd4e2930700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc19e920 con 0x7fd4dc101910 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.379+0000 7fd4e2930700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc19eae0 con 0x7fd4dc104240 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.379+0000 7fd4e0ecd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4dc106b70 0x7fd4dc19e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.379+0000 7fd4e0ecd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4dc106b70 0x7fd4dc19e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54298/0 (socket says 172.21.15.33:54298) 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.379+0000 7fd4e0ecd700 1 -- 172.21.15.33:0/1964012775 learned_addr learned my addr 172.21.15.33:0/1964012775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.379+0000 7fd4dbfff700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4dc101910 0x7fd4dc196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.379+0000 7fd4db7fe700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4dc104240 0x7fd4dc197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.380+0000 7fd4e0ecd700 1 -- 172.21.15.33:0/1964012775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4dc104240 msgr2=0x7fd4dc197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.380+0000 7fd4e0ecd700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4dc104240 0x7fd4dc197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.380+0000 7fd4e0ecd700 1 -- 172.21.15.33:0/1964012775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4dc101910 msgr2=0x7fd4dc196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.380+0000 7fd4e0ecd700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4dc101910 0x7fd4dc196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.380+0000 7fd4e0ecd700 1 -- 172.21.15.33:0/1964012775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0095f0 con 0x7fd4dc106b70 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.380+0000 7fd4e0ecd700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4dc106b70 0x7fd4dc19e1e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400ef30 tx=0x7fd4d400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.380+0000 7fd4d97fa700 1 -- 172.21.15.33:0/1964012775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4018070 con 0x7fd4dc106b70 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.380+0000 7fd4e2930700 1 -- 172.21.15.33:0/1964012775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc19ed60 con 0x7fd4dc106b70 2024-05-08T21:18:56.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.380+0000 7fd4d97fa700 1 -- 172.21.15.33:0/1964012775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4d400f040 con 0x7fd4dc106b70 2024-05-08T21:18:56.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.380+0000 7fd4d97fa700 1 -- 172.21.15.33:0/1964012775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4003d00 con 0x7fd4dc106b70 2024-05-08T21:18:56.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.380+0000 7fd4e2930700 1 -- 172.21.15.33:0/1964012775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc19f1c0 con 0x7fd4dc106b70 2024-05-08T21:18:56.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.381+0000 7fd4e2930700 1 -- 172.21.15.33:0/1964012775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc0fce80 con 0x7fd4dc106b70 2024-05-08T21:18:56.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.382+0000 7fd4d97fa700 1 -- 172.21.15.33:0/1964012775 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fd4d400caa0 con 0x7fd4dc106b70 2024-05-08T21:18:56.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.383+0000 7fd4d97fa700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4c40a11b0 0x7fd4c40a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:56.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.383+0000 7fd4d97fa700 1 -- 172.21.15.33:0/1964012775 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd4d40cd4b0 con 0x7fd4dc106b70 2024-05-08T21:18:56.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.383+0000 7fd4dbfff700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4c40a11b0 0x7fd4c40a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:56.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.383+0000 7fd4dbfff700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4c40a11b0 0x7fd4c40a3670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc005910 tx=0x7fd4cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:56.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.387+0000 7fd4d97fa700 1 -- 172.21.15.33:0/1964012775 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd4d4096800 con 0x7fd4dc106b70 2024-05-08T21:18:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:56 smithi033 ceph-mon[42528]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:56 smithi033 ceph-mon[42528]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:56 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/120064500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:56 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/888422728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:56.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:56 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2204946967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:56.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.609+0000 7fd4e2930700 1 -- 172.21.15.33:0/1964012775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd4dc10c6b0 con 0x7fd4dc106b70 2024-05-08T21:18:56.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.610+0000 7fd4d97fa700 1 -- 172.21.15.33:0/1964012775 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4d4099e20 con 0x7fd4dc106b70 2024-05-08T21:18:56.612 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T21:18:56.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.612+0000 7fd4e2930700 1 -- 172.21.15.33:0/1964012775 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4c40a11b0 msgr2=0x7fd4c40a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.612+0000 7fd4e2930700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4c40a11b0 0x7fd4c40a3670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc005910 tx=0x7fd4cc005fb0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.612+0000 7fd4e2930700 1 -- 172.21.15.33:0/1964012775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4dc106b70 msgr2=0x7fd4dc19e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.612+0000 7fd4e2930700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4dc106b70 0x7fd4dc19e1e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400ef30 tx=0x7fd4d400c450 comp rx=0 tx=0).stop 2024-05-08T21:18:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.612+0000 7fd4e2930700 1 -- 172.21.15.33:0/1964012775 shutdown_connections 2024-05-08T21:18:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.612+0000 7fd4e2930700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4dc106b70 0x7fd4dc19e1e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.612+0000 7fd4e2930700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4c40a11b0 0x7fd4c40a3670 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.612+0000 7fd4e2930700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4dc101910 0x7fd4dc196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.612+0000 7fd4e2930700 1 --2- 172.21.15.33:0/1964012775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4dc104240 0x7fd4dc197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.612+0000 7fd4e2930700 1 -- 172.21.15.33:0/1964012775 >> 172.21.15.33:0/1964012775 conn(0x7fd4dc0fb300 msgr2=0x7fd4dc103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.612+0000 7fd4e2930700 1 -- 172.21.15.33:0/1964012775 shutdown_connections 2024-05-08T21:18:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.612+0000 7fd4e2930700 1 -- 172.21.15.33:0/1964012775 wait complete. 2024-05-08T21:18:56.629 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T21:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.767+0000 7fe52efe3700 1 -- 172.21.15.33:0/126315743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5280ffd60 msgr2=0x7fe528100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.767+0000 7fe52efe3700 1 --2- 172.21.15.33:0/126315743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5280ffd60 0x7fe528100180 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe5240099c0 tx=0x7fe524009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.767+0000 7fe52efe3700 1 -- 172.21.15.33:0/126315743 shutdown_connections 2024-05-08T21:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.767+0000 7fe52efe3700 1 --2- 172.21.15.33:0/126315743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5281018f0 0x7fe5281081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.767+0000 7fe52efe3700 1 --2- 172.21.15.33:0/126315743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5280ffd60 0x7fe528100180 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.767+0000 7fe52efe3700 1 --2- 172.21.15.33:0/126315743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe528100f50 0x7fe5281013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.767+0000 7fe52efe3700 1 -- 172.21.15.33:0/126315743 >> 172.21.15.33:0/126315743 conn(0x7fe5280fb290 msgr2=0x7fe5280fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.767+0000 7fe52efe3700 1 -- 172.21.15.33:0/126315743 shutdown_connections 2024-05-08T21:18:56.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.767+0000 7fe52efe3700 1 -- 172.21.15.33:0/126315743 wait complete. 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.768+0000 7fe52efe3700 1 Processor -- start 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.768+0000 7fe52efe3700 1 -- start start 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.768+0000 7fe52efe3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5280ffd60 0x7fe528198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.768+0000 7fe52efe3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe528100f50 0x7fe528199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.768+0000 7fe52efe3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5281018f0 0x7fe5281a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.768+0000 7fe52efe3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281a0a70 con 0x7fe5280ffd60 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.768+0000 7fe52efe3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281a0bb0 con 0x7fe528100f50 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.768+0000 7fe52efe3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5281a0d40 con 0x7fe5281018f0 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.768+0000 7fe52cd7f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5280ffd60 0x7fe528198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.768+0000 7fe52cd7f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5280ffd60 0x7fe528198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54318/0 (socket says 172.21.15.33:54318) 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.768+0000 7fe52cd7f700 1 -- 172.21.15.33:0/1905617117 learned_addr learned my addr 172.21.15.33:0/1905617117 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.768+0000 7fe52d580700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5281018f0 0x7fe5281a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe51ffff700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe528100f50 0x7fe528199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:56.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe52cd7f700 1 -- 172.21.15.33:0/1905617117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5281018f0 msgr2=0x7fe5281a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:56.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe52cd7f700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5281018f0 0x7fe5281a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe52cd7f700 1 -- 172.21.15.33:0/1905617117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe528100f50 msgr2=0x7fe528199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:56.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe52cd7f700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe528100f50 0x7fe528199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:56.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe52cd7f700 1 -- 172.21.15.33:0/1905617117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5240095f0 con 0x7fe5280ffd60 2024-05-08T21:18:56.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe51ffff700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe528100f50 0x7fe528199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:56.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe52cd7f700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5280ffd60 0x7fe528198f40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe524006010 tx=0x7fe52400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:56.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe51dffb700 1 -- 172.21.15.33:0/1905617117 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52401d070 con 0x7fe5280ffd60 2024-05-08T21:18:56.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe51dffb700 1 -- 172.21.15.33:0/1905617117 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5240054c0 con 0x7fe5280ffd60 2024-05-08T21:18:56.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe52efe3700 1 -- 172.21.15.33:0/1905617117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5281a0fc0 con 0x7fe5280ffd60 2024-05-08T21:18:56.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe51dffb700 1 -- 172.21.15.33:0/1905617117 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe524004bf0 con 0x7fe5280ffd60 2024-05-08T21:18:56.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.769+0000 7fe52efe3700 1 -- 172.21.15.33:0/1905617117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5281a1380 con 0x7fe5280ffd60 2024-05-08T21:18:56.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.771+0000 7fe52efe3700 1 -- 172.21.15.33:0/1905617117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe528067d10 con 0x7fe5280ffd60 2024-05-08T21:18:56.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.772+0000 7fe51dffb700 1 -- 172.21.15.33:0/1905617117 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fe524004d50 con 0x7fe5280ffd60 2024-05-08T21:18:56.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.773+0000 7fe51dffb700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe5100a1230 0x7fe5100a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:56.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.773+0000 7fe51ffff700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe5100a1230 0x7fe5100a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:56.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.773+0000 7fe51dffb700 1 -- 172.21.15.33:0/1905617117 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe5240cd8b0 con 0x7fe5280ffd60 2024-05-08T21:18:56.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.773+0000 7fe51ffff700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe5100a1230 0x7fe5100a36f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe518009cb0 tx=0x7fe518009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.778+0000 7fe51dffb700 1 -- 172.21.15.33:0/1905617117 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe524096d80 con 0x7fe5280ffd60 2024-05-08T21:18:56.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:56 smithi071 ceph-mon[47074]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:56 smithi071 ceph-mon[47074]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:56 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/120064500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:56 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/888422728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:56.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:56 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2204946967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:56.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:56 smithi082 ceph-mon[47512]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:56.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:56 smithi082 ceph-mon[47512]: from='client.15600 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T21:18:56.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:56 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/120064500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:56.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:56 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/888422728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:18:56.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:56 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2204946967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:18:56.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.994+0000 7fe52efe3700 1 -- 172.21.15.33:0/1905617117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe52810b690 con 0x7fe5280ffd60 2024-05-08T21:18:56.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.995+0000 7fe51dffb700 1 -- 172.21.15.33:0/1905617117 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe524026020 con 0x7fe5280ffd60 2024-05-08T21:18:56.997 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:18:56.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.997+0000 7fe52efe3700 1 -- 172.21.15.33:0/1905617117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe5100a1230 msgr2=0x7fe5100a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:56.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.997+0000 7fe52efe3700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe5100a1230 0x7fe5100a36f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe518009cb0 tx=0x7fe518009260 comp rx=0 tx=0).stop 2024-05-08T21:18:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.997+0000 7fe52efe3700 1 -- 172.21.15.33:0/1905617117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5280ffd60 msgr2=0x7fe528198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.997+0000 7fe52efe3700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5280ffd60 0x7fe528198f40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe524006010 tx=0x7fe52400f690 comp rx=0 tx=0).stop 2024-05-08T21:18:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.997+0000 7fe52efe3700 1 -- 172.21.15.33:0/1905617117 shutdown_connections 2024-05-08T21:18:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.997+0000 7fe52efe3700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5281018f0 0x7fe5281a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.997+0000 7fe52efe3700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5280ffd60 0x7fe528198f40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.997+0000 7fe52efe3700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe5100a1230 0x7fe5100a36f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.997+0000 7fe52efe3700 1 --2- 172.21.15.33:0/1905617117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe528100f50 0x7fe528199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.997+0000 7fe52efe3700 1 -- 172.21.15.33:0/1905617117 >> 172.21.15.33:0/1905617117 conn(0x7fe5280fb290 msgr2=0x7fe5280681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.997+0000 7fe52efe3700 1 -- 172.21.15.33:0/1905617117 shutdown_connections 2024-05-08T21:18:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:56.997+0000 7fe52efe3700 1 -- 172.21.15.33:0/1905617117 wait complete. 2024-05-08T21:18:57.013 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:18:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.148+0000 7fc53bca7700 1 -- 172.21.15.33:0/3985494268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5341010b0 msgr2=0x7fc534107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.148+0000 7fc53bca7700 1 --2- 172.21.15.33:0/3985494268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5341010b0 0x7fc534107960 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00b0d0 tx=0x7fc52c00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.148+0000 7fc53bca7700 1 -- 172.21.15.33:0/3985494268 shutdown_connections 2024-05-08T21:18:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.148+0000 7fc53bca7700 1 --2- 172.21.15.33:0/3985494268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5341010b0 0x7fc534107960 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.148+0000 7fc53bca7700 1 --2- 172.21.15.33:0/3985494268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5340ff520 0x7fc5340ff940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.148+0000 7fc53bca7700 1 --2- 172.21.15.33:0/3985494268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc534100710 0x7fc534100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.148+0000 7fc53bca7700 1 -- 172.21.15.33:0/3985494268 >> 172.21.15.33:0/3985494268 conn(0x7fc5340faa90 msgr2=0x7fc5340fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.148+0000 7fc53bca7700 1 -- 172.21.15.33:0/3985494268 shutdown_connections 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.148+0000 7fc53bca7700 1 -- 172.21.15.33:0/3985494268 wait complete. 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53bca7700 1 Processor -- start 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53bca7700 1 -- start start 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53bca7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5340ff520 0x7fc534196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53bca7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc534100710 0x7fc534197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53bca7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5341010b0 0x7fc53419e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53bca7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53419e950 con 0x7fc5341010b0 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53bca7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53419ea90 con 0x7fc5340ff520 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53bca7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53419ec20 con 0x7fc534100710 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53a244700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5341010b0 0x7fc53419e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53a244700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5341010b0 0x7fc53419e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54344/0 (socket says 172.21.15.33:54344) 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53a244700 1 -- 172.21.15.33:0/2457834279 learned_addr learned my addr 172.21.15.33:0/2457834279 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc539242700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc534100710 0x7fc534197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc539a43700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5340ff520 0x7fc534196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53a244700 1 -- 172.21.15.33:0/2457834279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc534100710 msgr2=0x7fc534197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53a244700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc534100710 0x7fc534197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53a244700 1 -- 172.21.15.33:0/2457834279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5340ff520 msgr2=0x7fc534196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53a244700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5340ff520 0x7fc534196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.149+0000 7fc53a244700 1 -- 172.21.15.33:0/2457834279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c009d00 con 0x7fc5341010b0 2024-05-08T21:18:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.150+0000 7fc53a244700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5341010b0 0x7fc53419e380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc52c007b60 tx=0x7fc52c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.150+0000 7fc526ffd700 1 -- 172.21.15.33:0/2457834279 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c00e040 con 0x7fc5341010b0 2024-05-08T21:18:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.150+0000 7fc526ffd700 1 -- 172.21.15.33:0/2457834279 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc52c01d480 con 0x7fc5341010b0 2024-05-08T21:18:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.150+0000 7fc53bca7700 1 -- 172.21.15.33:0/2457834279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53419eea0 con 0x7fc5341010b0 2024-05-08T21:18:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.150+0000 7fc526ffd700 1 -- 172.21.15.33:0/2457834279 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c01b6b0 con 0x7fc5341010b0 2024-05-08T21:18:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.150+0000 7fc53bca7700 1 -- 172.21.15.33:0/2457834279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53419f2c0 con 0x7fc5341010b0 2024-05-08T21:18:57.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.151+0000 7fc53bca7700 1 -- 172.21.15.33:0/2457834279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc534067480 con 0x7fc5341010b0 2024-05-08T21:18:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.153+0000 7fc526ffd700 1 -- 172.21.15.33:0/2457834279 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135924+0+0 (secure 0 0 0) 0x7fc52c01d5f0 con 0x7fc5341010b0 2024-05-08T21:18:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.153+0000 7fc526ffd700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5200a11b0 0x7fc5200a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.153+0000 7fc539a43700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5200a11b0 0x7fc5200a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.153+0000 7fc526ffd700 1 -- 172.21.15.33:0/2457834279 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc52c0cef90 con 0x7fc5341010b0 2024-05-08T21:18:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.154+0000 7fc539a43700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5200a11b0 0x7fc5200a3670 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc528005950 tx=0x7fc5280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.157+0000 7fc526ffd700 1 -- 172.21.15.33:0/2457834279 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc52c098340 con 0x7fc5341010b0 2024-05-08T21:18:57.411 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:57 smithi033 ceph-mon[42528]: pgmap v287: 129 pgs: 129 active+clean; 646 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-08T21:18:57.411 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-05-08T21:18:57.411 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-05-08T21:18:57.411 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-05-08T21:18:57.411 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:57 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-08T21:18:57.411 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:57 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1964012775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:57.411 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:57 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1905617117' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:57.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.433+0000 7fc526ffd700 1 -- 172.21.15.33:0/2457834279 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc52c098710 con 0x7fc5341010b0 2024-05-08T21:18:57.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.434+0000 7fc53bca7700 1 -- 172.21.15.33:0/2457834279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc53410a3f0 con 0x7fc5341010b0 2024-05-08T21:18:57.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.435+0000 7fc526ffd700 1 -- 172.21.15.33:0/2457834279 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10899 (secure 0 0 0) 0x7fc52c05d830 con 0x7fc5341010b0 2024-05-08T21:18:57.438 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:57.438 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:18:49.368057+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:18:57.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.438+0000 7fc53bca7700 1 -- 172.21.15.33:0/2457834279 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5200a11b0 msgr2=0x7fc5200a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:57.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.438+0000 7fc53bca7700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5200a11b0 0x7fc5200a3670 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc528005950 tx=0x7fc5280058e0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.438+0000 7fc53bca7700 1 -- 172.21.15.33:0/2457834279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5341010b0 msgr2=0x7fc53419e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:57.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.438+0000 7fc53bca7700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5341010b0 0x7fc53419e380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc52c007b60 tx=0x7fc52c009570 comp rx=0 tx=0).stop 2024-05-08T21:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.438+0000 7fc53bca7700 1 -- 172.21.15.33:0/2457834279 shutdown_connections 2024-05-08T21:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.438+0000 7fc53bca7700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5341010b0 0x7fc53419e380 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.438+0000 7fc53bca7700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5200a11b0 0x7fc5200a3670 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.438+0000 7fc53bca7700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5340ff520 0x7fc534196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.438+0000 7fc53bca7700 1 --2- 172.21.15.33:0/2457834279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc534100710 0x7fc534197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.438+0000 7fc53bca7700 1 -- 172.21.15.33:0/2457834279 >> 172.21.15.33:0/2457834279 conn(0x7fc5340faa90 msgr2=0x7fc534105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.438+0000 7fc53bca7700 1 -- 172.21.15.33:0/2457834279 shutdown_connections 2024-05-08T21:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.438+0000 7fc53bca7700 1 -- 172.21.15.33:0/2457834279 wait complete. 2024-05-08T21:18:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-05-08T21:18:57.454 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:18:57.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.590+0000 7f2056d05700 1 -- 172.21.15.33:0/2038358306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2050100f70 msgr2=0x7f20501013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:57.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.590+0000 7f2056d05700 1 --2- 172.21.15.33:0/2038358306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2050100f70 0x7f20501013d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f20400099c0 tx=0x7f2040009cd0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.590+0000 7f2056d05700 1 -- 172.21.15.33:0/2038358306 shutdown_connections 2024-05-08T21:18:57.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.590+0000 7f2056d05700 1 --2- 172.21.15.33:0/2038358306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2050101910 0x7f20501081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.590+0000 7f2056d05700 1 --2- 172.21.15.33:0/2038358306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20500ffd80 0x7f20501001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.590+0000 7f2056d05700 1 --2- 172.21.15.33:0/2038358306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2050100f70 0x7f20501013d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.590+0000 7f2056d05700 1 -- 172.21.15.33:0/2038358306 >> 172.21.15.33:0/2038358306 conn(0x7f20500fb310 msgr2=0x7f20500fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:57.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.591+0000 7f2056d05700 1 -- 172.21.15.33:0/2038358306 shutdown_connections 2024-05-08T21:18:57.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.591+0000 7f2056d05700 1 -- 172.21.15.33:0/2038358306 wait complete. 2024-05-08T21:18:57.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.591+0000 7f2056d05700 1 Processor -- start 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.591+0000 7f2056d05700 1 -- start start 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.591+0000 7f2056d05700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20500ffd80 0x7f2050103eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.591+0000 7f2056d05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2050100f70 0x7f2050102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.591+0000 7f2056d05700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2050101910 0x7f2050102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.591+0000 7f2056d05700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2050103220 con 0x7f2050100f70 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.591+0000 7f2056d05700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2050103390 con 0x7f2050101910 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.591+0000 7f2056d05700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20501a0d10 con 0x7f20500ffd80 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2047fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2050100f70 0x7f2050102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2047fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2050100f70 0x7f2050102540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54366/0 (socket says 172.21.15.33:54366) 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2047fff700 1 -- 172.21.15.33:0/472386921 learned_addr learned my addr 172.21.15.33:0/472386921 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f20552a2700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2050101910 0x7f2050102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2054aa1700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20500ffd80 0x7f2050103eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2047fff700 1 -- 172.21.15.33:0/472386921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20500ffd80 msgr2=0x7f2050103eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2047fff700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20500ffd80 0x7f2050103eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2047fff700 1 -- 172.21.15.33:0/472386921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2050101910 msgr2=0x7f2050102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2047fff700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2050101910 0x7f2050102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2047fff700 1 -- 172.21.15.33:0/472386921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20400095f0 con 0x7f2050100f70 2024-05-08T21:18:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2054aa1700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20500ffd80 0x7f2050103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2047fff700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2050100f70 0x7f2050102540 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2040000c00 tx=0x7f204000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2045ffb700 1 -- 172.21.15.33:0/472386921 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204001d070 con 0x7f2050100f70 2024-05-08T21:18:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2045ffb700 1 -- 172.21.15.33:0/472386921 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f204000bd30 con 0x7f2050100f70 2024-05-08T21:18:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2045ffb700 1 -- 172.21.15.33:0/472386921 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20400173f0 con 0x7f2050100f70 2024-05-08T21:18:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.592+0000 7f2056d05700 1 -- 172.21.15.33:0/472386921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20501a0f40 con 0x7f2050100f70 2024-05-08T21:18:57.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.593+0000 7f2056d05700 1 -- 172.21.15.33:0/472386921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20501a12b0 con 0x7f2050100f70 2024-05-08T21:18:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.594+0000 7f2056d05700 1 -- 172.21.15.33:0/472386921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2050067d10 con 0x7f2050100f70 2024-05-08T21:18:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.595+0000 7f2045ffb700 1 -- 172.21.15.33:0/472386921 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2040022470 con 0x7f2050100f70 2024-05-08T21:18:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.595+0000 7f2045ffb700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f20380a1120 0x7f20380a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.595+0000 7f2054aa1700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f20380a1120 0x7f20380a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.595+0000 7f2045ffb700 1 -- 172.21.15.33:0/472386921 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f20400cddd0 con 0x7f2050100f70 2024-05-08T21:18:57.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.596+0000 7f2054aa1700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f20380a1120 0x7f20380a35e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f204c005950 tx=0x7f204c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:57.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.600+0000 7f2045ffb700 1 -- 172.21.15.33:0/472386921 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f204005dd60 con 0x7f2050100f70 2024-05-08T21:18:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:57 smithi071 ceph-mon[47074]: pgmap v287: 129 pgs: 129 active+clean; 646 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-08T21:18:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-05-08T21:18:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-05-08T21:18:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-05-08T21:18:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:57 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-08T21:18:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:57 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1964012775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:57 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1905617117' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.825+0000 7f2056d05700 1 -- 172.21.15.33:0/472386921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f205010b440 con 0x7f2050100f70 2024-05-08T21:18:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.827+0000 7f2045ffb700 1 -- 172.21.15.33:0/472386921 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24561 (secure 0 0 0) 0x7f20400d1030 con 0x7f2050100f70 2024-05-08T21:18:57.829 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:18:57.829 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:18:57.406727+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:18:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.829+0000 7f2056d05700 1 -- 172.21.15.33:0/472386921 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f20380a1120 msgr2=0x7f20380a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.829+0000 7f2056d05700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f20380a1120 0x7f20380a35e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f204c005950 tx=0x7f204c0058e0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.829+0000 7f2056d05700 1 -- 172.21.15.33:0/472386921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2050100f70 msgr2=0x7f2050102540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.829+0000 7f2056d05700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2050100f70 0x7f2050102540 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2040000c00 tx=0x7f204000fa00 comp rx=0 tx=0).stop 2024-05-08T21:18:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.829+0000 7f2056d05700 1 -- 172.21.15.33:0/472386921 shutdown_connections 2024-05-08T21:18:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.829+0000 7f2056d05700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f20380a1120 0x7f20380a35e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.829+0000 7f2056d05700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2050101910 0x7f2050102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.829+0000 7f2056d05700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20500ffd80 0x7f2050103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.829+0000 7f2056d05700 1 --2- 172.21.15.33:0/472386921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2050100f70 0x7f2050102540 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.829+0000 7f2056d05700 1 -- 172.21.15.33:0/472386921 >> 172.21.15.33:0/472386921 conn(0x7f20500fb310 msgr2=0x7f2050068160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.829+0000 7f2056d05700 1 -- 172.21.15.33:0/472386921 shutdown_connections 2024-05-08T21:18:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:57.829+0000 7f2056d05700 1 -- 172.21.15.33:0/472386921 wait complete. 2024-05-08T21:18:57.845 DEBUG:tasks.fwd_scrub:Yielding 2024-05-08T21:18:57.846 INFO:teuthology.run_tasks:Running task workunit... 2024-05-08T21:18:57.858 INFO:tasks.workunit:Pulling workunits from ref d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:18:57.858 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-08T21:18:57.858 INFO:tasks.workunit:timeout=3h 2024-05-08T21:18:57.858 INFO:tasks.workunit:cleanup=True 2024-05-08T21:18:57.858 DEBUG:teuthology.orchestra.run.smithi033:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-08T21:18:57.859 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-08T21:18:57.859 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:18:57.859 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:18:57.860 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:57 smithi082 ceph-mon[47512]: pgmap v287: 129 pgs: 129 active+clean; 646 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-08T21:18:57.860 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-05-08T21:18:57.860 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-05-08T21:18:57.860 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]: dispatch 2024-05-08T21:18:57.860 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:57 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-08T21:18:57.860 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:57 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1964012775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:18:57.860 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:57 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1905617117' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:18:57.880 INFO:teuthology.orchestra.run.smithi033.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-08T21:18:57.881 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-08T21:18:57.881 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 7ch/124d Inode: 1 Links: 2 2024-05-08T21:18:57.881 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T21:18:57.881 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-08T21:18:57.881 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-05-08 21:17:31.752561190 +0000 2024-05-08T21:18:57.881 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-05-08 21:17:31.752561190 +0000 2024-05-08T21:18:57.881 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-05-08 21:18:47.973127365 +0000 2024-05-08T21:18:57.881 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-05-08T21:18:57.881 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-08T21:18:57.881 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-08T21:18:57.957 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d21e4f961318b56f0d915caa2490c3711bf13c34 2024-05-08T21:18:58.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.001+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3872505223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e4100a70 msgr2=0x7fb5e4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.001+0000 7fb5ea31a700 1 --2- 172.21.15.33:0/3872505223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e4100a70 0x7fb5e4100ef0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00b0a0 tx=0x7fb5dc00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.001+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3872505223 shutdown_connections 2024-05-08T21:18:58.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.001+0000 7fb5ea31a700 1 --2- 172.21.15.33:0/3872505223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e4100a70 0x7fb5e4100ef0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.001+0000 7fb5ea31a700 1 --2- 172.21.15.33:0/3872505223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5e40ff750 0x7fb5e40ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.001+0000 7fb5ea31a700 1 --2- 172.21.15.33:0/3872505223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e41000b0 0x7fb5e4100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.001+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3872505223 >> 172.21.15.33:0/3872505223 conn(0x7fb5e40fb290 msgr2=0x7fb5e40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:58.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.001+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3872505223 shutdown_connections 2024-05-08T21:18:58.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.001+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3872505223 wait complete. 2024-05-08T21:18:58.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.002+0000 7fb5ea31a700 1 Processor -- start 2024-05-08T21:18:58.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.002+0000 7fb5ea31a700 1 -- start start 2024-05-08T21:18:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5ea31a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e40ff750 0x7fb5e4198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5ea31a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e41000b0 0x7fb5e41994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5ea31a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5e4100a70 0x7fb5e41a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5ea31a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e41a0ac0 con 0x7fb5e41000b0 2024-05-08T21:18:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5ea31a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e41a0c00 con 0x7fb5e4100a70 2024-05-08T21:18:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5ea31a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e41a0d90 con 0x7fb5e40ff750 2024-05-08T21:18:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e41000b0 0x7fb5e41994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e41000b0 0x7fb5e41994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54374/0 (socket says 172.21.15.33:54374) 2024-05-08T21:18:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5e37fe700 1 -- 172.21.15.33:0/3087279932 learned_addr learned my addr 172.21.15.33:0/3087279932 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5e88b7700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5e4100a70 0x7fb5e41a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e40ff750 0x7fb5e4198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5e37fe700 1 -- 172.21.15.33:0/3087279932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e40ff750 msgr2=0x7fb5e4198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e40ff750 0x7fb5e4198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5e37fe700 1 -- 172.21.15.33:0/3087279932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5e4100a70 msgr2=0x7fb5e41a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5e4100a70 0x7fb5e41a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5e37fe700 1 -- 172.21.15.33:0/3087279932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8009520 con 0x7fb5e41000b0 2024-05-08T21:18:58.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.003+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e41000b0 0x7fb5e41994d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800eb30 tx=0x7fb5d800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:58.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.004+0000 7fb5e17fa700 1 -- 172.21.15.33:0/3087279932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8009880 con 0x7fb5e41000b0 2024-05-08T21:18:58.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.004+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3087279932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5dc009d00 con 0x7fb5e41000b0 2024-05-08T21:18:58.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.004+0000 7fb5e17fa700 1 -- 172.21.15.33:0/3087279932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5d80044b0 con 0x7fb5e41000b0 2024-05-08T21:18:58.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.004+0000 7fb5e17fa700 1 -- 172.21.15.33:0/3087279932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8003ce0 con 0x7fb5e41000b0 2024-05-08T21:18:58.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.004+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3087279932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e41a13f0 con 0x7fb5e41000b0 2024-05-08T21:18:58.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.006+0000 7fb5e17fa700 1 -- 172.21.15.33:0/3087279932 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb5d8003e40 con 0x7fb5e41000b0 2024-05-08T21:18:58.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.007+0000 7fb5e17fa700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5cc0a1120 0x7fb5cc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.007+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5cc0a1120 0x7fb5cc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.007+0000 7fb5e17fa700 1 -- 172.21.15.33:0/3087279932 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fb5d8014070 con 0x7fb5e41000b0 2024-05-08T21:18:58.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5f64d4b80 1 Processor -- start 2024-05-08T21:18:58.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5f64d4b80 1 -- start start 2024-05-08T21:18:58.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5cc0a1120 0x7fb5cc0a35e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009c60 tx=0x7fb5d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5f64d4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db645c0 0x558f7db649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5f64d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5f64d4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db658e0 0x558f7db65d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5f64d4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7dabb1d0 con 0x558f7db658e0 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5f64d4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7dab8b90 con 0x558f7db645c0 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5f64d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7dab4480 con 0x558f7db64f20 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5e88b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db658e0 0x558f7db65d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5e88b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db658e0 0x558f7db65d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54378/0 (socket says 172.21.15.33:54378) 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5e88b7700 1 -- 172.21.15.33:0/3690585092 learned_addr learned my addr 172.21.15.33:0/3690585092 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/3690585092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db645c0 0x558f7db649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.008+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/3690585092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.009+0000 7fb5e88b7700 1 -- 172.21.15.33:0/3690585092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 msgr2=0x558f7db653a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.009+0000 7fb5e88b7700 1 --2- 172.21.15.33:0/3690585092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db653a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.009+0000 7fb5e88b7700 1 -- 172.21.15.33:0/3690585092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db645c0 msgr2=0x558f7db649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.009+0000 7fb5e88b7700 1 --2- 172.21.15.33:0/3690585092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db645c0 0x558f7db649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.009+0000 7fb5e88b7700 1 -- 172.21.15.33:0/3690585092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f7d935430 con 0x558f7db658e0 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.009+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/3690585092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db645c0 0x558f7db649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.009+0000 7fb5e88b7700 1 --2- 172.21.15.33:0/3690585092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db658e0 0x558f7db65d60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009c20 tx=0x7fb5dc009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8435a94cad2768c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.009+0000 7fb5ca7fc700 1 -- 172.21.15.33:0/3690585092 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00e040 con 0x558f7db658e0 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.009+0000 7fb5ca7fc700 1 -- 172.21.15.33:0/3690585092 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5dc009490 con 0x558f7db658e0 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.009+0000 7fb5ca7fc700 1 -- 172.21.15.33:0/3690585092 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc003a00 con 0x558f7db658e0 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.010+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/3690585092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db658e0 msgr2=0x558f7db65d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.010+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/3690585092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db658e0 0x558f7db65d60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc009c20 tx=0x7fb5dc009c50 comp rx=0 tx=0).stop 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.010+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/3690585092 shutdown_connections 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.010+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/3690585092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db658e0 0x558f7db65d60 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.010+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/3690585092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db645c0 0x558f7db649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.010+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/3690585092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db653a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.010+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/3690585092 >> 172.21.15.33:0/3690585092 conn(0x558f7dac2290 msgr2=0x558f7db5ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:58.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.010+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/3690585092 shutdown_connections 2024-05-08T21:18:58.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.010+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/3690585092 wait complete. 2024-05-08T21:18:58.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.010+0000 7fb5f64d4b80 1 -- start start 2024-05-08T21:18:58.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5f64d4b80 1 Processor -- start 2024-05-08T21:18:58.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5f64d4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db6de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5f64d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db6e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5f64d4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db658e0 0x558f7db753a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5f64d4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7da505e0 con 0x558f7db645c0 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5f64d4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7da50280 con 0x558f7db658e0 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5f64d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7d937720 con 0x558f7db64f20 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5f64d4b80 10 client.0 _ll_register_callbacks cb 0x558f7da59bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db6de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5e37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db6e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db6de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54390/0 (socket says 172.21.15.33:54390) 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5e3fff700 1 -- 172.21.15.33:0/37872388 learned_addr learned my addr 172.21.15.33:0/37872388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.011+0000 7fb5e88b7700 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db658e0 0x558f7db753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5e37fe700 1 -- 172.21.15.33:0/37872388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db658e0 msgr2=0x558f7db753a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db658e0 0x558f7db753a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5e37fe700 1 -- 172.21.15.33:0/37872388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 msgr2=0x558f7db6de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db6de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5e37fe700 1 -- 172.21.15.33:0/37872388 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d4017070 con 0x558f7db64f20 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db6de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db6e360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80ca870 tx=0x7fb5d800ce80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3e4ad99c4f3665b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5c37fe700 1 -- 172.21.15.33:0/37872388 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80b6070 con 0x558f7db64f20 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5c37fe700 1 -- 172.21.15.33:0/37872388 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fb5d800cbd0 con 0x558f7db64f20 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/37872388 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f7da0d2f0 con 0x558f7db64f20 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5c37fe700 1 -- 172.21.15.33:0/37872388 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5d80b0070 con 0x558f7db64f20 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5f64d4b80 10 client.25543 fetch_fsmap learned FSMap version 29 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/37872388 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fb5e410bdd0 con 0x558f7db64f20 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.012+0000 7fb5c37fe700 1 -- 172.21.15.33:0/37872388 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558f7da0d2f0 con 0x558f7db64f20 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.013+0000 7fb5c37fe700 1 -- 172.21.15.33:0/37872388 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb5d809a420 con 0x558f7db64f20 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.013+0000 7fb5f64d4b80 10 client.25543 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.013+0000 7fb5f64d4b80 10 client.25543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.013+0000 7fb5f64d4b80 20 client.25543 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:58.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.013+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x558f7db79bc0 0x558f7db7bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.013+0000 7fb5f64d4b80 4 client.25543 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.013+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/37872388 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f7db33890 con 0x558f7db79bc0 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.013+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x558f7db79bc0 0x558f7db7bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.013+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x558f7db79bc0 0x558f7db7bfe0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37d04912b79a950b server_cookie=1b280d765877e543 in_seq=0 out_seq=0 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.013+0000 7fb5c37fe700 10 client.25543 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5c37fe700 1 -- 172.21.15.33:0/37872388 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f7db33890 con 0x558f7db79bc0 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5c37fe700 10 client.25543 handle_command_reply: tid=0 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5f64d4b80 1 client.25543 shutdown 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/37872388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 msgr2=0x558f7db6e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db6e360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80ca870 tx=0x7fb5d800ce80 comp rx=0 tx=0).stop 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/37872388 shutdown_connections 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x558f7db79bc0 0x558f7db7bfe0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db658e0 0x558f7db753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db6de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5c37fe700 0 client.25543 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:58.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/37872388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db6e360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/37872388 >> 172.21.15.33:0/37872388 conn(0x558f7dac2290 msgr2=0x558f7db5e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:58.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/37872388 shutdown_connections 2024-05-08T21:18:58.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.014+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/37872388 wait complete. 2024-05-08T21:18:58.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.015+0000 7fb5f64d4b80 20 client.25543 trim_cache size 0 max 16384 2024-05-08T21:18:58.022 INFO:tasks.workunit.client.0.smithi033.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-08T21:18:58.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.033+0000 7fb5f64d4b80 1 Processor -- start 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.033+0000 7fb5f64d4b80 1 -- start start 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.033+0000 7fb5f64d4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db68810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.033+0000 7fb5f64d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db75360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.033+0000 7fb5f64d4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db658e0 0x558f7db68d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.033+0000 7fb5f64d4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7db799c0 con 0x558f7db645c0 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.033+0000 7fb5f64d4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7db605f0 con 0x558f7db658e0 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.033+0000 7fb5f64d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7db59880 con 0x558f7db64f20 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.033+0000 7fb5e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db68810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.033+0000 7fb5e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db68810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54398/0 (socket says 172.21.15.33:54398) 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.033+0000 7fb5e3fff700 1 -- 172.21.15.33:0/2579749383 learned_addr learned my addr 172.21.15.33:0/2579749383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.033+0000 7fb5e88b7700 1 --2- 172.21.15.33:0/2579749383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db658e0 0x558f7db68d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.034+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/2579749383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db75360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.034+0000 7fb5e3fff700 1 -- 172.21.15.33:0/2579749383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 msgr2=0x558f7db75360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.034+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/2579749383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db75360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.034+0000 7fb5e3fff700 1 -- 172.21.15.33:0/2579749383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db658e0 msgr2=0x558f7db68d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.034+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/2579749383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db658e0 0x558f7db68d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.034+0000 7fb5e3fff700 1 -- 172.21.15.33:0/2579749383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f7d8e1be0 con 0x558f7db645c0 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.034+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/2579749383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db75360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.034+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/2579749383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db68810 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4011ab0 tx=0x7fb5d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba4acbe62032e6a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.034+0000 7fb5caffd700 1 -- 172.21.15.33:0/2579749383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4022030 con 0x558f7db645c0 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.034+0000 7fb5caffd700 1 -- 172.21.15.33:0/2579749383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5d4023040 con 0x558f7db645c0 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.034+0000 7fb5caffd700 1 -- 172.21.15.33:0/2579749383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d402b460 con 0x558f7db645c0 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.035+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2579749383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 msgr2=0x558f7db68810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.035+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/2579749383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db68810 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4011ab0 tx=0x7fb5d4014000 comp rx=0 tx=0).stop 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.035+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2579749383 shutdown_connections 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.035+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/2579749383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db658e0 0x558f7db68d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.035+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/2579749383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db68810 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.035+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/2579749383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db64f20 0x558f7db75360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.035+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2579749383 >> 172.21.15.33:0/2579749383 conn(0x558f7dac2290 msgr2=0x558f7db80940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.035+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2579749383 shutdown_connections 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.035+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2579749383 wait complete. 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.035+0000 7fb5f64d4b80 1 -- start start 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5f64d4b80 1 Processor -- start 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5f64d4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db72010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5f64d4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db64f20 0x558f7db72550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5f64d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db658e0 0x558f7dbd00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5f64d4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7db75480 con 0x558f7db645c0 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5f64d4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7db755f0 con 0x558f7db64f20 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5f64d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f7dbd05f0 con 0x558f7db658e0 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5f64d4b80 10 client.0 _ll_register_callbacks cb 0x558f7da59bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db72010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db72010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54406/0 (socket says 172.21.15.33:54406) 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5e3fff700 1 -- 172.21.15.33:0/2508475550 learned_addr learned my addr 172.21.15.33:0/2508475550 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db64f20 0x558f7db72550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5e88b7700 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db658e0 0x558f7dbd00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5e3fff700 1 -- 172.21.15.33:0/2508475550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db658e0 msgr2=0x558f7dbd00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db658e0 0x558f7dbd00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5e3fff700 1 -- 172.21.15.33:0/2508475550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db64f20 msgr2=0x558f7db72550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db64f20 0x558f7db72550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5e3fff700 1 -- 172.21.15.33:0/2508475550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d401dbb0 con 0x558f7db645c0 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.036+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db64f20 0x558f7db72550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5e3fff700 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db72010 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40097c0 tx=0x7fb5d40057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f68567cb86cc22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5c37fe700 1 -- 172.21.15.33:0/2508475550 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4022070 con 0x558f7db645c0 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5c37fe700 1 -- 172.21.15.33:0/2508475550 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fb5d4005250 con 0x558f7db645c0 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2508475550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5dc009490 con 0x558f7db645c0 2024-05-08T21:18:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5c37fe700 1 -- 172.21.15.33:0/2508475550 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5d4005410 con 0x558f7db645c0 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5f64d4b80 10 client.15660 fetch_fsmap learned FSMap version 29 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2508475550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x558f7da7a5e0 con 0x558f7db645c0 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5c37fe700 1 -- 172.21.15.33:0/2508475550 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5dc009490 con 0x558f7db645c0 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5c37fe700 1 -- 172.21.15.33:0/2508475550 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb5d4032070 con 0x558f7db645c0 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5f64d4b80 10 client.15660 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5f64d4b80 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5f64d4b80 20 client.15660 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x558f7dbd1e80 0x558f7dbd42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5f64d4b80 4 client.15660 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.037+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2508475550 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558f7daacf40 con 0x558f7dbd1e80 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.038+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x558f7dbd1e80 0x558f7dbd42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.038+0000 7fb5c37fe700 10 client.15660 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.038+0000 7fb5e37fe700 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x558f7dbd1e80 0x558f7dbd42a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1012a712b0e314fc server_cookie=42955fe52f413217 in_seq=0 out_seq=0 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.039+0000 7fb5c37fe700 1 -- 172.21.15.33:0/2508475550 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558f7daacf40 con 0x558f7dbd1e80 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.039+0000 7fb5c37fe700 10 client.15660 handle_command_reply: tid=0 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.039+0000 7fb5f64d4b80 1 client.15660 shutdown 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.039+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2508475550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 msgr2=0x558f7db72010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.039+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db72010 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40097c0 tx=0x7fb5d40057a0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.039+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2508475550 shutdown_connections 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.039+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x558f7dbd1e80 0x558f7dbd42a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.039+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f7db658e0 0x558f7dbd00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.039+0000 7fb5c37fe700 0 client.15660 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.039+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558f7db645c0 0x558f7db72010 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.039+0000 7fb5f64d4b80 1 --2- 172.21.15.33:0/2508475550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558f7db64f20 0x558f7db72550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.039+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2508475550 >> 172.21.15.33:0/2508475550 conn(0x558f7dac2290 msgr2=0x558f7db68810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:58.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2508475550 shutdown_connections 2024-05-08T21:18:58.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5f64d4b80 1 -- 172.21.15.33:0/2508475550 wait complete. 2024-05-08T21:18:58.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5f64d4b80 20 client.15660 trim_cache size 0 max 16384 2024-05-08T21:18:58.048 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:58.048 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:18:58.048 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "bd4115b1-0e8f-4705-8033-a7625d0dd4c0", 2024-05-08T21:18:58.048 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:18:58.048 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:58.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3087279932 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5cc0a1120 msgr2=0x7fb5cc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5ea31a700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5cc0a1120 0x7fb5cc0a35e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009c60 tx=0x7fb5d4009260 comp rx=0 tx=0).stop 2024-05-08T21:18:58.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3087279932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e41000b0 msgr2=0x7fb5e41994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5ea31a700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e41000b0 0x7fb5e41994d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800eb30 tx=0x7fb5d800ee40 comp rx=0 tx=0).stop 2024-05-08T21:18:58.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3087279932 shutdown_connections 2024-05-08T21:18:58.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5ea31a700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5cc0a1120 0x7fb5cc0a35e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5ea31a700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5e4100a70 0x7fb5e41a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5ea31a700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5e40ff750 0x7fb5e4198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5ea31a700 1 --2- 172.21.15.33:0/3087279932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5e41000b0 0x7fb5e41994d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3087279932 >> 172.21.15.33:0/3087279932 conn(0x7fb5e40fb290 msgr2=0x7fb5e40fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3087279932 shutdown_connections 2024-05-08T21:18:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.040+0000 7fb5ea31a700 1 -- 172.21.15.33:0/3087279932 wait complete. 2024-05-08T21:18:58.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:18:58.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.240+0000 7f21abbe0700 1 -- 172.21.15.33:0/3054681372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21a40ff120 msgr2=0x7f21a40ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.240+0000 7f21abbe0700 1 --2- 172.21.15.33:0/3054681372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21a40ff120 0x7f21a40ff540 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009990 tx=0x7f21a0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.240+0000 7f21abbe0700 1 -- 172.21.15.33:0/3054681372 shutdown_connections 2024-05-08T21:18:58.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.240+0000 7f21abbe0700 1 --2- 172.21.15.33:0/3054681372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21a4102a90 0x7f21a4109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.240+0000 7f21abbe0700 1 --2- 172.21.15.33:0/3054681372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21a40ff120 0x7f21a40ff540 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.240+0000 7f21abbe0700 1 --2- 172.21.15.33:0/3054681372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a41020f0 0x7f21a4102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.240+0000 7f21abbe0700 1 -- 172.21.15.33:0/3054681372 >> 172.21.15.33:0/3054681372 conn(0x7f21a40faa50 msgr2=0x7f21a40fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:58.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.240+0000 7f21abbe0700 1 -- 172.21.15.33:0/3054681372 shutdown_connections 2024-05-08T21:18:58.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.240+0000 7f21abbe0700 1 -- 172.21.15.33:0/3054681372 wait complete. 2024-05-08T21:18:58.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.241+0000 7f21abbe0700 1 Processor -- start 2024-05-08T21:18:58.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.241+0000 7f21abbe0700 1 -- start start 2024-05-08T21:18:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.241+0000 7f21abbe0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21a40ff120 0x7f21a4196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.241+0000 7f21abbe0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a41020f0 0x7f21a4197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.241+0000 7f21abbe0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21a4102a90 0x7f21a419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.241+0000 7f21abbe0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a419e8f0 con 0x7f21a40ff120 2024-05-08T21:18:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.241+0000 7f21abbe0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a419ea30 con 0x7f21a41020f0 2024-05-08T21:18:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.241+0000 7f21abbe0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a419ebc0 con 0x7f21a4102a90 2024-05-08T21:18:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f21a997c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21a40ff120 0x7f21a4196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f21a997c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21a40ff120 0x7f21a4196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54418/0 (socket says 172.21.15.33:54418) 2024-05-08T21:18:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f21a997c700 1 -- 172.21.15.33:0/2156049740 learned_addr learned my addr 172.21.15.33:0/2156049740 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f21aa17d700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21a4102a90 0x7f21a419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f21a917b700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a41020f0 0x7f21a4197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f21a997c700 1 -- 172.21.15.33:0/2156049740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21a4102a90 msgr2=0x7f21a419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f21a997c700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21a4102a90 0x7f21a419e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f21a997c700 1 -- 172.21.15.33:0/2156049740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a41020f0 msgr2=0x7f21a4197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f21a997c700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a41020f0 0x7f21a4197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f21a997c700 1 -- 172.21.15.33:0/2156049740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a00095f0 con 0x7f21a40ff120 2024-05-08T21:18:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f21a997c700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21a40ff120 0x7f21a4196dc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f21a0005910 tx=0x7f21a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f219affd700 1 -- 172.21.15.33:0/2156049740 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a001d070 con 0x7f21a40ff120 2024-05-08T21:18:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f219affd700 1 -- 172.21.15.33:0/2156049740 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21a000beb0 con 0x7f21a40ff120 2024-05-08T21:18:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f219affd700 1 -- 172.21.15.33:0/2156049740 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0022e40 con 0x7f21a40ff120 2024-05-08T21:18:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.242+0000 7f21abbe0700 1 -- 172.21.15.33:0/2156049740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a419ee40 con 0x7f21a40ff120 2024-05-08T21:18:58.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.243+0000 7f21abbe0700 1 -- 172.21.15.33:0/2156049740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a419f1b0 con 0x7f21a40ff120 2024-05-08T21:18:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.245+0000 7f219affd700 1 -- 172.21.15.33:0/2156049740 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f21a00225e0 con 0x7f21a40ff120 2024-05-08T21:18:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.245+0000 7f219affd700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21900a1120 0x7f21900a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.246+0000 7f21a917b700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21900a1120 0x7f21900a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.246+0000 7f219affd700 1 -- 172.21.15.33:0/2156049740 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f21a00ce1d0 con 0x7f21a40ff120 2024-05-08T21:18:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.246+0000 7f21a917b700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21900a1120 0x7f21900a35e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2194009bf0 tx=0x7f2194009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.246+0000 7f21b7d9ab80 1 Processor -- start 2024-05-08T21:18:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21b7d9ab80 1 -- start start 2024-05-08T21:18:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21b7d9ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 0x55e0b3d72960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21b7d9ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d73320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21b7d9ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d73860 0x55e0b3d73ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21b7d9ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3cc9150 con 0x55e0b3d72540 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21b7d9ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3cc6b10 con 0x55e0b3d73860 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21b7d9ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3cc2400 con 0x55e0b3d72ea0 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21a997c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 0x55e0b3d72960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21aa17d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d73860 0x55e0b3d73ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21a997c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 0x55e0b3d72960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54434/0 (socket says 172.21.15.33:54434) 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21a997c700 1 -- 172.21.15.33:0/2405080241 learned_addr learned my addr 172.21.15.33:0/2405080241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21a917b700 1 --2- 172.21.15.33:0/2405080241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d73320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21aa17d700 1 -- 172.21.15.33:0/2405080241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d72ea0 msgr2=0x55e0b3d73320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21aa17d700 1 --2- 172.21.15.33:0/2405080241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d73320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21aa17d700 1 -- 172.21.15.33:0/2405080241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 msgr2=0x55e0b3d72960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21aa17d700 1 --2- 172.21.15.33:0/2405080241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 0x55e0b3d72960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21aa17d700 1 -- 172.21.15.33:0/2405080241 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0b3b433b0 con 0x55e0b3d73860 2024-05-08T21:18:58.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.247+0000 7f21a997c700 1 --2- 172.21.15.33:0/2405080241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 0x55e0b3d72960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:18:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.248+0000 7f21aa17d700 1 --2- 172.21.15.33:0/2405080241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d73860 0x55e0b3d73ce0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f219c003ec0 tx=0x7f219c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30eb6f511e0555e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.248+0000 7f218ffff700 1 -- 172.21.15.33:0/2405080241 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c019070 con 0x55e0b3d73860 2024-05-08T21:18:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.248+0000 7f218ffff700 1 -- 172.21.15.33:0/2405080241 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f219c00f040 con 0x55e0b3d73860 2024-05-08T21:18:58.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.248+0000 7f218ffff700 1 -- 172.21.15.33:0/2405080241 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c0078c0 con 0x55e0b3d73860 2024-05-08T21:18:58.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.249+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2405080241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d73860 msgr2=0x55e0b3d73ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.249+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/2405080241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d73860 0x55e0b3d73ce0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f219c003ec0 tx=0x7f219c009c20 comp rx=0 tx=0).stop 2024-05-08T21:18:58.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.249+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2405080241 shutdown_connections 2024-05-08T21:18:58.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.249+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/2405080241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d73860 0x55e0b3d73ce0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.249+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/2405080241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 0x55e0b3d72960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.249+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/2405080241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d73320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.249+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2405080241 >> 172.21.15.33:0/2405080241 conn(0x55e0b3cd0210 msgr2=0x55e0b3d6cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.249+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2405080241 shutdown_connections 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.249+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2405080241 wait complete. 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.249+0000 7f21b7d9ab80 1 -- start start 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.249+0000 7f21b7d9ab80 1 Processor -- start 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21b7d9ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 0x55e0b3d7bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21b7d9ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d7c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21b7d9ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d73860 0x55e0b3d83340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21b7d9ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3c5e560 con 0x55e0b3d72540 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21b7d9ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3c5e200 con 0x55e0b3d72ea0 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21b7d9ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3b456a0 con 0x55e0b3d73860 2024-05-08T21:18:58.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21b7d9ab80 10 client.0 _ll_register_callbacks cb 0x55e0b3c67b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21a997c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 0x55e0b3d7bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21a997c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 0x55e0b3d7bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54448/0 (socket says 172.21.15.33:54448) 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21a997c700 1 -- 172.21.15.33:0/2445763112 learned_addr learned my addr 172.21.15.33:0/2445763112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21a917b700 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d7c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21aa17d700 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d73860 0x55e0b3d83340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21a997c700 1 -- 172.21.15.33:0/2445763112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d73860 msgr2=0x55e0b3d83340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21a997c700 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d73860 0x55e0b3d83340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21a997c700 1 -- 172.21.15.33:0/2445763112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d72ea0 msgr2=0x55e0b3d7c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21a997c700 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d7c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21a997c700 1 -- 172.21.15.33:0/2445763112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a419f1b0 con 0x55e0b3d72540 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.250+0000 7f21aa17d700 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d73860 0x55e0b3d83340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f21a997c700 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 0x55e0b3d7bdc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f21a0005cd0 tx=0x7f21a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7796a3fe277865fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f218d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:58.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f218d7fa700 1 -- 172.21.15.33:0/2445763112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a00b0030 con 0x55e0b3d72540 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f218d7fa700 1 -- 172.21.15.33:0/2445763112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f21a00b1070 con 0x55e0b3d72540 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2445763112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0b3c1b270 con 0x55e0b3d72540 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f218d7fa700 1 -- 172.21.15.33:0/2445763112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21a009b9f0 con 0x55e0b3d72540 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f21b7d9ab80 10 client.15672 fetch_fsmap learned FSMap version 29 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2445763112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f219c003ac0 con 0x55e0b3d72540 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f218d7fa700 1 -- 172.21.15.33:0/2445763112 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e0b3c1b270 con 0x55e0b3d72540 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f218d7fa700 1 -- 172.21.15.33:0/2445763112 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f21a00a10b0 con 0x55e0b3d72540 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f21b7d9ab80 10 client.15672 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f21b7d9ab80 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f21b7d9ab80 20 client.15672 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e0b3d87860 0x55e0b3d89c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f21b7d9ab80 4 client.15672 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.251+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2445763112 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0b3d41810 con 0x55e0b3d87860 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.252+0000 7f21a917b700 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e0b3d87860 0x55e0b3d89c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.252+0000 7f218d7fa700 10 client.15672 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.252+0000 7f21a917b700 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e0b3d87860 0x55e0b3d89c80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a666e33c40081b38 server_cookie=ed7e122e29c2910a in_seq=0 out_seq=0 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f218d7fa700 1 -- 172.21.15.33:0/2445763112 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0b3d41810 con 0x55e0b3d87860 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f218d7fa700 10 client.15672 handle_command_reply: tid=0 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f21b7d9ab80 1 client.15672 shutdown 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2445763112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 msgr2=0x55e0b3d7bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 0x55e0b3d7bdc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f21a0005cd0 tx=0x7f21a0005fd0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2445763112 shutdown_connections 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e0b3d87860 0x55e0b3d89c80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d73860 0x55e0b3d83340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72540 0x55e0b3d7bdc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/2445763112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d7c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f218d7fa700 0 client.15672 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2445763112 >> 172.21.15.33:0/2445763112 conn(0x55e0b3cd0210 msgr2=0x55e0b3d6c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2445763112 shutdown_connections 2024-05-08T21:18:58.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/2445763112 wait complete. 2024-05-08T21:18:58.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.253+0000 7f21b7d9ab80 20 client.15672 trim_cache size 0 max 16384 2024-05-08T21:18:58.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.272+0000 7f21b7d9ab80 1 Processor -- start 2024-05-08T21:18:58.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.272+0000 7f21b7d9ab80 1 -- start start 2024-05-08T21:18:58.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.272+0000 7f21b7d9ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d72540 0x55e0b3d839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.272+0000 7f21b7d9ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d83300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.272+0000 7f21b7d9ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d73860 0x55e0b3d79e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.272+0000 7f21b7d9ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3d6e830 con 0x55e0b3d72ea0 2024-05-08T21:18:58.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.272+0000 7f21b7d9ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3d65040 con 0x55e0b3d73860 2024-05-08T21:18:58.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.272+0000 7f21b7d9ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3d770c0 con 0x55e0b3d72540 2024-05-08T21:18:58.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.272+0000 7f21a917b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d83300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.272+0000 7f21a917b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d83300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54450/0 (socket says 172.21.15.33:54450) 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f21a917b700 1 -- 172.21.15.33:0/1751185217 learned_addr learned my addr 172.21.15.33:0/1751185217 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f21a997c700 1 --2- 172.21.15.33:0/1751185217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d72540 0x55e0b3d839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f21aa17d700 1 --2- 172.21.15.33:0/1751185217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d73860 0x55e0b3d79e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f21a917b700 1 -- 172.21.15.33:0/1751185217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d72540 msgr2=0x55e0b3d839f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f21a917b700 1 --2- 172.21.15.33:0/1751185217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d72540 0x55e0b3d839f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f21a917b700 1 -- 172.21.15.33:0/1751185217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d73860 msgr2=0x55e0b3d79e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f21a917b700 1 --2- 172.21.15.33:0/1751185217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d73860 0x55e0b3d79e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f21a917b700 1 -- 172.21.15.33:0/1751185217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0b3aefb60 con 0x55e0b3d72ea0 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f21aa17d700 1 --2- 172.21.15.33:0/1751185217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d73860 0x55e0b3d79e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f21a917b700 1 --2- 172.21.15.33:0/1751185217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d83300 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2194011c80 tx=0x7f2194014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d574ef8a8cc0f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f2198da4700 1 -- 172.21.15.33:0/1751185217 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194024040 con 0x55e0b3d72ea0 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f2198da4700 1 -- 172.21.15.33:0/1751185217 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2194016070 con 0x55e0b3d72ea0 2024-05-08T21:18:58.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.273+0000 7f2198da4700 1 -- 172.21.15.33:0/1751185217 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21940038e0 con 0x55e0b3d72ea0 2024-05-08T21:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.274+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/1751185217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 msgr2=0x55e0b3d83300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.274+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/1751185217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d83300 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2194011c80 tx=0x7f2194014000 comp rx=0 tx=0).stop 2024-05-08T21:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.274+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/1751185217 shutdown_connections 2024-05-08T21:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.274+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/1751185217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d73860 0x55e0b3d79e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.274+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/1751185217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d72540 0x55e0b3d839f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.274+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/1751185217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d83300 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.274+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/1751185217 >> 172.21.15.33:0/1751185217 conn(0x55e0b3cd0210 msgr2=0x55e0b3d7b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.274+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/1751185217 shutdown_connections 2024-05-08T21:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.274+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/1751185217 wait complete. 2024-05-08T21:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21b7d9ab80 1 -- start start 2024-05-08T21:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21b7d9ab80 1 Processor -- start 2024-05-08T21:18:58.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21b7d9ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d72540 0x55e0b3d82fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21b7d9ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d7faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21b7d9ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d73860 0x55e0b3d7ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21b7d9ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3d80520 con 0x55e0b3d72ea0 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21b7d9ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3d80690 con 0x55e0b3d72540 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21b7d9ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0b3dddd50 con 0x55e0b3d73860 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21b7d9ab80 10 client.0 _ll_register_callbacks cb 0x55e0b3c67b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21a917b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d7faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21a917b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d7faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54460/0 (socket says 172.21.15.33:54460) 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21a917b700 1 -- 172.21.15.33:0/3871495625 learned_addr learned my addr 172.21.15.33:0/3871495625 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.275+0000 7f21aa17d700 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d73860 0x55e0b3d7ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.276+0000 7f21a997c700 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d72540 0x55e0b3d82fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.276+0000 7f21a917b700 1 -- 172.21.15.33:0/3871495625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d73860 msgr2=0x55e0b3d7ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.276+0000 7f21a917b700 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d73860 0x55e0b3d7ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.276+0000 7f21a917b700 1 -- 172.21.15.33:0/3871495625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d72540 msgr2=0x55e0b3d82fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:18:58.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.276+0000 7f21a917b700 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d72540 0x55e0b3d82fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.276+0000 7f21a917b700 1 -- 172.21.15.33:0/3871495625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2194012450 con 0x55e0b3d72ea0 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.276+0000 7f21a917b700 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d7faa0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2194012010 tx=0x7f2194012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6db03820e1de60b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.276+0000 7f218d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.276+0000 7f218d7fa700 1 -- 172.21.15.33:0/3871495625 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194024050 con 0x55e0b3d72ea0 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.276+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/3871495625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f219c00f040 con 0x55e0b3d72ea0 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.276+0000 7f218d7fa700 1 -- 172.21.15.33:0/3871495625 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f2194018070 con 0x55e0b3d72ea0 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.277+0000 7f218d7fa700 1 -- 172.21.15.33:0/3871495625 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2194005240 con 0x55e0b3d72ea0 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.277+0000 7f21b7d9ab80 10 client.15678 fetch_fsmap learned FSMap version 29 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.277+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/3871495625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e0b3c88560 con 0x55e0b3d72ea0 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.277+0000 7f218d7fa700 1 -- 172.21.15.33:0/3871495625 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f219c00f040 con 0x55e0b3d72ea0 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.277+0000 7f218d7fa700 1 -- 172.21.15.33:0/3871495625 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2194030020 con 0x55e0b3d72ea0 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.277+0000 7f21b7d9ab80 10 client.15678 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.277+0000 7f21b7d9ab80 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:18:58.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.277+0000 7f21b7d9ab80 20 client.15678 populate_metadata read hostname 'smithi033' 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.277+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e0b3ddf5c0 0x55e0b3de19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.277+0000 7f21b7d9ab80 4 client.15678 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.277+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/3871495625 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e0b3cbaec0 con 0x55e0b3ddf5c0 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.277+0000 7f21a997c700 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e0b3ddf5c0 0x55e0b3de19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f218d7fa700 10 client.15678 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21a997c700 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e0b3ddf5c0 0x55e0b3de19e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48b88594f1b32d43 server_cookie=ca5be03b0250621d in_seq=0 out_seq=0 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f218d7fa700 1 -- 172.21.15.33:0/3871495625 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e0b3cbaec0 con 0x55e0b3ddf5c0 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f218d7fa700 10 client.15678 handle_command_reply: tid=0 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21b7d9ab80 1 client.15678 shutdown 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/3871495625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 msgr2=0x55e0b3d7faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d7faa0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2194012010 tx=0x7f2194012e60 comp rx=0 tx=0).stop 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/3871495625 shutdown_connections 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e0b3ddf5c0 0x55e0b3de19e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e0b3d73860 0x55e0b3d7ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e0b3d72540 0x55e0b3d82fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21b7d9ab80 1 --2- 172.21.15.33:0/3871495625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0b3d72ea0 0x55e0b3d7faa0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/3871495625 >> 172.21.15.33:0/3871495625 conn(0x55e0b3cd0210 msgr2=0x55e0b3d7bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:58.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f218d7fa700 0 client.15678 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:18:58.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/3871495625 shutdown_connections 2024-05-08T21:18:58.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21b7d9ab80 1 -- 172.21.15.33:0/3871495625 wait complete. 2024-05-08T21:18:58.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.278+0000 7f21b7d9ab80 20 client.15678 trim_cache size 0 max 16384 2024-05-08T21:18:58.286 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:18:58.286 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:18:58.286 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:18:58.286 INFO:teuthology.orchestra.run.smithi033.stdout: "bd4115b1-0e8f-4705-8033-a7625d0dd4c0": { 2024-05-08T21:18:58.287 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:18:58.287 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "bd4115b1-0e8f-4705-8033-a7625d0dd4c0", 2024-05-08T21:18:58.287 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:18:58.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:18:58.287 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:18:58.287 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:18:58.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.279+0000 7f21abbe0700 1 -- 172.21.15.33:0/2156049740 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21900a1120 msgr2=0x7f21900a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.279+0000 7f21abbe0700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21900a1120 0x7f21900a35e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2194009bf0 tx=0x7f2194009260 comp rx=0 tx=0).stop 2024-05-08T21:18:58.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.279+0000 7f21abbe0700 1 -- 172.21.15.33:0/2156049740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21a40ff120 msgr2=0x7f21a4196dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:18:58.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.279+0000 7f21abbe0700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21a40ff120 0x7f21a4196dc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f21a0005910 tx=0x7f21a0004730 comp rx=0 tx=0).stop 2024-05-08T21:18:58.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.279+0000 7f21abbe0700 1 -- 172.21.15.33:0/2156049740 shutdown_connections 2024-05-08T21:18:58.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.279+0000 7f21abbe0700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21a4102a90 0x7f21a419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.279+0000 7f21abbe0700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21a40ff120 0x7f21a4196dc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.279+0000 7f21abbe0700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21900a1120 0x7f21900a35e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.279+0000 7f21abbe0700 1 --2- 172.21.15.33:0/2156049740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a41020f0 0x7f21a4197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:18:58.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.279+0000 7f21abbe0700 1 -- 172.21.15.33:0/2156049740 >> 172.21.15.33:0/2156049740 conn(0x7f21a40faa50 msgr2=0x7f21a41055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:18:58.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.279+0000 7f21abbe0700 1 -- 172.21.15.33:0/2156049740 shutdown_connections 2024-05-08T21:18:58.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:18:58.279+0000 7f21abbe0700 1 -- 172.21.15.33:0/2156049740 wait complete. 2024-05-08T21:18:58.295 INFO:tasks.cephfs.filesystem:scrub status for tag:bd4115b1-0e8f-4705-8033-a7625d0dd4c0 - {'path': '/', 'tag': 'bd4115b1-0e8f-4705-8033-a7625d0dd4c0', 'options': 'recursive,force'} 2024-05-08T21:18:58.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-05-08T21:18:58.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-05-08T21:18:58.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-05-08T21:18:58.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:58 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-08T21:18:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:58 smithi033 ceph-mon[42528]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T21:18:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:58 smithi033 ceph-mon[42528]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-08T21:18:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:58 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2457834279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:58 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/472386921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:58 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:18:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:58 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:18:58.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:58 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:18:58.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:58 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:18:58.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-05-08T21:18:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-05-08T21:18:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-05-08T21:18:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:58 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-08T21:18:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:58 smithi071 ceph-mon[47074]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T21:18:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:58 smithi071 ceph-mon[47074]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-08T21:18:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:58 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2457834279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:58 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/472386921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:58 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:18:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:58 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:18:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:58 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:18:58.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:58 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:18:58.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-05-08T21:18:58.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-05-08T21:18:58.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2a", "id": [3, 8]}]': finished 2024-05-08T21:18:58.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:58 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-08T21:18:58.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:58 smithi082 ceph-mon[47512]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T21:18:58.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:58 smithi082 ceph-mon[47512]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-05-08T21:18:58.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:58 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2457834279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:18:58.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:58 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/472386921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:18:58.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:58 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:18:58.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:58 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:18:58.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:58 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:18:58.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:58 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:18:59.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:59 smithi033 ceph-mon[42528]: pgmap v289: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-08T21:18:59.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:18:59 smithi033 ceph-mon[42528]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T21:18:59.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:59 smithi071 ceph-mon[47074]: pgmap v289: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-08T21:18:59.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:18:59 smithi071 ceph-mon[47074]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T21:18:59.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:59 smithi082 ceph-mon[47512]: pgmap v289: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-05-08T21:18:59.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:18:59 smithi082 ceph-mon[47512]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T21:19:01.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:01 smithi033 ceph-mon[42528]: pgmap v291: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-08T21:19:01.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:01 smithi071 ceph-mon[47074]: pgmap v291: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-08T21:19:01.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:01 smithi082 ceph-mon[47512]: pgmap v291: 129 pgs: 129 active+clean; 651 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-08T21:19:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:03 smithi033 ceph-mon[42528]: pgmap v292: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-05-08T21:19:03.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:03 smithi071 ceph-mon[47074]: pgmap v292: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-05-08T21:19:03.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:03 smithi082 ceph-mon[47512]: pgmap v292: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-05-08T21:19:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:05 smithi033 ceph-mon[42528]: pgmap v293: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-08T21:19:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:05 smithi071 ceph-mon[47074]: pgmap v293: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-08T21:19:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:05 smithi082 ceph-mon[47512]: pgmap v293: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-08T21:19:07.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:07 smithi033 ceph-mon[42528]: pgmap v294: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-08T21:19:07.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:07 smithi071 ceph-mon[47074]: pgmap v294: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-08T21:19:07.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:07 smithi082 ceph-mon[47512]: pgmap v294: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-08T21:19:09.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:09 smithi033 ceph-mon[42528]: pgmap v295: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 818 B/s wr, 10 op/s 2024-05-08T21:19:09.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:09 smithi071 ceph-mon[47074]: pgmap v295: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 818 B/s wr, 10 op/s 2024-05-08T21:19:09.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:09 smithi082 ceph-mon[47512]: pgmap v295: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 818 B/s wr, 10 op/s 2024-05-08T21:19:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:11 smithi033 ceph-mon[42528]: pgmap v296: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 785 B/s wr, 10 op/s 2024-05-08T21:19:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:11 smithi071 ceph-mon[47074]: pgmap v296: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 785 B/s wr, 10 op/s 2024-05-08T21:19:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:11 smithi082 ceph-mon[47512]: pgmap v296: 129 pgs: 129 active+clean; 656 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 785 B/s wr, 10 op/s 2024-05-08T21:19:12.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:12 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:19:12.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:12 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:19:12.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:12 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:19:12.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:12 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:19:12.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:12 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:19:12.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:12 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:19:13.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:13 smithi071 ceph-mon[47074]: pgmap v297: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-08T21:19:13.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:19:13.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:19:13.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:13 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:13 smithi082 ceph-mon[47512]: pgmap v297: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-08T21:19:13.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:19:13.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:19:13.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:13 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:13 smithi033 ceph-mon[42528]: pgmap v297: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-08T21:19:13.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:13.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:19:13.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:19:13.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:13 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:19:15.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:15 smithi071 ceph-mon[47074]: pgmap v298: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-08T21:19:15.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:15 smithi082 ceph-mon[47512]: pgmap v298: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-08T21:19:15.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:15 smithi033 ceph-mon[42528]: pgmap v298: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-08T21:19:17.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:17 smithi071 ceph-mon[47074]: pgmap v299: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-08T21:19:17.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:17 smithi082 ceph-mon[47512]: pgmap v299: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-08T21:19:17.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:17 smithi033 ceph-mon[42528]: pgmap v299: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-05-08T21:19:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:19 smithi071 ceph-mon[47074]: pgmap v300: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T21:19:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:19 smithi082 ceph-mon[47512]: pgmap v300: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T21:19:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:19 smithi033 ceph-mon[42528]: pgmap v300: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-08T21:19:20.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:20 smithi071 ceph-mon[47074]: pgmap v301: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T21:19:20.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:20 smithi082 ceph-mon[47512]: pgmap v301: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T21:19:20.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:20 smithi033 ceph-mon[42528]: pgmap v301: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-08T21:19:23.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:23 smithi071 ceph-mon[47074]: pgmap v302: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-08T21:19:23.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:23 smithi082 ceph-mon[47512]: pgmap v302: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-08T21:19:23.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:23 smithi033 ceph-mon[42528]: pgmap v302: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-08T21:19:25.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:25 smithi071 ceph-mon[47074]: pgmap v303: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:19:25.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:25 smithi082 ceph-mon[47512]: pgmap v303: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:19:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:25 smithi033 ceph-mon[42528]: pgmap v303: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:19:27.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:27 smithi071 ceph-mon[47074]: pgmap v304: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:19:27.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:27 smithi082 ceph-mon[47512]: pgmap v304: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:19:27.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:27 smithi033 ceph-mon[42528]: pgmap v304: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:19:28.296 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:19:28.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.448+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/1822615266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e4101950 msgr2=0x7fd7e4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.448+0000 7fd7ea7ec700 1 --2- 172.21.15.33:0/1822615266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e4101950 0x7fd7e4108200 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009fc0 tx=0x7fd7dc00b330 comp rx=0 tx=0).stop 2024-05-08T21:19:28.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.448+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/1822615266 shutdown_connections 2024-05-08T21:19:28.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.448+0000 7fd7ea7ec700 1 --2- 172.21.15.33:0/1822615266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e4101950 0x7fd7e4108200 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.448+0000 7fd7ea7ec700 1 --2- 172.21.15.33:0/1822615266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e40ffdc0 0x7fd7e41001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.448+0000 7fd7ea7ec700 1 --2- 172.21.15.33:0/1822615266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7e4100fb0 0x7fd7e4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.448+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/1822615266 >> 172.21.15.33:0/1822615266 conn(0x7fd7e40fb330 msgr2=0x7fd7e40fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:28.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.449+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/1822615266 shutdown_connections 2024-05-08T21:19:28.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.449+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/1822615266 wait complete. 2024-05-08T21:19:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.449+0000 7fd7ea7ec700 1 Processor -- start 2024-05-08T21:19:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.449+0000 7fd7ea7ec700 1 -- start start 2024-05-08T21:19:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7ea7ec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e40ffdc0 0x7fd7e4196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7ea7ec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7e4100fb0 0x7fd7e41973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7ea7ec700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e4101950 0x7fd7e419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7ea7ec700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e419e990 con 0x7fd7e4100fb0 2024-05-08T21:19:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7ea7ec700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e419ead0 con 0x7fd7e4101950 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7ea7ec700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e419ec60 con 0x7fd7e40ffdc0 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7e4100fb0 0x7fd7e41973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7e4100fb0 0x7fd7e41973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48836/0 (socket says 172.21.15.33:48836) 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7e37fe700 1 -- 172.21.15.33:0/3752648152 learned_addr learned my addr 172.21.15.33:0/3752648152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7e3fff700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e40ffdc0 0x7fd7e4196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7e8d89700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e4101950 0x7fd7e419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7e37fe700 1 -- 172.21.15.33:0/3752648152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e40ffdc0 msgr2=0x7fd7e4196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7e37fe700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e40ffdc0 0x7fd7e4196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7e37fe700 1 -- 172.21.15.33:0/3752648152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e4101950 msgr2=0x7fd7e419e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7e37fe700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e4101950 0x7fd7e419e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7e37fe700 1 -- 172.21.15.33:0/3752648152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d80095f0 con 0x7fd7e4100fb0 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.450+0000 7fd7e8d89700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e4101950 0x7fd7e419e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.451+0000 7fd7e37fe700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7e4100fb0 0x7fd7e41973f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800ec10 tx=0x7fd7d800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.451+0000 7fd7e17fa700 1 -- 172.21.15.33:0/3752648152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d800cd40 con 0x7fd7e4100fb0 2024-05-08T21:19:28.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.451+0000 7fd7e17fa700 1 -- 172.21.15.33:0/3752648152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7d80044b0 con 0x7fd7e4100fb0 2024-05-08T21:19:28.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.451+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/3752648152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7dc009c40 con 0x7fd7e4100fb0 2024-05-08T21:19:28.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.451+0000 7fd7e17fa700 1 -- 172.21.15.33:0/3752648152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8010430 con 0x7fd7e4100fb0 2024-05-08T21:19:28.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.451+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/3752648152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e419f260 con 0x7fd7e4100fb0 2024-05-08T21:19:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.453+0000 7fd7e17fa700 1 -- 172.21.15.33:0/3752648152 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd7d8010590 con 0x7fd7e4100fb0 2024-05-08T21:19:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.454+0000 7fd7e17fa700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7cc0a1120 0x7fd7cc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.454+0000 7fd7e17fa700 1 -- 172.21.15.33:0/3752648152 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd7d8014070 con 0x7fd7e4100fb0 2024-05-08T21:19:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.454+0000 7fd7e3fff700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7cc0a1120 0x7fd7cc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7e3fff700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7cc0a1120 0x7fd7cc0a35e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4100e10 tx=0x7fd7d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7f69a6b80 1 Processor -- start 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7f69a6b80 1 -- start start 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7f69a6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 0x555638de3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7f69a6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de3ea0 0x555638de4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7f69a6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de4860 0x555638de4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7f69a6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555638d3a150 con 0x555638de4860 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7f69a6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555638d37b10 con 0x555638de3540 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7f69a6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555638d33400 con 0x555638de3ea0 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7e8d89700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de4860 0x555638de4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7e8d89700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de4860 0x555638de4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48846/0 (socket says 172.21.15.33:48846) 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7e8d89700 1 -- 172.21.15.33:0/4162392286 learned_addr learned my addr 172.21.15.33:0/4162392286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7e3fff700 1 --2- 172.21.15.33:0/4162392286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 0x555638de3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7e37fe700 1 --2- 172.21.15.33:0/4162392286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de3ea0 0x555638de4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7e8d89700 1 -- 172.21.15.33:0/4162392286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de3ea0 msgr2=0x555638de4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7e8d89700 1 --2- 172.21.15.33:0/4162392286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de3ea0 0x555638de4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7e8d89700 1 -- 172.21.15.33:0/4162392286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 msgr2=0x555638de3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:28.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7e8d89700 1 --2- 172.21.15.33:0/4162392286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 0x555638de3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.455+0000 7fd7e8d89700 1 -- 172.21.15.33:0/4162392286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555638bb43b0 con 0x555638de4860 2024-05-08T21:19:28.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.456+0000 7fd7e8d89700 1 --2- 172.21.15.33:0/4162392286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de4860 0x555638de4ce0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009be0 tx=0x7fd7dc0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e941b2380b3535e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:28.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.456+0000 7fd7ca7fc700 1 -- 172.21.15.33:0/4162392286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc00e050 con 0x555638de4860 2024-05-08T21:19:28.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.456+0000 7fd7ca7fc700 1 -- 172.21.15.33:0/4162392286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7dc007620 con 0x555638de4860 2024-05-08T21:19:28.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.456+0000 7fd7ca7fc700 1 -- 172.21.15.33:0/4162392286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc00bcc0 con 0x555638de4860 2024-05-08T21:19:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.457+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4162392286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de4860 msgr2=0x555638de4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.457+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/4162392286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de4860 0x555638de4ce0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009be0 tx=0x7fd7dc0092e0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.457+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4162392286 shutdown_connections 2024-05-08T21:19:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.457+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/4162392286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de4860 0x555638de4ce0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.457+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/4162392286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 0x555638de3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.457+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/4162392286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de3ea0 0x555638de4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.457+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4162392286 >> 172.21.15.33:0/4162392286 conn(0x555638d41210 msgr2=0x555638dddde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.457+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4162392286 shutdown_connections 2024-05-08T21:19:28.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.457+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4162392286 wait complete. 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.457+0000 7fd7f69a6b80 1 -- start start 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.457+0000 7fd7f69a6b80 1 Processor -- start 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7f69a6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 0x555638decd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7f69a6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 0x555638ded2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7f69a6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de4860 0x555638df42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7f69a6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555638ccf560 con 0x555638de3ea0 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7f69a6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555638ccf200 con 0x555638de3540 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7f69a6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555638bb66a0 con 0x555638de4860 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7f69a6b80 10 client.0 _ll_register_callbacks cb 0x555638cd8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 0x555638ded2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 0x555638ded2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48856/0 (socket says 172.21.15.33:48856) 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7e3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 0x555638decd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7e37fe700 1 -- 172.21.15.33:0/542563128 learned_addr learned my addr 172.21.15.33:0/542563128 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7e8d89700 1 --2- 172.21.15.33:0/542563128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de4860 0x555638df42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7e37fe700 1 -- 172.21.15.33:0/542563128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de4860 msgr2=0x555638df42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7e37fe700 1 --2- 172.21.15.33:0/542563128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de4860 0x555638df42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7e37fe700 1 -- 172.21.15.33:0/542563128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 msgr2=0x555638decd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7e37fe700 1 --2- 172.21.15.33:0/542563128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 0x555638decd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7e37fe700 1 -- 172.21.15.33:0/542563128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e4108b50 con 0x555638de3ea0 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7e37fe700 1 --2- 172.21.15.33:0/542563128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 0x555638ded2b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8099f70 tx=0x7fd7d8099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f523e4ac3a81d1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.458+0000 7fd7c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7c37fe700 1 -- 172.21.15.33:0/542563128 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8003710 con 0x555638de3ea0 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7c37fe700 1 -- 172.21.15.33:0/542563128 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd7d809a970 con 0x555638de3ea0 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/542563128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555638c8c270 con 0x555638de3ea0 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7c37fe700 1 -- 172.21.15.33:0/542563128 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7d80965b0 con 0x555638de3ea0 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7f69a6b80 10 client.15690 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/542563128 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555638b60b60 con 0x555638de3ea0 2024-05-08T21:19:28.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7c37fe700 1 -- 172.21.15.33:0/542563128 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555638c8c270 con 0x555638de3ea0 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7c37fe700 1 -- 172.21.15.33:0/542563128 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7d8099880 con 0x555638de3ea0 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7f69a6b80 10 client.15690 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7f69a6b80 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7f69a6b80 20 client.15690 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/542563128 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555638df8bd0 0x555638dfaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7f69a6b80 4 client.15690 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.459+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/542563128 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555638db2810 con 0x555638df8bd0 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.460+0000 7fd7e3fff700 1 --2- 172.21.15.33:0/542563128 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555638df8bd0 0x555638dfaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.460+0000 7fd7e3fff700 1 --2- 172.21.15.33:0/542563128 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555638df8bd0 0x555638dfaff0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bad5ff8d83a4143d server_cookie=a9335f09327dce59 in_seq=0 out_seq=0 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.460+0000 7fd7c37fe700 10 client.15690 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7c37fe700 1 -- 172.21.15.33:0/542563128 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555638db2810 con 0x555638df8bd0 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7c37fe700 10 client.15690 handle_command_reply: tid=0 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7f69a6b80 1 client.15690 shutdown 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/542563128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 msgr2=0x555638ded2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/542563128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 0x555638ded2b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8099f70 tx=0x7fd7d8099bd0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/542563128 shutdown_connections 2024-05-08T21:19:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/542563128 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555638df8bd0 0x555638dfaff0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/542563128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de4860 0x555638df42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7c37fe700 0 client.15690 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/542563128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 0x555638decd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/542563128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 0x555638ded2b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/542563128 >> 172.21.15.33:0/542563128 conn(0x555638d41210 msgr2=0x555638ddd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/542563128 shutdown_connections 2024-05-08T21:19:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/542563128 wait complete. 2024-05-08T21:19:28.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.461+0000 7fd7f69a6b80 20 client.15690 trim_cache size 0 max 16384 2024-05-08T21:19:28.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.479+0000 7fd7f69a6b80 1 Processor -- start 2024-05-08T21:19:28.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.479+0000 7fd7f69a6b80 1 -- start start 2024-05-08T21:19:28.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.479+0000 7fd7f69a6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3540 0x555638dec280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.479+0000 7fd7f69a6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de3ea0 0x555638df42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.479+0000 7fd7f69a6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de4860 0x555638de76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.479+0000 7fd7f69a6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555638debdb0 con 0x555638de3540 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.479+0000 7fd7f69a6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555638df5470 con 0x555638de4860 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.479+0000 7fd7f69a6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555638dec8e0 con 0x555638de3ea0 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.479+0000 7fd7e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3540 0x555638dec280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3540 0x555638dec280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48864/0 (socket says 172.21.15.33:48864) 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7e3fff700 1 -- 172.21.15.33:0/785030780 learned_addr learned my addr 172.21.15.33:0/785030780 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7e37fe700 1 --2- 172.21.15.33:0/785030780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de3ea0 0x555638df42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7e8d89700 1 --2- 172.21.15.33:0/785030780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de4860 0x555638de76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7e3fff700 1 -- 172.21.15.33:0/785030780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de3ea0 msgr2=0x555638df42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7e3fff700 1 --2- 172.21.15.33:0/785030780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de3ea0 0x555638df42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7e3fff700 1 -- 172.21.15.33:0/785030780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de4860 msgr2=0x555638de76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7e3fff700 1 --2- 172.21.15.33:0/785030780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de4860 0x555638de76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7e3fff700 1 -- 172.21.15.33:0/785030780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555638cf9560 con 0x555638de3540 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7e37fe700 1 --2- 172.21.15.33:0/785030780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de3ea0 0x555638df42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7e8d89700 1 --2- 172.21.15.33:0/785030780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de4860 0x555638de76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7e3fff700 1 --2- 172.21.15.33:0/785030780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3540 0x555638dec280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x555638dfb900 tx=0x7fd7d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234d4903c0b48d05 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7caffd700 1 -- 172.21.15.33:0/785030780 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4022030 con 0x555638de3540 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7caffd700 1 -- 172.21.15.33:0/785030780 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7d4023040 con 0x555638de3540 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.480+0000 7fd7caffd700 1 -- 172.21.15.33:0/785030780 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d4003a00 con 0x555638de3540 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.481+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/785030780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3540 msgr2=0x555638dec280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.481+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/785030780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3540 0x555638dec280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x555638dfb900 tx=0x7fd7d4014000 comp rx=0 tx=0).stop 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.481+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/785030780 shutdown_connections 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.481+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/785030780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de4860 0x555638de76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.481+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/785030780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3540 0x555638dec280 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.481+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/785030780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de3ea0 0x555638df42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.481+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/785030780 >> 172.21.15.33:0/785030780 conn(0x555638d41210 msgr2=0x555638de4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:28.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.481+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/785030780 shutdown_connections 2024-05-08T21:19:28.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.481+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/785030780 wait complete. 2024-05-08T21:19:28.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.481+0000 7fd7f69a6b80 1 -- start start 2024-05-08T21:19:28.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7f69a6b80 1 Processor -- start 2024-05-08T21:19:28.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7f69a6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 0x555638df0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7f69a6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 0x555638df06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7f69a6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de4860 0x555638df3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7f69a6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555638df3bd0 con 0x555638de3ea0 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7f69a6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555638df3d40 con 0x555638de3540 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7f69a6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555638df3f00 con 0x555638de4860 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7f69a6b80 10 client.0 _ll_register_callbacks cb 0x555638cd8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 0x555638df06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7e37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 0x555638df06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48870/0 (socket says 172.21.15.33:48870) 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7e37fe700 1 -- 172.21.15.33:0/4174080298 learned_addr learned my addr 172.21.15.33:0/4174080298 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7e8d89700 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de4860 0x555638df3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7e3fff700 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 0x555638df0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7e37fe700 1 -- 172.21.15.33:0/4174080298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de4860 msgr2=0x555638df3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7e37fe700 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de4860 0x555638df3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7e37fe700 1 -- 172.21.15.33:0/4174080298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 msgr2=0x555638df0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:28.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7e37fe700 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 0x555638df0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.482+0000 7fd7e37fe700 1 -- 172.21.15.33:0/4174080298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d4017070 con 0x555638de3ea0 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7e37fe700 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 0x555638df06c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8085010 tx=0x7fd7d8015fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497f9f4b8d690050 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7c37fe700 1 -- 172.21.15.33:0/4174080298 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8099960 con 0x555638de3ea0 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7c37fe700 1 -- 172.21.15.33:0/4174080298 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd7d8003710 con 0x555638de3ea0 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4174080298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7dc007620 con 0x555638de3ea0 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7c37fe700 1 -- 172.21.15.33:0/4174080298 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7d80a2440 con 0x555638de3ea0 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7f69a6b80 10 client.15696 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4174080298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555638bb8cd0 con 0x555638de3ea0 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7c37fe700 1 -- 172.21.15.33:0/4174080298 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7dc007620 con 0x555638de3ea0 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7c37fe700 1 -- 172.21.15.33:0/4174080298 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7d8096060 con 0x555638de3ea0 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7f69a6b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7f69a6b80 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7f69a6b80 20 client.15696 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:28.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555638e4fcb0 0x555638e520d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7f69a6b80 4 client.15696 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.483+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4174080298 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555638d2bec0 con 0x555638e4fcb0 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.484+0000 7fd7e3fff700 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555638e4fcb0 0x555638e520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.484+0000 7fd7e3fff700 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555638e4fcb0 0x555638e520d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=567609bb651b21fc server_cookie=28ca13d332916e6 in_seq=0 out_seq=0 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.484+0000 7fd7c37fe700 10 client.15696 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.484+0000 7fd7c37fe700 1 -- 172.21.15.33:0/4174080298 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555638d2bec0 con 0x555638e4fcb0 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.484+0000 7fd7c37fe700 10 client.15696 handle_command_reply: tid=0 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.484+0000 7fd7f69a6b80 1 client.15696 shutdown 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.484+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4174080298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 msgr2=0x555638df06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.484+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 0x555638df06c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8085010 tx=0x7fd7d8015fc0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4174080298 shutdown_connections 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555638e4fcb0 0x555638e520d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555638de4860 0x555638df3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555638de3540 0x555638df0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7f69a6b80 1 --2- 172.21.15.33:0/4174080298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555638de3ea0 0x555638df06c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7c37fe700 0 client.15696 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4174080298 >> 172.21.15.33:0/4174080298 conn(0x555638d41210 msgr2=0x555638de76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4174080298 shutdown_connections 2024-05-08T21:19:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7f69a6b80 1 -- 172.21.15.33:0/4174080298 wait complete. 2024-05-08T21:19:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7f69a6b80 20 client.15696 trim_cache size 0 max 16384 2024-05-08T21:19:28.493 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:19:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:19:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:19:28.493 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:19:28.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/3752648152 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7cc0a1120 msgr2=0x7fd7cc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7ea7ec700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7cc0a1120 0x7fd7cc0a35e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4100e10 tx=0x7fd7d4009260 comp rx=0 tx=0).stop 2024-05-08T21:19:28.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/3752648152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7e4100fb0 msgr2=0x7fd7e41973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7ea7ec700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7e4100fb0 0x7fd7e41973f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800ec10 tx=0x7fd7d800ef20 comp rx=0 tx=0).stop 2024-05-08T21:19:28.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/3752648152 shutdown_connections 2024-05-08T21:19:28.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7ea7ec700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7cc0a1120 0x7fd7cc0a35e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7ea7ec700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e4101950 0x7fd7e419e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7ea7ec700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7e40ffdc0 0x7fd7e4196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7ea7ec700 1 --2- 172.21.15.33:0/3752648152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7e4100fb0 0x7fd7e41973f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.485+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/3752648152 >> 172.21.15.33:0/3752648152 conn(0x7fd7e40fb330 msgr2=0x7fd7e4069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:28.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.486+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/3752648152 shutdown_connections 2024-05-08T21:19:28.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.486+0000 7fd7ea7ec700 1 -- 172.21.15.33:0/3752648152 wait complete. 2024-05-08T21:19:28.501 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:19:28.502 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:19:28.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.639+0000 7fa208612700 1 -- 172.21.15.33:0/1844059873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2001042d0 msgr2=0x7fa2001066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.639+0000 7fa208612700 1 --2- 172.21.15.33:0/1844059873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2001042d0 0x7fa2001066c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009990 tx=0x7fa1f0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.639+0000 7fa208612700 1 -- 172.21.15.33:0/1844059873 shutdown_connections 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.639+0000 7fa208612700 1 --2- 172.21.15.33:0/1844059873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa200106c00 0x7fa200108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.639+0000 7fa208612700 1 --2- 172.21.15.33:0/1844059873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2001019a0 0x7fa200103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.639+0000 7fa208612700 1 --2- 172.21.15.33:0/1844059873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2001042d0 0x7fa2001066c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.639+0000 7fa208612700 1 -- 172.21.15.33:0/1844059873 >> 172.21.15.33:0/1844059873 conn(0x7fa2000fb310 msgr2=0x7fa2000fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.639+0000 7fa208612700 1 -- 172.21.15.33:0/1844059873 shutdown_connections 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.639+0000 7fa208612700 1 -- 172.21.15.33:0/1844059873 wait complete. 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa208612700 1 Processor -- start 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa208612700 1 -- start start 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa208612700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2001019a0 0x7fa200198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa208612700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2001042d0 0x7fa2001993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa208612700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa200106c00 0x7fa2001a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa208612700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001a09e0 con 0x7fa200106c00 2024-05-08T21:19:28.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa208612700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001a0b20 con 0x7fa2001019a0 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa208612700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2001a0cb0 con 0x7fa2001042d0 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa206baf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa200106c00 0x7fa2001a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa2063ae700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2001019a0 0x7fa200198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa206baf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa200106c00 0x7fa2001a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48884/0 (socket says 172.21.15.33:48884) 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa206baf700 1 -- 172.21.15.33:0/3622747833 learned_addr learned my addr 172.21.15.33:0/3622747833 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa205bad700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2001042d0 0x7fa2001993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.640+0000 7fa206baf700 1 -- 172.21.15.33:0/3622747833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2001042d0 msgr2=0x7fa2001993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.641+0000 7fa206baf700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2001042d0 0x7fa2001993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.641+0000 7fa206baf700 1 -- 172.21.15.33:0/3622747833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2001019a0 msgr2=0x7fa200198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.641+0000 7fa206baf700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2001019a0 0x7fa200198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.641+0000 7fa206baf700 1 -- 172.21.15.33:0/3622747833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f00095f0 con 0x7fa200106c00 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.641+0000 7fa2063ae700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2001019a0 0x7fa200198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.641+0000 7fa205bad700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2001042d0 0x7fa2001993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.641+0000 7fa206baf700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa200106c00 0x7fa2001a0410 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800ef30 tx=0x7fa1f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:28.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.641+0000 7fa1f77fe700 1 -- 172.21.15.33:0/3622747833 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8018070 con 0x7fa200106c00 2024-05-08T21:19:28.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.641+0000 7fa1f77fe700 1 -- 172.21.15.33:0/3622747833 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa1f800f040 con 0x7fa200106c00 2024-05-08T21:19:28.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.641+0000 7fa208612700 1 -- 172.21.15.33:0/3622747833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2001a0f30 con 0x7fa200106c00 2024-05-08T21:19:28.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.641+0000 7fa1f77fe700 1 -- 172.21.15.33:0/3622747833 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800cc70 con 0x7fa200106c00 2024-05-08T21:19:28.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.641+0000 7fa208612700 1 -- 172.21.15.33:0/3622747833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2001a1360 con 0x7fa200106c00 2024-05-08T21:19:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.642+0000 7fa208612700 1 -- 172.21.15.33:0/3622747833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa200108780 con 0x7fa200106c00 2024-05-08T21:19:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.644+0000 7fa1f77fe700 1 -- 172.21.15.33:0/3622747833 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa1f8003a00 con 0x7fa200106c00 2024-05-08T21:19:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.644+0000 7fa1f77fe700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa1ec0a1120 0x7fa1ec0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.644+0000 7fa2063ae700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa1ec0a1120 0x7fa1ec0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.644+0000 7fa1f77fe700 1 -- 172.21.15.33:0/3622747833 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa1f80ccd60 con 0x7fa200106c00 2024-05-08T21:19:28.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.645+0000 7fa2063ae700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa1ec0a1120 0x7fa1ec0a35e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc005950 tx=0x7fa1fc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:28.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.648+0000 7fa1f77fe700 1 -- 172.21.15.33:0/3622747833 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa1f8096200 con 0x7fa200106c00 2024-05-08T21:19:28.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.918+0000 7fa208612700 1 -- 172.21.15.33:0/3622747833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa200067d10 con 0x7fa200106c00 2024-05-08T21:19:28.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.918+0000 7fa1f77fe700 1 -- 172.21.15.33:0/3622747833 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10899 (secure 0 0 0) 0x7fa1f8003cb0 con 0x7fa200106c00 2024-05-08T21:19:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:19:28.921 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:18:49.368057+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:19:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.921+0000 7fa208612700 1 -- 172.21.15.33:0/3622747833 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa1ec0a1120 msgr2=0x7fa1ec0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.921+0000 7fa208612700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa1ec0a1120 0x7fa1ec0a35e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc005950 tx=0x7fa1fc00b410 comp rx=0 tx=0).stop 2024-05-08T21:19:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.921+0000 7fa208612700 1 -- 172.21.15.33:0/3622747833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa200106c00 msgr2=0x7fa2001a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.921+0000 7fa208612700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa200106c00 0x7fa2001a0410 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800ef30 tx=0x7fa1f800c3a0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.921+0000 7fa208612700 1 -- 172.21.15.33:0/3622747833 shutdown_connections 2024-05-08T21:19:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.921+0000 7fa208612700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa200106c00 0x7fa2001a0410 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.921+0000 7fa208612700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa1ec0a1120 0x7fa1ec0a35e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.921+0000 7fa208612700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2001019a0 0x7fa200198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.921+0000 7fa208612700 1 --2- 172.21.15.33:0/3622747833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2001042d0 0x7fa2001993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.921+0000 7fa208612700 1 -- 172.21.15.33:0/3622747833 >> 172.21.15.33:0/3622747833 conn(0x7fa2000fb310 msgr2=0x7fa200101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.921+0000 7fa208612700 1 -- 172.21.15.33:0/3622747833 shutdown_connections 2024-05-08T21:19:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:28.921+0000 7fa208612700 1 -- 172.21.15.33:0/3622747833 wait complete. 2024-05-08T21:19:28.924 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-05-08T21:19:28.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:19:29.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.074+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/367672779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c68101900 msgr2=0x7f2c68103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.074+0000 7f2c6d6b5700 1 --2- 172.21.15.33:0/367672779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c68101900 0x7f2c68103cf0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009990 tx=0x7f2c58009ca0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.074+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/367672779 shutdown_connections 2024-05-08T21:19:29.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.074+0000 7f2c6d6b5700 1 --2- 172.21.15.33:0/367672779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c68106b60 0x7f2c68108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.074+0000 7f2c6d6b5700 1 --2- 172.21.15.33:0/367672779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c68101900 0x7f2c68103cf0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.074+0000 7f2c6d6b5700 1 --2- 172.21.15.33:0/367672779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c68104230 0x7f2c68106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.074+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/367672779 >> 172.21.15.33:0/367672779 conn(0x7f2c680fb310 msgr2=0x7f2c680fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.074+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/367672779 shutdown_connections 2024-05-08T21:19:29.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.074+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/367672779 wait complete. 2024-05-08T21:19:29.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.075+0000 7f2c6d6b5700 1 Processor -- start 2024-05-08T21:19:29.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.075+0000 7f2c6d6b5700 1 -- start start 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.075+0000 7f2c6d6b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c68101900 0x7f2c68101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.075+0000 7f2c6d6b5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c68104230 0x7f2c680ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.075+0000 7f2c6d6b5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c68106b60 0x7f2c680ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.075+0000 7f2c6d6b5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68100590 con 0x7f2c68101900 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.075+0000 7f2c6d6b5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c68100700 con 0x7f2c68104230 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.075+0000 7f2c6d6b5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c681a0d30 con 0x7f2c68106b60 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.075+0000 7f2c66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c68101900 0x7f2c68101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.075+0000 7f2c66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c68101900 0x7f2c68101220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48902/0 (socket says 172.21.15.33:48902) 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.075+0000 7f2c66ffd700 1 -- 172.21.15.33:0/602892515 learned_addr learned my addr 172.21.15.33:0/602892515 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.075+0000 7f2c667fc700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c68104230 0x7f2c680ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c677fe700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c68106b60 0x7f2c680ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c66ffd700 1 -- 172.21.15.33:0/602892515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c68106b60 msgr2=0x7f2c680ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c68106b60 0x7f2c680ffdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c66ffd700 1 -- 172.21.15.33:0/602892515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c68104230 msgr2=0x7f2c680ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c68104230 0x7f2c680ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c66ffd700 1 -- 172.21.15.33:0/602892515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c580095f0 con 0x7f2c68101900 2024-05-08T21:19:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c677fe700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c68106b60 0x7f2c680ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c68101900 0x7f2c68101220 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2c580098e0 tx=0x7f2c58003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c4ffff700 1 -- 172.21.15.33:0/602892515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5801d070 con 0x7f2c68101900 2024-05-08T21:19:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c4ffff700 1 -- 172.21.15.33:0/602892515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c5800bd90 con 0x7f2c68101900 2024-05-08T21:19:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/602892515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c681a0f60 con 0x7f2c68101900 2024-05-08T21:19:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c4ffff700 1 -- 172.21.15.33:0/602892515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58022d20 con 0x7f2c68101900 2024-05-08T21:19:29.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.076+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/602892515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c681a12d0 con 0x7f2c68101900 2024-05-08T21:19:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.079+0000 7f2c4ffff700 1 -- 172.21.15.33:0/602892515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2c58003f40 con 0x7f2c68101900 2024-05-08T21:19:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.080+0000 7f2c4ffff700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2c500a1120 0x7f2c500a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.080+0000 7f2c4ffff700 1 -- 172.21.15.33:0/602892515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f2c580ce2b0 con 0x7f2c68101900 2024-05-08T21:19:29.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.080+0000 7f2c667fc700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2c500a1120 0x7f2c500a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.080+0000 7f2c667fc700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2c500a1120 0x7f2c500a35e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c007c60 tx=0x7f2c5c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.080+0000 7f2c7986fb80 1 Processor -- start 2024-05-08T21:19:29.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.080+0000 7f2c7986fb80 1 -- start start 2024-05-08T21:19:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c7986fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea59960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c7986fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 0x56510ea5a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c7986fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea5ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c7986fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510e9b0150 con 0x56510ea59540 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c7986fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510e9adb10 con 0x56510ea59ea0 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c7986fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510e9a9400 con 0x56510ea5a860 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea59960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea59960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48908/0 (socket says 172.21.15.33:48908) 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c66ffd700 1 -- 172.21.15.33:0/2715325940 learned_addr learned my addr 172.21.15.33:0/2715325940 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c667fc700 1 --2- 172.21.15.33:0/2715325940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 0x56510ea5a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c677fe700 1 --2- 172.21.15.33:0/2715325940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea5ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c66ffd700 1 -- 172.21.15.33:0/2715325940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 msgr2=0x56510ea5ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/2715325940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea5ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c66ffd700 1 -- 172.21.15.33:0/2715325940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 msgr2=0x56510ea5a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/2715325940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 0x56510ea5a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c66ffd700 1 -- 172.21.15.33:0/2715325940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56510e82a3b0 con 0x56510ea59540 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c677fe700 1 --2- 172.21.15.33:0/2715325940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea5ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/2715325940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea59960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2c5809b1f0 tx=0x7f2c5809b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775a700247776ae5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.081+0000 7f2c4d7fa700 1 -- 172.21.15.33:0/2715325940 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5809f120 con 0x56510ea59540 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.082+0000 7f2c4d7fa700 1 -- 172.21.15.33:0/2715325940 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c5809b4f0 con 0x56510ea59540 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.082+0000 7f2c4d7fa700 1 -- 172.21.15.33:0/2715325940 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5809ac40 con 0x56510ea59540 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.082+0000 7f2c7986fb80 1 -- 172.21.15.33:0/2715325940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 msgr2=0x56510ea59960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.082+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/2715325940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea59960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2c5809b1f0 tx=0x7f2c5809b100 comp rx=0 tx=0).stop 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.082+0000 7f2c7986fb80 1 -- 172.21.15.33:0/2715325940 shutdown_connections 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.082+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/2715325940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea5ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.082+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/2715325940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea59960 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.082+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/2715325940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 0x56510ea5a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.082+0000 7f2c7986fb80 1 -- 172.21.15.33:0/2715325940 >> 172.21.15.33:0/2715325940 conn(0x56510e9b7210 msgr2=0x56510ea53de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.082+0000 7f2c7986fb80 1 -- 172.21.15.33:0/2715325940 shutdown_connections 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.082+0000 7f2c7986fb80 1 -- 172.21.15.33:0/2715325940 wait complete. 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c7986fb80 1 -- start start 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c7986fb80 1 Processor -- start 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c7986fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea59540 0x56510ea62a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c7986fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59ea0 0x56510ea62fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c7986fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea5a860 0x56510ea6a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c7986fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510e945560 con 0x56510ea59ea0 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c7986fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510e945200 con 0x56510ea5a860 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c7986fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510e82c6a0 con 0x56510ea59540 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c7986fb80 10 client.0 _ll_register_callbacks cb 0x56510e94eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c667fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59ea0 0x56510ea62fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c667fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59ea0 0x56510ea62fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48916/0 (socket says 172.21.15.33:48916) 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c667fc700 1 -- 172.21.15.33:0/3679474615 learned_addr learned my addr 172.21.15.33:0/3679474615 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea59540 0x56510ea62a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.083+0000 7f2c677fe700 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea5a860 0x56510ea6a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c667fc700 1 -- 172.21.15.33:0/3679474615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea59540 msgr2=0x56510ea62a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c667fc700 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea59540 0x56510ea62a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c667fc700 1 -- 172.21.15.33:0/3679474615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea5a860 msgr2=0x56510ea6a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c667fc700 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea5a860 0x56510ea6a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c667fc700 1 -- 172.21.15.33:0/3679474615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c681a12d0 con 0x56510ea59ea0 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c677fe700 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea5a860 0x56510ea6a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c667fc700 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59ea0 0x56510ea62fc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0116f0 tx=0x7f2c5c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac93f43598beb62d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c4aff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c4aff5700 1 -- 172.21.15.33:0/3679474615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c010ee0 con 0x56510ea59ea0 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c4aff5700 1 -- 172.21.15.33:0/3679474615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f2c5c017070 con 0x56510ea59ea0 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3679474615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56510e902270 con 0x56510ea59ea0 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c4aff5700 1 -- 172.21.15.33:0/3679474615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c5c011e70 con 0x56510ea59ea0 2024-05-08T21:19:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c7986fb80 10 client.15714 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3679474615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56510e7d6b60 con 0x56510ea59ea0 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.084+0000 7f2c4aff5700 1 -- 172.21.15.33:0/3679474615 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56510e902270 con 0x56510ea59ea0 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.085+0000 7f2c4aff5700 1 -- 172.21.15.33:0/3679474615 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c5c01f430 con 0x56510ea59ea0 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.085+0000 7f2c7986fb80 10 client.15714 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.085+0000 7f2c7986fb80 10 client.15714 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.085+0000 7f2c7986fb80 20 client.15714 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.085+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x56510ea6e990 0x56510ea70db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.085+0000 7f2c7986fb80 4 client.15714 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.085+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3679474615 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56510ea28810 con 0x56510ea6e990 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.085+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x56510ea6e990 0x56510ea70db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.085+0000 7f2c4aff5700 10 client.15714 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.085+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x56510ea6e990 0x56510ea70db0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=84bf2a6c31fa1ca server_cookie=1e512c844d40968 in_seq=0 out_seq=0 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c4aff5700 1 -- 172.21.15.33:0/3679474615 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56510ea28810 con 0x56510ea6e990 2024-05-08T21:19:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c4aff5700 10 client.15714 handle_command_reply: tid=0 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c7986fb80 1 client.15714 shutdown 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3679474615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59ea0 msgr2=0x56510ea62fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59ea0 0x56510ea62fc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0116f0 tx=0x7f2c5c0047c0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3679474615 shutdown_connections 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea5a860 0x56510ea6a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x56510ea6e990 0x56510ea70db0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea59540 0x56510ea62a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/3679474615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59ea0 0x56510ea62fc0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c4aff5700 0 client.15714 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3679474615 >> 172.21.15.33:0/3679474615 conn(0x56510e9b7210 msgr2=0x56510ea535a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3679474615 shutdown_connections 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3679474615 wait complete. 2024-05-08T21:19:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.086+0000 7f2c7986fb80 20 client.15714 trim_cache size 0 max 16384 2024-05-08T21:19:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.100+0000 7f2c7986fb80 1 Processor -- start 2024-05-08T21:19:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c7986fb80 1 -- start start 2024-05-08T21:19:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c7986fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea61f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c7986fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 0x56510ea69fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c7986fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea6a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c7986fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510ea62620 con 0x56510ea59540 2024-05-08T21:19:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c7986fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510ea55570 con 0x56510ea59ea0 2024-05-08T21:19:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c7986fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510ea61a50 con 0x56510ea5a860 2024-05-08T21:19:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea61f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea61f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48922/0 (socket says 172.21.15.33:48922) 2024-05-08T21:19:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c66ffd700 1 -- 172.21.15.33:0/2217028355 learned_addr learned my addr 172.21.15.33:0/2217028355 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c667fc700 1 --2- 172.21.15.33:0/2217028355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 0x56510ea69fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c677fe700 1 --2- 172.21.15.33:0/2217028355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea6a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c66ffd700 1 -- 172.21.15.33:0/2217028355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 msgr2=0x56510ea6a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/2217028355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea6a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c66ffd700 1 -- 172.21.15.33:0/2217028355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 msgr2=0x56510ea69fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/2217028355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 0x56510ea69fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c66ffd700 1 -- 172.21.15.33:0/2217028355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56510e96f560 con 0x56510ea59540 2024-05-08T21:19:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.101+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/2217028355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea61f90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2c580979f0 tx=0x7f2c5801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db2d74719dd37e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.102+0000 7f2c4dffb700 1 -- 172.21.15.33:0/2217028355 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c580ce130 con 0x56510ea59540 2024-05-08T21:19:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.102+0000 7f2c4dffb700 1 -- 172.21.15.33:0/2217028355 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c580a38e0 con 0x56510ea59540 2024-05-08T21:19:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.102+0000 7f2c4dffb700 1 -- 172.21.15.33:0/2217028355 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c580a3450 con 0x56510ea59540 2024-05-08T21:19:29.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.102+0000 7f2c7986fb80 1 -- 172.21.15.33:0/2217028355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 msgr2=0x56510ea61f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.102+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/2217028355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea61f90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2c580979f0 tx=0x7f2c5801a040 comp rx=0 tx=0).stop 2024-05-08T21:19:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.102+0000 7f2c7986fb80 1 -- 172.21.15.33:0/2217028355 shutdown_connections 2024-05-08T21:19:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.102+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/2217028355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea6a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.102+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/2217028355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea61f90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.102+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/2217028355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 0x56510ea69fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.102+0000 7f2c7986fb80 1 -- 172.21.15.33:0/2217028355 >> 172.21.15.33:0/2217028355 conn(0x56510e9b7210 msgr2=0x56510ea5ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.102+0000 7f2c7986fb80 1 -- 172.21.15.33:0/2217028355 shutdown_connections 2024-05-08T21:19:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.102+0000 7f2c7986fb80 1 -- 172.21.15.33:0/2217028355 wait complete. 2024-05-08T21:19:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c7986fb80 1 -- start start 2024-05-08T21:19:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c7986fb80 1 Processor -- start 2024-05-08T21:19:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c7986fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea65850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c7986fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 0x56510ea65d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c7986fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea68d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c7986fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510ea692c0 con 0x56510ea59540 2024-05-08T21:19:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c7986fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510ea69430 con 0x56510ea59ea0 2024-05-08T21:19:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c7986fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510ea695f0 con 0x56510ea5a860 2024-05-08T21:19:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c7986fb80 10 client.0 _ll_register_callbacks cb 0x56510e94eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea65850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea65850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48936/0 (socket says 172.21.15.33:48936) 2024-05-08T21:19:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c66ffd700 1 -- 172.21.15.33:0/3931535492 learned_addr learned my addr 172.21.15.33:0/3931535492 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c667fc700 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 0x56510ea65d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c677fe700 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea68d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c66ffd700 1 -- 172.21.15.33:0/3931535492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 msgr2=0x56510ea68d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea68d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c66ffd700 1 -- 172.21.15.33:0/3931535492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 msgr2=0x56510ea65d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 0x56510ea65d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.103+0000 7f2c66ffd700 1 -- 172.21.15.33:0/3931535492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56510e82a3b0 con 0x56510ea59540 2024-05-08T21:19:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c66ffd700 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea65850 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2c580b5040 tx=0x7f2c58005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a949ec0843290d26 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c4aff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c4aff5700 1 -- 172.21.15.33:0/3931535492 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5809aa50 con 0x56510ea59540 2024-05-08T21:19:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c4aff5700 1 -- 172.21.15.33:0/3931535492 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f2c5809abb0 con 0x56510ea59540 2024-05-08T21:19:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3931535492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c5809b4f0 con 0x56510ea59540 2024-05-08T21:19:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c4aff5700 1 -- 172.21.15.33:0/3931535492 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c580a3450 con 0x56510ea59540 2024-05-08T21:19:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c7986fb80 10 client.15720 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3931535492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56510e82ecd0 con 0x56510ea59540 2024-05-08T21:19:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c4aff5700 1 -- 172.21.15.33:0/3931535492 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c5809b4f0 con 0x56510ea59540 2024-05-08T21:19:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c4aff5700 1 -- 172.21.15.33:0/3931535492 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c580972e0 con 0x56510ea59540 2024-05-08T21:19:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c7986fb80 10 client.15720 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c7986fb80 10 client.15720 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:19:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c7986fb80 20 client.15720 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x56510eac5430 0x56510eac7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c7986fb80 4 client.15720 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:19:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.104+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3931535492 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56510e9a1ec0 con 0x56510eac5430 2024-05-08T21:19:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c667fc700 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x56510eac5430 0x56510eac7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c4aff5700 10 client.15720 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:19:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c667fc700 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x56510eac5430 0x56510eac7850 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed5f4236dc6f0e85 server_cookie=e28da81379f84aa9 in_seq=0 out_seq=0 2024-05-08T21:19:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c4aff5700 1 -- 172.21.15.33:0/3931535492 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56510e9a1ec0 con 0x56510eac5430 2024-05-08T21:19:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c4aff5700 10 client.15720 handle_command_reply: tid=0 2024-05-08T21:19:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c7986fb80 1 client.15720 shutdown 2024-05-08T21:19:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3931535492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 msgr2=0x56510ea65850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea65850 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2c580b5040 tx=0x7f2c58005fb0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3931535492 shutdown_connections 2024-05-08T21:19:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x56510eac5430 0x56510eac7850 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56510ea5a860 0x56510ea68d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56510ea59540 0x56510ea65850 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c4aff5700 0 client.15720 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:19:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c7986fb80 1 --2- 172.21.15.33:0/3931535492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56510ea59ea0 0x56510ea65d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.105+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3931535492 >> 172.21.15.33:0/3931535492 conn(0x56510e9b7210 msgr2=0x56510ea60d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3931535492 shutdown_connections 2024-05-08T21:19:29.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c7986fb80 1 -- 172.21.15.33:0/3931535492 wait complete. 2024-05-08T21:19:29.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c7986fb80 20 client.15720 trim_cache size 0 max 16384 2024-05-08T21:19:29.115 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:19:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/602892515 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2c500a1120 msgr2=0x7f2c500a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c6d6b5700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2c500a1120 0x7f2c500a35e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c007c60 tx=0x7f2c5c0073d0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/602892515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c68101900 msgr2=0x7f2c68101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c6d6b5700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c68101900 0x7f2c68101220 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2c580098e0 tx=0x7f2c58003a50 comp rx=0 tx=0).stop 2024-05-08T21:19:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/602892515 shutdown_connections 2024-05-08T21:19:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c6d6b5700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c68106b60 0x7f2c680ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c6d6b5700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c68101900 0x7f2c68101220 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c6d6b5700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2c500a1120 0x7f2c500a35e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c6d6b5700 1 --2- 172.21.15.33:0/602892515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c68104230 0x7f2c680ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/602892515 >> 172.21.15.33:0/602892515 conn(0x7f2c680fb310 msgr2=0x7f2c68102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/602892515 shutdown_connections 2024-05-08T21:19:29.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.106+0000 7f2c6d6b5700 1 -- 172.21.15.33:0/602892515 wait complete. 2024-05-08T21:19:29.122 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:19:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.260+0000 7f6756adf700 1 -- 172.21.15.33:0/3948675316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750102a90 msgr2=0x7f6750109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.260+0000 7f6756adf700 1 --2- 172.21.15.33:0/3948675316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750102a90 0x7f6750109340 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f674800b0a0 tx=0x7f674800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.260+0000 7f6756adf700 1 -- 172.21.15.33:0/3948675316 shutdown_connections 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.260+0000 7f6756adf700 1 --2- 172.21.15.33:0/3948675316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750102a90 0x7f6750109340 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.260+0000 7f6756adf700 1 --2- 172.21.15.33:0/3948675316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67500ff120 0x7f67500ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.260+0000 7f6756adf700 1 --2- 172.21.15.33:0/3948675316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67501020f0 0x7f6750102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.260+0000 7f6756adf700 1 -- 172.21.15.33:0/3948675316 >> 172.21.15.33:0/3948675316 conn(0x7f67500faa50 msgr2=0x7f67500fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.260+0000 7f6756adf700 1 -- 172.21.15.33:0/3948675316 shutdown_connections 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.260+0000 7f6756adf700 1 -- 172.21.15.33:0/3948675316 wait complete. 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f6756adf700 1 Processor -- start 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f6756adf700 1 -- start start 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f6756adf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67500ff120 0x7f6750198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f6756adf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67501020f0 0x7f6750199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f6756adf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750102a90 0x7f67501a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f6756adf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67501a0b20 con 0x7f6750102a90 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f6756adf700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67501a0c60 con 0x7f67500ff120 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f6756adf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67501a0df0 con 0x7f67501020f0 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f675507c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750102a90 0x7f67501a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f675507c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750102a90 0x7f67501a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48956/0 (socket says 172.21.15.33:48956) 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f675507c700 1 -- 172.21.15.33:0/3151480389 learned_addr learned my addr 172.21.15.33:0/3151480389 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f674ffff700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67501020f0 0x7f6750199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f675487b700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67500ff120 0x7f6750198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f675507c700 1 -- 172.21.15.33:0/3151480389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67501020f0 msgr2=0x7f6750199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f675507c700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67501020f0 0x7f6750199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f675507c700 1 -- 172.21.15.33:0/3151480389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67500ff120 msgr2=0x7f6750198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.261+0000 7f675507c700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67500ff120 0x7f6750198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.262+0000 7f675507c700 1 -- 172.21.15.33:0/3151480389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6748009d00 con 0x7f6750102a90 2024-05-08T21:19:29.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.262+0000 7f675507c700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750102a90 0x7f67501a0550 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6748009820 tx=0x7f674800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.262+0000 7f674dffb700 1 -- 172.21.15.33:0/3151480389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674800e040 con 0x7f6750102a90 2024-05-08T21:19:29.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.262+0000 7f674dffb700 1 -- 172.21.15.33:0/3151480389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6748004c50 con 0x7f6750102a90 2024-05-08T21:19:29.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.262+0000 7f6756adf700 1 -- 172.21.15.33:0/3151480389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67501a1070 con 0x7f6750102a90 2024-05-08T21:19:29.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.262+0000 7f674dffb700 1 -- 172.21.15.33:0/3151480389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67480052a0 con 0x7f6750102a90 2024-05-08T21:19:29.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.262+0000 7f6756adf700 1 -- 172.21.15.33:0/3151480389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67501a1540 con 0x7f6750102a90 2024-05-08T21:19:29.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.265+0000 7f674dffb700 1 -- 172.21.15.33:0/3151480389 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6748005400 con 0x7f6750102a90 2024-05-08T21:19:29.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.266+0000 7f674dffb700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f67400a11a0 0x7f67400a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.266+0000 7f674dffb700 1 -- 172.21.15.33:0/3151480389 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f67480ce170 con 0x7f6750102a90 2024-05-08T21:19:29.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.266+0000 7f675487b700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f67400a11a0 0x7f67400a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.266+0000 7f675487b700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f67400a11a0 0x7f67400a3660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6744005d90 tx=0x7f6744005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.266+0000 7f6762c99b80 1 Processor -- start 2024-05-08T21:19:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f6762c99b80 1 -- start start 2024-05-08T21:19:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f6762c99b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7c540 0x5578edc7c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f6762c99b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 0x5578edc7d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f6762c99b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc7dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f6762c99b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578edbd3150 con 0x5578edc7cea0 2024-05-08T21:19:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f6762c99b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578edbd0b10 con 0x5578edc7d860 2024-05-08T21:19:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f6762c99b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578edbcc400 con 0x5578edc7c540 2024-05-08T21:19:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f674ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 0x5578edc7d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f674ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 0x5578edc7d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48964/0 (socket says 172.21.15.33:48964) 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f674ffff700 1 -- 172.21.15.33:0/419940742 learned_addr learned my addr 172.21.15.33:0/419940742 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f675507c700 1 --2- 172.21.15.33:0/419940742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc7dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f675487b700 1 --2- 172.21.15.33:0/419940742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7c540 0x5578edc7c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f675507c700 1 -- 172.21.15.33:0/419940742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7c540 msgr2=0x5578edc7c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f675507c700 1 --2- 172.21.15.33:0/419940742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7c540 0x5578edc7c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f675507c700 1 -- 172.21.15.33:0/419940742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 msgr2=0x5578edc7d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f675507c700 1 --2- 172.21.15.33:0/419940742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 0x5578edc7d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f675507c700 1 -- 172.21.15.33:0/419940742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578eda4d3b0 con 0x5578edc7d860 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.267+0000 7f675507c700 1 --2- 172.21.15.33:0/419940742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc7dce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67480cb8c0 tx=0x7f674809b1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e764d9577e7a744 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.268+0000 7f673affd700 1 -- 172.21.15.33:0/419940742 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674809b2d0 con 0x5578edc7d860 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.268+0000 7f673affd700 1 -- 172.21.15.33:0/419940742 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f674809f100 con 0x5578edc7d860 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.268+0000 7f673affd700 1 -- 172.21.15.33:0/419940742 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748099790 con 0x5578edc7d860 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.268+0000 7f6762c99b80 1 -- 172.21.15.33:0/419940742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 msgr2=0x5578edc7dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.268+0000 7f6762c99b80 1 --2- 172.21.15.33:0/419940742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc7dce0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67480cb8c0 tx=0x7f674809b1d0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.268+0000 7f6762c99b80 1 -- 172.21.15.33:0/419940742 shutdown_connections 2024-05-08T21:19:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.268+0000 7f6762c99b80 1 --2- 172.21.15.33:0/419940742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc7dce0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.268+0000 7f6762c99b80 1 --2- 172.21.15.33:0/419940742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7c540 0x5578edc7c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.268+0000 7f6762c99b80 1 --2- 172.21.15.33:0/419940742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 0x5578edc7d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.268+0000 7f6762c99b80 1 -- 172.21.15.33:0/419940742 >> 172.21.15.33:0/419940742 conn(0x5578edbda210 msgr2=0x5578edc76de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.268+0000 7f6762c99b80 1 -- 172.21.15.33:0/419940742 shutdown_connections 2024-05-08T21:19:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.268+0000 7f6762c99b80 1 -- 172.21.15.33:0/419940742 wait complete. 2024-05-08T21:19:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.269+0000 7f6762c99b80 1 -- start start 2024-05-08T21:19:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.269+0000 7f6762c99b80 1 Processor -- start 2024-05-08T21:19:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.269+0000 7f6762c99b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7c540 0x5578edc85c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.269+0000 7f6762c99b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 0x5578edc861c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.269+0000 7f6762c99b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc8d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.269+0000 7f6762c99b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578edb68560 con 0x5578edc7cea0 2024-05-08T21:19:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.269+0000 7f6762c99b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578edb68200 con 0x5578edc7d860 2024-05-08T21:19:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.269+0000 7f6762c99b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578eda4f6a0 con 0x5578edc7c540 2024-05-08T21:19:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.269+0000 7f6762c99b80 10 client.0 _ll_register_callbacks cb 0x5578edb71b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.269+0000 7f674ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 0x5578edc861c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.269+0000 7f674ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 0x5578edc861c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48976/0 (socket says 172.21.15.33:48976) 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f674ffff700 1 -- 172.21.15.33:0/4045225070 learned_addr learned my addr 172.21.15.33:0/4045225070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f675487b700 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7c540 0x5578edc85c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f675507c700 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc8d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f674ffff700 1 -- 172.21.15.33:0/4045225070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7c540 msgr2=0x5578edc85c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f674ffff700 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7c540 0x5578edc85c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f674ffff700 1 -- 172.21.15.33:0/4045225070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 msgr2=0x5578edc8d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f674ffff700 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc8d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f674ffff700 1 -- 172.21.15.33:0/4045225070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67501a1540 con 0x5578edc7cea0 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f674ffff700 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 0x5578edc861c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f673c011890 tx=0x7f673c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3242c999b47889b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f672ffff700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f672ffff700 1 -- 172.21.15.33:0/4045225070 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c016070 con 0x5578edc7cea0 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f672ffff700 1 -- 172.21.15.33:0/4045225070 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f673c004260 con 0x5578edc7cea0 2024-05-08T21:19:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f6762c99b80 1 -- 172.21.15.33:0/4045225070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578edb25270 con 0x5578edc7cea0 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f672ffff700 1 -- 172.21.15.33:0/4045225070 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f673c004420 con 0x5578edc7cea0 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f6762c99b80 10 client.15732 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.270+0000 7f6762c99b80 1 -- 172.21.15.33:0/4045225070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5578ed9f9b60 con 0x5578edc7cea0 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.271+0000 7f672ffff700 1 -- 172.21.15.33:0/4045225070 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5578edb25270 con 0x5578edc7cea0 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.271+0000 7f672ffff700 1 -- 172.21.15.33:0/4045225070 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f673c00f6c0 con 0x5578edc7cea0 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.271+0000 7f6762c99b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.271+0000 7f6762c99b80 10 client.15732 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.271+0000 7f6762c99b80 20 client.15732 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.271+0000 7f6762c99b80 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5578edc91b10 0x5578edc93f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.271+0000 7f6762c99b80 4 client.15732 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.271+0000 7f6762c99b80 1 -- 172.21.15.33:0/4045225070 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578edc4b810 con 0x5578edc91b10 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.271+0000 7f675487b700 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5578edc91b10 0x5578edc93f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.272+0000 7f672ffff700 10 client.15732 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.272+0000 7f675487b700 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5578edc91b10 0x5578edc93f30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e6c82a652391af0e server_cookie=7c06c566d0fa8c3d in_seq=0 out_seq=0 2024-05-08T21:19:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.272+0000 7f672ffff700 1 -- 172.21.15.33:0/4045225070 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578edc4b810 con 0x5578edc91b10 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.272+0000 7f672ffff700 10 client.15732 handle_command_reply: tid=0 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.272+0000 7f6762c99b80 1 client.15732 shutdown 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.273+0000 7f6762c99b80 1 -- 172.21.15.33:0/4045225070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 msgr2=0x5578edc861c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.273+0000 7f6762c99b80 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 0x5578edc861c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f673c011890 tx=0x7f673c011ba0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.273+0000 7f6762c99b80 1 -- 172.21.15.33:0/4045225070 shutdown_connections 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.273+0000 7f6762c99b80 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc8d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.273+0000 7f6762c99b80 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5578edc91b10 0x5578edc93f30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.273+0000 7f6762c99b80 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7c540 0x5578edc85c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.273+0000 7f672ffff700 0 client.15732 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.273+0000 7f6762c99b80 1 --2- 172.21.15.33:0/4045225070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7cea0 0x5578edc861c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.273+0000 7f6762c99b80 1 -- 172.21.15.33:0/4045225070 >> 172.21.15.33:0/4045225070 conn(0x5578edbda210 msgr2=0x5578edc765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.273+0000 7f6762c99b80 1 -- 172.21.15.33:0/4045225070 shutdown_connections 2024-05-08T21:19:29.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.273+0000 7f6762c99b80 1 -- 172.21.15.33:0/4045225070 wait complete. 2024-05-08T21:19:29.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.273+0000 7f6762c99b80 20 client.15732 trim_cache size 0 max 16384 2024-05-08T21:19:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f6762c99b80 1 Processor -- start 2024-05-08T21:19:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f6762c99b80 1 -- start start 2024-05-08T21:19:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f6762c99b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7c540 0x5578edc93f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f6762c99b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7cea0 0x5578edc8d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f6762c99b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc85190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f6762c99b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578edc99140 con 0x5578edc7c540 2024-05-08T21:19:29.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f6762c99b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578edc78830 con 0x5578edc7d860 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f6762c99b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578edc86900 con 0x5578edc7cea0 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f675487b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7c540 0x5578edc93f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f675487b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7c540 0x5578edc93f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48984/0 (socket says 172.21.15.33:48984) 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f675487b700 1 -- 172.21.15.33:0/3274865737 learned_addr learned my addr 172.21.15.33:0/3274865737 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f675507c700 1 --2- 172.21.15.33:0/3274865737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc85190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f674ffff700 1 --2- 172.21.15.33:0/3274865737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7cea0 0x5578edc8d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.287+0000 7f675487b700 1 -- 172.21.15.33:0/3274865737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7cea0 msgr2=0x5578edc8d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.288+0000 7f675487b700 1 --2- 172.21.15.33:0/3274865737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7cea0 0x5578edc8d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.288+0000 7f675487b700 1 -- 172.21.15.33:0/3274865737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 msgr2=0x5578edc85190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.288+0000 7f675487b700 1 --2- 172.21.15.33:0/3274865737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc85190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.288+0000 7f675487b700 1 -- 172.21.15.33:0/3274865737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578edb92560 con 0x5578edc7c540 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.288+0000 7f674ffff700 1 --2- 172.21.15.33:0/3274865737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7cea0 0x5578edc8d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.288+0000 7f675487b700 1 --2- 172.21.15.33:0/3274865737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7c540 0x5578edc93f30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f674400b470 tx=0x7f6744012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d786f94c5e99a32e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.288+0000 7f673b7fe700 1 -- 172.21.15.33:0/3274865737 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744024040 con 0x5578edc7c540 2024-05-08T21:19:29.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.288+0000 7f673b7fe700 1 -- 172.21.15.33:0/3274865737 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6744011e70 con 0x5578edc7c540 2024-05-08T21:19:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.288+0000 7f673b7fe700 1 -- 172.21.15.33:0/3274865737 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674401fbc0 con 0x5578edc7c540 2024-05-08T21:19:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.288+0000 7f6762c99b80 1 -- 172.21.15.33:0/3274865737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7c540 msgr2=0x5578edc93f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.288+0000 7f6762c99b80 1 --2- 172.21.15.33:0/3274865737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7c540 0x5578edc93f30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f674400b470 tx=0x7f6744012000 comp rx=0 tx=0).stop 2024-05-08T21:19:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.289+0000 7f6762c99b80 1 -- 172.21.15.33:0/3274865737 shutdown_connections 2024-05-08T21:19:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.289+0000 7f6762c99b80 1 --2- 172.21.15.33:0/3274865737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7d860 0x5578edc85190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.289+0000 7f6762c99b80 1 --2- 172.21.15.33:0/3274865737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7c540 0x5578edc93f30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.289+0000 7f6762c99b80 1 --2- 172.21.15.33:0/3274865737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7cea0 0x5578edc8d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.289+0000 7f6762c99b80 1 -- 172.21.15.33:0/3274865737 >> 172.21.15.33:0/3274865737 conn(0x5578edbda210 msgr2=0x5578edc83e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.289+0000 7f6762c99b80 1 -- 172.21.15.33:0/3274865737 shutdown_connections 2024-05-08T21:19:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.289+0000 7f6762c99b80 1 -- 172.21.15.33:0/3274865737 wait complete. 2024-05-08T21:19:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.289+0000 7f6762c99b80 1 -- start start 2024-05-08T21:19:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.289+0000 7f6762c99b80 1 Processor -- start 2024-05-08T21:19:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f6762c99b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7c540 0x5578edc8b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f6762c99b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7cea0 0x5578edce3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f6762c99b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7d860 0x5578edce6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f6762c99b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578edce6bf0 con 0x5578edc7d860 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f6762c99b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578edce6d60 con 0x5578edc7c540 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f6762c99b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578edce6f20 con 0x5578edc7cea0 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f6762c99b80 10 client.0 _ll_register_callbacks cb 0x5578edb71b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f675507c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7d860 0x5578edce6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f675507c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7d860 0x5578edce6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48988/0 (socket says 172.21.15.33:48988) 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f675507c700 1 -- 172.21.15.33:0/133407470 learned_addr learned my addr 172.21.15.33:0/133407470 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f675487b700 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7c540 0x5578edc8b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f674ffff700 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7cea0 0x5578edce3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f675507c700 1 -- 172.21.15.33:0/133407470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7cea0 msgr2=0x5578edce3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f675507c700 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7cea0 0x5578edce3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f675507c700 1 -- 172.21.15.33:0/133407470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7c540 msgr2=0x5578edc8b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f675507c700 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7c540 0x5578edc8b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f675507c700 1 -- 172.21.15.33:0/133407470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674400f9e0 con 0x5578edc7d860 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f674ffff700 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7cea0 0x5578edce3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f675507c700 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7d860 0x5578edce6530 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f674809b260 tx=0x7f674809aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bd1bf745fac46d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.290+0000 7f672ffff700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f672ffff700 1 -- 172.21.15.33:0/133407470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674809b2d0 con 0x5578edc7d860 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f672ffff700 1 -- 172.21.15.33:0/133407470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f67480a36c0 con 0x5578edc7d860 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f6762c99b80 1 -- 172.21.15.33:0/133407470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f674809f100 con 0x5578edc7d860 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f672ffff700 1 -- 172.21.15.33:0/133407470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f674809a7c0 con 0x5578edc7d860 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f6762c99b80 10 client.15738 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f6762c99b80 1 -- 172.21.15.33:0/133407470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5578eda51cd0 con 0x5578edc7d860 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f672ffff700 1 -- 172.21.15.33:0/133407470 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f674809f100 con 0x5578edc7d860 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f672ffff700 1 -- 172.21.15.33:0/133407470 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f67480a3230 con 0x5578edc7d860 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f6762c99b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f6762c99b80 10 client.15738 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:19:29.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f6762c99b80 20 client.15738 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f6762c99b80 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5578edce8480 0x5578edcea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f6762c99b80 4 client.15738 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f6762c99b80 1 -- 172.21.15.33:0/133407470 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5578edbc4ec0 con 0x5578edce8480 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.291+0000 7f675487b700 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5578edce8480 0x5578edcea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f675487b700 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5578edce8480 0x5578edcea8a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2dc3fec086082ce server_cookie=a3d3ef55cebea473 in_seq=0 out_seq=0 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f672ffff700 10 client.15738 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f672ffff700 1 -- 172.21.15.33:0/133407470 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5578edbc4ec0 con 0x5578edce8480 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f672ffff700 10 client.15738 handle_command_reply: tid=0 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f6762c99b80 1 client.15738 shutdown 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f6762c99b80 1 -- 172.21.15.33:0/133407470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7d860 msgr2=0x5578edce6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f6762c99b80 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7d860 0x5578edce6530 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f674809b260 tx=0x7f674809aec0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f6762c99b80 1 -- 172.21.15.33:0/133407470 shutdown_connections 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f6762c99b80 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578edc7d860 0x5578edce6530 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f6762c99b80 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578edc7c540 0x5578edc8b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f6762c99b80 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5578edce8480 0x5578edcea8a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f6762c99b80 1 --2- 172.21.15.33:0/133407470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5578edc7cea0 0x5578edce3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f672ffff700 0 client.15738 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:19:29.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.292+0000 7f6762c99b80 1 -- 172.21.15.33:0/133407470 >> 172.21.15.33:0/133407470 conn(0x5578edbda210 msgr2=0x5578edc83d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6762c99b80 1 -- 172.21.15.33:0/133407470 shutdown_connections 2024-05-08T21:19:29.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6762c99b80 1 -- 172.21.15.33:0/133407470 wait complete. 2024-05-08T21:19:29.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6762c99b80 20 client.15738 trim_cache size 0 max 16384 2024-05-08T21:19:29.301 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:19:29.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6756adf700 1 -- 172.21.15.33:0/3151480389 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f67400a11a0 msgr2=0x7f67400a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6756adf700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f67400a11a0 0x7f67400a3660 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6744005d90 tx=0x7f6744005ce0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6756adf700 1 -- 172.21.15.33:0/3151480389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750102a90 msgr2=0x7f67501a0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6756adf700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750102a90 0x7f67501a0550 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6748009820 tx=0x7f674800bab0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6756adf700 1 -- 172.21.15.33:0/3151480389 shutdown_connections 2024-05-08T21:19:29.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6756adf700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6750102a90 0x7f67501a0550 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6756adf700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f67400a11a0 0x7f67400a3660 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6756adf700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67500ff120 0x7f6750198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6756adf700 1 --2- 172.21.15.33:0/3151480389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67501020f0 0x7f6750199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6756adf700 1 -- 172.21.15.33:0/3151480389 >> 172.21.15.33:0/3151480389 conn(0x7f67500faa50 msgr2=0x7f6750107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6756adf700 1 -- 172.21.15.33:0/3151480389 shutdown_connections 2024-05-08T21:19:29.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.293+0000 7f6756adf700 1 -- 172.21.15.33:0/3151480389 wait complete. 2024-05-08T21:19:29.309 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:19:29.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.447+0000 7f8ef4bad700 1 -- 172.21.15.33:0/2756668109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef01042a0 msgr2=0x7f8ef0106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.447+0000 7f8ef4bad700 1 --2- 172.21.15.33:0/2756668109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef01042a0 0x7f8ef0106690 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009990 tx=0x7f8ed8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.447+0000 7f8ef4bad700 1 -- 172.21.15.33:0/2756668109 shutdown_connections 2024-05-08T21:19:29.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.447+0000 7f8ef4bad700 1 --2- 172.21.15.33:0/2756668109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef0106bd0 0x7f8ef0108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.447+0000 7f8ef4bad700 1 --2- 172.21.15.33:0/2756668109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef0101970 0x7f8ef0103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.447+0000 7f8ef4bad700 1 --2- 172.21.15.33:0/2756668109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef01042a0 0x7f8ef0106690 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.447+0000 7f8ef4bad700 1 -- 172.21.15.33:0/2756668109 >> 172.21.15.33:0/2756668109 conn(0x7f8ef00fb300 msgr2=0x7f8ef00fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.447+0000 7f8ef4bad700 1 -- 172.21.15.33:0/2756668109 shutdown_connections 2024-05-08T21:19:29.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.447+0000 7f8ef4bad700 1 -- 172.21.15.33:0/2756668109 wait complete. 2024-05-08T21:19:29.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.448+0000 7f8ef4bad700 1 Processor -- start 2024-05-08T21:19:29.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.448+0000 7f8ef4bad700 1 -- start start 2024-05-08T21:19:29.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.448+0000 7f8ef4bad700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef0101970 0x7f8ef0196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.448+0000 7f8ef4bad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef01042a0 0x7f8ef01972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.448+0000 7f8ef4bad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0106bd0 0x7f8ef019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.448+0000 7f8ef4bad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef019e840 con 0x7f8ef0106bd0 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.448+0000 7f8ef4bad700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef019e9b0 con 0x7f8ef0101970 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.448+0000 7f8ef4bad700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef019eb70 con 0x7f8ef01042a0 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.448+0000 7f8eeed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0106bd0 0x7f8ef019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.448+0000 7f8eeed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0106bd0 0x7f8ef019e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49000/0 (socket says 172.21.15.33:49000) 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.448+0000 7f8eeed9d700 1 -- 172.21.15.33:0/72410936 learned_addr learned my addr 172.21.15.33:0/72410936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.448+0000 7f8ee7fff700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef01042a0 0x7f8ef01972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8eee59c700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef0101970 0x7f8ef0196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8eeed9d700 1 -- 172.21.15.33:0/72410936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef01042a0 msgr2=0x7f8ef01972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8eeed9d700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef01042a0 0x7f8ef01972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8eeed9d700 1 -- 172.21.15.33:0/72410936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef0101970 msgr2=0x7f8ef0196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8eeed9d700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef0101970 0x7f8ef0196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8eeed9d700 1 -- 172.21.15.33:0/72410936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed80095f0 con 0x7f8ef0106bd0 2024-05-08T21:19:29.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8eee59c700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef0101970 0x7f8ef0196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8eeed9d700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0106bd0 0x7f8ef019e270 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800ef30 tx=0x7f8ee800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8ee77fe700 1 -- 172.21.15.33:0/72410936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8018070 con 0x7f8ef0106bd0 2024-05-08T21:19:29.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8ef4bad700 1 -- 172.21.15.33:0/72410936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef019edf0 con 0x7f8ef0106bd0 2024-05-08T21:19:29.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8ee77fe700 1 -- 172.21.15.33:0/72410936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ee800f040 con 0x7f8ef0106bd0 2024-05-08T21:19:29.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8ee77fe700 1 -- 172.21.15.33:0/72410936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800cc70 con 0x7f8ef0106bd0 2024-05-08T21:19:29.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.449+0000 7f8ef4bad700 1 -- 172.21.15.33:0/72410936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef019f1c0 con 0x7f8ef0106bd0 2024-05-08T21:19:29.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.452+0000 7f8ee77fe700 1 -- 172.21.15.33:0/72410936 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8ee8007500 con 0x7f8ef0106bd0 2024-05-08T21:19:29.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.453+0000 7f8ee77fe700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8edc0a1120 0x7f8edc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.453+0000 7f8ee77fe700 1 -- 172.21.15.33:0/72410936 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8ee80cc800 con 0x7f8ef0106bd0 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.453+0000 7f8eee59c700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8edc0a1120 0x7f8edc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8eee59c700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8edc0a1120 0x7f8edc0a35e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8ef01981d0 tx=0x7f8ee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8f00d67b80 1 Processor -- start 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8f00d67b80 1 -- start start 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8f00d67b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41e540 0x55c8ca41e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8f00d67b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca41f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8f00d67b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41f860 0x55c8ca41fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8f00d67b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ca375150 con 0x55c8ca41eea0 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8f00d67b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ca372b10 con 0x55c8ca41e540 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8f00d67b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ca36e400 con 0x55c8ca41f860 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8ee7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca41f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8ee7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca41f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49004/0 (socket says 172.21.15.33:49004) 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8eee59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41e540 0x55c8ca41e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8ee7fff700 1 -- 172.21.15.33:0/642949028 learned_addr learned my addr 172.21.15.33:0/642949028 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.454+0000 7f8eeed9d700 1 --2- 172.21.15.33:0/642949028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41f860 0x55c8ca41fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.455+0000 7f8eee59c700 1 -- 172.21.15.33:0/642949028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41f860 msgr2=0x55c8ca41fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.455+0000 7f8eee59c700 1 --2- 172.21.15.33:0/642949028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41f860 0x55c8ca41fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.455+0000 7f8eee59c700 1 -- 172.21.15.33:0/642949028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 msgr2=0x55c8ca41f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.455+0000 7f8eee59c700 1 --2- 172.21.15.33:0/642949028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca41f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.455+0000 7f8eee59c700 1 -- 172.21.15.33:0/642949028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8ca1ef3b0 con 0x55c8ca41e540 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.455+0000 7f8eeed9d700 1 --2- 172.21.15.33:0/642949028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41f860 0x55c8ca41fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.455+0000 7f8ee7fff700 1 --2- 172.21.15.33:0/642949028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca41f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.455+0000 7f8eee59c700 1 --2- 172.21.15.33:0/642949028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41e540 0x55c8ca41e960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0011a50 tx=0x7f8ee0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d98505edf292ea7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.455+0000 7f8ee4ff9700 1 -- 172.21.15.33:0/642949028 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0012bc0 con 0x55c8ca41e540 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.455+0000 7f8ee4ff9700 1 -- 172.21.15.33:0/642949028 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ee0016070 con 0x55c8ca41e540 2024-05-08T21:19:29.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.455+0000 7f8ee4ff9700 1 -- 172.21.15.33:0/642949028 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee001e610 con 0x55c8ca41e540 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.456+0000 7f8f00d67b80 1 -- 172.21.15.33:0/642949028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41e540 msgr2=0x55c8ca41e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.456+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/642949028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41e540 0x55c8ca41e960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0011a50 tx=0x7f8ee0014000 comp rx=0 tx=0).stop 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.456+0000 7f8f00d67b80 1 -- 172.21.15.33:0/642949028 shutdown_connections 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.456+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/642949028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41f860 0x55c8ca41fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.456+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/642949028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41e540 0x55c8ca41e960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.456+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/642949028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca41f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.456+0000 7f8f00d67b80 1 -- 172.21.15.33:0/642949028 >> 172.21.15.33:0/642949028 conn(0x55c8ca37c210 msgr2=0x55c8ca418de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.456+0000 7f8f00d67b80 1 -- 172.21.15.33:0/642949028 shutdown_connections 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.456+0000 7f8f00d67b80 1 -- 172.21.15.33:0/642949028 wait complete. 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.456+0000 7f8f00d67b80 1 -- start start 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8f00d67b80 1 Processor -- start 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8f00d67b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41e540 0x55c8ca427dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8f00d67b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca428300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8f00d67b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41f860 0x55c8ca42f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8f00d67b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ca30a560 con 0x55c8ca41eea0 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8f00d67b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ca30a200 con 0x55c8ca41e540 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8f00d67b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ca1f16a0 con 0x55c8ca41f860 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8f00d67b80 10 client.0 _ll_register_callbacks cb 0x55c8ca313b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8ee7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca428300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8ee7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca428300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49018/0 (socket says 172.21.15.33:49018) 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8ee7fff700 1 -- 172.21.15.33:0/4130237067 learned_addr learned my addr 172.21.15.33:0/4130237067 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8eee59c700 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41e540 0x55c8ca427dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8eeed9d700 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41f860 0x55c8ca42f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8ee7fff700 1 -- 172.21.15.33:0/4130237067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41f860 msgr2=0x55c8ca42f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8ee7fff700 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41f860 0x55c8ca42f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8ee7fff700 1 -- 172.21.15.33:0/4130237067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41e540 msgr2=0x55c8ca427dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8ee7fff700 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41e540 0x55c8ca427dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8ee7fff700 1 -- 172.21.15.33:0/4130237067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef019f1c0 con 0x55c8ca41eea0 2024-05-08T21:19:29.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.457+0000 7f8eeed9d700 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41f860 0x55c8ca42f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8ee7fff700 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca428300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8000c00 tx=0x7f8ed80053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e750e51149140 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8ed5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8ed5ffb700 1 -- 172.21.15.33:0/4130237067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed801d070 con 0x55c8ca41eea0 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8ed5ffb700 1 -- 172.21.15.33:0/4130237067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8ed8003b10 con 0x55c8ca41eea0 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8f00d67b80 1 -- 172.21.15.33:0/4130237067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8ca2c7270 con 0x55c8ca41eea0 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8ed5ffb700 1 -- 172.21.15.33:0/4130237067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ed8003cd0 con 0x55c8ca41eea0 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8f00d67b80 10 client.15750 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8f00d67b80 1 -- 172.21.15.33:0/4130237067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f8ee0011c60 con 0x55c8ca41eea0 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8ed5ffb700 1 -- 172.21.15.33:0/4130237067 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c8ca2c7270 con 0x55c8ca41eea0 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8ed5ffb700 1 -- 172.21.15.33:0/4130237067 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8ed8004860 con 0x55c8ca41eea0 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8f00d67b80 10 client.15750 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8f00d67b80 10 client.15750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8f00d67b80 20 client.15750 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c8ca433c40 0x55c8ca436060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8f00d67b80 4 client.15750 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.458+0000 7f8f00d67b80 1 -- 172.21.15.33:0/4130237067 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8ca3ed810 con 0x55c8ca433c40 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.459+0000 7f8eee59c700 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c8ca433c40 0x55c8ca436060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.459+0000 7f8ed5ffb700 10 client.15750 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.459+0000 7f8eee59c700 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c8ca433c40 0x55c8ca436060 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62b4df1e9676c7cf server_cookie=dfe71d1efcb8be1e in_seq=0 out_seq=0 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8ed5ffb700 1 -- 172.21.15.33:0/4130237067 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c8ca3ed810 con 0x55c8ca433c40 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8ed5ffb700 10 client.15750 handle_command_reply: tid=0 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8f00d67b80 1 client.15750 shutdown 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8f00d67b80 1 -- 172.21.15.33:0/4130237067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 msgr2=0x55c8ca428300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca428300 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8000c00 tx=0x7f8ed80053b0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8f00d67b80 1 -- 172.21.15.33:0/4130237067 shutdown_connections 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c8ca433c40 0x55c8ca436060 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41f860 0x55c8ca42f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8ed5ffb700 0 client.15750 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:19:29.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41e540 0x55c8ca427dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/4130237067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca428300 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8f00d67b80 1 -- 172.21.15.33:0/4130237067 >> 172.21.15.33:0/4130237067 conn(0x55c8ca37c210 msgr2=0x55c8ca4185a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8f00d67b80 1 -- 172.21.15.33:0/4130237067 shutdown_connections 2024-05-08T21:19:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8f00d67b80 1 -- 172.21.15.33:0/4130237067 wait complete. 2024-05-08T21:19:29.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.460+0000 7f8f00d67b80 20 client.15750 trim_cache size 0 max 16384 2024-05-08T21:19:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.474+0000 7f8f00d67b80 1 Processor -- start 2024-05-08T21:19:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.474+0000 7f8f00d67b80 1 -- start start 2024-05-08T21:19:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8f00d67b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41e540 0x55c8ca436030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8f00d67b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41eea0 0x55c8ca41e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8f00d67b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41f860 0x55c8ca42f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8f00d67b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ca43b250 con 0x55c8ca41f860 2024-05-08T21:19:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8f00d67b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ca428a40 con 0x55c8ca41eea0 2024-05-08T21:19:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8f00d67b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ca425ad0 con 0x55c8ca41e540 2024-05-08T21:19:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8eeed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41f860 0x55c8ca42f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8eeed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41f860 0x55c8ca42f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49024/0 (socket says 172.21.15.33:49024) 2024-05-08T21:19:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8eeed9d700 1 -- 172.21.15.33:0/3666960321 learned_addr learned my addr 172.21.15.33:0/3666960321 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8eee59c700 1 --2- 172.21.15.33:0/3666960321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41e540 0x55c8ca436030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8ee7fff700 1 --2- 172.21.15.33:0/3666960321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41eea0 0x55c8ca41e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8eeed9d700 1 -- 172.21.15.33:0/3666960321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41e540 msgr2=0x55c8ca436030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8eeed9d700 1 --2- 172.21.15.33:0/3666960321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41e540 0x55c8ca436030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8eeed9d700 1 -- 172.21.15.33:0/3666960321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41eea0 msgr2=0x55c8ca41e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8eeed9d700 1 --2- 172.21.15.33:0/3666960321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41eea0 0x55c8ca41e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8eeed9d700 1 -- 172.21.15.33:0/3666960321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8ca19bb60 con 0x55c8ca41f860 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8ee7fff700 1 --2- 172.21.15.33:0/3666960321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41eea0 0x55c8ca41e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8eeed9d700 1 --2- 172.21.15.33:0/3666960321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41f860 0x55c8ca42f9f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800ef00 tx=0x7f8ee8099280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8dcdf3e763bf39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8ee57fa700 1 -- 172.21.15.33:0/3666960321 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80b1070 con 0x55c8ca41f860 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8ee57fa700 1 -- 172.21.15.33:0/3666960321 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ee80b2220 con 0x55c8ca41f860 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.475+0000 7f8ee57fa700 1 -- 172.21.15.33:0/3666960321 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80a1d10 con 0x55c8ca41f860 2024-05-08T21:19:29.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.476+0000 7f8f00d67b80 1 -- 172.21.15.33:0/3666960321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41f860 msgr2=0x55c8ca42f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.476+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/3666960321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41f860 0x55c8ca42f9f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800ef00 tx=0x7f8ee8099280 comp rx=0 tx=0).stop 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.476+0000 7f8f00d67b80 1 -- 172.21.15.33:0/3666960321 shutdown_connections 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.476+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/3666960321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41f860 0x55c8ca42f9f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.476+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/3666960321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41e540 0x55c8ca436030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.476+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/3666960321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41eea0 0x55c8ca41e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.476+0000 7f8f00d67b80 1 -- 172.21.15.33:0/3666960321 >> 172.21.15.33:0/3666960321 conn(0x55c8ca37c210 msgr2=0x55c8ca41fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.476+0000 7f8f00d67b80 1 -- 172.21.15.33:0/3666960321 shutdown_connections 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.476+0000 7f8f00d67b80 1 -- 172.21.15.33:0/3666960321 wait complete. 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.476+0000 7f8f00d67b80 1 -- start start 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.476+0000 7f8f00d67b80 1 Processor -- start 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8f00d67b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41e540 0x55c8ca488120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8f00d67b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca48a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8f00d67b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41f860 0x55c8ca48cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8f00d67b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ca48d100 con 0x55c8ca41eea0 2024-05-08T21:19:29.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8f00d67b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ca48d240 con 0x55c8ca41f860 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8f00d67b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ca48d400 con 0x55c8ca41e540 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8f00d67b80 10 client.0 _ll_register_callbacks cb 0x55c8ca313b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8ee7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca48a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8ee7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca48a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49026/0 (socket says 172.21.15.33:49026) 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8ee7fff700 1 -- 172.21.15.33:0/1676679836 learned_addr learned my addr 172.21.15.33:0/1676679836 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8eeed9d700 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41f860 0x55c8ca48cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8eee59c700 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41e540 0x55c8ca488120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8ee7fff700 1 -- 172.21.15.33:0/1676679836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41e540 msgr2=0x55c8ca488120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8ee7fff700 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41e540 0x55c8ca488120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8ee7fff700 1 -- 172.21.15.33:0/1676679836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41f860 msgr2=0x55c8ca48cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8ee7fff700 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41f860 0x55c8ca48cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8ee7fff700 1 -- 172.21.15.33:0/1676679836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef019edf0 con 0x55c8ca41eea0 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8eee59c700 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41e540 0x55c8ca488120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8ee7fff700 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca48a670 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80098e0 tx=0x7f8ed8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4c1d2800ee4104 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.477+0000 7f8ed5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8ed5ffb700 1 -- 172.21.15.33:0/1676679836 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed801d040 con 0x55c8ca41eea0 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8ed5ffb700 1 -- 172.21.15.33:0/1676679836 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8ed8000e40 con 0x55c8ca41eea0 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8f00d67b80 1 -- 172.21.15.33:0/1676679836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ee0016070 con 0x55c8ca41eea0 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8ed5ffb700 1 -- 172.21.15.33:0/1676679836 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ed8005910 con 0x55c8ca41eea0 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8f00d67b80 10 client.15756 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8f00d67b80 1 -- 172.21.15.33:0/1676679836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c8ca334560 con 0x55c8ca41eea0 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8ed5ffb700 1 -- 172.21.15.33:0/1676679836 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ee0016070 con 0x55c8ca41eea0 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8ed5ffb700 1 -- 172.21.15.33:0/1676679836 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8ed803e030 con 0x55c8ca41eea0 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8f00d67b80 10 client.15756 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8f00d67b80 10 client.15756 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8f00d67b80 20 client.15756 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c8ca48ea10 0x55c8ca490e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8f00d67b80 4 client.15756 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.478+0000 7f8f00d67b80 1 -- 172.21.15.33:0/1676679836 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c8ca366ec0 con 0x55c8ca48ea10 2024-05-08T21:19:29.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8eee59c700 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c8ca48ea10 0x55c8ca490e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8eee59c700 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c8ca48ea10 0x55c8ca490e30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4fb345c5468793f1 server_cookie=90668421b341b5d8 in_seq=0 out_seq=0 2024-05-08T21:19:29.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8ed5ffb700 10 client.15756 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:19:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8ed5ffb700 1 -- 172.21.15.33:0/1676679836 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c8ca366ec0 con 0x55c8ca48ea10 2024-05-08T21:19:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8ed5ffb700 10 client.15756 handle_command_reply: tid=0 2024-05-08T21:19:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8f00d67b80 1 client.15756 shutdown 2024-05-08T21:19:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8f00d67b80 1 -- 172.21.15.33:0/1676679836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 msgr2=0x55c8ca48a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca48a670 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80098e0 tx=0x7f8ed8009310 comp rx=0 tx=0).stop 2024-05-08T21:19:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8f00d67b80 1 -- 172.21.15.33:0/1676679836 shutdown_connections 2024-05-08T21:19:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c8ca48ea10 0x55c8ca490e30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8ca41f860 0x55c8ca48cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8ed5ffb700 0 client.15756 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:19:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8ca41e540 0x55c8ca488120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8f00d67b80 1 --2- 172.21.15.33:0/1676679836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8ca41eea0 0x55c8ca48a670 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8f00d67b80 1 -- 172.21.15.33:0/1676679836 >> 172.21.15.33:0/1676679836 conn(0x55c8ca37c210 msgr2=0x55c8ca425dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8f00d67b80 1 -- 172.21.15.33:0/1676679836 shutdown_connections 2024-05-08T21:19:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.479+0000 7f8f00d67b80 1 -- 172.21.15.33:0/1676679836 wait complete. 2024-05-08T21:19:29.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8f00d67b80 20 client.15756 trim_cache size 0 max 16384 2024-05-08T21:19:29.488 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:19:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8ef4bad700 1 -- 172.21.15.33:0/72410936 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8edc0a1120 msgr2=0x7f8edc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8ef4bad700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8edc0a1120 0x7f8edc0a35e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8ef01981d0 tx=0x7f8ee0009260 comp rx=0 tx=0).stop 2024-05-08T21:19:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8ef4bad700 1 -- 172.21.15.33:0/72410936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0106bd0 msgr2=0x7f8ef019e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8ef4bad700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0106bd0 0x7f8ef019e270 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800ef30 tx=0x7f8ee800c3a0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8ef4bad700 1 -- 172.21.15.33:0/72410936 shutdown_connections 2024-05-08T21:19:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8ef4bad700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0106bd0 0x7f8ef019e270 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8ef4bad700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8edc0a1120 0x7f8edc0a35e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8ef4bad700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ef0101970 0x7f8ef0196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8ef4bad700 1 --2- 172.21.15.33:0/72410936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef01042a0 0x7f8ef01972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8ef4bad700 1 -- 172.21.15.33:0/72410936 >> 172.21.15.33:0/72410936 conn(0x7f8ef00fb300 msgr2=0x7f8ef0103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8ef4bad700 1 -- 172.21.15.33:0/72410936 shutdown_connections 2024-05-08T21:19:29.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.480+0000 7f8ef4bad700 1 -- 172.21.15.33:0/72410936 wait complete. 2024-05-08T21:19:29.495 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:29 smithi033 ceph-mon[42528]: pgmap v305: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:19:29.495 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:29 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3622747833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:19:29.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:19:29.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:29 smithi071 ceph-mon[47074]: pgmap v305: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:19:29.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:29 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3622747833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:19:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:29 smithi082 ceph-mon[47512]: pgmap v305: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:19:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:29 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3622747833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:19:29.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.633+0000 7fceaaa60700 1 -- 172.21.15.33:0/1414586022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea4101fe0 msgr2=0x7fcea4102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.633+0000 7fceaaa60700 1 --2- 172.21.15.33:0/1414586022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea4101fe0 0x7fcea4102460 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009990 tx=0x7fce8c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.633+0000 7fceaaa60700 1 -- 172.21.15.33:0/1414586022 shutdown_connections 2024-05-08T21:19:29.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.633+0000 7fceaaa60700 1 --2- 172.21.15.33:0/1414586022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcea41029a0 0x7fcea41092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.633+0000 7fceaaa60700 1 --2- 172.21.15.33:0/1414586022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea4100e90 0x7fcea41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.633+0000 7fceaaa60700 1 --2- 172.21.15.33:0/1414586022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea4101fe0 0x7fcea4102460 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.633+0000 7fceaaa60700 1 -- 172.21.15.33:0/1414586022 >> 172.21.15.33:0/1414586022 conn(0x7fcea40fc400 msgr2=0x7fcea40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.633+0000 7fceaaa60700 1 -- 172.21.15.33:0/1414586022 shutdown_connections 2024-05-08T21:19:29.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.633+0000 7fceaaa60700 1 -- 172.21.15.33:0/1414586022 wait complete. 2024-05-08T21:19:29.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.634+0000 7fceaaa60700 1 Processor -- start 2024-05-08T21:19:29.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.634+0000 7fceaaa60700 1 -- start start 2024-05-08T21:19:29.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.634+0000 7fceaaa60700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea4100e90 0x7fcea4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.634+0000 7fceaaa60700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea4101fe0 0x7fcea4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.634+0000 7fceaaa60700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcea41029a0 0x7fcea40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.634+0000 7fceaaa60700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4073f80 con 0x7fcea4100e90 2024-05-08T21:19:29.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.634+0000 7fceaaa60700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40740f0 con 0x7fcea41029a0 2024-05-08T21:19:29.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.634+0000 7fceaaa60700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea41a0e00 con 0x7fcea4101fe0 2024-05-08T21:19:29.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea4100e90 0x7fcea4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea4100e90 0x7fcea4074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49044/0 (socket says 172.21.15.33:49044) 2024-05-08T21:19:29.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea3fff700 1 -- 172.21.15.33:0/2989873683 learned_addr learned my addr 172.21.15.33:0/2989873683 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea37fe700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea4101fe0 0x7fcea4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea8ffd700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcea41029a0 0x7fcea40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea3fff700 1 -- 172.21.15.33:0/2989873683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea4101fe0 msgr2=0x7fcea4073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea3fff700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea4101fe0 0x7fcea4073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea3fff700 1 -- 172.21.15.33:0/2989873683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcea41029a0 msgr2=0x7fcea40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea3fff700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcea41029a0 0x7fcea40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea3fff700 1 -- 172.21.15.33:0/2989873683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c0095f0 con 0x7fcea4100e90 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea8ffd700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcea41029a0 0x7fcea40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea3fff700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea4100e90 0x7fcea4074c60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fce9400d950 tx=0x7fce9400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea17fa700 1 -- 172.21.15.33:0/2989873683 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94009940 con 0x7fcea4100e90 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea17fa700 1 -- 172.21.15.33:0/2989873683 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce94010460 con 0x7fcea4100e90 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fceaaa60700 1 -- 172.21.15.33:0/2989873683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea41a0fa0 con 0x7fcea4100e90 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fcea17fa700 1 -- 172.21.15.33:0/2989873683 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9400f620 con 0x7fcea4100e90 2024-05-08T21:19:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.635+0000 7fceaaa60700 1 -- 172.21.15.33:0/2989873683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea41a13d0 con 0x7fcea4100e90 2024-05-08T21:19:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.638+0000 7fcea17fa700 1 -- 172.21.15.33:0/2989873683 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fce940105d0 con 0x7fcea4100e90 2024-05-08T21:19:29.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.638+0000 7fcea17fa700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fce900a11a0 0x7fce900a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.639+0000 7fcea17fa700 1 -- 172.21.15.33:0/2989873683 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fce940cca70 con 0x7fcea4100e90 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.639+0000 7fcea37fe700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fce900a11a0 0x7fce900a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.639+0000 7fceb6c1ab80 1 Processor -- start 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.639+0000 7fcea37fe700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fce900a11a0 0x7fce900a3660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00b5c0 tx=0x7fce8c003720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.639+0000 7fceb6c1ab80 1 -- start start 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.639+0000 7fceb6c1ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9aaf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.639+0000 7fceb6c1ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9aafea0 0x563fb9ab0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.639+0000 7fceb6c1ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 0x563fb9ab0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.639+0000 7fceb6c1ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb9a06150 con 0x563fb9ab0860 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.639+0000 7fceb6c1ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb9a03b10 con 0x563fb9aaf540 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.639+0000 7fceb6c1ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb99ff400 con 0x563fb9aafea0 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fcea8ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 0x563fb9ab0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fcea8ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 0x563fb9ab0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49060/0 (socket says 172.21.15.33:49060) 2024-05-08T21:19:29.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fcea8ffd700 1 -- 172.21.15.33:0/3050995439 learned_addr learned my addr 172.21.15.33:0/3050995439 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fcea3fff700 1 --2- 172.21.15.33:0/3050995439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9aaf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fcea37fe700 1 --2- 172.21.15.33:0/3050995439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9aafea0 0x563fb9ab0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fcea8ffd700 1 -- 172.21.15.33:0/3050995439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9aafea0 msgr2=0x563fb9ab0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fcea8ffd700 1 --2- 172.21.15.33:0/3050995439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9aafea0 0x563fb9ab0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fcea8ffd700 1 -- 172.21.15.33:0/3050995439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 msgr2=0x563fb9aaf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fcea8ffd700 1 --2- 172.21.15.33:0/3050995439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9aaf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fcea8ffd700 1 -- 172.21.15.33:0/3050995439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fb98803b0 con 0x563fb9ab0860 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fcea8ffd700 1 --2- 172.21.15.33:0/3050995439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 0x563fb9ab0ce0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009e60 tx=0x7fce9c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3125058a26d553 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fce9a7fc700 1 -- 172.21.15.33:0/3050995439 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c014610 con 0x563fb9ab0860 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fce9a7fc700 1 -- 172.21.15.33:0/3050995439 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce9c00f050 con 0x563fb9ab0860 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.640+0000 7fce9a7fc700 1 -- 172.21.15.33:0/3050995439 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c013930 con 0x563fb9ab0860 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.641+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3050995439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 msgr2=0x563fb9ab0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.641+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/3050995439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 0x563fb9ab0ce0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009e60 tx=0x7fce9c0077e0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.641+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3050995439 shutdown_connections 2024-05-08T21:19:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.641+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/3050995439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 0x563fb9ab0ce0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.641+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/3050995439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9aaf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.641+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/3050995439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9aafea0 0x563fb9ab0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.641+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3050995439 >> 172.21.15.33:0/3050995439 conn(0x563fb9a0d210 msgr2=0x563fb9aa9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.641+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3050995439 shutdown_connections 2024-05-08T21:19:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.641+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3050995439 wait complete. 2024-05-08T21:19:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.641+0000 7fceb6c1ab80 1 -- start start 2024-05-08T21:19:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fceb6c1ab80 1 Processor -- start 2024-05-08T21:19:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fceb6c1ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aaf540 0x563fb9ab8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fceb6c1ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aafea0 0x563fb9ab92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fceb6c1ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9ab0860 0x563fb9ac0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fceb6c1ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb999b560 con 0x563fb9aaf540 2024-05-08T21:19:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fceb6c1ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb999b200 con 0x563fb9aafea0 2024-05-08T21:19:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fceb6c1ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb98826a0 con 0x563fb9ab0860 2024-05-08T21:19:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fceb6c1ab80 10 client.0 _ll_register_callbacks cb 0x563fb99a4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fcea3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aaf540 0x563fb9ab8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fcea3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aaf540 0x563fb9ab8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49072/0 (socket says 172.21.15.33:49072) 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fcea3fff700 1 -- 172.21.15.33:0/788324105 learned_addr learned my addr 172.21.15.33:0/788324105 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fcea37fe700 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aafea0 0x563fb9ab92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.642+0000 7fcea8ffd700 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9ab0860 0x563fb9ac0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fcea3fff700 1 -- 172.21.15.33:0/788324105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9ab0860 msgr2=0x563fb9ac0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fcea3fff700 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9ab0860 0x563fb9ac0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fcea3fff700 1 -- 172.21.15.33:0/788324105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aafea0 msgr2=0x563fb9ab92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fcea3fff700 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aafea0 0x563fb9ab92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fcea3fff700 1 -- 172.21.15.33:0/788324105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c009a90 con 0x563fb9aaf540 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fcea3fff700 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aaf540 0x563fb9ab8da0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fce9400b2f0 tx=0x7fce94099740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b71f72ba0a0cf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fce837fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fce837fe700 1 -- 172.21.15.33:0/788324105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940991f0 con 0x563fb9aaf540 2024-05-08T21:19:29.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fce837fe700 1 -- 172.21.15.33:0/788324105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fce94099b00 con 0x563fb9aaf540 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/788324105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fb9958270 con 0x563fb9aaf540 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fce837fe700 1 -- 172.21.15.33:0/788324105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce9409d930 con 0x563fb9aaf540 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fceb6c1ab80 10 client.15768 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/788324105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563fb982cb60 con 0x563fb9aaf540 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.643+0000 7fce837fe700 1 -- 172.21.15.33:0/788324105 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563fb9958270 con 0x563fb9aaf540 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.644+0000 7fce837fe700 1 -- 172.21.15.33:0/788324105 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce940b2290 con 0x563fb9aaf540 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.644+0000 7fceb6c1ab80 10 client.15768 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.644+0000 7fceb6c1ab80 10 client.15768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.644+0000 7fceb6c1ab80 20 client.15768 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.644+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563fb9ac4a10 0x563fb9ac6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.644+0000 7fceb6c1ab80 4 client.15768 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.644+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/788324105 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fb9a7e810 con 0x563fb9ac4a10 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.644+0000 7fcea37fe700 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563fb9ac4a10 0x563fb9ac6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fcea37fe700 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563fb9ac4a10 0x563fb9ac6e30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc3a54540206ceab server_cookie=f8a60b0bd7aa26a8 in_seq=0 out_seq=0 2024-05-08T21:19:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fce837fe700 10 client.15768 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fce837fe700 1 -- 172.21.15.33:0/788324105 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563fb9a7e810 con 0x563fb9ac4a10 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fce837fe700 10 client.15768 handle_command_reply: tid=0 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fceb6c1ab80 1 client.15768 shutdown 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/788324105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aaf540 msgr2=0x563fb9ab8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aaf540 0x563fb9ab8da0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fce9400b2f0 tx=0x7fce94099740 comp rx=0 tx=0).stop 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/788324105 shutdown_connections 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563fb9ac4a10 0x563fb9ac6e30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9ab0860 0x563fb9ac0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aaf540 0x563fb9ab8da0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fce837fe700 0 client.15768 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/788324105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aafea0 0x563fb9ab92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.645+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/788324105 >> 172.21.15.33:0/788324105 conn(0x563fb9a0d210 msgr2=0x563fb9aa95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.646+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/788324105 shutdown_connections 2024-05-08T21:19:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.646+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/788324105 wait complete. 2024-05-08T21:19:29.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.646+0000 7fceb6c1ab80 20 client.15768 trim_cache size 0 max 16384 2024-05-08T21:19:29.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.659+0000 7fceb6c1ab80 1 Processor -- start 2024-05-08T21:19:29.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.659+0000 7fceb6c1ab80 1 -- start start 2024-05-08T21:19:29.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fceb6c1ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9ac6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fceb6c1ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aafea0 0x563fb9ac02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fceb6c1ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9ab0860 0x563fb9ac09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fceb6c1ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb9ab7d60 con 0x563fb9aafea0 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fceb6c1ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb9aaa9a0 con 0x563fb9aaf540 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fceb6c1ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb9ab9f90 con 0x563fb9ab0860 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fcea37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aafea0 0x563fb9ac02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fcea37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aafea0 0x563fb9ac02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49074/0 (socket says 172.21.15.33:49074) 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fcea37fe700 1 -- 172.21.15.33:0/1137666013 learned_addr learned my addr 172.21.15.33:0/1137666013 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fcea3fff700 1 --2- 172.21.15.33:0/1137666013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9ac6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fcea8ffd700 1 --2- 172.21.15.33:0/1137666013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9ab0860 0x563fb9ac09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fcea37fe700 1 -- 172.21.15.33:0/1137666013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9ab0860 msgr2=0x563fb9ac09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fcea37fe700 1 --2- 172.21.15.33:0/1137666013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9ab0860 0x563fb9ac09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fcea37fe700 1 -- 172.21.15.33:0/1137666013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 msgr2=0x563fb9ac6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fcea37fe700 1 --2- 172.21.15.33:0/1137666013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9ac6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fcea37fe700 1 -- 172.21.15.33:0/1137666013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fb99c5560 con 0x563fb9aafea0 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fcea3fff700 1 --2- 172.21.15.33:0/1137666013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9ac6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fcea37fe700 1 --2- 172.21.15.33:0/1137666013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aafea0 0x563fb9ac02e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fce8c000c00 tx=0x7fce8c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf5137f51d5b7fd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.660+0000 7fce9affd700 1 -- 172.21.15.33:0/1137666013 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c005660 con 0x563fb9aafea0 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.661+0000 7fce9affd700 1 -- 172.21.15.33:0/1137666013 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce8c02c040 con 0x563fb9aafea0 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.661+0000 7fce9affd700 1 -- 172.21.15.33:0/1137666013 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c004900 con 0x563fb9aafea0 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.661+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/1137666013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aafea0 msgr2=0x563fb9ac02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.661+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/1137666013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aafea0 0x563fb9ac02e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fce8c000c00 tx=0x7fce8c023000 comp rx=0 tx=0).stop 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.661+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/1137666013 shutdown_connections 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.661+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/1137666013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9ab0860 0x563fb9ac09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.661+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/1137666013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9ac6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.661+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/1137666013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9aafea0 0x563fb9ac02e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.661+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/1137666013 >> 172.21.15.33:0/1137666013 conn(0x563fb9a0d210 msgr2=0x563fb9ab0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.661+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/1137666013 shutdown_connections 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.661+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/1137666013 wait complete. 2024-05-08T21:19:29.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.661+0000 7fceb6c1ab80 1 -- start start 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fceb6c1ab80 1 Processor -- start 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fceb6c1ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9abba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fceb6c1ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9aafea0 0x563fb9abbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fceb6c1ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 0x563fb9abeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fceb6c1ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb9abf3f0 con 0x563fb9ab0860 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fceb6c1ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb9abf560 con 0x563fb9aaf540 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fceb6c1ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fb9abf720 con 0x563fb9aafea0 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fceb6c1ab80 10 client.0 _ll_register_callbacks cb 0x563fb99a4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fcea8ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 0x563fb9abeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fcea8ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 0x563fb9abeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49086/0 (socket says 172.21.15.33:49086) 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fcea8ffd700 1 -- 172.21.15.33:0/3751977431 learned_addr learned my addr 172.21.15.33:0/3751977431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fcea3fff700 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9abba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.662+0000 7fcea37fe700 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9aafea0 0x563fb9abbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fcea8ffd700 1 -- 172.21.15.33:0/3751977431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9aafea0 msgr2=0x563fb9abbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fcea8ffd700 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9aafea0 0x563fb9abbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fcea8ffd700 1 -- 172.21.15.33:0/3751977431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 msgr2=0x563fb9abba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fcea8ffd700 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9abba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fcea8ffd700 1 -- 172.21.15.33:0/3751977431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea4109bf0 con 0x563fb9ab0860 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fcea8ffd700 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 0x563fb9abeeb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x563fb9aaad40 tx=0x7fce9c014640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd810a446f916896 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fce837fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fce837fe700 1 -- 172.21.15.33:0/3751977431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c023070 con 0x563fb9ab0860 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fce837fe700 1 -- 172.21.15.33:0/3751977431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fce9c014730 con 0x563fb9ab0860 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3751977431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce9c00f050 con 0x563fb9ab0860 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fce837fe700 1 -- 172.21.15.33:0/3751977431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce9c0090b0 con 0x563fb9ab0860 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fceb6c1ab80 10 client.15774 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3751977431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563fb9884cd0 con 0x563fb9ab0860 2024-05-08T21:19:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.663+0000 7fce837fe700 1 -- 172.21.15.33:0/3751977431 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce9c00f050 con 0x563fb9ab0860 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fce837fe700 1 -- 172.21.15.33:0/3751977431 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce9c013a10 con 0x563fb9ab0860 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fceb6c1ab80 10 client.15774 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fceb6c1ab80 10 client.15774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fceb6c1ab80 20 client.15774 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563fb9b1b830 0x563fb9b1dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fceb6c1ab80 4 client.15774 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3751977431 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563fb99f7ec0 con 0x563fb9b1b830 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fcea3fff700 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563fb9b1b830 0x563fb9b1dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fce837fe700 10 client.15774 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fcea3fff700 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563fb9b1b830 0x563fb9b1dc50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a9c26e3d90aa953 server_cookie=c992f945fa0e3f7d in_seq=0 out_seq=0 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fce837fe700 1 -- 172.21.15.33:0/3751977431 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563fb99f7ec0 con 0x563fb9b1b830 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fce837fe700 10 client.15774 handle_command_reply: tid=0 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.664+0000 7fceb6c1ab80 1 client.15774 shutdown 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3751977431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 msgr2=0x563fb9abeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 0x563fb9abeeb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x563fb9aaad40 tx=0x7fce9c014640 comp rx=0 tx=0).stop 2024-05-08T21:19:29.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3751977431 shutdown_connections 2024-05-08T21:19:29.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563fb9b1b830 0x563fb9b1dc50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563fb9ab0860 0x563fb9abeeb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563fb9aaf540 0x563fb9abba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fce837fe700 0 client.15774 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:19:29.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceb6c1ab80 1 --2- 172.21.15.33:0/3751977431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fb9aafea0 0x563fb9abbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3751977431 >> 172.21.15.33:0/3751977431 conn(0x563fb9a0d210 msgr2=0x563fb9ab0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3751977431 shutdown_connections 2024-05-08T21:19:29.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceb6c1ab80 1 -- 172.21.15.33:0/3751977431 wait complete. 2024-05-08T21:19:29.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceb6c1ab80 20 client.15774 trim_cache size 0 max 16384 2024-05-08T21:19:29.673 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:19:29.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceaaa60700 1 -- 172.21.15.33:0/2989873683 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fce900a11a0 msgr2=0x7fce900a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceaaa60700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fce900a11a0 0x7fce900a3660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00b5c0 tx=0x7fce8c003720 comp rx=0 tx=0).stop 2024-05-08T21:19:29.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceaaa60700 1 -- 172.21.15.33:0/2989873683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea4100e90 msgr2=0x7fcea4074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceaaa60700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea4100e90 0x7fcea4074c60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fce9400d950 tx=0x7fce9400dc60 comp rx=0 tx=0).stop 2024-05-08T21:19:29.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceaaa60700 1 -- 172.21.15.33:0/2989873683 shutdown_connections 2024-05-08T21:19:29.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceaaa60700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fce900a11a0 0x7fce900a3660 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceaaa60700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcea41029a0 0x7fcea40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceaaa60700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcea4100e90 0x7fcea4074c60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceaaa60700 1 --2- 172.21.15.33:0/2989873683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea4101fe0 0x7fcea4073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceaaa60700 1 -- 172.21.15.33:0/2989873683 >> 172.21.15.33:0/2989873683 conn(0x7fcea40fc400 msgr2=0x7fcea4105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.665+0000 7fceaaa60700 1 -- 172.21.15.33:0/2989873683 shutdown_connections 2024-05-08T21:19:29.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.666+0000 7fceaaa60700 1 -- 172.21.15.33:0/2989873683 wait complete. 2024-05-08T21:19:29.681 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:19:29.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.819+0000 7f9234a3d700 1 -- 172.21.15.33:0/2862062073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9230106bc0 msgr2=0x7f9230108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.819+0000 7f9234a3d700 1 --2- 172.21.15.33:0/2862062073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9230106bc0 0x7f9230108fd0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f922800b0a0 tx=0x7f922800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.819+0000 7f9234a3d700 1 -- 172.21.15.33:0/2862062073 shutdown_connections 2024-05-08T21:19:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.819+0000 7f9234a3d700 1 --2- 172.21.15.33:0/2862062073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9230106bc0 0x7f9230108fd0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.819+0000 7f9234a3d700 1 --2- 172.21.15.33:0/2862062073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230101960 0x7f9230103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.819+0000 7f9234a3d700 1 --2- 172.21.15.33:0/2862062073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9230104290 0x7f9230106680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.819+0000 7f9234a3d700 1 -- 172.21.15.33:0/2862062073 >> 172.21.15.33:0/2862062073 conn(0x7f92300fb310 msgr2=0x7f92300fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.819+0000 7f9234a3d700 1 -- 172.21.15.33:0/2862062073 shutdown_connections 2024-05-08T21:19:29.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.819+0000 7f9234a3d700 1 -- 172.21.15.33:0/2862062073 wait complete. 2024-05-08T21:19:29.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.820+0000 7f9234a3d700 1 Processor -- start 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.820+0000 7f9234a3d700 1 -- start start 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.820+0000 7f9234a3d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9230101960 0x7f9230196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.820+0000 7f9234a3d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230104290 0x7f92301973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.820+0000 7f9234a3d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9230106bc0 0x7f923019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.820+0000 7f9234a3d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923019e900 con 0x7f9230101960 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.820+0000 7f9234a3d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923019ea70 con 0x7f9230104290 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.820+0000 7f9234a3d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923019ec30 con 0x7f9230106bc0 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9230101960 0x7f9230196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9230101960 0x7f9230196e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49104/0 (socket says 172.21.15.33:49104) 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922e59c700 1 -- 172.21.15.33:0/1529507316 learned_addr learned my addr 172.21.15.33:0/1529507316 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922ed9d700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9230106bc0 0x7f923019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922dd9b700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230104290 0x7f92301973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922e59c700 1 -- 172.21.15.33:0/1529507316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9230106bc0 msgr2=0x7f923019e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922e59c700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9230106bc0 0x7f923019e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922e59c700 1 -- 172.21.15.33:0/1529507316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230104290 msgr2=0x7f92301973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922e59c700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230104290 0x7f92301973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922e59c700 1 -- 172.21.15.33:0/1529507316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9220009520 con 0x7f9230101960 2024-05-08T21:19:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922dd9b700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230104290 0x7f92301973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922ed9d700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9230106bc0 0x7f923019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f922e59c700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9230101960 0x7f9230196e60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f921800e9a0 tx=0x7f921800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f92277fe700 1 -- 172.21.15.33:0/1529507316 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9218009790 con 0x7f9230101960 2024-05-08T21:19:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f92277fe700 1 -- 172.21.15.33:0/1529507316 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f921800cbd0 con 0x7f9230101960 2024-05-08T21:19:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f9234a3d700 1 -- 172.21.15.33:0/1529507316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9228009d00 con 0x7f9230101960 2024-05-08T21:19:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f92277fe700 1 -- 172.21.15.33:0/1529507316 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9218018680 con 0x7f9230101960 2024-05-08T21:19:29.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.821+0000 7f9234a3d700 1 -- 172.21.15.33:0/1529507316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923019f1d0 con 0x7f9230101960 2024-05-08T21:19:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.824+0000 7f92277fe700 1 -- 172.21.15.33:0/1529507316 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f921800cd40 con 0x7f9230101960 2024-05-08T21:19:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.824+0000 7f92277fe700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f921c0a1120 0x7f921c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.824+0000 7f922dd9b700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f921c0a1120 0x7f921c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.824+0000 7f92277fe700 1 -- 172.21.15.33:0/1529507316 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9218014070 con 0x7f9230101960 2024-05-08T21:19:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.825+0000 7f922dd9b700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f921c0a1120 0x7f921c0a35e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9220009e40 tx=0x7f9220009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.825+0000 7f9240bf7b80 1 Processor -- start 2024-05-08T21:19:29.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f9240bf7b80 1 -- start start 2024-05-08T21:19:29.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f9240bf7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941540 0x5584e9941960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f9240bf7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941ea0 0x5584e9942320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f9240bf7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 0x5584e9942ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f9240bf7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584e9898150 con 0x5584e9942860 2024-05-08T21:19:29.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f9240bf7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584e9895b10 con 0x5584e9941540 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f9240bf7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584e9891400 con 0x5584e9941ea0 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941540 0x5584e9941960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 0x5584e9942ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 0x5584e9942ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49116/0 (socket says 172.21.15.33:49116) 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941540 0x5584e9941960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:50872/0 (socket says 172.21.15.33:50872) 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922ed9d700 1 -- 172.21.15.33:0/46718076 learned_addr learned my addr 172.21.15.33:0/46718076 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922dd9b700 1 --2- 172.21.15.33:0/46718076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941ea0 0x5584e9942320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922ed9d700 1 -- 172.21.15.33:0/46718076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941ea0 msgr2=0x5584e9942320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922ed9d700 1 --2- 172.21.15.33:0/46718076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941ea0 0x5584e9942320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922ed9d700 1 -- 172.21.15.33:0/46718076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941540 msgr2=0x5584e9941960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922ed9d700 1 --2- 172.21.15.33:0/46718076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941540 0x5584e9941960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922ed9d700 1 -- 172.21.15.33:0/46718076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584e97123b0 con 0x5584e9942860 2024-05-08T21:19:29.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922dd9b700 1 --2- 172.21.15.33:0/46718076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941ea0 0x5584e9942320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.826+0000 7f922e59c700 1 --2- 172.21.15.33:0/46718076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941540 0x5584e9941960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.827+0000 7f922ed9d700 1 --2- 172.21.15.33:0/46718076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 0x5584e9942ce0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f922800b780 tx=0x7f92280092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2818630811ba64bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.827+0000 7f9224ff9700 1 -- 172.21.15.33:0/46718076 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922800e050 con 0x5584e9942860 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.827+0000 7f9224ff9700 1 -- 172.21.15.33:0/46718076 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92280094d0 con 0x5584e9942860 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.827+0000 7f9224ff9700 1 -- 172.21.15.33:0/46718076 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92280054e0 con 0x5584e9942860 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 -- 172.21.15.33:0/46718076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 msgr2=0x5584e9942ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/46718076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 0x5584e9942ce0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f922800b780 tx=0x7f92280092e0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 -- 172.21.15.33:0/46718076 shutdown_connections 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/46718076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 0x5584e9942ce0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/46718076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941540 0x5584e9941960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/46718076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941ea0 0x5584e9942320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 -- 172.21.15.33:0/46718076 >> 172.21.15.33:0/46718076 conn(0x5584e989f210 msgr2=0x5584e993bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 -- 172.21.15.33:0/46718076 shutdown_connections 2024-05-08T21:19:29.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 -- 172.21.15.33:0/46718076 wait complete. 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 -- start start 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 Processor -- start 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941540 0x5584e994ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.828+0000 7f9240bf7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941ea0 0x5584e994b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f9240bf7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 0x5584e99522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f9240bf7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584e982d560 con 0x5584e9942860 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f9240bf7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584e982d200 con 0x5584e9941ea0 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f9240bf7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584e97146a0 con 0x5584e9941540 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f9240bf7b80 10 client.0 _ll_register_callbacks cb 0x5584e9836b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f922ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 0x5584e99522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f922ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 0x5584e99522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49118/0 (socket says 172.21.15.33:49118) 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f922e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941540 0x5584e994ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f922ed9d700 1 -- 172.21.15.33:0/2769905025 learned_addr learned my addr 172.21.15.33:0/2769905025 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f922dd9b700 1 --2- 172.21.15.33:0/2769905025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941ea0 0x5584e994b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f922ed9d700 1 -- 172.21.15.33:0/2769905025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941540 msgr2=0x5584e994ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f922ed9d700 1 --2- 172.21.15.33:0/2769905025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941540 0x5584e994ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f922ed9d700 1 -- 172.21.15.33:0/2769905025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941ea0 msgr2=0x5584e994b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f922ed9d700 1 --2- 172.21.15.33:0/2769905025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941ea0 0x5584e994b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f922ed9d700 1 -- 172.21.15.33:0/2769905025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923019f1d0 con 0x5584e9942860 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f922ed9d700 1 --2- 172.21.15.33:0/2769905025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 0x5584e99522d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9228007af0 tx=0x7f92280099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf2b539c6e56f16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f9215ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f9215ffb700 1 -- 172.21.15.33:0/2769905025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9228003e10 con 0x5584e9942860 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f9215ffb700 1 -- 172.21.15.33:0/2769905025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f92280051e0 con 0x5584e9942860 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.829+0000 7f9240bf7b80 1 -- 172.21.15.33:0/2769905025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584e97ea270 con 0x5584e9942860 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.830+0000 7f9215ffb700 1 -- 172.21.15.33:0/2769905025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92280074a0 con 0x5584e9942860 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.830+0000 7f9240bf7b80 10 client.15786 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.830+0000 7f9240bf7b80 1 -- 172.21.15.33:0/2769905025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f9230109bc0 con 0x5584e9942860 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.830+0000 7f9215ffb700 1 -- 172.21.15.33:0/2769905025 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5584e97ea270 con 0x5584e9942860 2024-05-08T21:19:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.830+0000 7f9215ffb700 1 -- 172.21.15.33:0/2769905025 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f922800b0d0 con 0x5584e9942860 2024-05-08T21:19:29.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.830+0000 7f9240bf7b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:29.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.830+0000 7f9240bf7b80 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:19:29.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.830+0000 7f9240bf7b80 20 client.15786 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:29.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.830+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/2769905025 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5584e9956740 0x5584e9958b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.830+0000 7f9240bf7b80 4 client.15786 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:19:29.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.830+0000 7f9240bf7b80 1 -- 172.21.15.33:0/2769905025 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584e9910810 con 0x5584e9956740 2024-05-08T21:19:29.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.830+0000 7f922e59c700 1 --2- 172.21.15.33:0/2769905025 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5584e9956740 0x5584e9958b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.831+0000 7f922e59c700 1 --2- 172.21.15.33:0/2769905025 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5584e9956740 0x5584e9958b60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d1fcf1e9230d69e server_cookie=ba6da089fcd9d24f in_seq=0 out_seq=0 2024-05-08T21:19:29.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.831+0000 7f9215ffb700 10 client.15786 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:29.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9215ffb700 1 -- 172.21.15.33:0/2769905025 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5584e9910810 con 0x5584e9956740 2024-05-08T21:19:29.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9215ffb700 10 client.15786 handle_command_reply: tid=0 2024-05-08T21:19:29.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9240bf7b80 1 client.15786 shutdown 2024-05-08T21:19:29.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9240bf7b80 1 -- 172.21.15.33:0/2769905025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 msgr2=0x5584e99522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/2769905025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 0x5584e99522d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9228007af0 tx=0x7f92280099e0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9240bf7b80 1 -- 172.21.15.33:0/2769905025 shutdown_connections 2024-05-08T21:19:29.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/2769905025 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5584e9956740 0x5584e9958b60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/2769905025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9942860 0x5584e99522d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/2769905025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941540 0x5584e994ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/2769905025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941ea0 0x5584e994b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9215ffb700 0 client.15786 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:29.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9240bf7b80 1 -- 172.21.15.33:0/2769905025 >> 172.21.15.33:0/2769905025 conn(0x5584e989f210 msgr2=0x5584e993b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9240bf7b80 1 -- 172.21.15.33:0/2769905025 shutdown_connections 2024-05-08T21:19:29.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9240bf7b80 1 -- 172.21.15.33:0/2769905025 wait complete. 2024-05-08T21:19:29.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.832+0000 7f9240bf7b80 20 client.15786 trim_cache size 0 max 16384 2024-05-08T21:19:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.846+0000 7f9240bf7b80 1 Processor -- start 2024-05-08T21:19:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f9240bf7b80 1 -- start start 2024-05-08T21:19:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f9240bf7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941540 0x5584e9958b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f9240bf7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941ea0 0x5584e9952980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f9240bf7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9942860 0x5584e994a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f9240bf7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584e995dd70 con 0x5584e9941ea0 2024-05-08T21:19:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f9240bf7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584e9936800 con 0x5584e9941540 2024-05-08T21:19:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f9240bf7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584e99489d0 con 0x5584e9942860 2024-05-08T21:19:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f922dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941ea0 0x5584e9952980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f922dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941ea0 0x5584e9952980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49124/0 (socket says 172.21.15.33:49124) 2024-05-08T21:19:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f922dd9b700 1 -- 172.21.15.33:0/1808922719 learned_addr learned my addr 172.21.15.33:0/1808922719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f922ed9d700 1 --2- 172.21.15.33:0/1808922719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9942860 0x5584e994a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f922e59c700 1 --2- 172.21.15.33:0/1808922719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941540 0x5584e9958b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f922dd9b700 1 -- 172.21.15.33:0/1808922719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9942860 msgr2=0x5584e994a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f922dd9b700 1 --2- 172.21.15.33:0/1808922719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9942860 0x5584e994a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f922dd9b700 1 -- 172.21.15.33:0/1808922719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941540 msgr2=0x5584e9958b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f922dd9b700 1 --2- 172.21.15.33:0/1808922719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941540 0x5584e9958b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f922dd9b700 1 -- 172.21.15.33:0/1808922719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584e96beb60 con 0x5584e9941ea0 2024-05-08T21:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f922ed9d700 1 --2- 172.21.15.33:0/1808922719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9942860 0x5584e994a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f922dd9b700 1 --2- 172.21.15.33:0/1808922719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941ea0 0x5584e9952980 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9220012600 tx=0x7f9220014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254d016c2ffd56c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.847+0000 7f92257fa700 1 -- 172.21.15.33:0/1808922719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9220011cc0 con 0x5584e9941ea0 2024-05-08T21:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.848+0000 7f92257fa700 1 -- 172.21.15.33:0/1808922719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9220016070 con 0x5584e9941ea0 2024-05-08T21:19:29.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.848+0000 7f92257fa700 1 -- 172.21.15.33:0/1808922719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92200209c0 con 0x5584e9941ea0 2024-05-08T21:19:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.848+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1808922719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941ea0 msgr2=0x5584e9952980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.848+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/1808922719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941ea0 0x5584e9952980 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9220012600 tx=0x7f9220014000 comp rx=0 tx=0).stop 2024-05-08T21:19:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.848+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1808922719 shutdown_connections 2024-05-08T21:19:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.848+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/1808922719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9942860 0x5584e994a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.848+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/1808922719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9941540 0x5584e9958b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.848+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/1808922719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941ea0 0x5584e9952980 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.848+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1808922719 >> 172.21.15.33:0/1808922719 conn(0x5584e989f210 msgr2=0x5584e9942c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.848+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1808922719 shutdown_connections 2024-05-08T21:19:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.848+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1808922719 wait complete. 2024-05-08T21:19:29.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f9240bf7b80 1 -- start start 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f9240bf7b80 1 Processor -- start 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f9240bf7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941540 0x5584e994dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f9240bf7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941ea0 0x5584e994e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f9240bf7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9942860 0x5584e9951150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f9240bf7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584e9951690 con 0x5584e9941540 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f9240bf7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584e9951800 con 0x5584e9942860 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f9240bf7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584e9951970 con 0x5584e9941ea0 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f9240bf7b80 10 client.0 _ll_register_callbacks cb 0x5584e9836b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f922e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941540 0x5584e994dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f922e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941540 0x5584e994dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49138/0 (socket says 172.21.15.33:49138) 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f922e59c700 1 -- 172.21.15.33:0/1297621315 learned_addr learned my addr 172.21.15.33:0/1297621315 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f922dd9b700 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941ea0 0x5584e994e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f922ed9d700 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9942860 0x5584e9951150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f922e59c700 1 -- 172.21.15.33:0/1297621315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941ea0 msgr2=0x5584e994e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f922e59c700 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941ea0 0x5584e994e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f922e59c700 1 -- 172.21.15.33:0/1297621315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9942860 msgr2=0x5584e9951150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f922e59c700 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9942860 0x5584e9951150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f922e59c700 1 -- 172.21.15.33:0/1297621315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9220011e20 con 0x5584e9941540 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f922dd9b700 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941ea0 0x5584e994e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.849+0000 7f922ed9d700 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9942860 0x5584e9951150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f922e59c700 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941540 0x5584e994dd80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9218010bd0 tx=0x7f92180964b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ef4a8f885553fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9215ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9215ffb700 1 -- 172.21.15.33:0/1297621315 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9218099530 con 0x5584e9941540 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9215ffb700 1 -- 172.21.15.33:0/1297621315 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9218099690 con 0x5584e9941540 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1297621315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92280094d0 con 0x5584e9941540 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9215ffb700 1 -- 172.21.15.33:0/1297621315 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92180a2030 con 0x5584e9941540 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9240bf7b80 10 client.15792 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:29.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1297621315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5584e9857560 con 0x5584e9941540 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9215ffb700 1 -- 172.21.15.33:0/1297621315 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92280094d0 con 0x5584e9941540 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9215ffb700 1 -- 172.21.15.33:0/1297621315 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f92180cacd0 con 0x5584e9941540 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9240bf7b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9240bf7b80 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9240bf7b80 20 client.15792 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5584e99ad770 0x5584e99afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9240bf7b80 4 client.15792 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.850+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1297621315 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5584e9889ec0 con 0x5584e99ad770 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.851+0000 7f922dd9b700 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5584e99ad770 0x5584e99afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.851+0000 7f9215ffb700 10 client.15792 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.851+0000 7f922dd9b700 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5584e99ad770 0x5584e99afb90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c55b8eb0e691a3d server_cookie=12c3d447a23c482 in_seq=0 out_seq=0 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.851+0000 7f9215ffb700 1 -- 172.21.15.33:0/1297621315 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5584e9889ec0 con 0x5584e99ad770 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.851+0000 7f9215ffb700 10 client.15792 handle_command_reply: tid=0 2024-05-08T21:19:29.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.851+0000 7f9240bf7b80 1 client.15792 shutdown 2024-05-08T21:19:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.851+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1297621315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941540 msgr2=0x5584e994dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.851+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941540 0x5584e994dd80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9218010bd0 tx=0x7f92180964b0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.851+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1297621315 shutdown_connections 2024-05-08T21:19:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.851+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5584e99ad770 0x5584e99afb90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584e9942860 0x5584e9951150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5584e9941540 0x5584e994dd80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9240bf7b80 1 --2- 172.21.15.33:0/1297621315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5584e9941ea0 0x5584e994e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1297621315 >> 172.21.15.33:0/1297621315 conn(0x5584e989f210 msgr2=0x5584e9945730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9215ffb700 0 client.15792 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1297621315 shutdown_connections 2024-05-08T21:19:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9240bf7b80 1 -- 172.21.15.33:0/1297621315 wait complete. 2024-05-08T21:19:29.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9240bf7b80 20 client.15792 trim_cache size 0 max 16384 2024-05-08T21:19:29.861 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:19:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9234a3d700 1 -- 172.21.15.33:0/1529507316 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f921c0a1120 msgr2=0x7f921c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9234a3d700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f921c0a1120 0x7f921c0a35e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9220009e40 tx=0x7f9220009260 comp rx=0 tx=0).stop 2024-05-08T21:19:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9234a3d700 1 -- 172.21.15.33:0/1529507316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9230101960 msgr2=0x7f9230196e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9234a3d700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9230101960 0x7f9230196e60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f921800e9a0 tx=0x7f921800ed60 comp rx=0 tx=0).stop 2024-05-08T21:19:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9234a3d700 1 -- 172.21.15.33:0/1529507316 shutdown_connections 2024-05-08T21:19:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9234a3d700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9230106bc0 0x7f923019e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9234a3d700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9230101960 0x7f9230196e60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9234a3d700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f921c0a1120 0x7f921c0a35e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9234a3d700 1 --2- 172.21.15.33:0/1529507316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230104290 0x7f92301973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9234a3d700 1 -- 172.21.15.33:0/1529507316 >> 172.21.15.33:0/1529507316 conn(0x7f92300fb310 msgr2=0x7f92301022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9234a3d700 1 -- 172.21.15.33:0/1529507316 shutdown_connections 2024-05-08T21:19:29.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:29.852+0000 7f9234a3d700 1 -- 172.21.15.33:0/1529507316 wait complete. 2024-05-08T21:19:30.868 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:19:30.869 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:19:31.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.017+0000 7f217b2ff700 1 -- 172.21.15.33:0/3072672059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2174101940 msgr2=0x7f2174103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.017+0000 7f217b2ff700 1 --2- 172.21.15.33:0/3072672059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2174101940 0x7f2174103d30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2170009990 tx=0x7f2170009ca0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.017+0000 7f217b2ff700 1 -- 172.21.15.33:0/3072672059 shutdown_connections 2024-05-08T21:19:31.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.017+0000 7f217b2ff700 1 --2- 172.21.15.33:0/3072672059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2174106ba0 0x7f2174108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.017+0000 7f217b2ff700 1 --2- 172.21.15.33:0/3072672059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2174101940 0x7f2174103d30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.017+0000 7f217b2ff700 1 --2- 172.21.15.33:0/3072672059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2174104270 0x7f2174106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.017+0000 7f217b2ff700 1 -- 172.21.15.33:0/3072672059 >> 172.21.15.33:0/3072672059 conn(0x7f21740fb290 msgr2=0x7f21740fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:31.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.017+0000 7f217b2ff700 1 -- 172.21.15.33:0/3072672059 shutdown_connections 2024-05-08T21:19:31.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.017+0000 7f217b2ff700 1 -- 172.21.15.33:0/3072672059 wait complete. 2024-05-08T21:19:31.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.018+0000 7f217b2ff700 1 Processor -- start 2024-05-08T21:19:31.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.018+0000 7f217b2ff700 1 -- start start 2024-05-08T21:19:31.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.018+0000 7f217b2ff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2174101940 0x7f2174196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.018+0000 7f217b2ff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2174104270 0x7f2174197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.018+0000 7f217b2ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2174106ba0 0x7f217419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217b2ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217419e7e0 con 0x7f2174106ba0 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217b2ff700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217419e950 con 0x7f2174101940 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217b2ff700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217419eb10 con 0x7f2174104270 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217989c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2174106ba0 0x7f217419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217989c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2174106ba0 0x7f217419e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46554/0 (socket says 172.21.15.33:46554) 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217989c700 1 -- 172.21.15.33:0/3498595542 learned_addr learned my addr 172.21.15.33:0/3498595542 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217909b700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2174101940 0x7f2174196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217889a700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2174104270 0x7f2174197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217989c700 1 -- 172.21.15.33:0/3498595542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2174104270 msgr2=0x7f2174197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217989c700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2174104270 0x7f2174197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217989c700 1 -- 172.21.15.33:0/3498595542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2174101940 msgr2=0x7f2174196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217989c700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2174101940 0x7f2174196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217989c700 1 -- 172.21.15.33:0/3498595542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21700095f0 con 0x7f2174106ba0 2024-05-08T21:19:31.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217889a700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2174104270 0x7f2174197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:31.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217989c700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2174106ba0 0x7f217419e210 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f216c007ee0 tx=0x7f216c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:31.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f216a7fc700 1 -- 172.21.15.33:0/3498595542 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c017070 con 0x7f2174106ba0 2024-05-08T21:19:31.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f216a7fc700 1 -- 172.21.15.33:0/3498595542 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f216c00f040 con 0x7f2174106ba0 2024-05-08T21:19:31.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f216a7fc700 1 -- 172.21.15.33:0/3498595542 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c005710 con 0x7f2174106ba0 2024-05-08T21:19:31.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217b2ff700 1 -- 172.21.15.33:0/3498595542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217419edf0 con 0x7f2174106ba0 2024-05-08T21:19:31.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.019+0000 7f217b2ff700 1 -- 172.21.15.33:0/3498595542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217419f220 con 0x7f2174106ba0 2024-05-08T21:19:31.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.023+0000 7f216a7fc700 1 -- 172.21.15.33:0/3498595542 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f216c005870 con 0x7f2174106ba0 2024-05-08T21:19:31.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.023+0000 7f21874b9b80 1 Processor -- start 2024-05-08T21:19:31.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.023+0000 7f21874b9b80 1 -- start start 2024-05-08T21:19:31.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f21874b9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8e9d0 0x562473a8edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f21874b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 0x562473a8f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f21874b9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8fcf0 0x562473a90170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f21874b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624739e11d0 con 0x562473a8f330 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f21874b9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624739deb90 con 0x562473a8fcf0 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f21874b9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624739da480 con 0x562473a8e9d0 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f217909b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 0x562473a8f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f217909b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 0x562473a8f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46564/0 (socket says 172.21.15.33:46564) 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f217909b700 1 -- 172.21.15.33:0/4268701981 learned_addr learned my addr 172.21.15.33:0/4268701981 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f217889a700 1 --2- 172.21.15.33:0/4268701981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8fcf0 0x562473a90170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f216a7fc700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21600a11a0 0x7f21600a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f217889a700 1 --2- 172.21.15.33:0/4268701981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8e9d0 0x562473a8edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f217909b700 1 -- 172.21.15.33:0/4268701981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8e9d0 msgr2=0x562473a8edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f217909b700 1 --2- 172.21.15.33:0/4268701981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8e9d0 0x562473a8edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f217909b700 1 -- 172.21.15.33:0/4268701981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8fcf0 msgr2=0x562473a90170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f217909b700 1 --2- 172.21.15.33:0/4268701981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8fcf0 0x562473a90170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f216a7fc700 1 -- 172.21.15.33:0/3498595542 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f216c0cc3c0 con 0x7f2174106ba0 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f217909b700 1 -- 172.21.15.33:0/4268701981 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56247385b430 con 0x562473a8f330 2024-05-08T21:19:31.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.024+0000 7f217989c700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21600a11a0 0x7f21600a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.025+0000 7f217909b700 1 --2- 172.21.15.33:0/4268701981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 0x562473a8f7b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2170009990 tx=0x7f2170009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2c9dff2dc1e88f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.025+0000 7f215f7fe700 1 -- 172.21.15.33:0/4268701981 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217001d050 con 0x562473a8f330 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.025+0000 7f215f7fe700 1 -- 172.21.15.33:0/4268701981 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f217001e040 con 0x562473a8f330 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.025+0000 7f215f7fe700 1 -- 172.21.15.33:0/4268701981 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170003e50 con 0x562473a8f330 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.026+0000 7f21874b9b80 1 -- 172.21.15.33:0/4268701981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 msgr2=0x562473a8f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.026+0000 7f217989c700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21600a11a0 0x7f21600a3660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f216c0c9b40 tx=0x7f216c084040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.026+0000 7f21874b9b80 1 --2- 172.21.15.33:0/4268701981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 0x562473a8f7b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2170009990 tx=0x7f2170009f60 comp rx=0 tx=0).stop 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.026+0000 7f21874b9b80 1 -- 172.21.15.33:0/4268701981 shutdown_connections 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.026+0000 7f21874b9b80 1 --2- 172.21.15.33:0/4268701981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8fcf0 0x562473a90170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.026+0000 7f21874b9b80 1 --2- 172.21.15.33:0/4268701981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8e9d0 0x562473a8edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.026+0000 7f21874b9b80 1 --2- 172.21.15.33:0/4268701981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 0x562473a8f7b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.026+0000 7f21874b9b80 1 -- 172.21.15.33:0/4268701981 >> 172.21.15.33:0/4268701981 conn(0x5624739e8290 msgr2=0x562473a84e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.026+0000 7f21874b9b80 1 -- 172.21.15.33:0/4268701981 shutdown_connections 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.026+0000 7f21874b9b80 1 -- 172.21.15.33:0/4268701981 wait complete. 2024-05-08T21:19:31.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.026+0000 7f21874b9b80 1 -- start start 2024-05-08T21:19:31.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.026+0000 7f21874b9b80 1 Processor -- start 2024-05-08T21:19:31.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f21874b9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8e9d0 0x562473a981e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f21874b9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8f330 0x562473a98720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f21874b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8fcf0 0x562473a9f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f21874b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624739765e0 con 0x562473a8fcf0 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f21874b9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562473976280 con 0x562473a8e9d0 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f21874b9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247385d720 con 0x562473a8f330 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f21874b9b80 10 client.0 _ll_register_callbacks cb 0x56247397fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f217889a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8e9d0 0x562473a981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f217889a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8fcf0 0x562473a9f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f217909b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8f330 0x562473a98720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f217889a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8fcf0 0x562473a9f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46568/0 (socket says 172.21.15.33:46568) 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f217889a700 1 -- 172.21.15.33:0/3619927970 learned_addr learned my addr 172.21.15.33:0/3619927970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f217889a700 1 -- 172.21.15.33:0/3619927970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8f330 msgr2=0x562473a98720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f217889a700 1 --2- 172.21.15.33:0/3619927970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8f330 0x562473a98720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f217889a700 1 -- 172.21.15.33:0/3619927970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8e9d0 msgr2=0x562473a981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f217889a700 1 --2- 172.21.15.33:0/3619927970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8e9d0 0x562473a981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.027+0000 7f217889a700 1 -- 172.21.15.33:0/3619927970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2174109ba0 con 0x562473a8fcf0 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f217889a700 1 --2- 172.21.15.33:0/3619927970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8fcf0 0x562473a9f760 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2164012fa0 tx=0x7f21640074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491a57fb9ca472c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f215cff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f215cff9700 1 -- 172.21.15.33:0/3619927970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21640078b0 con 0x562473a8fcf0 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f215cff9700 1 -- 172.21.15.33:0/3619927970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f216400fc50 con 0x562473a8fcf0 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f21874b9b80 1 -- 172.21.15.33:0/3619927970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624739332f0 con 0x562473a8fcf0 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f215cff9700 1 -- 172.21.15.33:0/3619927970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f216400fe10 con 0x562473a8fcf0 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f21874b9b80 10 client.15804 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f21874b9b80 1 -- 172.21.15.33:0/3619927970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562473807be0 con 0x562473a8fcf0 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f215cff9700 1 -- 172.21.15.33:0/3619927970 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5624739332f0 con 0x562473a8fcf0 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f215cff9700 1 -- 172.21.15.33:0/3619927970 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f21640052c0 con 0x562473a8fcf0 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f21874b9b80 10 client.15804 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f21874b9b80 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f21874b9b80 20 client.15804 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:31.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f21874b9b80 1 --2- 172.21.15.33:0/3619927970 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562473aa4050 0x562473aa6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f21874b9b80 4 client.15804 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.028+0000 7f21874b9b80 1 -- 172.21.15.33:0/3619927970 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562473a59890 con 0x562473aa4050 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.029+0000 7f217909b700 1 --2- 172.21.15.33:0/3619927970 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562473aa4050 0x562473aa6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.029+0000 7f215cff9700 10 client.15804 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.029+0000 7f217909b700 1 --2- 172.21.15.33:0/3619927970 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562473aa4050 0x562473aa6470 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2bf858b2cb57fb8 server_cookie=b48451d36421a7fe in_seq=0 out_seq=0 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f215cff9700 1 -- 172.21.15.33:0/3619927970 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562473a59890 con 0x562473aa4050 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f215cff9700 10 client.15804 handle_command_reply: tid=0 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f21874b9b80 1 client.15804 shutdown 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f21874b9b80 1 -- 172.21.15.33:0/3619927970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8fcf0 msgr2=0x562473a9f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f21874b9b80 1 --2- 172.21.15.33:0/3619927970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8fcf0 0x562473a9f760 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2164012fa0 tx=0x7f21640074f0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f21874b9b80 1 -- 172.21.15.33:0/3619927970 shutdown_connections 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f21874b9b80 1 --2- 172.21.15.33:0/3619927970 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562473aa4050 0x562473aa6470 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f21874b9b80 1 --2- 172.21.15.33:0/3619927970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8fcf0 0x562473a9f760 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f21874b9b80 1 --2- 172.21.15.33:0/3619927970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8e9d0 0x562473a981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f215cff9700 0 client.15804 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:31.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f21874b9b80 1 --2- 172.21.15.33:0/3619927970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8f330 0x562473a98720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f21874b9b80 1 -- 172.21.15.33:0/3619927970 >> 172.21.15.33:0/3619927970 conn(0x5624739e8290 msgr2=0x562473a84620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:31.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f21874b9b80 1 -- 172.21.15.33:0/3619927970 shutdown_connections 2024-05-08T21:19:31.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f21874b9b80 1 -- 172.21.15.33:0/3619927970 wait complete. 2024-05-08T21:19:31.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.030+0000 7f21874b9b80 20 client.15804 trim_cache size 0 max 16384 2024-05-08T21:19:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f21874b9b80 1 Processor -- start 2024-05-08T21:19:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f21874b9b80 1 -- start start 2024-05-08T21:19:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f21874b9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8e9d0 0x562473a986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f21874b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 0x562473a9f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f21874b9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8fcf0 0x562473a84e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f21874b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562473a81e40 con 0x562473a8f330 2024-05-08T21:19:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f21874b9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562473a98be0 con 0x562473a8fcf0 2024-05-08T21:19:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f21874b9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562473a7f880 con 0x562473a8e9d0 2024-05-08T21:19:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f217909b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 0x562473a9f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f217909b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 0x562473a9f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46580/0 (socket says 172.21.15.33:46580) 2024-05-08T21:19:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f217909b700 1 -- 172.21.15.33:0/3352287790 learned_addr learned my addr 172.21.15.33:0/3352287790 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f217889a700 1 --2- 172.21.15.33:0/3352287790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8e9d0 0x562473a986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f217889a700 1 --2- 172.21.15.33:0/3352287790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8fcf0 0x562473a84e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f217909b700 1 -- 172.21.15.33:0/3352287790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8e9d0 msgr2=0x562473a986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f217909b700 1 --2- 172.21.15.33:0/3352287790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8e9d0 0x562473a986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f217909b700 1 -- 172.21.15.33:0/3352287790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8fcf0 msgr2=0x562473a84e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f217909b700 1 --2- 172.21.15.33:0/3352287790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8fcf0 0x562473a84e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f217909b700 1 -- 172.21.15.33:0/3352287790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624739a05e0 con 0x562473a8f330 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.049+0000 7f217889a700 1 --2- 172.21.15.33:0/3352287790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8e9d0 0x562473a986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f217909b700 1 --2- 172.21.15.33:0/3352287790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 0x562473a9f680 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f21700098e0 tx=0x7f2170009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d9956bf05d2a0a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f215ffff700 1 -- 172.21.15.33:0/3352287790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217001d040 con 0x562473a8f330 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f215ffff700 1 -- 172.21.15.33:0/3352287790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f217000b600 con 0x562473a8f330 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f215ffff700 1 -- 172.21.15.33:0/3352287790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170005540 con 0x562473a8f330 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f21874b9b80 1 -- 172.21.15.33:0/3352287790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 msgr2=0x562473a9f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f21874b9b80 1 --2- 172.21.15.33:0/3352287790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 0x562473a9f680 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f21700098e0 tx=0x7f2170009310 comp rx=0 tx=0).stop 2024-05-08T21:19:31.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f21874b9b80 1 -- 172.21.15.33:0/3352287790 shutdown_connections 2024-05-08T21:19:31.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f21874b9b80 1 --2- 172.21.15.33:0/3352287790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8fcf0 0x562473a84e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f21874b9b80 1 --2- 172.21.15.33:0/3352287790 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8e9d0 0x562473a986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f21874b9b80 1 --2- 172.21.15.33:0/3352287790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8f330 0x562473a9f680 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f21874b9b80 1 -- 172.21.15.33:0/3352287790 >> 172.21.15.33:0/3352287790 conn(0x5624739e8290 msgr2=0x562473a97f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:31.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f21874b9b80 1 -- 172.21.15.33:0/3352287790 shutdown_connections 2024-05-08T21:19:31.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.050+0000 7f21874b9b80 1 -- 172.21.15.33:0/3352287790 wait complete. 2024-05-08T21:19:31.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.051+0000 7f21874b9b80 1 -- start start 2024-05-08T21:19:31.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.051+0000 7f21874b9b80 1 Processor -- start 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.051+0000 7f21874b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8e9d0 0x562473a9f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f21874b9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8f330 0x562473a9c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f21874b9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8fcf0 0x562473a9c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f21874b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562473afa540 con 0x562473a8e9d0 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f21874b9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562473afa680 con 0x562473a8f330 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f21874b9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562473afa7f0 con 0x562473a8fcf0 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f21874b9b80 10 client.0 _ll_register_callbacks cb 0x56247397fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f217889a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8e9d0 0x562473a9f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f217889a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8e9d0 0x562473a9f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46590/0 (socket says 172.21.15.33:46590) 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f217889a700 1 -- 172.21.15.33:0/1650006189 learned_addr learned my addr 172.21.15.33:0/1650006189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f217909b700 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8f330 0x562473a9c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f217889a700 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8fcf0 0x562473a9c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f217889a700 1 -- 172.21.15.33:0/1650006189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8fcf0 msgr2=0x562473a9c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f217889a700 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8fcf0 0x562473a9c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f217889a700 1 -- 172.21.15.33:0/1650006189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8f330 msgr2=0x562473a9c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f217889a700 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8f330 0x562473a9c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f217889a700 1 -- 172.21.15.33:0/1650006189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56247385b430 con 0x562473a8e9d0 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f217889a700 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8e9d0 0x562473a9f690 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2164001eb0 tx=0x7f21640074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8697407c4f22ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f215cff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f215cff9700 1 -- 172.21.15.33:0/1650006189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21640078b0 con 0x562473a8e9d0 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f215cff9700 1 -- 172.21.15.33:0/1650006189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f2164000e40 con 0x562473a8e9d0 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.052+0000 7f21874b9b80 1 -- 172.21.15.33:0/1650006189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f217001e040 con 0x562473a8e9d0 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.053+0000 7f215cff9700 1 -- 172.21.15.33:0/1650006189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2164011440 con 0x562473a8e9d0 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.053+0000 7f21874b9b80 10 client.15810 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.053+0000 7f21874b9b80 1 -- 172.21.15.33:0/1650006189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56247385fd50 con 0x562473a8e9d0 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.053+0000 7f215cff9700 1 -- 172.21.15.33:0/1650006189 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f217001e040 con 0x562473a8e9d0 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.053+0000 7f215cff9700 1 -- 172.21.15.33:0/1650006189 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2164011cd0 con 0x562473a8e9d0 2024-05-08T21:19:31.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.053+0000 7f21874b9b80 10 client.15810 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:31.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.053+0000 7f21874b9b80 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:19:31.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.053+0000 7f21874b9b80 20 client.15810 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:31.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.053+0000 7f21874b9b80 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562473afc010 0x562473afe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.053+0000 7f21874b9b80 4 client.15810 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:19:31.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.053+0000 7f21874b9b80 1 -- 172.21.15.33:0/1650006189 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5624739d2f40 con 0x562473afc010 2024-05-08T21:19:31.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.053+0000 7f217909b700 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562473afc010 0x562473afe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.054+0000 7f217909b700 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562473afc010 0x562473afe430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bce7cbdf3871029 server_cookie=8498effa9704e177 in_seq=0 out_seq=0 2024-05-08T21:19:31.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.054+0000 7f215cff9700 10 client.15810 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f215cff9700 1 -- 172.21.15.33:0/1650006189 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5624739d2f40 con 0x562473afc010 2024-05-08T21:19:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f215cff9700 10 client.15810 handle_command_reply: tid=0 2024-05-08T21:19:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f21874b9b80 1 client.15810 shutdown 2024-05-08T21:19:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f21874b9b80 1 -- 172.21.15.33:0/1650006189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8e9d0 msgr2=0x562473a9f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f21874b9b80 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8e9d0 0x562473a9f690 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2164001eb0 tx=0x7f21640074f0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f21874b9b80 1 -- 172.21.15.33:0/1650006189 shutdown_connections 2024-05-08T21:19:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f21874b9b80 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562473afc010 0x562473afe430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f21874b9b80 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562473a8fcf0 0x562473a9c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f21874b9b80 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562473a8e9d0 0x562473a9f690 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f21874b9b80 1 --2- 172.21.15.33:0/1650006189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562473a8f330 0x562473a9c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f21874b9b80 1 -- 172.21.15.33:0/1650006189 >> 172.21.15.33:0/1650006189 conn(0x5624739e8290 msgr2=0x562473a8d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f215cff9700 0 client.15810 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f21874b9b80 1 -- 172.21.15.33:0/1650006189 shutdown_connections 2024-05-08T21:19:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f21874b9b80 1 -- 172.21.15.33:0/1650006189 wait complete. 2024-05-08T21:19:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.055+0000 7f21874b9b80 20 client.15810 trim_cache size 0 max 16384 2024-05-08T21:19:31.063 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:19:31.063 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:19:31.063 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "faec4963-49bd-4375-acc9-3d0c24fb2927", 2024-05-08T21:19:31.063 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:19:31.063 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:19:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.056+0000 7f217b2ff700 1 -- 172.21.15.33:0/3498595542 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21600a11a0 msgr2=0x7f21600a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.056+0000 7f217b2ff700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21600a11a0 0x7f21600a3660 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f216c0c9b40 tx=0x7f216c084040 comp rx=0 tx=0).stop 2024-05-08T21:19:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.056+0000 7f217b2ff700 1 -- 172.21.15.33:0/3498595542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2174106ba0 msgr2=0x7f217419e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.056+0000 7f217b2ff700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2174106ba0 0x7f217419e210 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f216c007ee0 tx=0x7f216c00d3b0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.056+0000 7f217b2ff700 1 -- 172.21.15.33:0/3498595542 shutdown_connections 2024-05-08T21:19:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.056+0000 7f217b2ff700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2174106ba0 0x7f217419e210 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.056+0000 7f217b2ff700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21600a11a0 0x7f21600a3660 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.056+0000 7f217b2ff700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2174101940 0x7f2174196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.056+0000 7f217b2ff700 1 --2- 172.21.15.33:0/3498595542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2174104270 0x7f2174197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.056+0000 7f217b2ff700 1 -- 172.21.15.33:0/3498595542 >> 172.21.15.33:0/3498595542 conn(0x7f21740fb290 msgr2=0x7f21741023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.056+0000 7f217b2ff700 1 -- 172.21.15.33:0/3498595542 shutdown_connections 2024-05-08T21:19:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.056+0000 7f217b2ff700 1 -- 172.21.15.33:0/3498595542 wait complete. 2024-05-08T21:19:31.072 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:19:31.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.209+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/3649341358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fb4100e70 msgr2=0x7f9fb4101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.209+0000 7f9fbc5c3700 1 --2- 172.21.15.33:0/3649341358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fb4100e70 0x7f9fb4101290 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009990 tx=0x7f9fa8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.209+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/3649341358 shutdown_connections 2024-05-08T21:19:31.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.209+0000 7f9fbc5c3700 1 --2- 172.21.15.33:0/3649341358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fb4102980 0x7f9fb41092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.209+0000 7f9fbc5c3700 1 --2- 172.21.15.33:0/3649341358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fb4100e70 0x7f9fb4101290 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.209+0000 7f9fbc5c3700 1 --2- 172.21.15.33:0/3649341358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fb4101fc0 0x7f9fb4102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.209+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/3649341358 >> 172.21.15.33:0/3649341358 conn(0x7f9fb40fc400 msgr2=0x7f9fb40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:31.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.209+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/3649341358 shutdown_connections 2024-05-08T21:19:31.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.209+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/3649341358 wait complete. 2024-05-08T21:19:31.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fbc5c3700 1 Processor -- start 2024-05-08T21:19:31.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fbc5c3700 1 -- start start 2024-05-08T21:19:31.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fbc5c3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fb4100e70 0x7f9fb4074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fbc5c3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fb4101fc0 0x7f9fb4073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fbc5c3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fb4102980 0x7f9fb40738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fbc5c3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb4074070 con 0x7f9fb4101fc0 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fbc5c3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40741e0 con 0x7f9fb4102980 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fbc5c3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb41a0d60 con 0x7f9fb4100e70 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fb9b5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fb4101fc0 0x7f9fb4073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fb9b5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fb4101fc0 0x7f9fb4073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46608/0 (socket says 172.21.15.33:46608) 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fb9b5e700 1 -- 172.21.15.33:0/1310178831 learned_addr learned my addr 172.21.15.33:0/1310178831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fbab60700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fb4102980 0x7f9fb40738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.210+0000 7f9fba35f700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fb4100e70 0x7f9fb4074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.211+0000 7f9fb9b5e700 1 -- 172.21.15.33:0/1310178831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fb4100e70 msgr2=0x7f9fb4074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.211+0000 7f9fb9b5e700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fb4100e70 0x7f9fb4074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.211+0000 7f9fb9b5e700 1 -- 172.21.15.33:0/1310178831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fb4102980 msgr2=0x7f9fb40738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.211+0000 7f9fb9b5e700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fb4102980 0x7f9fb40738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.211+0000 7f9fb9b5e700 1 -- 172.21.15.33:0/1310178831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa80095f0 con 0x7f9fb4101fc0 2024-05-08T21:19:31.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.211+0000 7f9fb9b5e700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fb4101fc0 0x7f9fb4073390 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000e9a0 tx=0x7f9fb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:31.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.211+0000 7f9fa77fe700 1 -- 172.21.15.33:0/1310178831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0009790 con 0x7f9fb4101fc0 2024-05-08T21:19:31.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.211+0000 7f9fa77fe700 1 -- 172.21.15.33:0/1310178831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fb000ce40 con 0x7f9fb4101fc0 2024-05-08T21:19:31.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.211+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/1310178831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb41a0ff0 con 0x7f9fb4101fc0 2024-05-08T21:19:31.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.211+0000 7f9fa77fe700 1 -- 172.21.15.33:0/1310178831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0018420 con 0x7f9fb4101fc0 2024-05-08T21:19:31.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.211+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/1310178831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb41a14c0 con 0x7f9fb4101fc0 2024-05-08T21:19:31.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.213+0000 7f9fa77fe700 1 -- 172.21.15.33:0/1310178831 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9fb0010430 con 0x7f9fb4101fc0 2024-05-08T21:19:31.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.214+0000 7f9fa77fe700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9fa00a11a0 0x7f9fa00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.214+0000 7f9fa77fe700 1 -- 172.21.15.33:0/1310178831 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9fb0014070 con 0x7f9fb4101fc0 2024-05-08T21:19:31.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.214+0000 7f9fba35f700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9fa00a11a0 0x7f9fa00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.215+0000 7f9fba35f700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9fa00a11a0 0x7f9fa00a3660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8006010 tx=0x7f9fa800b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:31.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.215+0000 7f9fc877db80 1 Processor -- start 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.215+0000 7f9fc877db80 1 -- start start 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.215+0000 7f9fc877db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb8158d50 0x564bb8159170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.215+0000 7f9fc877db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb81596b0 0x564bb8159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.215+0000 7f9fc877db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb815a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.215+0000 7f9fc877db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bb80a7150 con 0x564bb8158d50 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.215+0000 7f9fc877db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bb80a4b10 con 0x564bb81596b0 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.215+0000 7f9fc877db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bb80a0400 con 0x564bb815a070 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.215+0000 7f9fba35f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb8158d50 0x564bb8159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fba35f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb8158d50 0x564bb8159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46620/0 (socket says 172.21.15.33:46620) 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fba35f700 1 -- 172.21.15.33:0/2567685515 learned_addr learned my addr 172.21.15.33:0/2567685515 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fbab60700 1 --2- 172.21.15.33:0/2567685515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb815a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fb9b5e700 1 --2- 172.21.15.33:0/2567685515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb81596b0 0x564bb8159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fba35f700 1 -- 172.21.15.33:0/2567685515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 msgr2=0x564bb815a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fba35f700 1 --2- 172.21.15.33:0/2567685515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb815a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fba35f700 1 -- 172.21.15.33:0/2567685515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb81596b0 msgr2=0x564bb8159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fba35f700 1 --2- 172.21.15.33:0/2567685515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb81596b0 0x564bb8159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fba35f700 1 -- 172.21.15.33:0/2567685515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bb7f213b0 con 0x564bb8158d50 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fbab60700 1 --2- 172.21.15.33:0/2567685515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb815a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fba35f700 1 --2- 172.21.15.33:0/2567685515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb8158d50 0x564bb8159170 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8015000 tx=0x7f9fa8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481dc32d137bc294 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fa4ff9700 1 -- 172.21.15.33:0/2567685515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8003c90 con 0x564bb8158d50 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fa4ff9700 1 -- 172.21.15.33:0/2567685515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fa801e040 con 0x564bb8158d50 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.216+0000 7f9fa4ff9700 1 -- 172.21.15.33:0/2567685515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8004820 con 0x564bb8158d50 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.217+0000 7f9fc877db80 1 -- 172.21.15.33:0/2567685515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb8158d50 msgr2=0x564bb8159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.217+0000 7f9fc877db80 1 --2- 172.21.15.33:0/2567685515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb8158d50 0x564bb8159170 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8015000 tx=0x7f9fa8020000 comp rx=0 tx=0).stop 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.217+0000 7f9fc877db80 1 -- 172.21.15.33:0/2567685515 shutdown_connections 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.217+0000 7f9fc877db80 1 --2- 172.21.15.33:0/2567685515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb815a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.217+0000 7f9fc877db80 1 --2- 172.21.15.33:0/2567685515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb8158d50 0x564bb8159170 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.217+0000 7f9fc877db80 1 --2- 172.21.15.33:0/2567685515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb81596b0 0x564bb8159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.217+0000 7f9fc877db80 1 -- 172.21.15.33:0/2567685515 >> 172.21.15.33:0/2567685515 conn(0x564bb80ae210 msgr2=0x564bb814ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.217+0000 7f9fc877db80 1 -- 172.21.15.33:0/2567685515 shutdown_connections 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.217+0000 7f9fc877db80 1 -- 172.21.15.33:0/2567685515 wait complete. 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fc877db80 1 -- start start 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fc877db80 1 Processor -- start 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fc877db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb81625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fc877db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 0x564bb8162b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fc877db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb8169b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fc877db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bb803c560 con 0x564bb81596b0 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fc877db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bb803c200 con 0x564bb8158d50 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fc877db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bb7f236a0 con 0x564bb815a070 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fc877db80 10 client.0 _ll_register_callbacks cb 0x564bb8045b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:31.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fb9b5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 0x564bb8162b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fb9b5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 0x564bb8162b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46630/0 (socket says 172.21.15.33:46630) 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fb9b5e700 1 -- 172.21.15.33:0/598332568 learned_addr learned my addr 172.21.15.33:0/598332568 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fba35f700 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb81625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fbab60700 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb8169b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fb9b5e700 1 -- 172.21.15.33:0/598332568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 msgr2=0x564bb8169b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fb9b5e700 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb8169b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fb9b5e700 1 -- 172.21.15.33:0/598332568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 msgr2=0x564bb81625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fb9b5e700 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb81625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.218+0000 7f9fb9b5e700 1 -- 172.21.15.33:0/598332568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb4109bd0 con 0x564bb81596b0 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.219+0000 7f9fba35f700 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb81625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.219+0000 7f9fb9b5e700 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 0x564bb8162b20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0099d50 tx=0x7f9fb009a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf83a8245b3cbb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.219+0000 7f9f9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.219+0000 7f9f9dffb700 1 -- 172.21.15.33:0/598332568 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00a1fb0 con 0x564bb81596b0 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.219+0000 7f9f9dffb700 1 -- 172.21.15.33:0/598332568 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9fb009dbf0 con 0x564bb81596b0 2024-05-08T21:19:31.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.219+0000 7f9fc877db80 1 -- 172.21.15.33:0/598332568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bb7ff9270 con 0x564bb81596b0 2024-05-08T21:19:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.224+0000 7f9f9dffb700 1 -- 172.21.15.33:0/598332568 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fb00b8170 con 0x564bb81596b0 2024-05-08T21:19:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.224+0000 7f9fc877db80 10 client.15822 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.224+0000 7f9fc877db80 1 -- 172.21.15.33:0/598332568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564bb7ecdb60 con 0x564bb81596b0 2024-05-08T21:19:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.224+0000 7f9f9dffb700 1 -- 172.21.15.33:0/598332568 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fb007c080 con 0x564bb81596b0 2024-05-08T21:19:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.224+0000 7f9f9dffb700 1 -- 172.21.15.33:0/598332568 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9fb0098d20 con 0x564bb81596b0 2024-05-08T21:19:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.224+0000 7f9fc877db80 10 client.15822 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.224+0000 7f9fc877db80 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.224+0000 7f9fc877db80 20 client.15822 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.224+0000 7f9fc877db80 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564bb816e320 0x564bb8170740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.224+0000 7f9fc877db80 4 client.15822 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.224+0000 7f9fc877db80 1 -- 172.21.15.33:0/598332568 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bb811f810 con 0x564bb816e320 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.225+0000 7f9fba35f700 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564bb816e320 0x564bb8170740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.225+0000 7f9f9dffb700 10 client.15822 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.225+0000 7f9fba35f700 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564bb816e320 0x564bb8170740 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d74d5280cda198c server_cookie=19a51001e0e80289 in_seq=0 out_seq=0 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9f9dffb700 1 -- 172.21.15.33:0/598332568 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564bb811f810 con 0x564bb816e320 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9f9dffb700 10 client.15822 handle_command_reply: tid=0 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9fc877db80 1 client.15822 shutdown 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9fc877db80 1 -- 172.21.15.33:0/598332568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 msgr2=0x564bb8162b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9fc877db80 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 0x564bb8162b20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0099d50 tx=0x7f9fb009a300 comp rx=0 tx=0).stop 2024-05-08T21:19:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9fc877db80 1 -- 172.21.15.33:0/598332568 shutdown_connections 2024-05-08T21:19:31.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9fc877db80 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564bb816e320 0x564bb8170740 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9fc877db80 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb8169b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9fc877db80 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb81625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9f9dffb700 0 client.15822 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:31.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9fc877db80 1 --2- 172.21.15.33:0/598332568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 0x564bb8162b20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9fc877db80 1 -- 172.21.15.33:0/598332568 >> 172.21.15.33:0/598332568 conn(0x564bb80ae210 msgr2=0x564bb814a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:31.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9fc877db80 1 -- 172.21.15.33:0/598332568 shutdown_connections 2024-05-08T21:19:31.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9fc877db80 1 -- 172.21.15.33:0/598332568 wait complete. 2024-05-08T21:19:31.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.226+0000 7f9fc877db80 20 client.15822 trim_cache size 0 max 16384 2024-05-08T21:19:31.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.244+0000 7f9fc877db80 1 Processor -- start 2024-05-08T21:19:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.244+0000 7f9fc877db80 1 -- start start 2024-05-08T21:19:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.244+0000 7f9fc877db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb8161af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.244+0000 7f9fc877db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 0x564bb8169b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.244+0000 7f9fc877db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb8160620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.244+0000 7f9fc877db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bb814c830 con 0x564bb81596b0 2024-05-08T21:19:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.244+0000 7f9fc877db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bb8145800 con 0x564bb8158d50 2024-05-08T21:19:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.244+0000 7f9fc877db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bb816e120 con 0x564bb815a070 2024-05-08T21:19:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fb9b5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 0x564bb8169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fb9b5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 0x564bb8169b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46638/0 (socket says 172.21.15.33:46638) 2024-05-08T21:19:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fb9b5e700 1 -- 172.21.15.33:0/1381374662 learned_addr learned my addr 172.21.15.33:0/1381374662 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fba35f700 1 --2- 172.21.15.33:0/1381374662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb8161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fbab60700 1 --2- 172.21.15.33:0/1381374662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb8160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fb9b5e700 1 -- 172.21.15.33:0/1381374662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 msgr2=0x564bb8160620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fb9b5e700 1 --2- 172.21.15.33:0/1381374662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb8160620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fb9b5e700 1 -- 172.21.15.33:0/1381374662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 msgr2=0x564bb8161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fb9b5e700 1 --2- 172.21.15.33:0/1381374662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb8161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fb9b5e700 1 -- 172.21.15.33:0/1381374662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bb8066560 con 0x564bb81596b0 2024-05-08T21:19:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fba35f700 1 --2- 172.21.15.33:0/1381374662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb8161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:19:31.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fb9b5e700 1 --2- 172.21.15.33:0/1381374662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 0x564bb8169b20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0000c00 tx=0x7f9fb0098370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea77485a5084baa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fa57fa700 1 -- 172.21.15.33:0/1381374662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00b8170 con 0x564bb81596b0 2024-05-08T21:19:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fa57fa700 1 -- 172.21.15.33:0/1381374662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564bb7ff9270 con 0x564bb81596b0 2024-05-08T21:19:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.245+0000 7f9fa57fa700 1 -- 172.21.15.33:0/1381374662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00a1db0 con 0x564bb81596b0 2024-05-08T21:19:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.246+0000 7f9fc877db80 1 -- 172.21.15.33:0/1381374662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 msgr2=0x564bb8169b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.246+0000 7f9fc877db80 1 --2- 172.21.15.33:0/1381374662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 0x564bb8169b20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0000c00 tx=0x7f9fb0098370 comp rx=0 tx=0).stop 2024-05-08T21:19:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.246+0000 7f9fc877db80 1 -- 172.21.15.33:0/1381374662 shutdown_connections 2024-05-08T21:19:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.246+0000 7f9fc877db80 1 --2- 172.21.15.33:0/1381374662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb815a070 0x564bb8160620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.246+0000 7f9fc877db80 1 --2- 172.21.15.33:0/1381374662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb8161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.246+0000 7f9fc877db80 1 --2- 172.21.15.33:0/1381374662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb81596b0 0x564bb8169b20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.246+0000 7f9fc877db80 1 -- 172.21.15.33:0/1381374662 >> 172.21.15.33:0/1381374662 conn(0x564bb80ae210 msgr2=0x564bb815a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.246+0000 7f9fc877db80 1 -- 172.21.15.33:0/1381374662 shutdown_connections 2024-05-08T21:19:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.246+0000 7f9fc877db80 1 -- 172.21.15.33:0/1381374662 wait complete. 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.246+0000 7f9fc877db80 1 -- start start 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fc877db80 1 Processor -- start 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fc877db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb81658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fc877db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb81596b0 0x564bb8165df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fc877db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb815a070 0x564bb8168dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fc877db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bb8169300 con 0x564bb815a070 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fc877db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bb8169470 con 0x564bb8158d50 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fc877db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bb8169630 con 0x564bb81596b0 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fc877db80 10 client.0 _ll_register_callbacks cb 0x564bb8045b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fbab60700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb815a070 0x564bb8168dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fbab60700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb815a070 0x564bb8168dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46644/0 (socket says 172.21.15.33:46644) 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fbab60700 1 -- 172.21.15.33:0/2091843662 learned_addr learned my addr 172.21.15.33:0/2091843662 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fb9b5e700 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb81596b0 0x564bb8165df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fba35f700 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb81658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fbab60700 1 -- 172.21.15.33:0/2091843662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb81596b0 msgr2=0x564bb8165df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fbab60700 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb81596b0 0x564bb8165df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fbab60700 1 -- 172.21.15.33:0/2091843662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 msgr2=0x564bb81658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fbab60700 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb81658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fbab60700 1 -- 172.21.15.33:0/2091843662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bb7f213b0 con 0x564bb815a070 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.247+0000 7f9fb9b5e700 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb81596b0 0x564bb8165df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9fbab60700 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb815a070 0x564bb8168dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9fac007c40 tx=0x7f9fac009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a468f9b93e261a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9f9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9f9dffb700 1 -- 172.21.15.33:0/2091843662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac00e040 con 0x564bb815a070 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9f9dffb700 1 -- 172.21.15.33:0/2091843662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9fac003e10 con 0x564bb815a070 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9fc877db80 1 -- 172.21.15.33:0/2091843662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fa801e040 con 0x564bb815a070 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9f9dffb700 1 -- 172.21.15.33:0/2091843662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fac004050 con 0x564bb815a070 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9fc877db80 10 client.15828 fetch_fsmap learned FSMap version 29 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9fc877db80 1 -- 172.21.15.33:0/2091843662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x564bb7ecdb60 con 0x564bb815a070 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9f9dffb700 1 -- 172.21.15.33:0/2091843662 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fa801e040 con 0x564bb815a070 2024-05-08T21:19:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9f9dffb700 1 -- 172.21.15.33:0/2091843662 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9fac017a40 con 0x564bb815a070 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9fc877db80 10 client.15828 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9fc877db80 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9fc877db80 20 client.15828 populate_metadata read hostname 'smithi033' 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9fc877db80 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564bb81c58a0 0x564bb81c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9fc877db80 4 client.15828 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.248+0000 7f9fc877db80 1 -- 172.21.15.33:0/2091843662 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564bb8098ec0 con 0x564bb81c58a0 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9fba35f700 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564bb81c58a0 0x564bb81c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9fba35f700 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564bb81c58a0 0x564bb81c7c90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9da2a0cdc979a96 server_cookie=58d658bbdd5f3c11 in_seq=0 out_seq=0 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9f9dffb700 10 client.15828 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9f9dffb700 1 -- 172.21.15.33:0/2091843662 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564bb8098ec0 con 0x564bb81c58a0 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9f9dffb700 10 client.15828 handle_command_reply: tid=0 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9fc877db80 1 client.15828 shutdown 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9fc877db80 1 -- 172.21.15.33:0/2091843662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb815a070 msgr2=0x564bb8168dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9fc877db80 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb815a070 0x564bb8168dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9fac007c40 tx=0x7f9fac009510 comp rx=0 tx=0).stop 2024-05-08T21:19:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9fc877db80 1 -- 172.21.15.33:0/2091843662 shutdown_connections 2024-05-08T21:19:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9fc877db80 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564bb81c58a0 0x564bb81c7c90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9fc877db80 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564bb815a070 0x564bb8168dc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9fc877db80 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bb8158d50 0x564bb81658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9f9dffb700 0 client.15828 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:19:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9fc877db80 1 --2- 172.21.15.33:0/2091843662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564bb81596b0 0x564bb8165df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9fc877db80 1 -- 172.21.15.33:0/2091843662 >> 172.21.15.33:0/2091843662 conn(0x564bb80ae210 msgr2=0x564bb81605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.249+0000 7f9fc877db80 1 -- 172.21.15.33:0/2091843662 shutdown_connections 2024-05-08T21:19:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fc877db80 1 -- 172.21.15.33:0/2091843662 wait complete. 2024-05-08T21:19:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fc877db80 20 client.15828 trim_cache size 0 max 16384 2024-05-08T21:19:31.260 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:19:31.261 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:19:31.261 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:19:31.261 INFO:teuthology.orchestra.run.smithi033.stdout: "faec4963-49bd-4375-acc9-3d0c24fb2927": { 2024-05-08T21:19:31.261 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:19:31.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "faec4963-49bd-4375-acc9-3d0c24fb2927", 2024-05-08T21:19:31.261 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:19:31.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:19:31.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:19:31.261 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:19:31.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/1310178831 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9fa00a11a0 msgr2=0x7f9fa00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fbc5c3700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9fa00a11a0 0x7f9fa00a3660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8006010 tx=0x7f9fa800b5d0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/1310178831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fb4101fc0 msgr2=0x7f9fb4073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:19:31.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fbc5c3700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fb4101fc0 0x7f9fb4073390 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000e9a0 tx=0x7f9fb000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/1310178831 shutdown_connections 2024-05-08T21:19:31.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fbc5c3700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9fa00a11a0 0x7f9fa00a3660 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fbc5c3700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fb4102980 0x7f9fb40738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fbc5c3700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fb4100e70 0x7f9fb4074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fbc5c3700 1 --2- 172.21.15.33:0/1310178831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fb4101fc0 0x7f9fb4073390 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:19:31.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/1310178831 >> 172.21.15.33:0/1310178831 conn(0x7f9fb40fc400 msgr2=0x7f9fb4105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:19:31.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/1310178831 shutdown_connections 2024-05-08T21:19:31.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:19:31.250+0000 7f9fbc5c3700 1 -- 172.21.15.33:0/1310178831 wait complete. 2024-05-08T21:19:31.266 INFO:tasks.cephfs.filesystem:scrub status for tag:faec4963-49bd-4375-acc9-3d0c24fb2927 - {'path': '/', 'tag': 'faec4963-49bd-4375-acc9-3d0c24fb2927', 'options': 'recursive,force'} 2024-05-08T21:19:31.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:31 smithi071 ceph-mon[47074]: pgmap v306: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:19:31.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:31 smithi082 ceph-mon[47512]: pgmap v306: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:19:31.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:31 smithi033 ceph-mon[42528]: pgmap v306: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:19:32.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:32 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:19:32.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:32 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:19:32.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:32 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:19:32.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:32 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:19:32.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:32 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:19:32.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:32 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:19:32.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:32 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:19:32.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:32 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:19:32.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:32 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:19:32.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:32 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:19:32.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:32 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:19:32.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:32 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:19:33.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:33 smithi071 ceph-mon[47074]: pgmap v307: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-05-08T21:19:33.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:33 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:19:33.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:33 smithi082 ceph-mon[47512]: pgmap v307: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-05-08T21:19:33.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:33 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:19:33.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:33 smithi033 ceph-mon[42528]: pgmap v307: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-05-08T21:19:33.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:33 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:19:35.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:35 smithi071 ceph-mon[47074]: pgmap v308: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:19:35.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:35 smithi082 ceph-mon[47512]: pgmap v308: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:19:35.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:35 smithi033 ceph-mon[42528]: pgmap v308: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:19:37.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:37 smithi071 ceph-mon[47074]: pgmap v309: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:19:37.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:37 smithi082 ceph-mon[47512]: pgmap v309: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:19:37.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:37 smithi033 ceph-mon[42528]: pgmap v309: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:19:39.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:39 smithi071 ceph-mon[47074]: pgmap v310: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-08T21:19:39.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:39 smithi082 ceph-mon[47512]: pgmap v310: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-08T21:19:39.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:39 smithi033 ceph-mon[42528]: pgmap v310: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-08T21:19:41.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:41 smithi071 ceph-mon[47074]: pgmap v311: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-08T21:19:41.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:41 smithi082 ceph-mon[47512]: pgmap v311: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-08T21:19:41.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:41 smithi033 ceph-mon[42528]: pgmap v311: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-08T21:19:43.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:43 smithi071 ceph-mon[47074]: pgmap v312: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-08T21:19:44.357 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:43 smithi033 ceph-mon[42528]: pgmap v312: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-08T21:19:44.357 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:43 smithi082 ceph-mon[47512]: pgmap v312: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-08T21:19:45.575 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:45 smithi071 ceph-mon[47074]: pgmap v313: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:19:45.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:45 smithi082 ceph-mon[47512]: pgmap v313: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:19:45.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:45 smithi033 ceph-mon[42528]: pgmap v313: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:19:47.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:47 smithi071 ceph-mon[47074]: pgmap v314: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:19:47.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:47 smithi082 ceph-mon[47512]: pgmap v314: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:19:47.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:47 smithi033 ceph-mon[42528]: pgmap v314: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:19:49.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:49 smithi071 ceph-mon[47074]: pgmap v315: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:19:49.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:49 smithi082 ceph-mon[47512]: pgmap v315: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:19:49.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:49 smithi033 ceph-mon[42528]: pgmap v315: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:19:51.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:51 smithi071 ceph-mon[47074]: pgmap v316: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:19:51.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:51 smithi082 ceph-mon[47512]: pgmap v316: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:19:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:51 smithi033 ceph-mon[42528]: pgmap v316: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:19:53.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:53 smithi071 ceph-mon[47074]: pgmap v317: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:19:53.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:53 smithi082 ceph-mon[47512]: pgmap v317: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:19:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:53 smithi033 ceph-mon[42528]: pgmap v317: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:19:55.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:55 smithi071 ceph-mon[47074]: pgmap v318: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:19:55.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:55 smithi082 ceph-mon[47512]: pgmap v318: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:19:55.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:55 smithi033 ceph-mon[42528]: pgmap v318: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:19:57.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:57 smithi071 ceph-mon[47074]: pgmap v319: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:19:57.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:57 smithi082 ceph-mon[47512]: pgmap v319: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:19:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:57 smithi033 ceph-mon[42528]: pgmap v319: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:19:59.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:19:59 smithi071 ceph-mon[47074]: pgmap v320: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:19:59.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:19:59 smithi082 ceph-mon[47512]: pgmap v320: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:19:59.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:19:59 smithi033 ceph-mon[42528]: pgmap v320: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:20:00.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:00 smithi071 ceph-mon[47074]: overall HEALTH_OK 2024-05-08T21:20:00.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:00 smithi082 ceph-mon[47512]: overall HEALTH_OK 2024-05-08T21:20:00.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:00 smithi033 ceph-mon[42528]: overall HEALTH_OK 2024-05-08T21:20:01.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:20:01.321 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:01 smithi071 ceph-mon[47074]: pgmap v321: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:20:01.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.442+0000 7fc18b872700 1 -- 172.21.15.33:0/3913554791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc184100090 msgr2=0x7fc184100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:01.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.442+0000 7fc18b872700 1 --2- 172.21.15.33:0/3913554791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc184100090 0x7fc184100510 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc1740099c0 tx=0x7fc174009cd0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.442+0000 7fc18b872700 1 -- 172.21.15.33:0/3913554791 shutdown_connections 2024-05-08T21:20:01.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.442+0000 7fc18b872700 1 --2- 172.21.15.33:0/3913554791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc184100a50 0x7fc184100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.442+0000 7fc18b872700 1 --2- 172.21.15.33:0/3913554791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1840ff730 0x7fc1840ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.442+0000 7fc18b872700 1 --2- 172.21.15.33:0/3913554791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc184100090 0x7fc184100510 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.442+0000 7fc18b872700 1 -- 172.21.15.33:0/3913554791 >> 172.21.15.33:0/3913554791 conn(0x7fc1840fb290 msgr2=0x7fc1840fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:01.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.442+0000 7fc18b872700 1 -- 172.21.15.33:0/3913554791 shutdown_connections 2024-05-08T21:20:01.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.442+0000 7fc18b872700 1 -- 172.21.15.33:0/3913554791 wait complete. 2024-05-08T21:20:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.443+0000 7fc18b872700 1 Processor -- start 2024-05-08T21:20:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.443+0000 7fc18b872700 1 -- start start 2024-05-08T21:20:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.443+0000 7fc18b872700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1840ff730 0x7fc184198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.443+0000 7fc18b872700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc184100090 0x7fc1841993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.443+0000 7fc18b872700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc184100a50 0x7fc1841a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.443+0000 7fc18b872700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1841a09d0 con 0x7fc1840ff730 2024-05-08T21:20:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.443+0000 7fc18b872700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1841a0b10 con 0x7fc184100a50 2024-05-08T21:20:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.443+0000 7fc18b872700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1841a0ca0 con 0x7fc184100090 2024-05-08T21:20:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.443+0000 7fc189e0f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc184100a50 0x7fc1841a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc189e0f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc184100a50 0x7fc1841a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:34676/0 (socket says 172.21.15.33:34676) 2024-05-08T21:20:01.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc189e0f700 1 -- 172.21.15.33:0/1105213107 learned_addr learned my addr 172.21.15.33:0/1105213107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc18960e700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1840ff730 0x7fc184198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc189e0f700 1 -- 172.21.15.33:0/1105213107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc184100090 msgr2=0x7fc1841993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc189e0f700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc184100090 0x7fc1841993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc189e0f700 1 -- 172.21.15.33:0/1105213107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1840ff730 msgr2=0x7fc184198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc189e0f700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1840ff730 0x7fc184198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc189e0f700 1 -- 172.21.15.33:0/1105213107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1740095f0 con 0x7fc184100a50 2024-05-08T21:20:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc18960e700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1840ff730 0x7fc184198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc189e0f700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc184100a50 0x7fc1841a0400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc17c007d70 tx=0x7fc17c00a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc17a7fc700 1 -- 172.21.15.33:0/1105213107 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00ad40 con 0x7fc184100a50 2024-05-08T21:20:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc17a7fc700 1 -- 172.21.15.33:0/1105213107 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc17c0128b0 con 0x7fc184100a50 2024-05-08T21:20:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc18b872700 1 -- 172.21.15.33:0/1105213107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1841a0f80 con 0x7fc184100a50 2024-05-08T21:20:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.444+0000 7fc17a7fc700 1 -- 172.21.15.33:0/1105213107 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0116b0 con 0x7fc184100a50 2024-05-08T21:20:01.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.445+0000 7fc18b872700 1 -- 172.21.15.33:0/1105213107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1841a13b0 con 0x7fc184100a50 2024-05-08T21:20:01.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.447+0000 7fc17a7fc700 1 -- 172.21.15.33:0/1105213107 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc17c012a20 con 0x7fc184100a50 2024-05-08T21:20:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.448+0000 7fc17a7fc700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1700a1120 0x7fc1700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.448+0000 7fc18960e700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1700a1120 0x7fc1700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.448+0000 7fc17a7fc700 1 -- 172.21.15.33:0/1105213107 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc17c0ccfe0 con 0x7fc184100a50 2024-05-08T21:20:01.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.449+0000 7fc18960e700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1700a1120 0x7fc1700a35e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc180009cb0 tx=0x7fc180009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.450+0000 7fc197a2cb80 1 Processor -- start 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.450+0000 7fc197a2cb80 1 -- start start 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.450+0000 7fc197a2cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31e540 0x55ff0d31e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.450+0000 7fc197a2cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 0x55ff0d31f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.450+0000 7fc197a2cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31f860 0x55ff0d31fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.450+0000 7fc197a2cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff0d275150 con 0x55ff0d31eea0 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.450+0000 7fc197a2cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff0d272b10 con 0x55ff0d31f860 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.450+0000 7fc197a2cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff0d26e400 con 0x55ff0d31e540 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc188e0d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 0x55ff0d31f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc188e0d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 0x55ff0d31f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50400/0 (socket says 172.21.15.33:50400) 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc188e0d700 1 -- 172.21.15.33:0/3425507725 learned_addr learned my addr 172.21.15.33:0/3425507725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc18960e700 1 --2- 172.21.15.33:0/3425507725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31e540 0x55ff0d31e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc189e0f700 1 --2- 172.21.15.33:0/3425507725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31f860 0x55ff0d31fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc188e0d700 1 -- 172.21.15.33:0/3425507725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31e540 msgr2=0x55ff0d31e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc188e0d700 1 --2- 172.21.15.33:0/3425507725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31e540 0x55ff0d31e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc188e0d700 1 -- 172.21.15.33:0/3425507725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31f860 msgr2=0x55ff0d31fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc188e0d700 1 --2- 172.21.15.33:0/3425507725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31f860 0x55ff0d31fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc188e0d700 1 -- 172.21.15.33:0/3425507725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff0d0ef3b0 con 0x55ff0d31eea0 2024-05-08T21:20:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc18960e700 1 --2- 172.21.15.33:0/3425507725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31e540 0x55ff0d31e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc188e0d700 1 --2- 172.21.15.33:0/3425507725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 0x55ff0d31f320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc17400bf10 tx=0x7fc174000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d43068041ef3f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.451+0000 7fc16f7fe700 1 -- 172.21.15.33:0/3425507725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17401d070 con 0x55ff0d31eea0 2024-05-08T21:20:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.452+0000 7fc16f7fe700 1 -- 172.21.15.33:0/3425507725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc17400bc20 con 0x55ff0d31eea0 2024-05-08T21:20:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.452+0000 7fc16f7fe700 1 -- 172.21.15.33:0/3425507725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1740039c0 con 0x55ff0d31eea0 2024-05-08T21:20:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.453+0000 7fc197a2cb80 1 -- 172.21.15.33:0/3425507725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 msgr2=0x55ff0d31f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.453+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/3425507725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 0x55ff0d31f320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc17400bf10 tx=0x7fc174000bc0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.453+0000 7fc197a2cb80 1 -- 172.21.15.33:0/3425507725 shutdown_connections 2024-05-08T21:20:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.453+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/3425507725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31f860 0x55ff0d31fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.453+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/3425507725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31e540 0x55ff0d31e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.453+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/3425507725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 0x55ff0d31f320 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.453+0000 7fc197a2cb80 1 -- 172.21.15.33:0/3425507725 >> 172.21.15.33:0/3425507725 conn(0x55ff0d27c210 msgr2=0x55ff0d318de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.453+0000 7fc197a2cb80 1 -- 172.21.15.33:0/3425507725 shutdown_connections 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.453+0000 7fc197a2cb80 1 -- 172.21.15.33:0/3425507725 wait complete. 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.453+0000 7fc197a2cb80 1 -- start start 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.453+0000 7fc197a2cb80 1 Processor -- start 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.454+0000 7fc197a2cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31e540 0x55ff0d327cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.454+0000 7fc197a2cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 0x55ff0d328230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.454+0000 7fc197a2cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31f860 0x55ff0d32f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.454+0000 7fc197a2cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff0d20a560 con 0x55ff0d31eea0 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.454+0000 7fc197a2cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff0d20a200 con 0x55ff0d31e540 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.454+0000 7fc197a2cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff0d0f16a0 con 0x55ff0d31f860 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.454+0000 7fc197a2cb80 10 client.0 _ll_register_callbacks cb 0x55ff0d213b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.454+0000 7fc18960e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31e540 0x55ff0d327cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.454+0000 7fc18960e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31e540 0x55ff0d327cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:34688/0 (socket says 172.21.15.33:34688) 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.454+0000 7fc18960e700 1 -- 172.21.15.33:0/4168186355 learned_addr learned my addr 172.21.15.33:0/4168186355 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc189e0f700 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31f860 0x55ff0d32f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc18960e700 1 -- 172.21.15.33:0/4168186355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31f860 msgr2=0x55ff0d32f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc18960e700 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31f860 0x55ff0d32f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc18960e700 1 -- 172.21.15.33:0/4168186355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 msgr2=0x55ff0d328230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc188e0d700 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 0x55ff0d328230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc18960e700 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 0x55ff0d328230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc18960e700 1 -- 172.21.15.33:0/4168186355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc180011c60 con 0x55ff0d31e540 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc188e0d700 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 0x55ff0d328230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc18960e700 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31e540 0x55ff0d327cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc180009b70 tx=0x7fc1800047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f693f416fcb2f4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc16cff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc16cff9700 1 -- 172.21.15.33:0/4168186355 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180012860 con 0x55ff0d31e540 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc16cff9700 1 -- 172.21.15.33:0/4168186355 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc180018070 con 0x55ff0d31e540 2024-05-08T21:20:01.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.455+0000 7fc197a2cb80 1 -- 172.21.15.33:0/4168186355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff0d1c7270 con 0x55ff0d31e540 2024-05-08T21:20:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.456+0000 7fc16cff9700 1 -- 172.21.15.33:0/4168186355 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc180011eb0 con 0x55ff0d31e540 2024-05-08T21:20:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.456+0000 7fc197a2cb80 10 client.25700 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.456+0000 7fc197a2cb80 1 -- 172.21.15.33:0/4168186355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fc18410bdd0 con 0x55ff0d31e540 2024-05-08T21:20:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.456+0000 7fc16cff9700 1 -- 172.21.15.33:0/4168186355 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ff0d1c7270 con 0x55ff0d31e540 2024-05-08T21:20:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.456+0000 7fc16cff9700 1 -- 172.21.15.33:0/4168186355 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1800218d0 con 0x55ff0d31e540 2024-05-08T21:20:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.456+0000 7fc197a2cb80 10 client.25700 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.456+0000 7fc197a2cb80 10 client.25700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.456+0000 7fc197a2cb80 20 client.25700 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.456+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ff0d333ab0 0x55ff0d335ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.456+0000 7fc197a2cb80 4 client.25700 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.456+0000 7fc197a2cb80 1 -- 172.21.15.33:0/4168186355 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff0d2ed810 con 0x55ff0d333ab0 2024-05-08T21:20:01.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.457+0000 7fc188e0d700 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ff0d333ab0 0x55ff0d335ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.457+0000 7fc16cff9700 10 client.25700 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.457+0000 7fc188e0d700 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ff0d333ab0 0x55ff0d335ed0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd560ed7447b6032 server_cookie=4431e0613c405c2 in_seq=0 out_seq=0 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc16cff9700 1 -- 172.21.15.33:0/4168186355 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ff0d2ed810 con 0x55ff0d333ab0 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc16cff9700 10 client.25700 handle_command_reply: tid=0 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc197a2cb80 1 client.25700 shutdown 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc197a2cb80 1 -- 172.21.15.33:0/4168186355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31e540 msgr2=0x55ff0d327cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31e540 0x55ff0d327cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc180009b70 tx=0x7fc1800047c0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc197a2cb80 1 -- 172.21.15.33:0/4168186355 shutdown_connections 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ff0d333ab0 0x55ff0d335ed0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31f860 0x55ff0d32f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc16cff9700 0 client.25700 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31e540 0x55ff0d327cf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/4168186355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31eea0 0x55ff0d328230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc197a2cb80 1 -- 172.21.15.33:0/4168186355 >> 172.21.15.33:0/4168186355 conn(0x55ff0d27c210 msgr2=0x55ff0d3185a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:01.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc197a2cb80 1 -- 172.21.15.33:0/4168186355 shutdown_connections 2024-05-08T21:20:01.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc197a2cb80 1 -- 172.21.15.33:0/4168186355 wait complete. 2024-05-08T21:20:01.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.458+0000 7fc197a2cb80 20 client.25700 trim_cache size 0 max 16384 2024-05-08T21:20:01.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.477+0000 7fc197a2cb80 1 Processor -- start 2024-05-08T21:20:01.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.477+0000 7fc197a2cb80 1 -- start start 2024-05-08T21:20:01.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.477+0000 7fc197a2cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31e540 0x55ff0d3226f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.477+0000 7fc197a2cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31eea0 0x55ff0d322c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.477+0000 7fc197a2cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31f860 0x55ff0d32f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.477+0000 7fc197a2cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff0d331030 con 0x55ff0d31e540 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.477+0000 7fc197a2cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff0d330420 con 0x55ff0d31f860 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.477+0000 7fc197a2cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff0d326d50 con 0x55ff0d31eea0 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.477+0000 7fc18960e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31e540 0x55ff0d3226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc18960e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31e540 0x55ff0d3226f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50418/0 (socket says 172.21.15.33:50418) 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc18960e700 1 -- 172.21.15.33:0/3637565805 learned_addr learned my addr 172.21.15.33:0/3637565805 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc188e0d700 1 --2- 172.21.15.33:0/3637565805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31eea0 0x55ff0d322c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc189e0f700 1 --2- 172.21.15.33:0/3637565805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31f860 0x55ff0d32f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc18960e700 1 -- 172.21.15.33:0/3637565805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31eea0 msgr2=0x55ff0d322c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc18960e700 1 --2- 172.21.15.33:0/3637565805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31eea0 0x55ff0d322c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc18960e700 1 -- 172.21.15.33:0/3637565805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31f860 msgr2=0x55ff0d32f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc18960e700 1 --2- 172.21.15.33:0/3637565805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31f860 0x55ff0d32f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc18960e700 1 -- 172.21.15.33:0/3637565805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff0d09bb60 con 0x55ff0d31e540 2024-05-08T21:20:01.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc189e0f700 1 --2- 172.21.15.33:0/3637565805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31f860 0x55ff0d32f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:01.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc18960e700 1 --2- 172.21.15.33:0/3637565805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31e540 0x55ff0d3226f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc1800096d0 tx=0x7fc180005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4845400c53f3f96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:01.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc16ffff700 1 -- 172.21.15.33:0/3637565805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180003d40 con 0x55ff0d31e540 2024-05-08T21:20:01.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.478+0000 7fc16ffff700 1 -- 172.21.15.33:0/3637565805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc180009ea0 con 0x55ff0d31e540 2024-05-08T21:20:01.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.479+0000 7fc16ffff700 1 -- 172.21.15.33:0/3637565805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180012a20 con 0x55ff0d31e540 2024-05-08T21:20:01.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.479+0000 7fc197a2cb80 1 -- 172.21.15.33:0/3637565805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31e540 msgr2=0x55ff0d3226f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.479+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/3637565805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31e540 0x55ff0d3226f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc1800096d0 tx=0x7fc180005b60 comp rx=0 tx=0).stop 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.479+0000 7fc197a2cb80 1 -- 172.21.15.33:0/3637565805 shutdown_connections 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.479+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/3637565805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31f860 0x55ff0d32f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.479+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/3637565805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31e540 0x55ff0d3226f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.479+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/3637565805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31eea0 0x55ff0d322c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.479+0000 7fc197a2cb80 1 -- 172.21.15.33:0/3637565805 >> 172.21.15.33:0/3637565805 conn(0x55ff0d27c210 msgr2=0x55ff0d327200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.479+0000 7fc197a2cb80 1 -- 172.21.15.33:0/3637565805 shutdown_connections 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.479+0000 7fc197a2cb80 1 -- 172.21.15.33:0/3637565805 wait complete. 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.480+0000 7fc197a2cb80 1 -- start start 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.480+0000 7fc197a2cb80 1 Processor -- start 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.480+0000 7fc197a2cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31e540 0x55ff0d32eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc197a2cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31eea0 0x55ff0d32f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc197a2cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31f860 0x55ff0d32d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc197a2cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff0d32d6c0 con 0x55ff0d31f860 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc197a2cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff0d32d800 con 0x55ff0d31eea0 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc197a2cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff0d32d9c0 con 0x55ff0d31e540 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc197a2cb80 10 client.0 _ll_register_callbacks cb 0x55ff0d213b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc189e0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31f860 0x55ff0d32d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc189e0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31f860 0x55ff0d32d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50432/0 (socket says 172.21.15.33:50432) 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc189e0f700 1 -- 172.21.15.33:0/6389308 learned_addr learned my addr 172.21.15.33:0/6389308 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc18960e700 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31e540 0x55ff0d32eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc188e0d700 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31eea0 0x55ff0d32f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc189e0f700 1 -- 172.21.15.33:0/6389308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31e540 msgr2=0x55ff0d32eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc189e0f700 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31e540 0x55ff0d32eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc189e0f700 1 -- 172.21.15.33:0/6389308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31eea0 msgr2=0x55ff0d32f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc189e0f700 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31eea0 0x55ff0d32f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc189e0f700 1 -- 172.21.15.33:0/6389308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18410bdd0 con 0x55ff0d31f860 2024-05-08T21:20:01.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc188e0d700 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31eea0 0x55ff0d32f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc189e0f700 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31f860 0x55ff0d32d140 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0a1fe0 tx=0x7fc17c012ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71eb394c4c52f61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.481+0000 7fc16cff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc16cff9700 1 -- 172.21.15.33:0/6389308 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c09dec0 con 0x55ff0d31f860 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc16cff9700 1 -- 172.21.15.33:0/6389308 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc17c0b1070 con 0x55ff0d31f860 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc197a2cb80 1 -- 172.21.15.33:0/6389308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc17400bc20 con 0x55ff0d31f860 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc16cff9700 1 -- 172.21.15.33:0/6389308 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc17c0a2510 con 0x55ff0d31f860 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc197a2cb80 10 client.15843 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc197a2cb80 1 -- 172.21.15.33:0/6389308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55ff0d234560 con 0x55ff0d31f860 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc16cff9700 1 -- 172.21.15.33:0/6389308 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc17c063080 con 0x55ff0d31f860 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc16cff9700 1 -- 172.21.15.33:0/6389308 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc17c0a21a0 con 0x55ff0d31f860 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc197a2cb80 10 client.15843 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc197a2cb80 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc197a2cb80 20 client.15843 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:01.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ff0d393b70 0x55ff0d395f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc197a2cb80 4 client.15843 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:20:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.482+0000 7fc197a2cb80 1 -- 172.21.15.33:0/6389308 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ff0d266ec0 con 0x55ff0d393b70 2024-05-08T21:20:01.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.483+0000 7fc18960e700 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ff0d393b70 0x55ff0d395f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.483+0000 7fc16cff9700 10 client.15843 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.483+0000 7fc18960e700 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ff0d393b70 0x55ff0d395f90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51452c87e3c9d146 server_cookie=600fb48b4deefbf5 in_seq=0 out_seq=0 2024-05-08T21:20:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.483+0000 7fc16cff9700 1 -- 172.21.15.33:0/6389308 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ff0d266ec0 con 0x55ff0d393b70 2024-05-08T21:20:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.483+0000 7fc16cff9700 10 client.15843 handle_command_reply: tid=0 2024-05-08T21:20:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.483+0000 7fc197a2cb80 1 client.15843 shutdown 2024-05-08T21:20:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.484+0000 7fc197a2cb80 1 -- 172.21.15.33:0/6389308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31f860 msgr2=0x55ff0d32d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:01.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.484+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31f860 0x55ff0d32d140 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0a1fe0 tx=0x7fc17c012ef0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.484+0000 7fc197a2cb80 1 -- 172.21.15.33:0/6389308 shutdown_connections 2024-05-08T21:20:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.484+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ff0d393b70 0x55ff0d395f90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.484+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ff0d31f860 0x55ff0d32d140 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.484+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff0d31e540 0x55ff0d32eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.484+0000 7fc197a2cb80 1 --2- 172.21.15.33:0/6389308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ff0d31eea0 0x55ff0d32f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.484+0000 7fc197a2cb80 1 -- 172.21.15.33:0/6389308 >> 172.21.15.33:0/6389308 conn(0x55ff0d27c210 msgr2=0x55ff0d3226f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.484+0000 7fc16cff9700 0 client.15843 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.484+0000 7fc197a2cb80 1 -- 172.21.15.33:0/6389308 shutdown_connections 2024-05-08T21:20:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.484+0000 7fc197a2cb80 1 -- 172.21.15.33:0/6389308 wait complete. 2024-05-08T21:20:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.484+0000 7fc197a2cb80 20 client.15843 trim_cache size 0 max 16384 2024-05-08T21:20:01.491 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:20:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:20:01.491 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:20:01.491 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:20:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.485+0000 7fc18b872700 1 -- 172.21.15.33:0/1105213107 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1700a1120 msgr2=0x7fc1700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.485+0000 7fc18b872700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1700a1120 0x7fc1700a35e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc180009cb0 tx=0x7fc180009260 comp rx=0 tx=0).stop 2024-05-08T21:20:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.485+0000 7fc18b872700 1 -- 172.21.15.33:0/1105213107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc184100a50 msgr2=0x7fc1841a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.485+0000 7fc18b872700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc184100a50 0x7fc1841a0400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc17c007d70 tx=0x7fc17c00a340 comp rx=0 tx=0).stop 2024-05-08T21:20:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.485+0000 7fc18b872700 1 -- 172.21.15.33:0/1105213107 shutdown_connections 2024-05-08T21:20:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.485+0000 7fc18b872700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc1700a1120 0x7fc1700a35e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.485+0000 7fc18b872700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc184100a50 0x7fc1841a0400 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.485+0000 7fc18b872700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1840ff730 0x7fc184198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.485+0000 7fc18b872700 1 --2- 172.21.15.33:0/1105213107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc184100090 0x7fc1841993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.485+0000 7fc18b872700 1 -- 172.21.15.33:0/1105213107 >> 172.21.15.33:0/1105213107 conn(0x7fc1840fb290 msgr2=0x7fc1840fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:01.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.485+0000 7fc18b872700 1 -- 172.21.15.33:0/1105213107 shutdown_connections 2024-05-08T21:20:01.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.485+0000 7fc18b872700 1 -- 172.21.15.33:0/1105213107 wait complete. 2024-05-08T21:20:01.502 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:20:01.502 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:20:01.530 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:01 smithi033 ceph-mon[42528]: pgmap v321: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:20:01.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:01 smithi082 ceph-mon[47512]: pgmap v321: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:20:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.685+0000 7f7d549d1700 1 -- 172.21.15.33:0/2646354427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d500ff510 msgr2=0x7f7d500ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.685+0000 7f7d549d1700 1 --2- 172.21.15.33:0/2646354427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d500ff510 0x7f7d500ff930 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000aad0 tx=0x7f7d4000ade0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.685+0000 7f7d549d1700 1 -- 172.21.15.33:0/2646354427 shutdown_connections 2024-05-08T21:20:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.685+0000 7f7d549d1700 1 --2- 172.21.15.33:0/2646354427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d501010a0 0x7f7d50107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.685+0000 7f7d549d1700 1 --2- 172.21.15.33:0/2646354427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d500ff510 0x7f7d500ff930 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.685+0000 7f7d549d1700 1 --2- 172.21.15.33:0/2646354427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d50100700 0x7f7d50100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.685+0000 7f7d549d1700 1 -- 172.21.15.33:0/2646354427 >> 172.21.15.33:0/2646354427 conn(0x7f7d500faa60 msgr2=0x7f7d500fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.685+0000 7f7d549d1700 1 -- 172.21.15.33:0/2646354427 shutdown_connections 2024-05-08T21:20:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.685+0000 7f7d549d1700 1 -- 172.21.15.33:0/2646354427 wait complete. 2024-05-08T21:20:01.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.686+0000 7f7d549d1700 1 Processor -- start 2024-05-08T21:20:01.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d549d1700 1 -- start start 2024-05-08T21:20:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d549d1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d500ff510 0x7f7d50196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d549d1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d50100700 0x7f7d501972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d549d1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d501010a0 0x7f7d5019e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d549d1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5019e860 con 0x7f7d50100700 2024-05-08T21:20:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d549d1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5019e9d0 con 0x7f7d500ff510 2024-05-08T21:20:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d549d1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5019eb90 con 0x7f7d501010a0 2024-05-08T21:20:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d4dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d50100700 0x7f7d501972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d4dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d50100700 0x7f7d501972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50458/0 (socket says 172.21.15.33:50458) 2024-05-08T21:20:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d4dd9b700 1 -- 172.21.15.33:0/222165013 learned_addr learned my addr 172.21.15.33:0/222165013 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d4e59c700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d500ff510 0x7f7d50196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d4ed9d700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d501010a0 0x7f7d5019e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d4e59c700 1 -- 172.21.15.33:0/222165013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d501010a0 msgr2=0x7f7d5019e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d4e59c700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d501010a0 0x7f7d5019e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d4e59c700 1 -- 172.21.15.33:0/222165013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d50100700 msgr2=0x7f7d501972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d4e59c700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d50100700 0x7f7d501972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d4e59c700 1 -- 172.21.15.33:0/222165013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4000a700 con 0x7f7d500ff510 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.687+0000 7f7d4dd9b700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d50100700 0x7f7d501972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.688+0000 7f7d4e59c700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d500ff510 0x7f7d50196d70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000a9f0 tx=0x7f7d4000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.688+0000 7f7d3f7fe700 1 -- 172.21.15.33:0/222165013 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40005920 con 0x7f7d500ff510 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.688+0000 7f7d3f7fe700 1 -- 172.21.15.33:0/222165013 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d4000cab0 con 0x7f7d500ff510 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.688+0000 7f7d3f7fe700 1 -- 172.21.15.33:0/222165013 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4001f400 con 0x7f7d500ff510 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.688+0000 7f7d549d1700 1 -- 172.21.15.33:0/222165013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5019ee10 con 0x7f7d500ff510 2024-05-08T21:20:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.688+0000 7f7d549d1700 1 -- 172.21.15.33:0/222165013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5019f1b0 con 0x7f7d500ff510 2024-05-08T21:20:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.689+0000 7f7d549d1700 1 -- 172.21.15.33:0/222165013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d50068160 con 0x7f7d500ff510 2024-05-08T21:20:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.691+0000 7f7d3f7fe700 1 -- 172.21.15.33:0/222165013 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7d40016420 con 0x7f7d500ff510 2024-05-08T21:20:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.691+0000 7f7d3f7fe700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7d380a1120 0x7f7d380a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.692+0000 7f7d4dd9b700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7d380a1120 0x7f7d380a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.692+0000 7f7d3f7fe700 1 -- 172.21.15.33:0/222165013 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7d40012070 con 0x7f7d500ff510 2024-05-08T21:20:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.692+0000 7f7d4dd9b700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7d380a1120 0x7f7d380a35e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7d44005950 tx=0x7f7d4400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:01.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:01.695+0000 7f7d3f7fe700 1 -- 172.21.15.33:0/222165013 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7d40098090 con 0x7f7d500ff510 2024-05-08T21:20:02.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.019+0000 7f7d549d1700 1 -- 172.21.15.33:0/222165013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d5010a5d0 con 0x7f7d500ff510 2024-05-08T21:20:02.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.020+0000 7f7d3f7fe700 1 -- 172.21.15.33:0/222165013 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10899 (secure 0 0 0) 0x7f7d4000dea0 con 0x7f7d500ff510 2024-05-08T21:20:02.022 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:20:02.023 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:18:49.368057+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:20:02.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.023+0000 7f7d549d1700 1 -- 172.21.15.33:0/222165013 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7d380a1120 msgr2=0x7f7d380a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.023+0000 7f7d549d1700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7d380a1120 0x7f7d380a35e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7d44005950 tx=0x7f7d4400b500 comp rx=0 tx=0).stop 2024-05-08T21:20:02.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.023+0000 7f7d549d1700 1 -- 172.21.15.33:0/222165013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d500ff510 msgr2=0x7f7d50196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.023+0000 7f7d549d1700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d500ff510 0x7f7d50196d70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000a9f0 tx=0x7f7d4000ee60 comp rx=0 tx=0).stop 2024-05-08T21:20:02.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.023+0000 7f7d549d1700 1 -- 172.21.15.33:0/222165013 shutdown_connections 2024-05-08T21:20:02.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.023+0000 7f7d549d1700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d501010a0 0x7f7d5019e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.023+0000 7f7d549d1700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7d380a1120 0x7f7d380a35e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.023+0000 7f7d549d1700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d500ff510 0x7f7d50196d70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.023+0000 7f7d549d1700 1 --2- 172.21.15.33:0/222165013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d50100700 0x7f7d501972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.023+0000 7f7d549d1700 1 -- 172.21.15.33:0/222165013 >> 172.21.15.33:0/222165013 conn(0x7f7d500faa60 msgr2=0x7f7d50105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.023+0000 7f7d549d1700 1 -- 172.21.15.33:0/222165013 shutdown_connections 2024-05-08T21:20:02.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.023+0000 7f7d549d1700 1 -- 172.21.15.33:0/222165013 wait complete. 2024-05-08T21:20:02.026 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-05-08T21:20:02.040 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:20:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.235+0000 7f2995696700 1 -- 172.21.15.33:0/285471482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2990100090 msgr2=0x7f2990100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.235+0000 7f2995696700 1 --2- 172.21.15.33:0/285471482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2990100090 0x7f2990100510 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f29840099c0 tx=0x7f2984009cd0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.236+0000 7f2995696700 1 -- 172.21.15.33:0/285471482 shutdown_connections 2024-05-08T21:20:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.236+0000 7f2995696700 1 --2- 172.21.15.33:0/285471482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2990100a50 0x7f2990100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.236+0000 7f2995696700 1 --2- 172.21.15.33:0/285471482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29900ff730 0x7f29900ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.236+0000 7f2995696700 1 --2- 172.21.15.33:0/285471482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2990100090 0x7f2990100510 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.236+0000 7f2995696700 1 -- 172.21.15.33:0/285471482 >> 172.21.15.33:0/285471482 conn(0x7f29900fb290 msgr2=0x7f29900fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.236+0000 7f2995696700 1 -- 172.21.15.33:0/285471482 shutdown_connections 2024-05-08T21:20:02.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.236+0000 7f2995696700 1 -- 172.21.15.33:0/285471482 wait complete. 2024-05-08T21:20:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.236+0000 7f2995696700 1 Processor -- start 2024-05-08T21:20:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.237+0000 7f2995696700 1 -- start start 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.237+0000 7f2995696700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29900ff730 0x7f2990198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.237+0000 7f2995696700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2990100090 0x7f2990199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.237+0000 7f2995696700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2990100a50 0x7f29901a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.237+0000 7f2995696700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29901a0a00 con 0x7f2990100090 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.237+0000 7f2995696700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29901a0b40 con 0x7f29900ff730 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.237+0000 7f2995696700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29901a0cd0 con 0x7f2990100a50 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.237+0000 7f298f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2990100a50 0x7f29901a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.237+0000 7f298f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2990100a50 0x7f29901a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51214/0 (socket says 172.21.15.33:51214) 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.237+0000 7f298f7fe700 1 -- 172.21.15.33:0/2912153235 learned_addr learned my addr 172.21.15.33:0/2912153235 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.237+0000 7f298effd700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29900ff730 0x7f2990198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f298e7fc700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2990100090 0x7f2990199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f298f7fe700 1 -- 172.21.15.33:0/2912153235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29900ff730 msgr2=0x7f2990198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f298f7fe700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29900ff730 0x7f2990198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f298f7fe700 1 -- 172.21.15.33:0/2912153235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2990100090 msgr2=0x7f2990199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f298f7fe700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2990100090 0x7f2990199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f298f7fe700 1 -- 172.21.15.33:0/2912153235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29840095f0 con 0x7f2990100a50 2024-05-08T21:20:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f298e7fc700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2990100090 0x7f2990199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f298f7fe700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2990100a50 0x7f29901a0430 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f298800ef30 tx=0x7f298800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f2977fff700 1 -- 172.21.15.33:0/2912153235 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988018070 con 0x7f2990100a50 2024-05-08T21:20:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f2995696700 1 -- 172.21.15.33:0/2912153235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29901a0f50 con 0x7f2990100a50 2024-05-08T21:20:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f2977fff700 1 -- 172.21.15.33:0/2912153235 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f298800f040 con 0x7f2990100a50 2024-05-08T21:20:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f2977fff700 1 -- 172.21.15.33:0/2912153235 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298800cc70 con 0x7f2990100a50 2024-05-08T21:20:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.238+0000 7f2995696700 1 -- 172.21.15.33:0/2912153235 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29901a13d0 con 0x7f2990100a50 2024-05-08T21:20:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.242+0000 7f2977fff700 1 -- 172.21.15.33:0/2912153235 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2988004000 con 0x7f2990100a50 2024-05-08T21:20:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.242+0000 7f29a1850b80 1 Processor -- start 2024-05-08T21:20:02.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.242+0000 7f29a1850b80 1 -- start start 2024-05-08T21:20:02.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.242+0000 7f29a1850b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd00960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.242+0000 7f29a1850b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd01320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.242+0000 7f29a1850b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efd01ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.242+0000 7f29a1850b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1efc57150 con 0x55c1efd00ea0 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.242+0000 7f29a1850b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1efc54b10 con 0x55c1efd01860 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.242+0000 7f29a1850b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1efc50400 con 0x55c1efd00540 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f2977fff700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f29780a1120 0x7f29780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f298e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efd01ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f298e7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efd01ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:34764/0 (socket says 172.21.15.33:34764) 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f298e7fc700 1 -- 172.21.15.33:0/3689038983 learned_addr learned my addr 172.21.15.33:0/3689038983 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f298effd700 1 --2- 172.21.15.33:0/3689038983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd01320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f298e7fc700 1 --2- 172.21.15.33:0/3689038983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd00960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f2977fff700 1 -- 172.21.15.33:0/2912153235 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f29880cc850 con 0x7f2990100a50 2024-05-08T21:20:02.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f298e7fc700 1 -- 172.21.15.33:0/3689038983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 msgr2=0x55c1efd00960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f298e7fc700 1 --2- 172.21.15.33:0/3689038983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd00960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f298e7fc700 1 -- 172.21.15.33:0/3689038983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 msgr2=0x55c1efd01320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f298e7fc700 1 --2- 172.21.15.33:0/3689038983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd01320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f298e7fc700 1 -- 172.21.15.33:0/3689038983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1efad13b0 con 0x55c1efd01860 2024-05-08T21:20:02.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.243+0000 7f298f7fe700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f29780a1120 0x7f29780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.244+0000 7f298f7fe700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f29780a1120 0x7f29780a35e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2988004d10 tx=0x7f2988009ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.245+0000 7f298e7fc700 1 --2- 172.21.15.33:0/3689038983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efd01ce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f29840099c0 tx=0x7f2984003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b32ad230b3fb523 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.245+0000 7f29757fa700 1 -- 172.21.15.33:0/3689038983 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2984005900 con 0x55c1efd01860 2024-05-08T21:20:02.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.245+0000 7f29757fa700 1 -- 172.21.15.33:0/3689038983 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f29840044b0 con 0x55c1efd01860 2024-05-08T21:20:02.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.245+0000 7f29757fa700 1 -- 172.21.15.33:0/3689038983 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298400f460 con 0x55c1efd01860 2024-05-08T21:20:02.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.246+0000 7f29a1850b80 1 -- 172.21.15.33:0/3689038983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 msgr2=0x55c1efd01ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.246+0000 7f29a1850b80 1 --2- 172.21.15.33:0/3689038983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efd01ce0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f29840099c0 tx=0x7f2984003680 comp rx=0 tx=0).stop 2024-05-08T21:20:02.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.246+0000 7f29a1850b80 1 -- 172.21.15.33:0/3689038983 shutdown_connections 2024-05-08T21:20:02.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.246+0000 7f29a1850b80 1 --2- 172.21.15.33:0/3689038983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efd01ce0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.246+0000 7f29a1850b80 1 --2- 172.21.15.33:0/3689038983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd00960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.246+0000 7f29a1850b80 1 --2- 172.21.15.33:0/3689038983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd01320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.246+0000 7f29a1850b80 1 -- 172.21.15.33:0/3689038983 >> 172.21.15.33:0/3689038983 conn(0x55c1efc5e210 msgr2=0x55c1efcfade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.246+0000 7f29a1850b80 1 -- 172.21.15.33:0/3689038983 shutdown_connections 2024-05-08T21:20:02.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.246+0000 7f29a1850b80 1 -- 172.21.15.33:0/3689038983 wait complete. 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.247+0000 7f29a1850b80 1 -- start start 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.247+0000 7f29a1850b80 1 Processor -- start 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f29a1850b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd12550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f29a1850b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd12a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f29a1850b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efd19a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f29a1850b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1efbec560 con 0x55c1efd00ea0 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f29a1850b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1efbec200 con 0x55c1efd01860 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f29a1850b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1efad36a0 con 0x55c1efd00540 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f29a1850b80 10 client.0 _ll_register_callbacks cb 0x55c1efbf5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f298e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd12550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f298e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd12550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51232/0 (socket says 172.21.15.33:51232) 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f298e7fc700 1 -- 172.21.15.33:0/60331754 learned_addr learned my addr 172.21.15.33:0/60331754 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f298e7fc700 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efd19a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f298effd700 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd12a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f298e7fc700 1 -- 172.21.15.33:0/60331754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 msgr2=0x55c1efd19a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f298e7fc700 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efd19a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f298e7fc700 1 -- 172.21.15.33:0/60331754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 msgr2=0x55c1efd12a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.248+0000 7f298e7fc700 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd12a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.249+0000 7f298e7fc700 1 -- 172.21.15.33:0/60331754 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299010bdd0 con 0x55c1efd00540 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.249+0000 7f298effd700 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd12a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.249+0000 7f298e7fc700 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd12550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2984004740 tx=0x7f2984009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f283a76b78c3d6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.249+0000 7f2972ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.249+0000 7f2972ff5700 1 -- 172.21.15.33:0/60331754 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2984003760 con 0x55c1efd00540 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.249+0000 7f29a1850b80 1 -- 172.21.15.33:0/60331754 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1efba9270 con 0x55c1efd00540 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.249+0000 7f2972ff5700 1 -- 172.21.15.33:0/60331754 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f2984003d30 con 0x55c1efd00540 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.249+0000 7f2972ff5700 1 -- 172.21.15.33:0/60331754 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2984003ef0 con 0x55c1efd00540 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.249+0000 7f29a1850b80 10 client.25738 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.249+0000 7f29a1850b80 1 -- 172.21.15.33:0/60331754 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c1efa7db60 con 0x55c1efd00540 2024-05-08T21:20:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.249+0000 7f2972ff5700 1 -- 172.21.15.33:0/60331754 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c1efba9270 con 0x55c1efd00540 2024-05-08T21:20:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.250+0000 7f2972ff5700 1 -- 172.21.15.33:0/60331754 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f29840047b0 con 0x55c1efd00540 2024-05-08T21:20:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.250+0000 7f29a1850b80 10 client.25738 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.250+0000 7f29a1850b80 10 client.25738 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:20:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.250+0000 7f29a1850b80 20 client.25738 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.250+0000 7f29a1850b80 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c1efd1de80 0x55c1efd202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.250+0000 7f29a1850b80 4 client.25738 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.250+0000 7f29a1850b80 1 -- 172.21.15.33:0/60331754 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1efccf810 con 0x55c1efd1de80 2024-05-08T21:20:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.250+0000 7f298effd700 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c1efd1de80 0x55c1efd202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.250+0000 7f2972ff5700 10 client.25738 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:20:02.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.250+0000 7f298effd700 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c1efd1de80 0x55c1efd202a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e0944911cc44cc0e server_cookie=6c129fb6c51e455a in_seq=0 out_seq=0 2024-05-08T21:20:02.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f2972ff5700 1 -- 172.21.15.33:0/60331754 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c1efccf810 con 0x55c1efd1de80 2024-05-08T21:20:02.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f2972ff5700 10 client.25738 handle_command_reply: tid=0 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f29a1850b80 1 client.25738 shutdown 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f29a1850b80 1 -- 172.21.15.33:0/60331754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 msgr2=0x55c1efd12550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f29a1850b80 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd12550 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2984004740 tx=0x7f2984009cd0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f29a1850b80 1 -- 172.21.15.33:0/60331754 shutdown_connections 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f29a1850b80 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efd19a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f29a1850b80 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c1efd1de80 0x55c1efd202a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f29a1850b80 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd12550 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f29a1850b80 1 --2- 172.21.15.33:0/60331754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd12a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f29a1850b80 1 -- 172.21.15.33:0/60331754 >> 172.21.15.33:0/60331754 conn(0x55c1efc5e210 msgr2=0x55c1efcfa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f2972ff5700 0 client.25738 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f29a1850b80 1 -- 172.21.15.33:0/60331754 shutdown_connections 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f29a1850b80 1 -- 172.21.15.33:0/60331754 wait complete. 2024-05-08T21:20:02.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.251+0000 7f29a1850b80 20 client.25738 trim_cache size 0 max 16384 2024-05-08T21:20:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.266+0000 7f29a1850b80 1 Processor -- start 2024-05-08T21:20:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.266+0000 7f29a1850b80 1 -- start start 2024-05-08T21:20:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.266+0000 7f29a1850b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd11ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.266+0000 7f29a1850b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd19a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.266+0000 7f29a1850b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efcfadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.266+0000 7f29a1850b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1efcf5880 con 0x55c1efd00ea0 2024-05-08T21:20:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.266+0000 7f29a1850b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1efcfc860 con 0x55c1efd01860 2024-05-08T21:20:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.266+0000 7f29a1850b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1efd25480 con 0x55c1efd00540 2024-05-08T21:20:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.266+0000 7f298e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd11ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.266+0000 7f298e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd11ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51248/0 (socket says 172.21.15.33:51248) 2024-05-08T21:20:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.266+0000 7f298e7fc700 1 -- 172.21.15.33:0/521783559 learned_addr learned my addr 172.21.15.33:0/521783559 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.266+0000 7f298e7fc700 1 --2- 172.21.15.33:0/521783559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efcfadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.267+0000 7f298effd700 1 --2- 172.21.15.33:0/521783559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd19a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.267+0000 7f298e7fc700 1 -- 172.21.15.33:0/521783559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 msgr2=0x55c1efcfadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.267+0000 7f298e7fc700 1 --2- 172.21.15.33:0/521783559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efcfadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.267+0000 7f298e7fc700 1 -- 172.21.15.33:0/521783559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 msgr2=0x55c1efd19a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.267+0000 7f298e7fc700 1 --2- 172.21.15.33:0/521783559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd19a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.267+0000 7f298e7fc700 1 -- 172.21.15.33:0/521783559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1efc16560 con 0x55c1efd00540 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.267+0000 7f298e7fc700 1 --2- 172.21.15.33:0/521783559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd11ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f298400bc20 tx=0x7f2984003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7db57ad134507e40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.267+0000 7f2975ffb700 1 -- 172.21.15.33:0/521783559 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29840047b0 con 0x55c1efd00540 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.267+0000 7f2975ffb700 1 -- 172.21.15.33:0/521783559 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2984004910 con 0x55c1efd00540 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.267+0000 7f2975ffb700 1 -- 172.21.15.33:0/521783559 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2984027750 con 0x55c1efd00540 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.268+0000 7f29a1850b80 1 -- 172.21.15.33:0/521783559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 msgr2=0x55c1efd11ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.268+0000 7f29a1850b80 1 --2- 172.21.15.33:0/521783559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd11ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f298400bc20 tx=0x7f2984003680 comp rx=0 tx=0).stop 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.268+0000 7f29a1850b80 1 -- 172.21.15.33:0/521783559 shutdown_connections 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.268+0000 7f29a1850b80 1 --2- 172.21.15.33:0/521783559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd01860 0x55c1efcfadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.268+0000 7f29a1850b80 1 --2- 172.21.15.33:0/521783559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd11ad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.268+0000 7f29a1850b80 1 --2- 172.21.15.33:0/521783559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd00ea0 0x55c1efd19a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.268+0000 7f29a1850b80 1 -- 172.21.15.33:0/521783559 >> 172.21.15.33:0/521783559 conn(0x55c1efc5e210 msgr2=0x55c1efd01c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.268+0000 7f29a1850b80 1 -- 172.21.15.33:0/521783559 shutdown_connections 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.268+0000 7f29a1850b80 1 -- 172.21.15.33:0/521783559 wait complete. 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.268+0000 7f29a1850b80 1 -- start start 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f29a1850b80 1 Processor -- start 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f29a1850b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd14c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f29a1850b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd00ea0 0x55c1efd15150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f29a1850b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd01860 0x55c1efd72320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f29a1850b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1efd19b20 con 0x55c1efd01860 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f29a1850b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1efd19c90 con 0x55c1efd00ea0 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f29a1850b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1efd19e50 con 0x55c1efd00540 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f29a1850b80 10 client.0 _ll_register_callbacks cb 0x55c1efbf5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f298e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd01860 0x55c1efd72320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f298e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd01860 0x55c1efd72320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50498/0 (socket says 172.21.15.33:50498) 2024-05-08T21:20:02.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f298e7fc700 1 -- 172.21.15.33:0/3657283819 learned_addr learned my addr 172.21.15.33:0/3657283819 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f298e7fc700 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd14c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.269+0000 7f298effd700 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd00ea0 0x55c1efd15150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f298e7fc700 1 -- 172.21.15.33:0/3657283819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 msgr2=0x55c1efd14c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f298e7fc700 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd14c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f298e7fc700 1 -- 172.21.15.33:0/3657283819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd00ea0 msgr2=0x55c1efd15150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f298e7fc700 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd00ea0 0x55c1efd15150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f298e7fc700 1 -- 172.21.15.33:0/3657283819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1efa7db60 con 0x55c1efd01860 2024-05-08T21:20:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f298e7fc700 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd01860 0x55c1efd72320 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2984001eb0 tx=0x7f29840046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15bc5b3fab5a945 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f2972ff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f2972ff5700 1 -- 172.21.15.33:0/3657283819 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2984027de0 con 0x55c1efd01860 2024-05-08T21:20:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f2972ff5700 1 -- 172.21.15.33:0/3657283819 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f29840274d0 con 0x55c1efd01860 2024-05-08T21:20:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f29a1850b80 1 -- 172.21.15.33:0/3657283819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29840044b0 con 0x55c1efd01860 2024-05-08T21:20:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f2972ff5700 1 -- 172.21.15.33:0/3657283819 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2984027690 con 0x55c1efd01860 2024-05-08T21:20:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f29a1850b80 10 client.15867 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.270+0000 7f29a1850b80 1 -- 172.21.15.33:0/3657283819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c1efad5cd0 con 0x55c1efd01860 2024-05-08T21:20:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.271+0000 7f2972ff5700 1 -- 172.21.15.33:0/3657283819 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f29840044b0 con 0x55c1efd01860 2024-05-08T21:20:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.271+0000 7f2972ff5700 1 -- 172.21.15.33:0/3657283819 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2984004ed0 con 0x55c1efd01860 2024-05-08T21:20:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.273+0000 7f29a1850b80 10 client.15867 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.273+0000 7f29a1850b80 10 client.15867 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:20:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.273+0000 7f29a1850b80 20 client.15867 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.273+0000 7f29a1850b80 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c1efd73a30 0x55c1efd75e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.273+0000 7f29a1850b80 4 client.15867 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:20:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.273+0000 7f29a1850b80 1 -- 172.21.15.33:0/3657283819 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c1efc48ec0 con 0x55c1efd73a30 2024-05-08T21:20:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.274+0000 7f298effd700 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c1efd73a30 0x55c1efd75e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.274+0000 7f298effd700 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c1efd73a30 0x55c1efd75e20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5cd26bf6823a778 server_cookie=7e01bc294fcb69fe in_seq=0 out_seq=0 2024-05-08T21:20:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.274+0000 7f2972ff5700 10 client.15867 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:20:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.274+0000 7f2972ff5700 1 -- 172.21.15.33:0/3657283819 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c1efc48ec0 con 0x55c1efd73a30 2024-05-08T21:20:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.274+0000 7f2972ff5700 10 client.15867 handle_command_reply: tid=0 2024-05-08T21:20:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.274+0000 7f29a1850b80 1 client.15867 shutdown 2024-05-08T21:20:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.275+0000 7f29a1850b80 1 -- 172.21.15.33:0/3657283819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd01860 msgr2=0x55c1efd72320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.275+0000 7f29a1850b80 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd01860 0x55c1efd72320 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2984001eb0 tx=0x7f29840046a0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.275+0000 7f29a1850b80 1 -- 172.21.15.33:0/3657283819 shutdown_connections 2024-05-08T21:20:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.275+0000 7f29a1850b80 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1efd01860 0x55c1efd72320 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.275+0000 7f29a1850b80 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55c1efd73a30 0x55c1efd75e20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.275+0000 7f29a1850b80 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1efd00540 0x55c1efd14c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.275+0000 7f29a1850b80 1 --2- 172.21.15.33:0/3657283819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1efd00ea0 0x55c1efd15150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.275+0000 7f29a1850b80 1 -- 172.21.15.33:0/3657283819 >> 172.21.15.33:0/3657283819 conn(0x55c1efc5e210 msgr2=0x55c1efd11ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.275+0000 7f2972ff5700 0 client.15867 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:20:02.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.276+0000 7f29a1850b80 1 -- 172.21.15.33:0/3657283819 shutdown_connections 2024-05-08T21:20:02.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.277+0000 7f29a1850b80 1 -- 172.21.15.33:0/3657283819 wait complete. 2024-05-08T21:20:02.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.277+0000 7f29a1850b80 20 client.15867 trim_cache size 0 max 16384 2024-05-08T21:20:02.280 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:20:02.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.279+0000 7f2995696700 1 -- 172.21.15.33:0/2912153235 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f29780a1120 msgr2=0x7f29780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.279+0000 7f2995696700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f29780a1120 0x7f29780a35e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2988004d10 tx=0x7f2988009ed0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.279+0000 7f2995696700 1 -- 172.21.15.33:0/2912153235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2990100a50 msgr2=0x7f29901a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.279+0000 7f2995696700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2990100a50 0x7f29901a0430 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f298800ef30 tx=0x7f298800c3a0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.279+0000 7f2995696700 1 -- 172.21.15.33:0/2912153235 shutdown_connections 2024-05-08T21:20:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.279+0000 7f2995696700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2990100a50 0x7f29901a0430 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.279+0000 7f2995696700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f29780a1120 0x7f29780a35e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.279+0000 7f2995696700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29900ff730 0x7f2990198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.279+0000 7f2995696700 1 --2- 172.21.15.33:0/2912153235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2990100090 0x7f2990199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.279+0000 7f2995696700 1 -- 172.21.15.33:0/2912153235 >> 172.21.15.33:0/2912153235 conn(0x7f29900fb290 msgr2=0x7f29900fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.279+0000 7f2995696700 1 -- 172.21.15.33:0/2912153235 shutdown_connections 2024-05-08T21:20:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.279+0000 7f2995696700 1 -- 172.21.15.33:0/2912153235 wait complete. 2024-05-08T21:20:02.294 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:02 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/222165013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:20:02.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:20:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.441+0000 7fcd34cc2700 1 -- 172.21.15.33:0/831706589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd30100e80 msgr2=0x7fcd301012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.441+0000 7fcd34cc2700 1 --2- 172.21.15.33:0/831706589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd30100e80 0x7fcd301012a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009990 tx=0x7fcd20009ca0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.441+0000 7fcd34cc2700 1 -- 172.21.15.33:0/831706589 shutdown_connections 2024-05-08T21:20:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.441+0000 7fcd34cc2700 1 --2- 172.21.15.33:0/831706589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd30102990 0x7fcd301092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.441+0000 7fcd34cc2700 1 --2- 172.21.15.33:0/831706589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd30100e80 0x7fcd301012a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.441+0000 7fcd34cc2700 1 --2- 172.21.15.33:0/831706589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd30101fd0 0x7fcd30102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.441+0000 7fcd34cc2700 1 -- 172.21.15.33:0/831706589 >> 172.21.15.33:0/831706589 conn(0x7fcd300fc3f0 msgr2=0x7fcd300fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.441+0000 7fcd34cc2700 1 -- 172.21.15.33:0/831706589 shutdown_connections 2024-05-08T21:20:02.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.441+0000 7fcd34cc2700 1 -- 172.21.15.33:0/831706589 wait complete. 2024-05-08T21:20:02.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.442+0000 7fcd34cc2700 1 Processor -- start 2024-05-08T21:20:02.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.442+0000 7fcd34cc2700 1 -- start start 2024-05-08T21:20:02.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.442+0000 7fcd34cc2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd30100e80 0x7fcd30074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.442+0000 7fcd34cc2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd30101fd0 0x7fcd30073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd34cc2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd30102990 0x7fcd300738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd34cc2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd30074070 con 0x7fcd30100e80 2024-05-08T21:20:02.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd34cc2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300741e0 con 0x7fcd30102990 2024-05-08T21:20:02.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd34cc2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd301a0db0 con 0x7fcd30101fd0 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd2ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd30102990 0x7fcd300738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd2ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd30102990 0x7fcd300738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:34824/0 (socket says 172.21.15.33:34824) 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd2ed9d700 1 -- 172.21.15.33:0/1806932548 learned_addr learned my addr 172.21.15.33:0/1806932548 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd30100e80 0x7fcd30074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd2ed9d700 1 -- 172.21.15.33:0/1806932548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd30101fd0 msgr2=0x7fcd30073390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd2ed9d700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd30101fd0 0x7fcd30073390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd2ed9d700 1 -- 172.21.15.33:0/1806932548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd30100e80 msgr2=0x7fcd30074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd2ed9d700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd30100e80 0x7fcd30074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd2ed9d700 1 -- 172.21.15.33:0/1806932548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd200095f0 con 0x7fcd30102990 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.443+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd30100e80 0x7fcd30074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.444+0000 7fcd2ed9d700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd30102990 0x7fcd300738d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800b970 tx=0x7fcd2800bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.444+0000 7fcd1f7fe700 1 -- 172.21.15.33:0/1806932548 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2800c950 con 0x7fcd30102990 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.444+0000 7fcd1f7fe700 1 -- 172.21.15.33:0/1806932548 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd28014440 con 0x7fcd30102990 2024-05-08T21:20:02.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.444+0000 7fcd34cc2700 1 -- 172.21.15.33:0/1806932548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd301a0fe0 con 0x7fcd30102990 2024-05-08T21:20:02.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.444+0000 7fcd1f7fe700 1 -- 172.21.15.33:0/1806932548 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280075d0 con 0x7fcd30102990 2024-05-08T21:20:02.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.444+0000 7fcd34cc2700 1 -- 172.21.15.33:0/1806932548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd301a1540 con 0x7fcd30102990 2024-05-08T21:20:02.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.447+0000 7fcd1f7fe700 1 -- 172.21.15.33:0/1806932548 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcd280145b0 con 0x7fcd30102990 2024-05-08T21:20:02.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.448+0000 7fcd1f7fe700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcd180a11a0 0x7fcd180a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.448+0000 7fcd1f7fe700 1 -- 172.21.15.33:0/1806932548 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcd28006e40 con 0x7fcd30102990 2024-05-08T21:20:02.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.449+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcd180a11a0 0x7fcd180a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.449+0000 7fcd40e7cb80 1 Processor -- start 2024-05-08T21:20:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.450+0000 7fcd40e7cb80 1 -- start start 2024-05-08T21:20:02.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.450+0000 7fcd40e7cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 0x560c4ad2e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.450+0000 7fcd40e7cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2eea0 0x560c4ad2f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.450+0000 7fcd40e7cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2f860 0x560c4ad2fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.450+0000 7fcd40e7cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ac85150 con 0x560c4ad2f860 2024-05-08T21:20:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.450+0000 7fcd40e7cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ac82b10 con 0x560c4ad2eea0 2024-05-08T21:20:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.450+0000 7fcd40e7cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ac7e400 con 0x560c4ad2e540 2024-05-08T21:20:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.450+0000 7fcd2e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 0x560c4ad2e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.450+0000 7fcd2e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 0x560c4ad2e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51278/0 (socket says 172.21.15.33:51278) 2024-05-08T21:20:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.450+0000 7fcd2e59c700 1 -- 172.21.15.33:0/2193592605 learned_addr learned my addr 172.21.15.33:0/2193592605 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.450+0000 7fcd2ed9d700 1 --2- 172.21.15.33:0/2193592605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2f860 0x560c4ad2fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.451+0000 7fcd2e59c700 1 -- 172.21.15.33:0/2193592605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2eea0 msgr2=0x560c4ad2f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.451+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/2193592605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2eea0 0x560c4ad2f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.451+0000 7fcd2e59c700 1 -- 172.21.15.33:0/2193592605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2f860 msgr2=0x560c4ad2fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.451+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/2193592605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2f860 0x560c4ad2fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.451+0000 7fcd2e59c700 1 -- 172.21.15.33:0/2193592605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c4aaff3b0 con 0x560c4ad2e540 2024-05-08T21:20:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.451+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/2193592605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 0x560c4ad2e960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009990 tx=0x7fcd200038f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a4f7efe28f8023e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.451+0000 7fcd1cff9700 1 -- 172.21.15.33:0/2193592605 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2002d030 con 0x560c4ad2e540 2024-05-08T21:20:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.451+0000 7fcd1cff9700 1 -- 172.21.15.33:0/2193592605 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd2001e070 con 0x560c4ad2e540 2024-05-08T21:20:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.451+0000 7fcd1cff9700 1 -- 172.21.15.33:0/2193592605 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2000fbc0 con 0x560c4ad2e540 2024-05-08T21:20:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.452+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/2193592605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 msgr2=0x560c4ad2e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.452+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/2193592605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 0x560c4ad2e960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009990 tx=0x7fcd200038f0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.452+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/2193592605 shutdown_connections 2024-05-08T21:20:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.452+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/2193592605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2f860 0x560c4ad2fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.452+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/2193592605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 0x560c4ad2e960 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.452+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/2193592605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2eea0 0x560c4ad2f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.452+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/2193592605 >> 172.21.15.33:0/2193592605 conn(0x560c4ac8c210 msgr2=0x560c4ad28de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.452+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/2193592605 shutdown_connections 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.452+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/2193592605 wait complete. 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd40e7cb80 1 -- start start 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd40e7cb80 1 Processor -- start 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd40e7cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2e540 0x560c4ad515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd40e7cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2eea0 0x560c4ad51b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd40e7cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2f860 0x560c4ad58b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd40e7cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ac1a560 con 0x560c4ad2eea0 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd40e7cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ac1a200 con 0x560c4ad2e540 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd40e7cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ab016a0 con 0x560c4ad2f860 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd40e7cb80 10 client.0 _ll_register_callbacks cb 0x560c4ac23b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd2ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2f860 0x560c4ad58b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd2ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2f860 0x560c4ad58b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51294/0 (socket says 172.21.15.33:51294) 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd2ed9d700 1 -- 172.21.15.33:0/3863003729 learned_addr learned my addr 172.21.15.33:0/3863003729 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.453+0000 7fcd2dd9b700 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2eea0 0x560c4ad51b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.454+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2e540 0x560c4ad515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.454+0000 7fcd2ed9d700 1 -- 172.21.15.33:0/3863003729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2e540 msgr2=0x560c4ad515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.454+0000 7fcd2ed9d700 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2e540 0x560c4ad515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.454+0000 7fcd2ed9d700 1 -- 172.21.15.33:0/3863003729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2eea0 msgr2=0x560c4ad51b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.454+0000 7fcd2ed9d700 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2eea0 0x560c4ad51b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.454+0000 7fcd2ed9d700 1 -- 172.21.15.33:0/3863003729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd30109be0 con 0x560c4ad2f860 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.454+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2e540 0x560c4ad515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.454+0000 7fcd2ed9d700 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2f860 0x560c4ad58b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd28096060 tx=0x7fcd280b7ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c52541480d301186 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.454+0000 7fcd15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.454+0000 7fcd15ffb700 1 -- 172.21.15.33:0/3863003729 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280995f0 con 0x560c4ad2f860 2024-05-08T21:20:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.454+0000 7fcd15ffb700 1 -- 172.21.15.33:0/3863003729 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcd28082e40 con 0x560c4ad2f860 2024-05-08T21:20:02.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.454+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3863003729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c4abd7270 con 0x560c4ad2f860 2024-05-08T21:20:02.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.455+0000 7fcd15ffb700 1 -- 172.21.15.33:0/3863003729 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd2809d420 con 0x560c4ad2f860 2024-05-08T21:20:02.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.455+0000 7fcd40e7cb80 10 client.25753 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:02.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.455+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3863003729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560c4aaabb60 con 0x560c4ad2f860 2024-05-08T21:20:02.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.455+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcd180a11a0 0x7fcd180a3660 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009990 tx=0x7fcd20004300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.455+0000 7fcd15ffb700 1 -- 172.21.15.33:0/3863003729 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560c4abd7270 con 0x560c4ad2f860 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.456+0000 7fcd15ffb700 1 -- 172.21.15.33:0/3863003729 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcd28084160 con 0x560c4ad2f860 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.456+0000 7fcd40e7cb80 10 client.25753 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.456+0000 7fcd40e7cb80 10 client.25753 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.456+0000 7fcd40e7cb80 20 client.25753 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.456+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x560c4ad5d480 0x560c4ad5f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.456+0000 7fcd40e7cb80 4 client.25753 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.456+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3863003729 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c4acfd810 con 0x560c4ad5d480 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.456+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x560c4ad5d480 0x560c4ad5f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.457+0000 7fcd15ffb700 10 client.25753 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.457+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x560c4ad5d480 0x560c4ad5f8a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e7dce83990b44d17 server_cookie=283a8e9b17e7365e in_seq=0 out_seq=0 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.457+0000 7fcd15ffb700 1 -- 172.21.15.33:0/3863003729 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c4acfd810 con 0x560c4ad5d480 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.457+0000 7fcd15ffb700 10 client.25753 handle_command_reply: tid=0 2024-05-08T21:20:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.457+0000 7fcd40e7cb80 1 client.25753 shutdown 2024-05-08T21:20:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.457+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3863003729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2f860 msgr2=0x560c4ad58b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.457+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2f860 0x560c4ad58b70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd28096060 tx=0x7fcd280b7ec0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.458+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3863003729 shutdown_connections 2024-05-08T21:20:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.458+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x560c4ad5d480 0x560c4ad5f8a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.458+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2f860 0x560c4ad58b70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.458+0000 7fcd15ffb700 0 client.25753 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:20:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.458+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2e540 0x560c4ad515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.458+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3863003729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2eea0 0x560c4ad51b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.458+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3863003729 >> 172.21.15.33:0/3863003729 conn(0x560c4ac8c210 msgr2=0x560c4ad285a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.458+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3863003729 shutdown_connections 2024-05-08T21:20:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.458+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3863003729 wait complete. 2024-05-08T21:20:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.458+0000 7fcd40e7cb80 20 client.25753 trim_cache size 0 max 16384 2024-05-08T21:20:02.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.472+0000 7fcd40e7cb80 1 Processor -- start 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.472+0000 7fcd40e7cb80 1 -- start start 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.472+0000 7fcd40e7cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 0x560c4ad59220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd40e7cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2eea0 0x560c4ad58ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd40e7cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2f860 0x560c4ad2fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd40e7cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ad2a570 con 0x560c4ad2f860 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd40e7cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ad29b60 con 0x560c4ad2eea0 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd40e7cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ad64ab0 con 0x560c4ad2e540 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd2e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 0x560c4ad59220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd2e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 0x560c4ad59220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51304/0 (socket says 172.21.15.33:51304) 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd2dd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2eea0 0x560c4ad58ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd2e59c700 1 -- 172.21.15.33:0/3602952085 learned_addr learned my addr 172.21.15.33:0/3602952085 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd2ed9d700 1 --2- 172.21.15.33:0/3602952085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2f860 0x560c4ad2fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd2e59c700 1 -- 172.21.15.33:0/3602952085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2eea0 msgr2=0x560c4ad58ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/3602952085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2eea0 0x560c4ad58ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd2e59c700 1 -- 172.21.15.33:0/3602952085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2f860 msgr2=0x560c4ad2fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/3602952085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2f860 0x560c4ad2fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd2e59c700 1 -- 172.21.15.33:0/3602952085 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c4ac44560 con 0x560c4ad2e540 2024-05-08T21:20:02.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.473+0000 7fcd2ed9d700 1 --2- 172.21.15.33:0/3602952085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2f860 0x560c4ad2fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:20:02.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.474+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/3602952085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 0x560c4ad59220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd200059d0 tx=0x7fcd20022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a99d4ad484d5420 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.474+0000 7fcd1d7fa700 1 -- 172.21.15.33:0/3602952085 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20039040 con 0x560c4ad2e540 2024-05-08T21:20:02.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.474+0000 7fcd1d7fa700 1 -- 172.21.15.33:0/3602952085 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd2003e8f0 con 0x560c4ad2e540 2024-05-08T21:20:02.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.474+0000 7fcd1d7fa700 1 -- 172.21.15.33:0/3602952085 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd200055e0 con 0x560c4ad2e540 2024-05-08T21:20:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.475+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3602952085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 msgr2=0x560c4ad59220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.475+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3602952085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 0x560c4ad59220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd200059d0 tx=0x7fcd20022000 comp rx=0 tx=0).stop 2024-05-08T21:20:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.475+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3602952085 shutdown_connections 2024-05-08T21:20:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.475+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3602952085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2f860 0x560c4ad2fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.475+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3602952085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2e540 0x560c4ad59220 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.475+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3602952085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2eea0 0x560c4ad58ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.475+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3602952085 >> 172.21.15.33:0/3602952085 conn(0x560c4ac8c210 msgr2=0x560c4ad25210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.475+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3602952085 shutdown_connections 2024-05-08T21:20:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.475+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3602952085 wait complete. 2024-05-08T21:20:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.475+0000 7fcd40e7cb80 1 -- start start 2024-05-08T21:20:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.475+0000 7fcd40e7cb80 1 Processor -- start 2024-05-08T21:20:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.475+0000 7fcd40e7cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2e540 0x560c4ad54cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd40e7cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2eea0 0x560c4ad57210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd40e7cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2f860 0x560c4ad57790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd40e7cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ad57ea0 con 0x560c4ad2e540 2024-05-08T21:20:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd40e7cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ad58010 con 0x560c4ad2f860 2024-05-08T21:20:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd40e7cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c4ad581d0 con 0x560c4ad2eea0 2024-05-08T21:20:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd40e7cb80 10 client.0 _ll_register_callbacks cb 0x560c4ac23b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd2e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2e540 0x560c4ad54cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd2dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2eea0 0x560c4ad57210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd2dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2eea0 0x560c4ad57210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51312/0 (socket says 172.21.15.33:51312) 2024-05-08T21:20:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd2e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2e540 0x560c4ad54cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50562/0 (socket says 172.21.15.33:50562) 2024-05-08T21:20:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd2dd9b700 1 -- 172.21.15.33:0/3759072389 learned_addr learned my addr 172.21.15.33:0/3759072389 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd2ed9d700 1 --2- 172.21.15.33:0/3759072389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2f860 0x560c4ad57790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd2dd9b700 1 -- 172.21.15.33:0/3759072389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2f860 msgr2=0x560c4ad57790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd2dd9b700 1 --2- 172.21.15.33:0/3759072389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2f860 0x560c4ad57790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd2dd9b700 1 -- 172.21.15.33:0/3759072389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2e540 msgr2=0x560c4ad54cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd2dd9b700 1 --2- 172.21.15.33:0/3759072389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2e540 0x560c4ad54cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.476+0000 7fcd2dd9b700 1 -- 172.21.15.33:0/3759072389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c4aaff3b0 con 0x560c4ad2eea0 2024-05-08T21:20:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd2dd9b700 1 --2- 172.21.15.33:0/3759072389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2eea0 0x560c4ad57210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd24012700 tx=0x7fcd24012a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e79b3bd1660b8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd15ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:20:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd15ffb700 1 -- 172.21.15.33:0/3759072389 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24017070 con 0x560c4ad2eea0 2024-05-08T21:20:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3759072389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd2001e070 con 0x560c4ad2eea0 2024-05-08T21:20:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd15ffb700 1 -- 172.21.15.33:0/3759072389 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcd240120d0 con 0x560c4ad2eea0 2024-05-08T21:20:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd15ffb700 1 -- 172.21.15.33:0/3759072389 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd24012290 con 0x560c4ad2eea0 2024-05-08T21:20:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd40e7cb80 10 client.25759 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:02.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3759072389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560c4ab03cd0 con 0x560c4ad2eea0 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd15ffb700 1 -- 172.21.15.33:0/3759072389 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd2001e070 con 0x560c4ad2eea0 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd15ffb700 1 -- 172.21.15.33:0/3759072389 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcd240048e0 con 0x560c4ad2eea0 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd40e7cb80 10 client.25759 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd40e7cb80 10 client.25759 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd40e7cb80 20 client.25759 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3759072389 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x560c4ad4b4e0 0x560c4ad4d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd40e7cb80 4 client.25759 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.477+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3759072389 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560c4ac76ec0 con 0x560c4ad4b4e0 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.478+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/3759072389 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x560c4ad4b4e0 0x560c4ad4d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.478+0000 7fcd2e59c700 1 --2- 172.21.15.33:0/3759072389 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x560c4ad4b4e0 0x560c4ad4d900 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f7bad3ef3678a1fa server_cookie=815d5737a89eecc8 in_seq=0 out_seq=0 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.478+0000 7fcd15ffb700 10 client.25759 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.478+0000 7fcd15ffb700 1 -- 172.21.15.33:0/3759072389 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560c4ac76ec0 con 0x560c4ad4b4e0 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.478+0000 7fcd15ffb700 10 client.25759 handle_command_reply: tid=0 2024-05-08T21:20:02.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.478+0000 7fcd40e7cb80 1 client.25759 shutdown 2024-05-08T21:20:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3759072389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2eea0 msgr2=0x560c4ad57210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3759072389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2eea0 0x560c4ad57210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd24012700 tx=0x7fcd24012a10 comp rx=0 tx=0).stop 2024-05-08T21:20:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3759072389 shutdown_connections 2024-05-08T21:20:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3759072389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c4ad2f860 0x560c4ad57790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3759072389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560c4ad2e540 0x560c4ad54cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3759072389 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x560c4ad4b4e0 0x560c4ad4d900 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd40e7cb80 1 --2- 172.21.15.33:0/3759072389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c4ad2eea0 0x560c4ad57210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3759072389 >> 172.21.15.33:0/3759072389 conn(0x560c4ac8c210 msgr2=0x560c4ad2fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd15ffb700 0 client.25759 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:20:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3759072389 shutdown_connections 2024-05-08T21:20:02.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd40e7cb80 1 -- 172.21.15.33:0/3759072389 wait complete. 2024-05-08T21:20:02.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd40e7cb80 20 client.25759 trim_cache size 0 max 16384 2024-05-08T21:20:02.487 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:20:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd34cc2700 1 -- 172.21.15.33:0/1806932548 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcd180a11a0 msgr2=0x7fcd180a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd34cc2700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcd180a11a0 0x7fcd180a3660 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009990 tx=0x7fcd20004300 comp rx=0 tx=0).stop 2024-05-08T21:20:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd34cc2700 1 -- 172.21.15.33:0/1806932548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd30102990 msgr2=0x7fcd300738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.479+0000 7fcd34cc2700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd30102990 0x7fcd300738d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800b970 tx=0x7fcd2800bd30 comp rx=0 tx=0).stop 2024-05-08T21:20:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.480+0000 7fcd34cc2700 1 -- 172.21.15.33:0/1806932548 shutdown_connections 2024-05-08T21:20:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.480+0000 7fcd34cc2700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcd180a11a0 0x7fcd180a3660 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.480+0000 7fcd34cc2700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd30102990 0x7fcd300738d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.480+0000 7fcd34cc2700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd30100e80 0x7fcd30074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.480+0000 7fcd34cc2700 1 --2- 172.21.15.33:0/1806932548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd30101fd0 0x7fcd30073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.480+0000 7fcd34cc2700 1 -- 172.21.15.33:0/1806932548 >> 172.21.15.33:0/1806932548 conn(0x7fcd300fc3f0 msgr2=0x7fcd301054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.480+0000 7fcd34cc2700 1 -- 172.21.15.33:0/1806932548 shutdown_connections 2024-05-08T21:20:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.480+0000 7fcd34cc2700 1 -- 172.21.15.33:0/1806932548 wait complete. 2024-05-08T21:20:02.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:20:02.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:02 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/222165013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:20:02.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:02 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/222165013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:20:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.655+0000 7f6ed54b1700 1 -- 172.21.15.33:0/3246365901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ed0074fe0 msgr2=0x7f6ed0073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.655+0000 7f6ed54b1700 1 --2- 172.21.15.33:0/3246365901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ed0074fe0 0x7f6ed0073430 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009990 tx=0x7f6ec0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.655+0000 7f6ed54b1700 1 -- 172.21.15.33:0/3246365901 shutdown_connections 2024-05-08T21:20:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.655+0000 7f6ed54b1700 1 --2- 172.21.15.33:0/3246365901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed010d440 0x7f6ed010f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.655+0000 7f6ed54b1700 1 --2- 172.21.15.33:0/3246365901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ed0074fe0 0x7f6ed0073430 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.655+0000 7f6ed54b1700 1 --2- 172.21.15.33:0/3246365901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ed0073a00 0x7f6ed0073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.655+0000 7f6ed54b1700 1 -- 172.21.15.33:0/3246365901 >> 172.21.15.33:0/3246365901 conn(0x7f6ed00fc400 msgr2=0x7f6ed00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.655+0000 7f6ed54b1700 1 -- 172.21.15.33:0/3246365901 shutdown_connections 2024-05-08T21:20:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.655+0000 7f6ed54b1700 1 -- 172.21.15.33:0/3246365901 wait complete. 2024-05-08T21:20:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6ed54b1700 1 Processor -- start 2024-05-08T21:20:02.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6ed54b1700 1 -- start start 2024-05-08T21:20:02.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6ed54b1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ed0073a00 0x7f6ed019d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6ed54b1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ed0074fe0 0x7f6ed019d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6ed54b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed010d440 0x7f6ed01a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6ed54b1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed01a4e90 con 0x7f6ed010d440 2024-05-08T21:20:02.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6ed54b1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed01a4fd0 con 0x7f6ed0073a00 2024-05-08T21:20:02.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6ed54b1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed01a5160 con 0x7f6ed0074fe0 2024-05-08T21:20:02.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6ecf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed010d440 0x7f6ed01a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6ecf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed010d440 0x7f6ed01a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50584/0 (socket says 172.21.15.33:50584) 2024-05-08T21:20:02.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6ecf7fe700 1 -- 172.21.15.33:0/2789231407 learned_addr learned my addr 172.21.15.33:0/2789231407 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6ece7fc700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ed0074fe0 0x7f6ed019d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.656+0000 7f6eceffd700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ed0073a00 0x7f6ed019d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.657+0000 7f6ecf7fe700 1 -- 172.21.15.33:0/2789231407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ed0074fe0 msgr2=0x7f6ed019d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.657+0000 7f6ecf7fe700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ed0074fe0 0x7f6ed019d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.657+0000 7f6ecf7fe700 1 -- 172.21.15.33:0/2789231407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ed0073a00 msgr2=0x7f6ed019d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.657+0000 7f6ecf7fe700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ed0073a00 0x7f6ed019d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.657+0000 7f6ecf7fe700 1 -- 172.21.15.33:0/2789231407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec00095f0 con 0x7f6ed010d440 2024-05-08T21:20:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.657+0000 7f6ecf7fe700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed010d440 0x7f6ed01a48c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800b970 tx=0x7f6ec800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.657+0000 7f6eb7fff700 1 -- 172.21.15.33:0/2789231407 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800c930 con 0x7f6ed010d440 2024-05-08T21:20:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.657+0000 7f6eb7fff700 1 -- 172.21.15.33:0/2789231407 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ec8014440 con 0x7f6ed010d440 2024-05-08T21:20:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.658+0000 7f6ed54b1700 1 -- 172.21.15.33:0/2789231407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed01a5440 con 0x7f6ed010d440 2024-05-08T21:20:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.658+0000 7f6ed54b1700 1 -- 172.21.15.33:0/2789231407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed01a59d0 con 0x7f6ed010d440 2024-05-08T21:20:02.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.658+0000 7f6eb7fff700 1 -- 172.21.15.33:0/2789231407 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80075d0 con 0x7f6ed010d440 2024-05-08T21:20:02.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.660+0000 7f6eb7fff700 1 -- 172.21.15.33:0/2789231407 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6ec8007730 con 0x7f6ed010d440 2024-05-08T21:20:02.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.661+0000 7f6eb7fff700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6eb80a11a0 0x7f6eb80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.661+0000 7f6eb7fff700 1 -- 172.21.15.33:0/2789231407 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f6ec8006e40 con 0x7f6ed010d440 2024-05-08T21:20:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.662+0000 7f6eceffd700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6eb80a11a0 0x7f6eb80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.662+0000 7f6eceffd700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6eb80a11a0 0x7f6eb80a3660 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0005910 tx=0x7f6ec000b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.663+0000 7f6ee166bb80 1 Processor -- start 2024-05-08T21:20:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.664+0000 7f6ee166bb80 1 -- start start 2024-05-08T21:20:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.664+0000 7f6ee166bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041540 0x5615ff041960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.664+0000 7f6ee166bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff041ea0 0x5615ff042320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.664+0000 7f6ee166bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff042860 0x5615ff042ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.664+0000 7f6ee166bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fef98150 con 0x5615ff041540 2024-05-08T21:20:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.664+0000 7f6ee166bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fef95b10 con 0x5615ff042860 2024-05-08T21:20:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.664+0000 7f6ee166bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fef91400 con 0x5615ff041ea0 2024-05-08T21:20:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.664+0000 7f6eceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041540 0x5615ff041960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.664+0000 7f6eceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041540 0x5615ff041960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50586/0 (socket says 172.21.15.33:50586) 2024-05-08T21:20:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.664+0000 7f6eceffd700 1 -- 172.21.15.33:0/3223841754 learned_addr learned my addr 172.21.15.33:0/3223841754 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.664+0000 7f6ecf7fe700 1 --2- 172.21.15.33:0/3223841754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff042860 0x5615ff042ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.664+0000 7f6ece7fc700 1 --2- 172.21.15.33:0/3223841754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff041ea0 0x5615ff042320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.665+0000 7f6eceffd700 1 -- 172.21.15.33:0/3223841754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff041ea0 msgr2=0x5615ff042320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.665+0000 7f6eceffd700 1 --2- 172.21.15.33:0/3223841754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff041ea0 0x5615ff042320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.665+0000 7f6eceffd700 1 -- 172.21.15.33:0/3223841754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff042860 msgr2=0x5615ff042ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.665+0000 7f6eceffd700 1 --2- 172.21.15.33:0/3223841754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff042860 0x5615ff042ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.665+0000 7f6eceffd700 1 -- 172.21.15.33:0/3223841754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615fee123b0 con 0x5615ff041540 2024-05-08T21:20:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.665+0000 7f6eceffd700 1 --2- 172.21.15.33:0/3223841754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041540 0x5615ff041960 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009990 tx=0x7f6ec0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45300aa215554665 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.665+0000 7f6eb57fa700 1 -- 172.21.15.33:0/3223841754 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0003c60 con 0x5615ff041540 2024-05-08T21:20:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.665+0000 7f6eb57fa700 1 -- 172.21.15.33:0/3223841754 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ec001e040 con 0x5615ff041540 2024-05-08T21:20:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.665+0000 7f6eb57fa700 1 -- 172.21.15.33:0/3223841754 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00049c0 con 0x5615ff041540 2024-05-08T21:20:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.666+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3223841754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041540 msgr2=0x5615ff041960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.666+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3223841754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041540 0x5615ff041960 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009990 tx=0x7f6ec0020000 comp rx=0 tx=0).stop 2024-05-08T21:20:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.666+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3223841754 shutdown_connections 2024-05-08T21:20:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.666+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3223841754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff042860 0x5615ff042ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.666+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3223841754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041540 0x5615ff041960 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.666+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3223841754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff041ea0 0x5615ff042320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.666+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3223841754 >> 172.21.15.33:0/3223841754 conn(0x5615fef9f210 msgr2=0x5615ff03bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.667+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3223841754 shutdown_connections 2024-05-08T21:20:02.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.667+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3223841754 wait complete. 2024-05-08T21:20:02.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.667+0000 7f6ee166bb80 1 -- start start 2024-05-08T21:20:02.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.667+0000 7f6ee166bb80 1 Processor -- start 2024-05-08T21:20:02.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ee166bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff041540 0x5615ff05bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ee166bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff05c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ee166bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff042860 0x5615ff063270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ee166bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fef2d560 con 0x5615ff041ea0 2024-05-08T21:20:02.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ee166bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fef2d200 con 0x5615ff042860 2024-05-08T21:20:02.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ee166bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615fee146a0 con 0x5615ff041540 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ee166bb80 10 client.0 _ll_register_callbacks cb 0x5615fef36b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ecf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff042860 0x5615ff063270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ecf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff042860 0x5615ff063270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:34884/0 (socket says 172.21.15.33:34884) 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ecf7fe700 1 -- 172.21.15.33:0/3876013997 learned_addr learned my addr 172.21.15.33:0/3876013997 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ece7fc700 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff05c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6eceffd700 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff041540 0x5615ff05bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ecf7fe700 1 -- 172.21.15.33:0/3876013997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff041540 msgr2=0x5615ff05bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ecf7fe700 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff041540 0x5615ff05bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ecf7fe700 1 -- 172.21.15.33:0/3876013997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 msgr2=0x5615ff05c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ecf7fe700 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff05c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ecf7fe700 1 -- 172.21.15.33:0/3876013997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed01101e0 con 0x5615ff042860 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6eceffd700 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff041540 0x5615ff05bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6ecf7fe700 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff042860 0x5615ff063270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8095270 tx=0x7f6ec8099a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcce3dd19d432390 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.668+0000 7f6eb2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:20:02.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.669+0000 7f6eb2ff5700 1 -- 172.21.15.33:0/3876013997 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec809d320 con 0x5615ff042860 2024-05-08T21:20:02.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.669+0000 7f6eb2ff5700 1 -- 172.21.15.33:0/3876013997 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f6ec80a1970 con 0x5615ff042860 2024-05-08T21:20:02.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.669+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3876013997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615feeea270 con 0x5615ff042860 2024-05-08T21:20:02.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.669+0000 7f6eb2ff5700 1 -- 172.21.15.33:0/3876013997 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ec80850b0 con 0x5615ff042860 2024-05-08T21:20:02.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.669+0000 7f6ee166bb80 10 client.25754 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:02.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.669+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3876013997 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5615fedbeb60 con 0x5615ff042860 2024-05-08T21:20:02.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.669+0000 7f6eb2ff5700 1 -- 172.21.15.33:0/3876013997 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5615feeea270 con 0x5615ff042860 2024-05-08T21:20:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.669+0000 7f6eb2ff5700 1 -- 172.21.15.33:0/3876013997 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ec80b7ae0 con 0x5615ff042860 2024-05-08T21:20:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.670+0000 7f6ee166bb80 10 client.25754 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.670+0000 7f6ee166bb80 10 client.25754 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:20:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.670+0000 7f6ee166bb80 20 client.25754 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.670+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5615ff078ba0 0x5615ff07afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.670+0000 7f6ee166bb80 4 client.25754 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.670+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3876013997 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615ff010810 con 0x5615ff078ba0 2024-05-08T21:20:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.670+0000 7f6eceffd700 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5615ff078ba0 0x5615ff07afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.671+0000 7f6eceffd700 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5615ff078ba0 0x5615ff07afc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc628514de980c17 server_cookie=b3df80875d65924 in_seq=0 out_seq=0 2024-05-08T21:20:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.671+0000 7f6eb2ff5700 10 client.25754 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:20:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.671+0000 7f6eb2ff5700 1 -- 172.21.15.33:0/3876013997 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5615ff010810 con 0x5615ff078ba0 2024-05-08T21:20:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.671+0000 7f6eb2ff5700 10 client.25754 handle_command_reply: tid=0 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.671+0000 7f6ee166bb80 1 client.25754 shutdown 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.672+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3876013997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff042860 msgr2=0x5615ff063270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.672+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff042860 0x5615ff063270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8095270 tx=0x7f6ec8099a30 comp rx=0 tx=0).stop 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.672+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3876013997 shutdown_connections 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.672+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5615ff078ba0 0x5615ff07afc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.672+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff042860 0x5615ff063270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.672+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff041540 0x5615ff05bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.672+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3876013997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff05c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.672+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3876013997 >> 172.21.15.33:0/3876013997 conn(0x5615fef9f210 msgr2=0x5615ff03b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.672+0000 7f6eb2ff5700 0 client.25754 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.672+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3876013997 shutdown_connections 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.672+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3876013997 wait complete. 2024-05-08T21:20:02.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.672+0000 7f6ee166bb80 20 client.25754 trim_cache size 0 max 16384 2024-05-08T21:20:02.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.699+0000 7f6ee166bb80 1 Processor -- start 2024-05-08T21:20:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.699+0000 7f6ee166bb80 1 -- start start 2024-05-08T21:20:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.699+0000 7f6ee166bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff041540 0x5615ff048cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.699+0000 7f6ee166bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff063190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.699+0000 7f6ee166bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 0x5615ff05bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.699+0000 7f6ee166bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ff05ce70 con 0x5615ff041ea0 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.699+0000 7f6ee166bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ff05ace0 con 0x5615ff041540 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.699+0000 7f6ee166bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ff036800 con 0x5615ff042860 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.700+0000 7f6ece7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff063190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.700+0000 7f6ece7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff063190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50612/0 (socket says 172.21.15.33:50612) 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.700+0000 7f6ece7fc700 1 -- 172.21.15.33:0/3798528330 learned_addr learned my addr 172.21.15.33:0/3798528330 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.700+0000 7f6ecf7fe700 1 --2- 172.21.15.33:0/3798528330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 0x5615ff05bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.700+0000 7f6eceffd700 1 --2- 172.21.15.33:0/3798528330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff041540 0x5615ff048cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.700+0000 7f6ecf7fe700 1 -- 172.21.15.33:0/3798528330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff041540 msgr2=0x5615ff048cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.700+0000 7f6ecf7fe700 1 --2- 172.21.15.33:0/3798528330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff041540 0x5615ff048cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.700+0000 7f6ecf7fe700 1 -- 172.21.15.33:0/3798528330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 msgr2=0x5615ff063190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.700+0000 7f6ecf7fe700 1 --2- 172.21.15.33:0/3798528330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff063190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.700+0000 7f6ecf7fe700 1 -- 172.21.15.33:0/3798528330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615fef57560 con 0x5615ff042860 2024-05-08T21:20:02.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.700+0000 7f6ecf7fe700 1 --2- 172.21.15.33:0/3798528330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 0x5615ff05bcf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800bda0 tx=0x7f6ec8097aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a623c51354326819 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.701+0000 7f6eb5ffb700 1 -- 172.21.15.33:0/3798528330 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80a1970 con 0x5615ff042860 2024-05-08T21:20:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.701+0000 7f6eb5ffb700 1 -- 172.21.15.33:0/3798528330 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ec809d320 con 0x5615ff042860 2024-05-08T21:20:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.701+0000 7f6eb5ffb700 1 -- 172.21.15.33:0/3798528330 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80cc2a0 con 0x5615ff042860 2024-05-08T21:20:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.701+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3798528330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 msgr2=0x5615ff05bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.701+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3798528330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 0x5615ff05bcf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800bda0 tx=0x7f6ec8097aa0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3798528330 shutdown_connections 2024-05-08T21:20:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3798528330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 0x5615ff05bcf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3798528330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff041540 0x5615ff048cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/3798528330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff063190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3798528330 >> 172.21.15.33:0/3798528330 conn(0x5615fef9f210 msgr2=0x5615ff042c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3798528330 shutdown_connections 2024-05-08T21:20:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 -- 172.21.15.33:0/3798528330 wait complete. 2024-05-08T21:20:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 -- start start 2024-05-08T21:20:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 Processor -- start 2024-05-08T21:20:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff041540 0x5615ff05dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff05e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 0x5615ff0cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ff05eac0 con 0x5615ff041ea0 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ff05ec30 con 0x5615ff041540 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615ff0cd640 con 0x5615ff042860 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.702+0000 7f6ee166bb80 10 client.0 _ll_register_callbacks cb 0x5615fef36b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6ecf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 0x5615ff0cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6eceffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff041540 0x5615ff05dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6ece7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff05e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6ecf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 0x5615ff0cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51364/0 (socket says 172.21.15.33:51364) 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6ecf7fe700 1 -- 172.21.15.33:0/1319525869 learned_addr learned my addr 172.21.15.33:0/1319525869 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6eceffd700 1 -- 172.21.15.33:0/1319525869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 msgr2=0x5615ff0cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6eceffd700 1 --2- 172.21.15.33:0/1319525869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 0x5615ff0cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6eceffd700 1 -- 172.21.15.33:0/1319525869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 msgr2=0x5615ff05e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6eceffd700 1 --2- 172.21.15.33:0/1319525869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff05e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6eceffd700 1 -- 172.21.15.33:0/1319525869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615fedbeb60 con 0x5615ff041540 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6ecf7fe700 1 --2- 172.21.15.33:0/1319525869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 0x5615ff0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:02.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6eceffd700 1 --2- 172.21.15.33:0/1319525869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff041540 0x5615ff05dfb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0000c00 tx=0x7f6ec0004160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4dcb29e5735d617 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6eb2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6eb2ff5700 1 -- 172.21.15.33:0/1319525869 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000bb20 con 0x5615ff041540 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.703+0000 7f6eb2ff5700 1 -- 172.21.15.33:0/1319525869 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f6ec000bc80 con 0x5615ff041540 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.704+0000 7f6ee166bb80 1 -- 172.21.15.33:0/1319525869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ec001e040 con 0x5615ff041540 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.704+0000 7f6eb2ff5700 1 -- 172.21.15.33:0/1319525869 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ec0004500 con 0x5615ff041540 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.704+0000 7f6ee166bb80 10 client.25760 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.704+0000 7f6ee166bb80 1 -- 172.21.15.33:0/1319525869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5615fee16cd0 con 0x5615ff041540 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.704+0000 7f6eb2ff5700 1 -- 172.21.15.33:0/1319525869 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ec001e040 con 0x5615ff041540 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.704+0000 7f6eb2ff5700 1 -- 172.21.15.33:0/1319525869 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ec0005520 con 0x5615ff041540 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.704+0000 7f6ee166bb80 10 client.25760 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.704+0000 7f6ee166bb80 10 client.25760 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.704+0000 7f6ee166bb80 20 client.25760 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.704+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/1319525869 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5615ff0cec00 0x5615ff0d1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.704+0000 7f6ee166bb80 4 client.25760 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:20:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.704+0000 7f6ee166bb80 1 -- 172.21.15.33:0/1319525869 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5615fef89ec0 con 0x5615ff0cec00 2024-05-08T21:20:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.705+0000 7f6ece7fc700 1 --2- 172.21.15.33:0/1319525869 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5615ff0cec00 0x5615ff0d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.705+0000 7f6ece7fc700 1 --2- 172.21.15.33:0/1319525869 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5615ff0cec00 0x5615ff0d1020 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8558bc4ff405b373 server_cookie=211ae76dd6349368 in_seq=0 out_seq=0 2024-05-08T21:20:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.705+0000 7f6eb2ff5700 10 client.25760 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:20:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.705+0000 7f6eb2ff5700 1 -- 172.21.15.33:0/1319525869 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5615fef89ec0 con 0x5615ff0cec00 2024-05-08T21:20:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.705+0000 7f6eb2ff5700 10 client.25760 handle_command_reply: tid=0 2024-05-08T21:20:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.705+0000 7f6ee166bb80 1 client.25760 shutdown 2024-05-08T21:20:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.706+0000 7f6ee166bb80 1 -- 172.21.15.33:0/1319525869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff041540 msgr2=0x5615ff05dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.706+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/1319525869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff041540 0x5615ff05dfb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0000c00 tx=0x7f6ec0004160 comp rx=0 tx=0).stop 2024-05-08T21:20:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.706+0000 7f6ee166bb80 1 -- 172.21.15.33:0/1319525869 shutdown_connections 2024-05-08T21:20:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.706+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/1319525869 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5615ff0cec00 0x5615ff0d1020 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.706+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/1319525869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5615ff042860 0x5615ff0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.706+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/1319525869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615ff041540 0x5615ff05dfb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.706+0000 7f6ee166bb80 1 --2- 172.21.15.33:0/1319525869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615ff041ea0 0x5615ff05e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.706+0000 7f6ee166bb80 1 -- 172.21.15.33:0/1319525869 >> 172.21.15.33:0/1319525869 conn(0x5615fef9f210 msgr2=0x5615ff05aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.706+0000 7f6eb2ff5700 0 client.25760 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:20:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.706+0000 7f6ee166bb80 1 -- 172.21.15.33:0/1319525869 shutdown_connections 2024-05-08T21:20:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.706+0000 7f6ee166bb80 1 -- 172.21.15.33:0/1319525869 wait complete. 2024-05-08T21:20:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.706+0000 7f6ee166bb80 20 client.25760 trim_cache size 0 max 16384 2024-05-08T21:20:02.712 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:20:02.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.707+0000 7f6ed54b1700 1 -- 172.21.15.33:0/2789231407 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6eb80a11a0 msgr2=0x7f6eb80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.707+0000 7f6ed54b1700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6eb80a11a0 0x7f6eb80a3660 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0005910 tx=0x7f6ec000b610 comp rx=0 tx=0).stop 2024-05-08T21:20:02.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.707+0000 7f6ed54b1700 1 -- 172.21.15.33:0/2789231407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed010d440 msgr2=0x7f6ed01a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.707+0000 7f6ed54b1700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed010d440 0x7f6ed01a48c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800b970 tx=0x7f6ec800bc80 comp rx=0 tx=0).stop 2024-05-08T21:20:02.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.707+0000 7f6ed54b1700 1 -- 172.21.15.33:0/2789231407 shutdown_connections 2024-05-08T21:20:02.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.707+0000 7f6ed54b1700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed010d440 0x7f6ed01a48c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.707+0000 7f6ed54b1700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6eb80a11a0 0x7f6eb80a3660 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.707+0000 7f6ed54b1700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ed0073a00 0x7f6ed019d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.707+0000 7f6ed54b1700 1 --2- 172.21.15.33:0/2789231407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ed0074fe0 0x7f6ed019d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.707+0000 7f6ed54b1700 1 -- 172.21.15.33:0/2789231407 >> 172.21.15.33:0/2789231407 conn(0x7f6ed00fc400 msgr2=0x7f6ed01007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.707+0000 7f6ed54b1700 1 -- 172.21.15.33:0/2789231407 shutdown_connections 2024-05-08T21:20:02.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.707+0000 7f6ed54b1700 1 -- 172.21.15.33:0/2789231407 wait complete. 2024-05-08T21:20:02.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:20:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.911+0000 7fa42765b700 1 -- 172.21.15.33:0/2291709668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa42010f330 msgr2=0x7fa4201117a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.911+0000 7fa42765b700 1 --2- 172.21.15.33:0/2291709668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa42010f330 0x7fa4201117a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa41800b0a0 tx=0x7fa41800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.911+0000 7fa42765b700 1 -- 172.21.15.33:0/2291709668 shutdown_connections 2024-05-08T21:20:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.911+0000 7fa42765b700 1 --2- 172.21.15.33:0/2291709668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa42010f330 0x7fa4201117a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.911+0000 7fa42765b700 1 --2- 172.21.15.33:0/2291709668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa420072ce0 0x7fa420107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.911+0000 7fa42765b700 1 --2- 172.21.15.33:0/2291709668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4201084e0 0x7fa420108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.911+0000 7fa42765b700 1 -- 172.21.15.33:0/2291709668 >> 172.21.15.33:0/2291709668 conn(0x7fa42006dcb0 msgr2=0x7fa4200700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.911+0000 7fa42765b700 1 -- 172.21.15.33:0/2291709668 shutdown_connections 2024-05-08T21:20:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.911+0000 7fa42765b700 1 -- 172.21.15.33:0/2291709668 wait complete. 2024-05-08T21:20:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.912+0000 7fa42765b700 1 Processor -- start 2024-05-08T21:20:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.912+0000 7fa42765b700 1 -- start start 2024-05-08T21:20:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.912+0000 7fa42765b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa420072ce0 0x7fa42010abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.912+0000 7fa42765b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4201084e0 0x7fa420109270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.912+0000 7fa42765b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42010f330 0x7fa4201097b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.912+0000 7fa42765b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4201a3460 con 0x7fa4201084e0 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.912+0000 7fa42765b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4201a35d0 con 0x7fa420072ce0 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.912+0000 7fa42765b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4201a3790 con 0x7fa42010f330 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.913+0000 7fa424bf6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4201084e0 0x7fa420109270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.913+0000 7fa425bf8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42010f330 0x7fa4201097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.913+0000 7fa424bf6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4201084e0 0x7fa420109270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50650/0 (socket says 172.21.15.33:50650) 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.913+0000 7fa424bf6700 1 -- 172.21.15.33:0/1144567333 learned_addr learned my addr 172.21.15.33:0/1144567333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.913+0000 7fa4253f7700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa420072ce0 0x7fa42010abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.913+0000 7fa425bf8700 1 -- 172.21.15.33:0/1144567333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa420072ce0 msgr2=0x7fa42010abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.913+0000 7fa425bf8700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa420072ce0 0x7fa42010abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.913+0000 7fa425bf8700 1 -- 172.21.15.33:0/1144567333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4201084e0 msgr2=0x7fa420109270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.913+0000 7fa425bf8700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4201084e0 0x7fa420109270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.913+0000 7fa425bf8700 1 -- 172.21.15.33:0/1144567333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c009520 con 0x7fa42010f330 2024-05-08T21:20:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.913+0000 7fa4253f7700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa420072ce0 0x7fa42010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.913+0000 7fa425bf8700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42010f330 0x7fa4201097b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa418007af0 tx=0x7fa418007870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.914+0000 7fa4167fc700 1 -- 172.21.15.33:0/1144567333 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41800e040 con 0x7fa42010f330 2024-05-08T21:20:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.914+0000 7fa4167fc700 1 -- 172.21.15.33:0/1144567333 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa41800be40 con 0x7fa42010f330 2024-05-08T21:20:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.914+0000 7fa4167fc700 1 -- 172.21.15.33:0/1144567333 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41801b6e0 con 0x7fa42010f330 2024-05-08T21:20:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.914+0000 7fa42765b700 1 -- 172.21.15.33:0/1144567333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa418009d00 con 0x7fa42010f330 2024-05-08T21:20:02.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.914+0000 7fa42765b700 1 -- 172.21.15.33:0/1144567333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa420109f30 con 0x7fa42010f330 2024-05-08T21:20:02.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.917+0000 7fa4167fc700 1 -- 172.21.15.33:0/1144567333 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa418003c00 con 0x7fa42010f330 2024-05-08T21:20:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.918+0000 7fa4167fc700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa40c0a1120 0x7fa40c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.918+0000 7fa4253f7700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa40c0a1120 0x7fa40c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.918+0000 7fa4167fc700 1 -- 172.21.15.33:0/1144567333 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa4180cddc0 con 0x7fa42010f330 2024-05-08T21:20:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.919+0000 7fa4253f7700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa40c0a1120 0x7fa40c0a35e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009fd0 tx=0x7fa41c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.919+0000 7fa433815b80 1 Processor -- start 2024-05-08T21:20:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.919+0000 7fa433815b80 1 -- start start 2024-05-08T21:20:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa433815b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d582960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa433815b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582ea0 0x55e29d583320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa433815b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d583ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa433815b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29d4d9150 con 0x55e29d583860 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa433815b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29d4d6b10 con 0x55e29d582540 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa433815b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29d4d2400 con 0x55e29d582ea0 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa425bf8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d583ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa425bf8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d583ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50658/0 (socket says 172.21.15.33:50658) 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa425bf8700 1 -- 172.21.15.33:0/1063995945 learned_addr learned my addr 172.21.15.33:0/1063995945 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa424bf6700 1 --2- 172.21.15.33:0/1063995945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582ea0 0x55e29d583320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa4253f7700 1 --2- 172.21.15.33:0/1063995945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d582960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa425bf8700 1 -- 172.21.15.33:0/1063995945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582ea0 msgr2=0x55e29d583320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa425bf8700 1 --2- 172.21.15.33:0/1063995945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582ea0 0x55e29d583320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa425bf8700 1 -- 172.21.15.33:0/1063995945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 msgr2=0x55e29d582960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa425bf8700 1 --2- 172.21.15.33:0/1063995945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d582960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.920+0000 7fa425bf8700 1 -- 172.21.15.33:0/1063995945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e29d3533b0 con 0x55e29d583860 2024-05-08T21:20:02.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.921+0000 7fa425bf8700 1 --2- 172.21.15.33:0/1063995945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d583ce0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa4180a2da0 tx=0x7fa41809ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca0f725a5024f31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.921+0000 7fa40b7fe700 1 -- 172.21.15.33:0/1063995945 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41809b5e0 con 0x55e29d583860 2024-05-08T21:20:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.921+0000 7fa40b7fe700 1 -- 172.21.15.33:0/1063995945 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa41809aee0 con 0x55e29d583860 2024-05-08T21:20:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.921+0000 7fa40b7fe700 1 -- 172.21.15.33:0/1063995945 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41809b3a0 con 0x55e29d583860 2024-05-08T21:20:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.922+0000 7fa433815b80 1 -- 172.21.15.33:0/1063995945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 msgr2=0x55e29d583ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.922+0000 7fa433815b80 1 --2- 172.21.15.33:0/1063995945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d583ce0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa4180a2da0 tx=0x7fa41809ade0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.922+0000 7fa433815b80 1 -- 172.21.15.33:0/1063995945 shutdown_connections 2024-05-08T21:20:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.922+0000 7fa433815b80 1 --2- 172.21.15.33:0/1063995945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d583ce0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.922+0000 7fa433815b80 1 --2- 172.21.15.33:0/1063995945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d582960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.922+0000 7fa433815b80 1 --2- 172.21.15.33:0/1063995945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582ea0 0x55e29d583320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.922+0000 7fa433815b80 1 -- 172.21.15.33:0/1063995945 >> 172.21.15.33:0/1063995945 conn(0x55e29d4e0210 msgr2=0x55e29d57cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.923+0000 7fa433815b80 1 -- 172.21.15.33:0/1063995945 shutdown_connections 2024-05-08T21:20:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.923+0000 7fa433815b80 1 -- 172.21.15.33:0/1063995945 wait complete. 2024-05-08T21:20:02.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.923+0000 7fa433815b80 1 -- start start 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.923+0000 7fa433815b80 1 Processor -- start 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa433815b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d58bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa433815b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d582ea0 0x55e29d58c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa433815b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d583860 0x55e29d593140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa433815b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29d46e560 con 0x55e29d582ea0 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa433815b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29d46e200 con 0x55e29d582540 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa433815b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29d3556a0 con 0x55e29d583860 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa433815b80 10 client.0 _ll_register_callbacks cb 0x55e29d477b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa4253f7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d58bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa4253f7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d58bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:34954/0 (socket says 172.21.15.33:34954) 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa4253f7700 1 -- 172.21.15.33:0/1125872665 learned_addr learned my addr 172.21.15.33:0/1125872665 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa424bf6700 1 --2- 172.21.15.33:0/1125872665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d582ea0 0x55e29d58c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa4253f7700 1 -- 172.21.15.33:0/1125872665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d583860 msgr2=0x55e29d593140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:20:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa4253f7700 1 --2- 172.21.15.33:0/1125872665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d583860 0x55e29d593140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa4253f7700 1 -- 172.21.15.33:0/1125872665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d582ea0 msgr2=0x55e29d58c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa4253f7700 1 --2- 172.21.15.33:0/1125872665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d582ea0 0x55e29d58c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.924+0000 7fa4253f7700 1 -- 172.21.15.33:0/1125872665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa420109f30 con 0x55e29d582540 2024-05-08T21:20:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.925+0000 7fa4253f7700 1 --2- 172.21.15.33:0/1125872665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d58bbc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00ebf0 tx=0x7fa41c013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f0e575dd118485 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.925+0000 7fa408ff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.925+0000 7fa408ff9700 1 -- 172.21.15.33:0/1125872665 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c021040 con 0x55e29d582540 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.925+0000 7fa408ff9700 1 -- 172.21.15.33:0/1125872665 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa41c01bd60 con 0x55e29d582540 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.925+0000 7fa433815b80 1 -- 172.21.15.33:0/1125872665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e29d42b270 con 0x55e29d582540 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.925+0000 7fa408ff9700 1 -- 172.21.15.33:0/1125872665 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa41c027430 con 0x55e29d582540 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.925+0000 7fa433815b80 10 client.25772 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.925+0000 7fa433815b80 1 -- 172.21.15.33:0/1125872665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e29d2ffb60 con 0x55e29d582540 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.926+0000 7fa408ff9700 1 -- 172.21.15.33:0/1125872665 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e29d42b270 con 0x55e29d582540 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.926+0000 7fa408ff9700 1 -- 172.21.15.33:0/1125872665 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa41c02f070 con 0x55e29d582540 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.926+0000 7fa433815b80 10 client.25772 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.926+0000 7fa433815b80 10 client.25772 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.926+0000 7fa433815b80 20 client.25772 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.926+0000 7fa433815b80 1 --2- 172.21.15.33:0/1125872665 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55e29d5979d0 0x55e29d599df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.926+0000 7fa433815b80 4 client.25772 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.926+0000 7fa433815b80 1 -- 172.21.15.33:0/1125872665 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e29d551810 con 0x55e29d5979d0 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.926+0000 7fa424bf6700 1 --2- 172.21.15.33:0/1125872665 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55e29d5979d0 0x55e29d599df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.927+0000 7fa424bf6700 1 --2- 172.21.15.33:0/1125872665 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55e29d5979d0 0x55e29d599df0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38b8c8797400dd1a server_cookie=7b9694c347a8d207 in_seq=0 out_seq=0 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.927+0000 7fa408ff9700 10 client.25772 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.927+0000 7fa408ff9700 1 -- 172.21.15.33:0/1125872665 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e29d551810 con 0x55e29d5979d0 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.927+0000 7fa408ff9700 10 client.25772 handle_command_reply: tid=0 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.927+0000 7fa433815b80 1 client.25772 shutdown 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.928+0000 7fa433815b80 1 -- 172.21.15.33:0/1125872665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 msgr2=0x55e29d58bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.928+0000 7fa433815b80 1 --2- 172.21.15.33:0/1125872665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d58bbc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00ebf0 tx=0x7fa41c013040 comp rx=0 tx=0).stop 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.928+0000 7fa433815b80 1 -- 172.21.15.33:0/1125872665 shutdown_connections 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.928+0000 7fa433815b80 1 --2- 172.21.15.33:0/1125872665 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55e29d5979d0 0x55e29d599df0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.928+0000 7fa433815b80 1 --2- 172.21.15.33:0/1125872665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d583860 0x55e29d593140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.928+0000 7fa433815b80 1 --2- 172.21.15.33:0/1125872665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d58bbc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.928+0000 7fa433815b80 1 --2- 172.21.15.33:0/1125872665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d582ea0 0x55e29d58c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.928+0000 7fa433815b80 1 -- 172.21.15.33:0/1125872665 >> 172.21.15.33:0/1125872665 conn(0x55e29d4e0210 msgr2=0x55e29d57c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.928+0000 7fa408ff9700 0 client.25772 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:20:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.928+0000 7fa433815b80 1 -- 172.21.15.33:0/1125872665 shutdown_connections 2024-05-08T21:20:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.928+0000 7fa433815b80 1 -- 172.21.15.33:0/1125872665 wait complete. 2024-05-08T21:20:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.929+0000 7fa433815b80 20 client.25772 trim_cache size 0 max 16384 2024-05-08T21:20:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa433815b80 1 Processor -- start 2024-05-08T21:20:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa433815b80 1 -- start start 2024-05-08T21:20:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa433815b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d586790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa433815b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582ea0 0x55e29d593100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa433815b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d586cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa433815b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29d594f00 con 0x55e29d583860 2024-05-08T21:20:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa433815b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29d58b780 con 0x55e29d582540 2024-05-08T21:20:02.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa433815b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29d57e570 con 0x55e29d582ea0 2024-05-08T21:20:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa425bf8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d586cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa4253f7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d586790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa425bf8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d586cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50674/0 (socket says 172.21.15.33:50674) 2024-05-08T21:20:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa4253f7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d586790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:34956/0 (socket says 172.21.15.33:34956) 2024-05-08T21:20:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.943+0000 7fa425bf8700 1 -- 172.21.15.33:0/2072400162 learned_addr learned my addr 172.21.15.33:0/2072400162 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.944+0000 7fa4253f7700 1 -- 172.21.15.33:0/2072400162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582ea0 msgr2=0x55e29d593100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:20:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.944+0000 7fa4253f7700 1 --2- 172.21.15.33:0/2072400162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582ea0 0x55e29d593100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.944+0000 7fa4253f7700 1 -- 172.21.15.33:0/2072400162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 msgr2=0x55e29d586cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.944+0000 7fa4253f7700 1 --2- 172.21.15.33:0/2072400162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d586cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.944+0000 7fa4253f7700 1 -- 172.21.15.33:0/2072400162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e29d498560 con 0x55e29d582540 2024-05-08T21:20:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.944+0000 7fa4253f7700 1 --2- 172.21.15.33:0/2072400162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d586790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa41c031020 tx=0x7fa41c011d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=605d51559b99ee4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.944+0000 7fa40bfff700 1 -- 172.21.15.33:0/2072400162 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c021070 con 0x55e29d582540 2024-05-08T21:20:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.944+0000 7fa40bfff700 1 -- 172.21.15.33:0/2072400162 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa41c0092d0 con 0x55e29d582540 2024-05-08T21:20:02.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.944+0000 7fa40bfff700 1 -- 172.21.15.33:0/2072400162 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c01aed0 con 0x55e29d582540 2024-05-08T21:20:02.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.945+0000 7fa433815b80 1 -- 172.21.15.33:0/2072400162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 msgr2=0x55e29d586790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.945+0000 7fa433815b80 1 --2- 172.21.15.33:0/2072400162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d586790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa41c031020 tx=0x7fa41c011d90 comp rx=0 tx=0).stop 2024-05-08T21:20:02.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.945+0000 7fa433815b80 1 -- 172.21.15.33:0/2072400162 shutdown_connections 2024-05-08T21:20:02.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.945+0000 7fa433815b80 1 --2- 172.21.15.33:0/2072400162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d586cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.946+0000 7fa433815b80 1 --2- 172.21.15.33:0/2072400162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582540 0x55e29d586790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.946+0000 7fa433815b80 1 --2- 172.21.15.33:0/2072400162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582ea0 0x55e29d593100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.946+0000 7fa433815b80 1 -- 172.21.15.33:0/2072400162 >> 172.21.15.33:0/2072400162 conn(0x55e29d4e0210 msgr2=0x55e29d589da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.946+0000 7fa433815b80 1 -- 172.21.15.33:0/2072400162 shutdown_connections 2024-05-08T21:20:02.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.946+0000 7fa433815b80 1 -- 172.21.15.33:0/2072400162 wait complete. 2024-05-08T21:20:02.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.946+0000 7fa433815b80 1 -- start start 2024-05-08T21:20:02.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.946+0000 7fa433815b80 1 Processor -- start 2024-05-08T21:20:02.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa433815b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582540 0x55e29d58f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa433815b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582ea0 0x55e29d58f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa433815b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d5925b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa433815b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29d592af0 con 0x55e29d583860 2024-05-08T21:20:02.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa433815b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29d592c60 con 0x55e29d582ea0 2024-05-08T21:20:02.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa433815b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e29d592e20 con 0x55e29d582540 2024-05-08T21:20:02.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa433815b80 10 client.0 _ll_register_callbacks cb 0x55e29d477b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:02.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa4253f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582540 0x55e29d58f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa4253f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582540 0x55e29d58f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51398/0 (socket says 172.21.15.33:51398) 2024-05-08T21:20:02.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa4253f7700 1 -- 172.21.15.33:0/1698593486 learned_addr learned my addr 172.21.15.33:0/1698593486 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:02.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa425bf8700 1 --2- 172.21.15.33:0/1698593486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d5925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa4253f7700 1 -- 172.21.15.33:0/1698593486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582ea0 msgr2=0x55e29d58f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa4253f7700 1 --2- 172.21.15.33:0/1698593486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582ea0 0x55e29d58f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa4253f7700 1 -- 172.21.15.33:0/1698593486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 msgr2=0x55e29d5925b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa4253f7700 1 --2- 172.21.15.33:0/1698593486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d5925b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.947+0000 7fa4253f7700 1 -- 172.21.15.33:0/1698593486 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e29d2ffb60 con 0x55e29d582540 2024-05-08T21:20:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.948+0000 7fa4253f7700 1 --2- 172.21.15.33:0/1698593486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582540 0x55e29d58f0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa41c01a900 tx=0x7fa41c01a930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d560fa004dfc3fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.948+0000 7fa408ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:20:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.948+0000 7fa408ff9700 1 -- 172.21.15.33:0/1698593486 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c021040 con 0x55e29d582540 2024-05-08T21:20:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.948+0000 7fa408ff9700 1 -- 172.21.15.33:0/1698593486 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa41c01bd60 con 0x55e29d582540 2024-05-08T21:20:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.948+0000 7fa433815b80 1 -- 172.21.15.33:0/1698593486 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa41809aee0 con 0x55e29d582540 2024-05-08T21:20:02.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.948+0000 7fa408ff9700 1 -- 172.21.15.33:0/1698593486 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa41c01aee0 con 0x55e29d582540 2024-05-08T21:20:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.948+0000 7fa433815b80 10 client.25789 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.948+0000 7fa433815b80 1 -- 172.21.15.33:0/1698593486 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e29d357cd0 con 0x55e29d582540 2024-05-08T21:20:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.948+0000 7fa408ff9700 1 -- 172.21.15.33:0/1698593486 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa41809aee0 con 0x55e29d582540 2024-05-08T21:20:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.949+0000 7fa408ff9700 1 -- 172.21.15.33:0/1698593486 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa41c00ec20 con 0x55e29d582540 2024-05-08T21:20:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.949+0000 7fa433815b80 10 client.25789 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.949+0000 7fa433815b80 10 client.25789 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:20:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.949+0000 7fa433815b80 20 client.25789 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.949+0000 7fa433815b80 1 --2- 172.21.15.33:0/1698593486 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55e29d5eedd0 0x55e29d5f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.949+0000 7fa433815b80 4 client.25789 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:20:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.949+0000 7fa433815b80 1 -- 172.21.15.33:0/1698593486 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e29d4caec0 con 0x55e29d5eedd0 2024-05-08T21:20:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.950+0000 7fa424bf6700 1 --2- 172.21.15.33:0/1698593486 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55e29d5eedd0 0x55e29d5f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:02.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.950+0000 7fa408ff9700 10 client.25789 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.950+0000 7fa424bf6700 1 --2- 172.21.15.33:0/1698593486 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55e29d5eedd0 0x55e29d5f11f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25c1b8f88ba74b5c server_cookie=25856b969e9fc6ca in_seq=0 out_seq=0 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa408ff9700 1 -- 172.21.15.33:0/1698593486 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e29d4caec0 con 0x55e29d5eedd0 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa408ff9700 10 client.25789 handle_command_reply: tid=0 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa433815b80 1 client.25789 shutdown 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa433815b80 1 -- 172.21.15.33:0/1698593486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582540 msgr2=0x55e29d58f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa433815b80 1 --2- 172.21.15.33:0/1698593486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582540 0x55e29d58f0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa41c01a900 tx=0x7fa41c01a930 comp rx=0 tx=0).stop 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa433815b80 1 -- 172.21.15.33:0/1698593486 shutdown_connections 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa433815b80 1 --2- 172.21.15.33:0/1698593486 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55e29d5eedd0 0x55e29d5f11f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa433815b80 1 --2- 172.21.15.33:0/1698593486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e29d583860 0x55e29d5925b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa433815b80 1 --2- 172.21.15.33:0/1698593486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e29d582540 0x55e29d58f0f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa408ff9700 0 client.25789 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa433815b80 1 --2- 172.21.15.33:0/1698593486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e29d582ea0 0x55e29d58f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa433815b80 1 -- 172.21.15.33:0/1698593486 >> 172.21.15.33:0/1698593486 conn(0x55e29d4e0210 msgr2=0x55e29d586790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa433815b80 1 -- 172.21.15.33:0/1698593486 shutdown_connections 2024-05-08T21:20:02.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.951+0000 7fa433815b80 1 -- 172.21.15.33:0/1698593486 wait complete. 2024-05-08T21:20:02.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.952+0000 7fa433815b80 20 client.25789 trim_cache size 0 max 16384 2024-05-08T21:20:02.956 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:20:02.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.955+0000 7fa42765b700 1 -- 172.21.15.33:0/1144567333 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa40c0a1120 msgr2=0x7fa40c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.955+0000 7fa42765b700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa40c0a1120 0x7fa40c0a35e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009fd0 tx=0x7fa41c009260 comp rx=0 tx=0).stop 2024-05-08T21:20:02.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.955+0000 7fa42765b700 1 -- 172.21.15.33:0/1144567333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42010f330 msgr2=0x7fa4201097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:02.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.955+0000 7fa42765b700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42010f330 0x7fa4201097b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa418007af0 tx=0x7fa418007870 comp rx=0 tx=0).stop 2024-05-08T21:20:02.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.955+0000 7fa42765b700 1 -- 172.21.15.33:0/1144567333 shutdown_connections 2024-05-08T21:20:02.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.955+0000 7fa42765b700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42010f330 0x7fa4201097b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.955+0000 7fa42765b700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa40c0a1120 0x7fa40c0a35e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.955+0000 7fa42765b700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa420072ce0 0x7fa42010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.955+0000 7fa42765b700 1 --2- 172.21.15.33:0/1144567333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4201084e0 0x7fa420109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:02.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.955+0000 7fa42765b700 1 -- 172.21.15.33:0/1144567333 >> 172.21.15.33:0/1144567333 conn(0x7fa42006dcb0 msgr2=0x7fa42010b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:02.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.956+0000 7fa42765b700 1 -- 172.21.15.33:0/1144567333 shutdown_connections 2024-05-08T21:20:02.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:02.956+0000 7fa42765b700 1 -- 172.21.15.33:0/1144567333 wait complete. 2024-05-08T21:20:02.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:20:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.118+0000 7fbe2f785700 1 -- 172.21.15.33:0/2563956191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe280ffd70 msgr2=0x7fbe28100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.118+0000 7fbe2f785700 1 --2- 172.21.15.33:0/2563956191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe280ffd70 0x7fbe28100190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009920 tx=0x7fbe24009c30 comp rx=0 tx=0).stop 2024-05-08T21:20:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.118+0000 7fbe2f785700 1 -- 172.21.15.33:0/2563956191 shutdown_connections 2024-05-08T21:20:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.118+0000 7fbe2f785700 1 --2- 172.21.15.33:0/2563956191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe28101900 0x7fbe281081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.118+0000 7fbe2f785700 1 --2- 172.21.15.33:0/2563956191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe280ffd70 0x7fbe28100190 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.118+0000 7fbe2f785700 1 --2- 172.21.15.33:0/2563956191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe28100f60 0x7fbe281013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.118+0000 7fbe2f785700 1 -- 172.21.15.33:0/2563956191 >> 172.21.15.33:0/2563956191 conn(0x7fbe280fb300 msgr2=0x7fbe280fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:03.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.119+0000 7fbe2f785700 1 -- 172.21.15.33:0/2563956191 shutdown_connections 2024-05-08T21:20:03.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.119+0000 7fbe2f785700 1 -- 172.21.15.33:0/2563956191 wait complete. 2024-05-08T21:20:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.119+0000 7fbe2f785700 1 Processor -- start 2024-05-08T21:20:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.120+0000 7fbe2f785700 1 -- start start 2024-05-08T21:20:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.120+0000 7fbe2f785700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe280ffd70 0x7fbe28069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.120+0000 7fbe2f785700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe28100f60 0x7fbe2819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.120+0000 7fbe2f785700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe28101900 0x7fbe2819e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.120+0000 7fbe2f785700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2819e8c0 con 0x7fbe28100f60 2024-05-08T21:20:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.120+0000 7fbe2f785700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2819ea30 con 0x7fbe28101900 2024-05-08T21:20:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.120+0000 7fbe2f785700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2819ebf0 con 0x7fbe280ffd70 2024-05-08T21:20:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.120+0000 7fbe2cd20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe28100f60 0x7fbe2819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.120+0000 7fbe2cd20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe28100f60 0x7fbe2819bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50712/0 (socket says 172.21.15.33:50712) 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.120+0000 7fbe2cd20700 1 -- 172.21.15.33:0/2659555270 learned_addr learned my addr 172.21.15.33:0/2659555270 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.120+0000 7fbe2dd22700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe28101900 0x7fbe2819e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.120+0000 7fbe2d521700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe280ffd70 0x7fbe28069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.121+0000 7fbe2cd20700 1 -- 172.21.15.33:0/2659555270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe280ffd70 msgr2=0x7fbe28069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.121+0000 7fbe2cd20700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe280ffd70 0x7fbe28069ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.121+0000 7fbe2cd20700 1 -- 172.21.15.33:0/2659555270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe28101900 msgr2=0x7fbe2819e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.121+0000 7fbe2cd20700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe28101900 0x7fbe2819e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.121+0000 7fbe2cd20700 1 -- 172.21.15.33:0/2659555270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe18009520 con 0x7fbe28100f60 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.121+0000 7fbe2cd20700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe28100f60 0x7fbe2819bb30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800eb30 tx=0x7fbe1800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.121+0000 7fbe1e7fc700 1 -- 172.21.15.33:0/2659555270 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18009880 con 0x7fbe28100f60 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.121+0000 7fbe2f785700 1 -- 172.21.15.33:0/2659555270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe20009d00 con 0x7fbe28100f60 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.121+0000 7fbe1e7fc700 1 -- 172.21.15.33:0/2659555270 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe180044b0 con 0x7fbe28100f60 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.121+0000 7fbe1e7fc700 1 -- 172.21.15.33:0/2659555270 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18003ce0 con 0x7fbe28100f60 2024-05-08T21:20:03.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.121+0000 7fbe2f785700 1 -- 172.21.15.33:0/2659555270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe24009520 con 0x7fbe28100f60 2024-05-08T21:20:03.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.126+0000 7fbe3b93fb80 1 Processor -- start 2024-05-08T21:20:03.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe3b93fb80 1 -- start start 2024-05-08T21:20:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe3b93fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2b540 0x565085a2b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe3b93fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 0x565085a2c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe3b93fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2c860 0x565085a2cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe3b93fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565085982150 con 0x565085a2bea0 2024-05-08T21:20:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe3b93fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56508597fb10 con 0x565085a2c860 2024-05-08T21:20:03.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe3b93fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56508597b400 con 0x565085a2b540 2024-05-08T21:20:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe2dd22700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2b540 0x565085a2b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe2dd22700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2b540 0x565085a2b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51424/0 (socket says 172.21.15.33:51424) 2024-05-08T21:20:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe2dd22700 1 -- 172.21.15.33:0/2410213972 learned_addr learned my addr 172.21.15.33:0/2410213972 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe1e7fc700 1 -- 172.21.15.33:0/2659555270 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbe18003e40 con 0x7fbe28100f60 2024-05-08T21:20:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe2dd22700 1 -- 172.21.15.33:0/2410213972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2c860 msgr2=0x565085a2cce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:20:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe2dd22700 1 --2- 172.21.15.33:0/2410213972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2c860 0x565085a2cce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe2dd22700 1 -- 172.21.15.33:0/2410213972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 msgr2=0x565085a2c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe2dd22700 1 --2- 172.21.15.33:0/2410213972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 0x565085a2c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.127+0000 7fbe2dd22700 1 -- 172.21.15.33:0/2410213972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650857fc3b0 con 0x565085a2b540 2024-05-08T21:20:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.128+0000 7fbe2dd22700 1 --2- 172.21.15.33:0/2410213972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2b540 0x565085a2b960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009560 tx=0x7fbe200099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5adac58c367d237 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.128+0000 7fbe1e7fc700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbe140a1220 0x7fbe140a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.128+0000 7fbe137fe700 1 -- 172.21.15.33:0/2410213972 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000e070 con 0x565085a2b540 2024-05-08T21:20:03.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.128+0000 7fbe137fe700 1 -- 172.21.15.33:0/2410213972 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe2000f040 con 0x565085a2b540 2024-05-08T21:20:03.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.128+0000 7fbe137fe700 1 -- 172.21.15.33:0/2410213972 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20004290 con 0x565085a2b540 2024-05-08T21:20:03.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.128+0000 7fbe1e7fc700 1 -- 172.21.15.33:0/2659555270 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbe18014070 con 0x7fbe28100f60 2024-05-08T21:20:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.129+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/2410213972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2b540 msgr2=0x565085a2b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.129+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/2410213972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2b540 0x565085a2b960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009560 tx=0x7fbe200099e0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.129+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/2410213972 shutdown_connections 2024-05-08T21:20:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.129+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/2410213972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2c860 0x565085a2cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.129+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/2410213972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2b540 0x565085a2b960 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.129+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/2410213972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 0x565085a2c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.129+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/2410213972 >> 172.21.15.33:0/2410213972 conn(0x565085989210 msgr2=0x565085a25de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:03.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.129+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/2410213972 shutdown_connections 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.129+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/2410213972 wait complete. 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.130+0000 7fbe3b93fb80 1 -- start start 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.130+0000 7fbe3b93fb80 1 Processor -- start 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.130+0000 7fbe2d521700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbe140a1220 0x7fbe140a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.131+0000 7fbe3b93fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 0x565085a3d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.131+0000 7fbe3b93fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2bea0 0x565085a3dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.131+0000 7fbe3b93fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2c860 0x565085a44cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.131+0000 7fbe3b93fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565085917560 con 0x565085a2c860 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.131+0000 7fbe3b93fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565085917200 con 0x565085a2b540 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.131+0000 7fbe3b93fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650857fe6a0 con 0x565085a2bea0 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.131+0000 7fbe3b93fb80 10 client.0 _ll_register_callbacks cb 0x565085920b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.131+0000 7fbe2d521700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 0x565085a3d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe2cd20700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2bea0 0x565085a3dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe2d521700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 0x565085a3d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:35010/0 (socket says 172.21.15.33:35010) 2024-05-08T21:20:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe2d521700 1 -- 172.21.15.33:0/3399023253 learned_addr learned my addr 172.21.15.33:0/3399023253 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe2cd20700 1 -- 172.21.15.33:0/3399023253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 msgr2=0x565085a3d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe2cd20700 1 --2- 172.21.15.33:0/3399023253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 0x565085a3d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe2cd20700 1 -- 172.21.15.33:0/3399023253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2c860 msgr2=0x565085a44cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe2cd20700 1 --2- 172.21.15.33:0/3399023253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2c860 0x565085a44cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe2cd20700 1 -- 172.21.15.33:0/3399023253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe200091e0 con 0x565085a2bea0 2024-05-08T21:20:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe2cd20700 1 --2- 172.21.15.33:0/3399023253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2bea0 0x565085a3dc30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x565085a26d40 tx=0x7fbe18099df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49b2b913425f8aaf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:20:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe10ff9700 1 -- 172.21.15.33:0/3399023253 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800cbd0 con 0x565085a2bea0 2024-05-08T21:20:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe2d521700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbe140a1220 0x7fbe140a36e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400fb00 tx=0x7fbe24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe10ff9700 1 -- 172.21.15.33:0/3399023253 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbe1809c030 con 0x565085a2bea0 2024-05-08T21:20:03.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/3399023253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650858d4270 con 0x565085a2bea0 2024-05-08T21:20:03.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.132+0000 7fbe10ff9700 1 -- 172.21.15.33:0/3399023253 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe1809a180 con 0x565085a2bea0 2024-05-08T21:20:03.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.133+0000 7fbe3b93fb80 10 client.25801 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:03.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.133+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/3399023253 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5650857a8b60 con 0x565085a2bea0 2024-05-08T21:20:03.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.133+0000 7fbe10ff9700 1 -- 172.21.15.33:0/3399023253 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe18080080 con 0x565085a2bea0 2024-05-08T21:20:03.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.133+0000 7fbe10ff9700 1 -- 172.21.15.33:0/3399023253 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbe1807a030 con 0x565085a2bea0 2024-05-08T21:20:03.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.133+0000 7fbe3b93fb80 10 client.25801 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:03.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.133+0000 7fbe3b93fb80 10 client.25801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:03.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.133+0000 7fbe3b93fb80 20 client.25801 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:03.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.133+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/3399023253 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x565085a49580 0x565085a4b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.133+0000 7fbe3b93fb80 4 client.25801 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.133+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/3399023253 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650859fa810 con 0x565085a49580 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.133+0000 7fbe2d521700 1 --2- 172.21.15.33:0/3399023253 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x565085a49580 0x565085a4b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.134+0000 7fbe10ff9700 10 client.25801 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.134+0000 7fbe2d521700 1 --2- 172.21.15.33:0/3399023253 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x565085a49580 0x565085a4b9a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e540e25e789b881c server_cookie=645b3d8532dc86a1 in_seq=0 out_seq=0 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.134+0000 7fbe10ff9700 1 -- 172.21.15.33:0/3399023253 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650859fa810 con 0x565085a49580 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.134+0000 7fbe10ff9700 10 client.25801 handle_command_reply: tid=0 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe3b93fb80 1 client.25801 shutdown 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/3399023253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2bea0 msgr2=0x565085a3dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/3399023253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2bea0 0x565085a3dc30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x565085a26d40 tx=0x7fbe18099df0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/3399023253 shutdown_connections 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/3399023253 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x565085a49580 0x565085a4b9a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/3399023253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2c860 0x565085a44cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe10ff9700 0 client.25801 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/3399023253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 0x565085a3d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/3399023253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2bea0 0x565085a3dc30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/3399023253 >> 172.21.15.33:0/3399023253 conn(0x565085989210 msgr2=0x565085a2a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/3399023253 shutdown_connections 2024-05-08T21:20:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/3399023253 wait complete. 2024-05-08T21:20:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.135+0000 7fbe3b93fb80 20 client.25801 trim_cache size 0 max 16384 2024-05-08T21:20:03.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.158+0000 7fbe3b93fb80 1 Processor -- start 2024-05-08T21:20:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.158+0000 7fbe3b93fb80 1 -- start start 2024-05-08T21:20:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.158+0000 7fbe3b93fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 0x565085a4b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.158+0000 7fbe3b93fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 0x565085a44c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.158+0000 7fbe3b93fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2c860 0x565085a32eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.158+0000 7fbe3b93fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565085a22dc0 con 0x565085a2bea0 2024-05-08T21:20:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.158+0000 7fbe3b93fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565085a49380 con 0x565085a2b540 2024-05-08T21:20:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.158+0000 7fbe3b93fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x565085a27570 con 0x565085a2c860 2024-05-08T21:20:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.158+0000 7fbe2cd20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 0x565085a44c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe2cd20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 0x565085a44c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50740/0 (socket says 172.21.15.33:50740) 2024-05-08T21:20:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe2cd20700 1 -- 172.21.15.33:0/1681210310 learned_addr learned my addr 172.21.15.33:0/1681210310 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe2dd22700 1 --2- 172.21.15.33:0/1681210310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2c860 0x565085a32eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe2d521700 1 --2- 172.21.15.33:0/1681210310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 0x565085a4b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe2cd20700 1 -- 172.21.15.33:0/1681210310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2c860 msgr2=0x565085a32eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe2cd20700 1 --2- 172.21.15.33:0/1681210310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2c860 0x565085a32eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe2cd20700 1 -- 172.21.15.33:0/1681210310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 msgr2=0x565085a4b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe2cd20700 1 --2- 172.21.15.33:0/1681210310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 0x565085a4b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe2cd20700 1 -- 172.21.15.33:0/1681210310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565085941560 con 0x565085a2bea0 2024-05-08T21:20:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe2cd20700 1 --2- 172.21.15.33:0/1681210310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 0x565085a44c70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800eb00 tx=0x7fbe1800c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dd85398136b251 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe13fff700 1 -- 172.21.15.33:0/1681210310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800cbd0 con 0x565085a2bea0 2024-05-08T21:20:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe13fff700 1 -- 172.21.15.33:0/1681210310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5650858d4270 con 0x565085a2bea0 2024-05-08T21:20:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.159+0000 7fbe13fff700 1 -- 172.21.15.33:0/1681210310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1809a1f0 con 0x565085a2bea0 2024-05-08T21:20:03.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.160+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1681210310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 msgr2=0x565085a44c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:03.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.160+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/1681210310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 0x565085a44c70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800eb00 tx=0x7fbe1800c6e0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.160+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1681210310 shutdown_connections 2024-05-08T21:20:03.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.160+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/1681210310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2c860 0x565085a32eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.160+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/1681210310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 0x565085a4b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.160+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/1681210310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 0x565085a44c70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.160+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1681210310 >> 172.21.15.33:0/1681210310 conn(0x565085989210 msgr2=0x565085a34430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:03.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.160+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1681210310 shutdown_connections 2024-05-08T21:20:03.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.161+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1681210310 wait complete. 2024-05-08T21:20:03.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.161+0000 7fbe3b93fb80 1 -- start start 2024-05-08T21:20:03.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.161+0000 7fbe3b93fb80 1 Processor -- start 2024-05-08T21:20:03.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe3b93fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 0x565085a421d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe3b93fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 0x565085a42710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe3b93fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2c860 0x565085a43720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe3b93fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565085a43c60 con 0x565085a2bea0 2024-05-08T21:20:03.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe3b93fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565085a43dd0 con 0x565085a2b540 2024-05-08T21:20:03.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe3b93fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x565085a43f90 con 0x565085a2c860 2024-05-08T21:20:03.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe3b93fb80 10 client.0 _ll_register_callbacks cb 0x565085920b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:03.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe2dd22700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2c860 0x565085a43720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe2dd22700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2c860 0x565085a43720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51452/0 (socket says 172.21.15.33:51452) 2024-05-08T21:20:03.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe2dd22700 1 -- 172.21.15.33:0/1355237975 learned_addr learned my addr 172.21.15.33:0/1355237975 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe2dd22700 1 -- 172.21.15.33:0/1355237975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 msgr2=0x565085a421d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe2dd22700 1 --2- 172.21.15.33:0/1355237975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 0x565085a421d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe2dd22700 1 -- 172.21.15.33:0/1355237975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 msgr2=0x565085a42710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe2dd22700 1 --2- 172.21.15.33:0/1355237975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 0x565085a42710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.162+0000 7fbe2dd22700 1 -- 172.21.15.33:0/1355237975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650857a8b60 con 0x565085a2c860 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.163+0000 7fbe2dd22700 1 --2- 172.21.15.33:0/1355237975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2c860 0x565085a43720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009cd0 tx=0x7fbe20016f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de720c5f294fe6c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.163+0000 7fbe10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.163+0000 7fbe10ff9700 1 -- 172.21.15.33:0/1355237975 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000e040 con 0x565085a2c860 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.163+0000 7fbe10ff9700 1 -- 172.21.15.33:0/1355237975 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbe20005070 con 0x565085a2c860 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.163+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1355237975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe2000f040 con 0x565085a2c860 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.163+0000 7fbe10ff9700 1 -- 172.21.15.33:0/1355237975 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe20005230 con 0x565085a2c860 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.163+0000 7fbe3b93fb80 10 client.25807 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.163+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1355237975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x565085800cd0 con 0x565085a2c860 2024-05-08T21:20:03.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.164+0000 7fbe10ff9700 1 -- 172.21.15.33:0/1355237975 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe2000f040 con 0x565085a2c860 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.164+0000 7fbe10ff9700 1 -- 172.21.15.33:0/1355237975 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbe2001c070 con 0x565085a2c860 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.164+0000 7fbe3b93fb80 10 client.25807 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.164+0000 7fbe3b93fb80 10 client.25807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.164+0000 7fbe3b93fb80 20 client.25807 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.164+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/1355237975 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x565085aa0080 0x565085aa24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.164+0000 7fbe3b93fb80 4 client.25807 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.164+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1355237975 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565085973ec0 con 0x565085aa0080 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.164+0000 7fbe2d521700 1 --2- 172.21.15.33:0/1355237975 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x565085aa0080 0x565085aa24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe10ff9700 10 client.25807 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe2d521700 1 --2- 172.21.15.33:0/1355237975 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x565085aa0080 0x565085aa24a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc047e83460cfca0 server_cookie=b65f21f2e5e1f180 in_seq=0 out_seq=0 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe10ff9700 1 -- 172.21.15.33:0/1355237975 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565085973ec0 con 0x565085aa0080 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe10ff9700 10 client.25807 handle_command_reply: tid=0 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe3b93fb80 1 client.25807 shutdown 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1355237975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2c860 msgr2=0x565085a43720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/1355237975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2c860 0x565085a43720 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009cd0 tx=0x7fbe20016f10 comp rx=0 tx=0).stop 2024-05-08T21:20:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1355237975 shutdown_connections 2024-05-08T21:20:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/1355237975 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x565085aa0080 0x565085aa24a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/1355237975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565085a2c860 0x565085a43720 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe10ff9700 0 client.25807 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/1355237975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565085a2b540 0x565085a421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe3b93fb80 1 --2- 172.21.15.33:0/1355237975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565085a2bea0 0x565085a42710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1355237975 >> 172.21.15.33:0/1355237975 conn(0x565085989210 msgr2=0x565085a34390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1355237975 shutdown_connections 2024-05-08T21:20:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.165+0000 7fbe3b93fb80 1 -- 172.21.15.33:0/1355237975 wait complete. 2024-05-08T21:20:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.166+0000 7fbe3b93fb80 20 client.25807 trim_cache size 0 max 16384 2024-05-08T21:20:03.172 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:20:03.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.166+0000 7fbe2f785700 1 -- 172.21.15.33:0/2659555270 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbe140a1220 msgr2=0x7fbe140a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:03.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.166+0000 7fbe2f785700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbe140a1220 0x7fbe140a36e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400fb00 tx=0x7fbe24009260 comp rx=0 tx=0).stop 2024-05-08T21:20:03.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.166+0000 7fbe2f785700 1 -- 172.21.15.33:0/2659555270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe28100f60 msgr2=0x7fbe2819bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:03.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.166+0000 7fbe2f785700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe28100f60 0x7fbe2819bb30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800eb30 tx=0x7fbe1800ee40 comp rx=0 tx=0).stop 2024-05-08T21:20:03.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.166+0000 7fbe2f785700 1 -- 172.21.15.33:0/2659555270 shutdown_connections 2024-05-08T21:20:03.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.166+0000 7fbe2f785700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbe140a1220 0x7fbe140a36e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.166+0000 7fbe2f785700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe28101900 0x7fbe2819e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.166+0000 7fbe2f785700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe280ffd70 0x7fbe28069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.166+0000 7fbe2f785700 1 --2- 172.21.15.33:0/2659555270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe28100f60 0x7fbe2819bb30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:03.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.166+0000 7fbe2f785700 1 -- 172.21.15.33:0/2659555270 >> 172.21.15.33:0/2659555270 conn(0x7fbe280fb300 msgr2=0x7fbe28104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:03.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.167+0000 7fbe2f785700 1 -- 172.21.15.33:0/2659555270 shutdown_connections 2024-05-08T21:20:03.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:03.167+0000 7fbe2f785700 1 -- 172.21.15.33:0/2659555270 wait complete. 2024-05-08T21:20:03.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:03 smithi071 ceph-mon[47074]: pgmap v322: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:20:03.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:03 smithi082 ceph-mon[47512]: pgmap v322: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:20:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:03 smithi033 ceph-mon[42528]: pgmap v322: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:20:04.185 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:20:04.185 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:20:04.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.384+0000 7fd794fd5700 1 -- 172.21.15.33:0/1469246984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790101950 msgr2=0x7fd790108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.384+0000 7fd794fd5700 1 --2- 172.21.15.33:0/1469246984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790101950 0x7fd790108200 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd78800b0d0 tx=0x7fd78800b3e0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.384+0000 7fd794fd5700 1 -- 172.21.15.33:0/1469246984 shutdown_connections 2024-05-08T21:20:04.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.384+0000 7fd794fd5700 1 --2- 172.21.15.33:0/1469246984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd790101950 0x7fd790108200 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.384+0000 7fd794fd5700 1 --2- 172.21.15.33:0/1469246984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7900ffdc0 0x7fd7901001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.384+0000 7fd794fd5700 1 --2- 172.21.15.33:0/1469246984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd790100fb0 0x7fd790101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.384+0000 7fd794fd5700 1 -- 172.21.15.33:0/1469246984 >> 172.21.15.33:0/1469246984 conn(0x7fd7900fb370 msgr2=0x7fd7900fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:04.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.384+0000 7fd794fd5700 1 -- 172.21.15.33:0/1469246984 shutdown_connections 2024-05-08T21:20:04.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.384+0000 7fd794fd5700 1 -- 172.21.15.33:0/1469246984 wait complete. 2024-05-08T21:20:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.385+0000 7fd794fd5700 1 Processor -- start 2024-05-08T21:20:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.385+0000 7fd794fd5700 1 -- start start 2024-05-08T21:20:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd794fd5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7900ffdc0 0x7fd790069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd794fd5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd790100fb0 0x7fd79019bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd794fd5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790101950 0x7fd79019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd794fd5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79019e960 con 0x7fd7900ffdc0 2024-05-08T21:20:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd794fd5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79019ead0 con 0x7fd790100fb0 2024-05-08T21:20:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd794fd5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79019ec90 con 0x7fd790101950 2024-05-08T21:20:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd78ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790101950 0x7fd79019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd78ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790101950 0x7fd79019e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51466/0 (socket says 172.21.15.33:51466) 2024-05-08T21:20:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd78ed9d700 1 -- 172.21.15.33:0/993393428 learned_addr learned my addr 172.21.15.33:0/993393428 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd78e59c700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7900ffdc0 0x7fd790069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd78dd9b700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd790100fb0 0x7fd79019bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd78ed9d700 1 -- 172.21.15.33:0/993393428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd790100fb0 msgr2=0x7fd79019bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd78ed9d700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd790100fb0 0x7fd79019bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd78ed9d700 1 -- 172.21.15.33:0/993393428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7900ffdc0 msgr2=0x7fd790069ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd78ed9d700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7900ffdc0 0x7fd790069ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd78ed9d700 1 -- 172.21.15.33:0/993393428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd784009520 con 0x7fd790101950 2024-05-08T21:20:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.386+0000 7fd78dd9b700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd790100fb0 0x7fd79019bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:04.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.387+0000 7fd78ed9d700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790101950 0x7fd79019e170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd7880079c0 tx=0x7fd78800cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:04.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.387+0000 7fd77f7fe700 1 -- 172.21.15.33:0/993393428 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800e040 con 0x7fd790101950 2024-05-08T21:20:04.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.387+0000 7fd77f7fe700 1 -- 172.21.15.33:0/993393428 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd788004c50 con 0x7fd790101950 2024-05-08T21:20:04.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.387+0000 7fd794fd5700 1 -- 172.21.15.33:0/993393428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd788009d00 con 0x7fd790101950 2024-05-08T21:20:04.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.387+0000 7fd77f7fe700 1 -- 172.21.15.33:0/993393428 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78801b6b0 con 0x7fd790101950 2024-05-08T21:20:04.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.387+0000 7fd794fd5700 1 -- 172.21.15.33:0/993393428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79019f1a0 con 0x7fd790101950 2024-05-08T21:20:04.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.390+0000 7fd77f7fe700 1 -- 172.21.15.33:0/993393428 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd78801b810 con 0x7fd790101950 2024-05-08T21:20:04.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.390+0000 7fd77f7fe700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7780a1220 0x7fd7780a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.390+0000 7fd77f7fe700 1 -- 172.21.15.33:0/993393428 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd7880cf380 con 0x7fd790101950 2024-05-08T21:20:04.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.392+0000 7fd7a118fb80 1 Processor -- start 2024-05-08T21:20:04.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.392+0000 7fd78e59c700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7780a1220 0x7fd7780a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.393+0000 7fd7a118fb80 1 -- start start 2024-05-08T21:20:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.393+0000 7fd7a118fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e18a3d0 0x555e4e18a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.393+0000 7fd7a118fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.393+0000 7fd7a118fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e198530 0x555e4e19a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.393+0000 7fd7a118fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4e0e11d0 con 0x555e4e198530 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.393+0000 7fd7a118fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4e0deb90 con 0x555e4e18a3d0 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.393+0000 7fd7a118fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4e0da480 con 0x555e4e18ad30 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.393+0000 7fd78e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e18a3d0 0x555e4e18a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.393+0000 7fd78e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e18a3d0 0x555e4e18a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:35058/0 (socket says 172.21.15.33:35058) 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.393+0000 7fd78e59c700 1 -- 172.21.15.33:0/3723774938 learned_addr learned my addr 172.21.15.33:0/3723774938 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.393+0000 7fd78ed9d700 1 --2- 172.21.15.33:0/3723774938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e198530 0x555e4e19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.394+0000 7fd78e59c700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7780a1220 0x7fd7780a36e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd7800115e0 tx=0x7fd7800119a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.394+0000 7fd78e59c700 1 -- 172.21.15.33:0/3723774938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 msgr2=0x555e4e197ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.394+0000 7fd78e59c700 1 --2- 172.21.15.33:0/3723774938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.394+0000 7fd78e59c700 1 -- 172.21.15.33:0/3723774938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e198530 msgr2=0x555e4e19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.394+0000 7fd78e59c700 1 --2- 172.21.15.33:0/3723774938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e198530 0x555e4e19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.394+0000 7fd78e59c700 1 -- 172.21.15.33:0/3723774938 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4df5b430 con 0x555e4e18a3d0 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.394+0000 7fd78ed9d700 1 --2- 172.21.15.33:0/3723774938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e198530 0x555e4e19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.394+0000 7fd78e59c700 1 --2- 172.21.15.33:0/3723774938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e18a3d0 0x555e4e18a7f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd780011fd0 tx=0x7fd780020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31ea20433c309483 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:04.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.394+0000 7fd77cff9700 1 -- 172.21.15.33:0/3723774938 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd780004220 con 0x555e4e18a3d0 2024-05-08T21:20:04.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.394+0000 7fd77cff9700 1 -- 172.21.15.33:0/3723774938 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd780018070 con 0x555e4e18a3d0 2024-05-08T21:20:04.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.394+0000 7fd77cff9700 1 -- 172.21.15.33:0/3723774938 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000f680 con 0x555e4e18a3d0 2024-05-08T21:20:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.395+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3723774938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e18a3d0 msgr2=0x555e4e18a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.395+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/3723774938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e18a3d0 0x555e4e18a7f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd780011fd0 tx=0x7fd780020000 comp rx=0 tx=0).stop 2024-05-08T21:20:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.395+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3723774938 shutdown_connections 2024-05-08T21:20:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.395+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/3723774938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e198530 0x555e4e19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.395+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/3723774938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e18a3d0 0x555e4e18a7f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.395+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/3723774938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.395+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3723774938 >> 172.21.15.33:0/3723774938 conn(0x555e4e0e8290 msgr2=0x555e4e184e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.396+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3723774938 shutdown_connections 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.396+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3723774938 wait complete. 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.396+0000 7fd7a118fb80 1 -- start start 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.396+0000 7fd7a118fb80 1 Processor -- start 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.396+0000 7fd7a118fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 0x555e4e1ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.396+0000 7fd7a118fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e18ad30 0x555e4e1adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd7a118fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e198530 0x555e4e1b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd7a118fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4e0765e0 con 0x555e4e18a3d0 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd7a118fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4e076280 con 0x555e4e18ad30 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd7a118fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4df5d720 con 0x555e4e198530 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd7a118fb80 10 client.0 _ll_register_callbacks cb 0x555e4e07fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd78ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e198530 0x555e4e1b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd78ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e198530 0x555e4e1b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51484/0 (socket says 172.21.15.33:51484) 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd78ed9d700 1 -- 172.21.15.33:0/1545317917 learned_addr learned my addr 172.21.15.33:0/1545317917 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd78ed9d700 1 -- 172.21.15.33:0/1545317917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e18ad30 msgr2=0x555e4e1adbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd78ed9d700 1 --2- 172.21.15.33:0/1545317917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e18ad30 0x555e4e1adbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd78ed9d700 1 -- 172.21.15.33:0/1545317917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 msgr2=0x555e4e1ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd78ed9d700 1 --2- 172.21.15.33:0/1545317917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 0x555e4e1ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd78ed9d700 1 -- 172.21.15.33:0/1545317917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd780020560 con 0x555e4e198530 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd78ed9d700 1 --2- 172.21.15.33:0/1545317917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e198530 0x555e4e1b4bf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd788009fd0 tx=0x7fd7880099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26983a750deb6fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.397+0000 7fd775ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.398+0000 7fd775ffb700 1 -- 172.21.15.33:0/1545317917 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7880a02d0 con 0x555e4e198530 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.398+0000 7fd775ffb700 1 -- 172.21.15.33:0/1545317917 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd7880a4890 con 0x555e4e198530 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.398+0000 7fd7a118fb80 1 -- 172.21.15.33:0/1545317917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e4e0332f0 con 0x555e4e198530 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.398+0000 7fd775ffb700 1 -- 172.21.15.33:0/1545317917 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7880cf170 con 0x555e4e198530 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.398+0000 7fd7a118fb80 10 client.25816 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.398+0000 7fd7a118fb80 1 -- 172.21.15.33:0/1545317917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555e4df07be0 con 0x555e4e198530 2024-05-08T21:20:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.398+0000 7fd775ffb700 1 -- 172.21.15.33:0/1545317917 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd78807f020 con 0x555e4e198530 2024-05-08T21:20:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.398+0000 7fd775ffb700 1 -- 172.21.15.33:0/1545317917 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd78809c7e0 con 0x555e4e198530 2024-05-08T21:20:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.398+0000 7fd7a118fb80 10 client.25816 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.399+0000 7fd7a118fb80 10 client.25816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.399+0000 7fd7a118fb80 20 client.25816 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.399+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/1545317917 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555e4e18f9b0 0x555e4e191dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.399+0000 7fd7a118fb80 4 client.25816 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.399+0000 7fd7a118fb80 1 -- 172.21.15.33:0/1545317917 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e4e159890 con 0x555e4e18f9b0 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.399+0000 7fd78e59c700 1 --2- 172.21.15.33:0/1545317917 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555e4e18f9b0 0x555e4e191dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.399+0000 7fd78e59c700 1 --2- 172.21.15.33:0/1545317917 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555e4e18f9b0 0x555e4e191dd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d218631d3701c8 server_cookie=f2d3ca1991a74316 in_seq=0 out_seq=0 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.399+0000 7fd775ffb700 10 client.25816 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd775ffb700 1 -- 172.21.15.33:0/1545317917 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e4e159890 con 0x555e4e18f9b0 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd775ffb700 10 client.25816 handle_command_reply: tid=0 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd7a118fb80 1 client.25816 shutdown 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd7a118fb80 1 -- 172.21.15.33:0/1545317917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e198530 msgr2=0x555e4e1b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/1545317917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e198530 0x555e4e1b4bf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd788009fd0 tx=0x7fd7880099e0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd7a118fb80 1 -- 172.21.15.33:0/1545317917 shutdown_connections 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/1545317917 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555e4e18f9b0 0x555e4e191dd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/1545317917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e198530 0x555e4e1b4bf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/1545317917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 0x555e4e1ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/1545317917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e18ad30 0x555e4e1adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd7a118fb80 1 -- 172.21.15.33:0/1545317917 >> 172.21.15.33:0/1545317917 conn(0x555e4e0e8290 msgr2=0x555e4e184620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd775ffb700 0 client.25816 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd7a118fb80 1 -- 172.21.15.33:0/1545317917 shutdown_connections 2024-05-08T21:20:04.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd7a118fb80 1 -- 172.21.15.33:0/1545317917 wait complete. 2024-05-08T21:20:04.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.400+0000 7fd7a118fb80 20 client.25816 trim_cache size 0 max 16384 2024-05-08T21:20:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.434+0000 7fd7a118fb80 1 Processor -- start 2024-05-08T21:20:04.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.435+0000 7fd7a118fb80 1 -- start start 2024-05-08T21:20:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.435+0000 7fd7a118fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 0x555e4e1ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.435+0000 7fd7a118fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e1b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.435+0000 7fd7a118fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 0x555e4e1ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.435+0000 7fd7a118fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4e18aa40 con 0x555e4e18a3d0 2024-05-08T21:20:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.435+0000 7fd7a118fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4e1ca9f0 con 0x555e4e198530 2024-05-08T21:20:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.435+0000 7fd7a118fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4e197990 con 0x555e4e18ad30 2024-05-08T21:20:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.435+0000 7fd78ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 0x555e4e1ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.435+0000 7fd78ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 0x555e4e1ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:35072/0 (socket says 172.21.15.33:35072) 2024-05-08T21:20:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.435+0000 7fd78e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 0x555e4e1ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.435+0000 7fd78ed9d700 1 -- 172.21.15.33:0/225220546 learned_addr learned my addr 172.21.15.33:0/225220546 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.435+0000 7fd78dd9b700 1 --2- 172.21.15.33:0/225220546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e1b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.436+0000 7fd78dd9b700 1 -- 172.21.15.33:0/225220546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 msgr2=0x555e4e1ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.436+0000 7fd78dd9b700 1 --2- 172.21.15.33:0/225220546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 0x555e4e1ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.436+0000 7fd78dd9b700 1 -- 172.21.15.33:0/225220546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 msgr2=0x555e4e1ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.436+0000 7fd78dd9b700 1 --2- 172.21.15.33:0/225220546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 0x555e4e1ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.436+0000 7fd78dd9b700 1 -- 172.21.15.33:0/225220546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4e181560 con 0x555e4e18ad30 2024-05-08T21:20:04.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.436+0000 7fd78dd9b700 1 --2- 172.21.15.33:0/225220546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e1b4b10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x555e4e17d2d0 tx=0x7fd7840113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aa713b3f3618d7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:04.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.436+0000 7fd77d7fa700 1 -- 172.21.15.33:0/225220546 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784011bb0 con 0x555e4e18ad30 2024-05-08T21:20:04.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.436+0000 7fd77d7fa700 1 -- 172.21.15.33:0/225220546 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd7840044b0 con 0x555e4e18ad30 2024-05-08T21:20:04.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.436+0000 7fd77d7fa700 1 -- 172.21.15.33:0/225220546 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784005020 con 0x555e4e18ad30 2024-05-08T21:20:04.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.437+0000 7fd7a118fb80 1 -- 172.21.15.33:0/225220546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 msgr2=0x555e4e1b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.437+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/225220546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e1b4b10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x555e4e17d2d0 tx=0x7fd7840113e0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.437+0000 7fd7a118fb80 1 -- 172.21.15.33:0/225220546 shutdown_connections 2024-05-08T21:20:04.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.437+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/225220546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 0x555e4e1ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.437+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/225220546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 0x555e4e1ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.437+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/225220546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e1b4b10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.437+0000 7fd7a118fb80 1 -- 172.21.15.33:0/225220546 >> 172.21.15.33:0/225220546 conn(0x555e4e0e8290 msgr2=0x555e4e1acb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.437+0000 7fd7a118fb80 1 -- 172.21.15.33:0/225220546 shutdown_connections 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.437+0000 7fd7a118fb80 1 -- 172.21.15.33:0/225220546 wait complete. 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.437+0000 7fd7a118fb80 1 -- start start 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd7a118fb80 1 Processor -- start 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd7a118fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 0x555e4e1b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd7a118fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e1b1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd7a118fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 0x555e4e1b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd7a118fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4e1b1d60 con 0x555e4e18a3d0 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd7a118fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4e1b4cd0 con 0x555e4e198530 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd7a118fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4e1b4e60 con 0x555e4e18ad30 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd7a118fb80 10 client.0 _ll_register_callbacks cb 0x555e4e07fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd78ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 0x555e4e1b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd78dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e1b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd78ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 0x555e4e1b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:35076/0 (socket says 172.21.15.33:35076) 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd78ed9d700 1 -- 172.21.15.33:0/3632767896 learned_addr learned my addr 172.21.15.33:0/3632767896 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd78dd9b700 1 --2- 172.21.15.33:0/3632767896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e1b1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51508/0 (socket says 172.21.15.33:51508) 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd78ed9d700 1 -- 172.21.15.33:0/3632767896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 msgr2=0x555e4e1b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd78ed9d700 1 --2- 172.21.15.33:0/3632767896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e1b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd78ed9d700 1 -- 172.21.15.33:0/3632767896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 msgr2=0x555e4e1b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd78ed9d700 1 --2- 172.21.15.33:0/3632767896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 0x555e4e1b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.438+0000 7fd78ed9d700 1 -- 172.21.15.33:0/3632767896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7840099d0 con 0x555e4e198530 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.439+0000 7fd78ed9d700 1 --2- 172.21.15.33:0/3632767896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 0x555e4e1b1680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd78809c1a0 tx=0x7fd78809b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8cc9162398a8ac9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.439+0000 7fd775ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.439+0000 7fd775ffb700 1 -- 172.21.15.33:0/3632767896 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78809c900 con 0x555e4e198530 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.439+0000 7fd775ffb700 1 -- 172.21.15.33:0/3632767896 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd788080080 con 0x555e4e198530 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.439+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3632767896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd780018070 con 0x555e4e198530 2024-05-08T21:20:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.439+0000 7fd775ffb700 1 -- 172.21.15.33:0/3632767896 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7880a4890 con 0x555e4e198530 2024-05-08T21:20:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.439+0000 7fd7a118fb80 10 client.25802 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.439+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3632767896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555e4e0a05e0 con 0x555e4e198530 2024-05-08T21:20:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.439+0000 7fd775ffb700 1 -- 172.21.15.33:0/3632767896 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd780018070 con 0x555e4e198530 2024-05-08T21:20:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.440+0000 7fd775ffb700 1 -- 172.21.15.33:0/3632767896 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd7880a4400 con 0x555e4e198530 2024-05-08T21:20:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.440+0000 7fd7a118fb80 10 client.25802 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.440+0000 7fd7a118fb80 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.440+0000 7fd7a118fb80 20 client.25802 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.440+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/3632767896 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555e4e218b70 0x555e4e21af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.440+0000 7fd7a118fb80 4 client.25802 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:20:04.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.440+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3632767896 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555e4e0d2f40 con 0x555e4e218b70 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.440+0000 7fd78e59c700 1 --2- 172.21.15.33:0/3632767896 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555e4e218b70 0x555e4e21af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.441+0000 7fd775ffb700 10 client.25802 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.441+0000 7fd78e59c700 1 --2- 172.21.15.33:0/3632767896 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555e4e218b70 0x555e4e21af90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1220a802736a2c78 server_cookie=b4f7728d6f284c38 in_seq=0 out_seq=0 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.442+0000 7fd775ffb700 1 -- 172.21.15.33:0/3632767896 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555e4e0d2f40 con 0x555e4e218b70 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.442+0000 7fd775ffb700 10 client.25802 handle_command_reply: tid=0 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.442+0000 7fd7a118fb80 1 client.25802 shutdown 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.443+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3632767896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 msgr2=0x555e4e1b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.443+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/3632767896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 0x555e4e1b1680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd78809c1a0 tx=0x7fd78809b9b0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.443+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3632767896 shutdown_connections 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.443+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/3632767896 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555e4e218b70 0x555e4e21af90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.443+0000 7fd775ffb700 0 client.25802 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.443+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/3632767896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555e4e198530 0x555e4e1b1680 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.443+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/3632767896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e4e18a3d0 0x555e4e1b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.443+0000 7fd7a118fb80 1 --2- 172.21.15.33:0/3632767896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4e18ad30 0x555e4e1b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.443+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3632767896 >> 172.21.15.33:0/3632767896 conn(0x555e4e0e8290 msgr2=0x555e4e1acb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.443+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3632767896 shutdown_connections 2024-05-08T21:20:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.443+0000 7fd7a118fb80 1 -- 172.21.15.33:0/3632767896 wait complete. 2024-05-08T21:20:04.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.444+0000 7fd7a118fb80 20 client.25802 trim_cache size 0 max 16384 2024-05-08T21:20:04.448 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:20:04.448 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:20:04.448 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "fd5cc1d8-a5ef-4070-a3c7-8a0aa2ba1526", 2024-05-08T21:20:04.448 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:20:04.448 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:20:04.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.445+0000 7fd794fd5700 1 -- 172.21.15.33:0/993393428 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7780a1220 msgr2=0x7fd7780a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.445+0000 7fd794fd5700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7780a1220 0x7fd7780a36e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd7800115e0 tx=0x7fd7800119a0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.445+0000 7fd794fd5700 1 -- 172.21.15.33:0/993393428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790101950 msgr2=0x7fd79019e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.445+0000 7fd794fd5700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790101950 0x7fd79019e170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd7880079c0 tx=0x7fd78800cab0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.445+0000 7fd794fd5700 1 -- 172.21.15.33:0/993393428 shutdown_connections 2024-05-08T21:20:04.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.445+0000 7fd794fd5700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd790101950 0x7fd79019e170 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.445+0000 7fd794fd5700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7900ffdc0 0x7fd790069ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.445+0000 7fd794fd5700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7780a1220 0x7fd7780a36e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.445+0000 7fd794fd5700 1 --2- 172.21.15.33:0/993393428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd790100fb0 0x7fd79019bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.445+0000 7fd794fd5700 1 -- 172.21.15.33:0/993393428 >> 172.21.15.33:0/993393428 conn(0x7fd7900fb370 msgr2=0x7fd790104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:04.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.445+0000 7fd794fd5700 1 -- 172.21.15.33:0/993393428 shutdown_connections 2024-05-08T21:20:04.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.445+0000 7fd794fd5700 1 -- 172.21.15.33:0/993393428 wait complete. 2024-05-08T21:20:04.462 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:20:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.605+0000 7f08dee66700 1 -- 172.21.15.33:0/2204417016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d8100e70 msgr2=0x7f08d8101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.605+0000 7f08dee66700 1 --2- 172.21.15.33:0/2204417016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d8100e70 0x7f08d8101290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009990 tx=0x7f08cc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.605+0000 7f08dee66700 1 -- 172.21.15.33:0/2204417016 shutdown_connections 2024-05-08T21:20:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.605+0000 7f08dee66700 1 --2- 172.21.15.33:0/2204417016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d8102980 0x7f08d81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.605+0000 7f08dee66700 1 --2- 172.21.15.33:0/2204417016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d8100e70 0x7f08d8101290 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.605+0000 7f08dee66700 1 --2- 172.21.15.33:0/2204417016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08d8101fc0 0x7f08d8102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.605+0000 7f08dee66700 1 -- 172.21.15.33:0/2204417016 >> 172.21.15.33:0/2204417016 conn(0x7f08d80fc400 msgr2=0x7f08d80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.606+0000 7f08dee66700 1 -- 172.21.15.33:0/2204417016 shutdown_connections 2024-05-08T21:20:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.606+0000 7f08dee66700 1 -- 172.21.15.33:0/2204417016 wait complete. 2024-05-08T21:20:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.606+0000 7f08dee66700 1 Processor -- start 2024-05-08T21:20:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.606+0000 7f08dee66700 1 -- start start 2024-05-08T21:20:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dee66700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08d8100e70 0x7f08d8196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dee66700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d8101fc0 0x7f08d81972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dee66700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d8102980 0x7f08d819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dee66700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d819e8d0 con 0x7f08d8101fc0 2024-05-08T21:20:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dee66700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d819ea10 con 0x7f08d8102980 2024-05-08T21:20:04.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dee66700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d819eba0 con 0x7f08d8100e70 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d8101fc0 0x7f08d81972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d8101fc0 0x7f08d81972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50838/0 (socket says 172.21.15.33:50838) 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08d7fff700 1 -- 172.21.15.33:0/2776296680 learned_addr learned my addr 172.21.15.33:0/2776296680 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dcc02700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08d8100e70 0x7f08d8196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dd403700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d8102980 0x7f08d819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dcc02700 1 -- 172.21.15.33:0/2776296680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d8102980 msgr2=0x7f08d819e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dcc02700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d8102980 0x7f08d819e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dcc02700 1 -- 172.21.15.33:0/2776296680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d8101fc0 msgr2=0x7f08d81972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dcc02700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d8101fc0 0x7f08d81972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08dcc02700 1 -- 172.21.15.33:0/2776296680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc0095f0 con 0x7f08d8100e70 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.607+0000 7f08d7fff700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d8101fc0 0x7f08d81972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.608+0000 7f08dcc02700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08d8100e70 0x7f08d8196da0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0098e0 tx=0x7f08cc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:04.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.608+0000 7f08d5ffb700 1 -- 172.21.15.33:0/2776296680 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc021070 con 0x7f08d8100e70 2024-05-08T21:20:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.608+0000 7f08d5ffb700 1 -- 172.21.15.33:0/2776296680 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08cc0054c0 con 0x7f08d8100e70 2024-05-08T21:20:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.608+0000 7f08d5ffb700 1 -- 172.21.15.33:0/2776296680 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc003680 con 0x7f08d8100e70 2024-05-08T21:20:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.608+0000 7f08dee66700 1 -- 172.21.15.33:0/2776296680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d80731d0 con 0x7f08d8100e70 2024-05-08T21:20:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.608+0000 7f08dee66700 1 -- 172.21.15.33:0/2776296680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d8073620 con 0x7f08d8100e70 2024-05-08T21:20:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.611+0000 7f08d5ffb700 1 -- 172.21.15.33:0/2776296680 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f08cc025070 con 0x7f08d8100e70 2024-05-08T21:20:04.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.612+0000 7f08d5ffb700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f08c80a1120 0x7f08c80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.612+0000 7f08d5ffb700 1 -- 172.21.15.33:0/2776296680 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f08cc0d1f10 con 0x7f08d8100e70 2024-05-08T21:20:04.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.612+0000 7f08eb020b80 1 Processor -- start 2024-05-08T21:20:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08eb020b80 1 -- start start 2024-05-08T21:20:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08eb020b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a08ed50 0x55dc4a08f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08eb020b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08f6b0 0x55dc4a08fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08eb020b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a090070 0x55dc4a0904f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08eb020b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49fdd150 con 0x55dc4a090070 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08eb020b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49fdab10 con 0x55dc4a08ed50 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08eb020b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49fd6400 con 0x55dc4a08f6b0 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08dcc02700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a08ed50 0x55dc4a08f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08dcc02700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a08ed50 0x55dc4a08f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:35096/0 (socket says 172.21.15.33:35096) 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08dcc02700 1 -- 172.21.15.33:0/3995964765 learned_addr learned my addr 172.21.15.33:0/3995964765 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08dd403700 1 --2- 172.21.15.33:0/3995964765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a090070 0x55dc4a0904f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08dcc02700 1 -- 172.21.15.33:0/3995964765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08f6b0 msgr2=0x55dc4a08fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08dcc02700 1 --2- 172.21.15.33:0/3995964765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08f6b0 0x55dc4a08fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08dcc02700 1 -- 172.21.15.33:0/3995964765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a090070 msgr2=0x55dc4a0904f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08dcc02700 1 --2- 172.21.15.33:0/3995964765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a090070 0x55dc4a0904f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.613+0000 7f08dcc02700 1 -- 172.21.15.33:0/3995964765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc49e573b0 con 0x55dc4a08ed50 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.614+0000 7f08dcc02700 1 --2- 172.21.15.33:0/3995964765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a08ed50 0x55dc4a08f170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0a6ef0 tx=0x7f08cc005ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5fb6ee0c495b77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:04.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.614+0000 7f08d7fff700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f08c80a1120 0x7f08c80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.614+0000 7f08c2ffd700 1 -- 172.21.15.33:0/3995964765 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0b6070 con 0x55dc4a08ed50 2024-05-08T21:20:04.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.614+0000 7f08c2ffd700 1 -- 172.21.15.33:0/3995964765 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08cc0a7420 con 0x55dc4a08ed50 2024-05-08T21:20:04.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.614+0000 7f08c2ffd700 1 -- 172.21.15.33:0/3995964765 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc09f4b0 con 0x55dc4a08ed50 2024-05-08T21:20:04.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.615+0000 7f08eb020b80 1 -- 172.21.15.33:0/3995964765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a08ed50 msgr2=0x55dc4a08f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.615+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3995964765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a08ed50 0x55dc4a08f170 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0a6ef0 tx=0x7f08cc005ac0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.615+0000 7f08d7fff700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f08c80a1120 0x7f08c80a35e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f08c40096b0 tx=0x7f08c40095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:04.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.615+0000 7f08eb020b80 1 -- 172.21.15.33:0/3995964765 shutdown_connections 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.615+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3995964765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a090070 0x55dc4a0904f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.615+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3995964765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a08ed50 0x55dc4a08f170 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.615+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3995964765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08f6b0 0x55dc4a08fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.615+0000 7f08eb020b80 1 -- 172.21.15.33:0/3995964765 >> 172.21.15.33:0/3995964765 conn(0x55dc49fe4210 msgr2=0x55dc4a080de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.615+0000 7f08eb020b80 1 -- 172.21.15.33:0/3995964765 shutdown_connections 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.615+0000 7f08eb020b80 1 -- 172.21.15.33:0/3995964765 wait complete. 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.615+0000 7f08eb020b80 1 -- start start 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.615+0000 7f08eb020b80 1 Processor -- start 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.616+0000 7f08eb020b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08ed50 0x55dc4a087860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.616+0000 7f08eb020b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08f6b0 0x55dc4a087da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.616+0000 7f08eb020b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a08cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.616+0000 7f08eb020b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49f72560 con 0x55dc4a08ed50 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.616+0000 7f08eb020b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49f72200 con 0x55dc4a090070 2024-05-08T21:20:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.616+0000 7f08eb020b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc49e596a0 con 0x55dc4a08f6b0 2024-05-08T21:20:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.616+0000 7f08eb020b80 10 client.0 _ll_register_callbacks cb 0x55dc49f7bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.616+0000 7f08dd403700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a08cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.616+0000 7f08dcc02700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08ed50 0x55dc4a087860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.616+0000 7f08dd403700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a08cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:35112/0 (socket says 172.21.15.33:35112) 2024-05-08T21:20:04.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.616+0000 7f08dd403700 1 -- 172.21.15.33:0/3014520307 learned_addr learned my addr 172.21.15.33:0/3014520307 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08dd403700 1 -- 172.21.15.33:0/3014520307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08f6b0 msgr2=0x55dc4a087da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08dd403700 1 --2- 172.21.15.33:0/3014520307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08f6b0 0x55dc4a087da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08dd403700 1 -- 172.21.15.33:0/3014520307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08ed50 msgr2=0x55dc4a087860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08dd403700 1 --2- 172.21.15.33:0/3014520307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08ed50 0x55dc4a087860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08dd403700 1 -- 172.21.15.33:0/3014520307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d8073620 con 0x55dc4a090070 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08dd403700 1 --2- 172.21.15.33:0/3014520307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a08cdd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f08d000ae40 tx=0x7f08d000ae70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e306064b34f9edaf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08b7fff700 1 -- 172.21.15.33:0/3014520307 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d00079c0 con 0x55dc4a090070 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08eb020b80 1 -- 172.21.15.33:0/3014520307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc49f2f270 con 0x55dc4a090070 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08b7fff700 1 -- 172.21.15.33:0/3014520307 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f08d0006e40 con 0x55dc4a090070 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08b7fff700 1 -- 172.21.15.33:0/3014520307 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08d0007b20 con 0x55dc4a090070 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08eb020b80 10 client.25814 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.617+0000 7f08eb020b80 1 -- 172.21.15.33:0/3014520307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dc49e03b60 con 0x55dc4a090070 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.618+0000 7f08b7fff700 1 -- 172.21.15.33:0/3014520307 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dc49f2f270 con 0x55dc4a090070 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.618+0000 7f08b7fff700 1 -- 172.21.15.33:0/3014520307 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08d0016cf0 con 0x55dc4a090070 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.618+0000 7f08eb020b80 10 client.25814 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.618+0000 7f08eb020b80 10 client.25814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.618+0000 7f08eb020b80 20 client.25814 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.618+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3014520307 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc4a0ac980 0x55dc4a0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.618+0000 7f08eb020b80 4 client.25814 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.618+0000 7f08eb020b80 1 -- 172.21.15.33:0/3014520307 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc4a055810 con 0x55dc4a0ac980 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.618+0000 7f08dcc02700 1 --2- 172.21.15.33:0/3014520307 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc4a0ac980 0x55dc4a0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08b7fff700 10 client.25814 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08dcc02700 1 --2- 172.21.15.33:0/3014520307 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc4a0ac980 0x55dc4a0aeda0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d1182b2dcf2c6ce server_cookie=a579a36393c7dc30 in_seq=0 out_seq=0 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08b7fff700 1 -- 172.21.15.33:0/3014520307 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc4a055810 con 0x55dc4a0ac980 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08b7fff700 10 client.25814 handle_command_reply: tid=0 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08eb020b80 1 client.25814 shutdown 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08eb020b80 1 -- 172.21.15.33:0/3014520307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 msgr2=0x55dc4a08cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3014520307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a08cdd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f08d000ae40 tx=0x7f08d000ae70 comp rx=0 tx=0).stop 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08eb020b80 1 -- 172.21.15.33:0/3014520307 shutdown_connections 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3014520307 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc4a0ac980 0x55dc4a0aeda0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3014520307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a08cdd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08b7fff700 0 client.25814 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3014520307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08ed50 0x55dc4a087860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3014520307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08f6b0 0x55dc4a087da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.619+0000 7f08eb020b80 1 -- 172.21.15.33:0/3014520307 >> 172.21.15.33:0/3014520307 conn(0x55dc49fe4210 msgr2=0x55dc4a0805a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:04.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.620+0000 7f08eb020b80 1 -- 172.21.15.33:0/3014520307 shutdown_connections 2024-05-08T21:20:04.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.620+0000 7f08eb020b80 1 -- 172.21.15.33:0/3014520307 wait complete. 2024-05-08T21:20:04.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.620+0000 7f08eb020b80 20 client.25814 trim_cache size 0 max 16384 2024-05-08T21:20:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08eb020b80 1 Processor -- start 2024-05-08T21:20:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08eb020b80 1 -- start start 2024-05-08T21:20:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08eb020b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 0x55dc4a08cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08eb020b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08f6b0 0x55dc4a096550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08eb020b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a0bcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08eb020b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4a0819d0 con 0x55dc4a08f6b0 2024-05-08T21:20:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08eb020b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4a08df50 con 0x55dc4a090070 2024-05-08T21:20:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08eb020b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4a0873d0 con 0x55dc4a08ed50 2024-05-08T21:20:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08f6b0 0x55dc4a096550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08dcc02700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 0x55dc4a08cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08dcc02700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 0x55dc4a08cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51554/0 (socket says 172.21.15.33:51554) 2024-05-08T21:20:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08dcc02700 1 -- 172.21.15.33:0/2696661731 learned_addr learned my addr 172.21.15.33:0/2696661731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08f6b0 0x55dc4a096550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50852/0 (socket says 172.21.15.33:50852) 2024-05-08T21:20:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.639+0000 7f08dd403700 1 --2- 172.21.15.33:0/2696661731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a0bcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.640+0000 7f08dd403700 1 -- 172.21.15.33:0/2696661731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 msgr2=0x55dc4a08cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.640+0000 7f08dd403700 1 --2- 172.21.15.33:0/2696661731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 0x55dc4a08cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.640+0000 7f08dd403700 1 -- 172.21.15.33:0/2696661731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08f6b0 msgr2=0x55dc4a096550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.640+0000 7f08dd403700 1 --2- 172.21.15.33:0/2696661731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08f6b0 0x55dc4a096550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.640+0000 7f08dd403700 1 -- 172.21.15.33:0/2696661731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc49f9c560 con 0x55dc4a090070 2024-05-08T21:20:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.640+0000 7f08dd403700 1 --2- 172.21.15.33:0/2696661731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a0bcc40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f08d0002230 tx=0x7f08d00155e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3adf59054f5674d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.640+0000 7f08c37fe700 1 -- 172.21.15.33:0/2696661731 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0007070 con 0x55dc4a090070 2024-05-08T21:20:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.640+0000 7f08c37fe700 1 -- 172.21.15.33:0/2696661731 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08d0011050 con 0x55dc4a090070 2024-05-08T21:20:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.640+0000 7f08c37fe700 1 -- 172.21.15.33:0/2696661731 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0007ea0 con 0x55dc4a090070 2024-05-08T21:20:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.641+0000 7f08eb020b80 1 -- 172.21.15.33:0/2696661731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 msgr2=0x55dc4a0bcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.641+0000 7f08eb020b80 1 --2- 172.21.15.33:0/2696661731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a0bcc40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f08d0002230 tx=0x7f08d00155e0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.641+0000 7f08eb020b80 1 -- 172.21.15.33:0/2696661731 shutdown_connections 2024-05-08T21:20:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.641+0000 7f08eb020b80 1 --2- 172.21.15.33:0/2696661731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a0bcc40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.641+0000 7f08eb020b80 1 --2- 172.21.15.33:0/2696661731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 0x55dc4a08cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.641+0000 7f08eb020b80 1 --2- 172.21.15.33:0/2696661731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08f6b0 0x55dc4a096550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.641+0000 7f08eb020b80 1 -- 172.21.15.33:0/2696661731 >> 172.21.15.33:0/2696661731 conn(0x55dc49fe4210 msgr2=0x55dc4a090450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.641+0000 7f08eb020b80 1 -- 172.21.15.33:0/2696661731 shutdown_connections 2024-05-08T21:20:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.641+0000 7f08eb020b80 1 -- 172.21.15.33:0/2696661731 wait complete. 2024-05-08T21:20:04.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.641+0000 7f08eb020b80 1 -- start start 2024-05-08T21:20:04.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.641+0000 7f08eb020b80 1 Processor -- start 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08eb020b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 0x55dc4a0ba030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08eb020b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08f6b0 0x55dc4a0ba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08eb020b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a101900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08eb020b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4a0bac50 con 0x55dc4a08f6b0 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08eb020b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4a101e40 con 0x55dc4a090070 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08eb020b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4a101fd0 con 0x55dc4a08ed50 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08eb020b80 10 client.0 _ll_register_callbacks cb 0x55dc49f7bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08dcc02700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 0x55dc4a0ba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08dd403700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08dcc02700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 0x55dc4a0ba030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51570/0 (socket says 172.21.15.33:51570) 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08dcc02700 1 -- 172.21.15.33:0/3274475584 learned_addr learned my addr 172.21.15.33:0/3274475584 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08dd403700 1 -- 172.21.15.33:0/3274475584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 msgr2=0x55dc4a0ba030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08dd403700 1 --2- 172.21.15.33:0/3274475584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 0x55dc4a0ba030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08dd403700 1 -- 172.21.15.33:0/3274475584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08f6b0 msgr2=0x55dc4a0ba570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08dd403700 1 --2- 172.21.15.33:0/3274475584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08f6b0 0x55dc4a0ba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08dd403700 1 -- 172.21.15.33:0/3274475584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc49e03b60 con 0x55dc4a090070 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.642+0000 7f08dcc02700 1 --2- 172.21.15.33:0/3274475584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 0x55dc4a0ba030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.643+0000 7f08dd403700 1 --2- 172.21.15.33:0/3274475584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a101900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08d00153c0 tx=0x7f08d00073a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0646ace1fb1afcb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.643+0000 7f08b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.643+0000 7f08b7fff700 1 -- 172.21.15.33:0/3274475584 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0007070 con 0x55dc4a090070 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.643+0000 7f08b7fff700 1 -- 172.21.15.33:0/3274475584 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f08d0015d00 con 0x55dc4a090070 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.643+0000 7f08eb020b80 1 -- 172.21.15.33:0/3274475584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08cc0a7420 con 0x55dc4a090070 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.643+0000 7f08b7fff700 1 -- 172.21.15.33:0/3274475584 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08d0015ec0 con 0x55dc4a090070 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.643+0000 7f08eb020b80 10 client.25820 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.643+0000 7f08eb020b80 1 -- 172.21.15.33:0/3274475584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55dc49e5bcd0 con 0x55dc4a090070 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.643+0000 7f08b7fff700 1 -- 172.21.15.33:0/3274475584 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08cc0a7420 con 0x55dc4a090070 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.643+0000 7f08b7fff700 1 -- 172.21.15.33:0/3274475584 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f08d002f040 con 0x55dc4a090070 2024-05-08T21:20:04.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.644+0000 7f08eb020b80 10 client.25820 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.644+0000 7f08eb020b80 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.644+0000 7f08eb020b80 20 client.25820 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.644+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3274475584 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc4a1034e0 0x55dc4a105900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.644+0000 7f08eb020b80 4 client.25820 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.644+0000 7f08eb020b80 1 -- 172.21.15.33:0/3274475584 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dc49fceec0 con 0x55dc4a1034e0 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.644+0000 7f08dcc02700 1 --2- 172.21.15.33:0/3274475584 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc4a1034e0 0x55dc4a105900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.644+0000 7f08b7fff700 10 client.25820 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.644+0000 7f08dcc02700 1 --2- 172.21.15.33:0/3274475584 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc4a1034e0 0x55dc4a105900 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbdb6168d046555c server_cookie=988c5ba66b7b19a6 in_seq=0 out_seq=0 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.644+0000 7f08b7fff700 1 -- 172.21.15.33:0/3274475584 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dc49fceec0 con 0x55dc4a1034e0 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.644+0000 7f08b7fff700 10 client.25820 handle_command_reply: tid=0 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.644+0000 7f08eb020b80 1 client.25820 shutdown 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08eb020b80 1 -- 172.21.15.33:0/3274475584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 msgr2=0x55dc4a101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3274475584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a101900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08d00153c0 tx=0x7f08d00073a0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08eb020b80 1 -- 172.21.15.33:0/3274475584 shutdown_connections 2024-05-08T21:20:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3274475584 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc4a1034e0 0x55dc4a105900 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3274475584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc4a090070 0x55dc4a101900 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3274475584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc4a08ed50 0x55dc4a0ba030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08eb020b80 1 --2- 172.21.15.33:0/3274475584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4a08f6b0 0x55dc4a0ba570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08b7fff700 0 client.25820 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08eb020b80 1 -- 172.21.15.33:0/3274475584 >> 172.21.15.33:0/3274475584 conn(0x55dc49fe4210 msgr2=0x55dc4a0b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08eb020b80 1 -- 172.21.15.33:0/3274475584 shutdown_connections 2024-05-08T21:20:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08eb020b80 1 -- 172.21.15.33:0/3274475584 wait complete. 2024-05-08T21:20:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08eb020b80 20 client.25820 trim_cache size 0 max 16384 2024-05-08T21:20:04.651 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:20:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:20:04.651 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:20:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: "fd5cc1d8-a5ef-4070-a3c7-8a0aa2ba1526": { 2024-05-08T21:20:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:20:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "fd5cc1d8-a5ef-4070-a3c7-8a0aa2ba1526", 2024-05-08T21:20:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:20:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:20:04.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:20:04.652 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:20:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08dee66700 1 -- 172.21.15.33:0/2776296680 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f08c80a1120 msgr2=0x7f08c80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08dee66700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f08c80a1120 0x7f08c80a35e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f08c40096b0 tx=0x7f08c40095c0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08dee66700 1 -- 172.21.15.33:0/2776296680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08d8100e70 msgr2=0x7f08d8196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.645+0000 7f08dee66700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08d8100e70 0x7f08d8196da0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0098e0 tx=0x7f08cc004730 comp rx=0 tx=0).stop 2024-05-08T21:20:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.646+0000 7f08dee66700 1 -- 172.21.15.33:0/2776296680 shutdown_connections 2024-05-08T21:20:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.646+0000 7f08dee66700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f08c80a1120 0x7f08c80a35e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.646+0000 7f08dee66700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d8102980 0x7f08d819e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.646+0000 7f08dee66700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08d8100e70 0x7f08d8196da0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.646+0000 7f08dee66700 1 --2- 172.21.15.33:0/2776296680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d8101fc0 0x7f08d81972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.646+0000 7f08dee66700 1 -- 172.21.15.33:0/2776296680 >> 172.21.15.33:0/2776296680 conn(0x7f08d80fc400 msgr2=0x7f08d81054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.646+0000 7f08dee66700 1 -- 172.21.15.33:0/2776296680 shutdown_connections 2024-05-08T21:20:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:04.646+0000 7f08dee66700 1 -- 172.21.15.33:0/2776296680 wait complete. 2024-05-08T21:20:04.662 INFO:tasks.cephfs.filesystem:scrub status for tag:fd5cc1d8-a5ef-4070-a3c7-8a0aa2ba1526 - {'path': '/', 'tag': 'fd5cc1d8-a5ef-4070-a3c7-8a0aa2ba1526', 'options': 'recursive,force'} 2024-05-08T21:20:05.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:05 smithi071 ceph-mon[47074]: pgmap v323: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:20:05.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:05 smithi082 ceph-mon[47512]: pgmap v323: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:20:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:05 smithi033 ceph-mon[42528]: pgmap v323: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:20:06.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:06 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:20:06.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:06 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:20:06.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:06 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:20:06.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:06 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:20:06.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:06 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:20:06.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:06 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:20:06.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:06 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:20:06.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:06 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:20:06.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:06 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:20:06.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:06 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:20:06.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:06 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:20:06.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:06 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:20:07.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:07 smithi071 ceph-mon[47074]: pgmap v324: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:20:07.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:07 smithi082 ceph-mon[47512]: pgmap v324: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:20:07.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:07 smithi033 ceph-mon[42528]: pgmap v324: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:20:08.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:08 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:20:08.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:08 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:20:08.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:08 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:20:09.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:09 smithi071 ceph-mon[47074]: pgmap v325: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-08T21:20:09.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:09 smithi082 ceph-mon[47512]: pgmap v325: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-08T21:20:09.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:09 smithi033 ceph-mon[42528]: pgmap v325: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-08T21:20:11.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:11 smithi071 ceph-mon[47074]: pgmap v326: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-08T21:20:11.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:11 smithi082 ceph-mon[47512]: pgmap v326: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-08T21:20:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:11 smithi033 ceph-mon[42528]: pgmap v326: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-05-08T21:20:13.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:13 smithi071 ceph-mon[47074]: pgmap v327: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-08T21:20:13.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:13 smithi082 ceph-mon[47512]: pgmap v327: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-08T21:20:13.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:13 smithi033 ceph-mon[42528]: pgmap v327: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-08T21:20:14.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:14 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:20:14.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:14 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:20:14.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:14 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:20:14.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:14 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:20:14.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:14 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:20:14.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:14 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:20:14.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:14 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:20:14.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:14 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:20:14.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:14 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:20:14.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:14 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:20:14.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:14 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:20:14.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:14 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:20:15.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:15 smithi071 ceph-mon[47074]: pgmap v328: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:20:15.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:15 smithi082 ceph-mon[47512]: pgmap v328: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:20:15.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:15 smithi033 ceph-mon[42528]: pgmap v328: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:20:17.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:17 smithi071 ceph-mon[47074]: pgmap v329: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:20:17.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:17 smithi082 ceph-mon[47512]: pgmap v329: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:20:17.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:17 smithi033 ceph-mon[42528]: pgmap v329: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-08T21:20:19.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:19 smithi071 ceph-mon[47074]: pgmap v330: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-05-08T21:20:19.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:19 smithi082 ceph-mon[47512]: pgmap v330: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-05-08T21:20:19.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:19 smithi033 ceph-mon[42528]: pgmap v330: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s 2024-05-08T21:20:21.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:21 smithi071 ceph-mon[47074]: pgmap v331: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:20:21.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:21 smithi082 ceph-mon[47512]: pgmap v331: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:20:21.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:21 smithi033 ceph-mon[42528]: pgmap v331: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T21:20:23.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:23 smithi071 ceph-mon[47074]: pgmap v332: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:20:23.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:23 smithi082 ceph-mon[47512]: pgmap v332: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:20:23.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:23 smithi033 ceph-mon[42528]: pgmap v332: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-05-08T21:20:25.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:25 smithi082 ceph-mon[47512]: pgmap v333: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:20:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:25 smithi033 ceph-mon[42528]: pgmap v333: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:20:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:25 smithi071 ceph-mon[47074]: pgmap v333: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:20:27.237 INFO:tasks.workunit.client.0.smithi033.stderr:Updating files: 85% (10461/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-08T21:20:27.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:27 smithi082 ceph-mon[47512]: pgmap v334: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:20:27.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:27 smithi033 ceph-mon[42528]: pgmap v334: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:20:27.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:27 smithi071 ceph-mon[47074]: pgmap v334: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-08T21:20:27.929 INFO:tasks.workunit.client.0.smithi033.stderr:Note: switching to 'd21e4f961318b56f0d915caa2490c3711bf13c34'. 2024-05-08T21:20:27.929 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:20:27.929 INFO:tasks.workunit.client.0.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T21:20:27.929 INFO:tasks.workunit.client.0.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr: git switch -c 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr:Or undo this operation with: 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr: git switch - 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:20:27.930 INFO:tasks.workunit.client.0.smithi033.stderr:HEAD is now at d21e4f96131 Merge PR #57343 into wip-pdonnell-testing-20240508.150423-reef 2024-05-08T21:20:27.936 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-08T21:20:27.976 INFO:tasks.workunit.client.0.smithi033.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-08T21:20:27.979 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T21:20:27.979 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-08T21:20:28.478 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-08T21:20:28.581 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-08T21:20:28.625 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T21:20:28.626 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T21:20:28.627 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-08T21:20:28.669 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T21:20:28.672 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:20:28.672 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-08T21:20:28.735 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-05-08T21:20:28.736 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-05-08T21:20:28.736 DEBUG:teuthology.orchestra.run.smithi033:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d21e4f961318b56f0d915caa2490c3711bf13c34 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-05-08T21:20:28.807 INFO:tasks.workunit.client.0.smithi033.stderr:+ set -e 2024-05-08T21:20:28.807 INFO:tasks.workunit.client.0.smithi033.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-05-08T21:20:28.811 INFO:tasks.workunit.client.0.smithi033.stderr:Cloning into 'xfstests-dev'... 2024-05-08T21:20:29.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:29 smithi082 ceph-mon[47512]: pgmap v335: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:20:29.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:29 smithi033 ceph-mon[42528]: pgmap v335: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:20:29.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:29 smithi071 ceph-mon[47074]: pgmap v335: 129 pgs: 129 active+clean; 662 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-08T21:20:31.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:31 smithi082 ceph-mon[47512]: pgmap v336: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-08T21:20:31.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:31 smithi033 ceph-mon[42528]: pgmap v336: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-08T21:20:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:31 smithi071 ceph-mon[47074]: pgmap v336: 129 pgs: 129 active+clean; 675 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-08T21:20:33.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:33 smithi082 ceph-mon[47512]: pgmap v337: 129 pgs: 129 active+clean; 847 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-08T21:20:33.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:33 smithi033 ceph-mon[42528]: pgmap v337: 129 pgs: 129 active+clean; 847 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-08T21:20:33.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:33 smithi071 ceph-mon[47074]: pgmap v337: 129 pgs: 129 active+clean; 847 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-05-08T21:20:34.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:20:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.830+0000 7f6b5481e700 1 -- 172.21.15.33:0/1485551532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500ff140 msgr2=0x7f6b500ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.830+0000 7f6b5481e700 1 --2- 172.21.15.33:0/1485551532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500ff140 0x7f6b500ff560 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009990 tx=0x7f6b44009ca0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.830+0000 7f6b5481e700 1 -- 172.21.15.33:0/1485551532 shutdown_connections 2024-05-08T21:20:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.830+0000 7f6b5481e700 1 --2- 172.21.15.33:0/1485551532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b50102b50 0x7f6b50109400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.830+0000 7f6b5481e700 1 --2- 172.21.15.33:0/1485551532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500ff140 0x7f6b500ff560 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.830+0000 7f6b5481e700 1 --2- 172.21.15.33:0/1485551532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b501021b0 0x7f6b50102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.830+0000 7f6b5481e700 1 -- 172.21.15.33:0/1485551532 >> 172.21.15.33:0/1485551532 conn(0x7f6b500faa50 msgr2=0x7f6b500fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.830+0000 7f6b5481e700 1 -- 172.21.15.33:0/1485551532 shutdown_connections 2024-05-08T21:20:34.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.830+0000 7f6b5481e700 1 -- 172.21.15.33:0/1485551532 wait complete. 2024-05-08T21:20:34.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.831+0000 7f6b5481e700 1 Processor -- start 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.831+0000 7f6b5481e700 1 -- start start 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b5481e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500ff140 0x7f6b50101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b5481e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b501021b0 0x7f6b50100160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b5481e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50102b50 0x7f6b501006a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b5481e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50100e40 con 0x7f6b500ff140 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b5481e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b50100fb0 con 0x7f6b501021b0 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b5481e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b501a0de0 con 0x7f6b50102b50 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b4e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500ff140 0x7f6b50101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b4e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500ff140 0x7f6b50101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41548/0 (socket says 172.21.15.33:41548) 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b4e59c700 1 -- 172.21.15.33:0/3045415277 learned_addr learned my addr 172.21.15.33:0/3045415277 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b4ed9d700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50102b50 0x7f6b501006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b4dd9b700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b501021b0 0x7f6b50100160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b4e59c700 1 -- 172.21.15.33:0/3045415277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50102b50 msgr2=0x7f6b501006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:34.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50102b50 0x7f6b501006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b4e59c700 1 -- 172.21.15.33:0/3045415277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b501021b0 msgr2=0x7f6b50100160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:34.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b501021b0 0x7f6b50100160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b4e59c700 1 -- 172.21.15.33:0/3045415277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b440095f0 con 0x7f6b500ff140 2024-05-08T21:20:34.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500ff140 0x7f6b50101ad0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6b44003f60 tx=0x7f6b44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:34.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b3f7fe700 1 -- 172.21.15.33:0/3045415277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4401d070 con 0x7f6b500ff140 2024-05-08T21:20:34.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b5481e700 1 -- 172.21.15.33:0/3045415277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b501a1010 con 0x7f6b500ff140 2024-05-08T21:20:34.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b3f7fe700 1 -- 172.21.15.33:0/3045415277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b4400bdb0 con 0x7f6b500ff140 2024-05-08T21:20:34.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.832+0000 7f6b3f7fe700 1 -- 172.21.15.33:0/3045415277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44022d60 con 0x7f6b500ff140 2024-05-08T21:20:34.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.833+0000 7f6b5481e700 1 -- 172.21.15.33:0/3045415277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b501a13d0 con 0x7f6b500ff140 2024-05-08T21:20:34.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.835+0000 7f6b3f7fe700 1 -- 172.21.15.33:0/3045415277 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6b44022470 con 0x7f6b500ff140 2024-05-08T21:20:34.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.836+0000 7f6b3f7fe700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b380a1120 0x7f6b380a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.836+0000 7f6b4dd9b700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b380a1120 0x7f6b380a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.836+0000 7f6b3f7fe700 1 -- 172.21.15.33:0/3045415277 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f6b440ce0b0 con 0x7f6b500ff140 2024-05-08T21:20:34.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.837+0000 7f6b4dd9b700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b380a1120 0x7f6b380a35e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009c60 tx=0x7f6b40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:34.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.837+0000 7f6b609d8b80 1 Processor -- start 2024-05-08T21:20:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.837+0000 7f6b609d8b80 1 -- start start 2024-05-08T21:20:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.837+0000 7f6b609d8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 0x562f23f8a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.837+0000 7f6b609d8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 0x562f23f8b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.837+0000 7f6b609d8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8b860 0x562f23f8bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.837+0000 7f6b609d8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f23ee1150 con 0x562f23f8a540 2024-05-08T21:20:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.837+0000 7f6b609d8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f23edeb10 con 0x562f23f8b860 2024-05-08T21:20:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.837+0000 7f6b609d8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f23eda400 con 0x562f23f8aea0 2024-05-08T21:20:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.837+0000 7f6b4e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 0x562f23f8a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.837+0000 7f6b4e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 0x562f23f8a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41550/0 (socket says 172.21.15.33:41550) 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.837+0000 7f6b4e59c700 1 -- 172.21.15.33:0/69337609 learned_addr learned my addr 172.21.15.33:0/69337609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.838+0000 7f6b4dd9b700 1 --2- 172.21.15.33:0/69337609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 0x562f23f8b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.838+0000 7f6b4ed9d700 1 --2- 172.21.15.33:0/69337609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8b860 0x562f23f8bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.838+0000 7f6b4e59c700 1 -- 172.21.15.33:0/69337609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 msgr2=0x562f23f8b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.838+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/69337609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 0x562f23f8b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.838+0000 7f6b4e59c700 1 -- 172.21.15.33:0/69337609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8b860 msgr2=0x562f23f8bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.838+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/69337609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8b860 0x562f23f8bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.838+0000 7f6b4e59c700 1 -- 172.21.15.33:0/69337609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f23d5b3b0 con 0x562f23f8a540 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.838+0000 7f6b4ed9d700 1 --2- 172.21.15.33:0/69337609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8b860 0x562f23f8bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.838+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/69337609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 0x562f23f8a960 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6b4409b160 tx=0x7f6b44004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb378fb58cb0fdc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.838+0000 7f6b3cff9700 1 -- 172.21.15.33:0/69337609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4409b8d0 con 0x562f23f8a540 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.838+0000 7f6b3cff9700 1 -- 172.21.15.33:0/69337609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b4409b1d0 con 0x562f23f8a540 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.838+0000 7f6b3cff9700 1 -- 172.21.15.33:0/69337609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4409a6c0 con 0x562f23f8a540 2024-05-08T21:20:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.839+0000 7f6b609d8b80 1 -- 172.21.15.33:0/69337609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 msgr2=0x562f23f8a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.839+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/69337609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 0x562f23f8a960 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6b4409b160 tx=0x7f6b44004320 comp rx=0 tx=0).stop 2024-05-08T21:20:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.839+0000 7f6b609d8b80 1 -- 172.21.15.33:0/69337609 shutdown_connections 2024-05-08T21:20:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.839+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/69337609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8b860 0x562f23f8bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.839+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/69337609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 0x562f23f8a960 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.839+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/69337609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 0x562f23f8b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.839+0000 7f6b609d8b80 1 -- 172.21.15.33:0/69337609 >> 172.21.15.33:0/69337609 conn(0x562f23ee8210 msgr2=0x562f23f84de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.839+0000 7f6b609d8b80 1 -- 172.21.15.33:0/69337609 shutdown_connections 2024-05-08T21:20:34.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.839+0000 7f6b609d8b80 1 -- 172.21.15.33:0/69337609 wait complete. 2024-05-08T21:20:34.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.839+0000 7f6b609d8b80 1 -- start start 2024-05-08T21:20:34.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b609d8b80 1 Processor -- start 2024-05-08T21:20:34.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b609d8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8a540 0x562f23f93a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b609d8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8aea0 0x562f23f93f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b609d8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8b860 0x562f23f9afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b609d8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f23e76560 con 0x562f23f8aea0 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b609d8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f23e76200 con 0x562f23f8a540 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b609d8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f23d5d6a0 con 0x562f23f8b860 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b609d8b80 10 client.0 _ll_register_callbacks cb 0x562f23e7fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b4dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8aea0 0x562f23f93f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b4dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8aea0 0x562f23f93f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41564/0 (socket says 172.21.15.33:41564) 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b4dd9b700 1 -- 172.21.15.33:0/3410698678 learned_addr learned my addr 172.21.15.33:0/3410698678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8a540 0x562f23f93a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b4ed9d700 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8b860 0x562f23f9afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b4dd9b700 1 -- 172.21.15.33:0/3410698678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8b860 msgr2=0x562f23f9afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b4dd9b700 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8b860 0x562f23f9afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b4dd9b700 1 -- 172.21.15.33:0/3410698678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8a540 msgr2=0x562f23f93a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b4dd9b700 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8a540 0x562f23f93a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.840+0000 7f6b4dd9b700 1 -- 172.21.15.33:0/3410698678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b501a13d0 con 0x562f23f8aea0 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b4dd9b700 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8aea0 0x562f23f93f80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6b40003900 tx=0x7f6b400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4700c5efeb8456ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b35ffb700 1 -- 172.21.15.33:0/3410698678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4001f840 con 0x562f23f8aea0 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b35ffb700 1 -- 172.21.15.33:0/3410698678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f6b4001f9a0 con 0x562f23f8aea0 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b609d8b80 1 -- 172.21.15.33:0/3410698678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f23e33270 con 0x562f23f8aea0 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b35ffb700 1 -- 172.21.15.33:0/3410698678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b40004580 con 0x562f23f8aea0 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b609d8b80 10 client.15957 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b609d8b80 1 -- 172.21.15.33:0/3410698678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562f23d07b60 con 0x562f23f8aea0 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b35ffb700 1 -- 172.21.15.33:0/3410698678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562f23e33270 con 0x562f23f8aea0 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b35ffb700 1 -- 172.21.15.33:0/3410698678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6b4001e430 con 0x562f23f8aea0 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b609d8b80 10 client.15957 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b609d8b80 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b609d8b80 20 client.15957 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:34.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562f23f9f860 0x562f23fa1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b609d8b80 4 client.15957 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.841+0000 7f6b609d8b80 1 -- 172.21.15.33:0/3410698678 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f23f59810 con 0x562f23f9f860 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.842+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562f23f9f860 0x562f23fa1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.842+0000 7f6b35ffb700 10 client.15957 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.842+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562f23f9f860 0x562f23fa1c80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4390a9888eed2a server_cookie=5ec4d7719f2704c3 in_seq=0 out_seq=0 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b35ffb700 1 -- 172.21.15.33:0/3410698678 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f23f59810 con 0x562f23f9f860 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b35ffb700 10 client.15957 handle_command_reply: tid=0 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b609d8b80 1 client.15957 shutdown 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b609d8b80 1 -- 172.21.15.33:0/3410698678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8aea0 msgr2=0x562f23f93f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8aea0 0x562f23f93f80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6b40003900 tx=0x7f6b400047c0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b609d8b80 1 -- 172.21.15.33:0/3410698678 shutdown_connections 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562f23f9f860 0x562f23fa1c80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8b860 0x562f23f9afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8a540 0x562f23f93a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/3410698678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8aea0 0x562f23f93f80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b35ffb700 0 client.15957 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b609d8b80 1 -- 172.21.15.33:0/3410698678 >> 172.21.15.33:0/3410698678 conn(0x562f23ee8210 msgr2=0x562f23f845a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b609d8b80 1 -- 172.21.15.33:0/3410698678 shutdown_connections 2024-05-08T21:20:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b609d8b80 1 -- 172.21.15.33:0/3410698678 wait complete. 2024-05-08T21:20:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.843+0000 7f6b609d8b80 20 client.15957 trim_cache size 0 max 16384 2024-05-08T21:20:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.861+0000 7f6b609d8b80 1 Processor -- start 2024-05-08T21:20:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.861+0000 7f6b609d8b80 1 -- start start 2024-05-08T21:20:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.861+0000 7f6b609d8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 0x562f23f91d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.861+0000 7f6b609d8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 0x562f23f9af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.861+0000 7f6b609d8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8b860 0x562f23f8e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.861+0000 7f6b609d8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f23f85b90 con 0x562f23f8a540 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.861+0000 7f6b609d8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f23fa6e70 con 0x562f23f8b860 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.861+0000 7f6b609d8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f23f919b0 con 0x562f23f8aea0 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b4e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 0x562f23f91d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b4e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 0x562f23f91d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41574/0 (socket says 172.21.15.33:41574) 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b4e59c700 1 -- 172.21.15.33:0/1026373915 learned_addr learned my addr 172.21.15.33:0/1026373915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b4ed9d700 1 --2- 172.21.15.33:0/1026373915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8b860 0x562f23f8e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b4dd9b700 1 --2- 172.21.15.33:0/1026373915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 0x562f23f9af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b4e59c700 1 -- 172.21.15.33:0/1026373915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 msgr2=0x562f23f9af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/1026373915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 0x562f23f9af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b4e59c700 1 -- 172.21.15.33:0/1026373915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8b860 msgr2=0x562f23f8e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/1026373915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8b860 0x562f23f8e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b4e59c700 1 -- 172.21.15.33:0/1026373915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f23ea0560 con 0x562f23f8a540 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/1026373915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 0x562f23f91d30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6b44097c40 tx=0x7f6b44004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685e21a1d66d107b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b3d7fa700 1 -- 172.21.15.33:0/1026373915 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b440b9040 con 0x562f23f8a540 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b3d7fa700 1 -- 172.21.15.33:0/1026373915 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b440a35c0 con 0x562f23f8a540 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.862+0000 7f6b3d7fa700 1 -- 172.21.15.33:0/1026373915 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b440a1320 con 0x562f23f8a540 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.863+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1026373915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 msgr2=0x562f23f91d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.863+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/1026373915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 0x562f23f91d30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6b44097c40 tx=0x7f6b44004320 comp rx=0 tx=0).stop 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.863+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1026373915 shutdown_connections 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.863+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/1026373915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8b860 0x562f23f8e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.863+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/1026373915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8a540 0x562f23f91d30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.863+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/1026373915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 0x562f23f9af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.863+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1026373915 >> 172.21.15.33:0/1026373915 conn(0x562f23ee8210 msgr2=0x562f23f8bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.863+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1026373915 shutdown_connections 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.863+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1026373915 wait complete. 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.863+0000 7f6b609d8b80 1 -- start start 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.863+0000 7f6b609d8b80 1 Processor -- start 2024-05-08T21:20:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b609d8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8a540 0x562f23f9a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b609d8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 0x562f23f9af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b609d8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8b860 0x562f23f97540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b609d8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f23f97c50 con 0x562f23f8b860 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b609d8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f23f97dc0 con 0x562f23f8a540 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b609d8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f23f97f80 con 0x562f23f8aea0 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b609d8b80 10 client.0 _ll_register_callbacks cb 0x562f23e7fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b4ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8b860 0x562f23f97540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b4ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8b860 0x562f23f97540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41578/0 (socket says 172.21.15.33:41578) 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b4ed9d700 1 -- 172.21.15.33:0/1007892855 learned_addr learned my addr 172.21.15.33:0/1007892855 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8a540 0x562f23f9a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b4dd9b700 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 0x562f23f9af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b4ed9d700 1 -- 172.21.15.33:0/1007892855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 msgr2=0x562f23f9af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b4ed9d700 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 0x562f23f9af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b4ed9d700 1 -- 172.21.15.33:0/1007892855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8a540 msgr2=0x562f23f9a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b4ed9d700 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8a540 0x562f23f9a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b4ed9d700 1 -- 172.21.15.33:0/1007892855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f23d5b3b0 con 0x562f23f8b860 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b4ed9d700 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8b860 0x562f23f97540 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6b480092f0 tx=0x7f6b48009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b8236ac35b69ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.864+0000 7f6b35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b35ffb700 1 -- 172.21.15.33:0/1007892855 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4800e040 con 0x562f23f8b860 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1007892855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b4409b1d0 con 0x562f23f8b860 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b35ffb700 1 -- 172.21.15.33:0/1007892855 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f6b48004db0 con 0x562f23f8b860 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b35ffb700 1 -- 172.21.15.33:0/1007892855 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b48018a20 con 0x562f23f8b860 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b609d8b80 10 client.15963 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1007892855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x562f23d5fcd0 con 0x562f23f8b860 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b35ffb700 1 -- 172.21.15.33:0/1007892855 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b4409b1d0 con 0x562f23f8b860 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b35ffb700 1 -- 172.21.15.33:0/1007892855 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6b48005850 con 0x562f23f8b860 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b609d8b80 10 client.15963 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b609d8b80 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b609d8b80 20 client.15963 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562f23ff7560 0x562f23ff9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b609d8b80 4 client.15963 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:20:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.865+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1007892855 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562f23ed2ec0 con 0x562f23ff7560 2024-05-08T21:20:34.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562f23ff7560 0x562f23ff9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b35ffb700 10 client.15963 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b4e59c700 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562f23ff7560 0x562f23ff9980 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e21e488adc1f11 server_cookie=55aab6939ec79f9b in_seq=0 out_seq=0 2024-05-08T21:20:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b35ffb700 1 -- 172.21.15.33:0/1007892855 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562f23ed2ec0 con 0x562f23ff7560 2024-05-08T21:20:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b35ffb700 10 client.15963 handle_command_reply: tid=0 2024-05-08T21:20:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b609d8b80 1 client.15963 shutdown 2024-05-08T21:20:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1007892855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8b860 msgr2=0x562f23f97540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8b860 0x562f23f97540 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6b480092f0 tx=0x7f6b48009320 comp rx=0 tx=0).stop 2024-05-08T21:20:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1007892855 shutdown_connections 2024-05-08T21:20:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562f23ff7560 0x562f23ff9980 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562f23f8b860 0x562f23f97540 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562f23f8a540 0x562f23f9a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b35ffb700 0 client.15963 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b609d8b80 1 --2- 172.21.15.33:0/1007892855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562f23f8aea0 0x562f23f9af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1007892855 >> 172.21.15.33:0/1007892855 conn(0x562f23ee8210 msgr2=0x562f23fa65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1007892855 shutdown_connections 2024-05-08T21:20:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.866+0000 7f6b609d8b80 1 -- 172.21.15.33:0/1007892855 wait complete. 2024-05-08T21:20:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b609d8b80 20 client.15963 trim_cache size 0 max 16384 2024-05-08T21:20:34.874 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:20:34.875 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:20:34.875 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:20:34.875 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:20:34.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b5481e700 1 -- 172.21.15.33:0/3045415277 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b380a1120 msgr2=0x7f6b380a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:34.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b5481e700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b380a1120 0x7f6b380a35e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009c60 tx=0x7f6b40009260 comp rx=0 tx=0).stop 2024-05-08T21:20:34.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b5481e700 1 -- 172.21.15.33:0/3045415277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500ff140 msgr2=0x7f6b50101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b5481e700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500ff140 0x7f6b50101ad0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6b44003f60 tx=0x7f6b44004730 comp rx=0 tx=0).stop 2024-05-08T21:20:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b5481e700 1 -- 172.21.15.33:0/3045415277 shutdown_connections 2024-05-08T21:20:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b5481e700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50102b50 0x7f6b501006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b5481e700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b500ff140 0x7f6b50101ad0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b5481e700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b380a1120 0x7f6b380a35e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b5481e700 1 --2- 172.21.15.33:0/3045415277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b501021b0 0x7f6b50100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b5481e700 1 -- 172.21.15.33:0/3045415277 >> 172.21.15.33:0/3045415277 conn(0x7f6b500faa50 msgr2=0x7f6b50105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b5481e700 1 -- 172.21.15.33:0/3045415277 shutdown_connections 2024-05-08T21:20:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:34.867+0000 7f6b5481e700 1 -- 172.21.15.33:0/3045415277 wait complete. 2024-05-08T21:20:34.882 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:20:34.882 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:20:35.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.018+0000 7fde09ffd700 1 -- 172.21.15.33:0/2534646977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde04100f70 msgr2=0x7fde041013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.018+0000 7fde09ffd700 1 --2- 172.21.15.33:0/2534646977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde04100f70 0x7fde041013d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fddf80098f0 tx=0x7fddf8009c00 comp rx=0 tx=0).stop 2024-05-08T21:20:35.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.018+0000 7fde09ffd700 1 -- 172.21.15.33:0/2534646977 shutdown_connections 2024-05-08T21:20:35.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.018+0000 7fde09ffd700 1 --2- 172.21.15.33:0/2534646977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde04101910 0x7fde041081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.018+0000 7fde09ffd700 1 --2- 172.21.15.33:0/2534646977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde040ffd80 0x7fde041001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.018+0000 7fde09ffd700 1 --2- 172.21.15.33:0/2534646977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde04100f70 0x7fde041013d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.018+0000 7fde09ffd700 1 -- 172.21.15.33:0/2534646977 >> 172.21.15.33:0/2534646977 conn(0x7fde040fb2f0 msgr2=0x7fde040fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.018+0000 7fde09ffd700 1 -- 172.21.15.33:0/2534646977 shutdown_connections 2024-05-08T21:20:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.018+0000 7fde09ffd700 1 -- 172.21.15.33:0/2534646977 wait complete. 2024-05-08T21:20:35.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.018+0000 7fde09ffd700 1 Processor -- start 2024-05-08T21:20:35.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde09ffd700 1 -- start start 2024-05-08T21:20:35.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde09ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde040ffd80 0x7fde04195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde09ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde04100f70 0x7fde04196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde09ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde04101910 0x7fde0419d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde09ffd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0419d770 con 0x7fde040ffd80 2024-05-08T21:20:35.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde09ffd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0419d8b0 con 0x7fde04100f70 2024-05-08T21:20:35.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde09ffd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0419da40 con 0x7fde04101910 2024-05-08T21:20:35.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde040ffd80 0x7fde04195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde037fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde040ffd80 0x7fde04195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41594/0 (socket says 172.21.15.33:41594) 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde037fe700 1 -- 172.21.15.33:0/323623106 learned_addr learned my addr 172.21.15.33:0/323623106 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde02ffd700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde04100f70 0x7fde04196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde03fff700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde04101910 0x7fde0419d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde037fe700 1 -- 172.21.15.33:0/323623106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde04101910 msgr2=0x7fde0419d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde037fe700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde04101910 0x7fde0419d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde037fe700 1 -- 172.21.15.33:0/323623106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde04100f70 msgr2=0x7fde04196180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde037fe700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde04100f70 0x7fde04196180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde037fe700 1 -- 172.21.15.33:0/323623106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40095f0 con 0x7fde040ffd80 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde03fff700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde04101910 0x7fde0419d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde02ffd700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde04100f70 0x7fde04196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.019+0000 7fde037fe700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde040ffd80 0x7fde04195c40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fddf400ebf0 tx=0x7fddf400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.020+0000 7fde00ff9700 1 -- 172.21.15.33:0/323623106 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4009a10 con 0x7fde040ffd80 2024-05-08T21:20:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.020+0000 7fde00ff9700 1 -- 172.21.15.33:0/323623106 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fddf400ce10 con 0x7fde040ffd80 2024-05-08T21:20:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.020+0000 7fde09ffd700 1 -- 172.21.15.33:0/323623106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf8009520 con 0x7fde040ffd80 2024-05-08T21:20:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.020+0000 7fde00ff9700 1 -- 172.21.15.33:0/323623106 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf40187c0 con 0x7fde040ffd80 2024-05-08T21:20:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.020+0000 7fde09ffd700 1 -- 172.21.15.33:0/323623106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0419e0a0 con 0x7fde040ffd80 2024-05-08T21:20:35.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.021+0000 7fde09ffd700 1 -- 172.21.15.33:0/323623106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde04107340 con 0x7fde040ffd80 2024-05-08T21:20:35.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.022+0000 7fde00ff9700 1 -- 172.21.15.33:0/323623106 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fddf4010430 con 0x7fde040ffd80 2024-05-08T21:20:35.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.022+0000 7fde00ff9700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fddec0a1120 0x7fddec0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.023+0000 7fde02ffd700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fddec0a1120 0x7fddec0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.023+0000 7fde00ff9700 1 -- 172.21.15.33:0/323623106 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fddf4014070 con 0x7fde040ffd80 2024-05-08T21:20:35.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.023+0000 7fde02ffd700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fddec0a1120 0x7fddec0a35e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fddf800fa00 tx=0x7fddf8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.028+0000 7fde00ff9700 1 -- 172.21.15.33:0/323623106 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fddf4096710 con 0x7fde040ffd80 2024-05-08T21:20:35.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.297+0000 7fde09ffd700 1 -- 172.21.15.33:0/323623106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde04067480 con 0x7fde040ffd80 2024-05-08T21:20:35.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.298+0000 7fde00ff9700 1 -- 172.21.15.33:0/323623106 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10899 (secure 0 0 0) 0x7fddf4099d30 con 0x7fde040ffd80 2024-05-08T21:20:35.300 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:20:35.300 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:18:49.368057+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:20:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.301+0000 7fde09ffd700 1 -- 172.21.15.33:0/323623106 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fddec0a1120 msgr2=0x7fddec0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.301+0000 7fde09ffd700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fddec0a1120 0x7fddec0a35e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fddf800fa00 tx=0x7fddf8019040 comp rx=0 tx=0).stop 2024-05-08T21:20:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.301+0000 7fde09ffd700 1 -- 172.21.15.33:0/323623106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde040ffd80 msgr2=0x7fde04195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.301+0000 7fde09ffd700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde040ffd80 0x7fde04195c40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fddf400ebf0 tx=0x7fddf400c2d0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.301+0000 7fde09ffd700 1 -- 172.21.15.33:0/323623106 shutdown_connections 2024-05-08T21:20:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.301+0000 7fde09ffd700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde04101910 0x7fde0419d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.301+0000 7fde09ffd700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde040ffd80 0x7fde04195c40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.301+0000 7fde09ffd700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fddec0a1120 0x7fddec0a35e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.301+0000 7fde09ffd700 1 --2- 172.21.15.33:0/323623106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde04100f70 0x7fde04196180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.301+0000 7fde09ffd700 1 -- 172.21.15.33:0/323623106 >> 172.21.15.33:0/323623106 conn(0x7fde040fb2f0 msgr2=0x7fde04104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.301+0000 7fde09ffd700 1 -- 172.21.15.33:0/323623106 shutdown_connections 2024-05-08T21:20:35.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.301+0000 7fde09ffd700 1 -- 172.21.15.33:0/323623106 wait complete. 2024-05-08T21:20:35.304 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-05-08T21:20:35.317 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:20:35.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.458+0000 7f8a0fa75700 1 -- 172.21.15.33:0/2687987388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a08100f70 msgr2=0x7f8a081013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.458+0000 7f8a0fa75700 1 --2- 172.21.15.33:0/2687987388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a08100f70 0x7f8a081013d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f89f80099c0 tx=0x7f89f8009cd0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.458+0000 7f8a0fa75700 1 -- 172.21.15.33:0/2687987388 shutdown_connections 2024-05-08T21:20:35.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.458+0000 7f8a0fa75700 1 --2- 172.21.15.33:0/2687987388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a08101910 0x7f8a081081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.458+0000 7f8a0fa75700 1 --2- 172.21.15.33:0/2687987388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a080ffd80 0x7f8a081001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.458+0000 7f8a0fa75700 1 --2- 172.21.15.33:0/2687987388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a08100f70 0x7f8a081013d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.458+0000 7f8a0fa75700 1 -- 172.21.15.33:0/2687987388 >> 172.21.15.33:0/2687987388 conn(0x7f8a080fb310 msgr2=0x7f8a080fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.458+0000 7f8a0fa75700 1 -- 172.21.15.33:0/2687987388 shutdown_connections 2024-05-08T21:20:35.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.458+0000 7f8a0fa75700 1 -- 172.21.15.33:0/2687987388 wait complete. 2024-05-08T21:20:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0fa75700 1 Processor -- start 2024-05-08T21:20:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0fa75700 1 -- start start 2024-05-08T21:20:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0fa75700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a080ffd80 0x7f8a08195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0fa75700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a08100f70 0x7f8a08196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0fa75700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a08101910 0x7f8a0819d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0fa75700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0819d720 con 0x7f8a080ffd80 2024-05-08T21:20:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0fa75700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0819d860 con 0x7f8a08100f70 2024-05-08T21:20:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0fa75700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0819d9f0 con 0x7f8a08101910 2024-05-08T21:20:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0d811700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a080ffd80 0x7f8a08195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0d811700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a080ffd80 0x7f8a08195bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41610/0 (socket says 172.21.15.33:41610) 2024-05-08T21:20:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0d811700 1 -- 172.21.15.33:0/116538039 learned_addr learned my addr 172.21.15.33:0/116538039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0e012700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a08101910 0x7f8a0819d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0d010700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a08100f70 0x7f8a08196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0d811700 1 -- 172.21.15.33:0/116538039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a08101910 msgr2=0x7f8a0819d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0d811700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a08101910 0x7f8a0819d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0d811700 1 -- 172.21.15.33:0/116538039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a08100f70 msgr2=0x7f8a08196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0d811700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a08100f70 0x7f8a08196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0d811700 1 -- 172.21.15.33:0/116538039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f80095f0 con 0x7f8a080ffd80 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.459+0000 7f8a0d010700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a08100f70 0x7f8a08196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.460+0000 7f8a0d811700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a080ffd80 0x7f8a08195bf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400e970 tx=0x7f8a0400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.460+0000 7f89feffd700 1 -- 172.21.15.33:0/116538039 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04009790 con 0x7f8a080ffd80 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.460+0000 7f89feffd700 1 -- 172.21.15.33:0/116538039 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a0400eec0 con 0x7f8a080ffd80 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.460+0000 7f8a0fa75700 1 -- 172.21.15.33:0/116538039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0819dcd0 con 0x7f8a080ffd80 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.460+0000 7f89feffd700 1 -- 172.21.15.33:0/116538039 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04003c10 con 0x7f8a080ffd80 2024-05-08T21:20:35.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.460+0000 7f8a0fa75700 1 -- 172.21.15.33:0/116538039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0819e190 con 0x7f8a080ffd80 2024-05-08T21:20:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.462+0000 7f89feffd700 1 -- 172.21.15.33:0/116538039 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8a04003db0 con 0x7f8a080ffd80 2024-05-08T21:20:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.463+0000 7f89feffd700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f89f40a1120 0x7f89f40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.463+0000 7f89feffd700 1 -- 172.21.15.33:0/116538039 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8a04014070 con 0x7f8a080ffd80 2024-05-08T21:20:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.463+0000 7f8a0d010700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f89f40a1120 0x7f89f40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.464+0000 7f8a0d010700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f89f40a1120 0x7f89f40a35e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f89f800b580 tx=0x7f89f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.464+0000 7f8a1bc2fb80 1 Processor -- start 2024-05-08T21:20:35.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.464+0000 7f8a1bc2fb80 1 -- start start 2024-05-08T21:20:35.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.464+0000 7f8a1bc2fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfb0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.464+0000 7f8a1bc2fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfb1300 0x55f37cfb1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.464+0000 7f8a1bc2fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfc0200 0x55f37cfc25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.464+0000 7f8a1bc2fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37cf0d150 con 0x55f37cfb27b0 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.464+0000 7f8a1bc2fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37cf0ab10 con 0x55f37cfc0200 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.464+0000 7f8a1bc2fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37cf06400 con 0x55f37cfb1300 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.464+0000 7f8a0d811700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfb0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.464+0000 7f8a0d811700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfb0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41622/0 (socket says 172.21.15.33:41622) 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.464+0000 7f8a0d811700 1 -- 172.21.15.33:0/555100082 learned_addr learned my addr 172.21.15.33:0/555100082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.465+0000 7f8a0e012700 1 --2- 172.21.15.33:0/555100082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfc0200 0x55f37cfc25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.465+0000 7f8a0d010700 1 --2- 172.21.15.33:0/555100082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfb1300 0x55f37cfb1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.465+0000 7f8a0d811700 1 -- 172.21.15.33:0/555100082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfb1300 msgr2=0x55f37cfb1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.465+0000 7f8a0d811700 1 --2- 172.21.15.33:0/555100082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfb1300 0x55f37cfb1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.465+0000 7f8a0d811700 1 -- 172.21.15.33:0/555100082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfc0200 msgr2=0x55f37cfc25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.465+0000 7f8a0d811700 1 --2- 172.21.15.33:0/555100082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfc0200 0x55f37cfc25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.465+0000 7f8a0d811700 1 -- 172.21.15.33:0/555100082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f37cd873b0 con 0x55f37cfb27b0 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.465+0000 7f8a0d811700 1 --2- 172.21.15.33:0/555100082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfb0dc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8a040964a0 tx=0x7f8a04099be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888bfef59bf1ab12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.465+0000 7f89f3fff700 1 -- 172.21.15.33:0/555100082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040b1070 con 0x55f37cfb27b0 2024-05-08T21:20:35.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.465+0000 7f89f3fff700 1 -- 172.21.15.33:0/555100082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a0409a6a0 con 0x55f37cfb27b0 2024-05-08T21:20:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.465+0000 7f89f3fff700 1 -- 172.21.15.33:0/555100082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0401fde0 con 0x55f37cfb27b0 2024-05-08T21:20:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.466+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/555100082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 msgr2=0x55f37cfb0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.466+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/555100082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfb0dc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8a040964a0 tx=0x7f8a04099be0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.466+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/555100082 shutdown_connections 2024-05-08T21:20:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.466+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/555100082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfc0200 0x55f37cfc25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.466+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/555100082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfb0dc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.466+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/555100082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfb1300 0x55f37cfb1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.466+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/555100082 >> 172.21.15.33:0/555100082 conn(0x55f37cf14210 msgr2=0x55f37cfb3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.466+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/555100082 shutdown_connections 2024-05-08T21:20:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.466+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/555100082 wait complete. 2024-05-08T21:20:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.466+0000 7f8a1bc2fb80 1 -- start start 2024-05-08T21:20:35.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a1bc2fb80 1 Processor -- start 2024-05-08T21:20:35.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a1bc2fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfb1300 0x55f37cfc4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a1bc2fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfc48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a1bc2fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfc0200 0x55f37cfcb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a1bc2fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37cea2560 con 0x55f37cfb27b0 2024-05-08T21:20:35.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a1bc2fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37cea2200 con 0x55f37cfb1300 2024-05-08T21:20:35.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a1bc2fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37cd896a0 con 0x55f37cfc0200 2024-05-08T21:20:35.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a1bc2fb80 10 client.0 _ll_register_callbacks cb 0x55f37ceabb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:35.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a0d010700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfc48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a0d010700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfc48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41628/0 (socket says 172.21.15.33:41628) 2024-05-08T21:20:35.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a0d010700 1 -- 172.21.15.33:0/1190766901 learned_addr learned my addr 172.21.15.33:0/1190766901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a0d811700 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfb1300 0x55f37cfc4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.467+0000 7f8a0e012700 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfc0200 0x55f37cfcb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f8a0d811700 1 -- 172.21.15.33:0/1190766901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfc0200 msgr2=0x55f37cfcb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f8a0d811700 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfc0200 0x55f37cfcb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f8a0d811700 1 -- 172.21.15.33:0/1190766901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 msgr2=0x55f37cfc48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f8a0d811700 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfc48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f8a0d811700 1 -- 172.21.15.33:0/1190766901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0819e190 con 0x55f37cfb1300 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f8a0d010700 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfc48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f8a0d811700 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfb1300 0x55f37cfc4390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a040964a0 tx=0x7f8a04095c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b83b2c3d08c0cacc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f89f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f89f17fa700 1 -- 172.21.15.33:0/1190766901 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0405fea0 con 0x55f37cfb1300 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1190766901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f37ce5f270 con 0x55f37cfb1300 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f89f17fa700 1 -- 172.21.15.33:0/1190766901 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8a040a2420 con 0x55f37cfb1300 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f89f17fa700 1 -- 172.21.15.33:0/1190766901 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a040b1050 con 0x55f37cfb1300 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f8a1bc2fb80 10 client.25856 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1190766901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f37cd33b60 con 0x55f37cfb1300 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f89f17fa700 1 -- 172.21.15.33:0/1190766901 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f37ce5f270 con 0x55f37cfb1300 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.468+0000 7f89f17fa700 1 -- 172.21.15.33:0/1190766901 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8a040cab10 con 0x55f37cfb1300 2024-05-08T21:20:35.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.469+0000 7f8a1bc2fb80 10 client.25856 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.469+0000 7f8a1bc2fb80 10 client.25856 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.469+0000 7f8a1bc2fb80 20 client.25856 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.469+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55f37cfcfc40 0x55f37cfd2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.469+0000 7f8a1bc2fb80 4 client.25856 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.469+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1190766901 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f37cf85810 con 0x55f37cfcfc40 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.469+0000 7f8a0d010700 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55f37cfcfc40 0x55f37cfd2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.469+0000 7f89f17fa700 10 client.25856 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.469+0000 7f8a0d010700 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55f37cfcfc40 0x55f37cfd2060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d60cb9a062b084d server_cookie=9c902f0c09c3299b in_seq=0 out_seq=0 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f89f17fa700 1 -- 172.21.15.33:0/1190766901 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f37cf85810 con 0x55f37cfcfc40 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f89f17fa700 10 client.25856 handle_command_reply: tid=0 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f8a1bc2fb80 1 client.25856 shutdown 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1190766901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfb1300 msgr2=0x55f37cfc4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfb1300 0x55f37cfc4390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a040964a0 tx=0x7f8a04095c70 comp rx=0 tx=0).stop 2024-05-08T21:20:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1190766901 shutdown_connections 2024-05-08T21:20:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55f37cfcfc40 0x55f37cfd2060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfc0200 0x55f37cfcb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f89f17fa700 0 client.25856 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:20:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfb1300 0x55f37cfc4390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/1190766901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfc48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1190766901 >> 172.21.15.33:0/1190766901 conn(0x55f37cf14210 msgr2=0x55f37cfb05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1190766901 shutdown_connections 2024-05-08T21:20:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1190766901 wait complete. 2024-05-08T21:20:35.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.470+0000 7f8a1bc2fb80 20 client.25856 trim_cache size 0 max 16384 2024-05-08T21:20:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.484+0000 7f8a1bc2fb80 1 Processor -- start 2024-05-08T21:20:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.484+0000 7f8a1bc2fb80 1 -- start start 2024-05-08T21:20:35.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a1bc2fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfb1300 0x55f37cfb3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a1bc2fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfcb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a1bc2fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfc0200 0x55f37cfb3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a1bc2fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37cfaddc0 con 0x55f37cfb27b0 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a1bc2fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37cfc5370 con 0x55f37cfc0200 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a1bc2fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37cfbc1e0 con 0x55f37cfb1300 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a0d010700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfcb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a0d010700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfcb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41634/0 (socket says 172.21.15.33:41634) 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a0d010700 1 -- 172.21.15.33:0/142629520 learned_addr learned my addr 172.21.15.33:0/142629520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a0d811700 1 --2- 172.21.15.33:0/142629520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfb1300 0x55f37cfb3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a0e012700 1 --2- 172.21.15.33:0/142629520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfc0200 0x55f37cfb3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a0d010700 1 -- 172.21.15.33:0/142629520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfb1300 msgr2=0x55f37cfb3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a0d010700 1 --2- 172.21.15.33:0/142629520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfb1300 0x55f37cfb3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a0d010700 1 -- 172.21.15.33:0/142629520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfc0200 msgr2=0x55f37cfb3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a0d010700 1 --2- 172.21.15.33:0/142629520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfc0200 0x55f37cfb3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a0d010700 1 -- 172.21.15.33:0/142629520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f37cecc560 con 0x55f37cfb27b0 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.485+0000 7f8a0d010700 1 --2- 172.21.15.33:0/142629520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfcb880 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009990 tx=0x7f89f8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69588c0b4343d07b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.486+0000 7f89fcda4700 1 -- 172.21.15.33:0/142629520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f801d040 con 0x55f37cfb27b0 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.486+0000 7f89fcda4700 1 -- 172.21.15.33:0/142629520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89f802b040 con 0x55f37cfb27b0 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.486+0000 7f89fcda4700 1 -- 172.21.15.33:0/142629520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80338a0 con 0x55f37cfb27b0 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.486+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/142629520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 msgr2=0x55f37cfcb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.486+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/142629520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfcb880 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009990 tx=0x7f89f8021000 comp rx=0 tx=0).stop 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.486+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/142629520 shutdown_connections 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.486+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/142629520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfc0200 0x55f37cfb3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.486+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/142629520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfb1300 0x55f37cfb3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.486+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/142629520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfcb880 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.486+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/142629520 >> 172.21.15.33:0/142629520 conn(0x55f37cf14210 msgr2=0x55f37cfc3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.486+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/142629520 shutdown_connections 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.486+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/142629520 wait complete. 2024-05-08T21:20:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.487+0000 7f8a1bc2fb80 1 -- start start 2024-05-08T21:20:35.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.487+0000 7f8a1bc2fb80 1 Processor -- start 2024-05-08T21:20:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.487+0000 7f8a1bc2fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfb1300 0x55f37cfc75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.487+0000 7f8a1bc2fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfc7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.487+0000 7f8a1bc2fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfc0200 0x55f37cfcab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.487+0000 7f8a1bc2fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37cfcb090 con 0x55f37cfb27b0 2024-05-08T21:20:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.487+0000 7f8a1bc2fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37cfcb200 con 0x55f37cfb1300 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.487+0000 7f8a1bc2fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f37cfcb3c0 con 0x55f37cfc0200 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.487+0000 7f8a1bc2fb80 10 client.0 _ll_register_callbacks cb 0x55f37ceabb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.487+0000 7f8a0d010700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfc7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.487+0000 7f8a0d010700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfc7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41650/0 (socket says 172.21.15.33:41650) 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f8a0d010700 1 -- 172.21.15.33:0/1938989390 learned_addr learned my addr 172.21.15.33:0/1938989390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f8a0e012700 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfc0200 0x55f37cfcab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f8a0d811700 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfb1300 0x55f37cfc75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f8a0d010700 1 -- 172.21.15.33:0/1938989390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfc0200 msgr2=0x55f37cfcab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f8a0d010700 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfc0200 0x55f37cfcab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f8a0d010700 1 -- 172.21.15.33:0/1938989390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfb1300 msgr2=0x55f37cfc75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f8a0d010700 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfb1300 0x55f37cfc75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f8a0d010700 1 -- 172.21.15.33:0/1938989390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a08108b10 con 0x55f37cfb27b0 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f8a0d010700 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfc7b30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f89f8036040 tx=0x7f89f8005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736fa53effd6bfdd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f89f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f89f17fa700 1 -- 172.21.15.33:0/1938989390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f801d050 con 0x55f37cfb27b0 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f89f17fa700 1 -- 172.21.15.33:0/1938989390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f89f8033020 con 0x55f37cfb27b0 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1938989390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a0409a6a0 con 0x55f37cfb27b0 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f89f17fa700 1 -- 172.21.15.33:0/1938989390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89f80331e0 con 0x55f37cfb27b0 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f8a1bc2fb80 10 client.15987 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.488+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1938989390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55f37cd8bcd0 con 0x55f37cfb27b0 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.489+0000 7f89f17fa700 1 -- 172.21.15.33:0/1938989390 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a0409a6a0 con 0x55f37cfb27b0 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.489+0000 7f89f17fa700 1 -- 172.21.15.33:0/1938989390 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f89f800fbb0 con 0x55f37cfb27b0 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.489+0000 7f8a1bc2fb80 10 client.15987 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.489+0000 7f8a1bc2fb80 10 client.15987 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.489+0000 7f8a1bc2fb80 20 client.15987 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.489+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55f37d026ba0 0x55f37d028f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.489+0000 7f8a1bc2fb80 4 client.15987 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:20:35.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.489+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1938989390 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f37cefeec0 con 0x55f37d026ba0 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.489+0000 7f8a0d811700 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55f37d026ba0 0x55f37d028f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.489+0000 7f8a0d811700 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55f37d026ba0 0x55f37d028f90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4aa60b538bcc53ce server_cookie=c0379af804630958 in_seq=0 out_seq=0 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.489+0000 7f89f17fa700 10 client.15987 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f89f17fa700 1 -- 172.21.15.33:0/1938989390 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f37cefeec0 con 0x55f37d026ba0 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f89f17fa700 10 client.15987 handle_command_reply: tid=0 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f8a1bc2fb80 1 client.15987 shutdown 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1938989390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 msgr2=0x55f37cfc7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfc7b30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f89f8036040 tx=0x7f89f8005ea0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1938989390 shutdown_connections 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55f37d026ba0 0x55f37d028f90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f37cfc0200 0x55f37cfcab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f37cfb1300 0x55f37cfc75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f8a1bc2fb80 1 --2- 172.21.15.33:0/1938989390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f37cfb27b0 0x55f37cfc7b30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f89f17fa700 0 client.15987 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:20:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1938989390 >> 172.21.15.33:0/1938989390 conn(0x55f37cf14210 msgr2=0x55f37cfbc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1938989390 shutdown_connections 2024-05-08T21:20:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f8a1bc2fb80 1 -- 172.21.15.33:0/1938989390 wait complete. 2024-05-08T21:20:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.490+0000 7f8a1bc2fb80 20 client.15987 trim_cache size 0 max 16384 2024-05-08T21:20:35.498 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:20:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.491+0000 7f8a0fa75700 1 -- 172.21.15.33:0/116538039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f89f40a1120 msgr2=0x7f89f40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.491+0000 7f8a0fa75700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f89f40a1120 0x7f89f40a35e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f89f800b580 tx=0x7f89f801a040 comp rx=0 tx=0).stop 2024-05-08T21:20:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.491+0000 7f8a0fa75700 1 -- 172.21.15.33:0/116538039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a080ffd80 msgr2=0x7f8a08195bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.491+0000 7f8a0fa75700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a080ffd80 0x7f8a08195bf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400e970 tx=0x7f8a0400ec80 comp rx=0 tx=0).stop 2024-05-08T21:20:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.491+0000 7f8a0fa75700 1 -- 172.21.15.33:0/116538039 shutdown_connections 2024-05-08T21:20:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.491+0000 7f8a0fa75700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a08101910 0x7f8a0819d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.491+0000 7f8a0fa75700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a080ffd80 0x7f8a08195bf0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.491+0000 7f8a0fa75700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f89f40a1120 0x7f89f40a35e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.491+0000 7f8a0fa75700 1 --2- 172.21.15.33:0/116538039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a08100f70 0x7f8a08196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.491+0000 7f8a0fa75700 1 -- 172.21.15.33:0/116538039 >> 172.21.15.33:0/116538039 conn(0x7f8a080fb310 msgr2=0x7f8a08104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.491+0000 7f8a0fa75700 1 -- 172.21.15.33:0/116538039 shutdown_connections 2024-05-08T21:20:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.491+0000 7f8a0fa75700 1 -- 172.21.15.33:0/116538039 wait complete. 2024-05-08T21:20:35.506 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:20:35.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:35 smithi082 ceph-mon[47512]: pgmap v338: 129 pgs: 129 active+clean; 5.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 395 KiB/s wr, 16 op/s 2024-05-08T21:20:35.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:35 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/323623106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:20:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.643+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/974658943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f98100fb0 msgr2=0x7f8f98101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.643+0000 7f8f9d9bc700 1 --2- 172.21.15.33:0/974658943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f98100fb0 0x7f8f98101410 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8f800099c0 tx=0x7f8f80009cd0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.643+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/974658943 shutdown_connections 2024-05-08T21:20:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.643+0000 7f8f9d9bc700 1 --2- 172.21.15.33:0/974658943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f98101950 0x7f8f98108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.643+0000 7f8f9d9bc700 1 --2- 172.21.15.33:0/974658943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f980ffdc0 0x7f8f981001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.643+0000 7f8f9d9bc700 1 --2- 172.21.15.33:0/974658943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f98100fb0 0x7f8f98101410 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.643+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/974658943 >> 172.21.15.33:0/974658943 conn(0x7f8f980fb370 msgr2=0x7f8f980fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.643+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/974658943 shutdown_connections 2024-05-08T21:20:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.643+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/974658943 wait complete. 2024-05-08T21:20:35.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.644+0000 7f8f9d9bc700 1 Processor -- start 2024-05-08T21:20:35.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.644+0000 7f8f9d9bc700 1 -- start start 2024-05-08T21:20:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f9d9bc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f980ffdc0 0x7f8f98103d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f9d9bc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f98100fb0 0x7f8f98102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f9d9bc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f98101950 0x7f8f981028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f9d9bc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98103070 con 0x7f8f98100fb0 2024-05-08T21:20:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f9d9bc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f981031e0 con 0x7f8f98101950 2024-05-08T21:20:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f9d9bc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f981a0d90 con 0x7f8f980ffdc0 2024-05-08T21:20:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f8e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f98100fb0 0x7f8f98102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f8e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f98100fb0 0x7f8f98102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41664/0 (socket says 172.21.15.33:41664) 2024-05-08T21:20:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f8e7fc700 1 -- 172.21.15.33:0/214173491 learned_addr learned my addr 172.21.15.33:0/214173491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f977fe700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f98101950 0x7f8f981028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f980ffdc0 0x7f8f98103d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f8e7fc700 1 -- 172.21.15.33:0/214173491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f980ffdc0 msgr2=0x7f8f98103d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f8e7fc700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f980ffdc0 0x7f8f98103d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f8e7fc700 1 -- 172.21.15.33:0/214173491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f98101950 msgr2=0x7f8f981028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f8e7fc700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f98101950 0x7f8f981028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f8e7fc700 1 -- 172.21.15.33:0/214173491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800095f0 con 0x7f8f98100fb0 2024-05-08T21:20:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f977fe700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f98101950 0x7f8f981028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f8e7fc700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f98100fb0 0x7f8f98102390 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000c00 tx=0x7f8f8000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.645+0000 7f8f9c9ba700 1 -- 172.21.15.33:0/214173491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8001d070 con 0x7f8f98100fb0 2024-05-08T21:20:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.646+0000 7f8f9c9ba700 1 -- 172.21.15.33:0/214173491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f8000bd30 con 0x7f8f98100fb0 2024-05-08T21:20:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.646+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/214173491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f981a0fc0 con 0x7f8f98100fb0 2024-05-08T21:20:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.646+0000 7f8f9c9ba700 1 -- 172.21.15.33:0/214173491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f800173f0 con 0x7f8f98100fb0 2024-05-08T21:20:35.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.646+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/214173491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f981a1330 con 0x7f8f98100fb0 2024-05-08T21:20:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.648+0000 7f8f9c9ba700 1 -- 172.21.15.33:0/214173491 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8f80022470 con 0x7f8f98100fb0 2024-05-08T21:20:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.648+0000 7f8f9c9ba700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8f840a1120 0x7f8f840a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.649+0000 7f8f9c9ba700 1 -- 172.21.15.33:0/214173491 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8f800cde80 con 0x7f8f98100fb0 2024-05-08T21:20:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.649+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8f840a1120 0x7f8f840a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.649+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8f840a1120 0x7f8f840a35e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8f98100e10 tx=0x7f8f8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8fa9b76b80 1 Processor -- start 2024-05-08T21:20:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8fa9b76b80 1 -- start start 2024-05-08T21:20:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8fa9b76b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bfe4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8fa9b76b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bfe5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8fa9b76b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 0x55e52bfe5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8fa9b76b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52bf3b150 con 0x55e52bfe5860 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8fa9b76b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52bf38b10 con 0x55e52bfe4ea0 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8fa9b76b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52bf34400 con 0x55e52bfe4540 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8f977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 0x55e52bfe5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8f977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 0x55e52bfe5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41678/0 (socket says 172.21.15.33:41678) 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8f977fe700 1 -- 172.21.15.33:0/3769662670 learned_addr learned my addr 172.21.15.33:0/3769662670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/3769662670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bfe4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8f8e7fc700 1 --2- 172.21.15.33:0/3769662670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bfe5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8f977fe700 1 -- 172.21.15.33:0/3769662670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 msgr2=0x55e52bfe4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8f977fe700 1 --2- 172.21.15.33:0/3769662670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bfe4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8f977fe700 1 -- 172.21.15.33:0/3769662670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 msgr2=0x55e52bfe5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8f977fe700 1 --2- 172.21.15.33:0/3769662670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bfe5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8f977fe700 1 -- 172.21.15.33:0/3769662670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e52bdb53b0 con 0x55e52bfe5860 2024-05-08T21:20:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8f8e7fc700 1 --2- 172.21.15.33:0/3769662670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bfe5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.650+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/3769662670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bfe4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.651+0000 7f8f977fe700 1 --2- 172.21.15.33:0/3769662670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 0x55e52bfe5ce0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8f90003eb0 tx=0x7f8f90009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a5c2898c7a104 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.651+0000 7f8f8dffb700 1 -- 172.21.15.33:0/3769662670 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f900092d0 con 0x55e52bfe5860 2024-05-08T21:20:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.651+0000 7f8f8dffb700 1 -- 172.21.15.33:0/3769662670 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f9000f040 con 0x55e52bfe5860 2024-05-08T21:20:35.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.651+0000 7f8f8dffb700 1 -- 172.21.15.33:0/3769662670 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9000e6f0 con 0x55e52bfe5860 2024-05-08T21:20:35.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.651+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/3769662670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 msgr2=0x55e52bfe5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.651+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/3769662670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 0x55e52bfe5ce0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8f90003eb0 tx=0x7f8f90009920 comp rx=0 tx=0).stop 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.652+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/3769662670 shutdown_connections 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.652+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/3769662670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 0x55e52bfe5ce0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.652+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/3769662670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bfe4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.652+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/3769662670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bfe5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.652+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/3769662670 >> 172.21.15.33:0/3769662670 conn(0x55e52bf42210 msgr2=0x55e52bfdede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.652+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/3769662670 shutdown_connections 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.652+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/3769662670 wait complete. 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.652+0000 7f8fa9b76b80 1 -- start start 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.652+0000 7f8fa9b76b80 1 Processor -- start 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8fa9b76b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bfede40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8fa9b76b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4ea0 0x55e52bfee380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8fa9b76b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe5860 0x55e52bff5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8fa9b76b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52bed0560 con 0x55e52bfe4ea0 2024-05-08T21:20:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8fa9b76b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52bed0200 con 0x55e52bfe5860 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8fa9b76b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52bdb76a0 con 0x55e52bfe4540 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f8e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4ea0 0x55e52bfee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8fa9b76b80 10 client.0 _ll_register_callbacks cb 0x55e52bed9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f8e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4ea0 0x55e52bfee380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41694/0 (socket says 172.21.15.33:41694) 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f8e7fc700 1 -- 172.21.15.33:0/91701775 learned_addr learned my addr 172.21.15.33:0/91701775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bfede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f977fe700 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe5860 0x55e52bff5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f8e7fc700 1 -- 172.21.15.33:0/91701775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 msgr2=0x55e52bfede40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f8e7fc700 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bfede40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f8e7fc700 1 -- 172.21.15.33:0/91701775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe5860 msgr2=0x55e52bff5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f8e7fc700 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe5860 0x55e52bff5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f8e7fc700 1 -- 172.21.15.33:0/91701775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8800be50 con 0x55e52bfe4ea0 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f977fe700 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe5860 0x55e52bff5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f8e7fc700 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4ea0 0x55e52bfee380 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8f800059d0 tx=0x7f8f80009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99064556f14df8a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.653+0000 7f8f76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8f76ffd700 1 -- 172.21.15.33:0/91701775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8009a6d0 con 0x55e52bfe4ea0 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8f76ffd700 1 -- 172.21.15.33:0/91701775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8f8009afe0 con 0x55e52bfe4ea0 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/91701775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e52be8d270 con 0x55e52bfe4ea0 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8f76ffd700 1 -- 172.21.15.33:0/91701775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f8009ee10 con 0x55e52bfe4ea0 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8fa9b76b80 10 client.15999 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/91701775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f8f90003b90 con 0x55e52bfe4ea0 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8f76ffd700 1 -- 172.21.15.33:0/91701775 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f80064080 con 0x55e52bfe4ea0 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8f76ffd700 1 -- 172.21.15.33:0/91701775 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8f8009dd80 con 0x55e52bfe4ea0 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8fa9b76b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8fa9b76b80 10 client.15999 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8fa9b76b80 20 client.15999 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e52bff9830 0x55e52bffbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8fa9b76b80 4 client.15999 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:35.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.654+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/91701775 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e52bfb3810 con 0x55e52bff9830 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.655+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e52bff9830 0x55e52bffbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.655+0000 7f8f76ffd700 10 client.15999 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.655+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e52bff9830 0x55e52bffbc20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fc4ff6575a2fd0bd server_cookie=bba7388dd049d0b9 in_seq=0 out_seq=0 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8f76ffd700 1 -- 172.21.15.33:0/91701775 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e52bfb3810 con 0x55e52bff9830 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8f76ffd700 10 client.15999 handle_command_reply: tid=0 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8fa9b76b80 1 client.15999 shutdown 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/91701775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4ea0 msgr2=0x55e52bfee380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4ea0 0x55e52bfee380 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8f800059d0 tx=0x7f8f80009310 comp rx=0 tx=0).stop 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/91701775 shutdown_connections 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe5860 0x55e52bff5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e52bff9830 0x55e52bffbc20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bfede40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/91701775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4ea0 0x55e52bfee380 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8f76ffd700 0 client.15999 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:20:35.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/91701775 >> 172.21.15.33:0/91701775 conn(0x55e52bf42210 msgr2=0x55e52bfde5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/91701775 shutdown_connections 2024-05-08T21:20:35.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/91701775 wait complete. 2024-05-08T21:20:35.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.656+0000 7f8fa9b76b80 20 client.15999 trim_cache size 0 max 16384 2024-05-08T21:20:35.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.670+0000 7f8fa9b76b80 1 Processor -- start 2024-05-08T21:20:35.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.670+0000 7f8fa9b76b80 1 -- start start 2024-05-08T21:20:35.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.670+0000 7f8fa9b76b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4540 0x55e52bffbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.670+0000 7f8fa9b76b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bff5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.670+0000 7f8fa9b76b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe5860 0x55e52bfed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.670+0000 7f8fa9b76b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52bfe0570 con 0x55e52bfe4540 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.670+0000 7f8fa9b76b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52c000df0 con 0x55e52bfe4ea0 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.670+0000 7f8fa9b76b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52bffc280 con 0x55e52bfe5860 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f96ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4540 0x55e52bffbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f96ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4540 0x55e52bffbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41710/0 (socket says 172.21.15.33:41710) 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f96ffd700 1 -- 172.21.15.33:0/3053425947 learned_addr learned my addr 172.21.15.33:0/3053425947 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f977fe700 1 --2- 172.21.15.33:0/3053425947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe5860 0x55e52bfed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f8e7fc700 1 --2- 172.21.15.33:0/3053425947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bff5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f96ffd700 1 -- 172.21.15.33:0/3053425947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe5860 msgr2=0x55e52bfed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/3053425947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe5860 0x55e52bfed350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f96ffd700 1 -- 172.21.15.33:0/3053425947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 msgr2=0x55e52bff5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/3053425947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bff5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f96ffd700 1 -- 172.21.15.33:0/3053425947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e52bd61b60 con 0x55e52bfe4540 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/3053425947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4540 0x55e52bffbc20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55e52bffc4c0 tx=0x7f8f88013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb0ab8790dba341 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f8effd700 1 -- 172.21.15.33:0/3053425947 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88024040 con 0x55e52bfe4540 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f8effd700 1 -- 172.21.15.33:0/3053425947 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f88015070 con 0x55e52bfe4540 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.671+0000 7f8f8effd700 1 -- 172.21.15.33:0/3053425947 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88003680 con 0x55e52bfe4540 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.672+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/3053425947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4540 msgr2=0x55e52bffbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.672+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/3053425947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4540 0x55e52bffbc20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55e52bffc4c0 tx=0x7f8f88013000 comp rx=0 tx=0).stop 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.672+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/3053425947 shutdown_connections 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.672+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/3053425947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe5860 0x55e52bfed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.672+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/3053425947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe4540 0x55e52bffbc20 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.672+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/3053425947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bff5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.672+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/3053425947 >> 172.21.15.33:0/3053425947 conn(0x55e52bf42210 msgr2=0x55e52bfdb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.672+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/3053425947 shutdown_connections 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.672+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/3053425947 wait complete. 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.672+0000 7f8fa9b76b80 1 -- start start 2024-05-08T21:20:35.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8fa9b76b80 1 Processor -- start 2024-05-08T21:20:35.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8fa9b76b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bff0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8fa9b76b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bff1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8fa9b76b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 0x55e52bff4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8fa9b76b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52bff4880 con 0x55e52bfe5860 2024-05-08T21:20:35.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8fa9b76b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52bff49f0 con 0x55e52bfe4ea0 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8fa9b76b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e52bff4bb0 con 0x55e52bfe4540 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8fa9b76b80 10 client.0 _ll_register_callbacks cb 0x55e52bed9b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8f977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 0x55e52bff4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8f977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 0x55e52bff4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41726/0 (socket says 172.21.15.33:41726) 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8f977fe700 1 -- 172.21.15.33:0/2560599801 learned_addr learned my addr 172.21.15.33:0/2560599801 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8f8e7fc700 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bff1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bff0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8f977fe700 1 -- 172.21.15.33:0/2560599801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 msgr2=0x55e52bff0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8f977fe700 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bff0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8f977fe700 1 -- 172.21.15.33:0/2560599801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 msgr2=0x55e52bff1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8f977fe700 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bff1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.673+0000 7f8f977fe700 1 -- 172.21.15.33:0/2560599801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f880115a0 con 0x55e52bfe5860 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8f977fe700 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 0x55e52bff4340 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8f90003e80 tx=0x7f8f90004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7ecafdde8a7c8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8f76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8f76ffd700 1 -- 172.21.15.33:0/2560599801 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90023070 con 0x55e52bfe5860 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8f76ffd700 1 -- 172.21.15.33:0/2560599801 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8f90007500 con 0x55e52bfe5860 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/2560599801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f9000f040 con 0x55e52bfe5860 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8f76ffd700 1 -- 172.21.15.33:0/2560599801 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f900076e0 con 0x55e52bfe5860 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8fa9b76b80 10 client.16005 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/2560599801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e52befa560 con 0x55e52bfe5860 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8f76ffd700 1 -- 172.21.15.33:0/2560599801 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f9000f040 con 0x55e52bfe5860 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8f76ffd700 1 -- 172.21.15.33:0/2560599801 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8f9000eac0 con 0x55e52bfe5860 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8fa9b76b80 10 client.16005 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8fa9b76b80 10 client.16005 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.674+0000 7f8fa9b76b80 20 client.16005 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.675+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e52c0508a0 0x55e52c052cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.675+0000 7f8fa9b76b80 4 client.16005 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.675+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/2560599801 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e52bf2cec0 con 0x55e52c0508a0 2024-05-08T21:20:35.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.675+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e52c0508a0 0x55e52c052cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.675+0000 7f8f76ffd700 10 client.16005 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:20:35.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.675+0000 7f8f96ffd700 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e52c0508a0 0x55e52c052cc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9664b849036d05e9 server_cookie=68e9bca6ef1bb122 in_seq=0 out_seq=0 2024-05-08T21:20:35.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.675+0000 7f8f76ffd700 1 -- 172.21.15.33:0/2560599801 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e52bf2cec0 con 0x55e52c0508a0 2024-05-08T21:20:35.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.675+0000 7f8f76ffd700 10 client.16005 handle_command_reply: tid=0 2024-05-08T21:20:35.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8fa9b76b80 1 client.16005 shutdown 2024-05-08T21:20:35.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/2560599801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 msgr2=0x55e52bff4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 0x55e52bff4340 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8f90003e80 tx=0x7f8f90004cc0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/2560599801 shutdown_connections 2024-05-08T21:20:35.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e52bfe5860 0x55e52bff4340 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55e52c0508a0 0x55e52c052cc0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e52bfe4540 0x55e52bff0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8fa9b76b80 1 --2- 172.21.15.33:0/2560599801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e52bfe4ea0 0x55e52bff1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8f76ffd700 0 client.16005 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:20:35.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/2560599801 >> 172.21.15.33:0/2560599801 conn(0x55e52bf42210 msgr2=0x55e52bfeb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/2560599801 shutdown_connections 2024-05-08T21:20:35.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8fa9b76b80 1 -- 172.21.15.33:0/2560599801 wait complete. 2024-05-08T21:20:35.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.676+0000 7f8fa9b76b80 20 client.16005 trim_cache size 0 max 16384 2024-05-08T21:20:35.686 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:20:35.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.677+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/214173491 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8f840a1120 msgr2=0x7f8f840a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.677+0000 7f8f9d9bc700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8f840a1120 0x7f8f840a35e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8f98100e10 tx=0x7f8f8800b410 comp rx=0 tx=0).stop 2024-05-08T21:20:35.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.677+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/214173491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f98100fb0 msgr2=0x7f8f98102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.677+0000 7f8f9d9bc700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f98100fb0 0x7f8f98102390 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000c00 tx=0x7f8f8000fa00 comp rx=0 tx=0).stop 2024-05-08T21:20:35.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.677+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/214173491 shutdown_connections 2024-05-08T21:20:35.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.677+0000 7f8f9d9bc700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8f840a1120 0x7f8f840a35e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.677+0000 7f8f9d9bc700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f98101950 0x7f8f981028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.677+0000 7f8f9d9bc700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f980ffdc0 0x7f8f98103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.677+0000 7f8f9d9bc700 1 --2- 172.21.15.33:0/214173491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f98100fb0 0x7f8f98102390 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.677+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/214173491 >> 172.21.15.33:0/214173491 conn(0x7f8f980fb370 msgr2=0x7f8f98069540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.677+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/214173491 shutdown_connections 2024-05-08T21:20:35.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.677+0000 7f8f9d9bc700 1 -- 172.21.15.33:0/214173491 wait complete. 2024-05-08T21:20:35.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:35 smithi033 ceph-mon[42528]: pgmap v338: 129 pgs: 129 active+clean; 5.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 395 KiB/s wr, 16 op/s 2024-05-08T21:20:35.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:35 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/323623106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:20:35.692 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:20:35.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:35 smithi071 ceph-mon[47074]: pgmap v338: 129 pgs: 129 active+clean; 5.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 395 KiB/s wr, 16 op/s 2024-05-08T21:20:35.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:35 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/323623106' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:20:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.829+0000 7f88b344c700 1 -- 172.21.15.33:0/2849283268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac101950 msgr2=0x7f88ac103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.829+0000 7f88b344c700 1 --2- 172.21.15.33:0/2849283268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac101950 0x7f88ac103d40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009990 tx=0x7f88a0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.829+0000 7f88b344c700 1 -- 172.21.15.33:0/2849283268 shutdown_connections 2024-05-08T21:20:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.829+0000 7f88b344c700 1 --2- 172.21.15.33:0/2849283268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88ac106bb0 0x7f88ac108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.829+0000 7f88b344c700 1 --2- 172.21.15.33:0/2849283268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac101950 0x7f88ac103d40 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.829+0000 7f88b344c700 1 --2- 172.21.15.33:0/2849283268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88ac104280 0x7f88ac106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.829+0000 7f88b344c700 1 -- 172.21.15.33:0/2849283268 >> 172.21.15.33:0/2849283268 conn(0x7f88ac0fb2a0 msgr2=0x7f88ac0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.829+0000 7f88b344c700 1 -- 172.21.15.33:0/2849283268 shutdown_connections 2024-05-08T21:20:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.829+0000 7f88b344c700 1 -- 172.21.15.33:0/2849283268 wait complete. 2024-05-08T21:20:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b344c700 1 Processor -- start 2024-05-08T21:20:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b344c700 1 -- start start 2024-05-08T21:20:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b344c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88ac101950 0x7f88ac196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b344c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88ac104280 0x7f88ac197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b344c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac106bb0 0x7f88ac19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b344c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac19e840 con 0x7f88ac106bb0 2024-05-08T21:20:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b344c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac19e9b0 con 0x7f88ac101950 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b344c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac19eb70 con 0x7f88ac104280 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b19e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac106bb0 0x7f88ac19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b19e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac106bb0 0x7f88ac19e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41738/0 (socket says 172.21.15.33:41738) 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b19e9700 1 -- 172.21.15.33:0/972423452 learned_addr learned my addr 172.21.15.33:0/972423452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b09e7700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88ac104280 0x7f88ac197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.830+0000 7f88b11e8700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88ac101950 0x7f88ac196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.831+0000 7f88b19e9700 1 -- 172.21.15.33:0/972423452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88ac104280 msgr2=0x7f88ac197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.831+0000 7f88b19e9700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88ac104280 0x7f88ac197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.831+0000 7f88b19e9700 1 -- 172.21.15.33:0/972423452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88ac101950 msgr2=0x7f88ac196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.831+0000 7f88b19e9700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88ac101950 0x7f88ac196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.831+0000 7f88b19e9700 1 -- 172.21.15.33:0/972423452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a00095f0 con 0x7f88ac106bb0 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.831+0000 7f88b19e9700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac106bb0 0x7f88ac19e270 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f88a4007ee0 tx=0x7f88a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.831+0000 7f889e7fc700 1 -- 172.21.15.33:0/972423452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4017070 con 0x7f88ac106bb0 2024-05-08T21:20:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.831+0000 7f889e7fc700 1 -- 172.21.15.33:0/972423452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88a400f040 con 0x7f88ac106bb0 2024-05-08T21:20:35.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.831+0000 7f88b344c700 1 -- 172.21.15.33:0/972423452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac19ee50 con 0x7f88ac106bb0 2024-05-08T21:20:35.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.831+0000 7f889e7fc700 1 -- 172.21.15.33:0/972423452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4005570 con 0x7f88ac106bb0 2024-05-08T21:20:35.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.831+0000 7f88b344c700 1 -- 172.21.15.33:0/972423452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac19f270 con 0x7f88ac106bb0 2024-05-08T21:20:35.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.833+0000 7f889e7fc700 1 -- 172.21.15.33:0/972423452 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f88a400db20 con 0x7f88ac106bb0 2024-05-08T21:20:35.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.834+0000 7f889e7fc700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f88980a11a0 0x7f88980a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.834+0000 7f88b11e8700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f88980a11a0 0x7f88980a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.834+0000 7f889e7fc700 1 -- 172.21.15.33:0/972423452 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f88a40cc380 con 0x7f88ac106bb0 2024-05-08T21:20:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.835+0000 7f88b11e8700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f88980a11a0 0x7f88980a3660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f88a000b2e0 tx=0x7f88a000b6d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.835+0000 7f88bf606b80 1 Processor -- start 2024-05-08T21:20:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.835+0000 7f88bf606b80 1 -- start start 2024-05-08T21:20:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.835+0000 7f88bf606b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 0x55e321691960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.835+0000 7f88bf606b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 0x55e321692320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.835+0000 7f88bf606b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321692860 0x55e321692ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.835+0000 7f88bf606b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3215e8150 con 0x55e321691ea0 2024-05-08T21:20:35.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.835+0000 7f88bf606b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3215e5b10 con 0x55e321691540 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.835+0000 7f88bf606b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3215e1400 con 0x55e321692860 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.835+0000 7f88b09e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 0x55e321692320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88b09e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 0x55e321692320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41744/0 (socket says 172.21.15.33:41744) 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88b09e7700 1 -- 172.21.15.33:0/869363166 learned_addr learned my addr 172.21.15.33:0/869363166 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88b19e9700 1 --2- 172.21.15.33:0/869363166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321692860 0x55e321692ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88b11e8700 1 --2- 172.21.15.33:0/869363166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 0x55e321691960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88b09e7700 1 -- 172.21.15.33:0/869363166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321692860 msgr2=0x55e321692ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88b09e7700 1 --2- 172.21.15.33:0/869363166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321692860 0x55e321692ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88b09e7700 1 -- 172.21.15.33:0/869363166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 msgr2=0x55e321691960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88b09e7700 1 --2- 172.21.15.33:0/869363166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 0x55e321691960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88b09e7700 1 -- 172.21.15.33:0/869363166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3214623b0 con 0x55e321691ea0 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88b11e8700 1 --2- 172.21.15.33:0/869363166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 0x55e321691960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88b09e7700 1 --2- 172.21.15.33:0/869363166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 0x55e321692320 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009d80 tx=0x7f88a8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474cb4d95cb8a175 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88977fe700 1 -- 172.21.15.33:0/869363166 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8004230 con 0x55e321691ea0 2024-05-08T21:20:35.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88977fe700 1 -- 172.21.15.33:0/869363166 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88a8004390 con 0x55e321691ea0 2024-05-08T21:20:35.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.836+0000 7f88977fe700 1 -- 172.21.15.33:0/869363166 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a80197a0 con 0x55e321691ea0 2024-05-08T21:20:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.837+0000 7f88bf606b80 1 -- 172.21.15.33:0/869363166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 msgr2=0x55e321692320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.837+0000 7f88bf606b80 1 --2- 172.21.15.33:0/869363166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 0x55e321692320 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009d80 tx=0x7f88a8011720 comp rx=0 tx=0).stop 2024-05-08T21:20:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.837+0000 7f88bf606b80 1 -- 172.21.15.33:0/869363166 shutdown_connections 2024-05-08T21:20:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.837+0000 7f88bf606b80 1 --2- 172.21.15.33:0/869363166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321692860 0x55e321692ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.837+0000 7f88bf606b80 1 --2- 172.21.15.33:0/869363166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 0x55e321691960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.837+0000 7f88bf606b80 1 --2- 172.21.15.33:0/869363166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 0x55e321692320 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.837+0000 7f88bf606b80 1 -- 172.21.15.33:0/869363166 >> 172.21.15.33:0/869363166 conn(0x55e3215ef210 msgr2=0x55e32168bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.837+0000 7f88bf606b80 1 -- 172.21.15.33:0/869363166 shutdown_connections 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.837+0000 7f88bf606b80 1 -- 172.21.15.33:0/869363166 wait complete. 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.837+0000 7f88bf606b80 1 -- start start 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88bf606b80 1 Processor -- start 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88bf606b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691540 0x55e32169ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88bf606b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691ea0 0x55e32169b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88bf606b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321692860 0x55e3216a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88bf606b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32157d560 con 0x55e321691540 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88bf606b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e32157d200 con 0x55e321691ea0 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88bf606b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3214646a0 con 0x55e321692860 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88bf606b80 10 client.0 _ll_register_callbacks cb 0x55e321586b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88b11e8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691540 0x55e32169ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88b11e8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691540 0x55e32169ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41754/0 (socket says 172.21.15.33:41754) 2024-05-08T21:20:35.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88b11e8700 1 -- 172.21.15.33:0/1800161696 learned_addr learned my addr 172.21.15.33:0/1800161696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88b19e9700 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321692860 0x55e3216a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.838+0000 7f88b09e7700 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691ea0 0x55e32169b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f88b11e8700 1 -- 172.21.15.33:0/1800161696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321692860 msgr2=0x55e3216a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f88b11e8700 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321692860 0x55e3216a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f88b11e8700 1 -- 172.21.15.33:0/1800161696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691ea0 msgr2=0x55e32169b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f88b11e8700 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691ea0 0x55e32169b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f88b11e8700 1 -- 172.21.15.33:0/1800161696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a80099e0 con 0x55e321691540 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f88b11e8700 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691540 0x55e32169ad70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f88a00095c0 tx=0x7f88a001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77332d40a4f51a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f8894ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f8894ff9700 1 -- 172.21.15.33:0/1800161696 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a000f670 con 0x55e321691540 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f8894ff9700 1 -- 172.21.15.33:0/1800161696 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f88a000f7d0 con 0x55e321691540 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f88bf606b80 1 -- 172.21.15.33:0/1800161696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e32153a270 con 0x55e321691540 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f8894ff9700 1 -- 172.21.15.33:0/1800161696 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88a000f990 con 0x55e321691540 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f88bf606b80 10 client.16017 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:35.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f88bf606b80 1 -- 172.21.15.33:0/1800161696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e32140eb60 con 0x55e321691540 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.839+0000 7f8894ff9700 1 -- 172.21.15.33:0/1800161696 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e32153a270 con 0x55e321691540 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.840+0000 7f8894ff9700 1 -- 172.21.15.33:0/1800161696 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f88a00249a0 con 0x55e321691540 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.840+0000 7f88bf606b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.840+0000 7f88bf606b80 10 client.16017 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.840+0000 7f88bf606b80 20 client.16017 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.840+0000 7f88bf606b80 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55e3216a6b70 0x55e3216a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.840+0000 7f88bf606b80 4 client.16017 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.840+0000 7f88bf606b80 1 -- 172.21.15.33:0/1800161696 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e321660810 con 0x55e3216a6b70 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.840+0000 7f88b09e7700 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55e3216a6b70 0x55e3216a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.840+0000 7f88b09e7700 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55e3216a6b70 0x55e3216a8f90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bac542946e1a0932 server_cookie=9c4d9963e3eeeec7 in_seq=0 out_seq=0 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.840+0000 7f8894ff9700 10 client.16017 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.841+0000 7f8894ff9700 1 -- 172.21.15.33:0/1800161696 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e321660810 con 0x55e3216a6b70 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.841+0000 7f8894ff9700 10 client.16017 handle_command_reply: tid=0 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.841+0000 7f88bf606b80 1 client.16017 shutdown 2024-05-08T21:20:35.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.841+0000 7f88bf606b80 1 -- 172.21.15.33:0/1800161696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691540 msgr2=0x55e32169ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.841+0000 7f88bf606b80 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691540 0x55e32169ad70 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f88a00095c0 tx=0x7f88a001a050 comp rx=0 tx=0).stop 2024-05-08T21:20:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.841+0000 7f88bf606b80 1 -- 172.21.15.33:0/1800161696 shutdown_connections 2024-05-08T21:20:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.841+0000 7f88bf606b80 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55e3216a6b70 0x55e3216a8f90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.841+0000 7f88bf606b80 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321692860 0x55e3216a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.841+0000 7f88bf606b80 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691540 0x55e32169ad70 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.841+0000 7f8894ff9700 0 client.16017 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:20:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.841+0000 7f88bf606b80 1 --2- 172.21.15.33:0/1800161696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691ea0 0x55e32169b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.841+0000 7f88bf606b80 1 -- 172.21.15.33:0/1800161696 >> 172.21.15.33:0/1800161696 conn(0x55e3215ef210 msgr2=0x55e32168b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.842+0000 7f88bf606b80 1 -- 172.21.15.33:0/1800161696 shutdown_connections 2024-05-08T21:20:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.842+0000 7f88bf606b80 1 -- 172.21.15.33:0/1800161696 wait complete. 2024-05-08T21:20:35.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.842+0000 7f88bf606b80 20 client.16017 trim_cache size 0 max 16384 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.855+0000 7f88bf606b80 1 Processor -- start 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88bf606b80 1 -- start start 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88bf606b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 0x55e321691920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88bf606b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321691ea0 0x55e3216a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88bf606b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321692860 0x55e32169a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88bf606b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3216ae1c0 con 0x55e321692860 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88bf606b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3216a6870 con 0x55e321691540 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88bf606b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e321684040 con 0x55e321691ea0 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b19e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321692860 0x55e32169a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b09e7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321691ea0 0x55e3216a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b11e8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 0x55e321691920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b11e8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 0x55e321691920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44922/0 (socket says 172.21.15.33:44922) 2024-05-08T21:20:35.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b19e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321692860 0x55e32169a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41764/0 (socket says 172.21.15.33:41764) 2024-05-08T21:20:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b11e8700 1 -- 172.21.15.33:0/2238583119 learned_addr learned my addr 172.21.15.33:0/2238583119 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b09e7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321691ea0 0x55e3216a22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:36198/0 (socket says 172.21.15.33:36198) 2024-05-08T21:20:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b11e8700 1 -- 172.21.15.33:0/2238583119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321691ea0 msgr2=0x55e3216a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b11e8700 1 --2- 172.21.15.33:0/2238583119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321691ea0 0x55e3216a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b11e8700 1 -- 172.21.15.33:0/2238583119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321692860 msgr2=0x55e32169a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b11e8700 1 --2- 172.21.15.33:0/2238583119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321692860 0x55e32169a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b11e8700 1 -- 172.21.15.33:0/2238583119 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3215a7560 con 0x55e321691540 2024-05-08T21:20:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b09e7700 1 --2- 172.21.15.33:0/2238583119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321691ea0 0x55e3216a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.856+0000 7f88b11e8700 1 --2- 172.21.15.33:0/2238583119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 0x55e321691920 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f88a0006010 tx=0x7f88a0003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3641fcf4f38538c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.857+0000 7f8897fff700 1 -- 172.21.15.33:0/2238583119 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a000bd70 con 0x55e321691540 2024-05-08T21:20:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.857+0000 7f8897fff700 1 -- 172.21.15.33:0/2238583119 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88a001e040 con 0x55e321691540 2024-05-08T21:20:35.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.857+0000 7f8897fff700 1 -- 172.21.15.33:0/2238583119 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a000f670 con 0x55e321691540 2024-05-08T21:20:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.857+0000 7f88bf606b80 1 -- 172.21.15.33:0/2238583119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 msgr2=0x55e321691920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.857+0000 7f88bf606b80 1 --2- 172.21.15.33:0/2238583119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 0x55e321691920 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f88a0006010 tx=0x7f88a0003e70 comp rx=0 tx=0).stop 2024-05-08T21:20:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.857+0000 7f88bf606b80 1 -- 172.21.15.33:0/2238583119 shutdown_connections 2024-05-08T21:20:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.857+0000 7f88bf606b80 1 --2- 172.21.15.33:0/2238583119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321692860 0x55e32169a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.857+0000 7f88bf606b80 1 --2- 172.21.15.33:0/2238583119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321691540 0x55e321691920 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.857+0000 7f88bf606b80 1 --2- 172.21.15.33:0/2238583119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321691ea0 0x55e3216a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.857+0000 7f88bf606b80 1 -- 172.21.15.33:0/2238583119 >> 172.21.15.33:0/2238583119 conn(0x55e3215ef210 msgr2=0x55e321692c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.857+0000 7f88bf606b80 1 -- 172.21.15.33:0/2238583119 shutdown_connections 2024-05-08T21:20:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.857+0000 7f88bf606b80 1 -- 172.21.15.33:0/2238583119 wait complete. 2024-05-08T21:20:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88bf606b80 1 -- start start 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88bf606b80 1 Processor -- start 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88bf606b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321691540 0x55e32169d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88bf606b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 0x55e32169dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88bf606b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321692860 0x55e3216a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88bf606b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3216a12a0 con 0x55e321691ea0 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88bf606b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3216a1410 con 0x55e321692860 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88bf606b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3216a15d0 con 0x55e321691540 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88bf606b80 10 client.0 _ll_register_callbacks cb 0x55e321586b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88b09e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 0x55e32169dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88b09e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 0x55e32169dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41776/0 (socket says 172.21.15.33:41776) 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88b09e7700 1 -- 172.21.15.33:0/1532802967 learned_addr learned my addr 172.21.15.33:0/1532802967 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.858+0000 7f88b19e9700 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321692860 0x55e3216a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f88b11e8700 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321691540 0x55e32169d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f88b09e7700 1 -- 172.21.15.33:0/1532802967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321691540 msgr2=0x55e32169d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f88b09e7700 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321691540 0x55e32169d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f88b09e7700 1 -- 172.21.15.33:0/1532802967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321692860 msgr2=0x55e3216a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f88b09e7700 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321692860 0x55e3216a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f88b09e7700 1 -- 172.21.15.33:0/1532802967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3214623b0 con 0x55e321691ea0 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f88b09e7700 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 0x55e32169dd40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f88a800a010 tx=0x7f88a8004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f428cde8752d4b4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f8894ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f8894ff9700 1 -- 172.21.15.33:0/1532802967 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8003d50 con 0x55e321691ea0 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f88bf606b80 1 -- 172.21.15.33:0/1532802967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88a8004390 con 0x55e321691ea0 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f8894ff9700 1 -- 172.21.15.33:0/1532802967 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f88a802ab00 con 0x55e321691ea0 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f8894ff9700 1 -- 172.21.15.33:0/1532802967 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88a802acc0 con 0x55e321691ea0 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f88bf606b80 10 client.16023 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f88bf606b80 1 -- 172.21.15.33:0/1532802967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e32140eb60 con 0x55e321691ea0 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.859+0000 7f8894ff9700 1 -- 172.21.15.33:0/1532802967 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88a8004390 con 0x55e321691ea0 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.860+0000 7f8894ff9700 1 -- 172.21.15.33:0/1532802967 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f88a8011440 con 0x55e321691ea0 2024-05-08T21:20:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.860+0000 7f88bf606b80 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.860+0000 7f88bf606b80 10 client.16023 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.860+0000 7f88bf606b80 20 client.16023 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.860+0000 7f88bf606b80 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55e3216fd7d0 0x55e3216ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.860+0000 7f88bf606b80 4 client.16023 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.860+0000 7f88bf606b80 1 -- 172.21.15.33:0/1532802967 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e3215d9ec0 con 0x55e3216fd7d0 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.860+0000 7f88b11e8700 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55e3216fd7d0 0x55e3216ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f8894ff9700 10 client.16023 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88b11e8700 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55e3216fd7d0 0x55e3216ffbf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90be0dd698072384 server_cookie=fef8fcc31d546f7 in_seq=0 out_seq=0 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f8894ff9700 1 -- 172.21.15.33:0/1532802967 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e3215d9ec0 con 0x55e3216fd7d0 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f8894ff9700 10 client.16023 handle_command_reply: tid=0 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88bf606b80 1 client.16023 shutdown 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88bf606b80 1 -- 172.21.15.33:0/1532802967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 msgr2=0x55e32169dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88bf606b80 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 0x55e32169dd40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f88a800a010 tx=0x7f88a8004660 comp rx=0 tx=0).stop 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88bf606b80 1 -- 172.21.15.33:0/1532802967 shutdown_connections 2024-05-08T21:20:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88bf606b80 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55e3216fd7d0 0x55e3216ffbf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88bf606b80 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e321692860 0x55e3216a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88bf606b80 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e321691540 0x55e32169d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f8894ff9700 0 client.16023 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:20:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88bf606b80 1 --2- 172.21.15.33:0/1532802967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e321691ea0 0x55e32169dd40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88bf606b80 1 -- 172.21.15.33:0/1532802967 >> 172.21.15.33:0/1532802967 conn(0x55e3215ef210 msgr2=0x55e32169a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88bf606b80 1 -- 172.21.15.33:0/1532802967 shutdown_connections 2024-05-08T21:20:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88bf606b80 1 -- 172.21.15.33:0/1532802967 wait complete. 2024-05-08T21:20:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.861+0000 7f88bf606b80 20 client.16023 trim_cache size 0 max 16384 2024-05-08T21:20:35.869 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:20:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.862+0000 7f88b344c700 1 -- 172.21.15.33:0/972423452 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f88980a11a0 msgr2=0x7f88980a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.862+0000 7f88b344c700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f88980a11a0 0x7f88980a3660 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f88a000b2e0 tx=0x7f88a000b6d0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.862+0000 7f88b344c700 1 -- 172.21.15.33:0/972423452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac106bb0 msgr2=0x7f88ac19e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.862+0000 7f88b344c700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac106bb0 0x7f88ac19e270 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f88a4007ee0 tx=0x7f88a400d3b0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.862+0000 7f88b344c700 1 -- 172.21.15.33:0/972423452 shutdown_connections 2024-05-08T21:20:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.862+0000 7f88b344c700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88ac106bb0 0x7f88ac19e270 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.862+0000 7f88b344c700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f88980a11a0 0x7f88980a3660 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.862+0000 7f88b344c700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88ac101950 0x7f88ac196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.862+0000 7f88b344c700 1 --2- 172.21.15.33:0/972423452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88ac104280 0x7f88ac197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.862+0000 7f88b344c700 1 -- 172.21.15.33:0/972423452 >> 172.21.15.33:0/972423452 conn(0x7f88ac0fb2a0 msgr2=0x7f88ac1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.862+0000 7f88b344c700 1 -- 172.21.15.33:0/972423452 shutdown_connections 2024-05-08T21:20:35.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:35.862+0000 7f88b344c700 1 -- 172.21.15.33:0/972423452 wait complete. 2024-05-08T21:20:35.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:20:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.014+0000 7fbc233e5700 1 -- 172.21.15.33:0/2138501017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c101910 msgr2=0x7fbc1c1081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.014+0000 7fbc233e5700 1 --2- 172.21.15.33:0/2138501017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c101910 0x7fbc1c1081c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009fc0 tx=0x7fbc1400b330 comp rx=0 tx=0).stop 2024-05-08T21:20:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.014+0000 7fbc233e5700 1 -- 172.21.15.33:0/2138501017 shutdown_connections 2024-05-08T21:20:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.014+0000 7fbc233e5700 1 --2- 172.21.15.33:0/2138501017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c101910 0x7fbc1c1081c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.014+0000 7fbc233e5700 1 --2- 172.21.15.33:0/2138501017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1c0ffd80 0x7fbc1c1001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.014+0000 7fbc233e5700 1 --2- 172.21.15.33:0/2138501017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc1c100f70 0x7fbc1c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.014+0000 7fbc233e5700 1 -- 172.21.15.33:0/2138501017 >> 172.21.15.33:0/2138501017 conn(0x7fbc1c0fb310 msgr2=0x7fbc1c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.014+0000 7fbc233e5700 1 -- 172.21.15.33:0/2138501017 shutdown_connections 2024-05-08T21:20:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.014+0000 7fbc233e5700 1 -- 172.21.15.33:0/2138501017 wait complete. 2024-05-08T21:20:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc233e5700 1 Processor -- start 2024-05-08T21:20:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc233e5700 1 -- start start 2024-05-08T21:20:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc233e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c0ffd80 0x7fbc1c103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc233e5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1c100f70 0x7fbc1c102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc233e5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc1c101910 0x7fbc1c102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc233e5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c102f90 con 0x7fbc1c0ffd80 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc233e5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c103100 con 0x7fbc1c101910 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc233e5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c1a0e90 con 0x7fbc1c100f70 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc21181700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c0ffd80 0x7fbc1c103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc21181700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c0ffd80 0x7fbc1c103c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41808/0 (socket says 172.21.15.33:41808) 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc21181700 1 -- 172.21.15.33:0/2512050607 learned_addr learned my addr 172.21.15.33:0/2512050607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc20980700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1c100f70 0x7fbc1c102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.015+0000 7fbc21982700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc1c101910 0x7fbc1c102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc21181700 1 -- 172.21.15.33:0/2512050607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1c100f70 msgr2=0x7fbc1c102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc21181700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1c100f70 0x7fbc1c102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc21181700 1 -- 172.21.15.33:0/2512050607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc1c101910 msgr2=0x7fbc1c102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc21181700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc1c101910 0x7fbc1c102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc21181700 1 -- 172.21.15.33:0/2512050607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc18009520 con 0x7fbc1c0ffd80 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc21982700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc1c101910 0x7fbc1c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc20980700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1c100f70 0x7fbc1c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc21181700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c0ffd80 0x7fbc1c103c70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800ea50 tx=0x7fbc1800ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc127fc700 1 -- 172.21.15.33:0/2512050607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1800cc90 con 0x7fbc1c0ffd80 2024-05-08T21:20:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc233e5700 1 -- 172.21.15.33:0/2512050607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc14009c40 con 0x7fbc1c0ffd80 2024-05-08T21:20:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc127fc700 1 -- 172.21.15.33:0/2512050607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc1800cdf0 con 0x7fbc1c0ffd80 2024-05-08T21:20:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc127fc700 1 -- 172.21.15.33:0/2512050607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18018680 con 0x7fbc1c0ffd80 2024-05-08T21:20:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.016+0000 7fbc233e5700 1 -- 172.21.15.33:0/2512050607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c1a1470 con 0x7fbc1c0ffd80 2024-05-08T21:20:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.019+0000 7fbc127fc700 1 -- 172.21.15.33:0/2512050607 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbc18010430 con 0x7fbc1c0ffd80 2024-05-08T21:20:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.019+0000 7fbc127fc700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbc080a1120 0x7fbc080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.019+0000 7fbc127fc700 1 -- 172.21.15.33:0/2512050607 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbc18014070 con 0x7fbc1c0ffd80 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.019+0000 7fbc20980700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbc080a1120 0x7fbc080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc2f59fb80 1 Processor -- start 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc2f59fb80 1 -- start start 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc20980700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbc080a1120 0x7fbc080a35e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c005fd0 tx=0x7fbc0c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc2f59fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb6e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc2f59fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6eea0 0x560d8fb6f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc2f59fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6f860 0x560d8fb6fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc2f59fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8fac5150 con 0x560d8fb6e540 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc2f59fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8fac2b10 con 0x560d8fb6f860 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc2f59fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8fabe400 con 0x560d8fb6eea0 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc21181700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb6e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc21181700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb6e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41810/0 (socket says 172.21.15.33:41810) 2024-05-08T21:20:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc21181700 1 -- 172.21.15.33:0/975796895 learned_addr learned my addr 172.21.15.33:0/975796895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.020+0000 7fbc20980700 1 --2- 172.21.15.33:0/975796895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6eea0 0x560d8fb6f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.021+0000 7fbc21982700 1 --2- 172.21.15.33:0/975796895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6f860 0x560d8fb6fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.021+0000 7fbc21181700 1 -- 172.21.15.33:0/975796895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6eea0 msgr2=0x560d8fb6f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.021+0000 7fbc21181700 1 --2- 172.21.15.33:0/975796895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6eea0 0x560d8fb6f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.021+0000 7fbc21181700 1 -- 172.21.15.33:0/975796895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6f860 msgr2=0x560d8fb6fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.021+0000 7fbc21181700 1 --2- 172.21.15.33:0/975796895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6f860 0x560d8fb6fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.021+0000 7fbc21181700 1 -- 172.21.15.33:0/975796895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d8f93f3b0 con 0x560d8fb6e540 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.021+0000 7fbc20980700 1 --2- 172.21.15.33:0/975796895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6eea0 0x560d8fb6f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.021+0000 7fbc21181700 1 --2- 172.21.15.33:0/975796895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb6e960 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbc180a2250 tx=0x7fbc18099f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a00ce354de9491 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.021+0000 7fbc077fe700 1 -- 172.21.15.33:0/975796895 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1809a300 con 0x560d8fb6e540 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.021+0000 7fbc077fe700 1 -- 172.21.15.33:0/975796895 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc1809e130 con 0x560d8fb6e540 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.021+0000 7fbc077fe700 1 -- 172.21.15.33:0/975796895 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc180987c0 con 0x560d8fb6e540 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.022+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/975796895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 msgr2=0x560d8fb6e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.022+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/975796895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb6e960 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbc180a2250 tx=0x7fbc18099f10 comp rx=0 tx=0).stop 2024-05-08T21:20:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.022+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/975796895 shutdown_connections 2024-05-08T21:20:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.022+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/975796895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6f860 0x560d8fb6fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.022+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/975796895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb6e960 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.022+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/975796895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6eea0 0x560d8fb6f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.022+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/975796895 >> 172.21.15.33:0/975796895 conn(0x560d8facc210 msgr2=0x560d8fb68de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.022+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/975796895 shutdown_connections 2024-05-08T21:20:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.022+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/975796895 wait complete. 2024-05-08T21:20:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.022+0000 7fbc2f59fb80 1 -- start start 2024-05-08T21:20:36.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.023+0000 7fbc2f59fb80 1 Processor -- start 2024-05-08T21:20:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.023+0000 7fbc2f59fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb77dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.023+0000 7fbc2f59fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6eea0 0x560d8fb78300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.023+0000 7fbc2f59fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6f860 0x560d8fb7f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.023+0000 7fbc2f59fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8fa5a560 con 0x560d8fb6e540 2024-05-08T21:20:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.023+0000 7fbc2f59fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8fa5a200 con 0x560d8fb6eea0 2024-05-08T21:20:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.023+0000 7fbc2f59fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8f9416a0 con 0x560d8fb6f860 2024-05-08T21:20:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.023+0000 7fbc2f59fb80 10 client.0 _ll_register_callbacks cb 0x560d8fa63b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.023+0000 7fbc21181700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb77dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.023+0000 7fbc21181700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb77dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41814/0 (socket says 172.21.15.33:41814) 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.023+0000 7fbc21181700 1 -- 172.21.15.33:0/1952724438 learned_addr learned my addr 172.21.15.33:0/1952724438 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.023+0000 7fbc21982700 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6f860 0x560d8fb7f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc20980700 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6eea0 0x560d8fb78300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc21181700 1 -- 172.21.15.33:0/1952724438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6f860 msgr2=0x560d8fb7f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc21181700 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6f860 0x560d8fb7f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc21181700 1 -- 172.21.15.33:0/1952724438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6eea0 msgr2=0x560d8fb78300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc21181700 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6eea0 0x560d8fb78300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc21181700 1 -- 172.21.15.33:0/1952724438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0c011c60 con 0x560d8fb6e540 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc21982700 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6f860 0x560d8fb7f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc21181700 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb77dc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbc1809a290 tx=0x7fbc18099bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9c67510e1760b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc04ff9700 1 -- 172.21.15.33:0/1952724438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc180987a0 con 0x560d8fb6e540 2024-05-08T21:20:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc04ff9700 1 -- 172.21.15.33:0/1952724438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbc180a2780 con 0x560d8fb6e540 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1952724438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d8fa17270 con 0x560d8fb6e540 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc04ff9700 1 -- 172.21.15.33:0/1952724438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc1809a300 con 0x560d8fb6e540 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc2f59fb80 10 client.16035 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1952724438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7fbc1c1a1470 con 0x560d8fb6e540 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.024+0000 7fbc04ff9700 1 -- 172.21.15.33:0/1952724438 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560d8fa17270 con 0x560d8fb6e540 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.025+0000 7fbc04ff9700 1 -- 172.21.15.33:0/1952724438 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbc18063070 con 0x560d8fb6e540 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.025+0000 7fbc2f59fb80 10 client.16035 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.025+0000 7fbc2f59fb80 10 client.16035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.025+0000 7fbc2f59fb80 20 client.16035 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.025+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x560d8fb837e0 0x560d8fb85c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.025+0000 7fbc2f59fb80 4 client.16035 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.025+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1952724438 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d8fb3d810 con 0x560d8fb837e0 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.025+0000 7fbc20980700 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x560d8fb837e0 0x560d8fb85c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.025+0000 7fbc20980700 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x560d8fb837e0 0x560d8fb85c00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c833432c1a5cd2 server_cookie=97a6e370bca2f8c4 in_seq=0 out_seq=0 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.025+0000 7fbc04ff9700 10 client.16035 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc04ff9700 1 -- 172.21.15.33:0/1952724438 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d8fb3d810 con 0x560d8fb837e0 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc04ff9700 10 client.16035 handle_command_reply: tid=0 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc2f59fb80 1 client.16035 shutdown 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1952724438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 msgr2=0x560d8fb77dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb77dc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbc1809a290 tx=0x7fbc18099bb0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1952724438 shutdown_connections 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x560d8fb837e0 0x560d8fb85c00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6f860 0x560d8fb7f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb77dc0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc04ff9700 0 client.16035 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/1952724438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6eea0 0x560d8fb78300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1952724438 >> 172.21.15.33:0/1952724438 conn(0x560d8facc210 msgr2=0x560d8fb685a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:36.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.026+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1952724438 shutdown_connections 2024-05-08T21:20:36.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.027+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1952724438 wait complete. 2024-05-08T21:20:36.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.027+0000 7fbc2f59fb80 20 client.16035 trim_cache size 0 max 16384 2024-05-08T21:20:36.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.040+0000 7fbc2f59fb80 1 Processor -- start 2024-05-08T21:20:36.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.040+0000 7fbc2f59fb80 1 -- start start 2024-05-08T21:20:36.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc2f59fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb85bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc2f59fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6eea0 0x560d8fb7f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc2f59fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6f860 0x560d8fb7f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc2f59fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8fb8adf0 con 0x560d8fb6e540 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc2f59fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8fb75a50 con 0x560d8fb6f860 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc2f59fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8fb6e920 con 0x560d8fb6eea0 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc21181700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb85bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc21181700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb85bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41826/0 (socket says 172.21.15.33:41826) 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc21181700 1 -- 172.21.15.33:0/1860341932 learned_addr learned my addr 172.21.15.33:0/1860341932 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc21982700 1 --2- 172.21.15.33:0/1860341932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6f860 0x560d8fb7f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc20980700 1 --2- 172.21.15.33:0/1860341932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6eea0 0x560d8fb7f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc21181700 1 -- 172.21.15.33:0/1860341932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6eea0 msgr2=0x560d8fb7f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc21181700 1 --2- 172.21.15.33:0/1860341932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6eea0 0x560d8fb7f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc21181700 1 -- 172.21.15.33:0/1860341932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6f860 msgr2=0x560d8fb7f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc21181700 1 --2- 172.21.15.33:0/1860341932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6f860 0x560d8fb7f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc21181700 1 -- 172.21.15.33:0/1860341932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d8f8ebb60 con 0x560d8fb6e540 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc21181700 1 --2- 172.21.15.33:0/1860341932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb85bd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbc18099e60 tx=0x7fbc180985b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e39818089e68e13 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc07fff700 1 -- 172.21.15.33:0/1860341932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18099c60 con 0x560d8fb6e540 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc07fff700 1 -- 172.21.15.33:0/1860341932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc180a2780 con 0x560d8fb6e540 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.041+0000 7fbc07fff700 1 -- 172.21.15.33:0/1860341932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1809a300 con 0x560d8fb6e540 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.042+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1860341932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 msgr2=0x560d8fb85bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.042+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/1860341932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb85bd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbc18099e60 tx=0x7fbc180985b0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.042+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1860341932 shutdown_connections 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.042+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/1860341932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6f860 0x560d8fb7f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.042+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/1860341932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6e540 0x560d8fb85bd0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.042+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/1860341932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6eea0 0x560d8fb7f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.042+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1860341932 >> 172.21.15.33:0/1860341932 conn(0x560d8facc210 msgr2=0x560d8fb6fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:36.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.042+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1860341932 shutdown_connections 2024-05-08T21:20:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.042+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/1860341932 wait complete. 2024-05-08T21:20:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc2f59fb80 1 -- start start 2024-05-08T21:20:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc2f59fb80 1 Processor -- start 2024-05-08T21:20:36.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc2f59fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6e540 0x560d8fb7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc2f59fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6eea0 0x560d8fb7aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc2f59fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6f860 0x560d8fb7dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc2f59fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8fb7e200 con 0x560d8fb6eea0 2024-05-08T21:20:36.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc2f59fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8fb7e370 con 0x560d8fb6e540 2024-05-08T21:20:36.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc2f59fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d8fb7e530 con 0x560d8fb6f860 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc2f59fb80 10 client.0 _ll_register_callbacks cb 0x560d8fa63b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc20980700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6eea0 0x560d8fb7aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc20980700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6eea0 0x560d8fb7aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41830/0 (socket says 172.21.15.33:41830) 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc20980700 1 -- 172.21.15.33:0/881713734 learned_addr learned my addr 172.21.15.33:0/881713734 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc21181700 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6e540 0x560d8fb7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.043+0000 7fbc21982700 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6f860 0x560d8fb7dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc20980700 1 -- 172.21.15.33:0/881713734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6f860 msgr2=0x560d8fb7dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc20980700 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6f860 0x560d8fb7dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc20980700 1 -- 172.21.15.33:0/881713734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6e540 msgr2=0x560d8fb7a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc20980700 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6e540 0x560d8fb7a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc20980700 1 -- 172.21.15.33:0/881713734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c1a1470 con 0x560d8fb6eea0 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc20980700 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6eea0 0x560d8fb7aca0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009ac0 tx=0x7fbc0c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd05da9de29c36f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc04ff9700 1 -- 172.21.15.33:0/881713734 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c026070 con 0x560d8fb6eea0 2024-05-08T21:20:36.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc04ff9700 1 -- 172.21.15.33:0/881713734 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbc0c018070 con 0x560d8fb6eea0 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/881713734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc1809e130 con 0x560d8fb6eea0 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc04ff9700 1 -- 172.21.15.33:0/881713734 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc0c012ac0 con 0x560d8fb6eea0 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc2f59fb80 10 client.16041 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.044+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/881713734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560d8fa84560 con 0x560d8fb6eea0 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.045+0000 7fbc04ff9700 1 -- 172.21.15.33:0/881713734 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc1809e130 con 0x560d8fb6eea0 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.045+0000 7fbc04ff9700 1 -- 172.21.15.33:0/881713734 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbc0c02e020 con 0x560d8fb6eea0 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.045+0000 7fbc2f59fb80 10 client.16041 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.045+0000 7fbc2f59fb80 10 client.16041 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.045+0000 7fbc2f59fb80 20 client.16041 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.045+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x560d8fbda440 0x560d8fbdc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.045+0000 7fbc2f59fb80 4 client.16041 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.045+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/881713734 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560d8fab6ec0 con 0x560d8fbda440 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.045+0000 7fbc21181700 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x560d8fbda440 0x560d8fbdc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.045+0000 7fbc04ff9700 10 client.16041 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:20:36.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.045+0000 7fbc21181700 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x560d8fbda440 0x560d8fbdc860 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=363d45f80884dc9f server_cookie=68275bb83f515e44 in_seq=0 out_seq=0 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc04ff9700 1 -- 172.21.15.33:0/881713734 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560d8fab6ec0 con 0x560d8fbda440 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc04ff9700 10 client.16041 handle_command_reply: tid=0 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc2f59fb80 1 client.16041 shutdown 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/881713734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6eea0 msgr2=0x560d8fb7aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6eea0 0x560d8fb7aca0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009ac0 tx=0x7fbc0c0047c0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/881713734 shutdown_connections 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x560d8fbda440 0x560d8fbdc860 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d8fb6f860 0x560d8fb7dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d8fb6e540 0x560d8fb7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc2f59fb80 1 --2- 172.21.15.33:0/881713734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560d8fb6eea0 0x560d8fb7aca0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc04ff9700 0 client.16041 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/881713734 >> 172.21.15.33:0/881713734 conn(0x560d8facc210 msgr2=0x560d8fb7f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/881713734 shutdown_connections 2024-05-08T21:20:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc2f59fb80 1 -- 172.21.15.33:0/881713734 wait complete. 2024-05-08T21:20:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.046+0000 7fbc2f59fb80 20 client.16041 trim_cache size 0 max 16384 2024-05-08T21:20:36.054 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:20:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.047+0000 7fbc233e5700 1 -- 172.21.15.33:0/2512050607 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbc080a1120 msgr2=0x7fbc080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.047+0000 7fbc233e5700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbc080a1120 0x7fbc080a35e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c005fd0 tx=0x7fbc0c009310 comp rx=0 tx=0).stop 2024-05-08T21:20:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.047+0000 7fbc233e5700 1 -- 172.21.15.33:0/2512050607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c0ffd80 msgr2=0x7fbc1c103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.047+0000 7fbc233e5700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c0ffd80 0x7fbc1c103c70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800ea50 tx=0x7fbc1800ee10 comp rx=0 tx=0).stop 2024-05-08T21:20:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.047+0000 7fbc233e5700 1 -- 172.21.15.33:0/2512050607 shutdown_connections 2024-05-08T21:20:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.047+0000 7fbc233e5700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbc080a1120 0x7fbc080a35e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.047+0000 7fbc233e5700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc1c101910 0x7fbc1c102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.047+0000 7fbc233e5700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc1c0ffd80 0x7fbc1c103c70 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.047+0000 7fbc233e5700 1 --2- 172.21.15.33:0/2512050607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc1c100f70 0x7fbc1c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.047+0000 7fbc233e5700 1 -- 172.21.15.33:0/2512050607 >> 172.21.15.33:0/2512050607 conn(0x7fbc1c0fb310 msgr2=0x7fbc1c0694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.047+0000 7fbc233e5700 1 -- 172.21.15.33:0/2512050607 shutdown_connections 2024-05-08T21:20:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.047+0000 7fbc233e5700 1 -- 172.21.15.33:0/2512050607 wait complete. 2024-05-08T21:20:36.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:20:36.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.200+0000 7f3bd52d8700 1 -- 172.21.15.33:0/1630046551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01029b0 msgr2=0x7f3bd01092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.200+0000 7f3bd52d8700 1 --2- 172.21.15.33:0/1630046551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01029b0 0x7f3bd01092d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800b0a0 tx=0x7f3bc800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.200+0000 7f3bd52d8700 1 -- 172.21.15.33:0/1630046551 shutdown_connections 2024-05-08T21:20:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.200+0000 7f3bd52d8700 1 --2- 172.21.15.33:0/1630046551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01029b0 0x7f3bd01092d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.200+0000 7f3bd52d8700 1 --2- 172.21.15.33:0/1630046551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd0100ea0 0x7f3bd01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.200+0000 7f3bd52d8700 1 --2- 172.21.15.33:0/1630046551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd0101ff0 0x7f3bd0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.200+0000 7f3bd52d8700 1 -- 172.21.15.33:0/1630046551 >> 172.21.15.33:0/1630046551 conn(0x7f3bd00fc430 msgr2=0x7f3bd00fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.200+0000 7f3bd52d8700 1 -- 172.21.15.33:0/1630046551 shutdown_connections 2024-05-08T21:20:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.200+0000 7f3bd52d8700 1 -- 172.21.15.33:0/1630046551 wait complete. 2024-05-08T21:20:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.201+0000 7f3bd52d8700 1 Processor -- start 2024-05-08T21:20:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.201+0000 7f3bd52d8700 1 -- start start 2024-05-08T21:20:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.201+0000 7f3bd52d8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd0100ea0 0x7f3bd0074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.201+0000 7f3bd52d8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd0101ff0 0x7f3bd0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.201+0000 7f3bd52d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01029b0 0x7f3bd00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.201+0000 7f3bd52d8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd0074070 con 0x7f3bd01029b0 2024-05-08T21:20:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.201+0000 7f3bd52d8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00741e0 con 0x7f3bd0101ff0 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.201+0000 7f3bd52d8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd01a0e70 con 0x7f3bd0100ea0 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.201+0000 7f3bcf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01029b0 0x7f3bd00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.201+0000 7f3bcf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01029b0 0x7f3bd00738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41848/0 (socket says 172.21.15.33:41848) 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.201+0000 7f3bcf7fe700 1 -- 172.21.15.33:0/3972945722 learned_addr learned my addr 172.21.15.33:0/3972945722 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bceffd700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd0100ea0 0x7f3bd0074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd0101ff0 0x7f3bd0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bcf7fe700 1 -- 172.21.15.33:0/3972945722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd0100ea0 msgr2=0x7f3bd0074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bcf7fe700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd0100ea0 0x7f3bd0074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bcf7fe700 1 -- 172.21.15.33:0/3972945722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd0101ff0 msgr2=0x7f3bd0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bcf7fe700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd0101ff0 0x7f3bd0073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bcf7fe700 1 -- 172.21.15.33:0/3972945722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8009d00 con 0x7f3bd01029b0 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd0101ff0 0x7f3bd0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bcf7fe700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01029b0 0x7f3bd00738d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009570 tx=0x7f3bc80095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bc7fff700 1 -- 172.21.15.33:0/3972945722 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800e040 con 0x7f3bd01029b0 2024-05-08T21:20:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bd52d8700 1 -- 172.21.15.33:0/3972945722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd01a10a0 con 0x7f3bd01029b0 2024-05-08T21:20:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bc7fff700 1 -- 172.21.15.33:0/3972945722 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bc800be90 con 0x7f3bd01029b0 2024-05-08T21:20:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bc7fff700 1 -- 172.21.15.33:0/3972945722 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8005040 con 0x7f3bd01029b0 2024-05-08T21:20:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.202+0000 7f3bd52d8700 1 -- 172.21.15.33:0/3972945722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd01a1520 con 0x7f3bd01029b0 2024-05-08T21:20:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.205+0000 7f3bc7fff700 1 -- 172.21.15.33:0/3972945722 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3bc801d470 con 0x7f3bd01029b0 2024-05-08T21:20:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.205+0000 7f3bc7fff700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3bbc0a1120 0x7f3bbc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.205+0000 7f3bc7fff700 1 -- 172.21.15.33:0/3972945722 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3bc80ce1b0 con 0x7f3bd01029b0 2024-05-08T21:20:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.205+0000 7f3bceffd700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3bbc0a1120 0x7f3bbc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.206+0000 7f3bceffd700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3bbc0a1120 0x7f3bbc0a35e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0009c60 tx=0x7f3bc0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.206+0000 7f3be1492b80 1 Processor -- start 2024-05-08T21:20:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.206+0000 7f3be1492b80 1 -- start start 2024-05-08T21:20:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3be1492b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882540 0x563116882960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3be1492b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882ea0 0x563116883320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3be1492b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x563116883ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3be1492b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631167d9150 con 0x563116882ea0 2024-05-08T21:20:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3be1492b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631167d6b10 con 0x563116883860 2024-05-08T21:20:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3be1492b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631167d2400 con 0x563116882540 2024-05-08T21:20:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882ea0 0x563116883320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882ea0 0x563116883320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41852/0 (socket says 172.21.15.33:41852) 2024-05-08T21:20:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3bce7fc700 1 -- 172.21.15.33:0/2411356281 learned_addr learned my addr 172.21.15.33:0/2411356281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3bceffd700 1 --2- 172.21.15.33:0/2411356281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882540 0x563116882960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3bcf7fe700 1 --2- 172.21.15.33:0/2411356281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x563116883ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3bce7fc700 1 -- 172.21.15.33:0/2411356281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882540 msgr2=0x563116882960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/2411356281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882540 0x563116882960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3bce7fc700 1 -- 172.21.15.33:0/2411356281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 msgr2=0x563116883ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/2411356281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x563116883ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3bce7fc700 1 -- 172.21.15.33:0/2411356281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631166533b0 con 0x563116882ea0 2024-05-08T21:20:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3bceffd700 1 --2- 172.21.15.33:0/2411356281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882540 0x563116882960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.207+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/2411356281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882ea0 0x563116883320 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009de0 tx=0x7f3bb80116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a100df83fca3da7f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.208+0000 7f3bc57fa700 1 -- 172.21.15.33:0/2411356281 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8011ed0 con 0x563116882ea0 2024-05-08T21:20:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.208+0000 7f3bc57fa700 1 -- 172.21.15.33:0/2411356281 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bb80128e0 con 0x563116882ea0 2024-05-08T21:20:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.208+0000 7f3bc57fa700 1 -- 172.21.15.33:0/2411356281 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb800f8a0 con 0x563116882ea0 2024-05-08T21:20:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.208+0000 7f3be1492b80 1 -- 172.21.15.33:0/2411356281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882ea0 msgr2=0x563116883320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.208+0000 7f3be1492b80 1 --2- 172.21.15.33:0/2411356281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882ea0 0x563116883320 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009de0 tx=0x7f3bb80116c0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 -- 172.21.15.33:0/2411356281 shutdown_connections 2024-05-08T21:20:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 --2- 172.21.15.33:0/2411356281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x563116883ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 --2- 172.21.15.33:0/2411356281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882540 0x563116882960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 --2- 172.21.15.33:0/2411356281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882ea0 0x563116883320 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 -- 172.21.15.33:0/2411356281 >> 172.21.15.33:0/2411356281 conn(0x5631167e0210 msgr2=0x56311687cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 -- 172.21.15.33:0/2411356281 shutdown_connections 2024-05-08T21:20:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 -- 172.21.15.33:0/2411356281 wait complete. 2024-05-08T21:20:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 -- start start 2024-05-08T21:20:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 Processor -- start 2024-05-08T21:20:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x56311688bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882ea0 0x56311688c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x563116893320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56311676e560 con 0x563116882540 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56311676e200 con 0x563116883860 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631166556a0 con 0x563116882ea0 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3be1492b80 10 client.0 _ll_register_callbacks cb 0x563116777b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x56311688bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x56311688bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41862/0 (socket says 172.21.15.33:41862) 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.209+0000 7f3bceffd700 1 -- 172.21.15.33:0/687975462 learned_addr learned my addr 172.21.15.33:0/687975462 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882ea0 0x56311688c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bcf7fe700 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x563116893320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bceffd700 1 -- 172.21.15.33:0/687975462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882ea0 msgr2=0x56311688c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bceffd700 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882ea0 0x56311688c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bceffd700 1 -- 172.21.15.33:0/687975462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 msgr2=0x563116893320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bceffd700 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x563116893320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bceffd700 1 -- 172.21.15.33:0/687975462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb8009a10 con 0x563116882540 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bcf7fe700 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x563116893320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bceffd700 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x56311688bdf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0003980 tx=0x7f3bc00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b2c6de9773075a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bb67fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bb67fc700 1 -- 172.21.15.33:0/687975462 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc001eab0 con 0x563116882540 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3be1492b80 1 -- 172.21.15.33:0/687975462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56311672b270 con 0x563116882540 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bb67fc700 1 -- 172.21.15.33:0/687975462 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3bc001ec10 con 0x563116882540 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3bb67fc700 1 -- 172.21.15.33:0/687975462 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bc001edd0 con 0x563116882540 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3be1492b80 10 client.16053 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.210+0000 7f3be1492b80 1 -- 172.21.15.33:0/687975462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x5631165ffb60 con 0x563116882540 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.211+0000 7f3bb67fc700 1 -- 172.21.15.33:0/687975462 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56311672b270 con 0x563116882540 2024-05-08T21:20:36.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.211+0000 7f3bb67fc700 1 -- 172.21.15.33:0/687975462 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3bc001c430 con 0x563116882540 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.211+0000 7f3be1492b80 10 client.16053 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.211+0000 7f3be1492b80 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.211+0000 7f3be1492b80 20 client.16053 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.211+0000 7f3be1492b80 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x563116897c00 0x56311689a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.211+0000 7f3be1492b80 4 client.16053 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.211+0000 7f3be1492b80 1 -- 172.21.15.33:0/687975462 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563116851810 con 0x563116897c00 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.211+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x563116897c00 0x56311689a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.211+0000 7f3bb67fc700 10 client.16053 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.211+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x563116897c00 0x56311689a020 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8ae3f3accbce6ce server_cookie=65faad66c13343fa in_seq=0 out_seq=0 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3bb67fc700 1 -- 172.21.15.33:0/687975462 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563116851810 con 0x563116897c00 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3bb67fc700 10 client.16053 handle_command_reply: tid=0 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3be1492b80 1 client.16053 shutdown 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3be1492b80 1 -- 172.21.15.33:0/687975462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 msgr2=0x56311688bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3be1492b80 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x56311688bdf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0003980 tx=0x7f3bc00047c0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3be1492b80 1 -- 172.21.15.33:0/687975462 shutdown_connections 2024-05-08T21:20:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3be1492b80 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x563116897c00 0x56311689a020 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3be1492b80 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x563116893320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3be1492b80 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x56311688bdf0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3bb67fc700 0 client.16053 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3be1492b80 1 --2- 172.21.15.33:0/687975462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882ea0 0x56311688c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3be1492b80 1 -- 172.21.15.33:0/687975462 >> 172.21.15.33:0/687975462 conn(0x5631167e0210 msgr2=0x56311687c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3be1492b80 1 -- 172.21.15.33:0/687975462 shutdown_connections 2024-05-08T21:20:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.212+0000 7f3be1492b80 1 -- 172.21.15.33:0/687975462 wait complete. 2024-05-08T21:20:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.213+0000 7f3be1492b80 20 client.16053 trim_cache size 0 max 16384 2024-05-08T21:20:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3be1492b80 1 Processor -- start 2024-05-08T21:20:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3be1492b80 1 -- start start 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3be1492b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x5631168939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3be1492b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116882ea0 0x5631168932e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3be1492b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116883860 0x563116889df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3be1492b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56311687e5f0 con 0x563116882540 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3be1492b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56311689f260 con 0x563116882ea0 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3be1492b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563116889aa0 con 0x563116883860 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x5631168939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x5631168939d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41874/0 (socket says 172.21.15.33:41874) 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3bceffd700 1 -- 172.21.15.33:0/1774549373 learned_addr learned my addr 172.21.15.33:0/1774549373 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3bcf7fe700 1 --2- 172.21.15.33:0/1774549373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116883860 0x563116889df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.227+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/1774549373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116882ea0 0x5631168932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.228+0000 7f3bceffd700 1 -- 172.21.15.33:0/1774549373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116883860 msgr2=0x563116889df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.228+0000 7f3bceffd700 1 --2- 172.21.15.33:0/1774549373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116883860 0x563116889df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.228+0000 7f3bceffd700 1 -- 172.21.15.33:0/1774549373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116882ea0 msgr2=0x5631168932e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.228+0000 7f3bceffd700 1 --2- 172.21.15.33:0/1774549373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116882ea0 0x5631168932e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.228+0000 7f3bceffd700 1 -- 172.21.15.33:0/1774549373 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563116798560 con 0x563116882540 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.228+0000 7f3bceffd700 1 --2- 172.21.15.33:0/1774549373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x5631168939d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000a010 tx=0x7f3bc0004010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7400898e4e8d6ded server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.228+0000 7f3bc5ffb700 1 -- 172.21.15.33:0/1774549373 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc001ca00 con 0x563116882540 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.228+0000 7f3bc5ffb700 1 -- 172.21.15.33:0/1774549373 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bc001e700 con 0x563116882540 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.228+0000 7f3bc5ffb700 1 -- 172.21.15.33:0/1774549373 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc00376d0 con 0x563116882540 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 -- 172.21.15.33:0/1774549373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 msgr2=0x5631168939d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 --2- 172.21.15.33:0/1774549373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x5631168939d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000a010 tx=0x7f3bc0004010 comp rx=0 tx=0).stop 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 -- 172.21.15.33:0/1774549373 shutdown_connections 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 --2- 172.21.15.33:0/1774549373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116883860 0x563116889df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 --2- 172.21.15.33:0/1774549373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x5631168939d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 --2- 172.21.15.33:0/1774549373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116882ea0 0x5631168932e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 -- 172.21.15.33:0/1774549373 >> 172.21.15.33:0/1774549373 conn(0x5631167e0210 msgr2=0x563116883c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 -- 172.21.15.33:0/1774549373 shutdown_connections 2024-05-08T21:20:36.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 -- 172.21.15.33:0/1774549373 wait complete. 2024-05-08T21:20:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 -- start start 2024-05-08T21:20:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 Processor -- start 2024-05-08T21:20:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x56311688eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882ea0 0x56311688f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x5631168923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631168928f0 con 0x563116882540 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563116892a60 con 0x563116883860 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563116892c20 con 0x563116882ea0 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.229+0000 7f3be1492b80 10 client.0 _ll_register_callbacks cb 0x563116777b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x56311688eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x56311688eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41890/0 (socket says 172.21.15.33:41890) 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bceffd700 1 -- 172.21.15.33:0/3012016264 learned_addr learned my addr 172.21.15.33:0/3012016264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882ea0 0x56311688f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bcf7fe700 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x5631168923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bce7fc700 1 -- 172.21.15.33:0/3012016264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 msgr2=0x5631168923b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x5631168923b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bce7fc700 1 -- 172.21.15.33:0/3012016264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 msgr2=0x56311688eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x56311688eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bce7fc700 1 -- 172.21.15.33:0/3012016264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631165ffb60 con 0x563116882ea0 2024-05-08T21:20:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bceffd700 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x56311688eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bce7fc700 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882ea0 0x56311688f3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8000c00 tx=0x7f3bb80092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10280b0d293b2afb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bb67fc700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:20:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bb67fc700 1 -- 172.21.15.33:0/3012016264 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8012d60 con 0x563116882ea0 2024-05-08T21:20:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3be1492b80 1 -- 172.21.15.33:0/3012016264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3bb80128e0 con 0x563116882ea0 2024-05-08T21:20:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.230+0000 7f3bb67fc700 1 -- 172.21.15.33:0/3012016264 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3bb8012450 con 0x563116882ea0 2024-05-08T21:20:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.231+0000 7f3bb67fc700 1 -- 172.21.15.33:0/3012016264 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bb8012610 con 0x563116882ea0 2024-05-08T21:20:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.231+0000 7f3be1492b80 10 client.25948 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.231+0000 7f3be1492b80 1 -- 172.21.15.33:0/3012016264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x563116657cd0 con 0x563116882ea0 2024-05-08T21:20:36.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.231+0000 7f3bb67fc700 1 -- 172.21.15.33:0/3012016264 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3bb80128e0 con 0x563116882ea0 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.231+0000 7f3bb67fc700 1 -- 172.21.15.33:0/3012016264 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3bb80113e0 con 0x563116882ea0 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.231+0000 7f3be1492b80 10 client.25948 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.231+0000 7f3be1492b80 10 client.25948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.231+0000 7f3be1492b80 20 client.25948 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.231+0000 7f3be1492b80 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5631168eee60 0x5631168f1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.231+0000 7f3be1492b80 4 client.25948 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.231+0000 7f3be1492b80 1 -- 172.21.15.33:0/3012016264 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5631167caec0 con 0x5631168eee60 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.231+0000 7f3bceffd700 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5631168eee60 0x5631168f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3bb67fc700 10 client.25948 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3bceffd700 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5631168eee60 0x5631168f1250 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845d47a7fea401e1 server_cookie=811ce2259a14df52 in_seq=0 out_seq=0 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3bb67fc700 1 -- 172.21.15.33:0/3012016264 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5631167caec0 con 0x5631168eee60 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3bb67fc700 10 client.25948 handle_command_reply: tid=0 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3be1492b80 1 client.25948 shutdown 2024-05-08T21:20:36.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3be1492b80 1 -- 172.21.15.33:0/3012016264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882ea0 msgr2=0x56311688f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3be1492b80 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882ea0 0x56311688f3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8000c00 tx=0x7f3bb80092f0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3be1492b80 1 -- 172.21.15.33:0/3012016264 shutdown_connections 2024-05-08T21:20:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3be1492b80 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5631168eee60 0x5631168f1250 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3be1492b80 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563116883860 0x5631168923b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3bb67fc700 0 client.25948 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3be1492b80 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563116882540 0x56311688eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3be1492b80 1 --2- 172.21.15.33:0/3012016264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563116882ea0 0x56311688f3e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3be1492b80 1 -- 172.21.15.33:0/3012016264 >> 172.21.15.33:0/3012016264 conn(0x5631167e0210 msgr2=0x56311688b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3be1492b80 1 -- 172.21.15.33:0/3012016264 shutdown_connections 2024-05-08T21:20:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3be1492b80 1 -- 172.21.15.33:0/3012016264 wait complete. 2024-05-08T21:20:36.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.232+0000 7f3be1492b80 20 client.25948 trim_cache size 0 max 16384 2024-05-08T21:20:36.241 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:20:36.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.233+0000 7f3bd52d8700 1 -- 172.21.15.33:0/3972945722 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3bbc0a1120 msgr2=0x7f3bbc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.233+0000 7f3bd52d8700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3bbc0a1120 0x7f3bbc0a35e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0009c60 tx=0x7f3bc0009260 comp rx=0 tx=0).stop 2024-05-08T21:20:36.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.233+0000 7f3bd52d8700 1 -- 172.21.15.33:0/3972945722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01029b0 msgr2=0x7f3bd00738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:36.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.233+0000 7f3bd52d8700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01029b0 0x7f3bd00738d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009570 tx=0x7f3bc80095a0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.233+0000 7f3bd52d8700 1 -- 172.21.15.33:0/3972945722 shutdown_connections 2024-05-08T21:20:36.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.233+0000 7f3bd52d8700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01029b0 0x7f3bd00738d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.233+0000 7f3bd52d8700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bd0100ea0 0x7f3bd0074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.233+0000 7f3bd52d8700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3bbc0a1120 0x7f3bbc0a35e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.233+0000 7f3bd52d8700 1 --2- 172.21.15.33:0/3972945722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd0101ff0 0x7f3bd0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:36.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.233+0000 7f3bd52d8700 1 -- 172.21.15.33:0/3972945722 >> 172.21.15.33:0/3972945722 conn(0x7f3bd00fc430 msgr2=0x7f3bd0105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:36.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.233+0000 7f3bd52d8700 1 -- 172.21.15.33:0/3972945722 shutdown_connections 2024-05-08T21:20:36.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:36.233+0000 7f3bd52d8700 1 -- 172.21.15.33:0/3972945722 wait complete. 2024-05-08T21:20:37.249 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:20:37.250 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:20:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.399+0000 7fd42398a700 1 -- 172.21.15.33:0/2080825694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd41c0ffd60 msgr2=0x7fd41c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.399+0000 7fd42398a700 1 --2- 172.21.15.33:0/2080825694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd41c0ffd60 0x7fd41c100180 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd4180099c0 tx=0x7fd418009cd0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.399+0000 7fd42398a700 1 -- 172.21.15.33:0/2080825694 shutdown_connections 2024-05-08T21:20:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.399+0000 7fd42398a700 1 --2- 172.21.15.33:0/2080825694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd41c1018f0 0x7fd41c1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.399+0000 7fd42398a700 1 --2- 172.21.15.33:0/2080825694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd41c0ffd60 0x7fd41c100180 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.399+0000 7fd42398a700 1 --2- 172.21.15.33:0/2080825694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c100f50 0x7fd41c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.399+0000 7fd42398a700 1 -- 172.21.15.33:0/2080825694 >> 172.21.15.33:0/2080825694 conn(0x7fd41c0fb290 msgr2=0x7fd41c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.399+0000 7fd42398a700 1 -- 172.21.15.33:0/2080825694 shutdown_connections 2024-05-08T21:20:37.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.399+0000 7fd42398a700 1 -- 172.21.15.33:0/2080825694 wait complete. 2024-05-08T21:20:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd42398a700 1 Processor -- start 2024-05-08T21:20:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd42398a700 1 -- start start 2024-05-08T21:20:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd42398a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0ffd60 0x7fd41c195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd42398a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd41c100f50 0x7fd41c196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd42398a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd41c1018f0 0x7fd41c19d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd42398a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c19d740 con 0x7fd41c100f50 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd42398a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c19d880 con 0x7fd41c1018f0 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd42398a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c19da10 con 0x7fd41c0ffd60 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd420f25700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd41c100f50 0x7fd41c196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd420f25700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd41c100f50 0x7fd41c196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41914/0 (socket says 172.21.15.33:41914) 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd420f25700 1 -- 172.21.15.33:0/1363469740 learned_addr learned my addr 172.21.15.33:0/1363469740 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd421726700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0ffd60 0x7fd41c195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.400+0000 7fd421f27700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd41c1018f0 0x7fd41c19d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.401+0000 7fd420f25700 1 -- 172.21.15.33:0/1363469740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0ffd60 msgr2=0x7fd41c195c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.401+0000 7fd420f25700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0ffd60 0x7fd41c195c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.401+0000 7fd420f25700 1 -- 172.21.15.33:0/1363469740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd41c1018f0 msgr2=0x7fd41c19d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.401+0000 7fd420f25700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd41c1018f0 0x7fd41c19d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.401+0000 7fd420f25700 1 -- 172.21.15.33:0/1363469740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4180095f0 con 0x7fd41c100f50 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.401+0000 7fd421f27700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd41c1018f0 0x7fd41c19d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.401+0000 7fd420f25700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd41c100f50 0x7fd41c196150 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00eab0 tx=0x7fd40c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:37.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.401+0000 7fd4127fc700 1 -- 172.21.15.33:0/1363469740 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c013070 con 0x7fd41c100f50 2024-05-08T21:20:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.401+0000 7fd4127fc700 1 -- 172.21.15.33:0/1363469740 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd40c00be50 con 0x7fd41c100f50 2024-05-08T21:20:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.401+0000 7fd42398a700 1 -- 172.21.15.33:0/1363469740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c0681c0 con 0x7fd41c100f50 2024-05-08T21:20:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.401+0000 7fd4127fc700 1 -- 172.21.15.33:0/1363469740 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c017e40 con 0x7fd41c100f50 2024-05-08T21:20:37.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.401+0000 7fd42398a700 1 -- 172.21.15.33:0/1363469740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c0686d0 con 0x7fd41c100f50 2024-05-08T21:20:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.403+0000 7fd4127fc700 1 -- 172.21.15.33:0/1363469740 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd40c00b9c0 con 0x7fd41c100f50 2024-05-08T21:20:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.404+0000 7fd4127fc700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4080a1120 0x7fd4080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.404+0000 7fd4127fc700 1 -- 172.21.15.33:0/1363469740 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd40c0ce320 con 0x7fd41c100f50 2024-05-08T21:20:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.404+0000 7fd421726700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4080a1120 0x7fd4080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.404+0000 7fd421726700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4080a1120 0x7fd4080a35e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd418000c00 tx=0x7fd418003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.405+0000 7fd42fb44b80 1 Processor -- start 2024-05-08T21:20:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.405+0000 7fd42fb44b80 1 -- start start 2024-05-08T21:20:37.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.405+0000 7fd42fb44b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e2685fd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.405+0000 7fd42fb44b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e2685fe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.405+0000 7fd42fb44b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 0x55e2685fed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.405+0000 7fd42fb44b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2685541d0 con 0x55e2685fdf20 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.405+0000 7fd42fb44b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268551b90 con 0x55e2685fd5c0 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.405+0000 7fd42fb44b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26854d480 con 0x55e2685fe8e0 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.405+0000 7fd420f25700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e2685fe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.405+0000 7fd420f25700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e2685fe3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41930/0 (socket says 172.21.15.33:41930) 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.405+0000 7fd420f25700 1 -- 172.21.15.33:0/3115833381 learned_addr learned my addr 172.21.15.33:0/3115833381 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.405+0000 7fd421726700 1 --2- 172.21.15.33:0/3115833381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e2685fd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.406+0000 7fd421f27700 1 --2- 172.21.15.33:0/3115833381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 0x55e2685fed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.406+0000 7fd420f25700 1 -- 172.21.15.33:0/3115833381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 msgr2=0x55e2685fed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.406+0000 7fd420f25700 1 --2- 172.21.15.33:0/3115833381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 0x55e2685fed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.406+0000 7fd420f25700 1 -- 172.21.15.33:0/3115833381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 msgr2=0x55e2685fd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.406+0000 7fd420f25700 1 --2- 172.21.15.33:0/3115833381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e2685fd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.406+0000 7fd420f25700 1 -- 172.21.15.33:0/3115833381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2683ce430 con 0x55e2685fdf20 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.406+0000 7fd420f25700 1 --2- 172.21.15.33:0/3115833381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e2685fe3a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd40c09b380 tx=0x7fd40c09b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921e824ed9827943 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.406+0000 7fd4077fe700 1 -- 172.21.15.33:0/3115833381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c017580 con 0x55e2685fdf20 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.406+0000 7fd4077fe700 1 -- 172.21.15.33:0/3115833381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd40c0ce160 con 0x55e2685fdf20 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.406+0000 7fd4077fe700 1 -- 172.21.15.33:0/3115833381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c09f220 con 0x55e2685fdf20 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.407+0000 7fd42fb44b80 1 -- 172.21.15.33:0/3115833381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 msgr2=0x55e2685fe3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.407+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/3115833381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e2685fe3a0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd40c09b380 tx=0x7fd40c09b2c0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.407+0000 7fd42fb44b80 1 -- 172.21.15.33:0/3115833381 shutdown_connections 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.407+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/3115833381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 0x55e2685fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.407+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/3115833381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e2685fd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.407+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/3115833381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e2685fe3a0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.407+0000 7fd42fb44b80 1 -- 172.21.15.33:0/3115833381 >> 172.21.15.33:0/3115833381 conn(0x55e26855b290 msgr2=0x55e2685f7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.407+0000 7fd42fb44b80 1 -- 172.21.15.33:0/3115833381 shutdown_connections 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.407+0000 7fd42fb44b80 1 -- 172.21.15.33:0/3115833381 wait complete. 2024-05-08T21:20:37.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.407+0000 7fd42fb44b80 1 -- start start 2024-05-08T21:20:37.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.407+0000 7fd42fb44b80 1 Processor -- start 2024-05-08T21:20:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd42fb44b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fd5c0 0x55e268606e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd42fb44b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fdf20 0x55e268607360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd42fb44b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fe8e0 0x55e26860e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd42fb44b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2684e95e0 con 0x55e2685fe8e0 2024-05-08T21:20:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd42fb44b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2684e9280 con 0x55e2685fdf20 2024-05-08T21:20:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd42fb44b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2683d0720 con 0x55e2685fd5c0 2024-05-08T21:20:37.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd42fb44b80 10 client.0 _ll_register_callbacks cb 0x55e2684f2bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd421726700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fd5c0 0x55e268606e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd421726700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fd5c0 0x55e268606e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:36370/0 (socket says 172.21.15.33:36370) 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd421726700 1 -- 172.21.15.33:0/1576518080 learned_addr learned my addr 172.21.15.33:0/1576518080 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd421f27700 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fe8e0 0x55e26860e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd420f25700 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fdf20 0x55e268607360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd421f27700 1 -- 172.21.15.33:0/1576518080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fd5c0 msgr2=0x55e268606e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd421f27700 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fd5c0 0x55e268606e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd421f27700 1 -- 172.21.15.33:0/1576518080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fdf20 msgr2=0x55e268607360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd421f27700 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fdf20 0x55e268607360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.408+0000 7fd421f27700 1 -- 172.21.15.33:0/1576518080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c0686d0 con 0x55e2685fe8e0 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.409+0000 7fd421f27700 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fe8e0 0x55e26860e3a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55e2685f8dc0 tx=0x7fd414007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbca33f40111f6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.409+0000 7fd404ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.409+0000 7fd404ff9700 1 -- 172.21.15.33:0/1576518080 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd414014800 con 0x55e2685fe8e0 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.409+0000 7fd404ff9700 1 -- 172.21.15.33:0/1576518080 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd414014960 con 0x55e2685fe8e0 2024-05-08T21:20:37.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.409+0000 7fd42fb44b80 1 -- 172.21.15.33:0/1576518080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2684a62f0 con 0x55e2685fe8e0 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.409+0000 7fd404ff9700 1 -- 172.21.15.33:0/1576518080 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd414014b20 con 0x55e2685fe8e0 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.409+0000 7fd42fb44b80 10 client.16071 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.409+0000 7fd42fb44b80 1 -- 172.21.15.33:0/1576518080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e26837abe0 con 0x55e2685fe8e0 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.409+0000 7fd404ff9700 1 -- 172.21.15.33:0/1576518080 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e2684a62f0 con 0x55e2685fe8e0 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.410+0000 7fd404ff9700 1 -- 172.21.15.33:0/1576518080 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd414013350 con 0x55e2685fe8e0 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.410+0000 7fd42fb44b80 10 client.16071 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.410+0000 7fd42fb44b80 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.410+0000 7fd42fb44b80 20 client.16071 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.410+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e268612d30 0x55e268615150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.410+0000 7fd42fb44b80 4 client.16071 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.410+0000 7fd42fb44b80 1 -- 172.21.15.33:0/1576518080 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2685cc890 con 0x55e268612d30 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.411+0000 7fd421726700 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e268612d30 0x55e268615150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.412+0000 7fd421726700 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e268612d30 0x55e268615150 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7122b4d176ba3d2 server_cookie=64d300551e2b88f in_seq=0 out_seq=0 2024-05-08T21:20:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.412+0000 7fd404ff9700 10 client.16071 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.413+0000 7fd404ff9700 1 -- 172.21.15.33:0/1576518080 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e2685cc890 con 0x55e268612d30 2024-05-08T21:20:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.413+0000 7fd404ff9700 10 client.16071 handle_command_reply: tid=0 2024-05-08T21:20:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.413+0000 7fd42fb44b80 1 client.16071 shutdown 2024-05-08T21:20:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.413+0000 7fd42fb44b80 1 -- 172.21.15.33:0/1576518080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fe8e0 msgr2=0x55e26860e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.413+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fe8e0 0x55e26860e3a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x55e2685f8dc0 tx=0x7fd414007a80 comp rx=0 tx=0).stop 2024-05-08T21:20:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.413+0000 7fd42fb44b80 1 -- 172.21.15.33:0/1576518080 shutdown_connections 2024-05-08T21:20:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.413+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e268612d30 0x55e268615150 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.413+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fe8e0 0x55e26860e3a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.413+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fd5c0 0x55e268606e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.413+0000 7fd404ff9700 0 client.16071 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.413+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/1576518080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fdf20 0x55e268607360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.413+0000 7fd42fb44b80 1 -- 172.21.15.33:0/1576518080 >> 172.21.15.33:0/1576518080 conn(0x55e26855b290 msgr2=0x55e2685f7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.414+0000 7fd42fb44b80 1 -- 172.21.15.33:0/1576518080 shutdown_connections 2024-05-08T21:20:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.414+0000 7fd42fb44b80 1 -- 172.21.15.33:0/1576518080 wait complete. 2024-05-08T21:20:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.414+0000 7fd42fb44b80 20 client.16071 trim_cache size 0 max 16384 2024-05-08T21:20:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.438+0000 7fd42fb44b80 1 Processor -- start 2024-05-08T21:20:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd42fb44b80 1 -- start start 2024-05-08T21:20:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd42fb44b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e268606330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd42fb44b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e26860e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd42fb44b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 0x55e2686017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd42fb44b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268605db0 con 0x55e2685fdf20 2024-05-08T21:20:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd42fb44b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e268604a40 con 0x55e2685fd5c0 2024-05-08T21:20:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd42fb44b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2685f00c0 con 0x55e2685fe8e0 2024-05-08T21:20:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd420f25700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e26860e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd421726700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e268606330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd420f25700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e26860e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41948/0 (socket says 172.21.15.33:41948) 2024-05-08T21:20:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd420f25700 1 -- 172.21.15.33:0/4212989215 learned_addr learned my addr 172.21.15.33:0/4212989215 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:37.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd421f27700 1 --2- 172.21.15.33:0/4212989215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 0x55e2686017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd420f25700 1 -- 172.21.15.33:0/4212989215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 msgr2=0x55e2686017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd420f25700 1 --2- 172.21.15.33:0/4212989215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 0x55e2686017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd420f25700 1 -- 172.21.15.33:0/4212989215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 msgr2=0x55e268606330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd420f25700 1 --2- 172.21.15.33:0/4212989215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e268606330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd420f25700 1 -- 172.21.15.33:0/4212989215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2685135e0 con 0x55e2685fdf20 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.439+0000 7fd421726700 1 --2- 172.21.15.33:0/4212989215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e268606330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.440+0000 7fd420f25700 1 --2- 172.21.15.33:0/4212989215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e26860e2c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00ea80 tx=0x7fd40c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151ed64f6294f8cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.440+0000 7fd407fff700 1 -- 172.21.15.33:0/4212989215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c017580 con 0x55e2685fdf20 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.440+0000 7fd407fff700 1 -- 172.21.15.33:0/4212989215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd40c0860d0 con 0x55e2685fdf20 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.440+0000 7fd407fff700 1 -- 172.21.15.33:0/4212989215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c09a8e0 con 0x55e2685fdf20 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.441+0000 7fd42fb44b80 1 -- 172.21.15.33:0/4212989215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 msgr2=0x55e26860e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.441+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/4212989215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e26860e2c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00ea80 tx=0x7fd40c01f000 comp rx=0 tx=0).stop 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.441+0000 7fd42fb44b80 1 -- 172.21.15.33:0/4212989215 shutdown_connections 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.441+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/4212989215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 0x55e2686017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.441+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/4212989215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e268606330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.441+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/4212989215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e26860e2c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.441+0000 7fd42fb44b80 1 -- 172.21.15.33:0/4212989215 >> 172.21.15.33:0/4212989215 conn(0x55e26855b290 msgr2=0x55e2685fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.441+0000 7fd42fb44b80 1 -- 172.21.15.33:0/4212989215 shutdown_connections 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.441+0000 7fd42fb44b80 1 -- 172.21.15.33:0/4212989215 wait complete. 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.441+0000 7fd42fb44b80 1 -- start start 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.441+0000 7fd42fb44b80 1 Processor -- start 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd42fb44b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e26860a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd42fb44b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e26860a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd42fb44b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 0x55e26860d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd42fb44b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26860dbe0 con 0x55e2685fdf20 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd42fb44b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26860dd50 con 0x55e2685fd5c0 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd42fb44b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26860df10 con 0x55e2685fe8e0 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd42fb44b80 10 client.0 _ll_register_callbacks cb 0x55e2684f2bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd421726700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e26860a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd421726700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e26860a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:45086/0 (socket says 172.21.15.33:45086) 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd421726700 1 -- 172.21.15.33:0/390528120 learned_addr learned my addr 172.21.15.33:0/390528120 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:37.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd421f27700 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 0x55e26860d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd420f25700 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e26860a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd421726700 1 -- 172.21.15.33:0/390528120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 msgr2=0x55e26860d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd421726700 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 0x55e26860d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd421726700 1 -- 172.21.15.33:0/390528120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 msgr2=0x55e26860a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd421726700 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e26860a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd421726700 1 -- 172.21.15.33:0/390528120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c108af0 con 0x55e2685fd5c0 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd421726700 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e26860a190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd418009990 tx=0x7fd418009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=368a42f7843cbd49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.442+0000 7fd404ff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd404ff9700 1 -- 172.21.15.33:0/390528120 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd418031930 con 0x55e2685fd5c0 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd42fb44b80 1 -- 172.21.15.33:0/390528120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd40c0ce160 con 0x55e2685fd5c0 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd404ff9700 1 -- 172.21.15.33:0/390528120 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd418005500 con 0x55e2685fd5c0 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd404ff9700 1 -- 172.21.15.33:0/390528120 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4180056c0 con 0x55e2685fd5c0 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd42fb44b80 10 client.25949 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:37.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd42fb44b80 1 -- 172.21.15.33:0/390528120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55e2683ce430 con 0x55e2685fd5c0 2024-05-08T21:20:37.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd404ff9700 1 -- 172.21.15.33:0/390528120 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd40c0ce160 con 0x55e2685fd5c0 2024-05-08T21:20:37.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd404ff9700 1 -- 172.21.15.33:0/390528120 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd41803a770 con 0x55e2685fd5c0 2024-05-08T21:20:37.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd42fb44b80 10 client.25949 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:37.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd42fb44b80 10 client.25949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:37.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd42fb44b80 20 client.25949 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:37.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e26866a2c0 0x55e26866c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd42fb44b80 4 client.25949 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:20:37.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.443+0000 7fd42fb44b80 1 -- 172.21.15.33:0/390528120 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e268545f40 con 0x55e26866a2c0 2024-05-08T21:20:37.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.444+0000 7fd420f25700 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e26866a2c0 0x55e26866c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.444+0000 7fd404ff9700 10 client.25949 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:37.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.444+0000 7fd420f25700 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e26866a2c0 0x55e26866c6e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d176eeac68964f5 server_cookie=dc62d583c7d7b6b5 in_seq=0 out_seq=0 2024-05-08T21:20:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd404ff9700 1 -- 172.21.15.33:0/390528120 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e268545f40 con 0x55e26866a2c0 2024-05-08T21:20:37.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd404ff9700 10 client.25949 handle_command_reply: tid=0 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd42fb44b80 1 client.25949 shutdown 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd42fb44b80 1 -- 172.21.15.33:0/390528120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 msgr2=0x55e26860a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e26860a190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd418009990 tx=0x7fd418009cd0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd42fb44b80 1 -- 172.21.15.33:0/390528120 shutdown_connections 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e26866a2c0 0x55e26866c6e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2685fe8e0 0x55e26860d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2685fd5c0 0x55e26860a190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd42fb44b80 1 --2- 172.21.15.33:0/390528120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2685fdf20 0x55e26860a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd404ff9700 0 client.25949 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd42fb44b80 1 -- 172.21.15.33:0/390528120 >> 172.21.15.33:0/390528120 conn(0x55e26855b290 msgr2=0x55e268606230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd42fb44b80 1 -- 172.21.15.33:0/390528120 shutdown_connections 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.510+0000 7fd42fb44b80 1 -- 172.21.15.33:0/390528120 wait complete. 2024-05-08T21:20:37.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.511+0000 7fd42fb44b80 20 client.25949 trim_cache size 0 max 16384 2024-05-08T21:20:37.514 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:20:37.514 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:20:37.514 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "377c3c63-ca93-4a03-be06-2107ebce40f4", 2024-05-08T21:20:37.514 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:20:37.514 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:20:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.511+0000 7fd42398a700 1 -- 172.21.15.33:0/1363469740 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4080a1120 msgr2=0x7fd4080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.511+0000 7fd42398a700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4080a1120 0x7fd4080a35e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd418000c00 tx=0x7fd418003680 comp rx=0 tx=0).stop 2024-05-08T21:20:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.511+0000 7fd42398a700 1 -- 172.21.15.33:0/1363469740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd41c100f50 msgr2=0x7fd41c196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.511+0000 7fd42398a700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd41c100f50 0x7fd41c196150 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00eab0 tx=0x7fd40c00edc0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.511+0000 7fd42398a700 1 -- 172.21.15.33:0/1363469740 shutdown_connections 2024-05-08T21:20:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.511+0000 7fd42398a700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4080a1120 0x7fd4080a35e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.511+0000 7fd42398a700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd41c1018f0 0x7fd41c19d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.511+0000 7fd42398a700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd41c0ffd60 0x7fd41c195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.511+0000 7fd42398a700 1 --2- 172.21.15.33:0/1363469740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd41c100f50 0x7fd41c196150 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.512+0000 7fd42398a700 1 -- 172.21.15.33:0/1363469740 >> 172.21.15.33:0/1363469740 conn(0x7fd41c0fb290 msgr2=0x7fd41c104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.512+0000 7fd42398a700 1 -- 172.21.15.33:0/1363469740 shutdown_connections 2024-05-08T21:20:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.512+0000 7fd42398a700 1 -- 172.21.15.33:0/1363469740 wait complete. 2024-05-08T21:20:37.530 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:20:37.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:37 smithi033 ceph-mon[42528]: pgmap v339: 129 pgs: 129 active+clean; 11 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 905 KiB/s wr, 27 op/s 2024-05-08T21:20:37.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:37 smithi082 ceph-mon[47512]: pgmap v339: 129 pgs: 129 active+clean; 11 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 905 KiB/s wr, 27 op/s 2024-05-08T21:20:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.711+0000 7f2327e9d700 1 -- 172.21.15.33:0/618211581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320100e70 msgr2=0x7f2320101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.711+0000 7f2327e9d700 1 --2- 172.21.15.33:0/618211581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320100e70 0x7f2320101290 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f231c009990 tx=0x7f231c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.711+0000 7f2327e9d700 1 -- 172.21.15.33:0/618211581 shutdown_connections 2024-05-08T21:20:37.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.711+0000 7f2327e9d700 1 --2- 172.21.15.33:0/618211581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2320102980 0x7f23201092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.711+0000 7f2327e9d700 1 --2- 172.21.15.33:0/618211581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320100e70 0x7f2320101290 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.711+0000 7f2327e9d700 1 --2- 172.21.15.33:0/618211581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320101fc0 0x7f2320102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.711+0000 7f2327e9d700 1 -- 172.21.15.33:0/618211581 >> 172.21.15.33:0/618211581 conn(0x7f23200fc400 msgr2=0x7f23200fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.711+0000 7f2327e9d700 1 -- 172.21.15.33:0/618211581 shutdown_connections 2024-05-08T21:20:37.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.711+0000 7f2327e9d700 1 -- 172.21.15.33:0/618211581 wait complete. 2024-05-08T21:20:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.712+0000 7f2327e9d700 1 Processor -- start 2024-05-08T21:20:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.712+0000 7f2327e9d700 1 -- start start 2024-05-08T21:20:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.712+0000 7f2327e9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320100e70 0x7f2320196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.712+0000 7f2327e9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2320101fc0 0x7f23201971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.712+0000 7f2327e9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320102980 0x7f232019e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.712+0000 7f2327e9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232019e790 con 0x7f2320100e70 2024-05-08T21:20:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.712+0000 7f2327e9d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232019e900 con 0x7f2320101fc0 2024-05-08T21:20:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.712+0000 7f2327e9d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232019eac0 con 0x7f2320102980 2024-05-08T21:20:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f2325c39700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320100e70 0x7f2320196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f2325438700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2320101fc0 0x7f23201971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f2325438700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2320101fc0 0x7f23201971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:45104/0 (socket says 172.21.15.33:45104) 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f2325c39700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320100e70 0x7f2320196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41976/0 (socket says 172.21.15.33:41976) 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f232643a700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320102980 0x7f232019e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f2325438700 1 -- 172.21.15.33:0/2197598816 learned_addr learned my addr 172.21.15.33:0/2197598816 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f2325438700 1 -- 172.21.15.33:0/2197598816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320102980 msgr2=0x7f232019e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f2325438700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320102980 0x7f232019e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f2325438700 1 -- 172.21.15.33:0/2197598816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320100e70 msgr2=0x7f2320196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f2325438700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320100e70 0x7f2320196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f2325438700 1 -- 172.21.15.33:0/2197598816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c0095f0 con 0x7f2320101fc0 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f2325438700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2320101fc0 0x7f23201971e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f231000e9a0 tx=0x7f231000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.713+0000 7f2316ffd700 1 -- 172.21.15.33:0/2197598816 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231000ca10 con 0x7f2320101fc0 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.714+0000 7f2327e9d700 1 -- 172.21.15.33:0/2197598816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232019eda0 con 0x7f2320101fc0 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.714+0000 7f2316ffd700 1 -- 172.21.15.33:0/2197598816 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2310010430 con 0x7f2320101fc0 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.714+0000 7f2327e9d700 1 -- 172.21.15.33:0/2197598816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f232019f270 con 0x7f2320101fc0 2024-05-08T21:20:37.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.714+0000 7f2316ffd700 1 -- 172.21.15.33:0/2197598816 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310018680 con 0x7f2320101fc0 2024-05-08T21:20:37.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.716+0000 7f2316ffd700 1 -- 172.21.15.33:0/2197598816 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2310010810 con 0x7f2320101fc0 2024-05-08T21:20:37.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.717+0000 7f2316ffd700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f230c0a1120 0x7f230c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.717+0000 7f2316ffd700 1 -- 172.21.15.33:0/2197598816 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f2310014070 con 0x7f2320101fc0 2024-05-08T21:20:37.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.717+0000 7f2325c39700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f230c0a1120 0x7f230c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.718+0000 7f2334057b80 1 Processor -- start 2024-05-08T21:20:37.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.718+0000 7f2325c39700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f230c0a1120 0x7f230c0a35e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f231c005910 tx=0x7f231c00b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:37.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.719+0000 7f2334057b80 1 -- start start 2024-05-08T21:20:37.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.719+0000 7f2334057b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 0x55b091e14960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.719+0000 7f2334057b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14ea0 0x55b091e15320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.719+0000 7f2334057b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e15ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.719+0000 7f2334057b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091d6b150 con 0x55b091e15860 2024-05-08T21:20:37.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.719+0000 7f2334057b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091d68b10 con 0x55b091e14ea0 2024-05-08T21:20:37.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.719+0000 7f2334057b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091d64400 con 0x55b091e14540 2024-05-08T21:20:37.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.719+0000 7f2325c39700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 0x55b091e14960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.719+0000 7f2325c39700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 0x55b091e14960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:36406/0 (socket says 172.21.15.33:36406) 2024-05-08T21:20:37.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.719+0000 7f2325c39700 1 -- 172.21.15.33:0/2291608397 learned_addr learned my addr 172.21.15.33:0/2291608397 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:37.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.719+0000 7f232643a700 1 --2- 172.21.15.33:0/2291608397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e15ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.720+0000 7f2325c39700 1 -- 172.21.15.33:0/2291608397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14ea0 msgr2=0x55b091e15320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:20:37.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.720+0000 7f2325c39700 1 --2- 172.21.15.33:0/2291608397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14ea0 0x55b091e15320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.720+0000 7f2325c39700 1 -- 172.21.15.33:0/2291608397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 msgr2=0x55b091e15ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.720+0000 7f2325c39700 1 --2- 172.21.15.33:0/2291608397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e15ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.720+0000 7f2325c39700 1 -- 172.21.15.33:0/2291608397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b091be53b0 con 0x55b091e14540 2024-05-08T21:20:37.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.720+0000 7f232643a700 1 --2- 172.21.15.33:0/2291608397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e15ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:37.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.720+0000 7f2325c39700 1 --2- 172.21.15.33:0/2291608397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 0x55b091e14960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f231c015000 tx=0x7f231c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db20fa4537a6539 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:37.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.720+0000 7f230bfff700 1 -- 172.21.15.33:0/2291608397 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c02b070 con 0x55b091e14540 2024-05-08T21:20:37.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.720+0000 7f230bfff700 1 -- 172.21.15.33:0/2291608397 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f231c01e040 con 0x55b091e14540 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.720+0000 7f230bfff700 1 -- 172.21.15.33:0/2291608397 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c00f920 con 0x55b091e14540 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.721+0000 7f2334057b80 1 -- 172.21.15.33:0/2291608397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 msgr2=0x55b091e14960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.721+0000 7f2334057b80 1 --2- 172.21.15.33:0/2291608397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 0x55b091e14960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f231c015000 tx=0x7f231c020000 comp rx=0 tx=0).stop 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.721+0000 7f2334057b80 1 -- 172.21.15.33:0/2291608397 shutdown_connections 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.721+0000 7f2334057b80 1 --2- 172.21.15.33:0/2291608397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e15ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.721+0000 7f2334057b80 1 --2- 172.21.15.33:0/2291608397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 0x55b091e14960 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.721+0000 7f2334057b80 1 --2- 172.21.15.33:0/2291608397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14ea0 0x55b091e15320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.721+0000 7f2334057b80 1 -- 172.21.15.33:0/2291608397 >> 172.21.15.33:0/2291608397 conn(0x55b091d72210 msgr2=0x55b091e0ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.721+0000 7f2334057b80 1 -- 172.21.15.33:0/2291608397 shutdown_connections 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.721+0000 7f2334057b80 1 -- 172.21.15.33:0/2291608397 wait complete. 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.722+0000 7f2334057b80 1 -- start start 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.722+0000 7f2334057b80 1 Processor -- start 2024-05-08T21:20:37.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.722+0000 7f2334057b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14540 0x55b091e1ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.722+0000 7f2334057b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e14ea0 0x55b091e204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f2334057b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e15860 0x55b091e1caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f2334057b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091d00560 con 0x55b091e14ea0 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f2334057b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091d00200 con 0x55b091e14540 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f2334057b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091be76a0 con 0x55b091e15860 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f2334057b80 10 client.0 _ll_register_callbacks cb 0x55b091d09b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f2325c39700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14540 0x55b091e1ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f2325c39700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14540 0x55b091e1ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:45120/0 (socket says 172.21.15.33:45120) 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f2325c39700 1 -- 172.21.15.33:0/3921195165 learned_addr learned my addr 172.21.15.33:0/3921195165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f232643a700 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e15860 0x55b091e1caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f2325438700 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e14ea0 0x55b091e204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f232643a700 1 -- 172.21.15.33:0/3921195165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14540 msgr2=0x55b091e1ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f232643a700 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14540 0x55b091e1ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f232643a700 1 -- 172.21.15.33:0/3921195165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e14ea0 msgr2=0x55b091e204a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f232643a700 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e14ea0 0x55b091e204a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f232643a700 1 -- 172.21.15.33:0/3921195165 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2318011230 con 0x55b091e15860 2024-05-08T21:20:37.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f232643a700 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e15860 0x55b091e1caf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2318007c60 tx=0x7f2318007f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5782b4e3ae5a5645 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:37.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.723+0000 7f23097fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:20:37.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f23097fa700 1 -- 172.21.15.33:0/3921195165 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231800e040 con 0x55b091e15860 2024-05-08T21:20:37.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f23097fa700 1 -- 172.21.15.33:0/3921195165 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f2318004db0 con 0x55b091e15860 2024-05-08T21:20:37.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f2334057b80 1 -- 172.21.15.33:0/3921195165 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b091cbd270 con 0x55b091e15860 2024-05-08T21:20:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f23097fa700 1 -- 172.21.15.33:0/3921195165 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2318011b50 con 0x55b091e15860 2024-05-08T21:20:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f2334057b80 10 client.25975 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f2334057b80 1 -- 172.21.15.33:0/3921195165 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f2320109bd0 con 0x55b091e15860 2024-05-08T21:20:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f23097fa700 1 -- 172.21.15.33:0/3921195165 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b091cbd270 con 0x55b091e15860 2024-05-08T21:20:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f23097fa700 1 -- 172.21.15.33:0/3921195165 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2318004270 con 0x55b091e15860 2024-05-08T21:20:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f2334057b80 10 client.25975 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f2334057b80 10 client.25975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f2334057b80 20 client.25975 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f2334057b80 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b091e32590 0x55b091e349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f2334057b80 4 client.25975 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.724+0000 7f2334057b80 1 -- 172.21.15.33:0/3921195165 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b091de3810 con 0x55b091e32590 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.725+0000 7f2325c39700 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b091e32590 0x55b091e349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.725+0000 7f23097fa700 10 client.25975 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.725+0000 7f2325c39700 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b091e32590 0x55b091e349b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16518e2aaf5371fa server_cookie=22b46c2fdb843b20 in_seq=0 out_seq=0 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f23097fa700 1 -- 172.21.15.33:0/3921195165 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b091de3810 con 0x55b091e32590 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f23097fa700 10 client.25975 handle_command_reply: tid=0 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f2334057b80 1 client.25975 shutdown 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f2334057b80 1 -- 172.21.15.33:0/3921195165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e15860 msgr2=0x55b091e1caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f2334057b80 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e15860 0x55b091e1caf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2318007c60 tx=0x7f2318007f70 comp rx=0 tx=0).stop 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f2334057b80 1 -- 172.21.15.33:0/3921195165 shutdown_connections 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f2334057b80 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b091e32590 0x55b091e349b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f2334057b80 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e15860 0x55b091e1caf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f2334057b80 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14540 0x55b091e1ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f2334057b80 1 --2- 172.21.15.33:0/3921195165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e14ea0 0x55b091e204a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f2334057b80 1 -- 172.21.15.33:0/3921195165 >> 172.21.15.33:0/3921195165 conn(0x55b091d72210 msgr2=0x55b091e0e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f23097fa700 0 client.25975 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f2334057b80 1 -- 172.21.15.33:0/3921195165 shutdown_connections 2024-05-08T21:20:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.726+0000 7f2334057b80 1 -- 172.21.15.33:0/3921195165 wait complete. 2024-05-08T21:20:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.727+0000 7f2334057b80 20 client.25975 trim_cache size 0 max 16384 2024-05-08T21:20:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f2334057b80 1 Processor -- start 2024-05-08T21:20:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f2334057b80 1 -- start start 2024-05-08T21:20:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f2334057b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 0x55b091e21720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f2334057b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14ea0 0x55b091e1be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f2334057b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e42850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f2334057b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091e09870 con 0x55b091e15860 2024-05-08T21:20:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f2334057b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091e10860 con 0x55b091e14ea0 2024-05-08T21:20:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f2334057b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091e39bd0 con 0x55b091e14540 2024-05-08T21:20:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f2325c39700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 0x55b091e21720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f2325438700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14ea0 0x55b091e1be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f2325c39700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 0x55b091e21720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:36426/0 (socket says 172.21.15.33:36426) 2024-05-08T21:20:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f2325c39700 1 -- 172.21.15.33:0/3662214776 learned_addr learned my addr 172.21.15.33:0/3662214776 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.751+0000 7f232643a700 1 --2- 172.21.15.33:0/3662214776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e42850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.752+0000 7f2325c39700 1 -- 172.21.15.33:0/3662214776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14ea0 msgr2=0x55b091e1be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.752+0000 7f2325c39700 1 --2- 172.21.15.33:0/3662214776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14ea0 0x55b091e1be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.752+0000 7f2325c39700 1 -- 172.21.15.33:0/3662214776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 msgr2=0x55b091e42850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.752+0000 7f2325c39700 1 --2- 172.21.15.33:0/3662214776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e42850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.752+0000 7f2325c39700 1 -- 172.21.15.33:0/3662214776 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b091b91b60 con 0x55b091e14540 2024-05-08T21:20:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.752+0000 7f2325c39700 1 --2- 172.21.15.33:0/3662214776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 0x55b091e21720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f231c003f60 tx=0x7f231c00b2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feb02606426428c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.752+0000 7f2314ff9700 1 -- 172.21.15.33:0/3662214776 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c02b040 con 0x55b091e14540 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.752+0000 7f2314ff9700 1 -- 172.21.15.33:0/3662214776 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f231c03b0f0 con 0x55b091e14540 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.752+0000 7f2314ff9700 1 -- 172.21.15.33:0/3662214776 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c00f730 con 0x55b091e14540 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.753+0000 7f2334057b80 1 -- 172.21.15.33:0/3662214776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 msgr2=0x55b091e21720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.753+0000 7f2334057b80 1 --2- 172.21.15.33:0/3662214776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 0x55b091e21720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f231c003f60 tx=0x7f231c00b2e0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.753+0000 7f2334057b80 1 -- 172.21.15.33:0/3662214776 shutdown_connections 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.753+0000 7f2334057b80 1 --2- 172.21.15.33:0/3662214776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e42850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.753+0000 7f2334057b80 1 --2- 172.21.15.33:0/3662214776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14540 0x55b091e21720 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.753+0000 7f2334057b80 1 --2- 172.21.15.33:0/3662214776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14ea0 0x55b091e1be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.753+0000 7f2334057b80 1 -- 172.21.15.33:0/3662214776 >> 172.21.15.33:0/3662214776 conn(0x55b091d72210 msgr2=0x55b091e0f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.753+0000 7f2334057b80 1 -- 172.21.15.33:0/3662214776 shutdown_connections 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.753+0000 7f2334057b80 1 -- 172.21.15.33:0/3662214776 wait complete. 2024-05-08T21:20:37.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.753+0000 7f2334057b80 1 -- start start 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.754+0000 7f2334057b80 1 Processor -- start 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.754+0000 7f2334057b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14540 0x55b091e3fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.754+0000 7f2334057b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14ea0 0x55b091e40330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.754+0000 7f2334057b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2334057b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091e41830 con 0x55b091e15860 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2334057b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091e419a0 con 0x55b091e14540 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2334057b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b091e41b60 con 0x55b091e14ea0 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2334057b80 10 client.0 _ll_register_callbacks cb 0x55b091d09b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f232643a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2325438700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14ea0 0x55b091e40330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2325438700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14ea0 0x55b091e40330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:36436/0 (socket says 172.21.15.33:36436) 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2325438700 1 -- 172.21.15.33:0/1025754557 learned_addr learned my addr 172.21.15.33:0/1025754557 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f232643a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e412f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42028/0 (socket says 172.21.15.33:42028) 2024-05-08T21:20:37.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2325438700 1 -- 172.21.15.33:0/1025754557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14540 msgr2=0x55b091e3fdf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2325438700 1 --2- 172.21.15.33:0/1025754557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14540 0x55b091e3fdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2325438700 1 -- 172.21.15.33:0/1025754557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 msgr2=0x55b091e412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2325438700 1 --2- 172.21.15.33:0/1025754557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e412f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2325438700 1 -- 172.21.15.33:0/1025754557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b091be53b0 con 0x55b091e14ea0 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f232643a700 1 --2- 172.21.15.33:0/1025754557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2325438700 1 --2- 172.21.15.33:0/1025754557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14ea0 0x55b091e40330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2310096960 tx=0x7f231009a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a3438b0ac1c073b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f23097fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f23097fa700 1 -- 172.21.15.33:0/1025754557 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231009dd20 con 0x55b091e14ea0 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f23097fa700 1 -- 172.21.15.33:0/1025754557 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f23100a2370 con 0x55b091e14ea0 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.755+0000 7f2334057b80 1 -- 172.21.15.33:0/1025754557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f231c01e040 con 0x55b091e14ea0 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.756+0000 7f23097fa700 1 -- 172.21.15.33:0/1025754557 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23100b8120 con 0x55b091e14ea0 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.756+0000 7f2334057b80 10 client.25981 fetch_fsmap learned FSMap version 29 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.756+0000 7f2334057b80 1 -- 172.21.15.33:0/1025754557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55b091d2a560 con 0x55b091e14ea0 2024-05-08T21:20:37.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.756+0000 7f23097fa700 1 -- 172.21.15.33:0/1025754557 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2310063020 con 0x55b091e14ea0 2024-05-08T21:20:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.756+0000 7f23097fa700 1 -- 172.21.15.33:0/1025754557 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 29) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2310094b00 con 0x55b091e14ea0 2024-05-08T21:20:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.756+0000 7f2334057b80 10 client.25981 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T21:20:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.756+0000 7f2334057b80 10 client.25981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:20:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.756+0000 7f2334057b80 20 client.25981 populate_metadata read hostname 'smithi033' 2024-05-08T21:20:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.756+0000 7f2334057b80 1 --2- 172.21.15.33:0/1025754557 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b091e89840 0x55b091e8bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:20:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.756+0000 7f2334057b80 4 client.25981 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:20:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.756+0000 7f2334057b80 1 -- 172.21.15.33:0/1025754557 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b091d5cec0 con 0x55b091e89840 2024-05-08T21:20:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.756+0000 7f2325c39700 1 --2- 172.21.15.33:0/1025754557 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b091e89840 0x55b091e8bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:20:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.757+0000 7f23097fa700 10 client.25981 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:37.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.757+0000 7f2325c39700 1 --2- 172.21.15.33:0/1025754557 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b091e89840 0x55b091e8bc30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe5e473fc5e2726 server_cookie=484e910d1af168e in_seq=0 out_seq=0 2024-05-08T21:20:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f23097fa700 1 -- 172.21.15.33:0/1025754557 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b091d5cec0 con 0x55b091e89840 2024-05-08T21:20:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f23097fa700 10 client.25981 handle_command_reply: tid=0 2024-05-08T21:20:37.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f2334057b80 1 client.25981 shutdown 2024-05-08T21:20:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f2334057b80 1 -- 172.21.15.33:0/1025754557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14ea0 msgr2=0x55b091e40330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f2334057b80 1 --2- 172.21.15.33:0/1025754557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14ea0 0x55b091e40330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2310096960 tx=0x7f231009a430 comp rx=0 tx=0).stop 2024-05-08T21:20:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f2334057b80 1 -- 172.21.15.33:0/1025754557 shutdown_connections 2024-05-08T21:20:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f2334057b80 1 --2- 172.21.15.33:0/1025754557 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b091e89840 0x55b091e8bc30 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f2334057b80 1 --2- 172.21.15.33:0/1025754557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b091e15860 0x55b091e412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f2334057b80 1 --2- 172.21.15.33:0/1025754557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b091e14540 0x55b091e3fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f23097fa700 0 client.25981 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:20:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f2334057b80 1 --2- 172.21.15.33:0/1025754557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b091e14ea0 0x55b091e40330 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f2334057b80 1 -- 172.21.15.33:0/1025754557 >> 172.21.15.33:0/1025754557 conn(0x55b091d72210 msgr2=0x55b091e18830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f2334057b80 1 -- 172.21.15.33:0/1025754557 shutdown_connections 2024-05-08T21:20:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f2334057b80 1 -- 172.21.15.33:0/1025754557 wait complete. 2024-05-08T21:20:37.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.758+0000 7f2334057b80 20 client.25981 trim_cache size 0 max 16384 2024-05-08T21:20:37.765 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:20:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:20:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:20:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "377c3c63-ca93-4a03-be06-2107ebce40f4": { 2024-05-08T21:20:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:20:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "377c3c63-ca93-4a03-be06-2107ebce40f4", 2024-05-08T21:20:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:20:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:20:37.765 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:20:37.765 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:20:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.759+0000 7f2327e9d700 1 -- 172.21.15.33:0/2197598816 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f230c0a1120 msgr2=0x7f230c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.759+0000 7f2327e9d700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f230c0a1120 0x7f230c0a35e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f231c005910 tx=0x7f231c00b5d0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.759+0000 7f2327e9d700 1 -- 172.21.15.33:0/2197598816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2320101fc0 msgr2=0x7f23201971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:20:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.759+0000 7f2327e9d700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2320101fc0 0x7f23201971e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f231000e9a0 tx=0x7f231000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.759+0000 7f2327e9d700 1 -- 172.21.15.33:0/2197598816 shutdown_connections 2024-05-08T21:20:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.759+0000 7f2327e9d700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2320102980 0x7f232019e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.759+0000 7f2327e9d700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2320100e70 0x7f2320196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.759+0000 7f2327e9d700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f230c0a1120 0x7f230c0a35e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.759+0000 7f2327e9d700 1 --2- 172.21.15.33:0/2197598816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2320101fc0 0x7f23201971e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:20:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.759+0000 7f2327e9d700 1 -- 172.21.15.33:0/2197598816 >> 172.21.15.33:0/2197598816 conn(0x7f23200fc400 msgr2=0x7f23201054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:20:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.759+0000 7f2327e9d700 1 -- 172.21.15.33:0/2197598816 shutdown_connections 2024-05-08T21:20:37.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:20:37.759+0000 7f2327e9d700 1 -- 172.21.15.33:0/2197598816 wait complete. 2024-05-08T21:20:37.777 INFO:tasks.cephfs.filesystem:scrub status for tag:377c3c63-ca93-4a03-be06-2107ebce40f4 - {'path': '/', 'tag': '377c3c63-ca93-4a03-be06-2107ebce40f4', 'options': 'recursive,force'} 2024-05-08T21:20:37.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:37 smithi071 ceph-mon[47074]: pgmap v339: 129 pgs: 129 active+clean; 11 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 905 KiB/s wr, 27 op/s 2024-05-08T21:20:39.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:39 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:20:39.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:39 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:20:39.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:39 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:20:39.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:39 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:20:39.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:39 smithi082 ceph-mon[47512]: pgmap v340: 129 pgs: 129 active+clean; 15 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-05-08T21:20:39.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:39 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:20:39.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:39 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:20:39.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:39 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:20:39.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:39 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:20:39.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:39 smithi033 ceph-mon[42528]: pgmap v340: 129 pgs: 129 active+clean; 15 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-05-08T21:20:39.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:39 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:20:39.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:39 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:20:39.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:39 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:20:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:39 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:20:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:39 smithi071 ceph-mon[47074]: pgmap v340: 129 pgs: 129 active+clean; 15 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.2 MiB/s wr, 38 op/s 2024-05-08T21:20:41.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:41 smithi082 ceph-mon[47512]: pgmap v341: 129 pgs: 129 active+clean; 17 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.4 MiB/s wr, 53 op/s 2024-05-08T21:20:41.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:41 smithi033 ceph-mon[42528]: pgmap v341: 129 pgs: 129 active+clean; 17 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.4 MiB/s wr, 53 op/s 2024-05-08T21:20:41.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:41 smithi071 ceph-mon[47074]: pgmap v341: 129 pgs: 129 active+clean; 17 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.4 MiB/s wr, 53 op/s 2024-05-08T21:20:43.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:43 smithi082 ceph-mon[47512]: pgmap v342: 129 pgs: 129 active+clean; 17 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.4 MiB/s wr, 60 op/s 2024-05-08T21:20:43.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:43 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:20:43.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:43 smithi033 ceph-mon[42528]: pgmap v342: 129 pgs: 129 active+clean; 17 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.4 MiB/s wr, 60 op/s 2024-05-08T21:20:43.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:43 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:20:43.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:43 smithi071 ceph-mon[47074]: pgmap v342: 129 pgs: 129 active+clean; 17 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.4 MiB/s wr, 60 op/s 2024-05-08T21:20:43.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:43 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:20:45.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:45 smithi082 ceph-mon[47512]: pgmap v343: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-05-08T21:20:45.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:45 smithi033 ceph-mon[42528]: pgmap v343: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-05-08T21:20:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:45 smithi071 ceph-mon[47074]: pgmap v343: 129 pgs: 129 active+clean; 18 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.4 MiB/s wr, 67 op/s 2024-05-08T21:20:47.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:47 smithi033 ceph-mon[42528]: pgmap v344: 129 pgs: 129 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-05-08T21:20:47.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:47 smithi071 ceph-mon[47074]: pgmap v344: 129 pgs: 129 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-05-08T21:20:47.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:47 smithi082 ceph-mon[47512]: pgmap v344: 129 pgs: 129 active+clean; 18 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.1 MiB/s wr, 86 op/s 2024-05-08T21:20:49.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:49 smithi033 ceph-mon[42528]: pgmap v345: 129 pgs: 129 active+clean; 19 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 778 KiB/s wr, 100 op/s 2024-05-08T21:20:49.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:49 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:20:49.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:49 smithi071 ceph-mon[47074]: pgmap v345: 129 pgs: 129 active+clean; 19 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 778 KiB/s wr, 100 op/s 2024-05-08T21:20:49.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:49 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:20:49.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:49 smithi082 ceph-mon[47512]: pgmap v345: 129 pgs: 129 active+clean; 19 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 778 KiB/s wr, 100 op/s 2024-05-08T21:20:49.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:49 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:20:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:51 smithi033 ceph-mon[42528]: pgmap v346: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 495 KiB/s wr, 123 op/s 2024-05-08T21:20:51.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:51 smithi071 ceph-mon[47074]: pgmap v346: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 495 KiB/s wr, 123 op/s 2024-05-08T21:20:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:51 smithi082 ceph-mon[47512]: pgmap v346: 129 pgs: 129 active+clean; 20 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 495 KiB/s wr, 123 op/s 2024-05-08T21:20:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:53 smithi033 ceph-mon[42528]: pgmap v347: 129 pgs: 129 active+clean; 26 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 918 KiB/s wr, 149 op/s 2024-05-08T21:20:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:53 smithi071 ceph-mon[47074]: pgmap v347: 129 pgs: 129 active+clean; 26 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 918 KiB/s wr, 149 op/s 2024-05-08T21:20:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:53 smithi082 ceph-mon[47512]: pgmap v347: 129 pgs: 129 active+clean; 26 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 918 KiB/s wr, 149 op/s 2024-05-08T21:20:54.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:54 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:20:54.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:54 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:20:54.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:54 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:20:55.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:55 smithi033 ceph-mon[42528]: pgmap v348: 129 pgs: 129 active+clean; 27 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 974 KiB/s wr, 189 op/s 2024-05-08T21:20:55.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:55 smithi071 ceph-mon[47074]: pgmap v348: 129 pgs: 129 active+clean; 27 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 974 KiB/s wr, 189 op/s 2024-05-08T21:20:55.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:55 smithi082 ceph-mon[47512]: pgmap v348: 129 pgs: 129 active+clean; 27 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 974 KiB/s wr, 189 op/s 2024-05-08T21:20:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:57 smithi033 ceph-mon[42528]: pgmap v349: 129 pgs: 129 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.3 MiB/s wr, 238 op/s 2024-05-08T21:20:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:57 smithi071 ceph-mon[47074]: pgmap v349: 129 pgs: 129 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.3 MiB/s wr, 238 op/s 2024-05-08T21:20:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:57 smithi082 ceph-mon[47512]: pgmap v349: 129 pgs: 129 active+clean; 30 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 1.3 MiB/s wr, 238 op/s 2024-05-08T21:20:58.650 INFO:tasks.workunit.client.0.smithi033.stderr:Updating files: 7% (321/4287) Updating files: 8% (343/4287) Updating files: 9% (386/4287) Updating files: 10% (429/4287) Updating files: 11% (472/4287) Updating files: 11% (490/4287) Updating files: 12% (515/4287) Updating files: 13% (558/4287) Updating files: 14% (601/4287) Updating files: 15% (644/4287) Updating files: 16% (686/4287) Updating files: 17% (729/4287) Updating files: 17% (731/4287) Updating files: 18% (772/4287) Updating files: 19% (815/4287) Updating files: 20% (858/4287) Updating files: 21% (901/4287) Updating files: 22% (944/4287) Updating files: 23% (987/4287) Updating files: 24% (1029/4287) Updating files: 25% (1072/4287) Updating files: 25% (1088/4287) Updating files: 26% (1115/4287) Updating files: 27% (1158/4287) Updating files: 28% (1201/4287) Updating files: 29% (1244/4287) Updating files: 30% (1287/4287) Updating files: 30% (1288/4287) Updating files: 31% (1329/4287) Updating files: 32% (1372/4287) Updating files: 33% (1415/4287) Updating files: 34% (1458/4287) Updating files: 34% (1488/4287) Updating files: 35% (1501/4287) Updating files: 36% (1544/4287) Updating files: 37% (1587/4287) Updating files: 38% (1630/4287) Updating files: 39% (1672/4287) Updating files: 40% (1715/4287) Updating files: 41% (1758/4287) Updating files: 41% (1782/4287) Updating files: 42% (1801/4287) Updating files: 43% (1844/4287) Updating files: 44% (1887/4287) Updating files: 45% (1930/4287) Updating files: 46% (1973/4287) Updating files: 47% (2015/4287) Updating files: 48% (2058/4287) Updating files: 48% (2098/4287) Updating files: 49% (2101/4287) Updating files: 50% (2144/4287) Updating files: 51% (2187/4287) Updating files: 52% (2230/4287) Updating files: 53% (2273/4287) Updating files: 54% (2315/4287) Updating files: 55% (2358/4287) Updating files: 56% (2401/4287) Updating files: 57% (2444/4287) Updating files: 58% (2487/4287) Updating files: 59% (2530/4287) Updating files: 60% (2573/4287) Updating files: 61% (2616/4287) Updating files: 62% (2658/4287) Updating files: 63% (2701/4287) Updating files: 63% (2712/4287) Updating files: 64% (2744/4287) Updating files: 65% (2787/4287) Updating files: 66% (2830/4287) Updating files: 67% (2873/4287) Updating files: 68% (2916/4287) Updating files: 69% (2959/4287) Updating files: 70% (3001/4287) Updating files: 70% (3022/4287) Updating files: 71% (3044/4287) Updating files: 72% (3087/4287) Updating files: 73% (3130/4287) Updating files: 74% (3173/4287) Updating files: 75% (3216/4287) Updating files: 75% (3237/4287) Updating files: 76% (3259/4287) Updating files: 77% (3301/4287) Updating files: 78% (3344/4287) Updating files: 79% (3387/4287) Updating files: 80% (3430/4287) Updating files: 81% (3473/4287) Updating files: 82% (3516/4287) Updating files: 83% (3559/4287) Updating files: 83% (3566/4287) Updating files: 84% (3602/4287) Updating files: 85% (3644/4287) Updating files: 86% (3687/4287) Updating files: 87% (3730/4287) Updating files: 88% (3773/4287) Updating files: 89% (3816/4287) Updating files: 90% (3859/4287) Updating files: 90% (3870/4287) Updating files: 91% (3902/4287) Updating files: 92% (3945/4287) Updating files: 93% (3987/4287) Updating files: 94% (4030/4287) Updating files: 95% (4073/4287) Updating files: 96% (4116/4287) Updating files: 96% (4131/4287) Updating files: 97% (4159/4287) Updating files: 98% (4202/4287) Updating files: 98% (4238/4287) Updating files: 99% (4245/4287) Updating files: 100% (4287/4287) Updating files: 100% (4287/4287), done. 2024-05-08T21:20:58.730 INFO:tasks.workunit.client.0.smithi033.stderr:+ cd xfstests-dev 2024-05-08T21:20:58.732 INFO:tasks.workunit.client.0.smithi033.stderr:+ git checkout 12973fc04fd10d4af086901e10ffa8e48866b735 2024-05-08T21:20:59.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:20:59 smithi033 ceph-mon[42528]: pgmap v350: 129 pgs: 129 active+clean; 31 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.3 MiB/s wr, 257 op/s 2024-05-08T21:20:59.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:20:59 smithi071 ceph-mon[47074]: pgmap v350: 129 pgs: 129 active+clean; 31 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.3 MiB/s wr, 257 op/s 2024-05-08T21:20:59.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:20:59 smithi082 ceph-mon[47512]: pgmap v350: 129 pgs: 129 active+clean; 31 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.3 MiB/s wr, 257 op/s 2024-05-08T21:21:01.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:01 smithi033 ceph-mon[42528]: pgmap v351: 129 pgs: 129 active+clean; 38 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 311 op/s 2024-05-08T21:21:01.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:01 smithi071 ceph-mon[47074]: pgmap v351: 129 pgs: 129 active+clean; 38 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 311 op/s 2024-05-08T21:21:01.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:01 smithi082 ceph-mon[47512]: pgmap v351: 129 pgs: 129 active+clean; 38 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 311 op/s 2024-05-08T21:21:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:03 smithi033 ceph-mon[42528]: pgmap v352: 129 pgs: 129 active+clean; 40 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 300 op/s 2024-05-08T21:21:03.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:03 smithi071 ceph-mon[47074]: pgmap v352: 129 pgs: 129 active+clean; 40 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 300 op/s 2024-05-08T21:21:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:03 smithi082 ceph-mon[47512]: pgmap v352: 129 pgs: 129 active+clean; 40 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 300 op/s 2024-05-08T21:21:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:05 smithi033 ceph-mon[42528]: pgmap v353: 129 pgs: 129 active+clean; 40 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.4 MiB/s wr, 261 op/s 2024-05-08T21:21:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:05 smithi071 ceph-mon[47074]: pgmap v353: 129 pgs: 129 active+clean; 40 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.4 MiB/s wr, 261 op/s 2024-05-08T21:21:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:05 smithi082 ceph-mon[47512]: pgmap v353: 129 pgs: 129 active+clean; 40 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 1.4 MiB/s wr, 261 op/s 2024-05-08T21:21:07.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:07 smithi033 ceph-mon[42528]: pgmap v354: 129 pgs: 129 active+clean; 42 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-05-08T21:21:07.779 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:21:07.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:07 smithi071 ceph-mon[47074]: pgmap v354: 129 pgs: 129 active+clean; 42 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-05-08T21:21:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:07 smithi082 ceph-mon[47512]: pgmap v354: 129 pgs: 129 active+clean; 42 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 240 op/s 2024-05-08T21:21:07.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.941+0000 7fab3f987700 1 -- 172.21.15.33:0/3848242619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab38100e70 msgr2=0x7fab38101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:07.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.941+0000 7fab3f987700 1 --2- 172.21.15.33:0/3848242619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab38100e70 0x7fab38101290 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab34009990 tx=0x7fab34009ca0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.941+0000 7fab3f987700 1 -- 172.21.15.33:0/3848242619 shutdown_connections 2024-05-08T21:21:07.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.941+0000 7fab3f987700 1 --2- 172.21.15.33:0/3848242619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab38102980 0x7fab381092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.941+0000 7fab3f987700 1 --2- 172.21.15.33:0/3848242619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab38100e70 0x7fab38101290 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.941+0000 7fab3f987700 1 --2- 172.21.15.33:0/3848242619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab38101fc0 0x7fab38102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.941+0000 7fab3f987700 1 -- 172.21.15.33:0/3848242619 >> 172.21.15.33:0/3848242619 conn(0x7fab380fc400 msgr2=0x7fab380fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:07.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.942+0000 7fab3f987700 1 -- 172.21.15.33:0/3848242619 shutdown_connections 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.942+0000 7fab3f987700 1 -- 172.21.15.33:0/3848242619 wait complete. 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.942+0000 7fab3f987700 1 Processor -- start 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.942+0000 7fab3f987700 1 -- start start 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.942+0000 7fab3f987700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab38100e70 0x7fab38196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.942+0000 7fab3f987700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab38101fc0 0x7fab38197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.942+0000 7fab3f987700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab38102980 0x7fab3819e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.942+0000 7fab3f987700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3819e7c0 con 0x7fab38102980 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.942+0000 7fab3f987700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3819e930 con 0x7fab38101fc0 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.942+0000 7fab3f987700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3819eaf0 con 0x7fab38100e70 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3d723700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab38100e70 0x7fab38196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3df24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab38102980 0x7fab3819e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3cf22700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab38101fc0 0x7fab38197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3d723700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab38100e70 0x7fab38196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:45314/0 (socket says 172.21.15.33:45314) 2024-05-08T21:21:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3df24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab38102980 0x7fab3819e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44216/0 (socket says 172.21.15.33:44216) 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3d723700 1 -- 172.21.15.33:0/3607930726 learned_addr learned my addr 172.21.15.33:0/3607930726 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3d723700 1 -- 172.21.15.33:0/3607930726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab38101fc0 msgr2=0x7fab38197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3d723700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab38101fc0 0x7fab38197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3d723700 1 -- 172.21.15.33:0/3607930726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab38102980 msgr2=0x7fab3819e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3d723700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab38102980 0x7fab3819e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3d723700 1 -- 172.21.15.33:0/3607930726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30009c40 con 0x7fab38100e70 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3df24700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab38102980 0x7fab3819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3cf22700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab38101fc0 0x7fab38197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3d723700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab38100e70 0x7fab38196d20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab340092e0 tx=0x7fab340057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab2e7fc700 1 -- 172.21.15.33:0/3607930726 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3401d070 con 0x7fab38100e70 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab2e7fc700 1 -- 172.21.15.33:0/3607930726 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab3400beb0 con 0x7fab38100e70 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3f987700 1 -- 172.21.15.33:0/3607930726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab340095f0 con 0x7fab38100e70 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab2e7fc700 1 -- 172.21.15.33:0/3607930726 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3400fda0 con 0x7fab38100e70 2024-05-08T21:21:07.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.943+0000 7fab3f987700 1 -- 172.21.15.33:0/3607930726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3819f0f0 con 0x7fab38100e70 2024-05-08T21:21:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.946+0000 7fab2e7fc700 1 -- 172.21.15.33:0/3607930726 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fab3400f560 con 0x7fab38100e70 2024-05-08T21:21:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.946+0000 7fab2e7fc700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fab240a1120 0x7fab240a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.946+0000 7fab3cf22700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fab240a1120 0x7fab240a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.946+0000 7fab2e7fc700 1 -- 172.21.15.33:0/3607930726 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fab340cdf20 con 0x7fab38100e70 2024-05-08T21:21:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.947+0000 7fab3cf22700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fab240a1120 0x7fab240a35e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fab28011490 tx=0x7fab28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.947+0000 7fab4bb41b80 1 Processor -- start 2024-05-08T21:21:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.947+0000 7fab4bb41b80 1 -- start start 2024-05-08T21:21:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab4bb41b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fa540 0x56308e1fa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab4bb41b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1faea0 0x56308e1fb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab4bb41b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1fb860 0x56308e1fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab4bb41b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308e151150 con 0x56308e1faea0 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab4bb41b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308e14eb10 con 0x56308e1fb860 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab4bb41b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308e14a400 con 0x56308e1fa540 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab3cf22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1faea0 0x56308e1fb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab3df24700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1fb860 0x56308e1fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab3df24700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1fb860 0x56308e1fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49190/0 (socket says 172.21.15.33:49190) 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab3df24700 1 -- 172.21.15.33:0/995577975 learned_addr learned my addr 172.21.15.33:0/995577975 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab3d723700 1 --2- 172.21.15.33:0/995577975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fa540 0x56308e1fa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab3df24700 1 -- 172.21.15.33:0/995577975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fa540 msgr2=0x56308e1fa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab3df24700 1 --2- 172.21.15.33:0/995577975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fa540 0x56308e1fa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab3df24700 1 -- 172.21.15.33:0/995577975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1faea0 msgr2=0x56308e1fb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab3df24700 1 --2- 172.21.15.33:0/995577975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1faea0 0x56308e1fb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab3df24700 1 -- 172.21.15.33:0/995577975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56308dfcb3b0 con 0x56308e1fb860 2024-05-08T21:21:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.948+0000 7fab3df24700 1 --2- 172.21.15.33:0/995577975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1fb860 0x56308e1fbce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fab30007640 tx=0x7fab30009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d667ab250984dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.949+0000 7fab237fe700 1 -- 172.21.15.33:0/995577975 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30009100 con 0x56308e1fb860 2024-05-08T21:21:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.949+0000 7fab237fe700 1 -- 172.21.15.33:0/995577975 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab3000f040 con 0x56308e1fb860 2024-05-08T21:21:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.949+0000 7fab237fe700 1 -- 172.21.15.33:0/995577975 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30015e00 con 0x56308e1fb860 2024-05-08T21:21:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.949+0000 7fab4bb41b80 1 -- 172.21.15.33:0/995577975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1fb860 msgr2=0x56308e1fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.949+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/995577975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1fb860 0x56308e1fbce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fab30007640 tx=0x7fab30009920 comp rx=0 tx=0).stop 2024-05-08T21:21:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.949+0000 7fab4bb41b80 1 -- 172.21.15.33:0/995577975 shutdown_connections 2024-05-08T21:21:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.949+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/995577975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1fb860 0x56308e1fbce0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.949+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/995577975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fa540 0x56308e1fa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.949+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/995577975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1faea0 0x56308e1fb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.949+0000 7fab4bb41b80 1 -- 172.21.15.33:0/995577975 >> 172.21.15.33:0/995577975 conn(0x56308e158210 msgr2=0x56308e1f4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.949+0000 7fab4bb41b80 1 -- 172.21.15.33:0/995577975 shutdown_connections 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.949+0000 7fab4bb41b80 1 -- 172.21.15.33:0/995577975 wait complete. 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.950+0000 7fab4bb41b80 1 -- start start 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.950+0000 7fab4bb41b80 1 Processor -- start 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.950+0000 7fab4bb41b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 0x56308e203de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.950+0000 7fab4bb41b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1faea0 0x56308e204320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.950+0000 7fab4bb41b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fb860 0x56308e20b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.950+0000 7fab4bb41b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308e0e6560 con 0x56308e1fa540 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.950+0000 7fab4bb41b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308e0e6200 con 0x56308e1faea0 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.950+0000 7fab4bb41b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308dfcd6a0 con 0x56308e1fb860 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab4bb41b80 10 client.0 _ll_register_callbacks cb 0x56308e0efb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:07.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3d723700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 0x56308e203de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3d723700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 0x56308e203de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44242/0 (socket says 172.21.15.33:44242) 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3df24700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fb860 0x56308e20b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3d723700 1 -- 172.21.15.33:0/3848563705 learned_addr learned my addr 172.21.15.33:0/3848563705 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3cf22700 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1faea0 0x56308e204320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3d723700 1 -- 172.21.15.33:0/3848563705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fb860 msgr2=0x56308e20b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3d723700 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fb860 0x56308e20b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3d723700 1 -- 172.21.15.33:0/3848563705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1faea0 msgr2=0x56308e204320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3d723700 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1faea0 0x56308e204320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3d723700 1 -- 172.21.15.33:0/3848563705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab30009260 con 0x56308e1fa540 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3cf22700 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1faea0 0x56308e204320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3df24700 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fb860 0x56308e20b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab3d723700 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 0x56308e203de0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fab34005aa0 tx=0x7fab3409ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3cc5b562c0cb6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab20ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab20ff9700 1 -- 172.21.15.33:0/3848563705 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3409e030 con 0x56308e1fa540 2024-05-08T21:21:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab20ff9700 1 -- 172.21.15.33:0/3848563705 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fab3409b080 con 0x56308e1fa540 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.951+0000 7fab4bb41b80 1 -- 172.21.15.33:0/3848563705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56308e0a3270 con 0x56308e1fa540 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.952+0000 7fab20ff9700 1 -- 172.21.15.33:0/3848563705 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab3409eeb0 con 0x56308e1fa540 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.952+0000 7fab4bb41b80 10 client.16107 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.952+0000 7fab4bb41b80 1 -- 172.21.15.33:0/3848563705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56308df77b60 con 0x56308e1fa540 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.952+0000 7fab20ff9700 1 -- 172.21.15.33:0/3848563705 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab34064080 con 0x56308e1fa540 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.952+0000 7fab20ff9700 1 -- 172.21.15.33:0/3848563705 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fab3409a570 con 0x56308e1fa540 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.952+0000 7fab4bb41b80 10 client.16107 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.952+0000 7fab4bb41b80 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.952+0000 7fab4bb41b80 20 client.16107 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.952+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56308e20f980 0x56308e211da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.952+0000 7fab4bb41b80 4 client.16107 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.952+0000 7fab4bb41b80 1 -- 172.21.15.33:0/3848563705 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56308e1c9810 con 0x56308e20f980 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.952+0000 7fab3cf22700 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56308e20f980 0x56308e211da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.953+0000 7fab3cf22700 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56308e20f980 0x56308e211da0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=912ae25ce83d6655 server_cookie=1fd226543c5cb438 in_seq=0 out_seq=0 2024-05-08T21:21:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.953+0000 7fab20ff9700 10 client.16107 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.953+0000 7fab20ff9700 1 -- 172.21.15.33:0/3848563705 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56308e1c9810 con 0x56308e20f980 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.953+0000 7fab20ff9700 10 client.16107 handle_command_reply: tid=0 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.953+0000 7fab4bb41b80 1 client.16107 shutdown 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.954+0000 7fab4bb41b80 1 -- 172.21.15.33:0/3848563705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 msgr2=0x56308e203de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.954+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 0x56308e203de0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fab34005aa0 tx=0x7fab3409ac90 comp rx=0 tx=0).stop 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.954+0000 7fab4bb41b80 1 -- 172.21.15.33:0/3848563705 shutdown_connections 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.954+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56308e20f980 0x56308e211da0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.954+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fb860 0x56308e20b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.954+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 0x56308e203de0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.954+0000 7fab20ff9700 0 client.16107 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.954+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/3848563705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1faea0 0x56308e204320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.954+0000 7fab4bb41b80 1 -- 172.21.15.33:0/3848563705 >> 172.21.15.33:0/3848563705 conn(0x56308e158210 msgr2=0x56308e1f45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.954+0000 7fab4bb41b80 1 -- 172.21.15.33:0/3848563705 shutdown_connections 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.954+0000 7fab4bb41b80 1 -- 172.21.15.33:0/3848563705 wait complete. 2024-05-08T21:21:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.954+0000 7fab4bb41b80 20 client.16107 trim_cache size 0 max 16384 2024-05-08T21:21:07.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.980+0000 7fab4bb41b80 1 Processor -- start 2024-05-08T21:21:07.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.980+0000 7fab4bb41b80 1 -- start start 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab4bb41b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 0x56308e20ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab4bb41b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1faea0 0x56308e20b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab4bb41b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1fb860 0x56308e1f1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab4bb41b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308e1f6570 con 0x56308e1fa540 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab4bb41b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308e216fb0 con 0x56308e1fb860 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab4bb41b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308e1ef800 con 0x56308e1faea0 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab3d723700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 0x56308e20ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab3d723700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 0x56308e20ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44258/0 (socket says 172.21.15.33:44258) 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab3d723700 1 -- 172.21.15.33:0/2657236024 learned_addr learned my addr 172.21.15.33:0/2657236024 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab3df24700 1 --2- 172.21.15.33:0/2657236024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1fb860 0x56308e1f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab3cf22700 1 --2- 172.21.15.33:0/2657236024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1faea0 0x56308e20b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab3d723700 1 -- 172.21.15.33:0/2657236024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1faea0 msgr2=0x56308e20b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab3d723700 1 --2- 172.21.15.33:0/2657236024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1faea0 0x56308e20b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab3d723700 1 -- 172.21.15.33:0/2657236024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1fb860 msgr2=0x56308e1f1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab3d723700 1 --2- 172.21.15.33:0/2657236024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1fb860 0x56308e1f1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab3d723700 1 -- 172.21.15.33:0/2657236024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56308e110560 con 0x56308e1fa540 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab3cf22700 1 --2- 172.21.15.33:0/2657236024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1faea0 0x56308e20b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab3d723700 1 --2- 172.21.15.33:0/2657236024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 0x56308e20ba10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fab34097580 tx=0x7fab34005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f7a9f43d92ae3f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.981+0000 7fab23fff700 1 -- 172.21.15.33:0/2657236024 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3409e040 con 0x56308e1fa540 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.982+0000 7fab23fff700 1 -- 172.21.15.33:0/2657236024 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56308e0a3270 con 0x56308e1fa540 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.982+0000 7fab23fff700 1 -- 172.21.15.33:0/2657236024 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab340a3470 con 0x56308e1fa540 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.982+0000 7fab4bb41b80 1 -- 172.21.15.33:0/2657236024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 msgr2=0x56308e20ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.982+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/2657236024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 0x56308e20ba10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fab34097580 tx=0x7fab34005870 comp rx=0 tx=0).stop 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.982+0000 7fab4bb41b80 1 -- 172.21.15.33:0/2657236024 shutdown_connections 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.982+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/2657236024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1fb860 0x56308e1f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.982+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/2657236024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fa540 0x56308e20ba10 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.982+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/2657236024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1faea0 0x56308e20b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.982+0000 7fab4bb41b80 1 -- 172.21.15.33:0/2657236024 >> 172.21.15.33:0/2657236024 conn(0x56308e158210 msgr2=0x56308e1fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:07.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.982+0000 7fab4bb41b80 1 -- 172.21.15.33:0/2657236024 shutdown_connections 2024-05-08T21:21:07.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.982+0000 7fab4bb41b80 1 -- 172.21.15.33:0/2657236024 wait complete. 2024-05-08T21:21:07.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.983+0000 7fab4bb41b80 1 -- start start 2024-05-08T21:21:07.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.983+0000 7fab4bb41b80 1 Processor -- start 2024-05-08T21:21:07.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.983+0000 7fab4bb41b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fa540 0x56308e206b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.983+0000 7fab4bb41b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1faea0 0x56308e2070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.983+0000 7fab4bb41b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fb860 0x56308e20a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.983+0000 7fab4bb41b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308e20a630 con 0x56308e1fb860 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.983+0000 7fab4bb41b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308e20a7a0 con 0x56308e1faea0 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.983+0000 7fab4bb41b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56308e20a960 con 0x56308e1fa540 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.983+0000 7fab4bb41b80 10 client.0 _ll_register_callbacks cb 0x56308e0efb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.983+0000 7fab3d723700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fa540 0x56308e206b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.983+0000 7fab3df24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fb860 0x56308e20a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab3cf22700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1faea0 0x56308e2070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab3d723700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fa540 0x56308e206b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:45340/0 (socket says 172.21.15.33:45340) 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab3d723700 1 -- 172.21.15.33:0/1612997529 learned_addr learned my addr 172.21.15.33:0/1612997529 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab3d723700 1 -- 172.21.15.33:0/1612997529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1faea0 msgr2=0x56308e2070d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab3d723700 1 --2- 172.21.15.33:0/1612997529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1faea0 0x56308e2070d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab3d723700 1 -- 172.21.15.33:0/1612997529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fb860 msgr2=0x56308e20a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:07.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab3d723700 1 --2- 172.21.15.33:0/1612997529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fb860 0x56308e20a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab3d723700 1 -- 172.21.15.33:0/1612997529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56308df77b60 con 0x56308e1fa540 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab3cf22700 1 --2- 172.21.15.33:0/1612997529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1faea0 0x56308e2070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab3d723700 1 --2- 172.21.15.33:0/1612997529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fa540 0x56308e206b90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab34005f50 tx=0x7fab340cb720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=183792b6f0246864 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab20ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab20ff9700 1 -- 172.21.15.33:0/1612997529 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3409e070 con 0x56308e1fa540 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab20ff9700 1 -- 172.21.15.33:0/1612997529 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fab3409b4f0 con 0x56308e1fa540 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab4bb41b80 1 -- 172.21.15.33:0/1612997529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab3000f040 con 0x56308e1fa540 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab20ff9700 1 -- 172.21.15.33:0/1612997529 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab340b93f0 con 0x56308e1fa540 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab4bb41b80 10 client.25999 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.984+0000 7fab4bb41b80 1 -- 172.21.15.33:0/1612997529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56308dfcfcd0 con 0x56308e1fa540 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.985+0000 7fab20ff9700 1 -- 172.21.15.33:0/1612997529 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab3000f040 con 0x56308e1fa540 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.985+0000 7fab20ff9700 1 -- 172.21.15.33:0/1612997529 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fab34096e70 con 0x56308e1fa540 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.985+0000 7fab4bb41b80 10 client.25999 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.985+0000 7fab4bb41b80 10 client.25999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.985+0000 7fab4bb41b80 20 client.25999 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:07.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.985+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/1612997529 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56308e266330 0x56308e268720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.985+0000 7fab4bb41b80 4 client.25999 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.985+0000 7fab4bb41b80 1 -- 172.21.15.33:0/1612997529 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56308e142ec0 con 0x56308e266330 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.985+0000 7fab3cf22700 1 --2- 172.21.15.33:0/1612997529 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56308e266330 0x56308e268720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.985+0000 7fab20ff9700 10 client.25999 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.985+0000 7fab3cf22700 1 --2- 172.21.15.33:0/1612997529 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56308e266330 0x56308e268720 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e717b4c1ee7938f server_cookie=c9b1e031d327822f in_seq=0 out_seq=0 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab20ff9700 1 -- 172.21.15.33:0/1612997529 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56308e142ec0 con 0x56308e266330 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab20ff9700 10 client.25999 handle_command_reply: tid=0 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab4bb41b80 1 client.25999 shutdown 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab4bb41b80 1 -- 172.21.15.33:0/1612997529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fa540 msgr2=0x56308e206b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/1612997529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fa540 0x56308e206b90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fab34005f50 tx=0x7fab340cb720 comp rx=0 tx=0).stop 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab4bb41b80 1 -- 172.21.15.33:0/1612997529 shutdown_connections 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/1612997529 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56308e266330 0x56308e268720 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/1612997529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56308e1fb860 0x56308e20a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/1612997529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56308e1fa540 0x56308e206b90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab20ff9700 0 client.25999 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:07.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab4bb41b80 1 --2- 172.21.15.33:0/1612997529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56308e1faea0 0x56308e2070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab4bb41b80 1 -- 172.21.15.33:0/1612997529 >> 172.21.15.33:0/1612997529 conn(0x56308e158210 msgr2=0x56308e201d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:07.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab4bb41b80 1 -- 172.21.15.33:0/1612997529 shutdown_connections 2024-05-08T21:21:07.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab4bb41b80 1 -- 172.21.15.33:0/1612997529 wait complete. 2024-05-08T21:21:07.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.986+0000 7fab4bb41b80 20 client.25999 trim_cache size 0 max 16384 2024-05-08T21:21:07.994 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:21:07.994 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:21:07.994 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:21:07.994 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:21:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.987+0000 7fab3f987700 1 -- 172.21.15.33:0/3607930726 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fab240a1120 msgr2=0x7fab240a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.987+0000 7fab3f987700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fab240a1120 0x7fab240a35e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fab28011490 tx=0x7fab28009260 comp rx=0 tx=0).stop 2024-05-08T21:21:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.987+0000 7fab3f987700 1 -- 172.21.15.33:0/3607930726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab38100e70 msgr2=0x7fab38196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:07.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.987+0000 7fab3f987700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab38100e70 0x7fab38196d20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab340092e0 tx=0x7fab340057c0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.987+0000 7fab3f987700 1 -- 172.21.15.33:0/3607930726 shutdown_connections 2024-05-08T21:21:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.987+0000 7fab3f987700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab38102980 0x7fab3819e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.987+0000 7fab3f987700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab38100e70 0x7fab38196d20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.987+0000 7fab3f987700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fab240a1120 0x7fab240a35e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.987+0000 7fab3f987700 1 --2- 172.21.15.33:0/3607930726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab38101fc0 0x7fab38197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.987+0000 7fab3f987700 1 -- 172.21.15.33:0/3607930726 >> 172.21.15.33:0/3607930726 conn(0x7fab380fc400 msgr2=0x7fab381054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.987+0000 7fab3f987700 1 -- 172.21.15.33:0/3607930726 shutdown_connections 2024-05-08T21:21:07.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:07.987+0000 7fab3f987700 1 -- 172.21.15.33:0/3607930726 wait complete. 2024-05-08T21:21:08.002 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:21:08.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:21:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.141+0000 7f17ddef1700 1 -- 172.21.15.33:0/1270080810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17d8101fd0 msgr2=0x7f17d8102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.141+0000 7f17ddef1700 1 --2- 172.21.15.33:0/1270080810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17d8101fd0 0x7f17d8102450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0098f0 tx=0x7f17cc009c00 comp rx=0 tx=0).stop 2024-05-08T21:21:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.142+0000 7f17ddef1700 1 -- 172.21.15.33:0/1270080810 shutdown_connections 2024-05-08T21:21:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.142+0000 7f17ddef1700 1 --2- 172.21.15.33:0/1270080810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17d8102990 0x7f17d81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.142+0000 7f17ddef1700 1 --2- 172.21.15.33:0/1270080810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d8100e80 0x7f17d81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.142+0000 7f17ddef1700 1 --2- 172.21.15.33:0/1270080810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17d8101fd0 0x7f17d8102450 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.142+0000 7f17ddef1700 1 -- 172.21.15.33:0/1270080810 >> 172.21.15.33:0/1270080810 conn(0x7f17d80fc410 msgr2=0x7f17d80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.142+0000 7f17ddef1700 1 -- 172.21.15.33:0/1270080810 shutdown_connections 2024-05-08T21:21:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.142+0000 7f17ddef1700 1 -- 172.21.15.33:0/1270080810 wait complete. 2024-05-08T21:21:08.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.142+0000 7f17ddef1700 1 Processor -- start 2024-05-08T21:21:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.142+0000 7f17ddef1700 1 -- start start 2024-05-08T21:21:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17ddef1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d8100e80 0x7f17d8196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17ddef1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17d8101fd0 0x7f17d8197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17ddef1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17d8102990 0x7f17d819e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17ddef1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d819e820 con 0x7f17d8102990 2024-05-08T21:21:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17ddef1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d819e990 con 0x7f17d8100e80 2024-05-08T21:21:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17ddef1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d819eb50 con 0x7f17d8101fd0 2024-05-08T21:21:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17d77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d8100e80 0x7f17d8196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17d8102990 0x7f17d819e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17d77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d8100e80 0x7f17d8196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49234/0 (socket says 172.21.15.33:49234) 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17d77fe700 1 -- 172.21.15.33:0/2375627628 learned_addr learned my addr 172.21.15.33:0/2375627628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17d6ffd700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17d8101fd0 0x7f17d8197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17d77fe700 1 -- 172.21.15.33:0/2375627628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17d8101fd0 msgr2=0x7f17d8197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17d77fe700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17d8101fd0 0x7f17d8197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17d77fe700 1 -- 172.21.15.33:0/2375627628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17d8102990 msgr2=0x7f17d819e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17d77fe700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17d8102990 0x7f17d819e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17d77fe700 1 -- 172.21.15.33:0/2375627628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c80095f0 con 0x7f17d8100e80 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.143+0000 7f17d77fe700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d8100e80 0x7f17d8196d00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f17c800ec10 tx=0x7f17c800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.144+0000 7f17d4ff9700 1 -- 172.21.15.33:0/2375627628 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c800cc80 con 0x7f17d8100e80 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.144+0000 7f17d4ff9700 1 -- 172.21.15.33:0/2375627628 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f17c800cde0 con 0x7f17d8100e80 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.144+0000 7f17d4ff9700 1 -- 172.21.15.33:0/2375627628 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8010430 con 0x7f17d8100e80 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.144+0000 7f17ddef1700 1 -- 172.21.15.33:0/2375627628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d0009c40 con 0x7f17d8100e80 2024-05-08T21:21:08.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.144+0000 7f17ddef1700 1 -- 172.21.15.33:0/2375627628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc009520 con 0x7f17d8100e80 2024-05-08T21:21:08.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.145+0000 7f17ddef1700 1 -- 172.21.15.33:0/2375627628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d81083e0 con 0x7f17d8100e80 2024-05-08T21:21:08.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.146+0000 7f17d4ff9700 1 -- 172.21.15.33:0/2375627628 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f17c8003eb0 con 0x7f17d8100e80 2024-05-08T21:21:08.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.147+0000 7f17d4ff9700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17c00a11a0 0x7f17c00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.147+0000 7f17d6ffd700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17c00a11a0 0x7f17c00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.147+0000 7f17d4ff9700 1 -- 172.21.15.33:0/2375627628 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f17c8014070 con 0x7f17d8100e80 2024-05-08T21:21:08.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.147+0000 7f17d6ffd700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17c00a11a0 0x7f17c00a3660 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f17cc005e90 tx=0x7f17cc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.151+0000 7f17d4ff9700 1 -- 172.21.15.33:0/2375627628 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f17c8096b80 con 0x7f17d8100e80 2024-05-08T21:21:08.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.471+0000 7f17ddef1700 1 -- 172.21.15.33:0/2375627628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17d8067d10 con 0x7f17d8100e80 2024-05-08T21:21:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.472+0000 7f17d4ff9700 1 -- 172.21.15.33:0/2375627628 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10906 (secure 0 0 0) 0x7f17c809a1a0 con 0x7f17d8100e80 2024-05-08T21:21:08.474 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:21:08.474 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:20:53.371310+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.475+0000 7f17ddef1700 1 -- 172.21.15.33:0/2375627628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17c00a11a0 msgr2=0x7f17c00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.475+0000 7f17ddef1700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17c00a11a0 0x7f17c00a3660 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f17cc005e90 tx=0x7f17cc019040 comp rx=0 tx=0).stop 2024-05-08T21:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.475+0000 7f17ddef1700 1 -- 172.21.15.33:0/2375627628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d8100e80 msgr2=0x7f17d8196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.475+0000 7f17ddef1700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d8100e80 0x7f17d8196d00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f17c800ec10 tx=0x7f17c800ef20 comp rx=0 tx=0).stop 2024-05-08T21:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.475+0000 7f17ddef1700 1 -- 172.21.15.33:0/2375627628 shutdown_connections 2024-05-08T21:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.475+0000 7f17ddef1700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17d8102990 0x7f17d819e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.475+0000 7f17ddef1700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17c00a11a0 0x7f17c00a3660 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.475+0000 7f17ddef1700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17d8100e80 0x7f17d8196d00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.475+0000 7f17ddef1700 1 --2- 172.21.15.33:0/2375627628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17d8101fd0 0x7f17d8197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.475+0000 7f17ddef1700 1 -- 172.21.15.33:0/2375627628 >> 172.21.15.33:0/2375627628 conn(0x7f17d80fc410 msgr2=0x7f17d8105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.475+0000 7f17ddef1700 1 -- 172.21.15.33:0/2375627628 shutdown_connections 2024-05-08T21:21:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.475+0000 7f17ddef1700 1 -- 172.21.15.33:0/2375627628 wait complete. 2024-05-08T21:21:08.478 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 31 2024-05-08T21:21:08.491 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:21:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.634+0000 7fa64fac8700 1 -- 172.21.15.33:0/2663708643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa648101940 msgr2=0x7fa648103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.634+0000 7fa64fac8700 1 --2- 172.21.15.33:0/2663708643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa648101940 0x7fa648103d30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa644009990 tx=0x7fa644009ca0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.634+0000 7fa64fac8700 1 -- 172.21.15.33:0/2663708643 shutdown_connections 2024-05-08T21:21:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.634+0000 7fa64fac8700 1 --2- 172.21.15.33:0/2663708643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa648106ba0 0x7fa648108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.634+0000 7fa64fac8700 1 --2- 172.21.15.33:0/2663708643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa648101940 0x7fa648103d30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.634+0000 7fa64fac8700 1 --2- 172.21.15.33:0/2663708643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648104270 0x7fa648106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.634+0000 7fa64fac8700 1 -- 172.21.15.33:0/2663708643 >> 172.21.15.33:0/2663708643 conn(0x7fa6480fb290 msgr2=0x7fa6480fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.635+0000 7fa64fac8700 1 -- 172.21.15.33:0/2663708643 shutdown_connections 2024-05-08T21:21:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.635+0000 7fa64fac8700 1 -- 172.21.15.33:0/2663708643 wait complete. 2024-05-08T21:21:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.635+0000 7fa64fac8700 1 Processor -- start 2024-05-08T21:21:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.635+0000 7fa64fac8700 1 -- start start 2024-05-08T21:21:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.635+0000 7fa64fac8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa648101940 0x7fa648196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.635+0000 7fa64fac8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa648104270 0x7fa648197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.635+0000 7fa64fac8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648106ba0 0x7fa64819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.635+0000 7fa64fac8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64819e7f0 con 0x7fa648101940 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.635+0000 7fa64fac8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64819e960 con 0x7fa648106ba0 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.635+0000 7fa64fac8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64819eb20 con 0x7fa648104270 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64d864700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa648101940 0x7fa648196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64d864700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa648101940 0x7fa648196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44322/0 (socket says 172.21.15.33:44322) 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64d063700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa648104270 0x7fa648197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64d864700 1 -- 172.21.15.33:0/249218046 learned_addr learned my addr 172.21.15.33:0/249218046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64e065700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648106ba0 0x7fa64819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64d864700 1 -- 172.21.15.33:0/249218046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa648104270 msgr2=0x7fa648197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64d864700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa648104270 0x7fa648197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64d864700 1 -- 172.21.15.33:0/249218046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648106ba0 msgr2=0x7fa64819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64d864700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648106ba0 0x7fa64819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64d864700 1 -- 172.21.15.33:0/249218046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa638009520 con 0x7fa648101940 2024-05-08T21:21:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64e065700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648106ba0 0x7fa64819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64d864700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa648101940 0x7fa648196cd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa644004710 tx=0x7fa644009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa63effd700 1 -- 172.21.15.33:0/249218046 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64401d070 con 0x7fa648101940 2024-05-08T21:21:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64fac8700 1 -- 172.21.15.33:0/249218046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6440095f0 con 0x7fa648101940 2024-05-08T21:21:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa63effd700 1 -- 172.21.15.33:0/249218046 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa64400be70 con 0x7fa648101940 2024-05-08T21:21:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa63effd700 1 -- 172.21.15.33:0/249218046 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644021410 con 0x7fa648101940 2024-05-08T21:21:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.636+0000 7fa64fac8700 1 -- 172.21.15.33:0/249218046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64819f040 con 0x7fa648101940 2024-05-08T21:21:08.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.639+0000 7fa63effd700 1 -- 172.21.15.33:0/249218046 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa64400f680 con 0x7fa648101940 2024-05-08T21:21:08.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.640+0000 7fa63effd700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa6340a1120 0x7fa6340a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.640+0000 7fa63effd700 1 -- 172.21.15.33:0/249218046 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa6440cded0 con 0x7fa648101940 2024-05-08T21:21:08.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.640+0000 7fa64d063700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa6340a1120 0x7fa6340a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.640+0000 7fa64d063700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa6340a1120 0x7fa6340a35e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa648198180 tx=0x7fa638009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.640+0000 7fa65bc82b80 1 Processor -- start 2024-05-08T21:21:08.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa65bc82b80 1 -- start start 2024-05-08T21:21:08.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa65bc82b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273540 0x565104273960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa65bc82b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273ea0 0x565104274320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa65bc82b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104274860 0x565104274ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa65bc82b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651041ca150 con 0x565104274860 2024-05-08T21:21:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa65bc82b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651041c7b10 con 0x565104273540 2024-05-08T21:21:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa65bc82b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651041c3400 con 0x565104273ea0 2024-05-08T21:21:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa64d864700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273540 0x565104273960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa64d063700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273ea0 0x565104274320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa64e065700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104274860 0x565104274ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa64d864700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273540 0x565104273960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49256/0 (socket says 172.21.15.33:49256) 2024-05-08T21:21:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa64d864700 1 -- 172.21.15.33:0/1858424348 learned_addr learned my addr 172.21.15.33:0/1858424348 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa64d063700 1 -- 172.21.15.33:0/1858424348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273540 msgr2=0x565104273960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa64d063700 1 --2- 172.21.15.33:0/1858424348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273540 0x565104273960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa64d063700 1 -- 172.21.15.33:0/1858424348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104274860 msgr2=0x565104274ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa64d063700 1 --2- 172.21.15.33:0/1858424348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104274860 0x565104274ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa64d063700 1 -- 172.21.15.33:0/1858424348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651040443b0 con 0x565104273ea0 2024-05-08T21:21:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.641+0000 7fa64d864700 1 --2- 172.21.15.33:0/1858424348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273540 0x565104273960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.642+0000 7fa64d063700 1 --2- 172.21.15.33:0/1858424348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273ea0 0x565104274320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa63800e540 tx=0x7fa638014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a13ba76a2202e65 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.642+0000 7fa633fff700 1 -- 172.21.15.33:0/1858424348 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638004030 con 0x565104273ea0 2024-05-08T21:21:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.642+0000 7fa633fff700 1 -- 172.21.15.33:0/1858424348 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa638021070 con 0x565104273ea0 2024-05-08T21:21:08.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.642+0000 7fa633fff700 1 -- 172.21.15.33:0/1858424348 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638026470 con 0x565104273ea0 2024-05-08T21:21:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.643+0000 7fa65bc82b80 1 -- 172.21.15.33:0/1858424348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273ea0 msgr2=0x565104274320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.643+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/1858424348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273ea0 0x565104274320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa63800e540 tx=0x7fa638014000 comp rx=0 tx=0).stop 2024-05-08T21:21:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.643+0000 7fa65bc82b80 1 -- 172.21.15.33:0/1858424348 shutdown_connections 2024-05-08T21:21:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.643+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/1858424348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104274860 0x565104274ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.643+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/1858424348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273540 0x565104273960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.643+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/1858424348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273ea0 0x565104274320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.643+0000 7fa65bc82b80 1 -- 172.21.15.33:0/1858424348 >> 172.21.15.33:0/1858424348 conn(0x5651041d1210 msgr2=0x56510426dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.643+0000 7fa65bc82b80 1 -- 172.21.15.33:0/1858424348 shutdown_connections 2024-05-08T21:21:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.643+0000 7fa65bc82b80 1 -- 172.21.15.33:0/1858424348 wait complete. 2024-05-08T21:21:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.643+0000 7fa65bc82b80 1 -- start start 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.643+0000 7fa65bc82b80 1 Processor -- start 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa65bc82b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 0x56510427ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa65bc82b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273ea0 0x56510427d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa65bc82b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104274860 0x565104284450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa65bc82b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510415f560 con 0x565104273ea0 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa65bc82b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510415f200 con 0x565104274860 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa65bc82b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651040466a0 con 0x565104273540 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa65bc82b80 10 client.0 _ll_register_callbacks cb 0x565104168b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64d864700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 0x56510427ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64d063700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273ea0 0x56510427d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64d864700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 0x56510427ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:45402/0 (socket says 172.21.15.33:45402) 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64d063700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273ea0 0x56510427d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44332/0 (socket says 172.21.15.33:44332) 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64d864700 1 -- 172.21.15.33:0/3493673219 learned_addr learned my addr 172.21.15.33:0/3493673219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64e065700 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104274860 0x565104284450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64d864700 1 -- 172.21.15.33:0/3493673219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104274860 msgr2=0x565104284450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64d864700 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104274860 0x565104284450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64d864700 1 -- 172.21.15.33:0/3493673219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273ea0 msgr2=0x56510427d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64d864700 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273ea0 0x56510427d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64d864700 1 -- 172.21.15.33:0/3493673219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64819f040 con 0x565104273540 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64e065700 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104274860 0x565104284450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa64d864700 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 0x56510427ced0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa644003f60 tx=0x7fa64409b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af59c68c208fc986 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.644+0000 7fa6317fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa6317fa700 1 -- 172.21.15.33:0/3493673219 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64409b030 con 0x565104273540 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa6317fa700 1 -- 172.21.15.33:0/3493673219 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa64409ee60 con 0x565104273540 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3493673219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56510411c270 con 0x565104273540 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa6317fa700 1 -- 172.21.15.33:0/3493673219 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6440a3420 con 0x565104273540 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa65bc82b80 10 client.26017 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3493673219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fa638017070 con 0x565104273540 2024-05-08T21:21:08.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa6317fa700 1 -- 172.21.15.33:0/3493673219 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa644064070 con 0x565104273540 2024-05-08T21:21:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa6317fa700 1 -- 172.21.15.33:0/3493673219 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa64400b9e0 con 0x565104273540 2024-05-08T21:21:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa65bc82b80 10 client.26017 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa65bc82b80 10 client.26017 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:21:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa65bc82b80 20 client.26017 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5651042888f0 0x56510428ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa65bc82b80 4 client.26017 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.645+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3493673219 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565104242810 con 0x5651042888f0 2024-05-08T21:21:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.646+0000 7fa64d063700 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5651042888f0 0x56510428ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.646+0000 7fa6317fa700 10 client.26017 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:21:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.646+0000 7fa64d063700 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5651042888f0 0x56510428ad10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9ac3dc30f64d2375 server_cookie=c5fde7b0a8f7a619 in_seq=0 out_seq=0 2024-05-08T21:21:08.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa6317fa700 1 -- 172.21.15.33:0/3493673219 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565104242810 con 0x5651042888f0 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa6317fa700 10 client.26017 handle_command_reply: tid=0 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa65bc82b80 1 client.26017 shutdown 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3493673219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 msgr2=0x56510427ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 0x56510427ced0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa644003f60 tx=0x7fa64409b570 comp rx=0 tx=0).stop 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3493673219 shutdown_connections 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104274860 0x565104284450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5651042888f0 0x56510428ad10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 0x56510427ced0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/3493673219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273ea0 0x56510427d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa6317fa700 0 client.26017 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3493673219 >> 172.21.15.33:0/3493673219 conn(0x5651041d1210 msgr2=0x56510426d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3493673219 shutdown_connections 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3493673219 wait complete. 2024-05-08T21:21:08.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.647+0000 7fa65bc82b80 20 client.26017 trim_cache size 0 max 16384 2024-05-08T21:21:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.661+0000 7fa65bc82b80 1 Processor -- start 2024-05-08T21:21:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.661+0000 7fa65bc82b80 1 -- start start 2024-05-08T21:21:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa65bc82b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 0x565104273920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa65bc82b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273ea0 0x565104284410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa65bc82b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104274860 0x56510427bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa65bc82b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56510426f8c0 con 0x565104274860 2024-05-08T21:21:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa65bc82b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565104266070 con 0x565104273ea0 2024-05-08T21:21:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa65bc82b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651042780c0 con 0x565104273540 2024-05-08T21:21:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64e065700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104274860 0x56510427bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64d063700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273ea0 0x565104284410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64d864700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 0x565104273920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64d063700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273ea0 0x565104284410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49270/0 (socket says 172.21.15.33:49270) 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64d864700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 0x565104273920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:45410/0 (socket says 172.21.15.33:45410) 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64d063700 1 -- 172.21.15.33:0/3764585977 learned_addr learned my addr 172.21.15.33:0/3764585977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64e065700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104274860 0x56510427bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44342/0 (socket says 172.21.15.33:44342) 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64d864700 1 -- 172.21.15.33:0/3764585977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273ea0 msgr2=0x565104284410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64d864700 1 --2- 172.21.15.33:0/3764585977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273ea0 0x565104284410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64d864700 1 -- 172.21.15.33:0/3764585977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104274860 msgr2=0x56510427bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64d864700 1 --2- 172.21.15.33:0/3764585977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104274860 0x56510427bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64d864700 1 -- 172.21.15.33:0/3764585977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565103ff0b60 con 0x565104273540 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64d063700 1 --2- 172.21.15.33:0/3764585977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273ea0 0x565104284410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64e065700 1 --2- 172.21.15.33:0/3764585977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104274860 0x56510427bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.662+0000 7fa64d864700 1 --2- 172.21.15.33:0/3764585977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 0x565104273920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa64409b690 tx=0x7fa6440cb6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18e828967fe11a34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.663+0000 7fa63cda4700 1 -- 172.21.15.33:0/3764585977 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64409ee60 con 0x565104273540 2024-05-08T21:21:08.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.663+0000 7fa63cda4700 1 -- 172.21.15.33:0/3764585977 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa644064070 con 0x565104273540 2024-05-08T21:21:08.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.663+0000 7fa63cda4700 1 -- 172.21.15.33:0/3764585977 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440993d0 con 0x565104273540 2024-05-08T21:21:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.663+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3764585977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 msgr2=0x565104273920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.663+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/3764585977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 0x565104273920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa64409b690 tx=0x7fa6440cb6d0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.663+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3764585977 shutdown_connections 2024-05-08T21:21:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.663+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/3764585977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104274860 0x56510427bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.663+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/3764585977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104273540 0x565104273920 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.663+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/3764585977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273ea0 0x565104284410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.663+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3764585977 >> 172.21.15.33:0/3764585977 conn(0x5651041d1210 msgr2=0x565104274c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.663+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3764585977 shutdown_connections 2024-05-08T21:21:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.663+0000 7fa65bc82b80 1 -- 172.21.15.33:0/3764585977 wait complete. 2024-05-08T21:21:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa65bc82b80 1 -- start start 2024-05-08T21:21:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa65bc82b80 1 Processor -- start 2024-05-08T21:21:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa65bc82b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273540 0x56510427f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa65bc82b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273ea0 0x56510427fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa65bc82b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104274860 0x565104282e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa65bc82b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651042833d0 con 0x565104273540 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa65bc82b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565104283540 con 0x565104273ea0 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa65bc82b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x565104283700 con 0x565104274860 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa65bc82b80 10 client.0 _ll_register_callbacks cb 0x565104168b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa64d864700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273540 0x56510427f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa64d864700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273540 0x56510427f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44344/0 (socket says 172.21.15.33:44344) 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa64d864700 1 -- 172.21.15.33:0/2179105530 learned_addr learned my addr 172.21.15.33:0/2179105530 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.664+0000 7fa64d063700 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273ea0 0x56510427fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa64e065700 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104274860 0x565104282e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa64d864700 1 -- 172.21.15.33:0/2179105530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104274860 msgr2=0x565104282e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa64d864700 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104274860 0x565104282e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa64d864700 1 -- 172.21.15.33:0/2179105530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273ea0 msgr2=0x56510427fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa64d864700 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273ea0 0x56510427fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa64d864700 1 -- 172.21.15.33:0/2179105530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651040443b0 con 0x565104273540 2024-05-08T21:21:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa64d864700 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273540 0x56510427f930 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa644019fd0 tx=0x7fa64409ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ee6a4c2d504726 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa6317fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa6317fa700 1 -- 172.21.15.33:0/2179105530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64409ee60 con 0x565104273540 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa6317fa700 1 -- 172.21.15.33:0/2179105530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa6440b3b50 con 0x565104273540 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa65bc82b80 1 -- 172.21.15.33:0/2179105530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa638021070 con 0x565104273540 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa6317fa700 1 -- 172.21.15.33:0/2179105530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa64400b9e0 con 0x565104273540 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa65bc82b80 10 client.16137 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa65bc82b80 1 -- 172.21.15.33:0/2179105530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fa640003a40 con 0x565104273540 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.665+0000 7fa6317fa700 1 -- 172.21.15.33:0/2179105530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa638021070 con 0x565104273540 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa6317fa700 1 -- 172.21.15.33:0/2179105530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa644096e20 con 0x565104273540 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa65bc82b80 10 client.16137 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa65bc82b80 10 client.16137 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa65bc82b80 20 client.16137 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5651042dee90 0x5651042e12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa65bc82b80 4 client.16137 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa65bc82b80 1 -- 172.21.15.33:0/2179105530 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5651041bbec0 con 0x5651042dee90 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa64d063700 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5651042dee90 0x5651042e12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa64d063700 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5651042dee90 0x5651042e12b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a367379d8d4e60d4 server_cookie=744064fd8e4ca944 in_seq=0 out_seq=0 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa6317fa700 10 client.16137 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa6317fa700 1 -- 172.21.15.33:0/2179105530 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5651041bbec0 con 0x5651042dee90 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa6317fa700 10 client.16137 handle_command_reply: tid=0 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.666+0000 7fa65bc82b80 1 client.16137 shutdown 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa65bc82b80 1 -- 172.21.15.33:0/2179105530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273540 msgr2=0x56510427f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273540 0x56510427f930 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa644019fd0 tx=0x7fa64409ac20 comp rx=0 tx=0).stop 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa65bc82b80 1 -- 172.21.15.33:0/2179105530 shutdown_connections 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5651042dee90 0x5651042e12b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565104274860 0x565104282e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565104273540 0x56510427f930 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa65bc82b80 1 --2- 172.21.15.33:0/2179105530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565104273ea0 0x56510427fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa6317fa700 0 client.16137 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:21:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa65bc82b80 1 -- 172.21.15.33:0/2179105530 >> 172.21.15.33:0/2179105530 conn(0x5651041d1210 msgr2=0x56510427ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa65bc82b80 1 -- 172.21.15.33:0/2179105530 shutdown_connections 2024-05-08T21:21:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa65bc82b80 1 -- 172.21.15.33:0/2179105530 wait complete. 2024-05-08T21:21:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa65bc82b80 20 client.16137 trim_cache size 0 max 16384 2024-05-08T21:21:08.676 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:21:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa64fac8700 1 -- 172.21.15.33:0/249218046 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa6340a1120 msgr2=0x7fa6340a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa64fac8700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa6340a1120 0x7fa6340a35e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa648198180 tx=0x7fa638009260 comp rx=0 tx=0).stop 2024-05-08T21:21:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa64fac8700 1 -- 172.21.15.33:0/249218046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa648101940 msgr2=0x7fa648196cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa64fac8700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa648101940 0x7fa648196cd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa644004710 tx=0x7fa644009580 comp rx=0 tx=0).stop 2024-05-08T21:21:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa64fac8700 1 -- 172.21.15.33:0/249218046 shutdown_connections 2024-05-08T21:21:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa64fac8700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa6340a1120 0x7fa6340a35e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa64fac8700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa648106ba0 0x7fa64819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa64fac8700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa648101940 0x7fa648196cd0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.667+0000 7fa64fac8700 1 --2- 172.21.15.33:0/249218046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa648104270 0x7fa648197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.668+0000 7fa64fac8700 1 -- 172.21.15.33:0/249218046 >> 172.21.15.33:0/249218046 conn(0x7fa6480fb290 msgr2=0x7fa648103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.668+0000 7fa64fac8700 1 -- 172.21.15.33:0/249218046 shutdown_connections 2024-05-08T21:21:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.668+0000 7fa64fac8700 1 -- 172.21.15.33:0/249218046 wait complete. 2024-05-08T21:21:08.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:21:08.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.828+0000 7f9ffcac1700 1 -- 172.21.15.33:0/1360977353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff8100f50 msgr2=0x7f9ff81013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.828+0000 7f9ffcac1700 1 --2- 172.21.15.33:0/1360977353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff8100f50 0x7f9ff81013b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0099c0 tx=0x7f9fec009cd0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.828+0000 7f9ffcac1700 1 -- 172.21.15.33:0/1360977353 shutdown_connections 2024-05-08T21:21:08.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.828+0000 7f9ffcac1700 1 --2- 172.21.15.33:0/1360977353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff81018f0 0x7f9ff81081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.828+0000 7f9ffcac1700 1 --2- 172.21.15.33:0/1360977353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff80ffd60 0x7f9ff8100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.828+0000 7f9ffcac1700 1 --2- 172.21.15.33:0/1360977353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff8100f50 0x7f9ff81013b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.828+0000 7f9ffcac1700 1 -- 172.21.15.33:0/1360977353 >> 172.21.15.33:0/1360977353 conn(0x7f9ff80fb290 msgr2=0x7f9ff80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.828+0000 7f9ffcac1700 1 -- 172.21.15.33:0/1360977353 shutdown_connections 2024-05-08T21:21:08.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.828+0000 7f9ffcac1700 1 -- 172.21.15.33:0/1360977353 wait complete. 2024-05-08T21:21:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.829+0000 7f9ffcac1700 1 Processor -- start 2024-05-08T21:21:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.829+0000 7f9ffcac1700 1 -- start start 2024-05-08T21:21:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.829+0000 7f9ffcac1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff80ffd60 0x7f9ff8195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.829+0000 7f9ffcac1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff8100f50 0x7f9ff8196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.829+0000 7f9ffcac1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81018f0 0x7f9ff819d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.829+0000 7f9ffcac1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff819d770 con 0x7f9ff80ffd60 2024-05-08T21:21:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.829+0000 7f9ffcac1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff819d8e0 con 0x7f9ff8100f50 2024-05-08T21:21:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.829+0000 7f9ffcac1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff819daa0 con 0x7f9ff81018f0 2024-05-08T21:21:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.829+0000 7f9ff659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff80ffd60 0x7f9ff8195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.829+0000 7f9ff5d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff8100f50 0x7f9ff8196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.829+0000 7f9ff659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff80ffd60 0x7f9ff8195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44362/0 (socket says 172.21.15.33:44362) 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.829+0000 7f9ff659c700 1 -- 172.21.15.33:0/3774535913 learned_addr learned my addr 172.21.15.33:0/3774535913 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9ff6d9d700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81018f0 0x7f9ff819d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9ff5d9b700 1 -- 172.21.15.33:0/3774535913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81018f0 msgr2=0x7f9ff819d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81018f0 0x7f9ff819d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9ff5d9b700 1 -- 172.21.15.33:0/3774535913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff80ffd60 msgr2=0x7f9ff8195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff80ffd60 0x7f9ff8195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9ff5d9b700 1 -- 172.21.15.33:0/3774535913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff0009c40 con 0x7f9ff8100f50 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9ff659c700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff80ffd60 0x7f9ff8195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff8100f50 0x7f9ff8196210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0098e0 tx=0x7f9fec00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9fe77fe700 1 -- 172.21.15.33:0/3774535913 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec01d070 con 0x7f9ff8100f50 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9ffcac1700 1 -- 172.21.15.33:0/3774535913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fec0095f0 con 0x7f9ff8100f50 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9fe77fe700 1 -- 172.21.15.33:0/3774535913 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fec0054b0 con 0x7f9ff8100f50 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9fe77fe700 1 -- 172.21.15.33:0/3774535913 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec004a90 con 0x7f9ff8100f50 2024-05-08T21:21:08.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.830+0000 7f9ffcac1700 1 -- 172.21.15.33:0/3774535913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff819dfb0 con 0x7f9ff8100f50 2024-05-08T21:21:08.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.833+0000 7f9fe77fe700 1 -- 172.21.15.33:0/3774535913 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9fec00fb80 con 0x7f9ff8100f50 2024-05-08T21:21:08.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.834+0000 7f9fe77fe700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9fe00a11a0 0x7f9fe00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.834+0000 7f9fe77fe700 1 -- 172.21.15.33:0/3774535913 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9fec0cd560 con 0x7f9ff8100f50 2024-05-08T21:21:08.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.834+0000 7f9ff659c700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9fe00a11a0 0x7f9fe00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.835+0000 7f9ff659c700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9fe00a11a0 0x7f9fe00a3660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8197100 tx=0x7f9fe8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.835+0000 7fa008c7bb80 1 Processor -- start 2024-05-08T21:21:08.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.835+0000 7fa008c7bb80 1 -- start start 2024-05-08T21:21:08.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.835+0000 7fa008c7bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4e750 0x55d77cb4eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.835+0000 7fa008c7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb4f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.835+0000 7fa008c7bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4fa70 0x55d77cb4fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.835+0000 7fa008c7bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77caa3150 con 0x55d77cb4fa70 2024-05-08T21:21:08.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.835+0000 7fa008c7bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77caa0b10 con 0x55d77cb4e750 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.835+0000 7fa008c7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77ca9c400 con 0x55d77cb4f0b0 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.835+0000 7f9ff659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4e750 0x55d77cb4eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.835+0000 7f9ff5d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb4f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9ff6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4fa70 0x55d77cb4fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9ff659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4e750 0x55d77cb4eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49296/0 (socket says 172.21.15.33:49296) 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9ff5d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb4f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:45450/0 (socket says 172.21.15.33:45450) 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9ff659c700 1 -- 172.21.15.33:0/1157233611 learned_addr learned my addr 172.21.15.33:0/1157233611 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9ff5d9b700 1 -- 172.21.15.33:0/1157233611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4e750 msgr2=0x55d77cb4eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/1157233611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4e750 0x55d77cb4eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9ff5d9b700 1 -- 172.21.15.33:0/1157233611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4fa70 msgr2=0x55d77cb4fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/1157233611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4fa70 0x55d77cb4fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9ff5d9b700 1 -- 172.21.15.33:0/1157233611 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d77c91d3b0 con 0x55d77cb4f0b0 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/1157233611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb4f530 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0cacb0 tx=0x7f9fec09a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e50b291deb47b81a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9fe4ff9700 1 -- 172.21.15.33:0/1157233611 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec09adc0 con 0x55d77cb4f0b0 2024-05-08T21:21:08.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9fe4ff9700 1 -- 172.21.15.33:0/1157233611 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fec09a6c0 con 0x55d77cb4f0b0 2024-05-08T21:21:08.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.836+0000 7f9fe4ff9700 1 -- 172.21.15.33:0/1157233611 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0cd2e0 con 0x55d77cb4f0b0 2024-05-08T21:21:08.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.837+0000 7fa008c7bb80 1 -- 172.21.15.33:0/1157233611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 msgr2=0x55d77cb4f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.837+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/1157233611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb4f530 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0cacb0 tx=0x7f9fec09a250 comp rx=0 tx=0).stop 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.837+0000 7fa008c7bb80 1 -- 172.21.15.33:0/1157233611 shutdown_connections 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.837+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/1157233611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4fa70 0x55d77cb4fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.837+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/1157233611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4e750 0x55d77cb4eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.837+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/1157233611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb4f530 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.837+0000 7fa008c7bb80 1 -- 172.21.15.33:0/1157233611 >> 172.21.15.33:0/1157233611 conn(0x55d77caaa210 msgr2=0x55d77cb46de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.837+0000 7fa008c7bb80 1 -- 172.21.15.33:0/1157233611 shutdown_connections 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.837+0000 7fa008c7bb80 1 -- 172.21.15.33:0/1157233611 wait complete. 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.837+0000 7fa008c7bb80 1 -- start start 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7fa008c7bb80 1 Processor -- start 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7fa008c7bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 0x55d77cb57d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7fa008c7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb58240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7fa008c7bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4fa70 0x55d77cb5f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7fa008c7bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77ca38560 con 0x55d77cb4e750 2024-05-08T21:21:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7fa008c7bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77ca38200 con 0x55d77cb4fa70 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7fa008c7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77c91f6a0 con 0x55d77cb4f0b0 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7fa008c7bb80 10 client.0 _ll_register_callbacks cb 0x55d77ca41b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7f9ff659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 0x55d77cb57d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7f9ff5d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb58240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7f9ff659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 0x55d77cb57d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44386/0 (socket says 172.21.15.33:44386) 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7f9ff659c700 1 -- 172.21.15.33:0/4211096934 learned_addr learned my addr 172.21.15.33:0/4211096934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7f9ff6d9d700 1 --2- 172.21.15.33:0/4211096934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4fa70 0x55d77cb5f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7f9ff5d9b700 1 -- 172.21.15.33:0/4211096934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4fa70 msgr2=0x55d77cb5f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/4211096934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4fa70 0x55d77cb5f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7f9ff5d9b700 1 -- 172.21.15.33:0/4211096934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 msgr2=0x55d77cb57d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/4211096934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 0x55d77cb57d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7f9ff5d9b700 1 -- 172.21.15.33:0/4211096934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff819dfb0 con 0x55d77cb4f0b0 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/4211096934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb58240 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55d77cb56890 tx=0x7f9fec0092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eeb8a6537e8095a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.838+0000 7f9fddffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7f9fddffb700 1 -- 172.21.15.33:0/4211096934 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec09e4f0 con 0x55d77cb4f0b0 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7f9fddffb700 1 -- 172.21.15.33:0/4211096934 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9fec0a2ab0 con 0x55d77cb4f0b0 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7fa008c7bb80 1 -- 172.21.15.33:0/4211096934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d77c9f5270 con 0x55d77cb4f0b0 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7f9fddffb700 1 -- 172.21.15.33:0/4211096934 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fec0cd2e0 con 0x55d77cb4f0b0 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7fa008c7bb80 10 client.26035 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7fa008c7bb80 1 -- 172.21.15.33:0/4211096934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d77c8c9b60 con 0x55d77cb4f0b0 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7f9fddffb700 1 -- 172.21.15.33:0/4211096934 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d77c9f5270 con 0x55d77cb4f0b0 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7f9fddffb700 1 -- 172.21.15.33:0/4211096934 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9fec0a2620 con 0x55d77cb4f0b0 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7fa008c7bb80 10 client.26035 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7fa008c7bb80 10 client.26035 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7fa008c7bb80 20 client.26035 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/4211096934 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55d77cb635e0 0x55d77cb659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7fa008c7bb80 4 client.26035 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.839+0000 7fa008c7bb80 1 -- 172.21.15.33:0/4211096934 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d77cb1b810 con 0x55d77cb635e0 2024-05-08T21:21:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.840+0000 7f9ff659c700 1 --2- 172.21.15.33:0/4211096934 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55d77cb635e0 0x55d77cb659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.840+0000 7f9ff659c700 1 --2- 172.21.15.33:0/4211096934 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55d77cb635e0 0x55d77cb659d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=308f7db6f6cfb102 server_cookie=2518c7fc67806c31 in_seq=0 out_seq=0 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.840+0000 7f9fddffb700 10 client.26035 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.841+0000 7f9fddffb700 1 -- 172.21.15.33:0/4211096934 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d77cb1b810 con 0x55d77cb635e0 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.841+0000 7f9fddffb700 10 client.26035 handle_command_reply: tid=0 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.841+0000 7fa008c7bb80 1 client.26035 shutdown 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.841+0000 7fa008c7bb80 1 -- 172.21.15.33:0/4211096934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 msgr2=0x55d77cb58240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.841+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/4211096934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb58240 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55d77cb56890 tx=0x7f9fec0092d0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.841+0000 7fa008c7bb80 1 -- 172.21.15.33:0/4211096934 shutdown_connections 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.841+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/4211096934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4fa70 0x55d77cb5f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.841+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/4211096934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 0x55d77cb57d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.841+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/4211096934 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55d77cb635e0 0x55d77cb659d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.841+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/4211096934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb58240 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.841+0000 7f9fddffb700 0 client.26035 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.841+0000 7fa008c7bb80 1 -- 172.21.15.33:0/4211096934 >> 172.21.15.33:0/4211096934 conn(0x55d77caaa210 msgr2=0x55d77cb465a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.842+0000 7fa008c7bb80 1 -- 172.21.15.33:0/4211096934 shutdown_connections 2024-05-08T21:21:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.842+0000 7fa008c7bb80 1 -- 172.21.15.33:0/4211096934 wait complete. 2024-05-08T21:21:08.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.842+0000 7fa008c7bb80 20 client.26035 trim_cache size 0 max 16384 2024-05-08T21:21:08.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7fa008c7bb80 1 Processor -- start 2024-05-08T21:21:08.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7fa008c7bb80 1 -- start start 2024-05-08T21:21:08.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7fa008c7bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4e750 0x55d77cb659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7fa008c7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb57280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7fa008c7bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4fa70 0x55d77cb5f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7fa008c7bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77cb60400 con 0x55d77cb4fa70 2024-05-08T21:21:08.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7fa008c7bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77cb57870 con 0x55d77cb4e750 2024-05-08T21:21:08.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7fa008c7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77cb43df0 con 0x55d77cb4f0b0 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7f9ff6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4fa70 0x55d77cb5f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7f9ff5d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb57280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7f9ff659c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4e750 0x55d77cb659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7f9ff6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4fa70 0x55d77cb5f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44396/0 (socket says 172.21.15.33:44396) 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7f9ff5d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb57280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:45480/0 (socket says 172.21.15.33:45480) 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.856+0000 7f9ff6d9d700 1 -- 172.21.15.33:0/1164629155 learned_addr learned my addr 172.21.15.33:0/1164629155 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.857+0000 7f9ff5d9b700 1 -- 172.21.15.33:0/1164629155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4e750 msgr2=0x55d77cb659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.857+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/1164629155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4e750 0x55d77cb659d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.857+0000 7f9ff5d9b700 1 -- 172.21.15.33:0/1164629155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4fa70 msgr2=0x55d77cb5f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.857+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/1164629155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4fa70 0x55d77cb5f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.857+0000 7f9ff5d9b700 1 -- 172.21.15.33:0/1164629155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d77ca62560 con 0x55d77cb4f0b0 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.857+0000 7f9ff6d9d700 1 --2- 172.21.15.33:0/1164629155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4fa70 0x55d77cb5f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.857+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/1164629155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb57280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9fec09a3c0 tx=0x7f9fec0963c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d444ec94a0ffcaa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.857+0000 7f9fe57fa700 1 -- 172.21.15.33:0/1164629155 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec099e60 con 0x55d77cb4f0b0 2024-05-08T21:21:08.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.857+0000 7f9fe57fa700 1 -- 172.21.15.33:0/1164629155 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fec09a200 con 0x55d77cb4f0b0 2024-05-08T21:21:08.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.857+0000 7f9fe57fa700 1 -- 172.21.15.33:0/1164629155 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0a2620 con 0x55d77cb4f0b0 2024-05-08T21:21:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 -- 172.21.15.33:0/1164629155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 msgr2=0x55d77cb57280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/1164629155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb57280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9fec09a3c0 tx=0x7f9fec0963c0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 -- 172.21.15.33:0/1164629155 shutdown_connections 2024-05-08T21:21:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/1164629155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4fa70 0x55d77cb5f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/1164629155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4e750 0x55d77cb659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/1164629155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4f0b0 0x55d77cb57280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 -- 172.21.15.33:0/1164629155 >> 172.21.15.33:0/1164629155 conn(0x55d77caaa210 msgr2=0x55d77cb43210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 -- 172.21.15.33:0/1164629155 shutdown_connections 2024-05-08T21:21:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 -- 172.21.15.33:0/1164629155 wait complete. 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 -- start start 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 Processor -- start 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 0x55d77cb5a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4f0b0 0x55d77cb5a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4fa70 0x55d77cb5d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77cb5de70 con 0x55d77cb4e750 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77cb5dfe0 con 0x55d77cb4f0b0 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d77cb5e1a0 con 0x55d77cb4fa70 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.858+0000 7fa008c7bb80 10 client.0 _ll_register_callbacks cb 0x55d77ca41b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 0x55d77cb5a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff6d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4fa70 0x55d77cb5d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 0x55d77cb5a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44406/0 (socket says 172.21.15.33:44406) 2024-05-08T21:21:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff659c700 1 -- 172.21.15.33:0/3561772177 learned_addr learned my addr 172.21.15.33:0/3561772177 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4f0b0 0x55d77cb5a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff659c700 1 -- 172.21.15.33:0/3561772177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4fa70 msgr2=0x55d77cb5d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff659c700 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4fa70 0x55d77cb5d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff659c700 1 -- 172.21.15.33:0/3561772177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4f0b0 msgr2=0x55d77cb5a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff659c700 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4f0b0 0x55d77cb5a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff659c700 1 -- 172.21.15.33:0/3561772177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff000e5d0 con 0x55d77cb4e750 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff6d9d700 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4fa70 0x55d77cb5d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4f0b0 0x55d77cb5a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9ff659c700 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 0x55d77cb5a470 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x55d77cb66250 tx=0x7f9fe8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944759f1dfa93998 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9fddffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9fddffb700 1 -- 172.21.15.33:0/3561772177 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8024040 con 0x55d77cb4e750 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7f9fddffb700 1 -- 172.21.15.33:0/3561772177 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9fe80295b0 con 0x55d77cb4e750 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.859+0000 7fa008c7bb80 1 -- 172.21.15.33:0/3561772177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fec09a6c0 con 0x55d77cb4e750 2024-05-08T21:21:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.860+0000 7f9fddffb700 1 -- 172.21.15.33:0/3561772177 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe8029770 con 0x55d77cb4e750 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.860+0000 7fa008c7bb80 10 client.16155 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.860+0000 7fa008c7bb80 1 -- 172.21.15.33:0/3561772177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d77c8c9b60 con 0x55d77cb4e750 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.860+0000 7f9fddffb700 1 -- 172.21.15.33:0/3561772177 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fec09a6c0 con 0x55d77cb4e750 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.860+0000 7f9fddffb700 1 -- 172.21.15.33:0/3561772177 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9fe80051b0 con 0x55d77cb4e750 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.860+0000 7fa008c7bb80 10 client.16155 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.860+0000 7fa008c7bb80 10 client.16155 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.860+0000 7fa008c7bb80 20 client.16155 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.860+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55d77cbb9de0 0x55d77cbbc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.860+0000 7fa008c7bb80 4 client.16155 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.860+0000 7fa008c7bb80 1 -- 172.21.15.33:0/3561772177 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d77ca94ec0 con 0x55d77cbb9de0 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.860+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55d77cbb9de0 0x55d77cbbc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.861+0000 7f9fddffb700 10 client.16155 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:21:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.861+0000 7f9ff5d9b700 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55d77cbb9de0 0x55d77cbbc200 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b6700b781256f7c server_cookie=a4252f867a17182 in_seq=0 out_seq=0 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7f9fddffb700 1 -- 172.21.15.33:0/3561772177 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d77ca94ec0 con 0x55d77cbb9de0 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7f9fddffb700 10 client.16155 handle_command_reply: tid=0 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7fa008c7bb80 1 client.16155 shutdown 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7fa008c7bb80 1 -- 172.21.15.33:0/3561772177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 msgr2=0x55d77cb5a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 0x55d77cb5a470 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x55d77cb66250 tx=0x7f9fe8004750 comp rx=0 tx=0).stop 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7fa008c7bb80 1 -- 172.21.15.33:0/3561772177 shutdown_connections 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55d77cbb9de0 0x55d77cbbc200 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d77cb4fa70 0x55d77cb5d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d77cb4e750 0x55d77cb5a470 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7f9fddffb700 0 client.16155 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7fa008c7bb80 1 --2- 172.21.15.33:0/3561772177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d77cb4f0b0 0x55d77cb5a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7fa008c7bb80 1 -- 172.21.15.33:0/3561772177 >> 172.21.15.33:0/3561772177 conn(0x55d77caaa210 msgr2=0x55d77cb4fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7fa008c7bb80 1 -- 172.21.15.33:0/3561772177 shutdown_connections 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7fa008c7bb80 1 -- 172.21.15.33:0/3561772177 wait complete. 2024-05-08T21:21:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7fa008c7bb80 20 client.16155 trim_cache size 0 max 16384 2024-05-08T21:21:08.870 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:21:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7f9ffcac1700 1 -- 172.21.15.33:0/3774535913 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9fe00a11a0 msgr2=0x7f9fe00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.862+0000 7f9ffcac1700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9fe00a11a0 0x7f9fe00a3660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8197100 tx=0x7f9fe8009260 comp rx=0 tx=0).stop 2024-05-08T21:21:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.863+0000 7f9ffcac1700 1 -- 172.21.15.33:0/3774535913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff8100f50 msgr2=0x7f9ff8196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.863+0000 7f9ffcac1700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff8100f50 0x7f9ff8196210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0098e0 tx=0x7f9fec00f690 comp rx=0 tx=0).stop 2024-05-08T21:21:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.863+0000 7f9ffcac1700 1 -- 172.21.15.33:0/3774535913 shutdown_connections 2024-05-08T21:21:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.863+0000 7f9ffcac1700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff81018f0 0x7f9ff819d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.863+0000 7f9ffcac1700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ff80ffd60 0x7f9ff8195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.863+0000 7f9ffcac1700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9fe00a11a0 0x7f9fe00a3660 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.863+0000 7f9ffcac1700 1 --2- 172.21.15.33:0/3774535913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff8100f50 0x7f9ff8196210 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.863+0000 7f9ffcac1700 1 -- 172.21.15.33:0/3774535913 >> 172.21.15.33:0/3774535913 conn(0x7f9ff80fb290 msgr2=0x7f9ff81043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.863+0000 7f9ffcac1700 1 -- 172.21.15.33:0/3774535913 shutdown_connections 2024-05-08T21:21:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:08.863+0000 7f9ffcac1700 1 -- 172.21.15.33:0/3774535913 wait complete. 2024-05-08T21:21:08.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:21:09.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.033+0000 7fb33bca4700 1 -- 172.21.15.33:0/366142813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb334101fc0 msgr2=0x7fb334102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.033+0000 7fb33bca4700 1 --2- 172.21.15.33:0/366142813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb334101fc0 0x7fb334102440 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb324009990 tx=0x7fb324009ca0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.033+0000 7fb33bca4700 1 -- 172.21.15.33:0/366142813 shutdown_connections 2024-05-08T21:21:09.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.033+0000 7fb33bca4700 1 --2- 172.21.15.33:0/366142813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb334102980 0x7fb3341092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.033+0000 7fb33bca4700 1 --2- 172.21.15.33:0/366142813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334100e70 0x7fb334101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.033+0000 7fb33bca4700 1 --2- 172.21.15.33:0/366142813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb334101fc0 0x7fb334102440 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.033+0000 7fb33bca4700 1 -- 172.21.15.33:0/366142813 >> 172.21.15.33:0/366142813 conn(0x7fb3340fc400 msgr2=0x7fb3340fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.033+0000 7fb33bca4700 1 -- 172.21.15.33:0/366142813 shutdown_connections 2024-05-08T21:21:09.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.034+0000 7fb33bca4700 1 -- 172.21.15.33:0/366142813 wait complete. 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.034+0000 7fb33bca4700 1 Processor -- start 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.034+0000 7fb33bca4700 1 -- start start 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.034+0000 7fb33bca4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334100e70 0x7fb334196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.034+0000 7fb33bca4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb334101fc0 0x7fb334197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.034+0000 7fb33bca4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb334102980 0x7fb33419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.034+0000 7fb33bca4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33419e830 con 0x7fb334100e70 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.034+0000 7fb33bca4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33419e9a0 con 0x7fb334101fc0 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.034+0000 7fb33bca4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33419eb60 con 0x7fb334102980 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb339a40700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334100e70 0x7fb334196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb339a40700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334100e70 0x7fb334196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44420/0 (socket says 172.21.15.33:44420) 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb339a40700 1 -- 172.21.15.33:0/116729478 learned_addr learned my addr 172.21.15.33:0/116729478 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb33923f700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb334101fc0 0x7fb334197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb33a241700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb334102980 0x7fb33419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb339a40700 1 -- 172.21.15.33:0/116729478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb334102980 msgr2=0x7fb33419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb339a40700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb334102980 0x7fb33419e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb339a40700 1 -- 172.21.15.33:0/116729478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb334101fc0 msgr2=0x7fb334197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb339a40700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb334101fc0 0x7fb334197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb339a40700 1 -- 172.21.15.33:0/116729478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb330009520 con 0x7fb334100e70 2024-05-08T21:21:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb339a40700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334100e70 0x7fb334196d40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb33000eb30 tx=0x7fb33000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb32affd700 1 -- 172.21.15.33:0/116729478 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330009880 con 0x7fb334100e70 2024-05-08T21:21:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb33bca4700 1 -- 172.21.15.33:0/116729478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3240095f0 con 0x7fb334100e70 2024-05-08T21:21:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb32affd700 1 -- 172.21.15.33:0/116729478 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3300044b0 con 0x7fb334100e70 2024-05-08T21:21:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb32affd700 1 -- 172.21.15.33:0/116729478 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330003ce0 con 0x7fb334100e70 2024-05-08T21:21:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.035+0000 7fb33bca4700 1 -- 172.21.15.33:0/116729478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33419f1c0 con 0x7fb334100e70 2024-05-08T21:21:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.038+0000 7fb32affd700 1 -- 172.21.15.33:0/116729478 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb330003e40 con 0x7fb334100e70 2024-05-08T21:21:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.038+0000 7fb32affd700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb3200a1120 0x7fb3200a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.039+0000 7fb32affd700 1 -- 172.21.15.33:0/116729478 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fb330014070 con 0x7fb334100e70 2024-05-08T21:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.039+0000 7fb33923f700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb3200a1120 0x7fb3200a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.039+0000 7fb33923f700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb3200a1120 0x7fb3200a35e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb324006010 tx=0x7fb32400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.039+0000 7fb347e5eb80 1 Processor -- start 2024-05-08T21:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.039+0000 7fb347e5eb80 1 -- start start 2024-05-08T21:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb347e5eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x55771663b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb347e5eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x55771663c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb347e5eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x55771663cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb347e5eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557716592150 con 0x55771663c860 2024-05-08T21:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb347e5eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771658fb10 con 0x55771663b540 2024-05-08T21:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb347e5eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771658b400 con 0x55771663bea0 2024-05-08T21:21:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb339a40700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x55771663b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb339a40700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x55771663b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49338/0 (socket says 172.21.15.33:49338) 2024-05-08T21:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb339a40700 1 -- 172.21.15.33:0/2054841262 learned_addr learned my addr 172.21.15.33:0/2054841262 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb33a241700 1 --2- 172.21.15.33:0/2054841262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x55771663cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb33923f700 1 --2- 172.21.15.33:0/2054841262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x55771663c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb339a40700 1 -- 172.21.15.33:0/2054841262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 msgr2=0x55771663c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb339a40700 1 --2- 172.21.15.33:0/2054841262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x55771663c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb339a40700 1 -- 172.21.15.33:0/2054841262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 msgr2=0x55771663cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb339a40700 1 --2- 172.21.15.33:0/2054841262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x55771663cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb339a40700 1 -- 172.21.15.33:0/2054841262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55771640c3b0 con 0x55771663b540 2024-05-08T21:21:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb33923f700 1 --2- 172.21.15.33:0/2054841262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x55771663c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.040+0000 7fb339a40700 1 --2- 172.21.15.33:0/2054841262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x55771663b960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb3300a2130 tx=0x7fb330014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8ac3ca064d2621a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.041+0000 7fb31ffff700 1 -- 172.21.15.33:0/2054841262 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33009a8e0 con 0x55771663b540 2024-05-08T21:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.041+0000 7fb31ffff700 1 -- 172.21.15.33:0/2054841262 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb33000cbd0 con 0x55771663b540 2024-05-08T21:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.041+0000 7fb31ffff700 1 -- 172.21.15.33:0/2054841262 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33001fde0 con 0x55771663b540 2024-05-08T21:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.041+0000 7fb347e5eb80 1 -- 172.21.15.33:0/2054841262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 msgr2=0x55771663b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.041+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/2054841262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x55771663b960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb3300a2130 tx=0x7fb330014000 comp rx=0 tx=0).stop 2024-05-08T21:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.041+0000 7fb347e5eb80 1 -- 172.21.15.33:0/2054841262 shutdown_connections 2024-05-08T21:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.041+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/2054841262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x55771663cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.041+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/2054841262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x55771663b960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.041+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/2054841262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x55771663c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.041+0000 7fb347e5eb80 1 -- 172.21.15.33:0/2054841262 >> 172.21.15.33:0/2054841262 conn(0x557716599210 msgr2=0x557716635de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.042+0000 7fb347e5eb80 1 -- 172.21.15.33:0/2054841262 shutdown_connections 2024-05-08T21:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.042+0000 7fb347e5eb80 1 -- 172.21.15.33:0/2054841262 wait complete. 2024-05-08T21:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.042+0000 7fb347e5eb80 1 -- start start 2024-05-08T21:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.042+0000 7fb347e5eb80 1 Processor -- start 2024-05-08T21:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb347e5eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x55771664d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb347e5eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x55771664dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb347e5eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x557716654c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb347e5eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557716527560 con 0x55771663c860 2024-05-08T21:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb347e5eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557716527200 con 0x55771663b540 2024-05-08T21:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb347e5eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771640e6a0 con 0x55771663bea0 2024-05-08T21:21:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb347e5eb80 10 client.0 _ll_register_callbacks cb 0x557716530b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb33923f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x55771664dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb33923f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x55771664dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:45514/0 (socket says 172.21.15.33:45514) 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb33923f700 1 -- 172.21.15.33:0/691378161 learned_addr learned my addr 172.21.15.33:0/691378161 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb339a40700 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x55771664d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb33a241700 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x557716654c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb33923f700 1 -- 172.21.15.33:0/691378161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 msgr2=0x55771664d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb33923f700 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x55771664d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb33923f700 1 -- 172.21.15.33:0/691378161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 msgr2=0x557716654c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb33923f700 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x557716654c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb33923f700 1 -- 172.21.15.33:0/691378161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33419f1c0 con 0x55771663bea0 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb33923f700 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x55771664dc30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb324009990 tx=0x7fb32401a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f443fb75671d83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb31d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.043+0000 7fb31d7fa700 1 -- 172.21.15.33:0/691378161 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32400bb70 con 0x55771663bea0 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb347e5eb80 1 -- 172.21.15.33:0/691378161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577164e4270 con 0x55771663bea0 2024-05-08T21:21:09.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb31d7fa700 1 -- 172.21.15.33:0/691378161 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fb32400bcd0 con 0x55771663bea0 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb31d7fa700 1 -- 172.21.15.33:0/691378161 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb32400be90 con 0x55771663bea0 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb347e5eb80 10 client.26053 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb347e5eb80 1 -- 172.21.15.33:0/691378161 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5577163b8b60 con 0x55771663bea0 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb31d7fa700 1 -- 172.21.15.33:0/691378161 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5577164e4270 con 0x55771663bea0 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb31d7fa700 1 -- 172.21.15.33:0/691378161 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb32400f580 con 0x55771663bea0 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb347e5eb80 10 client.26053 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb347e5eb80 10 client.26053 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb347e5eb80 20 client.26053 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55771666a4b0 0x55771666c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb347e5eb80 4 client.26053 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb347e5eb80 1 -- 172.21.15.33:0/691378161 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55771660a810 con 0x55771666a4b0 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.044+0000 7fb339a40700 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55771666a4b0 0x55771666c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.045+0000 7fb31d7fa700 10 client.26053 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.045+0000 7fb339a40700 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55771666a4b0 0x55771666c8d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0fd2b21d643f2a1 server_cookie=6fcefcef3d487a8 in_seq=0 out_seq=0 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.045+0000 7fb31d7fa700 1 -- 172.21.15.33:0/691378161 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55771660a810 con 0x55771666a4b0 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.045+0000 7fb31d7fa700 10 client.26053 handle_command_reply: tid=0 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.045+0000 7fb347e5eb80 1 client.26053 shutdown 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.045+0000 7fb347e5eb80 1 -- 172.21.15.33:0/691378161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 msgr2=0x55771664dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.045+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x55771664dc30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb324009990 tx=0x7fb32401a050 comp rx=0 tx=0).stop 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.046+0000 7fb347e5eb80 1 -- 172.21.15.33:0/691378161 shutdown_connections 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.046+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55771666a4b0 0x55771666c8d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.046+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x557716654c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.046+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x55771664d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.046+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/691378161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x55771664dc30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.046+0000 7fb31d7fa700 0 client.26053 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.046+0000 7fb347e5eb80 1 -- 172.21.15.33:0/691378161 >> 172.21.15.33:0/691378161 conn(0x557716599210 msgr2=0x5577166355a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.046+0000 7fb347e5eb80 1 -- 172.21.15.33:0/691378161 shutdown_connections 2024-05-08T21:21:09.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.046+0000 7fb347e5eb80 1 -- 172.21.15.33:0/691378161 wait complete. 2024-05-08T21:21:09.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.046+0000 7fb347e5eb80 20 client.26053 trim_cache size 0 max 16384 2024-05-08T21:21:09.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.060+0000 7fb347e5eb80 1 Processor -- start 2024-05-08T21:21:09.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.060+0000 7fb347e5eb80 1 -- start start 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.060+0000 7fb347e5eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663b540 0x55771666c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.060+0000 7fb347e5eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663bea0 0x557716654b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.060+0000 7fb347e5eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663c860 0x55771664cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.060+0000 7fb347e5eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771664d260 con 0x55771663b540 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.060+0000 7fb347e5eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771664e890 con 0x55771663bea0 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.060+0000 7fb347e5eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557716671a80 con 0x55771663c860 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb339a40700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663b540 0x55771666c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb339a40700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663b540 0x55771666c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44454/0 (socket says 172.21.15.33:44454) 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb339a40700 1 -- 172.21.15.33:0/2164876385 learned_addr learned my addr 172.21.15.33:0/2164876385 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb33a241700 1 --2- 172.21.15.33:0/2164876385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663c860 0x55771664cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb33923f700 1 --2- 172.21.15.33:0/2164876385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663bea0 0x557716654b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb339a40700 1 -- 172.21.15.33:0/2164876385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663c860 msgr2=0x55771664cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb339a40700 1 --2- 172.21.15.33:0/2164876385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663c860 0x55771664cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb339a40700 1 -- 172.21.15.33:0/2164876385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663bea0 msgr2=0x557716654b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb339a40700 1 --2- 172.21.15.33:0/2164876385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663bea0 0x557716654b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb339a40700 1 -- 172.21.15.33:0/2164876385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557716551560 con 0x55771663b540 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb339a40700 1 --2- 172.21.15.33:0/2164876385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663b540 0x55771666c8a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb33000c5b0 tx=0x7fb330003720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb1363997fc2c96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb328da4700 1 -- 172.21.15.33:0/2164876385 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3300983a0 con 0x55771663b540 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb328da4700 1 -- 172.21.15.33:0/2164876385 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb330062070 con 0x55771663b540 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.061+0000 7fb328da4700 1 -- 172.21.15.33:0/2164876385 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33009a440 con 0x55771663b540 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.062+0000 7fb347e5eb80 1 -- 172.21.15.33:0/2164876385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663b540 msgr2=0x55771666c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.062+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/2164876385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663b540 0x55771666c8a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb33000c5b0 tx=0x7fb330003720 comp rx=0 tx=0).stop 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.062+0000 7fb347e5eb80 1 -- 172.21.15.33:0/2164876385 shutdown_connections 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.062+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/2164876385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663c860 0x55771664cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.062+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/2164876385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663b540 0x55771666c8a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.062+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/2164876385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663bea0 0x557716654b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.062+0000 7fb347e5eb80 1 -- 172.21.15.33:0/2164876385 >> 172.21.15.33:0/2164876385 conn(0x557716599210 msgr2=0x557716632210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.062+0000 7fb347e5eb80 1 -- 172.21.15.33:0/2164876385 shutdown_connections 2024-05-08T21:21:09.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.062+0000 7fb347e5eb80 1 -- 172.21.15.33:0/2164876385 wait complete. 2024-05-08T21:21:09.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb347e5eb80 1 -- start start 2024-05-08T21:21:09.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb347e5eb80 1 Processor -- start 2024-05-08T21:21:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb347e5eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x557716651f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb347e5eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x557716652490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb347e5eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x5577166533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb347e5eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577166538f0 con 0x55771663c860 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb347e5eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557716653a60 con 0x55771663b540 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb347e5eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557716653c20 con 0x55771663bea0 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb347e5eb80 10 client.0 _ll_register_callbacks cb 0x557716530b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb339a40700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x557716651f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb339a40700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x557716651f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49354/0 (socket says 172.21.15.33:49354) 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb339a40700 1 -- 172.21.15.33:0/3912434505 learned_addr learned my addr 172.21.15.33:0/3912434505 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb33923f700 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x557716652490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.063+0000 7fb33a241700 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x5577166533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb339a40700 1 -- 172.21.15.33:0/3912434505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 msgr2=0x557716652490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb339a40700 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x557716652490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb339a40700 1 -- 172.21.15.33:0/3912434505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 msgr2=0x5577166533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb339a40700 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x5577166533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb339a40700 1 -- 172.21.15.33:0/3912434505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55771640c3b0 con 0x55771663b540 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb33a241700 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x5577166533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb339a40700 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x557716651f50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb330099ee0 tx=0x7fb330014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22103440a36cc262 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb31d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb31d7fa700 1 -- 172.21.15.33:0/3912434505 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330083070 con 0x55771663b540 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb347e5eb80 1 -- 172.21.15.33:0/3912434505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb33000cbd0 con 0x55771663b540 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb31d7fa700 1 -- 172.21.15.33:0/3912434505 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fb3300a2660 con 0x55771663b540 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb31d7fa700 1 -- 172.21.15.33:0/3912434505 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb33001fde0 con 0x55771663b540 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb347e5eb80 10 client.26039 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb347e5eb80 1 -- 172.21.15.33:0/3912434505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557716410cd0 con 0x55771663b540 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.064+0000 7fb31d7fa700 1 -- 172.21.15.33:0/3912434505 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb33000cbd0 con 0x55771663b540 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb31d7fa700 1 -- 172.21.15.33:0/3912434505 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb33009a4d0 con 0x55771663b540 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb347e5eb80 10 client.26039 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:09.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb347e5eb80 10 client.26039 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb347e5eb80 20 client.26039 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5577166c0a20 0x5577166c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb347e5eb80 4 client.26039 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb347e5eb80 1 -- 172.21.15.33:0/3912434505 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557716583ec0 con 0x5577166c0a20 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb33923f700 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5577166c0a20 0x5577166c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb31d7fa700 10 client.26039 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb33923f700 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5577166c0a20 0x5577166c2e10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d9e0b032f52bc04 server_cookie=6022ebcaa8c272e4 in_seq=0 out_seq=0 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb31d7fa700 1 -- 172.21.15.33:0/3912434505 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557716583ec0 con 0x5577166c0a20 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb31d7fa700 10 client.26039 handle_command_reply: tid=0 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb347e5eb80 1 client.26039 shutdown 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb347e5eb80 1 -- 172.21.15.33:0/3912434505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 msgr2=0x557716651f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.065+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x557716651f50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb330099ee0 tx=0x7fb330014000 comp rx=0 tx=0).stop 2024-05-08T21:21:09.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb347e5eb80 1 -- 172.21.15.33:0/3912434505 shutdown_connections 2024-05-08T21:21:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5577166c0a20 0x5577166c2e10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55771663c860 0x5577166533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55771663b540 0x557716651f50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb31d7fa700 0 client.26039 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:21:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb347e5eb80 1 --2- 172.21.15.33:0/3912434505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55771663bea0 0x557716652490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb347e5eb80 1 -- 172.21.15.33:0/3912434505 >> 172.21.15.33:0/3912434505 conn(0x557716599210 msgr2=0x55771663cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb347e5eb80 1 -- 172.21.15.33:0/3912434505 shutdown_connections 2024-05-08T21:21:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb347e5eb80 1 -- 172.21.15.33:0/3912434505 wait complete. 2024-05-08T21:21:09.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb347e5eb80 20 client.26039 trim_cache size 0 max 16384 2024-05-08T21:21:09.074 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:21:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb33bca4700 1 -- 172.21.15.33:0/116729478 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb3200a1120 msgr2=0x7fb3200a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb33bca4700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb3200a1120 0x7fb3200a35e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb324006010 tx=0x7fb32400b610 comp rx=0 tx=0).stop 2024-05-08T21:21:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb33bca4700 1 -- 172.21.15.33:0/116729478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334100e70 msgr2=0x7fb334196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb33bca4700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334100e70 0x7fb334196d40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb33000eb30 tx=0x7fb33000ee40 comp rx=0 tx=0).stop 2024-05-08T21:21:09.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb33bca4700 1 -- 172.21.15.33:0/116729478 shutdown_connections 2024-05-08T21:21:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb33bca4700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb334102980 0x7fb33419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb33bca4700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334100e70 0x7fb334196d40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb33bca4700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb3200a1120 0x7fb3200a35e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb33bca4700 1 --2- 172.21.15.33:0/116729478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb334101fc0 0x7fb334197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.066+0000 7fb33bca4700 1 -- 172.21.15.33:0/116729478 >> 172.21.15.33:0/116729478 conn(0x7fb3340fc400 msgr2=0x7fb334105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.067+0000 7fb33bca4700 1 -- 172.21.15.33:0/116729478 shutdown_connections 2024-05-08T21:21:09.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.067+0000 7fb33bca4700 1 -- 172.21.15.33:0/116729478 wait complete. 2024-05-08T21:21:09.082 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:21:09.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.225+0000 7fad5fe63700 1 -- 172.21.15.33:0/2231149875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad58100f50 msgr2=0x7fad581013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.225+0000 7fad5fe63700 1 --2- 172.21.15.33:0/2231149875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad58100f50 0x7fad581013b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fad480099c0 tx=0x7fad48009cd0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.225+0000 7fad5fe63700 1 -- 172.21.15.33:0/2231149875 shutdown_connections 2024-05-08T21:21:09.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.225+0000 7fad5fe63700 1 --2- 172.21.15.33:0/2231149875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad581018f0 0x7fad581081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.225+0000 7fad5fe63700 1 --2- 172.21.15.33:0/2231149875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad580ffd60 0x7fad58100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.225+0000 7fad5fe63700 1 --2- 172.21.15.33:0/2231149875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad58100f50 0x7fad581013b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.225+0000 7fad5fe63700 1 -- 172.21.15.33:0/2231149875 >> 172.21.15.33:0/2231149875 conn(0x7fad580fb290 msgr2=0x7fad580fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.225+0000 7fad5fe63700 1 -- 172.21.15.33:0/2231149875 shutdown_connections 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.225+0000 7fad5fe63700 1 -- 172.21.15.33:0/2231149875 wait complete. 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5fe63700 1 Processor -- start 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5fe63700 1 -- start start 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5fe63700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad580ffd60 0x7fad58195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5fe63700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad58100f50 0x7fad58196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5fe63700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad581018f0 0x7fad5819d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5fe63700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5819d770 con 0x7fad58100f50 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5fe63700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5819d8e0 con 0x7fad581018f0 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5fe63700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5819daa0 con 0x7fad580ffd60 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5d3fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad58100f50 0x7fad58196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5dbff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad580ffd60 0x7fad58195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5d3fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad58100f50 0x7fad58196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44500/0 (socket says 172.21.15.33:44500) 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5d3fe700 1 -- 172.21.15.33:0/3454721144 learned_addr learned my addr 172.21.15.33:0/3454721144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5e400700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad581018f0 0x7fad5819d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5dbff700 1 -- 172.21.15.33:0/3454721144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad581018f0 msgr2=0x7fad5819d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5dbff700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad581018f0 0x7fad5819d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5dbff700 1 -- 172.21.15.33:0/3454721144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad58100f50 msgr2=0x7fad58196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5dbff700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad58100f50 0x7fad58196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5dbff700 1 -- 172.21.15.33:0/3454721144 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad54009520 con 0x7fad580ffd60 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.226+0000 7fad5d3fe700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad58100f50 0x7fad58196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.227+0000 7fad5dbff700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad580ffd60 0x7fad58195cd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fad5400e9c0 tx=0x7fad5400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.227+0000 7fad4effd700 1 -- 172.21.15.33:0/3454721144 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5400cb70 con 0x7fad580ffd60 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.227+0000 7fad4effd700 1 -- 172.21.15.33:0/3454721144 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad5400ee50 con 0x7fad580ffd60 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.227+0000 7fad4effd700 1 -- 172.21.15.33:0/3454721144 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54018680 con 0x7fad580ffd60 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.227+0000 7fad5fe63700 1 -- 172.21.15.33:0/3454721144 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad50009c40 con 0x7fad580ffd60 2024-05-08T21:21:09.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.227+0000 7fad5fe63700 1 -- 172.21.15.33:0/3454721144 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad480095f0 con 0x7fad580ffd60 2024-05-08T21:21:09.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.229+0000 7fad4effd700 1 -- 172.21.15.33:0/3454721144 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fad54010430 con 0x7fad580ffd60 2024-05-08T21:21:09.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.230+0000 7fad4effd700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad440a11a0 0x7fad440a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.230+0000 7fad4effd700 1 -- 172.21.15.33:0/3454721144 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fad54014070 con 0x7fad580ffd60 2024-05-08T21:21:09.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.230+0000 7fad5d3fe700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad440a11a0 0x7fad440a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.230+0000 7fad5d3fe700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad440a11a0 0x7fad440a3660 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fad48006010 tx=0x7fad48003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad6c01db80 1 Processor -- start 2024-05-08T21:21:09.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad6c01db80 1 -- start start 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad6c01db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2b750 0x55f694b2bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad6c01db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b2c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad6c01db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2ca70 0x55f694b2cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad6c01db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f694a80150 con 0x55f694b2b750 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad6c01db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f694a7db10 con 0x55f694b2ca70 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad6c01db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f694a79400 con 0x55f694b2c0b0 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad5dbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2b750 0x55f694b2bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad5e400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2ca70 0x55f694b2cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad5dbff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2b750 0x55f694b2bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44504/0 (socket says 172.21.15.33:44504) 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad5e400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2ca70 0x55f694b2cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49382/0 (socket says 172.21.15.33:49382) 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad5dbff700 1 -- 172.21.15.33:0/2264049044 learned_addr learned my addr 172.21.15.33:0/2264049044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.231+0000 7fad5d3fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b2c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.232+0000 7fad5dbff700 1 -- 172.21.15.33:0/2264049044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 msgr2=0x55f694b2c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.232+0000 7fad5dbff700 1 --2- 172.21.15.33:0/2264049044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b2c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.232+0000 7fad5dbff700 1 -- 172.21.15.33:0/2264049044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2ca70 msgr2=0x55f694b2cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.232+0000 7fad5dbff700 1 --2- 172.21.15.33:0/2264049044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2ca70 0x55f694b2cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.232+0000 7fad5dbff700 1 -- 172.21.15.33:0/2264049044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6948fa3b0 con 0x55f694b2b750 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.232+0000 7fad5e400700 1 --2- 172.21.15.33:0/2264049044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2ca70 0x55f694b2cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.232+0000 7fad5dbff700 1 --2- 172.21.15.33:0/2264049044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2b750 0x55f694b2bb70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fad54010a10 tx=0x7fad5409a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da553efcbbe31c94 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.232+0000 7fad43fff700 1 -- 172.21.15.33:0/2264049044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5409a800 con 0x55f694b2b750 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.232+0000 7fad43fff700 1 -- 172.21.15.33:0/2264049044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad5409a100 con 0x55f694b2b750 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.232+0000 7fad43fff700 1 -- 172.21.15.33:0/2264049044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54099850 con 0x55f694b2b750 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.233+0000 7fad6c01db80 1 -- 172.21.15.33:0/2264049044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2b750 msgr2=0x55f694b2bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.233+0000 7fad6c01db80 1 --2- 172.21.15.33:0/2264049044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2b750 0x55f694b2bb70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fad54010a10 tx=0x7fad5409a000 comp rx=0 tx=0).stop 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.233+0000 7fad6c01db80 1 -- 172.21.15.33:0/2264049044 shutdown_connections 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.233+0000 7fad6c01db80 1 --2- 172.21.15.33:0/2264049044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2ca70 0x55f694b2cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.233+0000 7fad6c01db80 1 --2- 172.21.15.33:0/2264049044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2b750 0x55f694b2bb70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.233+0000 7fad6c01db80 1 --2- 172.21.15.33:0/2264049044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b2c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.233+0000 7fad6c01db80 1 -- 172.21.15.33:0/2264049044 >> 172.21.15.33:0/2264049044 conn(0x55f694a87210 msgr2=0x55f694b23de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.233+0000 7fad6c01db80 1 -- 172.21.15.33:0/2264049044 shutdown_connections 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.233+0000 7fad6c01db80 1 -- 172.21.15.33:0/2264049044 wait complete. 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.233+0000 7fad6c01db80 1 -- start start 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad6c01db80 1 Processor -- start 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad6c01db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b34fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad6c01db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2c0b0 0x55f694b35500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad6c01db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2ca70 0x55f694b3c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad6c01db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f694a15560 con 0x55f694b2c0b0 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad6c01db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f694a15200 con 0x55f694b2b750 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad6c01db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6948fc6a0 con 0x55f694b2ca70 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad6c01db80 10 client.0 _ll_register_callbacks cb 0x55f694a1eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad5dbff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b34fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad5dbff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b34fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49390/0 (socket says 172.21.15.33:49390) 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad5dbff700 1 -- 172.21.15.33:0/193619267 learned_addr learned my addr 172.21.15.33:0/193619267 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad5e400700 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2ca70 0x55f694b3c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad5d3fe700 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2c0b0 0x55f694b35500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.234+0000 7fad5dbff700 1 -- 172.21.15.33:0/193619267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2ca70 msgr2=0x55f694b3c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad5dbff700 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2ca70 0x55f694b3c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad5dbff700 1 -- 172.21.15.33:0/193619267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2c0b0 msgr2=0x55f694b35500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad5dbff700 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2c0b0 0x55f694b35500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad5dbff700 1 -- 172.21.15.33:0/193619267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5000e330 con 0x55f694b2b750 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad5dbff700 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b34fc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fad5409a090 tx=0x7fad54099f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c1b448173f26d31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad417fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad417fa700 1 -- 172.21.15.33:0/193619267 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5409a800 con 0x55f694b2b750 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad417fa700 1 -- 172.21.15.33:0/193619267 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fad5409df30 con 0x55f694b2b750 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad6c01db80 1 -- 172.21.15.33:0/193619267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6949d2270 con 0x55f694b2b750 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad417fa700 1 -- 172.21.15.33:0/193619267 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad540a22f0 con 0x55f694b2b750 2024-05-08T21:21:09.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad6c01db80 10 client.26051 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad6c01db80 1 -- 172.21.15.33:0/193619267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fad480095f0 con 0x55f694b2b750 2024-05-08T21:21:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.235+0000 7fad417fa700 1 -- 172.21.15.33:0/193619267 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f6949d2270 con 0x55f694b2b750 2024-05-08T21:21:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.236+0000 7fad417fa700 1 -- 172.21.15.33:0/193619267 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fad5409ee50 con 0x55f694b2b750 2024-05-08T21:21:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.236+0000 7fad6c01db80 10 client.26051 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.236+0000 7fad6c01db80 10 client.26051 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:21:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.236+0000 7fad6c01db80 20 client.26051 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.236+0000 7fad6c01db80 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f694b40830 0x55f694b42c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.236+0000 7fad6c01db80 4 client.26051 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.236+0000 7fad6c01db80 1 -- 172.21.15.33:0/193619267 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f694af8810 con 0x55f694b40830 2024-05-08T21:21:09.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.236+0000 7fad5d3fe700 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f694b40830 0x55f694b42c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.236+0000 7fad417fa700 10 client.26051 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:21:09.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.236+0000 7fad5d3fe700 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f694b40830 0x55f694b42c50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b4facfcea8e3845 server_cookie=8c0288c944bcc042 in_seq=0 out_seq=0 2024-05-08T21:21:09.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad417fa700 1 -- 172.21.15.33:0/193619267 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f694af8810 con 0x55f694b40830 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad417fa700 10 client.26051 handle_command_reply: tid=0 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad6c01db80 1 client.26051 shutdown 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad6c01db80 1 -- 172.21.15.33:0/193619267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 msgr2=0x55f694b34fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad6c01db80 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b34fc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fad5409a090 tx=0x7fad54099f60 comp rx=0 tx=0).stop 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad6c01db80 1 -- 172.21.15.33:0/193619267 shutdown_connections 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad6c01db80 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f694b40830 0x55f694b42c50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad6c01db80 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2ca70 0x55f694b3c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad6c01db80 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b34fc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad6c01db80 1 --2- 172.21.15.33:0/193619267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2c0b0 0x55f694b35500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad6c01db80 1 -- 172.21.15.33:0/193619267 >> 172.21.15.33:0/193619267 conn(0x55f694a87210 msgr2=0x55f694b235a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad417fa700 0 client.26051 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad6c01db80 1 -- 172.21.15.33:0/193619267 shutdown_connections 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad6c01db80 1 -- 172.21.15.33:0/193619267 wait complete. 2024-05-08T21:21:09.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.237+0000 7fad6c01db80 20 client.26051 trim_cache size 0 max 16384 2024-05-08T21:21:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.252+0000 7fad6c01db80 1 Processor -- start 2024-05-08T21:21:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad6c01db80 1 -- start start 2024-05-08T21:21:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad6c01db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b42c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad6c01db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b3c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad6c01db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2ca70 0x55f694b3cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad6c01db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f694b1e800 con 0x55f694b2ca70 2024-05-08T21:21:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad6c01db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f694b25570 con 0x55f694b2b750 2024-05-08T21:21:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad6c01db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f694b32c30 con 0x55f694b2c0b0 2024-05-08T21:21:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad5dbff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b42c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad5e400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2ca70 0x55f694b3cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad5d3fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b3c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad5dbff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b42c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49392/0 (socket says 172.21.15.33:49392) 2024-05-08T21:21:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad5dbff700 1 -- 172.21.15.33:0/1770070414 learned_addr learned my addr 172.21.15.33:0/1770070414 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad5d3fe700 1 -- 172.21.15.33:0/1770070414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 msgr2=0x55f694b42c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad5d3fe700 1 --2- 172.21.15.33:0/1770070414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b42c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad5d3fe700 1 -- 172.21.15.33:0/1770070414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2ca70 msgr2=0x55f694b3cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad5d3fe700 1 --2- 172.21.15.33:0/1770070414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2ca70 0x55f694b3cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.253+0000 7fad5d3fe700 1 -- 172.21.15.33:0/1770070414 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6948a6b60 con 0x55f694b2c0b0 2024-05-08T21:21:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.254+0000 7fad5d3fe700 1 --2- 172.21.15.33:0/1770070414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b3c500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55f694b43520 tx=0x7fad48021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12800d635fde1de4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.254+0000 7fad4cff9700 1 -- 172.21.15.33:0/1770070414 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48003d40 con 0x55f694b2c0b0 2024-05-08T21:21:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.254+0000 7fad4cff9700 1 -- 172.21.15.33:0/1770070414 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad4802a040 con 0x55f694b2c0b0 2024-05-08T21:21:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.254+0000 7fad4cff9700 1 -- 172.21.15.33:0/1770070414 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4800f6c0 con 0x55f694b2c0b0 2024-05-08T21:21:09.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 -- 172.21.15.33:0/1770070414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 msgr2=0x55f694b3c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 --2- 172.21.15.33:0/1770070414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b3c500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55f694b43520 tx=0x7fad48021000 comp rx=0 tx=0).stop 2024-05-08T21:21:09.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 -- 172.21.15.33:0/1770070414 shutdown_connections 2024-05-08T21:21:09.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 --2- 172.21.15.33:0/1770070414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2ca70 0x55f694b3cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 --2- 172.21.15.33:0/1770070414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b42c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 --2- 172.21.15.33:0/1770070414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b3c500 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 -- 172.21.15.33:0/1770070414 >> 172.21.15.33:0/1770070414 conn(0x55f694a87210 msgr2=0x55f694b2ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 -- 172.21.15.33:0/1770070414 shutdown_connections 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 -- 172.21.15.33:0/1770070414 wait complete. 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 -- start start 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 Processor -- start 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b37e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b38350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.255+0000 7fad6c01db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2ca70 0x55f694b3b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad6c01db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f694b3b860 con 0x55f694b2ca70 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad6c01db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f694b3b9d0 con 0x55f694b2b750 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad6c01db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f694b3bb90 con 0x55f694b2c0b0 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad6c01db80 10 client.0 _ll_register_callbacks cb 0x55f694a1eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5dbff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b37e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5e400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2ca70 0x55f694b3b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5dbff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b37e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49400/0 (socket says 172.21.15.33:49400) 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5e400700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2ca70 0x55f694b3b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44530/0 (socket says 172.21.15.33:44530) 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5dbff700 1 -- 172.21.15.33:0/406150585 learned_addr learned my addr 172.21.15.33:0/406150585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5d3fe700 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b38350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5dbff700 1 -- 172.21.15.33:0/406150585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 msgr2=0x55f694b38350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5dbff700 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b38350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5dbff700 1 -- 172.21.15.33:0/406150585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2ca70 msgr2=0x55f694b3b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5dbff700 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2ca70 0x55f694b3b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5dbff700 1 -- 172.21.15.33:0/406150585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad58108af0 con 0x55f694b2b750 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5e400700 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2ca70 0x55f694b3b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad5dbff700 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b37e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad5409f1e0 tx=0x7fad54099310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f08d2e89c7aa440 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad417fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad417fa700 1 -- 172.21.15.33:0/406150585 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54099a60 con 0x55f694b2b750 2024-05-08T21:21:09.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad417fa700 1 -- 172.21.15.33:0/406150585 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fad5409a800 con 0x55f694b2b750 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.256+0000 7fad6c01db80 1 -- 172.21.15.33:0/406150585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad5409a100 con 0x55f694b2b750 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.257+0000 7fad417fa700 1 -- 172.21.15.33:0/406150585 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad5409df30 con 0x55f694b2b750 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.257+0000 7fad6c01db80 10 client.26057 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.257+0000 7fad6c01db80 1 -- 172.21.15.33:0/406150585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f694a3f560 con 0x55f694b2b750 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.257+0000 7fad417fa700 1 -- 172.21.15.33:0/406150585 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad5409a100 con 0x55f694b2b750 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.257+0000 7fad417fa700 1 -- 172.21.15.33:0/406150585 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fad540b8310 con 0x55f694b2b750 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.257+0000 7fad6c01db80 10 client.26057 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.257+0000 7fad6c01db80 10 client.26057 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.257+0000 7fad6c01db80 20 client.26057 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.257+0000 7fad6c01db80 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f694b97300 0x55f694b99720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.257+0000 7fad6c01db80 4 client.26057 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.257+0000 7fad6c01db80 1 -- 172.21.15.33:0/406150585 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f694a71ec0 con 0x55f694b97300 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.257+0000 7fad5d3fe700 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f694b97300 0x55f694b99720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad417fa700 10 client.26057 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:21:09.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad5d3fe700 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f694b97300 0x55f694b99720 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43d69eab6f3caa9c server_cookie=4609aaa0b11cc255 in_seq=0 out_seq=0 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad417fa700 1 -- 172.21.15.33:0/406150585 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f694a71ec0 con 0x55f694b97300 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad417fa700 10 client.26057 handle_command_reply: tid=0 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad6c01db80 1 client.26057 shutdown 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad6c01db80 1 -- 172.21.15.33:0/406150585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 msgr2=0x55f694b37e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad6c01db80 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b37e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fad5409f1e0 tx=0x7fad54099310 comp rx=0 tx=0).stop 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad6c01db80 1 -- 172.21.15.33:0/406150585 shutdown_connections 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad6c01db80 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f694b97300 0x55f694b99720 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad6c01db80 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f694b2ca70 0x55f694b3b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad6c01db80 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f694b2b750 0x55f694b37e10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad417fa700 0 client.26057 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad6c01db80 1 --2- 172.21.15.33:0/406150585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f694b2c0b0 0x55f694b38350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad6c01db80 1 -- 172.21.15.33:0/406150585 >> 172.21.15.33:0/406150585 conn(0x55f694a87210 msgr2=0x55f694b344a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad6c01db80 1 -- 172.21.15.33:0/406150585 shutdown_connections 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad6c01db80 1 -- 172.21.15.33:0/406150585 wait complete. 2024-05-08T21:21:09.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.258+0000 7fad6c01db80 20 client.26057 trim_cache size 0 max 16384 2024-05-08T21:21:09.266 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:21:09.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.259+0000 7fad5fe63700 1 -- 172.21.15.33:0/3454721144 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad440a11a0 msgr2=0x7fad440a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.259+0000 7fad5fe63700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad440a11a0 0x7fad440a3660 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fad48006010 tx=0x7fad48003680 comp rx=0 tx=0).stop 2024-05-08T21:21:09.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.259+0000 7fad5fe63700 1 -- 172.21.15.33:0/3454721144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad580ffd60 msgr2=0x7fad58195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.259+0000 7fad5fe63700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad580ffd60 0x7fad58195cd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fad5400e9c0 tx=0x7fad5400ecd0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.259+0000 7fad5fe63700 1 -- 172.21.15.33:0/3454721144 shutdown_connections 2024-05-08T21:21:09.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.259+0000 7fad5fe63700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fad440a11a0 0x7fad440a3660 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.259+0000 7fad5fe63700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad581018f0 0x7fad5819d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.259+0000 7fad5fe63700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad580ffd60 0x7fad58195cd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.259+0000 7fad5fe63700 1 --2- 172.21.15.33:0/3454721144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad58100f50 0x7fad58196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.259+0000 7fad5fe63700 1 -- 172.21.15.33:0/3454721144 >> 172.21.15.33:0/3454721144 conn(0x7fad580fb290 msgr2=0x7fad581043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.259+0000 7fad5fe63700 1 -- 172.21.15.33:0/3454721144 shutdown_connections 2024-05-08T21:21:09.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.259+0000 7fad5fe63700 1 -- 172.21.15.33:0/3454721144 wait complete. 2024-05-08T21:21:09.275 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:21:09.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.426+0000 7f61c5d39700 1 -- 172.21.15.33:0/2517645729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0104210 msgr2=0x7f61c0106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.426+0000 7f61c5d39700 1 --2- 172.21.15.33:0/2517645729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0104210 0x7f61c0106600 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009990 tx=0x7f61b0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.426+0000 7f61c5d39700 1 -- 172.21.15.33:0/2517645729 shutdown_connections 2024-05-08T21:21:09.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.426+0000 7f61c5d39700 1 --2- 172.21.15.33:0/2517645729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61c0106b40 0x7f61c0108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.426+0000 7f61c5d39700 1 --2- 172.21.15.33:0/2517645729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c01018e0 0x7f61c0103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.426+0000 7f61c5d39700 1 --2- 172.21.15.33:0/2517645729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0104210 0x7f61c0106600 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.426+0000 7f61c5d39700 1 -- 172.21.15.33:0/2517645729 >> 172.21.15.33:0/2517645729 conn(0x7f61c00fb290 msgr2=0x7f61c00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.426+0000 7f61c5d39700 1 -- 172.21.15.33:0/2517645729 shutdown_connections 2024-05-08T21:21:09.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.427+0000 7f61c5d39700 1 -- 172.21.15.33:0/2517645729 wait complete. 2024-05-08T21:21:09.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.427+0000 7f61c5d39700 1 Processor -- start 2024-05-08T21:21:09.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61c5d39700 1 -- start start 2024-05-08T21:21:09.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61c5d39700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c01018e0 0x7f61c0196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61c5d39700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61c0104210 0x7f61c0197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61c5d39700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0106b40 0x7f61c019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61c5d39700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c019e820 con 0x7f61c0106b40 2024-05-08T21:21:09.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61c5d39700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c019e990 con 0x7f61c0104210 2024-05-08T21:21:09.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61c5d39700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c019eb50 con 0x7f61c01018e0 2024-05-08T21:21:09.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0106b40 0x7f61c019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61beffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61c0104210 0x7f61c0197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c01018e0 0x7f61c0196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0106b40 0x7f61c019e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44552/0 (socket says 172.21.15.33:44552) 2024-05-08T21:21:09.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61bffff700 1 -- 172.21.15.33:0/358050906 learned_addr learned my addr 172.21.15.33:0/358050906 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61beffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61c0104210 0x7f61c0197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49422/0 (socket says 172.21.15.33:49422) 2024-05-08T21:21:09.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61bffff700 1 -- 172.21.15.33:0/358050906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c01018e0 msgr2=0x7f61c0196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61bffff700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c01018e0 0x7f61c0196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61bffff700 1 -- 172.21.15.33:0/358050906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61c0104210 msgr2=0x7f61c0197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61bffff700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61c0104210 0x7f61c0197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61bffff700 1 -- 172.21.15.33:0/358050906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a8009520 con 0x7f61c0106b40 2024-05-08T21:21:09.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c01018e0 0x7f61c0196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:09.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.428+0000 7f61beffd700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61c0104210 0x7f61c0197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:09.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.429+0000 7f61bffff700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0106b40 0x7f61c019e250 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f61b800ef30 tx=0x7f61b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.429+0000 7f61bcff9700 1 -- 172.21.15.33:0/358050906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8018070 con 0x7f61c0106b40 2024-05-08T21:21:09.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.429+0000 7f61c5d39700 1 -- 172.21.15.33:0/358050906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b00095f0 con 0x7f61c0106b40 2024-05-08T21:21:09.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.429+0000 7f61bcff9700 1 -- 172.21.15.33:0/358050906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61b800f040 con 0x7f61c0106b40 2024-05-08T21:21:09.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.429+0000 7f61bcff9700 1 -- 172.21.15.33:0/358050906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b800cbc0 con 0x7f61c0106b40 2024-05-08T21:21:09.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.429+0000 7f61c5d39700 1 -- 172.21.15.33:0/358050906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c019f0d0 con 0x7f61c0106b40 2024-05-08T21:21:09.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.431+0000 7f61bcff9700 1 -- 172.21.15.33:0/358050906 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f61b800cd80 con 0x7f61c0106b40 2024-05-08T21:21:09.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.432+0000 7f61bcff9700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f61ac0a1120 0x7f61ac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.433+0000 7f61bcff9700 1 -- 172.21.15.33:0/358050906 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f61b80cc760 con 0x7f61c0106b40 2024-05-08T21:21:09.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.433+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f61ac0a1120 0x7f61ac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61d1ef3b80 1 Processor -- start 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61d1ef3b80 1 -- start start 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f61ac0a1120 0x7f61ac0a35e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009fd0 tx=0x7f61a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61d1ef3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 0x55bf58320d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61d1ef3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf583212b0 0x55bf58321730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61d1ef3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58321c70 0x55bf583220f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61d1ef3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf58273150 con 0x55bf58321c70 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61d1ef3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf58270b10 con 0x55bf583212b0 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61d1ef3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5826c400 con 0x55bf58320950 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 0x55bf58320d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 0x55bf58320d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:45622/0 (socket says 172.21.15.33:45622) 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61bf7fe700 1 -- 172.21.15.33:0/975412172 learned_addr learned my addr 172.21.15.33:0/975412172 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61beffd700 1 --2- 172.21.15.33:0/975412172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf583212b0 0x55bf58321730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61bf7fe700 1 -- 172.21.15.33:0/975412172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf583212b0 msgr2=0x55bf58321730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/975412172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf583212b0 0x55bf58321730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61bf7fe700 1 -- 172.21.15.33:0/975412172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58321c70 msgr2=0x55bf583220f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/975412172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58321c70 0x55bf583220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61bf7fe700 1 -- 172.21.15.33:0/975412172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf580ed3b0 con 0x55bf58320950 2024-05-08T21:21:09.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.434+0000 7f61beffd700 1 --2- 172.21.15.33:0/975412172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf583212b0 0x55bf58321730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:21:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.435+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/975412172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 0x55bf58320d70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f61a8004400 tx=0x7f61a8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=947470447f1de51f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.435+0000 7f61b5ffb700 1 -- 172.21.15.33:0/975412172 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8003ea0 con 0x55bf58320950 2024-05-08T21:21:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.436+0000 7f61b5ffb700 1 -- 172.21.15.33:0/975412172 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61a8004cc0 con 0x55bf58320950 2024-05-08T21:21:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.436+0000 7f61b5ffb700 1 -- 172.21.15.33:0/975412172 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a801e400 con 0x55bf58320950 2024-05-08T21:21:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.437+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/975412172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 msgr2=0x55bf58320d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.437+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/975412172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 0x55bf58320d70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f61a8004400 tx=0x7f61a8014000 comp rx=0 tx=0).stop 2024-05-08T21:21:09.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.437+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/975412172 shutdown_connections 2024-05-08T21:21:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.437+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/975412172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58321c70 0x55bf583220f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.437+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/975412172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 0x55bf58320d70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.437+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/975412172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf583212b0 0x55bf58321730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.437+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/975412172 >> 172.21.15.33:0/975412172 conn(0x55bf5827a210 msgr2=0x55bf58316de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.437+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/975412172 shutdown_connections 2024-05-08T21:21:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.437+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/975412172 wait complete. 2024-05-08T21:21:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.437+0000 7f61d1ef3b80 1 -- start start 2024-05-08T21:21:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.438+0000 7f61d1ef3b80 1 Processor -- start 2024-05-08T21:21:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.438+0000 7f61d1ef3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58320950 0x55bf5832a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.438+0000 7f61d1ef3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf583212b0 0x55bf5832a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.438+0000 7f61d1ef3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58321c70 0x55bf58331890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.438+0000 7f61d1ef3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf58208560 con 0x55bf58320950 2024-05-08T21:21:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.438+0000 7f61d1ef3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf58208200 con 0x55bf58321c70 2024-05-08T21:21:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.439+0000 7f61d1ef3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf580ef6a0 con 0x55bf583212b0 2024-05-08T21:21:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.439+0000 7f61d1ef3b80 10 client.0 _ll_register_callbacks cb 0x55bf58211b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.439+0000 7f61bffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58321c70 0x55bf58331890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.439+0000 7f61bffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58321c70 0x55bf58331890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49440/0 (socket says 172.21.15.33:49440) 2024-05-08T21:21:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.439+0000 7f61bffff700 1 -- 172.21.15.33:0/1447721775 learned_addr learned my addr 172.21.15.33:0/1447721775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.440+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58320950 0x55bf5832a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.440+0000 7f61beffd700 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf583212b0 0x55bf5832a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.441+0000 7f61bffff700 1 -- 172.21.15.33:0/1447721775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf583212b0 msgr2=0x55bf5832a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.441+0000 7f61bffff700 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf583212b0 0x55bf5832a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.441+0000 7f61bffff700 1 -- 172.21.15.33:0/1447721775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58320950 msgr2=0x55bf5832a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.441+0000 7f61bffff700 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58320950 0x55bf5832a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.441+0000 7f61bffff700 1 -- 172.21.15.33:0/1447721775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a8017050 con 0x55bf58321c70 2024-05-08T21:21:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.445+0000 7f61bffff700 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58321c70 0x55bf58331890 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f61b80995c0 tx=0x7f61b8099e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdfa708431fb75ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.445+0000 7f619effd700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:21:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.446+0000 7f619effd700 1 -- 172.21.15.33:0/1447721775 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8099fc0 con 0x55bf58321c70 2024-05-08T21:21:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.447+0000 7f619effd700 1 -- 172.21.15.33:0/1447721775 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f61b80998c0 con 0x55bf58321c70 2024-05-08T21:21:09.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.450+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1447721775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf581c5270 con 0x55bf58321c70 2024-05-08T21:21:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.453+0000 7f619effd700 1 -- 172.21.15.33:0/1447721775 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b809d6f0 con 0x55bf58321c70 2024-05-08T21:21:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.454+0000 7f61d1ef3b80 10 client.26069 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.454+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1447721775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55bf58099b60 con 0x55bf58321c70 2024-05-08T21:21:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.455+0000 7f619effd700 1 -- 172.21.15.33:0/1447721775 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bf581c5270 con 0x55bf58321c70 2024-05-08T21:21:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.455+0000 7f619effd700 1 -- 172.21.15.33:0/1447721775 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f61b80b9140 con 0x55bf58321c70 2024-05-08T21:21:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.455+0000 7f61d1ef3b80 10 client.26069 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.455+0000 7f61d1ef3b80 10 client.26069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.455+0000 7f61d1ef3b80 20 client.26069 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.455+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55bf58346ef0 0x55bf58349310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.456+0000 7f61d1ef3b80 4 client.26069 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:09.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.456+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1447721775 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf582eb810 con 0x55bf58346ef0 2024-05-08T21:21:09.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.457+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55bf58346ef0 0x55bf58349310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.458+0000 7f619effd700 10 client.26069 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:09.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.458+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55bf58346ef0 0x55bf58349310 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c67402eebb586b server_cookie=89c80103ac576e62 in_seq=0 out_seq=0 2024-05-08T21:21:09.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.460+0000 7f619effd700 1 -- 172.21.15.33:0/1447721775 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf582eb810 con 0x55bf58346ef0 2024-05-08T21:21:09.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.460+0000 7f619effd700 10 client.26069 handle_command_reply: tid=0 2024-05-08T21:21:09.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.460+0000 7f61d1ef3b80 1 client.26069 shutdown 2024-05-08T21:21:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.460+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1447721775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58321c70 msgr2=0x55bf58331890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.460+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58321c70 0x55bf58331890 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f61b80995c0 tx=0x7f61b8099e00 comp rx=0 tx=0).stop 2024-05-08T21:21:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.460+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1447721775 shutdown_connections 2024-05-08T21:21:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.460+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55bf58346ef0 0x55bf58349310 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.460+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58321c70 0x55bf58331890 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.460+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58320950 0x55bf5832a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.460+0000 7f619effd700 0 client.26069 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.460+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/1447721775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf583212b0 0x55bf5832a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.460+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1447721775 >> 172.21.15.33:0/1447721775 conn(0x55bf5827a210 msgr2=0x55bf583165a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.461+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1447721775 shutdown_connections 2024-05-08T21:21:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.461+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1447721775 wait complete. 2024-05-08T21:21:09.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.461+0000 7f61d1ef3b80 20 client.26069 trim_cache size 0 max 16384 2024-05-08T21:21:09.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.480+0000 7f61d1ef3b80 1 Processor -- start 2024-05-08T21:21:09.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.480+0000 7f61d1ef3b80 1 -- start start 2024-05-08T21:21:09.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61d1ef3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 0x55bf58324ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61d1ef3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf583212b0 0x55bf58325220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61d1ef3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58321c70 0x55bf583317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61d1ef3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf58317980 con 0x55bf58321c70 2024-05-08T21:21:09.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61d1ef3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf58343a20 con 0x55bf583212b0 2024-05-08T21:21:09.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61d1ef3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf58329e80 con 0x55bf58320950 2024-05-08T21:21:09.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61beffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf583212b0 0x55bf58325220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 0x55bf58324ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58321c70 0x55bf583317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 0x55bf58324ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:45646/0 (socket says 172.21.15.33:45646) 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61beffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf583212b0 0x55bf58325220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49448/0 (socket says 172.21.15.33:49448) 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61bf7fe700 1 -- 172.21.15.33:0/1437022818 learned_addr learned my addr 172.21.15.33:0/1437022818 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61bffff700 1 -- 172.21.15.33:0/1437022818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 msgr2=0x55bf58324ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61bffff700 1 --2- 172.21.15.33:0/1437022818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 0x55bf58324ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61bffff700 1 -- 172.21.15.33:0/1437022818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf583212b0 msgr2=0x55bf58325220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61bffff700 1 --2- 172.21.15.33:0/1437022818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf583212b0 0x55bf58325220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61bffff700 1 -- 172.21.15.33:0/1437022818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf58232560 con 0x55bf58321c70 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.481+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/1437022818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 0x55bf58324ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.482+0000 7f61bffff700 1 --2- 172.21.15.33:0/1437022818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58321c70 0x55bf583317b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f61b8098780 tx=0x7f61b8009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d24ea56b0d2d4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.482+0000 7f61b67fc700 1 -- 172.21.15.33:0/1437022818 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b80a19b0 con 0x55bf58321c70 2024-05-08T21:21:09.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.482+0000 7f61b67fc700 1 -- 172.21.15.33:0/1437022818 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61b8099220 con 0x55bf58321c70 2024-05-08T21:21:09.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.482+0000 7f61b67fc700 1 -- 172.21.15.33:0/1437022818 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b809d6f0 con 0x55bf58321c70 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.483+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1437022818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58321c70 msgr2=0x55bf583317b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.483+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/1437022818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58321c70 0x55bf583317b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f61b8098780 tx=0x7f61b8009ed0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.483+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1437022818 shutdown_connections 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.483+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/1437022818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf58321c70 0x55bf583317b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.483+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/1437022818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58320950 0x55bf58324ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.483+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/1437022818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf583212b0 0x55bf58325220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.483+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1437022818 >> 172.21.15.33:0/1437022818 conn(0x55bf5827a210 msgr2=0x55bf5834dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.483+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1437022818 shutdown_connections 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.483+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/1437022818 wait complete. 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.483+0000 7f61d1ef3b80 1 -- start start 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.483+0000 7f61d1ef3b80 1 Processor -- start 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61d1ef3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58320950 0x55bf5832f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61d1ef3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf583212b0 0x55bf5832f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61d1ef3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58321c70 0x55bf583306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61d1ef3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf58330c00 con 0x55bf583212b0 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61d1ef3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf58330d70 con 0x55bf58320950 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61d1ef3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf58330f30 con 0x55bf58321c70 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61d1ef3b80 10 client.0 _ll_register_callbacks cb 0x55bf58211b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58320950 0x55bf5832f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61beffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf583212b0 0x55bf5832f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61bf7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58320950 0x55bf5832f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49462/0 (socket says 172.21.15.33:49462) 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61bf7fe700 1 -- 172.21.15.33:0/3904189876 learned_addr learned my addr 172.21.15.33:0/3904189876 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61bffff700 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58321c70 0x55bf583306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61bf7fe700 1 -- 172.21.15.33:0/3904189876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58321c70 msgr2=0x55bf583306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58321c70 0x55bf583306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61bf7fe700 1 -- 172.21.15.33:0/3904189876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf583212b0 msgr2=0x55bf5832f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:09.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf583212b0 0x55bf5832f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61bf7fe700 1 -- 172.21.15.33:0/3904189876 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf580ed3b0 con 0x55bf58320950 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61bffff700 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58321c70 0x55bf583306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f61bf7fe700 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58320950 0x55bf5832f210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f61a8024000 tx=0x7f61a8009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fecde9d886337b3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.484+0000 7f619effd700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f619effd700 1 -- 172.21.15.33:0/3904189876 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a802d040 con 0x55bf58320950 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/3904189876 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61a8004cc0 con 0x55bf58320950 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f619effd700 1 -- 172.21.15.33:0/3904189876 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f61a8004910 con 0x55bf58320950 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f619effd700 1 -- 172.21.15.33:0/3904189876 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61a8004ad0 con 0x55bf58320950 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f61d1ef3b80 10 client.26075 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/3904189876 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55bf58099b60 con 0x55bf58320950 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f619effd700 1 -- 172.21.15.33:0/3904189876 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61a8004cc0 con 0x55bf58320950 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f619effd700 1 -- 172.21.15.33:0/3904189876 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f61a8024030 con 0x55bf58320950 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f61d1ef3b80 10 client.26075 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:09.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f61d1ef3b80 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f61d1ef3b80 20 client.26075 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55bf5839e0b0 0x55bf583a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f61d1ef3b80 4 client.26075 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.485+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/3904189876 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bf58264ec0 con 0x55bf5839e0b0 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61beffd700 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55bf5839e0b0 0x55bf583a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f619effd700 10 client.26075 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61beffd700 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55bf5839e0b0 0x55bf583a04d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb432ec38272287 server_cookie=af5649af56a8dc46 in_seq=0 out_seq=0 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f619effd700 1 -- 172.21.15.33:0/3904189876 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bf58264ec0 con 0x55bf5839e0b0 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f619effd700 10 client.26075 handle_command_reply: tid=0 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61d1ef3b80 1 client.26075 shutdown 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/3904189876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58320950 msgr2=0x55bf5832f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58320950 0x55bf5832f210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f61a8024000 tx=0x7f61a8009400 comp rx=0 tx=0).stop 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/3904189876 shutdown_connections 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55bf5839e0b0 0x55bf583a04d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf58321c70 0x55bf583306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f619effd700 0 client.26075 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf58320950 0x55bf5832f210 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61d1ef3b80 1 --2- 172.21.15.33:0/3904189876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf583212b0 0x55bf5832f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/3904189876 >> 172.21.15.33:0/3904189876 conn(0x55bf5827a210 msgr2=0x55bf58316d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/3904189876 shutdown_connections 2024-05-08T21:21:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.486+0000 7f61d1ef3b80 1 -- 172.21.15.33:0/3904189876 wait complete. 2024-05-08T21:21:09.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.487+0000 7f61d1ef3b80 20 client.26075 trim_cache size 0 max 16384 2024-05-08T21:21:09.495 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:21:09.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.487+0000 7f61c5d39700 1 -- 172.21.15.33:0/358050906 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f61ac0a1120 msgr2=0x7f61ac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.487+0000 7f61c5d39700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f61ac0a1120 0x7f61ac0a35e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009fd0 tx=0x7f61a8009260 comp rx=0 tx=0).stop 2024-05-08T21:21:09.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.487+0000 7f61c5d39700 1 -- 172.21.15.33:0/358050906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0106b40 msgr2=0x7f61c019e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:09.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.487+0000 7f61c5d39700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0106b40 0x7f61c019e250 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f61b800ef30 tx=0x7f61b800c3a0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.487+0000 7f61c5d39700 1 -- 172.21.15.33:0/358050906 shutdown_connections 2024-05-08T21:21:09.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.487+0000 7f61c5d39700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61c0106b40 0x7f61c019e250 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.487+0000 7f61c5d39700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c01018e0 0x7f61c0196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.487+0000 7f61c5d39700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f61ac0a1120 0x7f61ac0a35e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.487+0000 7f61c5d39700 1 --2- 172.21.15.33:0/358050906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61c0104210 0x7f61c0197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:09.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.487+0000 7f61c5d39700 1 -- 172.21.15.33:0/358050906 >> 172.21.15.33:0/358050906 conn(0x7f61c00fb290 msgr2=0x7f61c01023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:09.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.487+0000 7f61c5d39700 1 -- 172.21.15.33:0/358050906 shutdown_connections 2024-05-08T21:21:09.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:09.488+0000 7f61c5d39700 1 -- 172.21.15.33:0/358050906 wait complete. 2024-05-08T21:21:09.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:09 smithi033 ceph-mon[42528]: pgmap v355: 129 pgs: 129 active+clean; 47 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-05-08T21:21:09.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:09 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2375627628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:21:09.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:09 smithi071 ceph-mon[47074]: pgmap v355: 129 pgs: 129 active+clean; 47 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-05-08T21:21:09.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:09 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2375627628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:21:09.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:09 smithi082 ceph-mon[47512]: pgmap v355: 129 pgs: 129 active+clean; 47 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 225 op/s 2024-05-08T21:21:09.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:09 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2375627628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:21:10.504 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:21:10.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:21:10.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.661+0000 7f39ad22d700 1 -- 172.21.15.33:0/1839357506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8101930 msgr2=0x7f39a81081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.661+0000 7f39ad22d700 1 --2- 172.21.15.33:0/1839357506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8101930 0x7f39a81081e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009fc0 tx=0x7f39a000b330 comp rx=0 tx=0).stop 2024-05-08T21:21:10.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.662+0000 7f39ad22d700 1 -- 172.21.15.33:0/1839357506 shutdown_connections 2024-05-08T21:21:10.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.662+0000 7f39ad22d700 1 --2- 172.21.15.33:0/1839357506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8101930 0x7f39a81081e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.662+0000 7f39ad22d700 1 --2- 172.21.15.33:0/1839357506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a80ffda0 0x7f39a81001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.662+0000 7f39ad22d700 1 --2- 172.21.15.33:0/1839357506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39a8100f90 0x7f39a81013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.662+0000 7f39ad22d700 1 -- 172.21.15.33:0/1839357506 >> 172.21.15.33:0/1839357506 conn(0x7f39a80fb2d0 msgr2=0x7f39a80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:10.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.662+0000 7f39ad22d700 1 -- 172.21.15.33:0/1839357506 shutdown_connections 2024-05-08T21:21:10.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.662+0000 7f39ad22d700 1 -- 172.21.15.33:0/1839357506 wait complete. 2024-05-08T21:21:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.662+0000 7f39ad22d700 1 Processor -- start 2024-05-08T21:21:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39ad22d700 1 -- start start 2024-05-08T21:21:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39ad22d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a80ffda0 0x7f39a8103de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39ad22d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39a8100f90 0x7f39a81024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39ad22d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8101930 0x7f39a8102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39ad22d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a8103100 con 0x7f39a8101930 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39ad22d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a8103270 con 0x7f39a80ffda0 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39ad22d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a81a0ef0 con 0x7f39a8100f90 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39a6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a80ffda0 0x7f39a8103de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39a759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8101930 0x7f39a8102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39a6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a80ffda0 0x7f39a8103de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:45528/0 (socket says 172.21.15.33:45528) 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39a759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8101930 0x7f39a8102a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37992/0 (socket says 172.21.15.33:37992) 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39a6d9d700 1 -- 172.21.15.33:0/3524716934 learned_addr learned my addr 172.21.15.33:0/3524716934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.663+0000 7f39a659c700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39a8100f90 0x7f39a81024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.664+0000 7f39a6d9d700 1 -- 172.21.15.33:0/3524716934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39a8100f90 msgr2=0x7f39a81024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.664+0000 7f39a6d9d700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39a8100f90 0x7f39a81024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.664+0000 7f39a6d9d700 1 -- 172.21.15.33:0/3524716934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8101930 msgr2=0x7f39a8102a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.664+0000 7f39a6d9d700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8101930 0x7f39a8102a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.664+0000 7f39a6d9d700 1 -- 172.21.15.33:0/3524716934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39980095f0 con 0x7f39a80ffda0 2024-05-08T21:21:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.664+0000 7f39a759e700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8101930 0x7f39a8102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.664+0000 7f39a6d9d700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a80ffda0 0x7f39a8103de0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f399800ebf0 tx=0x7f399800ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.664+0000 7f398ffff700 1 -- 172.21.15.33:0/3524716934 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3998009a10 con 0x7f39a80ffda0 2024-05-08T21:21:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.664+0000 7f39ad22d700 1 -- 172.21.15.33:0/3524716934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a0009c40 con 0x7f39a80ffda0 2024-05-08T21:21:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.664+0000 7f398ffff700 1 -- 172.21.15.33:0/3524716934 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f399800cc90 con 0x7f39a80ffda0 2024-05-08T21:21:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.664+0000 7f398ffff700 1 -- 172.21.15.33:0/3524716934 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3998018680 con 0x7f39a80ffda0 2024-05-08T21:21:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.664+0000 7f39ad22d700 1 -- 172.21.15.33:0/3524716934 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a81a14d0 con 0x7f39a80ffda0 2024-05-08T21:21:10.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.666+0000 7f398ffff700 1 -- 172.21.15.33:0/3524716934 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3998010430 con 0x7f39a80ffda0 2024-05-08T21:21:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.667+0000 7f398ffff700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39900a1120 0x7f39900a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.667+0000 7f39a659c700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39900a1120 0x7f39900a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.667+0000 7f398ffff700 1 -- 172.21.15.33:0/3524716934 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3998014070 con 0x7f39a80ffda0 2024-05-08T21:21:10.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.668+0000 7f39a659c700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39900a1120 0x7f39900a35e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f399c009c60 tx=0x7f399c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:10.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.668+0000 7f39b93e7b80 1 Processor -- start 2024-05-08T21:21:10.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.668+0000 7f39b93e7b80 1 -- start start 2024-05-08T21:21:10.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39b93e7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39b93e7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39b93e7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b62d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39b93e7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2ab81d0 con 0x5599c2b615c0 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39b93e7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2ab5b90 con 0x5599c2b628e0 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39b93e7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2ab1480 con 0x5599c2b61f20 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39a6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39a659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39a6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38002/0 (socket says 172.21.15.33:38002) 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39a6d9d700 1 -- 172.21.15.33:0/777622545 learned_addr learned my addr 172.21.15.33:0/777622545 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39a759e700 1 --2- 172.21.15.33:0/777622545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b62d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39a659c700 1 -- 172.21.15.33:0/777622545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 msgr2=0x5599c2b62d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39a659c700 1 --2- 172.21.15.33:0/777622545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b62d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39a659c700 1 -- 172.21.15.33:0/777622545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 msgr2=0x5599c2b619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39a659c700 1 --2- 172.21.15.33:0/777622545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39a659c700 1 -- 172.21.15.33:0/777622545 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c2932430 con 0x5599c2b61f20 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f39a659c700 1 --2- 172.21.15.33:0/777622545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b623a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f399c00e3e0 tx=0x7f399c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94a69874382e1333 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:10.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.669+0000 7f398d7fa700 1 -- 172.21.15.33:0/777622545 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c00ed80 con 0x5599c2b61f20 2024-05-08T21:21:10.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.670+0000 7f398d7fa700 1 -- 172.21.15.33:0/777622545 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f399c021070 con 0x5599c2b61f20 2024-05-08T21:21:10.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.670+0000 7f398d7fa700 1 -- 172.21.15.33:0/777622545 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c026470 con 0x5599c2b61f20 2024-05-08T21:21:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.670+0000 7f39b93e7b80 1 -- 172.21.15.33:0/777622545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 msgr2=0x5599c2b623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.670+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/777622545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b623a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f399c00e3e0 tx=0x7f399c014000 comp rx=0 tx=0).stop 2024-05-08T21:21:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.670+0000 7f39b93e7b80 1 -- 172.21.15.33:0/777622545 shutdown_connections 2024-05-08T21:21:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.670+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/777622545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b62d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.670+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/777622545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.670+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/777622545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b623a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39b93e7b80 1 -- 172.21.15.33:0/777622545 >> 172.21.15.33:0/777622545 conn(0x5599c2abf290 msgr2=0x5599c2b5be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39b93e7b80 1 -- 172.21.15.33:0/777622545 shutdown_connections 2024-05-08T21:21:10.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39b93e7b80 1 -- 172.21.15.33:0/777622545 wait complete. 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39b93e7b80 1 -- start start 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39b93e7b80 1 Processor -- start 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39b93e7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b6ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39b93e7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b6b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39b93e7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39b93e7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2a4d5e0 con 0x5599c2b615c0 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39b93e7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2a4d280 con 0x5599c2b628e0 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39b93e7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2934720 con 0x5599c2b61f20 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39b93e7b80 10 client.0 _ll_register_callbacks cb 0x5599c2a56bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.671+0000 7f39a6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b6ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39a659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b6b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39a6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b6ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38014/0 (socket says 172.21.15.33:38014) 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39a759e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39a6d9d700 1 -- 172.21.15.33:0/1549190129 learned_addr learned my addr 172.21.15.33:0/1549190129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39a659c700 1 -- 172.21.15.33:0/1549190129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 msgr2=0x5599c2b723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39a659c700 1 --2- 172.21.15.33:0/1549190129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39a659c700 1 -- 172.21.15.33:0/1549190129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 msgr2=0x5599c2b6ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39a659c700 1 --2- 172.21.15.33:0/1549190129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b6ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39a659c700 1 -- 172.21.15.33:0/1549190129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c017070 con 0x5599c2b61f20 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39a6d9d700 1 --2- 172.21.15.33:0/1549190129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b6ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39a659c700 1 --2- 172.21.15.33:0/1549190129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b6b3a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f399c00e3b0 tx=0x7f399c0037b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d04f1844304b35d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f398aff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f398aff5700 1 -- 172.21.15.33:0/1549190129 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c02a040 con 0x5599c2b61f20 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f398aff5700 1 -- 172.21.15.33:0/1549190129 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f399c004100 con 0x5599c2b61f20 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1549190129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599c2a0a2f0 con 0x5599c2b61f20 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f398aff5700 1 -- 172.21.15.33:0/1549190129 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f399c0042c0 con 0x5599c2b61f20 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39b93e7b80 10 client.26107 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:10.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.672+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1549190129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5599c28debe0 con 0x5599c2b61f20 2024-05-08T21:21:10.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.673+0000 7f398aff5700 1 -- 172.21.15.33:0/1549190129 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5599c2a0a2f0 con 0x5599c2b61f20 2024-05-08T21:21:10.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.673+0000 7f398aff5700 1 -- 172.21.15.33:0/1549190129 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f399c026470 con 0x5599c2b61f20 2024-05-08T21:21:10.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.673+0000 7f39b93e7b80 10 client.26107 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:10.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.673+0000 7f39b93e7b80 10 client.26107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:10.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.673+0000 7f39b93e7b80 20 client.26107 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:10.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.673+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/1549190129 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5599c2b76b90 0x5599c2b78fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.673+0000 7f39b93e7b80 4 client.26107 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:10.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.673+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1549190129 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599c2b30890 con 0x5599c2b76b90 2024-05-08T21:21:10.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.673+0000 7f39a6d9d700 1 --2- 172.21.15.33:0/1549190129 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5599c2b76b90 0x5599c2b78fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.673+0000 7f398aff5700 10 client.26107 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:10.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.673+0000 7f39a6d9d700 1 --2- 172.21.15.33:0/1549190129 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5599c2b76b90 0x5599c2b78fb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=183c58e562ef7d5a server_cookie=1d75e490d80aa4c9 in_seq=0 out_seq=0 2024-05-08T21:21:10.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f398aff5700 1 -- 172.21.15.33:0/1549190129 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599c2b30890 con 0x5599c2b76b90 2024-05-08T21:21:10.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f398aff5700 10 client.26107 handle_command_reply: tid=0 2024-05-08T21:21:10.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f39b93e7b80 1 client.26107 shutdown 2024-05-08T21:21:10.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1549190129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 msgr2=0x5599c2b6b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/1549190129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b6b3a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f399c00e3b0 tx=0x7f399c0037b0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1549190129 shutdown_connections 2024-05-08T21:21:10.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/1549190129 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5599c2b76b90 0x5599c2b78fb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/1549190129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/1549190129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b6ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/1549190129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b6b3a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f398aff5700 0 client.26107 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:10.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1549190129 >> 172.21.15.33:0/1549190129 conn(0x5599c2abf290 msgr2=0x5599c2b5b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:10.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1549190129 shutdown_connections 2024-05-08T21:21:10.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1549190129 wait complete. 2024-05-08T21:21:10.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.675+0000 7f39b93e7b80 20 client.26107 trim_cache size 0 max 16384 2024-05-08T21:21:10.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39b93e7b80 1 Processor -- start 2024-05-08T21:21:10.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39b93e7b80 1 -- start start 2024-05-08T21:21:10.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39b93e7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b615c0 0x5599c2b6ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39b93e7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b61f20 0x5599c2b723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39b93e7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b628e0 0x5599c2b6b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39b93e7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2b73560 con 0x5599c2b61f20 2024-05-08T21:21:10.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39b93e7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2b6a9d0 con 0x5599c2b615c0 2024-05-08T21:21:10.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39b93e7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2b5cc10 con 0x5599c2b628e0 2024-05-08T21:21:10.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39a6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b615c0 0x5599c2b6ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39a659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b61f20 0x5599c2b723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39a659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b61f20 0x5599c2b723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38020/0 (socket says 172.21.15.33:38020) 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39a6d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b615c0 0x5599c2b6ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:45556/0 (socket says 172.21.15.33:45556) 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39a659c700 1 -- 172.21.15.33:0/2261728589 learned_addr learned my addr 172.21.15.33:0/2261728589 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.694+0000 7f39a759e700 1 --2- 172.21.15.33:0/2261728589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b628e0 0x5599c2b6b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.695+0000 7f39a659c700 1 -- 172.21.15.33:0/2261728589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b628e0 msgr2=0x5599c2b6b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.695+0000 7f39a659c700 1 --2- 172.21.15.33:0/2261728589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b628e0 0x5599c2b6b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.695+0000 7f39a659c700 1 -- 172.21.15.33:0/2261728589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b615c0 msgr2=0x5599c2b6ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.695+0000 7f39a659c700 1 --2- 172.21.15.33:0/2261728589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b615c0 0x5599c2b6ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.695+0000 7f39a659c700 1 -- 172.21.15.33:0/2261728589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599c2b58560 con 0x5599c2b61f20 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.695+0000 7f39a759e700 1 --2- 172.21.15.33:0/2261728589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b628e0 0x5599c2b6b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.695+0000 7f39a6d9d700 1 --2- 172.21.15.33:0/2261728589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b615c0 0x5599c2b6ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.695+0000 7f39a659c700 1 --2- 172.21.15.33:0/2261728589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b61f20 0x5599c2b723a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f399c00a010 tx=0x7f399c009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a3e6d9f68605ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:10.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.695+0000 7f398dffb700 1 -- 172.21.15.33:0/2261728589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c02a070 con 0x5599c2b61f20 2024-05-08T21:21:10.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.695+0000 7f398dffb700 1 -- 172.21.15.33:0/2261728589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f399c026a70 con 0x5599c2b61f20 2024-05-08T21:21:10.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.695+0000 7f398dffb700 1 -- 172.21.15.33:0/2261728589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c026e10 con 0x5599c2b61f20 2024-05-08T21:21:10.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.696+0000 7f39b93e7b80 1 -- 172.21.15.33:0/2261728589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b61f20 msgr2=0x5599c2b723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.696+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/2261728589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b61f20 0x5599c2b723a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f399c00a010 tx=0x7f399c009fa0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.696+0000 7f39b93e7b80 1 -- 172.21.15.33:0/2261728589 shutdown_connections 2024-05-08T21:21:10.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.696+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/2261728589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b628e0 0x5599c2b6b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.696+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/2261728589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b615c0 0x5599c2b6ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.696+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/2261728589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b61f20 0x5599c2b723a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.696+0000 7f39b93e7b80 1 -- 172.21.15.33:0/2261728589 >> 172.21.15.33:0/2261728589 conn(0x5599c2abf290 msgr2=0x5599c2b657b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.696+0000 7f39b93e7b80 1 -- 172.21.15.33:0/2261728589 shutdown_connections 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.696+0000 7f39b93e7b80 1 -- 172.21.15.33:0/2261728589 wait complete. 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.696+0000 7f39b93e7b80 1 -- start start 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.696+0000 7f39b93e7b80 1 Processor -- start 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39b93e7b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b71ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39b93e7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b6e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39b93e7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b6ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39b93e7b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2b6f5f0 con 0x5599c2b615c0 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39b93e7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2b72560 con 0x5599c2b628e0 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39b93e7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599c2b726f0 con 0x5599c2b61f20 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39b93e7b80 10 client.0 _ll_register_callbacks cb 0x5599c2a56bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39a6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b71ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39a6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b71ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38032/0 (socket says 172.21.15.33:38032) 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39a759e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b6ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39a6d9d700 1 -- 172.21.15.33:0/1450476936 learned_addr learned my addr 172.21.15.33:0/1450476936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39a659c700 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b6e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39a6d9d700 1 -- 172.21.15.33:0/1450476936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 msgr2=0x5599c2b6e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39a6d9d700 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b6e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39a6d9d700 1 -- 172.21.15.33:0/1450476936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 msgr2=0x5599c2b6ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39a6d9d700 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b6ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39a6d9d700 1 -- 172.21.15.33:0/1450476936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a81a14d0 con 0x5599c2b615c0 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39a759e700 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b6ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f39a6d9d700 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b71ee0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f399809a020 tx=0x7f3998085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e6156f537aa62e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.697+0000 7f398aff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f398aff5700 1 -- 172.21.15.33:0/1450476936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39980cbea0 con 0x5599c2b615c0 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f398aff5700 1 -- 172.21.15.33:0/1450476936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f399809aa20 con 0x5599c2b615c0 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1450476936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f399c021070 con 0x5599c2b615c0 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f398aff5700 1 -- 172.21.15.33:0/1450476936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f399809a320 con 0x5599c2b615c0 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f39b93e7b80 10 client.16224 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:10.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1450476936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f39a8108b30 con 0x5599c2b615c0 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f398aff5700 1 -- 172.21.15.33:0/1450476936 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f399c021070 con 0x5599c2b615c0 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f398aff5700 1 -- 172.21.15.33:0/1450476936 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3998099810 con 0x5599c2b615c0 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f39b93e7b80 10 client.16224 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f39b93e7b80 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f39b93e7b80 20 client.16224 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5599c2bce930 0x5599c2bd0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f39b93e7b80 4 client.16224 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1450476936 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5599c2aa9f40 con 0x5599c2bce930 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.698+0000 7f39a659c700 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5599c2bce930 0x5599c2bd0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.699+0000 7f398aff5700 10 client.16224 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.699+0000 7f39a659c700 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5599c2bce930 0x5599c2bd0d50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e8c77cc75e90a12 server_cookie=b8c5b31293cb0591 in_seq=0 out_seq=0 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f398aff5700 1 -- 172.21.15.33:0/1450476936 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5599c2aa9f40 con 0x5599c2bce930 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f398aff5700 10 client.16224 handle_command_reply: tid=0 2024-05-08T21:21:10.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f39b93e7b80 1 client.16224 shutdown 2024-05-08T21:21:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1450476936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 msgr2=0x5599c2b71ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b71ee0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f399809a020 tx=0x7f3998085040 comp rx=0 tx=0).stop 2024-05-08T21:21:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1450476936 shutdown_connections 2024-05-08T21:21:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5599c2bce930 0x5599c2bd0d50 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599c2b628e0 0x5599c2b6ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599c2b615c0 0x5599c2b71ee0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f398aff5700 0 client.16224 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f39b93e7b80 1 --2- 172.21.15.33:0/1450476936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5599c2b61f20 0x5599c2b6e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1450476936 >> 172.21.15.33:0/1450476936 conn(0x5599c2abf290 msgr2=0x5599c2b657b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1450476936 shutdown_connections 2024-05-08T21:21:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.700+0000 7f39b93e7b80 1 -- 172.21.15.33:0/1450476936 wait complete. 2024-05-08T21:21:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39b93e7b80 20 client.16224 trim_cache size 0 max 16384 2024-05-08T21:21:10.708 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:21:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:21:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "f50ed481-9aef-490b-ac79-9eecdf05e253", 2024-05-08T21:21:10.708 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:21:10.708 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:21:10.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39ad22d700 1 -- 172.21.15.33:0/3524716934 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39900a1120 msgr2=0x7f39900a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39ad22d700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39900a1120 0x7f39900a35e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f399c009c60 tx=0x7f399c009260 comp rx=0 tx=0).stop 2024-05-08T21:21:10.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39ad22d700 1 -- 172.21.15.33:0/3524716934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a80ffda0 msgr2=0x7f39a8103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39ad22d700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a80ffda0 0x7f39a8103de0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f399800ebf0 tx=0x7f399800ef00 comp rx=0 tx=0).stop 2024-05-08T21:21:10.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39ad22d700 1 -- 172.21.15.33:0/3524716934 shutdown_connections 2024-05-08T21:21:10.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39ad22d700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8101930 0x7f39a8102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39ad22d700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39900a1120 0x7f39900a35e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39ad22d700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a80ffda0 0x7f39a8103de0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39ad22d700 1 --2- 172.21.15.33:0/3524716934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39a8100f90 0x7f39a81024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39ad22d700 1 -- 172.21.15.33:0/3524716934 >> 172.21.15.33:0/3524716934 conn(0x7f39a80fb2d0 msgr2=0x7f39a8069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:10.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39ad22d700 1 -- 172.21.15.33:0/3524716934 shutdown_connections 2024-05-08T21:21:10.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.701+0000 7f39ad22d700 1 -- 172.21.15.33:0/3524716934 wait complete. 2024-05-08T21:21:10.719 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:21:10.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.885+0000 7efc24fa2700 1 -- 172.21.15.33:0/1176454321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc20106b40 msgr2=0x7efc20108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.885+0000 7efc24fa2700 1 --2- 172.21.15.33:0/1176454321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc20106b40 0x7efc20108fb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efc1800b0a0 tx=0x7efc1800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.885+0000 7efc24fa2700 1 -- 172.21.15.33:0/1176454321 shutdown_connections 2024-05-08T21:21:10.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.885+0000 7efc24fa2700 1 --2- 172.21.15.33:0/1176454321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc20106b40 0x7efc20108fb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.885+0000 7efc24fa2700 1 --2- 172.21.15.33:0/1176454321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc201018e0 0x7efc20103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.885+0000 7efc24fa2700 1 --2- 172.21.15.33:0/1176454321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc20104210 0x7efc20106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.886+0000 7efc24fa2700 1 -- 172.21.15.33:0/1176454321 >> 172.21.15.33:0/1176454321 conn(0x7efc200fb290 msgr2=0x7efc200fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:10.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.886+0000 7efc24fa2700 1 -- 172.21.15.33:0/1176454321 shutdown_connections 2024-05-08T21:21:10.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.886+0000 7efc24fa2700 1 -- 172.21.15.33:0/1176454321 wait complete. 2024-05-08T21:21:10.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc24fa2700 1 Processor -- start 2024-05-08T21:21:10.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc24fa2700 1 -- start start 2024-05-08T21:21:10.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc24fa2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc201018e0 0x7efc20196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc24fa2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc20104210 0x7efc201972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc24fa2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc20106b40 0x7efc2019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc24fa2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2019e8e0 con 0x7efc201018e0 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc24fa2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2019ea20 con 0x7efc20104210 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc24fa2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2019ebb0 con 0x7efc20106b40 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc1e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc201018e0 0x7efc20196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc1e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc201018e0 0x7efc20196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38056/0 (socket says 172.21.15.33:38056) 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc1e59c700 1 -- 172.21.15.33:0/3674110071 learned_addr learned my addr 172.21.15.33:0/3674110071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc1dd9b700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc20104210 0x7efc201972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc1ed9d700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc20106b40 0x7efc2019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc1e59c700 1 -- 172.21.15.33:0/3674110071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc20106b40 msgr2=0x7efc2019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc1e59c700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc20106b40 0x7efc2019e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc1e59c700 1 -- 172.21.15.33:0/3674110071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc20104210 msgr2=0x7efc201972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.887+0000 7efc1e59c700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc20104210 0x7efc201972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.888+0000 7efc1e59c700 1 -- 172.21.15.33:0/3674110071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc14009520 con 0x7efc201018e0 2024-05-08T21:21:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.888+0000 7efc1e59c700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc201018e0 0x7efc20196db0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efc1000e9a0 tx=0x7efc1000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.888+0000 7efc0f7fe700 1 -- 172.21.15.33:0/3674110071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10009790 con 0x7efc201018e0 2024-05-08T21:21:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.888+0000 7efc24fa2700 1 -- 172.21.15.33:0/3674110071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc18009d00 con 0x7efc201018e0 2024-05-08T21:21:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.888+0000 7efc0f7fe700 1 -- 172.21.15.33:0/3674110071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc10010430 con 0x7efc201018e0 2024-05-08T21:21:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.888+0000 7efc24fa2700 1 -- 172.21.15.33:0/3674110071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2019f1b0 con 0x7efc201018e0 2024-05-08T21:21:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.888+0000 7efc0f7fe700 1 -- 172.21.15.33:0/3674110071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1000cea0 con 0x7efc201018e0 2024-05-08T21:21:10.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.891+0000 7efc0f7fe700 1 -- 172.21.15.33:0/3674110071 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7efc10018680 con 0x7efc201018e0 2024-05-08T21:21:10.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.891+0000 7efc0f7fe700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7efc080a11a0 0x7efc080a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.892+0000 7efc1dd9b700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7efc080a11a0 0x7efc080a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.892+0000 7efc0f7fe700 1 -- 172.21.15.33:0/3674110071 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7efc10014070 con 0x7efc201018e0 2024-05-08T21:21:10.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.892+0000 7efc1dd9b700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7efc080a11a0 0x7efc080a3660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efc14009df0 tx=0x7efc14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:10.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.892+0000 7efc3115cb80 1 Processor -- start 2024-05-08T21:21:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc3115cb80 1 -- start start 2024-05-08T21:21:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc3115cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5540 0x55e9aaec5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc3115cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaec6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc3115cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaec6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc3115cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aae1c150 con 0x55e9aaec5ea0 2024-05-08T21:21:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc3115cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aae19b10 con 0x55e9aaec5540 2024-05-08T21:21:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc3115cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aae15400 con 0x55e9aaec6860 2024-05-08T21:21:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaec6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaec6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38068/0 (socket says 172.21.15.33:38068) 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5540 0x55e9aaec5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1dd9b700 1 -- 172.21.15.33:0/3895954215 learned_addr learned my addr 172.21.15.33:0/3895954215 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1ed9d700 1 --2- 172.21.15.33:0/3895954215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaec6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1ed9d700 1 -- 172.21.15.33:0/3895954215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5540 msgr2=0x55e9aaec5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1ed9d700 1 --2- 172.21.15.33:0/3895954215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5540 0x55e9aaec5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1ed9d700 1 -- 172.21.15.33:0/3895954215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 msgr2=0x55e9aaec6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1ed9d700 1 --2- 172.21.15.33:0/3895954215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaec6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1ed9d700 1 -- 172.21.15.33:0/3895954215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9aac963b0 con 0x55e9aaec6860 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1dd9b700 1 --2- 172.21.15.33:0/3895954215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaec6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1e59c700 1 --2- 172.21.15.33:0/3895954215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5540 0x55e9aaec5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.893+0000 7efc1ed9d700 1 --2- 172.21.15.33:0/3895954215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaec6ce0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc18007af0 tx=0x7efc18009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f41c2df8fbd1198 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.894+0000 7efc0cff9700 1 -- 172.21.15.33:0/3895954215 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1800e040 con 0x55e9aaec6860 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.894+0000 7efc0cff9700 1 -- 172.21.15.33:0/3895954215 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc180074e0 con 0x55e9aaec6860 2024-05-08T21:21:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.894+0000 7efc0cff9700 1 -- 172.21.15.33:0/3895954215 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18003c60 con 0x55e9aaec6860 2024-05-08T21:21:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.894+0000 7efc3115cb80 1 -- 172.21.15.33:0/3895954215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 msgr2=0x55e9aaec6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.894+0000 7efc3115cb80 1 --2- 172.21.15.33:0/3895954215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaec6ce0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc18007af0 tx=0x7efc18009c00 comp rx=0 tx=0).stop 2024-05-08T21:21:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.894+0000 7efc3115cb80 1 -- 172.21.15.33:0/3895954215 shutdown_connections 2024-05-08T21:21:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.894+0000 7efc3115cb80 1 --2- 172.21.15.33:0/3895954215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaec6ce0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.894+0000 7efc3115cb80 1 --2- 172.21.15.33:0/3895954215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5540 0x55e9aaec5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.895+0000 7efc3115cb80 1 --2- 172.21.15.33:0/3895954215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaec6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.895+0000 7efc3115cb80 1 -- 172.21.15.33:0/3895954215 >> 172.21.15.33:0/3895954215 conn(0x55e9aae23210 msgr2=0x55e9aaebfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.895+0000 7efc3115cb80 1 -- 172.21.15.33:0/3895954215 shutdown_connections 2024-05-08T21:21:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.895+0000 7efc3115cb80 1 -- 172.21.15.33:0/3895954215 wait complete. 2024-05-08T21:21:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.895+0000 7efc3115cb80 1 -- start start 2024-05-08T21:21:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.895+0000 7efc3115cb80 1 Processor -- start 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.895+0000 7efc3115cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5540 0x55e9aaeced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.895+0000 7efc3115cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaecf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.895+0000 7efc3115cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaed62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.895+0000 7efc3115cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aadb1560 con 0x55e9aaec5ea0 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.895+0000 7efc3115cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aadb1200 con 0x55e9aaec5540 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc3115cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aac986a0 con 0x55e9aaec6860 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaecf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5540 0x55e9aaeced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaed62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1e59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5540 0x55e9aaeced60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:45616/0 (socket says 172.21.15.33:45616) 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaed62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:37258/0 (socket says 172.21.15.33:37258) 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1e59c700 1 -- 172.21.15.33:0/1225169950 learned_addr learned my addr 172.21.15.33:0/1225169950 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaecf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38072/0 (socket says 172.21.15.33:38072) 2024-05-08T21:21:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc3115cb80 10 client.0 _ll_register_callbacks cb 0x55e9aadbab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1ed9d700 1 -- 172.21.15.33:0/1225169950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5540 msgr2=0x55e9aaeced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1ed9d700 1 --2- 172.21.15.33:0/1225169950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5540 0x55e9aaeced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1ed9d700 1 -- 172.21.15.33:0/1225169950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 msgr2=0x55e9aaecf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1ed9d700 1 --2- 172.21.15.33:0/1225169950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaecf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1ed9d700 1 -- 172.21.15.33:0/1225169950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc14017070 con 0x55e9aaec6860 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc1ed9d700 1 --2- 172.21.15.33:0/1225169950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaed62e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc1800b070 tx=0x7efc180093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0059de3416ed69a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc05ffb700 1 -- 172.21.15.33:0/1225169950 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1800e040 con 0x55e9aaec6860 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc3115cb80 1 -- 172.21.15.33:0/1225169950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9aad6e270 con 0x55e9aaec6860 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.896+0000 7efc05ffb700 1 -- 172.21.15.33:0/1225169950 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7efc180050c0 con 0x55e9aaec6860 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.897+0000 7efc05ffb700 1 -- 172.21.15.33:0/1225169950 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc18005280 con 0x55e9aaec6860 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.897+0000 7efc3115cb80 10 client.26125 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.897+0000 7efc3115cb80 1 -- 172.21.15.33:0/1225169950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7efc20109ba0 con 0x55e9aaec6860 2024-05-08T21:21:10.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.897+0000 7efc05ffb700 1 -- 172.21.15.33:0/1225169950 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e9aad6e270 con 0x55e9aaec6860 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.897+0000 7efc05ffb700 1 -- 172.21.15.33:0/1225169950 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efc18019070 con 0x55e9aaec6860 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.897+0000 7efc3115cb80 10 client.26125 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.897+0000 7efc3115cb80 10 client.26125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.897+0000 7efc3115cb80 20 client.26125 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.897+0000 7efc3115cb80 1 --2- 172.21.15.33:0/1225169950 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9aaeeb870 0x55e9aaeedc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.897+0000 7efc3115cb80 4 client.26125 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.897+0000 7efc3115cb80 1 -- 172.21.15.33:0/1225169950 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9aae94810 con 0x55e9aaeeb870 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.897+0000 7efc1e59c700 1 --2- 172.21.15.33:0/1225169950 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9aaeeb870 0x55e9aaeedc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.898+0000 7efc05ffb700 10 client.26125 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.898+0000 7efc1e59c700 1 --2- 172.21.15.33:0/1225169950 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9aaeeb870 0x55e9aaeedc90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8a48d80d409a82 server_cookie=f49ad685f08c61c6 in_seq=0 out_seq=0 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc05ffb700 1 -- 172.21.15.33:0/1225169950 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e9aae94810 con 0x55e9aaeeb870 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc05ffb700 10 client.26125 handle_command_reply: tid=0 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc3115cb80 1 client.26125 shutdown 2024-05-08T21:21:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc3115cb80 1 -- 172.21.15.33:0/1225169950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 msgr2=0x55e9aaed62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc3115cb80 1 --2- 172.21.15.33:0/1225169950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaed62e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc1800b070 tx=0x7efc180093e0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc3115cb80 1 -- 172.21.15.33:0/1225169950 shutdown_connections 2024-05-08T21:21:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc3115cb80 1 --2- 172.21.15.33:0/1225169950 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9aaeeb870 0x55e9aaeedc90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc3115cb80 1 --2- 172.21.15.33:0/1225169950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaed62e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc3115cb80 1 --2- 172.21.15.33:0/1225169950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5540 0x55e9aaeced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc05ffb700 0 client.26125 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc3115cb80 1 --2- 172.21.15.33:0/1225169950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaecf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc3115cb80 1 -- 172.21.15.33:0/1225169950 >> 172.21.15.33:0/1225169950 conn(0x55e9aae23210 msgr2=0x55e9aaec4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc3115cb80 1 -- 172.21.15.33:0/1225169950 shutdown_connections 2024-05-08T21:21:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc3115cb80 1 -- 172.21.15.33:0/1225169950 wait complete. 2024-05-08T21:21:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.899+0000 7efc3115cb80 20 client.26125 trim_cache size 0 max 16384 2024-05-08T21:21:10.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.919+0000 7efc3115cb80 1 Processor -- start 2024-05-08T21:21:10.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.919+0000 7efc3115cb80 1 -- start start 2024-05-08T21:21:10.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc3115cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec5540 0x55e9aaee79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc3115cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaed6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc3115cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec6860 0x55e9aaeccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc3115cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aaec15b0 con 0x55e9aaec5ea0 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc3115cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aaecd900 con 0x55e9aaec6860 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc3115cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aaeba880 con 0x55e9aaec5540 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc1dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaed6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc1e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec5540 0x55e9aaee79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc1e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec5540 0x55e9aaee79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:37272/0 (socket says 172.21.15.33:37272) 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc1ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec6860 0x55e9aaeccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc1dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaed6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38078/0 (socket says 172.21.15.33:38078) 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc1e59c700 1 -- 172.21.15.33:0/3176864736 learned_addr learned my addr 172.21.15.33:0/3176864736 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc1e59c700 1 -- 172.21.15.33:0/3176864736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec6860 msgr2=0x55e9aaeccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc1e59c700 1 --2- 172.21.15.33:0/3176864736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec6860 0x55e9aaeccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc1e59c700 1 -- 172.21.15.33:0/3176864736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 msgr2=0x55e9aaed6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc1e59c700 1 --2- 172.21.15.33:0/3176864736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaed6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc1e59c700 1 -- 172.21.15.33:0/3176864736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9aac42b60 con 0x55e9aaec5540 2024-05-08T21:21:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc1e59c700 1 --2- 172.21.15.33:0/3176864736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec5540 0x55e9aaee79a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efc1000c2d0 tx=0x7efc1001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2a4ac7fc7e2d5f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.920+0000 7efc0d7fa700 1 -- 172.21.15.33:0/3176864736 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1009a4a0 con 0x55e9aaec5540 2024-05-08T21:21:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.921+0000 7efc0d7fa700 1 -- 172.21.15.33:0/3176864736 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc1009dbd0 con 0x55e9aaec5540 2024-05-08T21:21:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.921+0000 7efc0d7fa700 1 -- 172.21.15.33:0/3176864736 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc100994f0 con 0x55e9aaec5540 2024-05-08T21:21:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.921+0000 7efc3115cb80 1 -- 172.21.15.33:0/3176864736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec5540 msgr2=0x55e9aaee79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.921+0000 7efc3115cb80 1 --2- 172.21.15.33:0/3176864736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec5540 0x55e9aaee79a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efc1000c2d0 tx=0x7efc1001a040 comp rx=0 tx=0).stop 2024-05-08T21:21:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.921+0000 7efc3115cb80 1 -- 172.21.15.33:0/3176864736 shutdown_connections 2024-05-08T21:21:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.921+0000 7efc3115cb80 1 --2- 172.21.15.33:0/3176864736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec6860 0x55e9aaeccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.921+0000 7efc3115cb80 1 --2- 172.21.15.33:0/3176864736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec5540 0x55e9aaee79a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.921+0000 7efc3115cb80 1 --2- 172.21.15.33:0/3176864736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5ea0 0x55e9aaed6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.921+0000 7efc3115cb80 1 -- 172.21.15.33:0/3176864736 >> 172.21.15.33:0/3176864736 conn(0x55e9aae23210 msgr2=0x55e9aaec6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.921+0000 7efc3115cb80 1 -- 172.21.15.33:0/3176864736 shutdown_connections 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc3115cb80 1 -- 172.21.15.33:0/3176864736 wait complete. 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc3115cb80 1 -- start start 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc3115cb80 1 Processor -- start 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc3115cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5540 0x55e9aaed1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc3115cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5ea0 0x55e9aaed1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc3115cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaed4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc3115cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aaed54f0 con 0x55e9aaec5540 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc3115cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aaed5660 con 0x55e9aaec5ea0 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc3115cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9aaed5820 con 0x55e9aaec6860 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc3115cb80 10 client.0 _ll_register_callbacks cb 0x55e9aadbab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc1e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5540 0x55e9aaed1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc1e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5540 0x55e9aaed1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38088/0 (socket says 172.21.15.33:38088) 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc1e59c700 1 -- 172.21.15.33:0/174460166 learned_addr learned my addr 172.21.15.33:0/174460166 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc1dd9b700 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5ea0 0x55e9aaed1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.922+0000 7efc1ed9d700 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaed4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc1e59c700 1 -- 172.21.15.33:0/174460166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 msgr2=0x55e9aaed4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc1e59c700 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaed4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc1e59c700 1 -- 172.21.15.33:0/174460166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5ea0 msgr2=0x55e9aaed1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc1e59c700 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5ea0 0x55e9aaed1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc1e59c700 1 -- 172.21.15.33:0/174460166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc18009d00 con 0x55e9aaec5540 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc1e59c700 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5540 0x55e9aaed1aa0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efc10099aa0 tx=0x7efc100994c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944e32bb6293005e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc05ffb700 1 -- 172.21.15.33:0/174460166 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc100a2220 con 0x55e9aaec5540 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc3115cb80 1 -- 172.21.15.33:0/174460166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc180074e0 con 0x55e9aaec5540 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc05ffb700 1 -- 172.21.15.33:0/174460166 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7efc10097ee0 con 0x55e9aaec5540 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc05ffb700 1 -- 172.21.15.33:0/174460166 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc100ca560 con 0x55e9aaec5540 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc3115cb80 10 client.16242 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:10.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc3115cb80 1 -- 172.21.15.33:0/174460166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55e9aaddb560 con 0x55e9aaec5540 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.923+0000 7efc05ffb700 1 -- 172.21.15.33:0/174460166 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc180074e0 con 0x55e9aaec5540 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.924+0000 7efc05ffb700 1 -- 172.21.15.33:0/174460166 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efc10098d00 con 0x55e9aaec5540 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.924+0000 7efc3115cb80 10 client.16242 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.924+0000 7efc3115cb80 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.924+0000 7efc3115cb80 20 client.16242 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.924+0000 7efc3115cb80 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9aaf427e0 0x55e9aaf44c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.924+0000 7efc3115cb80 4 client.16242 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.924+0000 7efc3115cb80 1 -- 172.21.15.33:0/174460166 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e9aae0dec0 con 0x55e9aaf427e0 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.924+0000 7efc1dd9b700 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9aaf427e0 0x55e9aaf44c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.925+0000 7efc1dd9b700 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9aaf427e0 0x55e9aaf44c00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c76347e171e445 server_cookie=5f136d6a6c9e540 in_seq=0 out_seq=0 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.925+0000 7efc05ffb700 10 client.16242 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.925+0000 7efc05ffb700 1 -- 172.21.15.33:0/174460166 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55e9aae0dec0 con 0x55e9aaf427e0 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.925+0000 7efc05ffb700 10 client.16242 handle_command_reply: tid=0 2024-05-08T21:21:10.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.925+0000 7efc3115cb80 1 client.16242 shutdown 2024-05-08T21:21:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.925+0000 7efc3115cb80 1 -- 172.21.15.33:0/174460166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5540 msgr2=0x55e9aaed1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.925+0000 7efc3115cb80 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5540 0x55e9aaed1aa0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efc10099aa0 tx=0x7efc100994c0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc3115cb80 1 -- 172.21.15.33:0/174460166 shutdown_connections 2024-05-08T21:21:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc3115cb80 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9aaf427e0 0x55e9aaf44c00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc3115cb80 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9aaec6860 0x55e9aaed4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc3115cb80 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9aaec5540 0x55e9aaed1aa0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc3115cb80 1 --2- 172.21.15.33:0/174460166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9aaec5ea0 0x55e9aaed1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc05ffb700 0 client.16242 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc3115cb80 1 -- 172.21.15.33:0/174460166 >> 172.21.15.33:0/174460166 conn(0x55e9aae23210 msgr2=0x55e9aaeccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc3115cb80 1 -- 172.21.15.33:0/174460166 shutdown_connections 2024-05-08T21:21:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc3115cb80 1 -- 172.21.15.33:0/174460166 wait complete. 2024-05-08T21:21:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc3115cb80 20 client.16242 trim_cache size 0 max 16384 2024-05-08T21:21:10.933 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:21:10.933 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "scrub active (1269 inodes in the stack)", 2024-05-08T21:21:10.933 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:21:10.933 INFO:teuthology.orchestra.run.smithi033.stdout: "f50ed481-9aef-490b-ac79-9eecdf05e253": { 2024-05-08T21:21:10.933 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:21:10.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "f50ed481-9aef-490b-ac79-9eecdf05e253", 2024-05-08T21:21:10.933 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:21:10.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:21:10.933 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:21:10.933 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:21:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc24fa2700 1 -- 172.21.15.33:0/3674110071 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7efc080a11a0 msgr2=0x7efc080a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc24fa2700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7efc080a11a0 0x7efc080a3660 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efc14009df0 tx=0x7efc14009260 comp rx=0 tx=0).stop 2024-05-08T21:21:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc24fa2700 1 -- 172.21.15.33:0/3674110071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc201018e0 msgr2=0x7efc20196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc24fa2700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc201018e0 0x7efc20196db0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7efc1000e9a0 tx=0x7efc1000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc24fa2700 1 -- 172.21.15.33:0/3674110071 shutdown_connections 2024-05-08T21:21:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc24fa2700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc20106b40 0x7efc2019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc24fa2700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc201018e0 0x7efc20196db0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc24fa2700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7efc080a11a0 0x7efc080a3660 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc24fa2700 1 --2- 172.21.15.33:0/3674110071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc20104210 0x7efc201972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:10.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.926+0000 7efc24fa2700 1 -- 172.21.15.33:0/3674110071 >> 172.21.15.33:0/3674110071 conn(0x7efc200fb290 msgr2=0x7efc20102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:10.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.927+0000 7efc24fa2700 1 -- 172.21.15.33:0/3674110071 shutdown_connections 2024-05-08T21:21:10.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:10.927+0000 7efc24fa2700 1 -- 172.21.15.33:0/3674110071 wait complete. 2024-05-08T21:21:10.942 INFO:tasks.cephfs.filesystem:scrub status for tag:f50ed481-9aef-490b-ac79-9eecdf05e253 - {'path': '/', 'tag': 'f50ed481-9aef-490b-ac79-9eecdf05e253', 'options': 'recursive,force'} 2024-05-08T21:21:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: pgmap v356: 129 pgs: 129 active+clean; 49 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-05-08T21:21:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:11.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:11.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: pgmap v356: 129 pgs: 129 active+clean; 49 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-05-08T21:21:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:11.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:11.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:11.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:11.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: pgmap v356: 129 pgs: 129 active+clean; 49 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-05-08T21:21:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:12 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:21:12.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:12 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:12 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:12.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:12 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:12.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:12 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:12.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:12 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:12 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:12 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:12.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:12 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:12.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:12 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:12 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:12 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:12 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:21:12.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:12 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:12 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:12.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:12 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:12.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:12 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:12.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:12 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:12 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:12 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:12.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:12 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:12.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:12 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:12 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:12 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:12 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:21:12.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:12 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:12 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:12.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:12 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:12.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:12 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:12.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:12 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:12 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:12 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:12.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:12 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:12.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:12 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:12 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:12.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:12 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:13.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:13 smithi033 ceph-mon[42528]: pgmap v357: 129 pgs: 129 active+clean; 56 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 199 op/s 2024-05-08T21:21:13.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:13 smithi071 ceph-mon[47074]: pgmap v357: 129 pgs: 129 active+clean; 56 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 199 op/s 2024-05-08T21:21:13.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:13 smithi082 ceph-mon[47512]: pgmap v357: 129 pgs: 129 active+clean; 56 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 199 op/s 2024-05-08T21:21:14.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:14 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:21:14.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:14 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:21:14.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:14 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:21:15.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:15 smithi033 ceph-mon[42528]: pgmap v358: 129 pgs: 129 active+clean; 56 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-05-08T21:21:15.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:21:15.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:21:15.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:21:15.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:15 smithi071 ceph-mon[47074]: pgmap v358: 129 pgs: 129 active+clean; 56 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-05-08T21:21:15.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:21:15.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:21:15.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:21:15.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:15 smithi082 ceph-mon[47512]: pgmap v358: 129 pgs: 129 active+clean; 56 MiB data, 592 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-05-08T21:21:15.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:21:15.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:21:15.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:21:17.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:17 smithi033 ceph-mon[42528]: pgmap v359: 129 pgs: 129 active+clean; 61 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-05-08T21:21:17.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:17 smithi071 ceph-mon[47074]: pgmap v359: 129 pgs: 129 active+clean; 61 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-05-08T21:21:17.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:17 smithi082 ceph-mon[47512]: pgmap v359: 129 pgs: 129 active+clean; 61 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 268 op/s 2024-05-08T21:21:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:18 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:21:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:18 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:21:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:18 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:21:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:18 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:21:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:18 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:21:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:18 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:21:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:18 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:21:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:18 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:21:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:18 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:21:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:18 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:21:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:18 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:21:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:18 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:21:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:18 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:21:18.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:18 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:21:18.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:18 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:21:19.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:19 smithi033 ceph-mon[42528]: pgmap v360: 129 pgs: 129 active+clean; 61 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-05-08T21:21:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:19 smithi071 ceph-mon[47074]: pgmap v360: 129 pgs: 129 active+clean; 61 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-05-08T21:21:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:19 smithi082 ceph-mon[47512]: pgmap v360: 129 pgs: 129 active+clean; 61 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 263 op/s 2024-05-08T21:21:21.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:21 smithi033 ceph-mon[42528]: pgmap v361: 129 pgs: 129 active+clean; 62 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 254 op/s 2024-05-08T21:21:21.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:21 smithi071 ceph-mon[47074]: pgmap v361: 129 pgs: 129 active+clean; 62 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 254 op/s 2024-05-08T21:21:21.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:21 smithi082 ceph-mon[47512]: pgmap v361: 129 pgs: 129 active+clean; 62 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 254 op/s 2024-05-08T21:21:23.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:23 smithi033 ceph-mon[42528]: pgmap v362: 129 pgs: 129 active+clean; 65 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-05-08T21:21:23.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:23 smithi071 ceph-mon[47074]: pgmap v362: 129 pgs: 129 active+clean; 65 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-05-08T21:21:23.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:23 smithi082 ceph-mon[47512]: pgmap v362: 129 pgs: 129 active+clean; 65 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 311 op/s 2024-05-08T21:21:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:25 smithi033 ceph-mon[42528]: pgmap v363: 129 pgs: 129 active+clean; 65 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.1 MiB/s wr, 267 op/s 2024-05-08T21:21:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:25 smithi071 ceph-mon[47074]: pgmap v363: 129 pgs: 129 active+clean; 65 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.1 MiB/s wr, 267 op/s 2024-05-08T21:21:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:25 smithi082 ceph-mon[47512]: pgmap v363: 129 pgs: 129 active+clean; 65 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 1.1 MiB/s wr, 267 op/s 2024-05-08T21:21:27.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:27 smithi033 ceph-mon[42528]: pgmap v364: 129 pgs: 129 active+clean; 75 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 317 op/s 2024-05-08T21:21:27.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:27 smithi071 ceph-mon[47074]: pgmap v364: 129 pgs: 129 active+clean; 75 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 317 op/s 2024-05-08T21:21:27.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:27 smithi082 ceph-mon[47512]: pgmap v364: 129 pgs: 129 active+clean; 75 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 317 op/s 2024-05-08T21:21:29.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:29 smithi033 ceph-mon[42528]: pgmap v365: 129 pgs: 129 active+clean; 76 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 258 op/s 2024-05-08T21:21:29.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:29 smithi071 ceph-mon[47074]: pgmap v365: 129 pgs: 129 active+clean; 76 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 258 op/s 2024-05-08T21:21:29.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:29 smithi082 ceph-mon[47512]: pgmap v365: 129 pgs: 129 active+clean; 76 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 258 op/s 2024-05-08T21:21:31.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:31 smithi033 ceph-mon[42528]: pgmap v366: 129 pgs: 129 active+clean; 81 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 294 op/s 2024-05-08T21:21:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:31 smithi071 ceph-mon[47074]: pgmap v366: 129 pgs: 129 active+clean; 81 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 294 op/s 2024-05-08T21:21:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:31 smithi082 ceph-mon[47512]: pgmap v366: 129 pgs: 129 active+clean; 81 MiB data, 706 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 294 op/s 2024-05-08T21:21:33.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:33 smithi033 ceph-mon[42528]: pgmap v367: 129 pgs: 129 active+clean; 88 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 294 op/s 2024-05-08T21:21:33.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:33 smithi071 ceph-mon[47074]: pgmap v367: 129 pgs: 129 active+clean; 88 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 294 op/s 2024-05-08T21:21:33.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:33 smithi082 ceph-mon[47512]: pgmap v367: 129 pgs: 129 active+clean; 88 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 294 op/s 2024-05-08T21:21:35.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:35 smithi033 ceph-mon[42528]: pgmap v368: 129 pgs: 129 active+clean; 88 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-05-08T21:21:35.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:35 smithi071 ceph-mon[47074]: pgmap v368: 129 pgs: 129 active+clean; 88 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-05-08T21:21:35.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:35 smithi082 ceph-mon[47512]: pgmap v368: 129 pgs: 129 active+clean; 88 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-05-08T21:21:37.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:37 smithi071 ceph-mon[47074]: pgmap v369: 129 pgs: 129 active+clean; 101 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-05-08T21:21:37.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:37 smithi082 ceph-mon[47512]: pgmap v369: 129 pgs: 129 active+clean; 101 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-05-08T21:21:37.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:37 smithi033 ceph-mon[42528]: pgmap v369: 129 pgs: 129 active+clean; 101 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 272 op/s 2024-05-08T21:21:39.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:39 smithi071 ceph-mon[47074]: pgmap v370: 129 pgs: 129 active+clean; 101 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2024-05-08T21:21:39.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:39 smithi082 ceph-mon[47512]: pgmap v370: 129 pgs: 129 active+clean; 101 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2024-05-08T21:21:39.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:39 smithi033 ceph-mon[42528]: pgmap v370: 129 pgs: 129 active+clean; 101 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 213 op/s 2024-05-08T21:21:40.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:21:41.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.105+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/3946340728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cf41018f0 msgr2=0x7f9cf41081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.106+0000 7f9cfc4e5700 1 --2- 172.21.15.33:0/3946340728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cf41018f0 0x7f9cf41081a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00b0d0 tx=0x7f9cec00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.106+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/3946340728 shutdown_connections 2024-05-08T21:21:41.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.106+0000 7f9cfc4e5700 1 --2- 172.21.15.33:0/3946340728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cf41018f0 0x7f9cf41081a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.106+0000 7f9cfc4e5700 1 --2- 172.21.15.33:0/3946340728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf40ffd60 0x7f9cf4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.106+0000 7f9cfc4e5700 1 --2- 172.21.15.33:0/3946340728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf4100f50 0x7f9cf41013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.106+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/3946340728 >> 172.21.15.33:0/3946340728 conn(0x7f9cf40fb290 msgr2=0x7f9cf40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.106+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/3946340728 shutdown_connections 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.106+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/3946340728 wait complete. 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.106+0000 7f9cfc4e5700 1 Processor -- start 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.106+0000 7f9cfc4e5700 1 -- start start 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cfc4e5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cf40ffd60 0x7f9cf4103c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cfc4e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf4100f50 0x7f9cf4102330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cfc4e5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf41018f0 0x7f9cf4102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cfc4e5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf4102f20 con 0x7f9cf4100f50 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cfc4e5700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf4103090 con 0x7f9cf40ffd60 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cfc4e5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf41a0f10 con 0x7f9cf41018f0 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cf9a80700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf4100f50 0x7f9cf4102330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cfa281700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cf40ffd60 0x7f9cf4103c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cf9a80700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf4100f50 0x7f9cf4102330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52580/0 (socket says 172.21.15.33:52580) 2024-05-08T21:21:41.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cf9a80700 1 -- 172.21.15.33:0/439285213 learned_addr learned my addr 172.21.15.33:0/439285213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cfaa82700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf41018f0 0x7f9cf4102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cf9a80700 1 -- 172.21.15.33:0/439285213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf41018f0 msgr2=0x7f9cf4102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cf9a80700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf41018f0 0x7f9cf4102870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cf9a80700 1 -- 172.21.15.33:0/439285213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cf40ffd60 msgr2=0x7f9cf4103c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cf9a80700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cf40ffd60 0x7f9cf4103c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cf9a80700 1 -- 172.21.15.33:0/439285213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec009d00 con 0x7f9cf4100f50 2024-05-08T21:21:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cfa281700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cf40ffd60 0x7f9cf4103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cf9a80700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf4100f50 0x7f9cf4102330 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400e9a0 tx=0x7f9ce400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9ceb7fe700 1 -- 172.21.15.33:0/439285213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4009790 con 0x7f9cf4100f50 2024-05-08T21:21:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/439285213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf41a1110 con 0x7f9cf4100f50 2024-05-08T21:21:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9ceb7fe700 1 -- 172.21.15.33:0/439285213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ce4010430 con 0x7f9cf4100f50 2024-05-08T21:21:41.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.107+0000 7f9ceb7fe700 1 -- 172.21.15.33:0/439285213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce400cea0 con 0x7f9cf4100f50 2024-05-08T21:21:41.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.108+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/439285213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf41a1620 con 0x7f9cf4100f50 2024-05-08T21:21:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.110+0000 7f9ceb7fe700 1 -- 172.21.15.33:0/439285213 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9ce40105a0 con 0x7f9cf4100f50 2024-05-08T21:21:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.110+0000 7f9ceb7fe700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9ce00a1120 0x7f9ce00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.111+0000 7f9cfa281700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9ce00a1120 0x7f9ce00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.111+0000 7f9ceb7fe700 1 -- 172.21.15.33:0/439285213 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9ce4014070 con 0x7f9cf4100f50 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.111+0000 7f9cfa281700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9ce00a1120 0x7f9ce00a35e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4100db0 tx=0x7f9cf0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9d0869fb80 1 Processor -- start 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9d0869fb80 1 -- start start 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9d0869fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57df540 0x5611f57df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9d0869fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 0x5611f57e0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9d0869fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9d0869fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611f5736150 con 0x5611f57dfea0 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9d0869fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611f5733b10 con 0x5611f57df540 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9d0869fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611f572f400 con 0x5611f57e0860 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9cf9a80700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 0x5611f57e0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9cfa281700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57df540 0x5611f57df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9cf9a80700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 0x5611f57e0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52584/0 (socket says 172.21.15.33:52584) 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9cf9a80700 1 -- 172.21.15.33:0/1852714057 learned_addr learned my addr 172.21.15.33:0/1852714057 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.112+0000 7f9cfaa82700 1 --2- 172.21.15.33:0/1852714057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.113+0000 7f9cf9a80700 1 -- 172.21.15.33:0/1852714057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 msgr2=0x5611f57e0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.113+0000 7f9cf9a80700 1 --2- 172.21.15.33:0/1852714057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57e0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.113+0000 7f9cf9a80700 1 -- 172.21.15.33:0/1852714057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57df540 msgr2=0x5611f57df960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.113+0000 7f9cf9a80700 1 --2- 172.21.15.33:0/1852714057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57df540 0x5611f57df960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.113+0000 7f9cf9a80700 1 -- 172.21.15.33:0/1852714057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611f55b03b0 con 0x5611f57dfea0 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.113+0000 7f9cfa281700 1 --2- 172.21.15.33:0/1852714057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57df540 0x5611f57df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.113+0000 7f9cf9a80700 1 --2- 172.21.15.33:0/1852714057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 0x5611f57e0320 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40963f0 tx=0x7f9ce4099920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb07446bc81fd841 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.113+0000 7f9ce8ff9700 1 -- 172.21.15.33:0/1852714057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4099ef0 con 0x5611f57dfea0 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.113+0000 7f9ce8ff9700 1 -- 172.21.15.33:0/1852714057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ce409dd20 con 0x5611f57dfea0 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.113+0000 7f9ce8ff9700 1 -- 172.21.15.33:0/1852714057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40a1ee0 con 0x5611f57dfea0 2024-05-08T21:21:41.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.114+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1852714057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 msgr2=0x5611f57e0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.114+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/1852714057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 0x5611f57e0320 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40963f0 tx=0x7f9ce4099920 comp rx=0 tx=0).stop 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.114+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1852714057 shutdown_connections 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.114+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/1852714057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57e0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.114+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/1852714057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57df540 0x5611f57df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.114+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/1852714057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 0x5611f57e0320 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.114+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1852714057 >> 172.21.15.33:0/1852714057 conn(0x5611f573d210 msgr2=0x5611f57d9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.114+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1852714057 shutdown_connections 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.114+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1852714057 wait complete. 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.114+0000 7f9d0869fb80 1 -- start start 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.114+0000 7f9d0869fb80 1 Processor -- start 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9d0869fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57df540 0x5611f57e8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9d0869fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57dfea0 0x5611f57e9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9d0869fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57f0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9d0869fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611f56cb560 con 0x5611f57df540 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9d0869fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611f56cb200 con 0x5611f57dfea0 2024-05-08T21:21:41.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9d0869fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611f55b26a0 con 0x5611f57e0860 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9d0869fb80 10 client.0 _ll_register_callbacks cb 0x5611f56d4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9cfa281700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57df540 0x5611f57e8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9cfa281700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57df540 0x5611f57e8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52592/0 (socket says 172.21.15.33:52592) 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9cfa281700 1 -- 172.21.15.33:0/1423982693 learned_addr learned my addr 172.21.15.33:0/1423982693 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9cfaa82700 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57f0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9cf9a80700 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57dfea0 0x5611f57e9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9cfa281700 1 -- 172.21.15.33:0/1423982693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 msgr2=0x5611f57f0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9cfa281700 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57f0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9cfa281700 1 -- 172.21.15.33:0/1423982693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57dfea0 msgr2=0x5611f57e9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9cfa281700 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57dfea0 0x5611f57e9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9cfa281700 1 -- 172.21.15.33:0/1423982693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf0017070 con 0x5611f57df540 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9cfa281700 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57df540 0x5611f57e8d10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0003b50 tx=0x7f9cf00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12c9793a877a1c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.115+0000 7f9cddffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9cddffb700 1 -- 172.21.15.33:0/1423982693 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf001fcd0 con 0x5611f57df540 2024-05-08T21:21:41.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9cddffb700 1 -- 172.21.15.33:0/1423982693 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9cf001fe30 con 0x5611f57df540 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1423982693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611f5688270 con 0x5611f57df540 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9cddffb700 1 -- 172.21.15.33:0/1423982693 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cf0004830 con 0x5611f57df540 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9d0869fb80 10 client.16254 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1423982693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f9cf41a1620 con 0x5611f57df540 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9cddffb700 1 -- 172.21.15.33:0/1423982693 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cf0031020 con 0x5611f57df540 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9cddffb700 1 -- 172.21.15.33:0/1423982693 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9cf0005540 con 0x5611f57df540 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9d0869fb80 10 client.16254 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9d0869fb80 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9d0869fb80 20 client.16254 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5611f57f4c20 0x5611f57f7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9d0869fb80 4 client.16254 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.116+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1423982693 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611f57ae810 con 0x5611f57f4c20 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.117+0000 7f9cf9a80700 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5611f57f4c20 0x5611f57f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.117+0000 7f9cf9a80700 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5611f57f4c20 0x5611f57f7040 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1887275e406df87 server_cookie=327e604143ca0e87 in_seq=0 out_seq=0 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.117+0000 7f9cddffb700 10 client.16254 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9cddffb700 1 -- 172.21.15.33:0/1423982693 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5611f57ae810 con 0x5611f57f4c20 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9cddffb700 10 client.16254 handle_command_reply: tid=0 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9d0869fb80 1 client.16254 shutdown 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1423982693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57df540 msgr2=0x5611f57e8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57df540 0x5611f57e8d10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0003b50 tx=0x7f9cf00047c0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1423982693 shutdown_connections 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5611f57f4c20 0x5611f57f7040 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57f0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57df540 0x5611f57e8d10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/1423982693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57dfea0 0x5611f57e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1423982693 >> 172.21.15.33:0/1423982693 conn(0x5611f573d210 msgr2=0x5611f57d95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9cddffb700 0 client.16254 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.118+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1423982693 shutdown_connections 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.119+0000 7f9d0869fb80 1 -- 172.21.15.33:0/1423982693 wait complete. 2024-05-08T21:21:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.119+0000 7f9d0869fb80 20 client.16254 trim_cache size 0 max 16384 2024-05-08T21:21:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9d0869fb80 1 Processor -- start 2024-05-08T21:21:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9d0869fb80 1 -- start start 2024-05-08T21:21:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9d0869fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57df540 0x5611f57d9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9d0869fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57dfea0 0x5611f57f0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9d0869fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57da2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9d0869fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611f57e69f0 con 0x5611f57df540 2024-05-08T21:21:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9d0869fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611f57e7c70 con 0x5611f57dfea0 2024-05-08T21:21:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9d0869fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611f57f4920 con 0x5611f57e0860 2024-05-08T21:21:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9cfaa82700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57da2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9cfa281700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57df540 0x5611f57d9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9cf9a80700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57dfea0 0x5611f57f0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9cfaa82700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57da2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:47870/0 (socket says 172.21.15.33:47870) 2024-05-08T21:21:41.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9cfaa82700 1 -- 172.21.15.33:0/23158798 learned_addr learned my addr 172.21.15.33:0/23158798 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:41.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9cfaa82700 1 -- 172.21.15.33:0/23158798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57dfea0 msgr2=0x5611f57f0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9cfaa82700 1 --2- 172.21.15.33:0/23158798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57dfea0 0x5611f57f0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9cfaa82700 1 -- 172.21.15.33:0/23158798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57df540 msgr2=0x5611f57d9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9cfaa82700 1 --2- 172.21.15.33:0/23158798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57df540 0x5611f57d9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.139+0000 7f9cfaa82700 1 -- 172.21.15.33:0/23158798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611f555cb60 con 0x5611f57e0860 2024-05-08T21:21:41.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.140+0000 7f9cfaa82700 1 --2- 172.21.15.33:0/23158798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57da2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00b0d0 tx=0x7f9cec00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9a527f7ed34e56d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.140+0000 7f9ce97fa700 1 -- 172.21.15.33:0/23158798 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00e040 con 0x5611f57e0860 2024-05-08T21:21:41.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.140+0000 7f9ce97fa700 1 -- 172.21.15.33:0/23158798 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cec0094b0 con 0x5611f57e0860 2024-05-08T21:21:41.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.140+0000 7f9ce97fa700 1 -- 172.21.15.33:0/23158798 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec003dc0 con 0x5611f57e0860 2024-05-08T21:21:41.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.141+0000 7f9d0869fb80 1 -- 172.21.15.33:0/23158798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 msgr2=0x5611f57da2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.141+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/23158798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57da2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00b0d0 tx=0x7f9cec00b040 comp rx=0 tx=0).stop 2024-05-08T21:21:41.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.141+0000 7f9d0869fb80 1 -- 172.21.15.33:0/23158798 shutdown_connections 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.141+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/23158798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57e0860 0x5611f57da2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.141+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/23158798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57df540 0x5611f57d9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.141+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/23158798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57dfea0 0x5611f57f0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.141+0000 7f9d0869fb80 1 -- 172.21.15.33:0/23158798 >> 172.21.15.33:0/23158798 conn(0x5611f573d210 msgr2=0x5611f57fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.141+0000 7f9d0869fb80 1 -- 172.21.15.33:0/23158798 shutdown_connections 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.141+0000 7f9d0869fb80 1 -- 172.21.15.33:0/23158798 wait complete. 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.141+0000 7f9d0869fb80 1 -- start start 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.141+0000 7f9d0869fb80 1 Processor -- start 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9d0869fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57df540 0x5611f57eb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9d0869fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 0x5611f57ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9d0869fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57e0860 0x5611f57eece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9d0869fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611f57ef220 con 0x5611f57dfea0 2024-05-08T21:21:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9d0869fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611f57ef390 con 0x5611f57e0860 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9d0869fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611f57ef550 con 0x5611f57df540 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9d0869fb80 10 client.0 _ll_register_callbacks cb 0x5611f56d4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cf9a80700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 0x5611f57ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cf9a80700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 0x5611f57ebe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52604/0 (socket says 172.21.15.33:52604) 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cf9a80700 1 -- 172.21.15.33:0/3201485292 learned_addr learned my addr 172.21.15.33:0/3201485292 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cfaa82700 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57e0860 0x5611f57eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cfa281700 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57df540 0x5611f57eb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cf9a80700 1 -- 172.21.15.33:0/3201485292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57df540 msgr2=0x5611f57eb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cf9a80700 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57df540 0x5611f57eb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cf9a80700 1 -- 172.21.15.33:0/3201485292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57e0860 msgr2=0x5611f57eece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cf9a80700 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57e0860 0x5611f57eece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cf9a80700 1 -- 172.21.15.33:0/3201485292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf4108af0 con 0x5611f57dfea0 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cfaa82700 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57e0860 0x5611f57eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cf9a80700 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 0x5611f57ebe00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4096960 tx=0x7f9ce40092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deea3a134b703948 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cddffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cddffb700 1 -- 172.21.15.33:0/3201485292 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4099ef0 con 0x5611f57dfea0 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9cddffb700 1 -- 172.21.15.33:0/3201485292 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9ce40a2370 con 0x5611f57dfea0 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.142+0000 7f9d0869fb80 1 -- 172.21.15.33:0/3201485292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ce409dd20 con 0x5611f57dfea0 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.143+0000 7f9cddffb700 1 -- 172.21.15.33:0/3201485292 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ce40b80d0 con 0x5611f57dfea0 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.143+0000 7f9d0869fb80 10 client.16260 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.143+0000 7f9d0869fb80 1 -- 172.21.15.33:0/3201485292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5611f56f5560 con 0x5611f57dfea0 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.143+0000 7f9cddffb700 1 -- 172.21.15.33:0/3201485292 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ce409dd20 con 0x5611f57dfea0 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.143+0000 7f9cddffb700 1 -- 172.21.15.33:0/3201485292 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9ce40a1ee0 con 0x5611f57dfea0 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.143+0000 7f9d0869fb80 10 client.16260 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.143+0000 7f9d0869fb80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.143+0000 7f9d0869fb80 20 client.16260 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.143+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5611f584b690 0x5611f584dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.143+0000 7f9d0869fb80 4 client.16260 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.143+0000 7f9d0869fb80 1 -- 172.21.15.33:0/3201485292 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5611f5727ec0 con 0x5611f584b690 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.143+0000 7f9cfa281700 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5611f584b690 0x5611f584dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9cddffb700 10 client.16260 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9cfa281700 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5611f584b690 0x5611f584dab0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=627b243cfb255de5 server_cookie=5b7d036aeae4225f in_seq=0 out_seq=0 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9cddffb700 1 -- 172.21.15.33:0/3201485292 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5611f5727ec0 con 0x5611f584b690 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9cddffb700 10 client.16260 handle_command_reply: tid=0 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9d0869fb80 1 client.16260 shutdown 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9d0869fb80 1 -- 172.21.15.33:0/3201485292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 msgr2=0x5611f57ebe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 0x5611f57ebe00 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4096960 tx=0x7f9ce40092d0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9d0869fb80 1 -- 172.21.15.33:0/3201485292 shutdown_connections 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5611f584b690 0x5611f584dab0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611f57e0860 0x5611f57eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5611f57df540 0x5611f57eb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9cddffb700 0 client.16260 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9d0869fb80 1 --2- 172.21.15.33:0/3201485292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f57dfea0 0x5611f57ebe00 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9d0869fb80 1 -- 172.21.15.33:0/3201485292 >> 172.21.15.33:0/3201485292 conn(0x5611f573d210 msgr2=0x5611f57e7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.144+0000 7f9d0869fb80 1 -- 172.21.15.33:0/3201485292 shutdown_connections 2024-05-08T21:21:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9d0869fb80 1 -- 172.21.15.33:0/3201485292 wait complete. 2024-05-08T21:21:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9d0869fb80 20 client.16260 trim_cache size 0 max 16384 2024-05-08T21:21:41.152 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:21:41.152 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:21:41.152 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:21:41.152 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:21:41.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/439285213 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9ce00a1120 msgr2=0x7f9ce00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9cfc4e5700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9ce00a1120 0x7f9ce00a35e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4100db0 tx=0x7f9cf0009260 comp rx=0 tx=0).stop 2024-05-08T21:21:41.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/439285213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf4100f50 msgr2=0x7f9cf4102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9cfc4e5700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf4100f50 0x7f9cf4102330 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400e9a0 tx=0x7f9ce400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/439285213 shutdown_connections 2024-05-08T21:21:41.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9cfc4e5700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf41018f0 0x7f9cf4102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9cfc4e5700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9ce00a1120 0x7f9ce00a35e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9cfc4e5700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cf40ffd60 0x7f9cf4103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9cfc4e5700 1 --2- 172.21.15.33:0/439285213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9cf4100f50 0x7f9cf4102330 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/439285213 >> 172.21.15.33:0/439285213 conn(0x7f9cf40fb290 msgr2=0x7f9cf4068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/439285213 shutdown_connections 2024-05-08T21:21:41.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.145+0000 7f9cfc4e5700 1 -- 172.21.15.33:0/439285213 wait complete. 2024-05-08T21:21:41.161 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:21:41.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:21:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.304+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2338145104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fd41042d0 msgr2=0x7f1fd41066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.304+0000 7f1fdb0f7700 1 --2- 172.21.15.33:0/2338145104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fd41042d0 0x7f1fd41066c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009990 tx=0x7f1fc0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.304+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2338145104 shutdown_connections 2024-05-08T21:21:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.304+0000 7f1fdb0f7700 1 --2- 172.21.15.33:0/2338145104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd4106c00 0x7f1fd4108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.304+0000 7f1fdb0f7700 1 --2- 172.21.15.33:0/2338145104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41019a0 0x7f1fd4103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.304+0000 7f1fdb0f7700 1 --2- 172.21.15.33:0/2338145104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fd41042d0 0x7f1fd41066c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.304+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2338145104 >> 172.21.15.33:0/2338145104 conn(0x7f1fd40fb310 msgr2=0x7f1fd40fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.304+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2338145104 shutdown_connections 2024-05-08T21:21:41.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.304+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2338145104 wait complete. 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fdb0f7700 1 Processor -- start 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fdb0f7700 1 -- start start 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fdb0f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41019a0 0x7f1fd4101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fdb0f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fd41042d0 0x7f1fd40ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fdb0f7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd4106c00 0x7f1fd40ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fdb0f7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd4100590 con 0x7f1fd41042d0 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fdb0f7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd4100700 con 0x7f1fd4106c00 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fdb0f7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd41a0da0 con 0x7f1fd41019a0 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fd41042d0 0x7f1fd40ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd8e93700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41019a0 0x7f1fd4101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd8e93700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41019a0 0x7f1fd4101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:47886/0 (socket says 172.21.15.33:47886) 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fd41042d0 0x7f1fd40ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52630/0 (socket says 172.21.15.33:52630) 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd8e93700 1 -- 172.21.15.33:0/2438357799 learned_addr learned my addr 172.21.15.33:0/2438357799 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:41.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd9694700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd4106c00 0x7f1fd40ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd3fff700 1 -- 172.21.15.33:0/2438357799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41019a0 msgr2=0x7f1fd4101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd3fff700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41019a0 0x7f1fd4101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd3fff700 1 -- 172.21.15.33:0/2438357799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd4106c00 msgr2=0x7f1fd40ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd3fff700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd4106c00 0x7f1fd40ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd3fff700 1 -- 172.21.15.33:0/2438357799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc00095f0 con 0x7f1fd41042d0 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd8e93700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41019a0 0x7f1fd4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.305+0000 7f1fd9694700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd4106c00 0x7f1fd40ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.306+0000 7f1fd3fff700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fd41042d0 0x7f1fd40ff950 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0003f60 tx=0x7f1fc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.306+0000 7f1fd1ffb700 1 -- 172.21.15.33:0/2438357799 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc001d070 con 0x7f1fd41042d0 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.306+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2438357799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd41a0fa0 con 0x7f1fd41042d0 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.306+0000 7f1fd1ffb700 1 -- 172.21.15.33:0/2438357799 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1fc000bdb0 con 0x7f1fd41042d0 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.306+0000 7f1fd1ffb700 1 -- 172.21.15.33:0/2438357799 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0022ca0 con 0x7f1fd41042d0 2024-05-08T21:21:41.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.306+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2438357799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd41a13c0 con 0x7f1fd41042d0 2024-05-08T21:21:41.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.307+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2438357799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd40fce90 con 0x7f1fd41042d0 2024-05-08T21:21:41.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.308+0000 7f1fd1ffb700 1 -- 172.21.15.33:0/2438357799 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1fc0022470 con 0x7f1fd41042d0 2024-05-08T21:21:41.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.309+0000 7f1fd1ffb700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1fc40a1120 0x7f1fc40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.309+0000 7f1fd1ffb700 1 -- 172.21.15.33:0/2438357799 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1fc00ce5e0 con 0x7f1fd41042d0 2024-05-08T21:21:41.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.309+0000 7f1fd8e93700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1fc40a1120 0x7f1fc40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.310+0000 7f1fd8e93700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1fc40a1120 0x7f1fc40a35e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8011550 tx=0x7f1fc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.313+0000 7f1fd1ffb700 1 -- 172.21.15.33:0/2438357799 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1fc0097a80 con 0x7f1fd41042d0 2024-05-08T21:21:41.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.618+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2438357799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fd4067d10 con 0x7f1fd41042d0 2024-05-08T21:21:41.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.619+0000 7f1fd1ffb700 1 -- 172.21.15.33:0/2438357799 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10906 (secure 0 0 0) 0x7f1fc009b0a0 con 0x7f1fd41042d0 2024-05-08T21:21:41.620 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:21:41.621 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:20:53.371310+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:21:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.621+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2438357799 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1fc40a1120 msgr2=0x7f1fc40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.621+0000 7f1fdb0f7700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1fc40a1120 0x7f1fc40a35e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8011550 tx=0x7f1fc8009260 comp rx=0 tx=0).stop 2024-05-08T21:21:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.621+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2438357799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fd41042d0 msgr2=0x7f1fd40ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.621+0000 7f1fdb0f7700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fd41042d0 0x7f1fd40ff950 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0003f60 tx=0x7f1fc0004730 comp rx=0 tx=0).stop 2024-05-08T21:21:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.621+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2438357799 shutdown_connections 2024-05-08T21:21:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.621+0000 7f1fdb0f7700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1fc40a1120 0x7f1fc40a35e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.621+0000 7f1fdb0f7700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fd4106c00 0x7f1fd40ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.621+0000 7f1fdb0f7700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fd41019a0 0x7f1fd4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.621+0000 7f1fdb0f7700 1 --2- 172.21.15.33:0/2438357799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fd41042d0 0x7f1fd40ff950 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.621+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2438357799 >> 172.21.15.33:0/2438357799 conn(0x7f1fd40fb310 msgr2=0x7f1fd4103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.621+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2438357799 shutdown_connections 2024-05-08T21:21:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.621+0000 7f1fdb0f7700 1 -- 172.21.15.33:0/2438357799 wait complete. 2024-05-08T21:21:41.625 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 31 2024-05-08T21:21:41.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:21:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.786+0000 7f47837d9700 1 -- 172.21.15.33:0/1020466599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f477c1018f0 msgr2=0x7f477c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.786+0000 7f47837d9700 1 --2- 172.21.15.33:0/1020466599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f477c1018f0 0x7f477c1081a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4774009fc0 tx=0x7f477400b330 comp rx=0 tx=0).stop 2024-05-08T21:21:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.786+0000 7f47837d9700 1 -- 172.21.15.33:0/1020466599 shutdown_connections 2024-05-08T21:21:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.786+0000 7f47837d9700 1 --2- 172.21.15.33:0/1020466599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f477c1018f0 0x7f477c1081a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.786+0000 7f47837d9700 1 --2- 172.21.15.33:0/1020466599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c0ffd60 0x7f477c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.786+0000 7f47837d9700 1 --2- 172.21.15.33:0/1020466599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f477c100f50 0x7f477c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.786+0000 7f47837d9700 1 -- 172.21.15.33:0/1020466599 >> 172.21.15.33:0/1020466599 conn(0x7f477c0fb290 msgr2=0x7f477c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.786+0000 7f47837d9700 1 -- 172.21.15.33:0/1020466599 shutdown_connections 2024-05-08T21:21:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.786+0000 7f47837d9700 1 -- 172.21.15.33:0/1020466599 wait complete. 2024-05-08T21:21:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.787+0000 7f47837d9700 1 Processor -- start 2024-05-08T21:21:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.787+0000 7f47837d9700 1 -- start start 2024-05-08T21:21:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.787+0000 7f47837d9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c0ffd60 0x7f477c196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.787+0000 7f47837d9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f477c100f50 0x7f477c197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.787+0000 7f47837d9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f477c1018f0 0x7f477c19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.787+0000 7f47837d9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c19e930 con 0x7f477c100f50 2024-05-08T21:21:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.787+0000 7f47837d9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c19ea70 con 0x7f477c1018f0 2024-05-08T21:21:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.787+0000 7f47837d9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c19ec00 con 0x7f477c0ffd60 2024-05-08T21:21:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.787+0000 7f4781575700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c0ffd60 0x7f477c196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f4781d76700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f477c1018f0 0x7f477c19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f4781575700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c0ffd60 0x7f477c196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:47904/0 (socket says 172.21.15.33:47904) 2024-05-08T21:21:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f4780d74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f477c100f50 0x7f477c197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f4781575700 1 -- 172.21.15.33:0/3802359352 learned_addr learned my addr 172.21.15.33:0/3802359352 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f4781d76700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f477c1018f0 0x7f477c19e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37202/0 (socket says 172.21.15.33:37202) 2024-05-08T21:21:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f4780d74700 1 -- 172.21.15.33:0/3802359352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c0ffd60 msgr2=0x7f477c196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f4780d74700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c0ffd60 0x7f477c196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f4780d74700 1 -- 172.21.15.33:0/3802359352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f477c1018f0 msgr2=0x7f477c19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f4780d74700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f477c1018f0 0x7f477c19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f4780d74700 1 -- 172.21.15.33:0/3802359352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4774009c40 con 0x7f477c100f50 2024-05-08T21:21:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f4781575700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c0ffd60 0x7f477c196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f4780d74700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f477c100f50 0x7f477c197390 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f477800e9f0 tx=0x7f477800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f476e7fc700 1 -- 172.21.15.33:0/3802359352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778004220 con 0x7f477c100f50 2024-05-08T21:21:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f476e7fc700 1 -- 172.21.15.33:0/3802359352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4778004cc0 con 0x7f477c100f50 2024-05-08T21:21:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f47837d9700 1 -- 172.21.15.33:0/3802359352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477c19eee0 con 0x7f477c100f50 2024-05-08T21:21:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f476e7fc700 1 -- 172.21.15.33:0/3802359352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477800ce00 con 0x7f477c100f50 2024-05-08T21:21:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.788+0000 7f47837d9700 1 -- 172.21.15.33:0/3802359352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c19f310 con 0x7f477c100f50 2024-05-08T21:21:41.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.791+0000 7f476e7fc700 1 -- 172.21.15.33:0/3802359352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4778004380 con 0x7f477c100f50 2024-05-08T21:21:41.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.791+0000 7f476e7fc700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f47680a1120 0x7f47680a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.791+0000 7f4781575700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f47680a1120 0x7f47680a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.791+0000 7f476e7fc700 1 -- 172.21.15.33:0/3802359352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f4778014070 con 0x7f477c100f50 2024-05-08T21:21:41.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.792+0000 7f4781575700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f47680a1120 0x7f47680a35e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4770009fd0 tx=0x7f4770009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.792+0000 7f478f993b80 1 Processor -- start 2024-05-08T21:21:41.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.792+0000 7f478f993b80 1 -- start start 2024-05-08T21:21:41.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f478f993b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 0x561394f9bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f478f993b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 0x561394f9c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f478f993b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9cc70 0x561394f9d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f478f993b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561394eee150 con 0x561394f9c2b0 2024-05-08T21:21:41.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f478f993b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561394eebb10 con 0x561394f9b950 2024-05-08T21:21:41.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f478f993b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561394ee7400 con 0x561394f9cc70 2024-05-08T21:21:41.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f4781575700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 0x561394f9bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f4780d74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 0x561394f9c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f4781d76700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9cc70 0x561394f9d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f4780d74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 0x561394f9c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52662/0 (socket says 172.21.15.33:52662) 2024-05-08T21:21:41.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f4780d74700 1 -- 172.21.15.33:0/1899396866 learned_addr learned my addr 172.21.15.33:0/1899396866 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:41.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f4781575700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 0x561394f9bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37204/0 (socket says 172.21.15.33:37204) 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f4781d76700 1 -- 172.21.15.33:0/1899396866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 msgr2=0x561394f9bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f4781d76700 1 --2- 172.21.15.33:0/1899396866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 0x561394f9bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f4781d76700 1 -- 172.21.15.33:0/1899396866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 msgr2=0x561394f9c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f4781d76700 1 --2- 172.21.15.33:0/1899396866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 0x561394f9c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f4781d76700 1 -- 172.21.15.33:0/1899396866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561394d683b0 con 0x561394f9cc70 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f4781d76700 1 --2- 172.21.15.33:0/1899396866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9cc70 0x561394f9d0f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4774007890 tx=0x7f47740092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff5e2b3081562599 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.793+0000 7f47677fe700 1 -- 172.21.15.33:0/1899396866 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477400e040 con 0x561394f9cc70 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.794+0000 7f47677fe700 1 -- 172.21.15.33:0/1899396866 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4774003a00 con 0x561394f9cc70 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.794+0000 7f47677fe700 1 -- 172.21.15.33:0/1899396866 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477400bce0 con 0x561394f9cc70 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.794+0000 7f478f993b80 1 -- 172.21.15.33:0/1899396866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9cc70 msgr2=0x561394f9d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.794+0000 7f478f993b80 1 --2- 172.21.15.33:0/1899396866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9cc70 0x561394f9d0f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4774007890 tx=0x7f47740092e0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.794+0000 7f478f993b80 1 -- 172.21.15.33:0/1899396866 shutdown_connections 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.794+0000 7f478f993b80 1 --2- 172.21.15.33:0/1899396866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9cc70 0x561394f9d0f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.794+0000 7f478f993b80 1 --2- 172.21.15.33:0/1899396866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 0x561394f9bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.794+0000 7f478f993b80 1 --2- 172.21.15.33:0/1899396866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 0x561394f9c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.794+0000 7f478f993b80 1 -- 172.21.15.33:0/1899396866 >> 172.21.15.33:0/1899396866 conn(0x561394ef5210 msgr2=0x561394f91de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.794+0000 7f478f993b80 1 -- 172.21.15.33:0/1899396866 shutdown_connections 2024-05-08T21:21:41.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.794+0000 7f478f993b80 1 -- 172.21.15.33:0/1899396866 wait complete. 2024-05-08T21:21:41.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.795+0000 7f478f993b80 1 -- start start 2024-05-08T21:21:41.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.795+0000 7f478f993b80 1 Processor -- start 2024-05-08T21:21:41.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.795+0000 7f478f993b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 0x561394fa5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.795+0000 7f478f993b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9c2b0 0x561394fa57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.795+0000 7f478f993b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9cc70 0x561394fac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.795+0000 7f478f993b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561394e83560 con 0x561394f9cc70 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.795+0000 7f478f993b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561394e83200 con 0x561394f9b950 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.795+0000 7f478f993b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561394d6a6a0 con 0x561394f9c2b0 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f478f993b80 10 client.0 _ll_register_callbacks cb 0x561394e8cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4781d76700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9cc70 0x561394fac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4781575700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 0x561394fa5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4781575700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 0x561394fa5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37212/0 (socket says 172.21.15.33:37212) 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4781575700 1 -- 172.21.15.33:0/856750874 learned_addr learned my addr 172.21.15.33:0/856750874 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4780d74700 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9c2b0 0x561394fa57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4781575700 1 -- 172.21.15.33:0/856750874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9c2b0 msgr2=0x561394fa57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4781575700 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9c2b0 0x561394fa57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4781575700 1 -- 172.21.15.33:0/856750874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9cc70 msgr2=0x561394fac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4781575700 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9cc70 0x561394fac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4781575700 1 -- 172.21.15.33:0/856750874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c108af0 con 0x561394f9b950 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4781d76700 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9cc70 0x561394fac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4781575700 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 0x561394fa5290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f47700092e0 tx=0x7f47700047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f470ab8c505e48fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4764ff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.796+0000 7f4764ff9700 1 -- 172.21.15.33:0/856750874 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477001f9c0 con 0x561394f9b950 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f478f993b80 1 -- 172.21.15.33:0/856750874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561394e40270 con 0x561394f9b950 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f4764ff9700 1 -- 172.21.15.33:0/856750874 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f477001fb20 con 0x561394f9b950 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f4764ff9700 1 -- 172.21.15.33:0/856750874 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f477001fce0 con 0x561394f9b950 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f478f993b80 10 client.26147 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f478f993b80 1 -- 172.21.15.33:0/856750874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x561394d14b60 con 0x561394f9b950 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f4764ff9700 1 -- 172.21.15.33:0/856750874 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561394e40270 con 0x561394f9b950 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f4764ff9700 1 -- 172.21.15.33:0/856750874 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f47700273e0 con 0x561394f9b950 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f478f993b80 10 client.26147 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f478f993b80 10 client.26147 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f478f993b80 20 client.26147 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:41.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f478f993b80 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x561394fc2060 0x561394fc4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f478f993b80 4 client.26147 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f478f993b80 1 -- 172.21.15.33:0/856750874 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561394f66810 con 0x561394fc2060 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.797+0000 7f4780d74700 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x561394fc2060 0x561394fc4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.798+0000 7f4780d74700 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x561394fc2060 0x561394fc4480 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6fcc9c43851d70ef server_cookie=75ce3c3a9672a37 in_seq=0 out_seq=0 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.798+0000 7f4764ff9700 10 client.26147 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.798+0000 7f4764ff9700 1 -- 172.21.15.33:0/856750874 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561394f66810 con 0x561394fc2060 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.798+0000 7f4764ff9700 10 client.26147 handle_command_reply: tid=0 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.798+0000 7f478f993b80 1 client.26147 shutdown 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.799+0000 7f478f993b80 1 -- 172.21.15.33:0/856750874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 msgr2=0x561394fa5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.799+0000 7f478f993b80 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 0x561394fa5290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f47700092e0 tx=0x7f47700047c0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.799+0000 7f478f993b80 1 -- 172.21.15.33:0/856750874 shutdown_connections 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.799+0000 7f478f993b80 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9cc70 0x561394fac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.799+0000 7f478f993b80 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9b950 0x561394fa5290 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.799+0000 7f478f993b80 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x561394fc2060 0x561394fc4480 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.799+0000 7f478f993b80 1 --2- 172.21.15.33:0/856750874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9c2b0 0x561394fa57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.799+0000 7f4764ff9700 0 client.26147 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:21:41.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.799+0000 7f478f993b80 1 -- 172.21.15.33:0/856750874 >> 172.21.15.33:0/856750874 conn(0x561394ef5210 msgr2=0x561394f96340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.799+0000 7f478f993b80 1 -- 172.21.15.33:0/856750874 shutdown_connections 2024-05-08T21:21:41.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.799+0000 7f478f993b80 1 -- 172.21.15.33:0/856750874 wait complete. 2024-05-08T21:21:41.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.799+0000 7f478f993b80 20 client.26147 trim_cache size 0 max 16384 2024-05-08T21:21:41.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:41 smithi071 ceph-mon[47074]: pgmap v371: 129 pgs: 129 active+clean; 109 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 261 op/s 2024-05-08T21:21:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.828+0000 7f478f993b80 1 Processor -- start 2024-05-08T21:21:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.828+0000 7f478f993b80 1 -- start start 2024-05-08T21:21:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f478f993b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9b950 0x561394fbde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f478f993b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9c2b0 0x561394fac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f478f993b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9cc70 0x561394f8e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f478f993b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561394f8a0f0 con 0x561394f9b950 2024-05-08T21:21:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f478f993b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561394f8c800 con 0x561394f9c2b0 2024-05-08T21:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f478f993b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561394f8e750 con 0x561394f9cc70 2024-05-08T21:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f4780d74700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9c2b0 0x561394fac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f4781575700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9b950 0x561394fbde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f4780d74700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9c2b0 0x561394fac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37216/0 (socket says 172.21.15.33:37216) 2024-05-08T21:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f4781575700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9b950 0x561394fbde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52678/0 (socket says 172.21.15.33:52678) 2024-05-08T21:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f4780d74700 1 -- 172.21.15.33:0/3984798690 learned_addr learned my addr 172.21.15.33:0/3984798690 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f4781d76700 1 --2- 172.21.15.33:0/3984798690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9cc70 0x561394f8e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f4780d74700 1 -- 172.21.15.33:0/3984798690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9cc70 msgr2=0x561394f8e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f4780d74700 1 --2- 172.21.15.33:0/3984798690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9cc70 0x561394f8e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f4780d74700 1 -- 172.21.15.33:0/3984798690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9b950 msgr2=0x561394fbde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f4780d74700 1 --2- 172.21.15.33:0/3984798690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9b950 0x561394fbde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f4780d74700 1 -- 172.21.15.33:0/3984798690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561394ead560 con 0x561394f9c2b0 2024-05-08T21:21:41.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.829+0000 7f4780d74700 1 --2- 172.21.15.33:0/3984798690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9c2b0 0x561394fac700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f47780ca2e0 tx=0x7f477809c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f0c718c17b47c16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.830+0000 7f4767fff700 1 -- 172.21.15.33:0/3984798690 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477809da60 con 0x561394f9c2b0 2024-05-08T21:21:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.830+0000 7f4767fff700 1 -- 172.21.15.33:0/3984798690 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47780a20b0 con 0x561394f9c2b0 2024-05-08T21:21:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.830+0000 7f4767fff700 1 -- 172.21.15.33:0/3984798690 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47780a1c20 con 0x561394f9c2b0 2024-05-08T21:21:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.831+0000 7f478f993b80 1 -- 172.21.15.33:0/3984798690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9c2b0 msgr2=0x561394fac700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.831+0000 7f478f993b80 1 --2- 172.21.15.33:0/3984798690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9c2b0 0x561394fac700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f47780ca2e0 tx=0x7f477809c040 comp rx=0 tx=0).stop 2024-05-08T21:21:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.831+0000 7f478f993b80 1 -- 172.21.15.33:0/3984798690 shutdown_connections 2024-05-08T21:21:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.831+0000 7f478f993b80 1 --2- 172.21.15.33:0/3984798690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9cc70 0x561394f8e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.831+0000 7f478f993b80 1 --2- 172.21.15.33:0/3984798690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9b950 0x561394fbde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.831+0000 7f478f993b80 1 --2- 172.21.15.33:0/3984798690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9c2b0 0x561394fac700 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.831+0000 7f478f993b80 1 -- 172.21.15.33:0/3984798690 >> 172.21.15.33:0/3984798690 conn(0x561394ef5210 msgr2=0x561394fa32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.831+0000 7f478f993b80 1 -- 172.21.15.33:0/3984798690 shutdown_connections 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.831+0000 7f478f993b80 1 -- 172.21.15.33:0/3984798690 wait complete. 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.831+0000 7f478f993b80 1 -- start start 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.831+0000 7f478f993b80 1 Processor -- start 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f478f993b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9b950 0x561394fa7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f478f993b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 0x561394fa8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f478f993b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9cc70 0x561394fab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f478f993b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561394fab830 con 0x561394f9c2b0 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f478f993b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561394fab9a0 con 0x561394f9cc70 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f478f993b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561394fabb60 con 0x561394f9b950 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f478f993b80 10 client.0 _ll_register_callbacks cb 0x561394e8cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f4780d74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 0x561394fa8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f4780d74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 0x561394fa8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52692/0 (socket says 172.21.15.33:52692) 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f4780d74700 1 -- 172.21.15.33:0/2088497573 learned_addr learned my addr 172.21.15.33:0/2088497573 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f4781575700 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9b950 0x561394fa7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f4781d76700 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9cc70 0x561394fab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f4780d74700 1 -- 172.21.15.33:0/2088497573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9b950 msgr2=0x561394fa7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.832+0000 7f4780d74700 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9b950 0x561394fa7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.833+0000 7f4780d74700 1 -- 172.21.15.33:0/2088497573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9cc70 msgr2=0x561394fab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.833+0000 7f4780d74700 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9cc70 0x561394fab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.833+0000 7f4780d74700 1 -- 172.21.15.33:0/2088497573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c19f310 con 0x561394f9c2b0 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.833+0000 7f4780d74700 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 0x561394fa8370 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4778061040 tx=0x7f477800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b7515af5a16a02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.833+0000 7f4764ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.833+0000 7f4764ff9700 1 -- 172.21.15.33:0/2088497573 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477809da60 con 0x561394f9c2b0 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.833+0000 7f4764ff9700 1 -- 172.21.15.33:0/2088497573 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f47780b2070 con 0x561394f9c2b0 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.833+0000 7f478f993b80 1 -- 172.21.15.33:0/2088497573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4774003a00 con 0x561394f9c2b0 2024-05-08T21:21:41.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.833+0000 7f4764ff9700 1 -- 172.21.15.33:0/2088497573 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4778083080 con 0x561394f9c2b0 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.833+0000 7f478f993b80 10 client.16284 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.833+0000 7f478f993b80 1 -- 172.21.15.33:0/2088497573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x561394d6ccd0 con 0x561394f9c2b0 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.833+0000 7f4764ff9700 1 -- 172.21.15.33:0/2088497573 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47780a8080 con 0x561394f9c2b0 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.834+0000 7f4764ff9700 1 -- 172.21.15.33:0/2088497573 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4778065390 con 0x561394f9c2b0 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.834+0000 7f478f993b80 10 client.16284 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.834+0000 7f478f993b80 10 client.16284 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.834+0000 7f478f993b80 20 client.16284 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.834+0000 7f478f993b80 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x561395018b00 0x56139501af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.834+0000 7f478f993b80 4 client.16284 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.834+0000 7f478f993b80 1 -- 172.21.15.33:0/2088497573 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561394edfec0 con 0x561395018b00 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.834+0000 7f4781575700 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x561395018b00 0x56139501af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.834+0000 7f4781575700 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x561395018b00 0x56139501af20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f69ed09d0eac737d server_cookie=bd70fcb013fa2c08 in_seq=0 out_seq=0 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.834+0000 7f4764ff9700 10 client.16284 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.835+0000 7f4764ff9700 1 -- 172.21.15.33:0/2088497573 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561394edfec0 con 0x561395018b00 2024-05-08T21:21:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.835+0000 7f4764ff9700 10 client.16284 handle_command_reply: tid=0 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.835+0000 7f478f993b80 1 client.16284 shutdown 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f478f993b80 1 -- 172.21.15.33:0/2088497573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 msgr2=0x561394fa8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f478f993b80 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 0x561394fa8370 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4778061040 tx=0x7f477800c7e0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f478f993b80 1 -- 172.21.15.33:0/2088497573 shutdown_connections 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f478f993b80 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561394f9cc70 0x561394fab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f478f993b80 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x561395018b00 0x56139501af20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f478f993b80 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561394f9b950 0x561394fa7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f478f993b80 1 --2- 172.21.15.33:0/2088497573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561394f9c2b0 0x561394fa8370 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f4764ff9700 0 client.16284 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f478f993b80 1 -- 172.21.15.33:0/2088497573 >> 172.21.15.33:0/2088497573 conn(0x561394ef5210 msgr2=0x561394fa4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f478f993b80 1 -- 172.21.15.33:0/2088497573 shutdown_connections 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f478f993b80 1 -- 172.21.15.33:0/2088497573 wait complete. 2024-05-08T21:21:41.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f478f993b80 20 client.16284 trim_cache size 0 max 16384 2024-05-08T21:21:41.843 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:21:41.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f47837d9700 1 -- 172.21.15.33:0/3802359352 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f47680a1120 msgr2=0x7f47680a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f47837d9700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f47680a1120 0x7f47680a35e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4770009fd0 tx=0x7f4770009310 comp rx=0 tx=0).stop 2024-05-08T21:21:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f47837d9700 1 -- 172.21.15.33:0/3802359352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f477c100f50 msgr2=0x7f477c197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f47837d9700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f477c100f50 0x7f477c197390 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f477800e9f0 tx=0x7f477800ed00 comp rx=0 tx=0).stop 2024-05-08T21:21:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f47837d9700 1 -- 172.21.15.33:0/3802359352 shutdown_connections 2024-05-08T21:21:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f47837d9700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f47680a1120 0x7f47680a35e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f47837d9700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f477c1018f0 0x7f477c19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f47837d9700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c0ffd60 0x7f477c196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.836+0000 7f47837d9700 1 --2- 172.21.15.33:0/3802359352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f477c100f50 0x7f477c197390 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.837+0000 7f47837d9700 1 -- 172.21.15.33:0/3802359352 >> 172.21.15.33:0/3802359352 conn(0x7f477c0fb290 msgr2=0x7f477c068160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.837+0000 7f47837d9700 1 -- 172.21.15.33:0/3802359352 shutdown_connections 2024-05-08T21:21:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.837+0000 7f47837d9700 1 -- 172.21.15.33:0/3802359352 wait complete. 2024-05-08T21:21:41.851 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:41 smithi033 ceph-mon[42528]: pgmap v371: 129 pgs: 129 active+clean; 109 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 261 op/s 2024-05-08T21:21:41.852 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:21:41.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:41 smithi082 ceph-mon[47512]: pgmap v371: 129 pgs: 129 active+clean; 109 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 261 op/s 2024-05-08T21:21:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.996+0000 7f038e41e700 1 -- 172.21.15.33:0/4208459588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03880ffd90 msgr2=0x7f03881001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.996+0000 7f038e41e700 1 --2- 172.21.15.33:0/4208459588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03880ffd90 0x7f03881001b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f03780099c0 tx=0x7f0378009cd0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.996+0000 7f038e41e700 1 -- 172.21.15.33:0/4208459588 shutdown_connections 2024-05-08T21:21:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.996+0000 7f038e41e700 1 --2- 172.21.15.33:0/4208459588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0388101920 0x7f03881081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.996+0000 7f038e41e700 1 --2- 172.21.15.33:0/4208459588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03880ffd90 0x7f03881001b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.996+0000 7f038e41e700 1 --2- 172.21.15.33:0/4208459588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0388100f80 0x7f03881013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.996+0000 7f038e41e700 1 -- 172.21.15.33:0/4208459588 >> 172.21.15.33:0/4208459588 conn(0x7f03880fb300 msgr2=0x7f03880fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.996+0000 7f038e41e700 1 -- 172.21.15.33:0/4208459588 shutdown_connections 2024-05-08T21:21:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.996+0000 7f038e41e700 1 -- 172.21.15.33:0/4208459588 wait complete. 2024-05-08T21:21:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.996+0000 7f038e41e700 1 Processor -- start 2024-05-08T21:21:41.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.996+0000 7f038e41e700 1 -- start start 2024-05-08T21:21:42.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f038e41e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03880ffd90 0x7f0388196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f038e41e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0388100f80 0x7f03881972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f038e41e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0388101920 0x7f038819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f038e41e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038819e870 con 0x7f03880ffd90 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f038e41e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038819e9e0 con 0x7f0388101920 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f038e41e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038819eba0 con 0x7f0388100f80 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f0387fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03880ffd90 0x7f0388196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f0387fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03880ffd90 0x7f0388196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52718/0 (socket says 172.21.15.33:52718) 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f0387fff700 1 -- 172.21.15.33:0/3487209040 learned_addr learned my addr 172.21.15.33:0/3487209040 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f03877fe700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0388100f80 0x7f03881972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f038c9bb700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0388101920 0x7f038819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f0387fff700 1 -- 172.21.15.33:0/3487209040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0388100f80 msgr2=0x7f03881972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f0387fff700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0388100f80 0x7f03881972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f0387fff700 1 -- 172.21.15.33:0/3487209040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0388101920 msgr2=0x7f038819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f0387fff700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0388101920 0x7f038819e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.997+0000 7f0387fff700 1 -- 172.21.15.33:0/3487209040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03780095f0 con 0x7f03880ffd90 2024-05-08T21:21:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.998+0000 7f0387fff700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03880ffd90 0x7f0388196d80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0378006010 tx=0x7f037800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.998+0000 7f03857fa700 1 -- 172.21.15.33:0/3487209040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037801d070 con 0x7f03880ffd90 2024-05-08T21:21:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.998+0000 7f038e41e700 1 -- 172.21.15.33:0/3487209040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038819ee20 con 0x7f03880ffd90 2024-05-08T21:21:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.998+0000 7f03857fa700 1 -- 172.21.15.33:0/3487209040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03780054e0 con 0x7f03880ffd90 2024-05-08T21:21:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.998+0000 7f03857fa700 1 -- 172.21.15.33:0/3487209040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378004b50 con 0x7f03880ffd90 2024-05-08T21:21:42.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:41.998+0000 7f038e41e700 1 -- 172.21.15.33:0/3487209040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038819f190 con 0x7f03880ffd90 2024-05-08T21:21:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.000+0000 7f03857fa700 1 -- 172.21.15.33:0/3487209040 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0378004cb0 con 0x7f03880ffd90 2024-05-08T21:21:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.001+0000 7f03857fa700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f03700a1120 0x7f03700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.001+0000 7f03857fa700 1 -- 172.21.15.33:0/3487209040 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f03780cd7f0 con 0x7f03880ffd90 2024-05-08T21:21:42.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.001+0000 7f03877fe700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f03700a1120 0x7f03700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.001+0000 7f03877fe700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f03700a1120 0x7f03700a35e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f037c009c60 tx=0x7f037c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f039a5d8b80 1 Processor -- start 2024-05-08T21:21:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f039a5d8b80 1 -- start start 2024-05-08T21:21:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f039a5d8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9ded70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f039a5d8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9df2b0 0x5571ea9df730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f039a5d8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 0x5571ea9e00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f039a5d8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ea931150 con 0x5571ea9dfc70 2024-05-08T21:21:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f039a5d8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ea92eb10 con 0x5571ea9df2b0 2024-05-08T21:21:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f039a5d8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ea92a400 con 0x5571ea9de950 2024-05-08T21:21:42.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f038c9bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 0x5571ea9e00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f0387fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9ded70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f038c9bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 0x5571ea9e00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52728/0 (socket says 172.21.15.33:52728) 2024-05-08T21:21:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f038c9bb700 1 -- 172.21.15.33:0/1398425040 learned_addr learned my addr 172.21.15.33:0/1398425040 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.002+0000 7f0387fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9ded70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:47976/0 (socket says 172.21.15.33:47976) 2024-05-08T21:21:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.003+0000 7f03877fe700 1 --2- 172.21.15.33:0/1398425040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9df2b0 0x5571ea9df730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.003+0000 7f038c9bb700 1 -- 172.21.15.33:0/1398425040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 msgr2=0x5571ea9ded70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.003+0000 7f038c9bb700 1 --2- 172.21.15.33:0/1398425040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9ded70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.003+0000 7f038c9bb700 1 -- 172.21.15.33:0/1398425040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9df2b0 msgr2=0x5571ea9df730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.003+0000 7f038c9bb700 1 --2- 172.21.15.33:0/1398425040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9df2b0 0x5571ea9df730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.003+0000 7f038c9bb700 1 -- 172.21.15.33:0/1398425040 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571ea7ab3b0 con 0x5571ea9dfc70 2024-05-08T21:21:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.003+0000 7f038c9bb700 1 --2- 172.21.15.33:0/1398425040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 0x5571ea9e00f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0380011680 tx=0x7f0380009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0815c6217ad8c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.003+0000 7f036e7fc700 1 -- 172.21.15.33:0/1398425040 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038000e040 con 0x5571ea9dfc70 2024-05-08T21:21:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.003+0000 7f036e7fc700 1 -- 172.21.15.33:0/1398425040 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f038000f040 con 0x5571ea9dfc70 2024-05-08T21:21:42.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.003+0000 7f036e7fc700 1 -- 172.21.15.33:0/1398425040 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03800046d0 con 0x5571ea9dfc70 2024-05-08T21:21:42.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.004+0000 7f039a5d8b80 1 -- 172.21.15.33:0/1398425040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 msgr2=0x5571ea9e00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.004+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/1398425040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 0x5571ea9e00f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0380011680 tx=0x7f0380009920 comp rx=0 tx=0).stop 2024-05-08T21:21:42.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.004+0000 7f039a5d8b80 1 -- 172.21.15.33:0/1398425040 shutdown_connections 2024-05-08T21:21:42.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.004+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/1398425040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 0x5571ea9e00f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.004+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/1398425040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9ded70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.004+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/1398425040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9df2b0 0x5571ea9df730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.004+0000 7f039a5d8b80 1 -- 172.21.15.33:0/1398425040 >> 172.21.15.33:0/1398425040 conn(0x5571ea938210 msgr2=0x5571ea9d4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.004+0000 7f039a5d8b80 1 -- 172.21.15.33:0/1398425040 shutdown_connections 2024-05-08T21:21:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.004+0000 7f039a5d8b80 1 -- 172.21.15.33:0/1398425040 wait complete. 2024-05-08T21:21:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.004+0000 7f039a5d8b80 1 -- start start 2024-05-08T21:21:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f039a5d8b80 1 Processor -- start 2024-05-08T21:21:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f039a5d8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9de950 0x5571ea9e8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f039a5d8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9df2b0 0x5571ea9e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f039a5d8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9dfc70 0x5571ea9ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f039a5d8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ea8c6560 con 0x5571ea9de950 2024-05-08T21:21:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f039a5d8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ea8c6200 con 0x5571ea9dfc70 2024-05-08T21:21:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f039a5d8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ea7ad6a0 con 0x5571ea9df2b0 2024-05-08T21:21:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f039a5d8b80 10 client.0 _ll_register_callbacks cb 0x5571ea8cfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f0387fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9de950 0x5571ea9e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f0387fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9de950 0x5571ea9e8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52730/0 (socket says 172.21.15.33:52730) 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f0387fff700 1 -- 172.21.15.33:0/2869741928 learned_addr learned my addr 172.21.15.33:0/2869741928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f03877fe700 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9df2b0 0x5571ea9e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f038c9bb700 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9dfc70 0x5571ea9ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f03877fe700 1 -- 172.21.15.33:0/2869741928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9dfc70 msgr2=0x5571ea9ef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f03877fe700 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9dfc70 0x5571ea9ef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f03877fe700 1 -- 172.21.15.33:0/2869741928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9de950 msgr2=0x5571ea9e8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f03877fe700 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9de950 0x5571ea9e8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.005+0000 7f03877fe700 1 -- 172.21.15.33:0/2869741928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0380011330 con 0x5571ea9df2b0 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f03877fe700 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9df2b0 0x5571ea9e8760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f037c011ab0 tx=0x7f037c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d72267eacb2e00a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f03677fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f03677fe700 1 -- 172.21.15.33:0/2869741928 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c022070 con 0x5571ea9df2b0 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2869741928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571ea883270 con 0x5571ea9df2b0 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f03677fe700 1 -- 172.21.15.33:0/2869741928 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f037c027420 con 0x5571ea9df2b0 2024-05-08T21:21:42.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f03677fe700 1 -- 172.21.15.33:0/2869741928 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f037c004560 con 0x5571ea9df2b0 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f039a5d8b80 10 client.26185 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2869741928 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5571ea757b60 con 0x5571ea9df2b0 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f03677fe700 1 -- 172.21.15.33:0/2869741928 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5571ea883270 con 0x5571ea9df2b0 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f03677fe700 1 -- 172.21.15.33:0/2869741928 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f037c01c5c0 con 0x5571ea9df2b0 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f039a5d8b80 10 client.26185 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f039a5d8b80 10 client.26185 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f039a5d8b80 20 client.26185 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5571ea9f4060 0x5571ea9f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f039a5d8b80 4 client.26185 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.006+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2869741928 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571ea9a9810 con 0x5571ea9f4060 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.007+0000 7f0387fff700 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5571ea9f4060 0x5571ea9f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.007+0000 7f03677fe700 10 client.26185 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.007+0000 7f0387fff700 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5571ea9f4060 0x5571ea9f6480 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=156998d039b93563 server_cookie=b9b717f360315293 in_seq=0 out_seq=0 2024-05-08T21:21:42.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f03677fe700 1 -- 172.21.15.33:0/2869741928 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5571ea9a9810 con 0x5571ea9f4060 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f03677fe700 10 client.26185 handle_command_reply: tid=0 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f039a5d8b80 1 client.26185 shutdown 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2869741928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9df2b0 msgr2=0x5571ea9e8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9df2b0 0x5571ea9e8760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f037c011ab0 tx=0x7f037c0047c0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2869741928 shutdown_connections 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9dfc70 0x5571ea9ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9de950 0x5571ea9e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5571ea9f4060 0x5571ea9f6480 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/2869741928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9df2b0 0x5571ea9e8760 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2869741928 >> 172.21.15.33:0/2869741928 conn(0x5571ea938210 msgr2=0x5571ea9d9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f03677fe700 0 client.26185 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2869741928 shutdown_connections 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2869741928 wait complete. 2024-05-08T21:21:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.008+0000 7f039a5d8b80 20 client.26185 trim_cache size 0 max 16384 2024-05-08T21:21:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f039a5d8b80 1 Processor -- start 2024-05-08T21:21:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f039a5d8b80 1 -- start start 2024-05-08T21:21:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f039a5d8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f039a5d8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9df2b0 0x5571ea9ef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f039a5d8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 0x5571ea9e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f039a5d8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ea9e5f50 con 0x5571ea9dfc70 2024-05-08T21:21:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f039a5d8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ea9d60d0 con 0x5571ea9df2b0 2024-05-08T21:21:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f039a5d8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ea9cfa90 con 0x5571ea9de950 2024-05-08T21:21:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f038c9bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 0x5571ea9e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f0387fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f03877fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9df2b0 0x5571ea9ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f038c9bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 0x5571ea9e62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52744/0 (socket says 172.21.15.33:52744) 2024-05-08T21:21:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f0387fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9e6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:47984/0 (socket says 172.21.15.33:47984) 2024-05-08T21:21:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f038c9bb700 1 -- 172.21.15.33:0/3938052496 learned_addr learned my addr 172.21.15.33:0/3938052496 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f038c9bb700 1 -- 172.21.15.33:0/3938052496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 msgr2=0x5571ea9e6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f038c9bb700 1 --2- 172.21.15.33:0/3938052496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9e6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f038c9bb700 1 -- 172.21.15.33:0/3938052496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9df2b0 msgr2=0x5571ea9ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f038c9bb700 1 --2- 172.21.15.33:0/3938052496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9df2b0 0x5571ea9ef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.023+0000 7f038c9bb700 1 -- 172.21.15.33:0/3938052496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571ea8f0560 con 0x5571ea9dfc70 2024-05-08T21:21:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.024+0000 7f0387fff700 1 --2- 172.21.15.33:0/3938052496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.024+0000 7f038c9bb700 1 --2- 172.21.15.33:0/3938052496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 0x5571ea9e62a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f03800060b0 tx=0x7f0380009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7abc39b5cc194fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.024+0000 7f036effd700 1 -- 172.21.15.33:0/3938052496 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038000e050 con 0x5571ea9dfc70 2024-05-08T21:21:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.024+0000 7f036effd700 1 -- 172.21.15.33:0/3938052496 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0380003e90 con 0x5571ea9dfc70 2024-05-08T21:21:42.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.024+0000 7f036effd700 1 -- 172.21.15.33:0/3938052496 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380005950 con 0x5571ea9dfc70 2024-05-08T21:21:42.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.025+0000 7f039a5d8b80 1 -- 172.21.15.33:0/3938052496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 msgr2=0x5571ea9e62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.025+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/3938052496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 0x5571ea9e62a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f03800060b0 tx=0x7f0380009640 comp rx=0 tx=0).stop 2024-05-08T21:21:42.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.025+0000 7f039a5d8b80 1 -- 172.21.15.33:0/3938052496 shutdown_connections 2024-05-08T21:21:42.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.025+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/3938052496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9dfc70 0x5571ea9e62a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.025+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/3938052496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.025+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/3938052496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9df2b0 0x5571ea9ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.025+0000 7f039a5d8b80 1 -- 172.21.15.33:0/3938052496 >> 172.21.15.33:0/3938052496 conn(0x5571ea938210 msgr2=0x5571ea9e0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.025+0000 7f039a5d8b80 1 -- 172.21.15.33:0/3938052496 shutdown_connections 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.025+0000 7f039a5d8b80 1 -- 172.21.15.33:0/3938052496 wait complete. 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.025+0000 7f039a5d8b80 1 -- start start 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f039a5d8b80 1 Processor -- start 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f039a5d8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f039a5d8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9df2b0 0x5571ea9eb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f039a5d8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9dfc70 0x5571ea9ee460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f039a5d8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ea9ee9a0 con 0x5571ea9df2b0 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f039a5d8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ea9eeb10 con 0x5571ea9dfc70 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f039a5d8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571ea9eecd0 con 0x5571ea9de950 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f039a5d8b80 10 client.0 _ll_register_callbacks cb 0x5571ea8cfb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f03877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9df2b0 0x5571ea9eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f03877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9df2b0 0x5571ea9eb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52754/0 (socket says 172.21.15.33:52754) 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f03877fe700 1 -- 172.21.15.33:0/2134106830 learned_addr learned my addr 172.21.15.33:0/2134106830 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f038c9bb700 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9dfc70 0x5571ea9ee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.026+0000 7f0387fff700 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f038c9bb700 1 -- 172.21.15.33:0/2134106830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 msgr2=0x5571ea9eaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f038c9bb700 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9eaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f038c9bb700 1 -- 172.21.15.33:0/2134106830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9df2b0 msgr2=0x5571ea9eb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f038c9bb700 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9df2b0 0x5571ea9eb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f038c9bb700 1 -- 172.21.15.33:0/2134106830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038819f190 con 0x5571ea9dfc70 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f03877fe700 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9df2b0 0x5571ea9eb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f038c9bb700 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9dfc70 0x5571ea9ee460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0380011c70 tx=0x7f0380009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f5b1c90eff63fcb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f03677fe700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f03677fe700 1 -- 172.21.15.33:0/2134106830 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0380005860 con 0x5571ea9dfc70 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2134106830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f038000f040 con 0x5571ea9dfc70 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f03677fe700 1 -- 172.21.15.33:0/2134106830 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f03800059c0 con 0x5571ea9dfc70 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f03677fe700 1 -- 172.21.15.33:0/2134106830 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f038000e020 con 0x5571ea9dfc70 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f039a5d8b80 10 client.26171 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.027+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2134106830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5571ea7ab3b0 con 0x5571ea9dfc70 2024-05-08T21:21:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f03677fe700 1 -- 172.21.15.33:0/2134106830 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f038000f040 con 0x5571ea9dfc70 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f03677fe700 1 -- 172.21.15.33:0/2134106830 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0380027590 con 0x5571ea9dfc70 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f039a5d8b80 10 client.26171 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f039a5d8b80 10 client.26171 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f039a5d8b80 20 client.26171 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5571eaa4abe0 0x5571eaa4d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f039a5d8b80 4 client.26171 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2134106830 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5571ea922ec0 con 0x5571eaa4abe0 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f0387fff700 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5571eaa4abe0 0x5571eaa4d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f0387fff700 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5571eaa4abe0 0x5571eaa4d000 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d09ffeb92d9bb2f6 server_cookie=ee8deb4c1471aecd in_seq=0 out_seq=0 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f03677fe700 10 client.26171 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f03677fe700 1 -- 172.21.15.33:0/2134106830 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5571ea922ec0 con 0x5571eaa4abe0 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.028+0000 7f03677fe700 10 client.26171 handle_command_reply: tid=0 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f039a5d8b80 1 client.26171 shutdown 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2134106830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9dfc70 msgr2=0x5571ea9ee460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9dfc70 0x5571ea9ee460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0380011c70 tx=0x7f0380009bc0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2134106830 shutdown_connections 2024-05-08T21:21:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571ea9dfc70 0x5571ea9ee460 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5571eaa4abe0 0x5571eaa4d000 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5571ea9de950 0x5571ea9eaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f039a5d8b80 1 --2- 172.21.15.33:0/2134106830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571ea9df2b0 0x5571ea9eb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f03677fe700 0 client.26171 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:21:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2134106830 >> 172.21.15.33:0/2134106830 conn(0x5571ea938210 msgr2=0x5571ea9d4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2134106830 shutdown_connections 2024-05-08T21:21:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f039a5d8b80 1 -- 172.21.15.33:0/2134106830 wait complete. 2024-05-08T21:21:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.029+0000 7f039a5d8b80 20 client.26171 trim_cache size 0 max 16384 2024-05-08T21:21:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:21:42.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.030+0000 7f038e41e700 1 -- 172.21.15.33:0/3487209040 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f03700a1120 msgr2=0x7f03700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.030+0000 7f038e41e700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f03700a1120 0x7f03700a35e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f037c009c60 tx=0x7f037c009260 comp rx=0 tx=0).stop 2024-05-08T21:21:42.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.030+0000 7f038e41e700 1 -- 172.21.15.33:0/3487209040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03880ffd90 msgr2=0x7f0388196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.030+0000 7f038e41e700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03880ffd90 0x7f0388196d80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0378006010 tx=0x7f037800f690 comp rx=0 tx=0).stop 2024-05-08T21:21:42.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.030+0000 7f038e41e700 1 -- 172.21.15.33:0/3487209040 shutdown_connections 2024-05-08T21:21:42.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.030+0000 7f038e41e700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f03700a1120 0x7f03700a35e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.030+0000 7f038e41e700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0388101920 0x7f038819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.030+0000 7f038e41e700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03880ffd90 0x7f0388196d80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.030+0000 7f038e41e700 1 --2- 172.21.15.33:0/3487209040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0388100f80 0x7f03881972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.030+0000 7f038e41e700 1 -- 172.21.15.33:0/3487209040 >> 172.21.15.33:0/3487209040 conn(0x7f03880fb300 msgr2=0x7f03880690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.030+0000 7f038e41e700 1 -- 172.21.15.33:0/3487209040 shutdown_connections 2024-05-08T21:21:42.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.030+0000 7f038e41e700 1 -- 172.21.15.33:0/3487209040 wait complete. 2024-05-08T21:21:42.045 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:21:42.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.203+0000 7fe986c1c700 1 -- 172.21.15.33:0/2423572997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9801013e0 msgr2=0x7fe980101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.203+0000 7fe986c1c700 1 --2- 172.21.15.33:0/2423572997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9801013e0 0x7fe980101800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe974009990 tx=0x7fe974009ca0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.203+0000 7fe986c1c700 1 -- 172.21.15.33:0/2423572997 shutdown_connections 2024-05-08T21:21:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.203+0000 7fe986c1c700 1 --2- 172.21.15.33:0/2423572997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe980100960 0x7fe980108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.203+0000 7fe986c1c700 1 --2- 172.21.15.33:0/2423572997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9801013e0 0x7fe980101800 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.203+0000 7fe986c1c700 1 --2- 172.21.15.33:0/2423572997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9800fffa0 0x7fe980100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.203+0000 7fe986c1c700 1 -- 172.21.15.33:0/2423572997 >> 172.21.15.33:0/2423572997 conn(0x7fe9800faa50 msgr2=0x7fe9800fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.203+0000 7fe986c1c700 1 -- 172.21.15.33:0/2423572997 shutdown_connections 2024-05-08T21:21:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.203+0000 7fe986c1c700 1 -- 172.21.15.33:0/2423572997 wait complete. 2024-05-08T21:21:42.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.204+0000 7fe986c1c700 1 Processor -- start 2024-05-08T21:21:42.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.204+0000 7fe986c1c700 1 -- start start 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe986c1c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9800fffa0 0x7fe980195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe986c1c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe980100960 0x7fe980196140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe986c1c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9801013e0 0x7fe98019d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe986c1c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98019d730 con 0x7fe9801013e0 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe986c1c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98019d870 con 0x7fe980100960 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe986c1c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98019da00 con 0x7fe9800fffa0 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe9849b8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9800fffa0 0x7fe980195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe9851b9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9801013e0 0x7fe98019d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe9849b8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9800fffa0 0x7fe980195c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:48018/0 (socket says 172.21.15.33:48018) 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe9849b8700 1 -- 172.21.15.33:0/2087633378 learned_addr learned my addr 172.21.15.33:0/2087633378 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe97ffff700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe980100960 0x7fe980196140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe9849b8700 1 -- 172.21.15.33:0/2087633378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe980100960 msgr2=0x7fe980196140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe9849b8700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe980100960 0x7fe980196140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe9849b8700 1 -- 172.21.15.33:0/2087633378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9801013e0 msgr2=0x7fe98019d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe9849b8700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9801013e0 0x7fe98019d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe9849b8700 1 -- 172.21.15.33:0/2087633378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9740095f0 con 0x7fe9800fffa0 2024-05-08T21:21:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe97ffff700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe980100960 0x7fe980196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.205+0000 7fe9849b8700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9800fffa0 0x7fe980195c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe974004800 tx=0x7fe974009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.206+0000 7fe97dffb700 1 -- 172.21.15.33:0/2087633378 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97400b710 con 0x7fe9800fffa0 2024-05-08T21:21:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.206+0000 7fe97dffb700 1 -- 172.21.15.33:0/2087633378 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe97400bd00 con 0x7fe9800fffa0 2024-05-08T21:21:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.206+0000 7fe986c1c700 1 -- 172.21.15.33:0/2087633378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9800fee90 con 0x7fe9800fffa0 2024-05-08T21:21:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.206+0000 7fe97dffb700 1 -- 172.21.15.33:0/2087633378 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974019cd0 con 0x7fe9800fffa0 2024-05-08T21:21:42.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.206+0000 7fe986c1c700 1 -- 172.21.15.33:0/2087633378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9800ff390 con 0x7fe9800fffa0 2024-05-08T21:21:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.208+0000 7fe97dffb700 1 -- 172.21.15.33:0/2087633378 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe974019510 con 0x7fe9800fffa0 2024-05-08T21:21:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.209+0000 7fe97dffb700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe9700a11a0 0x7fe9700a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.209+0000 7fe97dffb700 1 -- 172.21.15.33:0/2087633378 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe9740ce510 con 0x7fe9800fffa0 2024-05-08T21:21:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.209+0000 7fe97ffff700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe9700a11a0 0x7fe9700a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.209+0000 7fe97ffff700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe9700a11a0 0x7fe9700a3660 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe96c011490 tx=0x7fe96c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe992dd6b80 1 Processor -- start 2024-05-08T21:21:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe992dd6b80 1 -- start start 2024-05-08T21:21:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe992dd6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da5540 0x55bd87da5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe992dd6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87da6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe992dd6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da6860 0x55bd87da6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe992dd6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd87cfc150 con 0x55bd87da5ea0 2024-05-08T21:21:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe992dd6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd87cf9b10 con 0x55bd87da5540 2024-05-08T21:21:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe992dd6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd87cf5400 con 0x55bd87da6860 2024-05-08T21:21:42.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe9849b8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da5540 0x55bd87da5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe97ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87da6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe9849b8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da5540 0x55bd87da5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37302/0 (socket says 172.21.15.33:37302) 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe9849b8700 1 -- 172.21.15.33:0/2114028424 learned_addr learned my addr 172.21.15.33:0/2114028424 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe97ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87da6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52766/0 (socket says 172.21.15.33:52766) 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe9851b9700 1 --2- 172.21.15.33:0/2114028424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da6860 0x55bd87da6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe9849b8700 1 -- 172.21.15.33:0/2114028424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da6860 msgr2=0x55bd87da6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe9849b8700 1 --2- 172.21.15.33:0/2114028424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da6860 0x55bd87da6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe9849b8700 1 -- 172.21.15.33:0/2114028424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 msgr2=0x55bd87da6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.210+0000 7fe9849b8700 1 --2- 172.21.15.33:0/2114028424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87da6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.211+0000 7fe9849b8700 1 -- 172.21.15.33:0/2114028424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd87b763b0 con 0x55bd87da5540 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.211+0000 7fe97ffff700 1 --2- 172.21.15.33:0/2114028424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87da6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.211+0000 7fe9849b8700 1 --2- 172.21.15.33:0/2114028424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da5540 0x55bd87da5960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe9740a3530 tx=0x7fe974005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef777f375ea97e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.211+0000 7fe96affd700 1 -- 172.21.15.33:0/2114028424 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97409afd0 con 0x55bd87da5540 2024-05-08T21:21:42.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.211+0000 7fe96affd700 1 -- 172.21.15.33:0/2114028424 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe974016e80 con 0x55bd87da5540 2024-05-08T21:21:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.211+0000 7fe96affd700 1 -- 172.21.15.33:0/2114028424 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9740a3a60 con 0x55bd87da5540 2024-05-08T21:21:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.212+0000 7fe992dd6b80 1 -- 172.21.15.33:0/2114028424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da5540 msgr2=0x55bd87da5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.212+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/2114028424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da5540 0x55bd87da5960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe9740a3530 tx=0x7fe974005e80 comp rx=0 tx=0).stop 2024-05-08T21:21:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.212+0000 7fe992dd6b80 1 -- 172.21.15.33:0/2114028424 shutdown_connections 2024-05-08T21:21:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.212+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/2114028424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da6860 0x55bd87da6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.212+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/2114028424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da5540 0x55bd87da5960 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.212+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/2114028424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87da6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.212+0000 7fe992dd6b80 1 -- 172.21.15.33:0/2114028424 >> 172.21.15.33:0/2114028424 conn(0x55bd87d03210 msgr2=0x55bd87d9fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.212+0000 7fe992dd6b80 1 -- 172.21.15.33:0/2114028424 shutdown_connections 2024-05-08T21:21:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.212+0000 7fe992dd6b80 1 -- 172.21.15.33:0/2114028424 wait complete. 2024-05-08T21:21:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.212+0000 7fe992dd6b80 1 -- start start 2024-05-08T21:21:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.212+0000 7fe992dd6b80 1 Processor -- start 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe992dd6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87daeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe992dd6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87daf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe992dd6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da6860 0x55bd87db6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe992dd6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd87c91560 con 0x55bd87da5ea0 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe992dd6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd87c91200 con 0x55bd87da6860 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe992dd6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd87b786a0 con 0x55bd87da5540 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe992dd6b80 10 client.0 _ll_register_callbacks cb 0x55bd87c9ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe97ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87daf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe9851b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da6860 0x55bd87db6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe97ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87daf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52778/0 (socket says 172.21.15.33:52778) 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe9849b8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87daeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe97ffff700 1 -- 172.21.15.33:0/4010433881 learned_addr learned my addr 172.21.15.33:0/4010433881 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe9851b9700 1 -- 172.21.15.33:0/4010433881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 msgr2=0x55bd87daeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe9851b9700 1 --2- 172.21.15.33:0/4010433881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87daeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe9851b9700 1 -- 172.21.15.33:0/4010433881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 msgr2=0x55bd87daf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe9851b9700 1 --2- 172.21.15.33:0/4010433881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87daf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe9851b9700 1 -- 172.21.15.33:0/4010433881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96c017070 con 0x55bd87da6860 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe97ffff700 1 --2- 172.21.15.33:0/4010433881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87daf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe9851b9700 1 --2- 172.21.15.33:0/4010433881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da6860 0x55bd87db6040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe978010fc0 tx=0x7fe978016490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f890d5f3fa2bcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.213+0000 7fe95ffff700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe95ffff700 1 -- 172.21.15.33:0/4010433881 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978015070 con 0x55bd87da6860 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe95ffff700 1 -- 172.21.15.33:0/4010433881 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe978016a90 con 0x55bd87da6860 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe992dd6b80 1 -- 172.21.15.33:0/4010433881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd87c4e270 con 0x55bd87da6860 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe95ffff700 1 -- 172.21.15.33:0/4010433881 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe97800e040 con 0x55bd87da6860 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe992dd6b80 10 client.26183 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe992dd6b80 1 -- 172.21.15.33:0/4010433881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fe9800ff390 con 0x55bd87da6860 2024-05-08T21:21:42.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe95ffff700 1 -- 172.21.15.33:0/4010433881 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bd87c4e270 con 0x55bd87da6860 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe95ffff700 1 -- 172.21.15.33:0/4010433881 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe9780040b0 con 0x55bd87da6860 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe992dd6b80 10 client.26183 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe992dd6b80 10 client.26183 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe992dd6b80 20 client.26183 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/4010433881 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55bd87dba970 0x55bd87dbcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe992dd6b80 4 client.26183 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.214+0000 7fe992dd6b80 1 -- 172.21.15.33:0/4010433881 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd87d74810 con 0x55bd87dba970 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.215+0000 7fe9849b8700 1 --2- 172.21.15.33:0/4010433881 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55bd87dba970 0x55bd87dbcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.215+0000 7fe9849b8700 1 --2- 172.21.15.33:0/4010433881 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55bd87dba970 0x55bd87dbcd90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=707a6e08a9b3e393 server_cookie=ac36c1594cc72c90 in_seq=0 out_seq=0 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.215+0000 7fe95ffff700 10 client.26183 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.215+0000 7fe95ffff700 1 -- 172.21.15.33:0/4010433881 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bd87d74810 con 0x55bd87dba970 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.215+0000 7fe95ffff700 10 client.26183 handle_command_reply: tid=0 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe992dd6b80 1 client.26183 shutdown 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe992dd6b80 1 -- 172.21.15.33:0/4010433881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da6860 msgr2=0x55bd87db6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/4010433881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da6860 0x55bd87db6040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe978010fc0 tx=0x7fe978016490 comp rx=0 tx=0).stop 2024-05-08T21:21:42.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe992dd6b80 1 -- 172.21.15.33:0/4010433881 shutdown_connections 2024-05-08T21:21:42.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/4010433881 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55bd87dba970 0x55bd87dbcd90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/4010433881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da6860 0x55bd87db6040 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/4010433881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87daeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/4010433881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87daf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe95ffff700 0 client.26183 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:21:42.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe992dd6b80 1 -- 172.21.15.33:0/4010433881 >> 172.21.15.33:0/4010433881 conn(0x55bd87d03210 msgr2=0x55bd87d9f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe992dd6b80 1 -- 172.21.15.33:0/4010433881 shutdown_connections 2024-05-08T21:21:42.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe992dd6b80 1 -- 172.21.15.33:0/4010433881 wait complete. 2024-05-08T21:21:42.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.216+0000 7fe992dd6b80 20 client.26183 trim_cache size 0 max 16384 2024-05-08T21:21:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe992dd6b80 1 Processor -- start 2024-05-08T21:21:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe992dd6b80 1 -- start start 2024-05-08T21:21:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe992dd6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87db66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe992dd6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da5ea0 0x55bd87db6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe992dd6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da6860 0x55bd87dadfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe992dd6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd87dae660 con 0x55bd87da6860 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe992dd6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd87d9cdc0 con 0x55bd87da5ea0 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe992dd6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd87db7e00 con 0x55bd87da5540 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe9849b8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87db66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe9851b9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da6860 0x55bd87dadfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe97ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da5ea0 0x55bd87db6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe9849b8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87db66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:48046/0 (socket says 172.21.15.33:48046) 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe9851b9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da6860 0x55bd87dadfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52782/0 (socket says 172.21.15.33:52782) 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.232+0000 7fe9849b8700 1 -- 172.21.15.33:0/3281497999 learned_addr learned my addr 172.21.15.33:0/3281497999 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.233+0000 7fe9849b8700 1 -- 172.21.15.33:0/3281497999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da5ea0 msgr2=0x55bd87db6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.233+0000 7fe9849b8700 1 --2- 172.21.15.33:0/3281497999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da5ea0 0x55bd87db6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.233+0000 7fe9849b8700 1 -- 172.21.15.33:0/3281497999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da6860 msgr2=0x55bd87dadfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.233+0000 7fe9849b8700 1 --2- 172.21.15.33:0/3281497999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da6860 0x55bd87dadfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.233+0000 7fe9849b8700 1 -- 172.21.15.33:0/3281497999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd87b22b60 con 0x55bd87da5540 2024-05-08T21:21:42.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.233+0000 7fe9849b8700 1 --2- 172.21.15.33:0/3281497999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87db66f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe97409b370 tx=0x7fe974009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=823e24a3f2b892e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.233+0000 7fe96b7fe700 1 -- 172.21.15.33:0/3281497999 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97409bd70 con 0x55bd87da5540 2024-05-08T21:21:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.233+0000 7fe96b7fe700 1 -- 172.21.15.33:0/3281497999 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9740a3a60 con 0x55bd87da5540 2024-05-08T21:21:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.233+0000 7fe96b7fe700 1 -- 172.21.15.33:0/3281497999 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe974099b10 con 0x55bd87da5540 2024-05-08T21:21:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.234+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3281497999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 msgr2=0x55bd87db66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.234+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/3281497999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87db66f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe97409b370 tx=0x7fe974009310 comp rx=0 tx=0).stop 2024-05-08T21:21:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.234+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3281497999 shutdown_connections 2024-05-08T21:21:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.234+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/3281497999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da6860 0x55bd87dadfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.234+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/3281497999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87db66f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.234+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/3281497999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da5ea0 0x55bd87db6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.234+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3281497999 >> 172.21.15.33:0/3281497999 conn(0x55bd87d03210 msgr2=0x55bd87da6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.234+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3281497999 shutdown_connections 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.234+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3281497999 wait complete. 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.234+0000 7fe992dd6b80 1 -- start start 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe992dd6b80 1 Processor -- start 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe992dd6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87db3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe992dd6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87e0ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe992dd6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da6860 0x55bd87e0f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe992dd6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd87e0fa40 con 0x55bd87da5ea0 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe992dd6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd87e0fbb0 con 0x55bd87da6860 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe992dd6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd87e0fd70 con 0x55bd87da5540 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe992dd6b80 10 client.0 _ll_register_callbacks cb 0x55bd87c9ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe9851b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da6860 0x55bd87e0f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe9849b8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87db3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe97ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87e0ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe9849b8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87db3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:48062/0 (socket says 172.21.15.33:48062) 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe9851b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da6860 0x55bd87e0f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37326/0 (socket says 172.21.15.33:37326) 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.235+0000 7fe9849b8700 1 -- 172.21.15.33:0/3756996792 learned_addr learned my addr 172.21.15.33:0/3756996792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe9849b8700 1 -- 172.21.15.33:0/3756996792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da6860 msgr2=0x55bd87e0f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe9849b8700 1 --2- 172.21.15.33:0/3756996792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da6860 0x55bd87e0f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe9849b8700 1 -- 172.21.15.33:0/3756996792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 msgr2=0x55bd87e0ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe9849b8700 1 --2- 172.21.15.33:0/3756996792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87e0ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe9849b8700 1 -- 172.21.15.33:0/3756996792 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd87b763b0 con 0x55bd87da5540 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe9849b8700 1 --2- 172.21.15.33:0/3756996792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87db3ff0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe9740bb200 tx=0x7fe974005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24c00aaa64f8bc22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe95ffff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe95ffff700 1 -- 172.21.15.33:0/3756996792 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97409f4a0 con 0x55bd87da5540 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3756996792 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe974016e80 con 0x55bd87da5540 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe95ffff700 1 -- 172.21.15.33:0/3756996792 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe9740b5070 con 0x55bd87da5540 2024-05-08T21:21:42.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe95ffff700 1 -- 172.21.15.33:0/3756996792 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9740ce290 con 0x55bd87da5540 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe992dd6b80 10 client.26209 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3756996792 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55bd87cbb560 con 0x55bd87da5540 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.236+0000 7fe95ffff700 1 -- 172.21.15.33:0/3756996792 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe974016e80 con 0x55bd87da5540 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.237+0000 7fe95ffff700 1 -- 172.21.15.33:0/3756996792 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe97409ab60 con 0x55bd87da5540 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.237+0000 7fe992dd6b80 10 client.26209 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.237+0000 7fe992dd6b80 10 client.26209 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.237+0000 7fe992dd6b80 20 client.26209 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.237+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/3756996792 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55bd87e11260 0x55bd87e13680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.237+0000 7fe992dd6b80 4 client.26209 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.237+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3756996792 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bd87cedec0 con 0x55bd87e11260 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.237+0000 7fe97ffff700 1 --2- 172.21.15.33:0/3756996792 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55bd87e11260 0x55bd87e13680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.237+0000 7fe95ffff700 10 client.26209 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.237+0000 7fe97ffff700 1 --2- 172.21.15.33:0/3756996792 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55bd87e11260 0x55bd87e13680 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2eabafcf589c8c7 server_cookie=5fef93aed3d4a697 in_seq=0 out_seq=0 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe95ffff700 1 -- 172.21.15.33:0/3756996792 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bd87cedec0 con 0x55bd87e11260 2024-05-08T21:21:42.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe95ffff700 10 client.26209 handle_command_reply: tid=0 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe992dd6b80 1 client.26209 shutdown 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3756996792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 msgr2=0x55bd87db3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/3756996792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87db3ff0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe9740bb200 tx=0x7fe974005a90 comp rx=0 tx=0).stop 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3756996792 shutdown_connections 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/3756996792 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55bd87e11260 0x55bd87e13680 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/3756996792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd87da6860 0x55bd87e0f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/3756996792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd87da5540 0x55bd87db3ff0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe992dd6b80 1 --2- 172.21.15.33:0/3756996792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd87da5ea0 0x55bd87e0ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe95ffff700 0 client.26209 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3756996792 >> 172.21.15.33:0/3756996792 conn(0x55bd87d03210 msgr2=0x55bd87dadfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3756996792 shutdown_connections 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe992dd6b80 1 -- 172.21.15.33:0/3756996792 wait complete. 2024-05-08T21:21:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.238+0000 7fe992dd6b80 20 client.26209 trim_cache size 0 max 16384 2024-05-08T21:21:42.245 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:21:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.239+0000 7fe986c1c700 1 -- 172.21.15.33:0/2087633378 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe9700a11a0 msgr2=0x7fe9700a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.239+0000 7fe986c1c700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe9700a11a0 0x7fe9700a3660 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe96c011490 tx=0x7fe96c009260 comp rx=0 tx=0).stop 2024-05-08T21:21:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.239+0000 7fe986c1c700 1 -- 172.21.15.33:0/2087633378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9800fffa0 msgr2=0x7fe980195c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.239+0000 7fe986c1c700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9800fffa0 0x7fe980195c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe974004800 tx=0x7fe974009580 comp rx=0 tx=0).stop 2024-05-08T21:21:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.239+0000 7fe986c1c700 1 -- 172.21.15.33:0/2087633378 shutdown_connections 2024-05-08T21:21:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.239+0000 7fe986c1c700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9801013e0 0x7fe98019d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.239+0000 7fe986c1c700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9800fffa0 0x7fe980195c00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.239+0000 7fe986c1c700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe9700a11a0 0x7fe9700a3660 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.239+0000 7fe986c1c700 1 --2- 172.21.15.33:0/2087633378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe980100960 0x7fe980196140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.239+0000 7fe986c1c700 1 -- 172.21.15.33:0/2087633378 >> 172.21.15.33:0/2087633378 conn(0x7fe9800faa50 msgr2=0x7fe980104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.239+0000 7fe986c1c700 1 -- 172.21.15.33:0/2087633378 shutdown_connections 2024-05-08T21:21:42.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.239+0000 7fe986c1c700 1 -- 172.21.15.33:0/2087633378 wait complete. 2024-05-08T21:21:42.255 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:21:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.407+0000 7f1f724cd700 1 -- 172.21.15.33:0/2865692499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f6c0feed0 msgr2=0x7f1f6c0ff350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.407+0000 7f1f724cd700 1 --2- 172.21.15.33:0/2865692499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f6c0feed0 0x7f1f6c0ff350 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1f600098f0 tx=0x7f1f60009c00 comp rx=0 tx=0).stop 2024-05-08T21:21:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.407+0000 7f1f724cd700 1 -- 172.21.15.33:0/2865692499 shutdown_connections 2024-05-08T21:21:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.407+0000 7f1f724cd700 1 --2- 172.21.15.33:0/2865692499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c0ff890 0x7f1f6c109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.407+0000 7f1f724cd700 1 --2- 172.21.15.33:0/2865692499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f6c102460 0x7f1f6c102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.407+0000 7f1f724cd700 1 --2- 172.21.15.33:0/2865692499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f6c0feed0 0x7f1f6c0ff350 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.407+0000 7f1f724cd700 1 -- 172.21.15.33:0/2865692499 >> 172.21.15.33:0/2865692499 conn(0x7f1f6c0faa50 msgr2=0x7f1f6c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.407+0000 7f1f724cd700 1 -- 172.21.15.33:0/2865692499 shutdown_connections 2024-05-08T21:21:42.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.407+0000 7f1f724cd700 1 -- 172.21.15.33:0/2865692499 wait complete. 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f724cd700 1 Processor -- start 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f724cd700 1 -- start start 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f724cd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c0feed0 0x7f1f6c196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f724cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f6c0ff890 0x7f1f6c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f724cd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f6c102460 0x7f1f6c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f724cd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19e890 con 0x7f1f6c0feed0 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f724cd700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19ea00 con 0x7f1f6c102460 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f724cd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19ebc0 con 0x7f1f6c0ff890 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c0feed0 0x7f1f6c196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c0feed0 0x7f1f6c196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52798/0 (socket says 172.21.15.33:52798) 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f6bfff700 1 -- 172.21.15.33:0/2043375935 learned_addr learned my addr 172.21.15.33:0/2043375935 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f6b7fe700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f6c0ff890 0x7f1f6c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.408+0000 7f1f70a6a700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f6c102460 0x7f1f6c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.409+0000 7f1f6bfff700 1 -- 172.21.15.33:0/2043375935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f6c0ff890 msgr2=0x7f1f6c1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.409+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f6c0ff890 0x7f1f6c1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.409+0000 7f1f6bfff700 1 -- 172.21.15.33:0/2043375935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f6c102460 msgr2=0x7f1f6c19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.409+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f6c102460 0x7f1f6c19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.409+0000 7f1f6bfff700 1 -- 172.21.15.33:0/2043375935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f64009d00 con 0x7f1f6c0feed0 2024-05-08T21:21:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.409+0000 7f1f6b7fe700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f6c0ff890 0x7f1f6c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.409+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c0feed0 0x7f1f6c196da0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00e970 tx=0x7f1f5c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.409+0000 7f1f697fa700 1 -- 172.21.15.33:0/2043375935 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c009790 con 0x7f1f6c0feed0 2024-05-08T21:21:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.409+0000 7f1f697fa700 1 -- 172.21.15.33:0/2043375935 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f5c00c7e0 con 0x7f1f6c0feed0 2024-05-08T21:21:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.409+0000 7f1f724cd700 1 -- 172.21.15.33:0/2043375935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f60009520 con 0x7f1f6c0feed0 2024-05-08T21:21:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.409+0000 7f1f697fa700 1 -- 172.21.15.33:0/2043375935 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c003bd0 con 0x7f1f6c0feed0 2024-05-08T21:21:42.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.409+0000 7f1f724cd700 1 -- 172.21.15.33:0/2043375935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c19f1d0 con 0x7f1f6c0feed0 2024-05-08T21:21:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.412+0000 7f1f697fa700 1 -- 172.21.15.33:0/2043375935 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1f5c00c950 con 0x7f1f6c0feed0 2024-05-08T21:21:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.412+0000 7f1f697fa700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1f540a1120 0x7f1f540a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.412+0000 7f1f697fa700 1 -- 172.21.15.33:0/2043375935 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1f5c014070 con 0x7f1f6c0feed0 2024-05-08T21:21:42.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.412+0000 7f1f6b7fe700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1f540a1120 0x7f1f540a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.413+0000 7f1f6b7fe700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1f540a1120 0x7f1f540a35e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009fd0 tx=0x7f1f60019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f7e687b80 1 Processor -- start 2024-05-08T21:21:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f7e687b80 1 -- start start 2024-05-08T21:21:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f7e687b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658489960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f7e687b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b658489ea0 0x55b65848a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f7e687b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b65848a860 0x55b65848ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f7e687b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6583e0150 con 0x55b65848a860 2024-05-08T21:21:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f7e687b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6583ddb10 con 0x55b658489ea0 2024-05-08T21:21:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f7e687b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6583d9400 con 0x55b658489540 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658489960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658489960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:48082/0 (socket says 172.21.15.33:48082) 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b658489ea0 0x55b65848a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f6bfff700 1 -- 172.21.15.33:0/3173210216 learned_addr learned my addr 172.21.15.33:0/3173210216 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f70a6a700 1 --2- 172.21.15.33:0/3173210216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b65848a860 0x55b65848ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f6bfff700 1 -- 172.21.15.33:0/3173210216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b658489ea0 msgr2=0x55b65848a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.414+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/3173210216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b658489ea0 0x55b65848a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.415+0000 7f1f6bfff700 1 -- 172.21.15.33:0/3173210216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b65848a860 msgr2=0x55b65848ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.415+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/3173210216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b65848a860 0x55b65848ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.415+0000 7f1f6bfff700 1 -- 172.21.15.33:0/3173210216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b65825a3b0 con 0x55b658489540 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.415+0000 7f1f6b7fe700 1 --2- 172.21.15.33:0/3173210216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b658489ea0 0x55b65848a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.415+0000 7f1f70a6a700 1 --2- 172.21.15.33:0/3173210216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b65848a860 0x55b65848ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.415+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/3173210216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658489960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0963d0 tx=0x7f1f5c005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=329191391b64dca5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.415+0000 7f1f527fc700 1 -- 172.21.15.33:0/3173210216 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c09a5d0 con 0x55b658489540 2024-05-08T21:21:42.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.415+0000 7f1f527fc700 1 -- 172.21.15.33:0/3173210216 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f5c099ed0 con 0x55b658489540 2024-05-08T21:21:42.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.415+0000 7f1f527fc700 1 -- 172.21.15.33:0/3173210216 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c09a130 con 0x55b658489540 2024-05-08T21:21:42.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.416+0000 7f1f7e687b80 1 -- 172.21.15.33:0/3173210216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 msgr2=0x55b658489960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.416+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/3173210216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658489960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0963d0 tx=0x7f1f5c005910 comp rx=0 tx=0).stop 2024-05-08T21:21:42.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.416+0000 7f1f7e687b80 1 -- 172.21.15.33:0/3173210216 shutdown_connections 2024-05-08T21:21:42.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.416+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/3173210216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b65848a860 0x55b65848ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.416+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/3173210216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658489960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.416+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/3173210216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b658489ea0 0x55b65848a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.416+0000 7f1f7e687b80 1 -- 172.21.15.33:0/3173210216 >> 172.21.15.33:0/3173210216 conn(0x55b6583e7210 msgr2=0x55b658483de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.416+0000 7f1f7e687b80 1 -- 172.21.15.33:0/3173210216 shutdown_connections 2024-05-08T21:21:42.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.416+0000 7f1f7e687b80 1 -- 172.21.15.33:0/3173210216 wait complete. 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.416+0000 7f1f7e687b80 1 -- start start 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.416+0000 7f1f7e687b80 1 Processor -- start 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f7e687b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658492eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f7e687b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b658489ea0 0x55b6584933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f7e687b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b65848a860 0x55b65849a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f7e687b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b658375560 con 0x55b658489ea0 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f7e687b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b658375200 con 0x55b65848a860 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f7e687b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65825c6a0 con 0x55b658489540 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f7e687b80 10 client.0 _ll_register_callbacks cb 0x55b65837eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658492eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b658489ea0 0x55b6584933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f70a6a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b65848a860 0x55b65849a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658492eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:48084/0 (socket says 172.21.15.33:48084) 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f6bfff700 1 -- 172.21.15.33:0/1210248396 learned_addr learned my addr 172.21.15.33:0/1210248396 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f6bfff700 1 -- 172.21.15.33:0/1210248396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b65848a860 msgr2=0x55b65849a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/1210248396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b65848a860 0x55b65849a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f6bfff700 1 -- 172.21.15.33:0/1210248396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b658489ea0 msgr2=0x55b6584933f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/1210248396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b658489ea0 0x55b6584933f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f6bfff700 1 -- 172.21.15.33:0/1210248396 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f64009160 con 0x55b658489540 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/1210248396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658492eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0995f0 tx=0x7f1f5c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d366563479da62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.417+0000 7f1f4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f4b7fe700 1 -- 172.21.15.33:0/1210248396 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c09a5d0 con 0x55b658489540 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f4b7fe700 1 -- 172.21.15.33:0/1210248396 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1f5c09dd00 con 0x55b658489540 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f7e687b80 1 -- 172.21.15.33:0/1210248396 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b658332270 con 0x55b658489540 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f4b7fe700 1 -- 172.21.15.33:0/1210248396 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f5c098350 con 0x55b658489540 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f7e687b80 10 client.26221 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f7e687b80 1 -- 172.21.15.33:0/1210248396 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f1f6c19f1d0 con 0x55b658489540 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f4b7fe700 1 -- 172.21.15.33:0/1210248396 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b658332270 con 0x55b658489540 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f4b7fe700 1 -- 172.21.15.33:0/1210248396 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1f5c095d30 con 0x55b658489540 2024-05-08T21:21:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f7e687b80 10 client.26221 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f7e687b80 10 client.26221 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f7e687b80 20 client.26221 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/1210248396 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55b65849e850 0x55b6584a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f7e687b80 4 client.26221 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.418+0000 7f1f7e687b80 1 -- 172.21.15.33:0/1210248396 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b658458810 con 0x55b65849e850 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.419+0000 7f1f6b7fe700 1 --2- 172.21.15.33:0/1210248396 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55b65849e850 0x55b6584a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.419+0000 7f1f6b7fe700 1 --2- 172.21.15.33:0/1210248396 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55b65849e850 0x55b6584a0c70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d673a3412a072f6 server_cookie=324da6c646ca7fc7 in_seq=0 out_seq=0 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.419+0000 7f1f4b7fe700 10 client.26221 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f4b7fe700 1 -- 172.21.15.33:0/1210248396 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b658458810 con 0x55b65849e850 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f4b7fe700 10 client.26221 handle_command_reply: tid=0 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f7e687b80 1 client.26221 shutdown 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f7e687b80 1 -- 172.21.15.33:0/1210248396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 msgr2=0x55b658492eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/1210248396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658492eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0995f0 tx=0x7f1f5c0047c0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f7e687b80 1 -- 172.21.15.33:0/1210248396 shutdown_connections 2024-05-08T21:21:42.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/1210248396 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55b65849e850 0x55b6584a0c70 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/1210248396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b65848a860 0x55b65849a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/1210248396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658492eb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f4b7fe700 0 client.26221 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:21:42.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/1210248396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b658489ea0 0x55b6584933f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f7e687b80 1 -- 172.21.15.33:0/1210248396 >> 172.21.15.33:0/1210248396 conn(0x55b6583e7210 msgr2=0x55b6584835a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f7e687b80 1 -- 172.21.15.33:0/1210248396 shutdown_connections 2024-05-08T21:21:42.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f7e687b80 1 -- 172.21.15.33:0/1210248396 wait complete. 2024-05-08T21:21:42.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.420+0000 7f1f7e687b80 20 client.26221 trim_cache size 0 max 16384 2024-05-08T21:21:42.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f7e687b80 1 Processor -- start 2024-05-08T21:21:42.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f7e687b80 1 -- start start 2024-05-08T21:21:42.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f7e687b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658489920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f7e687b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b658489ea0 0x55b65849a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f7e687b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b65848a860 0x55b65848ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f7e687b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65848b180 con 0x55b65848a860 2024-05-08T21:21:42.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f7e687b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b658490b30 con 0x55b658489ea0 2024-05-08T21:21:42.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f7e687b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65847c040 con 0x55b658489540 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658489920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b658489ea0 0x55b65849a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658489920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:48100/0 (socket says 172.21.15.33:48100) 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f6bfff700 1 -- 172.21.15.33:0/4266323641 learned_addr learned my addr 172.21.15.33:0/4266323641 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.440+0000 7f1f70a6a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b65848a860 0x55b65848ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.441+0000 7f1f6bfff700 1 -- 172.21.15.33:0/4266323641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b658489ea0 msgr2=0x55b65849a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.441+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/4266323641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b658489ea0 0x55b65849a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.441+0000 7f1f6bfff700 1 -- 172.21.15.33:0/4266323641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b65848a860 msgr2=0x55b65848ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.441+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/4266323641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b65848a860 0x55b65848ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.441+0000 7f1f6bfff700 1 -- 172.21.15.33:0/4266323641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b658206b60 con 0x55b658489540 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.441+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/4266323641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658489920 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c096940 tx=0x7f1f5c005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ebaf3f9ece5d237 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.441+0000 7f1f52ffd700 1 -- 172.21.15.33:0/4266323641 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c09a5d0 con 0x55b658489540 2024-05-08T21:21:42.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.441+0000 7f1f52ffd700 1 -- 172.21.15.33:0/4266323641 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f5c09dd00 con 0x55b658489540 2024-05-08T21:21:42.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.441+0000 7f1f52ffd700 1 -- 172.21.15.33:0/4266323641 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c098270 con 0x55b658489540 2024-05-08T21:21:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.442+0000 7f1f7e687b80 1 -- 172.21.15.33:0/4266323641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 msgr2=0x55b658489920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.442+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/4266323641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658489920 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c096940 tx=0x7f1f5c005910 comp rx=0 tx=0).stop 2024-05-08T21:21:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.442+0000 7f1f7e687b80 1 -- 172.21.15.33:0/4266323641 shutdown_connections 2024-05-08T21:21:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.442+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/4266323641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b65848a860 0x55b65848ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.442+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/4266323641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489540 0x55b658489920 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.442+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/4266323641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b658489ea0 0x55b65849a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.442+0000 7f1f7e687b80 1 -- 172.21.15.33:0/4266323641 >> 172.21.15.33:0/4266323641 conn(0x55b6583e7210 msgr2=0x55b658480210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.442+0000 7f1f7e687b80 1 -- 172.21.15.33:0/4266323641 shutdown_connections 2024-05-08T21:21:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.442+0000 7f1f7e687b80 1 -- 172.21.15.33:0/4266323641 wait complete. 2024-05-08T21:21:42.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.443+0000 7f1f7e687b80 1 -- start start 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.443+0000 7f1f7e687b80 1 Processor -- start 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f7e687b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b658489540 0x55b658495760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f7e687b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489ea0 0x55b658495ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f7e687b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b65848a860 0x55b658498bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f7e687b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b658499110 con 0x55b658489540 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f7e687b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b658499280 con 0x55b65848a860 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f7e687b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b658499440 con 0x55b658489ea0 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f7e687b80 10 client.0 _ll_register_callbacks cb 0x55b65837eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b658489540 0x55b658495760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b658489540 0x55b658495760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52840/0 (socket says 172.21.15.33:52840) 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f6bfff700 1 -- 172.21.15.33:0/25796526 learned_addr learned my addr 172.21.15.33:0/25796526 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f6b7fe700 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489ea0 0x55b658495ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f70a6a700 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b65848a860 0x55b658498bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f6b7fe700 1 -- 172.21.15.33:0/25796526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b65848a860 msgr2=0x55b658498bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f6b7fe700 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b65848a860 0x55b658498bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f6b7fe700 1 -- 172.21.15.33:0/25796526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b658489540 msgr2=0x55b658495760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f6b7fe700 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b658489540 0x55b658495760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f6b7fe700 1 -- 172.21.15.33:0/25796526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c19f1d0 con 0x55b658489ea0 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.444+0000 7f1f70a6a700 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b65848a860 0x55b658498bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.445+0000 7f1f6b7fe700 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489ea0 0x55b658495ca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009230 tx=0x7f1f60009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6013a55fd384add server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.445+0000 7f1f4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.445+0000 7f1f4b7fe700 1 -- 172.21.15.33:0/25796526 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60035070 con 0x55b658489ea0 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.445+0000 7f1f4b7fe700 1 -- 172.21.15.33:0/25796526 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1f6000bb60 con 0x55b658489ea0 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.445+0000 7f1f7e687b80 1 -- 172.21.15.33:0/25796526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f5c099ed0 con 0x55b658489ea0 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.445+0000 7f1f4b7fe700 1 -- 172.21.15.33:0/25796526 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f60005610 con 0x55b658489ea0 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.445+0000 7f1f7e687b80 10 client.26227 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.445+0000 7f1f7e687b80 1 -- 172.21.15.33:0/25796526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b65839f560 con 0x55b658489ea0 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.445+0000 7f1f4b7fe700 1 -- 172.21.15.33:0/25796526 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f5c099ed0 con 0x55b658489ea0 2024-05-08T21:21:42.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.445+0000 7f1f4b7fe700 1 -- 172.21.15.33:0/25796526 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1f6003a4e0 con 0x55b658489ea0 2024-05-08T21:21:42.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.446+0000 7f1f7e687b80 10 client.26227 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:42.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.446+0000 7f1f7e687b80 10 client.26227 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:21:42.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.446+0000 7f1f7e687b80 20 client.26227 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:42.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.446+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55b6584f5250 0x55b6584f7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.446+0000 7f1f7e687b80 4 client.26227 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:21:42.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.446+0000 7f1f7e687b80 1 -- 172.21.15.33:0/25796526 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b6583d1ec0 con 0x55b6584f5250 2024-05-08T21:21:42.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.446+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55b6584f5250 0x55b6584f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.446+0000 7f1f4b7fe700 10 client.26227 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:21:42.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.446+0000 7f1f6bfff700 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55b6584f5250 0x55b6584f7670 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbba808b33b35894 server_cookie=be7f551db95fc544 in_seq=0 out_seq=0 2024-05-08T21:21:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.447+0000 7f1f4b7fe700 1 -- 172.21.15.33:0/25796526 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b6583d1ec0 con 0x55b6584f5250 2024-05-08T21:21:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.447+0000 7f1f4b7fe700 10 client.26227 handle_command_reply: tid=0 2024-05-08T21:21:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.447+0000 7f1f7e687b80 1 client.26227 shutdown 2024-05-08T21:21:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.447+0000 7f1f7e687b80 1 -- 172.21.15.33:0/25796526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489ea0 msgr2=0x55b658495ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.447+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489ea0 0x55b658495ca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009230 tx=0x7f1f60009c00 comp rx=0 tx=0).stop 2024-05-08T21:21:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.448+0000 7f1f7e687b80 1 -- 172.21.15.33:0/25796526 shutdown_connections 2024-05-08T21:21:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.448+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55b6584f5250 0x55b6584f7670 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.448+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b65848a860 0x55b658498bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.448+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b658489540 0x55b658495760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.448+0000 7f1f7e687b80 1 --2- 172.21.15.33:0/25796526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b658489ea0 0x55b658495ca0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.448+0000 7f1f7e687b80 1 -- 172.21.15.33:0/25796526 >> 172.21.15.33:0/25796526 conn(0x55b6583e7210 msgr2=0x55b658492320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.448+0000 7f1f4b7fe700 0 client.26227 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:21:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.448+0000 7f1f7e687b80 1 -- 172.21.15.33:0/25796526 shutdown_connections 2024-05-08T21:21:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.448+0000 7f1f7e687b80 1 -- 172.21.15.33:0/25796526 wait complete. 2024-05-08T21:21:42.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.448+0000 7f1f7e687b80 20 client.26227 trim_cache size 0 max 16384 2024-05-08T21:21:42.453 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:21:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.449+0000 7f1f724cd700 1 -- 172.21.15.33:0/2043375935 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1f540a1120 msgr2=0x7f1f540a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.449+0000 7f1f724cd700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1f540a1120 0x7f1f540a35e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009fd0 tx=0x7f1f60019040 comp rx=0 tx=0).stop 2024-05-08T21:21:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.449+0000 7f1f724cd700 1 -- 172.21.15.33:0/2043375935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c0feed0 msgr2=0x7f1f6c196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.449+0000 7f1f724cd700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c0feed0 0x7f1f6c196da0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00e970 tx=0x7f1f5c00ec80 comp rx=0 tx=0).stop 2024-05-08T21:21:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.449+0000 7f1f724cd700 1 -- 172.21.15.33:0/2043375935 shutdown_connections 2024-05-08T21:21:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.449+0000 7f1f724cd700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1f540a1120 0x7f1f540a35e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.449+0000 7f1f724cd700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f6c102460 0x7f1f6c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.449+0000 7f1f724cd700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c0feed0 0x7f1f6c196da0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.449+0000 7f1f724cd700 1 --2- 172.21.15.33:0/2043375935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f6c0ff890 0x7f1f6c1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.449+0000 7f1f724cd700 1 -- 172.21.15.33:0/2043375935 >> 172.21.15.33:0/2043375935 conn(0x7f1f6c0faa50 msgr2=0x7f1f6c1056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.450+0000 7f1f724cd700 1 -- 172.21.15.33:0/2043375935 shutdown_connections 2024-05-08T21:21:42.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.450+0000 7f1f724cd700 1 -- 172.21.15.33:0/2043375935 wait complete. 2024-05-08T21:21:42.469 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:21:42.507 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:42 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2438357799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:21:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.628+0000 7f8987b56700 1 -- 172.21.15.33:0/656794320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8980104220 msgr2=0x7f8980106610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.628+0000 7f8987b56700 1 --2- 172.21.15.33:0/656794320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8980104220 0x7f8980106610 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8970009990 tx=0x7f8970009ca0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.628+0000 7f8987b56700 1 -- 172.21.15.33:0/656794320 shutdown_connections 2024-05-08T21:21:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.628+0000 7f8987b56700 1 --2- 172.21.15.33:0/656794320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980106b50 0x7f8980108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.628+0000 7f8987b56700 1 --2- 172.21.15.33:0/656794320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89801018f0 0x7f8980103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.628+0000 7f8987b56700 1 --2- 172.21.15.33:0/656794320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8980104220 0x7f8980106610 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.628+0000 7f8987b56700 1 -- 172.21.15.33:0/656794320 >> 172.21.15.33:0/656794320 conn(0x7f89800fb300 msgr2=0x7f89800fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.628+0000 7f8987b56700 1 -- 172.21.15.33:0/656794320 shutdown_connections 2024-05-08T21:21:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.628+0000 7f8987b56700 1 -- 172.21.15.33:0/656794320 wait complete. 2024-05-08T21:21:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.628+0000 7f8987b56700 1 Processor -- start 2024-05-08T21:21:42.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f8987b56700 1 -- start start 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f8987b56700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89801018f0 0x7f8980196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f8987b56700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980104220 0x7f8980197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f8987b56700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8980106b50 0x7f898019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f8987b56700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898019e7f0 con 0x7f89801018f0 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f8987b56700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898019e960 con 0x7f8980106b50 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f8987b56700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898019eb20 con 0x7f8980104220 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89858f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89801018f0 0x7f8980196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89850f1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980104220 0x7f8980197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89858f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89801018f0 0x7f8980196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52858/0 (socket says 172.21.15.33:52858) 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89858f2700 1 -- 172.21.15.33:0/4249912786 learned_addr learned my addr 172.21.15.33:0/4249912786 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89860f3700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8980106b50 0x7f898019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89850f1700 1 -- 172.21.15.33:0/4249912786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8980106b50 msgr2=0x7f898019e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89850f1700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8980106b50 0x7f898019e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89850f1700 1 -- 172.21.15.33:0/4249912786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89801018f0 msgr2=0x7f8980196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89850f1700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89801018f0 0x7f8980196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89850f1700 1 -- 172.21.15.33:0/4249912786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89700095f0 con 0x7f8980104220 2024-05-08T21:21:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89858f2700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89801018f0 0x7f8980196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89860f3700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8980106b50 0x7f898019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.629+0000 7f89850f1700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980104220 0x7f8980197210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8970003f60 tx=0x7f8970004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.630+0000 7f8976ffd700 1 -- 172.21.15.33:0/4249912786 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897001d070 con 0x7f8980104220 2024-05-08T21:21:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.630+0000 7f8976ffd700 1 -- 172.21.15.33:0/4249912786 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f897000bdb0 con 0x7f8980104220 2024-05-08T21:21:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.630+0000 7f8987b56700 1 -- 172.21.15.33:0/4249912786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898019eda0 con 0x7f8980104220 2024-05-08T21:21:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.630+0000 7f8976ffd700 1 -- 172.21.15.33:0/4249912786 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8970022ca0 con 0x7f8980104220 2024-05-08T21:21:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.630+0000 7f8987b56700 1 -- 172.21.15.33:0/4249912786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898019f1c0 con 0x7f8980104220 2024-05-08T21:21:42.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.633+0000 7f8976ffd700 1 -- 172.21.15.33:0/4249912786 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8970022470 con 0x7f8980104220 2024-05-08T21:21:42.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.633+0000 7f8976ffd700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f896c0a1120 0x7f896c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.634+0000 7f89858f2700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f896c0a1120 0x7f896c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.634+0000 7f8976ffd700 1 -- 172.21.15.33:0/4249912786 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f89700ce2d0 con 0x7f8980104220 2024-05-08T21:21:42.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.634+0000 7f89858f2700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f896c0a1120 0x7f896c0a35e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8980103100 tx=0x7f897c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.634+0000 7f8993d10b80 1 Processor -- start 2024-05-08T21:21:42.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f8993d10b80 1 -- start start 2024-05-08T21:21:42.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f8993d10b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb1950 0x564350bb1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f8993d10b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 0x564350bb2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f8993d10b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bb30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f8993d10b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564350b04150 con 0x564350bb2c70 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f8993d10b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564350b01b10 con 0x564350bb22b0 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f8993d10b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564350afd400 con 0x564350bb1950 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f89850f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 0x564350bb2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f89858f2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb1950 0x564350bb1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f89850f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 0x564350bb2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37408/0 (socket says 172.21.15.33:37408) 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f89850f1700 1 -- 172.21.15.33:0/1940796330 learned_addr learned my addr 172.21.15.33:0/1940796330 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f89860f3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bb30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f89850f1700 1 -- 172.21.15.33:0/1940796330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb1950 msgr2=0x564350bb1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f89850f1700 1 --2- 172.21.15.33:0/1940796330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb1950 0x564350bb1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f89850f1700 1 -- 172.21.15.33:0/1940796330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 msgr2=0x564350bb30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f89850f1700 1 --2- 172.21.15.33:0/1940796330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bb30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f89850f1700 1 -- 172.21.15.33:0/1940796330 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56435097e3b0 con 0x564350bb22b0 2024-05-08T21:21:42.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.635+0000 7f89850f1700 1 --2- 172.21.15.33:0/1940796330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 0x564350bb2730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8970006010 tx=0x7f8970009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43aef764992d3c93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.636+0000 7f896bfff700 1 -- 172.21.15.33:0/1940796330 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89700ce160 con 0x564350bb22b0 2024-05-08T21:21:42.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.636+0000 7f896bfff700 1 -- 172.21.15.33:0/1940796330 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f897009aeb0 con 0x564350bb22b0 2024-05-08T21:21:42.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.636+0000 7f896bfff700 1 -- 172.21.15.33:0/1940796330 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897009f020 con 0x564350bb22b0 2024-05-08T21:21:42.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.637+0000 7f8993d10b80 1 -- 172.21.15.33:0/1940796330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 msgr2=0x564350bb2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.637+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1940796330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 0x564350bb2730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8970006010 tx=0x7f8970009ca0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.637+0000 7f8993d10b80 1 -- 172.21.15.33:0/1940796330 shutdown_connections 2024-05-08T21:21:42.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.637+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1940796330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bb30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.637+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1940796330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb1950 0x564350bb1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.637+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1940796330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 0x564350bb2730 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.637+0000 7f8993d10b80 1 -- 172.21.15.33:0/1940796330 >> 172.21.15.33:0/1940796330 conn(0x564350b0b210 msgr2=0x564350ba7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.637+0000 7f8993d10b80 1 -- 172.21.15.33:0/1940796330 shutdown_connections 2024-05-08T21:21:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.637+0000 7f8993d10b80 1 -- 172.21.15.33:0/1940796330 wait complete. 2024-05-08T21:21:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.637+0000 7f8993d10b80 1 -- start start 2024-05-08T21:21:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.637+0000 7f8993d10b80 1 Processor -- start 2024-05-08T21:21:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f8993d10b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 0x564350bbaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f8993d10b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb22b0 0x564350bbb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f8993d10b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bc2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f8993d10b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564350a99560 con 0x564350bb2c70 2024-05-08T21:21:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f8993d10b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564350a99200 con 0x564350bb1950 2024-05-08T21:21:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f8993d10b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643509806a0 con 0x564350bb22b0 2024-05-08T21:21:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f8993d10b80 10 client.0 _ll_register_callbacks cb 0x564350aa2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f89858f2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 0x564350bbaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f89850f1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb22b0 0x564350bbb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f89858f2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 0x564350bbaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37412/0 (socket says 172.21.15.33:37412) 2024-05-08T21:21:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f89858f2700 1 -- 172.21.15.33:0/1955301470 learned_addr learned my addr 172.21.15.33:0/1955301470 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f89860f3700 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bc2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f89858f2700 1 -- 172.21.15.33:0/1955301470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb22b0 msgr2=0x564350bbb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f89858f2700 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb22b0 0x564350bbb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f89858f2700 1 -- 172.21.15.33:0/1955301470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 msgr2=0x564350bc2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f89858f2700 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bc2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f89858f2700 1 -- 172.21.15.33:0/1955301470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8970097040 con 0x564350bb1950 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f89860f3700 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bc2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.638+0000 7f89850f1700 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb22b0 0x564350bbb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.639+0000 7f89858f2700 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 0x564350bbaf00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f897c0094f0 tx=0x7f897c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e02ae8c3c1870b16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.639+0000 7f89697fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.639+0000 7f89697fa700 1 -- 172.21.15.33:0/1955301470 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c01fcd0 con 0x564350bb1950 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.639+0000 7f8993d10b80 1 -- 172.21.15.33:0/1955301470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564350a56270 con 0x564350bb1950 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.639+0000 7f89697fa700 1 -- 172.21.15.33:0/1955301470 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f897c01fe30 con 0x564350bb1950 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.639+0000 7f89697fa700 1 -- 172.21.15.33:0/1955301470 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f897c003e60 con 0x564350bb1950 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.639+0000 7f8993d10b80 10 client.26219 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.639+0000 7f8993d10b80 1 -- 172.21.15.33:0/1955301470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56435092ab60 con 0x564350bb1950 2024-05-08T21:21:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.639+0000 7f89697fa700 1 -- 172.21.15.33:0/1955301470 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564350a56270 con 0x564350bb1950 2024-05-08T21:21:42.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.639+0000 7f89697fa700 1 -- 172.21.15.33:0/1955301470 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f897c027470 con 0x564350bb1950 2024-05-08T21:21:42.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.639+0000 7f8993d10b80 10 client.26219 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:42.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.639+0000 7f8993d10b80 10 client.26219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:42.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.640+0000 7f8993d10b80 20 client.26219 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:42.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.640+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564350bc6d10 0x564350bc9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.640+0000 7f8993d10b80 4 client.26219 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:42.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.640+0000 7f8993d10b80 1 -- 172.21.15.33:0/1955301470 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564350b7c810 con 0x564350bc6d10 2024-05-08T21:21:42.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.640+0000 7f89850f1700 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564350bc6d10 0x564350bc9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f89697fa700 10 client.26219 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:42.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f89850f1700 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564350bc6d10 0x564350bc9130 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f097770e8db89e9d server_cookie=e3512e364020b8ea in_seq=0 out_seq=0 2024-05-08T21:21:42.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f89697fa700 1 -- 172.21.15.33:0/1955301470 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564350b7c810 con 0x564350bc6d10 2024-05-08T21:21:42.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f89697fa700 10 client.26219 handle_command_reply: tid=0 2024-05-08T21:21:42.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f8993d10b80 1 client.26219 shutdown 2024-05-08T21:21:42.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f8993d10b80 1 -- 172.21.15.33:0/1955301470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 msgr2=0x564350bbaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 0x564350bbaf00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f897c0094f0 tx=0x7f897c0047c0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f8993d10b80 1 -- 172.21.15.33:0/1955301470 shutdown_connections 2024-05-08T21:21:42.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564350bc6d10 0x564350bc9130 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bc2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 0x564350bbaf00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1955301470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb22b0 0x564350bbb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f8993d10b80 1 -- 172.21.15.33:0/1955301470 >> 172.21.15.33:0/1955301470 conn(0x564350b0b210 msgr2=0x564350ba75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.641+0000 7f89697fa700 0 client.26219 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.642+0000 7f8993d10b80 1 -- 172.21.15.33:0/1955301470 shutdown_connections 2024-05-08T21:21:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.642+0000 7f8993d10b80 1 -- 172.21.15.33:0/1955301470 wait complete. 2024-05-08T21:21:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.642+0000 7f8993d10b80 20 client.26219 trim_cache size 0 max 16384 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.656+0000 7f8993d10b80 1 Processor -- start 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.656+0000 7f8993d10b80 1 -- start start 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f8993d10b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 0x564350bba410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f8993d10b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb22b0 0x564350bc2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f8993d10b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bb91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f8993d10b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564350bc3640 con 0x564350bb2c70 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f8993d10b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564350bc4190 con 0x564350bb1950 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f8993d10b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564350ba4df0 con 0x564350bb22b0 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f89858f2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 0x564350bba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f89860f3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bb91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f89858f2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 0x564350bba410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37418/0 (socket says 172.21.15.33:37418) 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f89858f2700 1 -- 172.21.15.33:0/1893117993 learned_addr learned my addr 172.21.15.33:0/1893117993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f89850f1700 1 --2- 172.21.15.33:0/1893117993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb22b0 0x564350bc2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f89858f2700 1 -- 172.21.15.33:0/1893117993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb22b0 msgr2=0x564350bc2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f89858f2700 1 --2- 172.21.15.33:0/1893117993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb22b0 0x564350bc2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f89858f2700 1 -- 172.21.15.33:0/1893117993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 msgr2=0x564350bb91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f89858f2700 1 --2- 172.21.15.33:0/1893117993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bb91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f89858f2700 1 -- 172.21.15.33:0/1893117993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564350ac3560 con 0x564350bb1950 2024-05-08T21:21:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f89860f3700 1 --2- 172.21.15.33:0/1893117993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bb91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.657+0000 7f89858f2700 1 --2- 172.21.15.33:0/1893117993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 0x564350bba410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f897c016040 tx=0x7f897c0092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd8c6b184567a044 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.658+0000 7f8974da4700 1 -- 172.21.15.33:0/1893117993 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c0275e0 con 0x564350bb1950 2024-05-08T21:21:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.658+0000 7f8974da4700 1 -- 172.21.15.33:0/1893117993 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f897c01f710 con 0x564350bb1950 2024-05-08T21:21:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.658+0000 7f8974da4700 1 -- 172.21.15.33:0/1893117993 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c0366a0 con 0x564350bb1950 2024-05-08T21:21:42.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.658+0000 7f8993d10b80 1 -- 172.21.15.33:0/1893117993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 msgr2=0x564350bba410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.658+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1893117993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 0x564350bba410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f897c016040 tx=0x7f897c0092d0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.658+0000 7f8993d10b80 1 -- 172.21.15.33:0/1893117993 shutdown_connections 2024-05-08T21:21:42.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.658+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1893117993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb2c70 0x564350bb91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.658+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1893117993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb1950 0x564350bba410 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.658+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1893117993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb22b0 0x564350bc2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.658+0000 7f8993d10b80 1 -- 172.21.15.33:0/1893117993 >> 172.21.15.33:0/1893117993 conn(0x564350b0b210 msgr2=0x564350bb3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.659+0000 7f8993d10b80 1 -- 172.21.15.33:0/1893117993 shutdown_connections 2024-05-08T21:21:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.659+0000 7f8993d10b80 1 -- 172.21.15.33:0/1893117993 wait complete. 2024-05-08T21:21:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.659+0000 7f8993d10b80 1 -- start start 2024-05-08T21:21:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.659+0000 7f8993d10b80 1 Processor -- start 2024-05-08T21:21:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.659+0000 7f8993d10b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb1950 0x564350bbd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.659+0000 7f8993d10b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 0x564350bbdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.659+0000 7f8993d10b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb2c70 0x564350c1b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.659+0000 7f8993d10b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564350bc2480 con 0x564350bb1950 2024-05-08T21:21:42.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.659+0000 7f8993d10b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564350bc25f0 con 0x564350bb22b0 2024-05-08T21:21:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.659+0000 7f8993d10b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564350bc2760 con 0x564350bb2c70 2024-05-08T21:21:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f8993d10b80 10 client.0 _ll_register_callbacks cb 0x564350aa2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89858f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb1950 0x564350bbd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89860f3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb2c70 0x564350c1b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89850f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 0x564350bbdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89858f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb1950 0x564350bbd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52898/0 (socket says 172.21.15.33:52898) 2024-05-08T21:21:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89858f2700 1 -- 172.21.15.33:0/1690421164 learned_addr learned my addr 172.21.15.33:0/1690421164 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89860f3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb2c70 0x564350c1b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:48154/0 (socket says 172.21.15.33:48154) 2024-05-08T21:21:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89850f1700 1 --2- 172.21.15.33:0/1690421164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 0x564350bbdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37434/0 (socket says 172.21.15.33:37434) 2024-05-08T21:21:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89858f2700 1 -- 172.21.15.33:0/1690421164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb2c70 msgr2=0x564350c1b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89858f2700 1 --2- 172.21.15.33:0/1690421164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb2c70 0x564350c1b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89858f2700 1 -- 172.21.15.33:0/1690421164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 msgr2=0x564350bbdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89858f2700 1 --2- 172.21.15.33:0/1690421164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 0x564350bbdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89858f2700 1 -- 172.21.15.33:0/1690421164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89780091c0 con 0x564350bb1950 2024-05-08T21:21:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89858f2700 1 --2- 172.21.15.33:0/1690421164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb1950 0x564350bbd660 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f897c027c00 tx=0x7f897c027c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849a19fbafd988c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89697fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:21:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89697fa700 1 -- 172.21.15.33:0/1690421164 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c01fce0 con 0x564350bb1950 2024-05-08T21:21:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f89697fa700 1 -- 172.21.15.33:0/1690421164 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f897c005050 con 0x564350bb1950 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.660+0000 7f8993d10b80 1 -- 172.21.15.33:0/1690421164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f897009aeb0 con 0x564350bb1950 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.661+0000 7f89697fa700 1 -- 172.21.15.33:0/1690421164 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f897c01fe40 con 0x564350bb1950 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.661+0000 7f8993d10b80 10 client.16356 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.661+0000 7f8993d10b80 1 -- 172.21.15.33:0/1690421164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56435092ab60 con 0x564350bb1950 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.661+0000 7f89697fa700 1 -- 172.21.15.33:0/1690421164 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f897009aeb0 con 0x564350bb1950 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.661+0000 7f89697fa700 1 -- 172.21.15.33:0/1690421164 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f897c004d10 con 0x564350bb1950 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.661+0000 7f8993d10b80 10 client.16356 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.661+0000 7f8993d10b80 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.661+0000 7f8993d10b80 20 client.16356 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.661+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1690421164 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564350c1cd70 0x564350c1f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.661+0000 7f8993d10b80 4 client.16356 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.661+0000 7f8993d10b80 1 -- 172.21.15.33:0/1690421164 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564350af5ec0 con 0x564350c1cd70 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.661+0000 7f89850f1700 1 --2- 172.21.15.33:0/1690421164 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564350c1cd70 0x564350c1f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f89697fa700 10 client.16356 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f89850f1700 1 --2- 172.21.15.33:0/1690421164 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564350c1cd70 0x564350c1f190 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f2d9ac2f9623fa7 server_cookie=832bbe1fccf165b2 in_seq=0 out_seq=0 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f89697fa700 1 -- 172.21.15.33:0/1690421164 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564350af5ec0 con 0x564350c1cd70 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f89697fa700 10 client.16356 handle_command_reply: tid=0 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f8993d10b80 1 client.16356 shutdown 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f8993d10b80 1 -- 172.21.15.33:0/1690421164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb1950 msgr2=0x564350bbd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1690421164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb1950 0x564350bbd660 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f897c027c00 tx=0x7f897c027c30 comp rx=0 tx=0).stop 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f8993d10b80 1 -- 172.21.15.33:0/1690421164 shutdown_connections 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1690421164 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564350c1cd70 0x564350c1f190 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1690421164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564350bb2c70 0x564350c1b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1690421164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564350bb1950 0x564350bbd660 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f89697fa700 0 client.16356 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f8993d10b80 1 --2- 172.21.15.33:0/1690421164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564350bb22b0 0x564350bbdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f8993d10b80 1 -- 172.21.15.33:0/1690421164 >> 172.21.15.33:0/1690421164 conn(0x564350b0b210 msgr2=0x564350bba410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f8993d10b80 1 -- 172.21.15.33:0/1690421164 shutdown_connections 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.662+0000 7f8993d10b80 1 -- 172.21.15.33:0/1690421164 wait complete. 2024-05-08T21:21:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8993d10b80 20 client.16356 trim_cache size 0 max 16384 2024-05-08T21:21:42.672 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:21:42.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8987b56700 1 -- 172.21.15.33:0/4249912786 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f896c0a1120 msgr2=0x7f896c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8987b56700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f896c0a1120 0x7f896c0a35e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8980103100 tx=0x7f897c009260 comp rx=0 tx=0).stop 2024-05-08T21:21:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8987b56700 1 -- 172.21.15.33:0/4249912786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980104220 msgr2=0x7f8980197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8987b56700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980104220 0x7f8980197210 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8970003f60 tx=0x7f8970004730 comp rx=0 tx=0).stop 2024-05-08T21:21:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8987b56700 1 -- 172.21.15.33:0/4249912786 shutdown_connections 2024-05-08T21:21:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8987b56700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f896c0a1120 0x7f896c0a35e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8987b56700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8980106b50 0x7f898019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8987b56700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89801018f0 0x7f8980196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8987b56700 1 --2- 172.21.15.33:0/4249912786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8980104220 0x7f8980197210 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8987b56700 1 -- 172.21.15.33:0/4249912786 >> 172.21.15.33:0/4249912786 conn(0x7f89800fb300 msgr2=0x7f8980103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8987b56700 1 -- 172.21.15.33:0/4249912786 shutdown_connections 2024-05-08T21:21:42.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:42.663+0000 7f8987b56700 1 -- 172.21.15.33:0/4249912786 wait complete. 2024-05-08T21:21:42.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:42 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2438357799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:21:42.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:42 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2438357799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:21:43.680 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:21:43.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:21:43.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:43 smithi071 ceph-mon[47074]: pgmap v372: 129 pgs: 129 active+clean; 112 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2024-05-08T21:21:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.841+0000 7f650da9d700 1 -- 172.21.15.33:0/857271795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6508101940 msgr2=0x7f6508103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.841+0000 7f650da9d700 1 --2- 172.21.15.33:0/857271795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6508101940 0x7f6508103d30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009990 tx=0x7f64f8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.841+0000 7f650da9d700 1 -- 172.21.15.33:0/857271795 shutdown_connections 2024-05-08T21:21:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.841+0000 7f650da9d700 1 --2- 172.21.15.33:0/857271795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508106ba0 0x7f6508108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.841+0000 7f650da9d700 1 --2- 172.21.15.33:0/857271795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6508101940 0x7f6508103d30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.841+0000 7f650da9d700 1 --2- 172.21.15.33:0/857271795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6508104270 0x7f6508106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.841+0000 7f650da9d700 1 -- 172.21.15.33:0/857271795 >> 172.21.15.33:0/857271795 conn(0x7f65080fb290 msgr2=0x7f65080fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.841+0000 7f650da9d700 1 -- 172.21.15.33:0/857271795 shutdown_connections 2024-05-08T21:21:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.841+0000 7f650da9d700 1 -- 172.21.15.33:0/857271795 wait complete. 2024-05-08T21:21:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.842+0000 7f650da9d700 1 Processor -- start 2024-05-08T21:21:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.842+0000 7f650da9d700 1 -- start start 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.842+0000 7f650da9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101940 0x7f6508196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.842+0000 7f650da9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6508104270 0x7f6508197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.842+0000 7f650da9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6508106ba0 0x7f650819e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.842+0000 7f650da9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650819e840 con 0x7f6508101940 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.842+0000 7f650da9d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650819e9b0 con 0x7f6508104270 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.842+0000 7f650da9d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650819eb70 con 0x7f6508106ba0 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.842+0000 7f65077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101940 0x7f6508196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.842+0000 7f6506ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6508104270 0x7f6508197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f65077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101940 0x7f6508196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52908/0 (socket says 172.21.15.33:52908) 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f65077fe700 1 -- 172.21.15.33:0/2821012381 learned_addr learned my addr 172.21.15.33:0/2821012381 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f6507fff700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6508106ba0 0x7f650819e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f65077fe700 1 -- 172.21.15.33:0/2821012381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6508106ba0 msgr2=0x7f650819e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f65077fe700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6508106ba0 0x7f650819e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f65077fe700 1 -- 172.21.15.33:0/2821012381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6508104270 msgr2=0x7f6508197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f65077fe700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6508104270 0x7f6508197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f65077fe700 1 -- 172.21.15.33:0/2821012381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6500009c40 con 0x7f6508101940 2024-05-08T21:21:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f65077fe700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101940 0x7f6508196d50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f64f8006010 tx=0x7f64f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f6504ff9700 1 -- 172.21.15.33:0/2821012381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f801d070 con 0x7f6508101940 2024-05-08T21:21:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f650da9d700 1 -- 172.21.15.33:0/2821012381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f80095f0 con 0x7f6508101940 2024-05-08T21:21:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f6504ff9700 1 -- 172.21.15.33:0/2821012381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64f8005a00 con 0x7f6508101940 2024-05-08T21:21:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f6504ff9700 1 -- 172.21.15.33:0/2821012381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f800fd30 con 0x7f6508101940 2024-05-08T21:21:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.843+0000 7f650da9d700 1 -- 172.21.15.33:0/2821012381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650819f090 con 0x7f6508101940 2024-05-08T21:21:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.846+0000 7f6504ff9700 1 -- 172.21.15.33:0/2821012381 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f64f800b920 con 0x7f6508101940 2024-05-08T21:21:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.846+0000 7f6504ff9700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f64f00a1120 0x7f64f00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.846+0000 7f6506ffd700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f64f00a1120 0x7f64f00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.846+0000 7f6504ff9700 1 -- 172.21.15.33:0/2821012381 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f64f80cde20 con 0x7f6508101940 2024-05-08T21:21:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.847+0000 7f6506ffd700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f64f00a1120 0x7f64f00a35e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009c60 tx=0x7f64fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.847+0000 7f6519c57b80 1 Processor -- start 2024-05-08T21:21:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.847+0000 7f6519c57b80 1 -- start start 2024-05-08T21:21:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6519c57b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb2bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6519c57b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2c330 0x55880fb2c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6519c57b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2ccf0 0x55880fb2d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6519c57b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880fa7e1d0 con 0x55880fb2c330 2024-05-08T21:21:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6519c57b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880fa7bb90 con 0x55880fb2b9d0 2024-05-08T21:21:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6519c57b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880fa77480 con 0x55880fb2ccf0 2024-05-08T21:21:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f65077fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb2bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6506ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2c330 0x55880fb2c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f65077fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb2bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37462/0 (socket says 172.21.15.33:37462) 2024-05-08T21:21:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6507fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2ccf0 0x55880fb2d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f65077fe700 1 -- 172.21.15.33:0/1492849990 learned_addr learned my addr 172.21.15.33:0/1492849990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6506ffd700 1 -- 172.21.15.33:0/1492849990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2ccf0 msgr2=0x55880fb2d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6506ffd700 1 --2- 172.21.15.33:0/1492849990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2ccf0 0x55880fb2d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6506ffd700 1 -- 172.21.15.33:0/1492849990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 msgr2=0x55880fb2bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6506ffd700 1 --2- 172.21.15.33:0/1492849990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb2bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6506ffd700 1 -- 172.21.15.33:0/1492849990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55880f8f8430 con 0x55880fb2c330 2024-05-08T21:21:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f65077fe700 1 --2- 172.21.15.33:0/1492849990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb2bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.848+0000 7f6506ffd700 1 --2- 172.21.15.33:0/1492849990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2c330 0x55880fb2c7b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00e3b0 tx=0x7f64fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88054c9007f2e23 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.849+0000 7f64edffb700 1 -- 172.21.15.33:0/1492849990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00ed50 con 0x55880fb2c330 2024-05-08T21:21:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.849+0000 7f64edffb700 1 -- 172.21.15.33:0/1492849990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64fc021070 con 0x55880fb2c330 2024-05-08T21:21:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.849+0000 7f64edffb700 1 -- 172.21.15.33:0/1492849990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc003e10 con 0x55880fb2c330 2024-05-08T21:21:43.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:43 smithi082 ceph-mon[47512]: pgmap v372: 129 pgs: 129 active+clean; 112 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.849+0000 7f6519c57b80 1 -- 172.21.15.33:0/1492849990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2c330 msgr2=0x55880fb2c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.849+0000 7f6519c57b80 1 --2- 172.21.15.33:0/1492849990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2c330 0x55880fb2c7b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00e3b0 tx=0x7f64fc014000 comp rx=0 tx=0).stop 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.849+0000 7f6519c57b80 1 -- 172.21.15.33:0/1492849990 shutdown_connections 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.849+0000 7f6519c57b80 1 --2- 172.21.15.33:0/1492849990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2ccf0 0x55880fb2d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.849+0000 7f6519c57b80 1 --2- 172.21.15.33:0/1492849990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb2bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.849+0000 7f6519c57b80 1 --2- 172.21.15.33:0/1492849990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2c330 0x55880fb2c7b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.849+0000 7f6519c57b80 1 -- 172.21.15.33:0/1492849990 >> 172.21.15.33:0/1492849990 conn(0x55880fa85290 msgr2=0x55880fb21e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.850+0000 7f6519c57b80 1 -- 172.21.15.33:0/1492849990 shutdown_connections 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.850+0000 7f6519c57b80 1 -- 172.21.15.33:0/1492849990 wait complete. 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.850+0000 7f6519c57b80 1 -- start start 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.850+0000 7f6519c57b80 1 Processor -- start 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.850+0000 7f6519c57b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2b9d0 0x55880fb35270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.850+0000 7f6519c57b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2c330 0x55880fb357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.850+0000 7f6519c57b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2ccf0 0x55880fb3c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.850+0000 7f6519c57b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880fa135e0 con 0x55880fb2ccf0 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.850+0000 7f6519c57b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880fa13280 con 0x55880fb2c330 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.850+0000 7f6519c57b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880f8fa720 con 0x55880fb2b9d0 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f6519c57b80 10 client.0 _ll_register_callbacks cb 0x55880fa1cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f65077fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2b9d0 0x55880fb35270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f6507fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2ccf0 0x55880fb3c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f65077fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2b9d0 0x55880fb35270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:48188/0 (socket says 172.21.15.33:48188) 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f65077fe700 1 -- 172.21.15.33:0/2467824218 learned_addr learned my addr 172.21.15.33:0/2467824218 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f6506ffd700 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2c330 0x55880fb357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f65077fe700 1 -- 172.21.15.33:0/2467824218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2c330 msgr2=0x55880fb357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f65077fe700 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2c330 0x55880fb357b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f65077fe700 1 -- 172.21.15.33:0/2467824218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2ccf0 msgr2=0x55880fb3c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f65077fe700 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2ccf0 0x55880fb3c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f65077fe700 1 -- 172.21.15.33:0/2467824218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650819f090 con 0x55880fb2b9d0 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f6507fff700 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2ccf0 0x55880fb3c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f65077fe700 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2b9d0 0x55880fb35270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009350 tx=0x7f64f809add0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a0a93d6d0b6d20c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f64e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f64e6ffd700 1 -- 172.21.15.33:0/2467824218 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80a10c0 con 0x55880fb2b9d0 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f64e6ffd700 1 -- 172.21.15.33:0/2467824218 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f64f809b680 con 0x55880fb2b9d0 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.851+0000 7f6519c57b80 1 -- 172.21.15.33:0/2467824218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55880f9d02f0 con 0x55880fb2b9d0 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.852+0000 7f64e6ffd700 1 -- 172.21.15.33:0/2467824218 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64f809a8e0 con 0x55880fb2b9d0 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.852+0000 7f6519c57b80 10 client.26257 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.852+0000 7f6519c57b80 1 -- 172.21.15.33:0/2467824218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f64fc017070 con 0x55880fb2b9d0 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.852+0000 7f64e6ffd700 1 -- 172.21.15.33:0/2467824218 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64f80b9030 con 0x55880fb2b9d0 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.852+0000 7f64e6ffd700 1 -- 172.21.15.33:0/2467824218 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f64f809a470 con 0x55880fb2b9d0 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.852+0000 7f6519c57b80 10 client.26257 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.852+0000 7f6519c57b80 10 client.26257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.852+0000 7f6519c57b80 20 client.26257 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.852+0000 7f6519c57b80 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55880fb40de0 0x55880fb43200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.852+0000 7f6519c57b80 4 client.26257 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.852+0000 7f6519c57b80 1 -- 172.21.15.33:0/2467824218 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55880faf6890 con 0x55880fb40de0 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.852+0000 7f6506ffd700 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55880fb40de0 0x55880fb43200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.853+0000 7f64e6ffd700 10 client.26257 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.853+0000 7f6506ffd700 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55880fb40de0 0x55880fb43200 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35ba7ba2f52ce087 server_cookie=41f5b7eece2780b2 in_seq=0 out_seq=0 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.853+0000 7f64e6ffd700 1 -- 172.21.15.33:0/2467824218 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55880faf6890 con 0x55880fb40de0 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.853+0000 7f64e6ffd700 10 client.26257 handle_command_reply: tid=0 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.853+0000 7f6519c57b80 1 client.26257 shutdown 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.854+0000 7f6519c57b80 1 -- 172.21.15.33:0/2467824218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2b9d0 msgr2=0x55880fb35270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.854+0000 7f6519c57b80 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2b9d0 0x55880fb35270 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009350 tx=0x7f64f809add0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.854+0000 7f6519c57b80 1 -- 172.21.15.33:0/2467824218 shutdown_connections 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.854+0000 7f6519c57b80 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55880fb40de0 0x55880fb43200 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.854+0000 7f6519c57b80 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2ccf0 0x55880fb3c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.854+0000 7f6519c57b80 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2b9d0 0x55880fb35270 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.854+0000 7f6519c57b80 1 --2- 172.21.15.33:0/2467824218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2c330 0x55880fb357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.854+0000 7f64e6ffd700 0 client.26257 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.854+0000 7f6519c57b80 1 -- 172.21.15.33:0/2467824218 >> 172.21.15.33:0/2467824218 conn(0x55880fa85290 msgr2=0x55880fb21620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:43.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.854+0000 7f6519c57b80 1 -- 172.21.15.33:0/2467824218 shutdown_connections 2024-05-08T21:21:43.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.854+0000 7f6519c57b80 1 -- 172.21.15.33:0/2467824218 wait complete. 2024-05-08T21:21:43.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.854+0000 7f6519c57b80 20 client.26257 trim_cache size 0 max 16384 2024-05-08T21:21:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f6519c57b80 1 Processor -- start 2024-05-08T21:21:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f6519c57b80 1 -- start start 2024-05-08T21:21:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f6519c57b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb47b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f6519c57b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2c330 0x55880fb3c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f6519c57b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2ccf0 0x55880fb2fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f6519c57b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880fb1c880 con 0x55880fb2c330 2024-05-08T21:21:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f6519c57b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880fb235f0 con 0x55880fb2b9d0 2024-05-08T21:21:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f6519c57b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880fb23830 con 0x55880fb2ccf0 2024-05-08T21:21:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f6506ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2c330 0x55880fb3c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f65077fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb47b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f6506ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2c330 0x55880fb3c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52926/0 (socket says 172.21.15.33:52926) 2024-05-08T21:21:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f6506ffd700 1 -- 172.21.15.33:0/3879520911 learned_addr learned my addr 172.21.15.33:0/3879520911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.874+0000 7f6507fff700 1 --2- 172.21.15.33:0/3879520911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2ccf0 0x55880fb2fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.875+0000 7f65077fe700 1 -- 172.21.15.33:0/3879520911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2ccf0 msgr2=0x55880fb2fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.875+0000 7f65077fe700 1 --2- 172.21.15.33:0/3879520911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2ccf0 0x55880fb2fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.875+0000 7f65077fe700 1 -- 172.21.15.33:0/3879520911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2c330 msgr2=0x55880fb3c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.875+0000 7f65077fe700 1 --2- 172.21.15.33:0/3879520911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2c330 0x55880fb3c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.875+0000 7f65077fe700 1 -- 172.21.15.33:0/3879520911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55880fb1e560 con 0x55880fb2b9d0 2024-05-08T21:21:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.875+0000 7f65077fe700 1 --2- 172.21.15.33:0/3879520911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb47b60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f64f80b9430 tx=0x7f64f8005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f732583637d8133 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.875+0000 7f64ee7fc700 1 -- 172.21.15.33:0/3879520911 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80a2ee0 con 0x55880fb2b9d0 2024-05-08T21:21:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.875+0000 7f64ee7fc700 1 -- 172.21.15.33:0/3879520911 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64f80b9070 con 0x55880fb2b9d0 2024-05-08T21:21:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.875+0000 7f64ee7fc700 1 -- 172.21.15.33:0/3879520911 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f809af80 con 0x55880fb2b9d0 2024-05-08T21:21:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.876+0000 7f6519c57b80 1 -- 172.21.15.33:0/3879520911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 msgr2=0x55880fb47b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.876+0000 7f6519c57b80 1 --2- 172.21.15.33:0/3879520911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb47b60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f64f80b9430 tx=0x7f64f8005910 comp rx=0 tx=0).stop 2024-05-08T21:21:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.876+0000 7f6519c57b80 1 -- 172.21.15.33:0/3879520911 shutdown_connections 2024-05-08T21:21:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.876+0000 7f6519c57b80 1 --2- 172.21.15.33:0/3879520911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2ccf0 0x55880fb2fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.876+0000 7f6519c57b80 1 --2- 172.21.15.33:0/3879520911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb47b60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.876+0000 7f6519c57b80 1 --2- 172.21.15.33:0/3879520911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2c330 0x55880fb3c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.876+0000 7f6519c57b80 1 -- 172.21.15.33:0/3879520911 >> 172.21.15.33:0/3879520911 conn(0x55880fa85290 msgr2=0x55880fb34780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.876+0000 7f6519c57b80 1 -- 172.21.15.33:0/3879520911 shutdown_connections 2024-05-08T21:21:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.876+0000 7f6519c57b80 1 -- 172.21.15.33:0/3879520911 wait complete. 2024-05-08T21:21:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.876+0000 7f6519c57b80 1 -- start start 2024-05-08T21:21:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.877+0000 7f6519c57b80 1 Processor -- start 2024-05-08T21:21:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.877+0000 7f6519c57b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb2abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.877+0000 7f6519c57b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2c330 0x55880fb2b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.877+0000 7f6519c57b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2ccf0 0x55880fb3c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.877+0000 7f6519c57b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880fb27700 con 0x55880fb2ccf0 2024-05-08T21:21:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.877+0000 7f6519c57b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880fb27870 con 0x55880fb2b9d0 2024-05-08T21:21:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.877+0000 7f6519c57b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55880fb279b0 con 0x55880fb2c330 2024-05-08T21:21:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.877+0000 7f6519c57b80 10 client.0 _ll_register_callbacks cb 0x55880fa1cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.877+0000 7f65077fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb2abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.877+0000 7f65077fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb2abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37504/0 (socket says 172.21.15.33:37504) 2024-05-08T21:21:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.877+0000 7f65077fe700 1 -- 172.21.15.33:0/1255083922 learned_addr learned my addr 172.21.15.33:0/1255083922 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f6506ffd700 1 --2- 172.21.15.33:0/1255083922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2c330 0x55880fb2b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f65077fe700 1 -- 172.21.15.33:0/1255083922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2c330 msgr2=0x55880fb2b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f65077fe700 1 --2- 172.21.15.33:0/1255083922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2c330 0x55880fb2b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f65077fe700 1 -- 172.21.15.33:0/1255083922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2ccf0 msgr2=0x55880fb3c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f65077fe700 1 --2- 172.21.15.33:0/1255083922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2ccf0 0x55880fb3c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f65077fe700 1 -- 172.21.15.33:0/1255083922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc017070 con 0x55880fb2b9d0 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f65077fe700 1 --2- 172.21.15.33:0/1255083922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb2abc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f64f8097480 tx=0x7f64f8099530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6db60759dc62e6cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f64e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f64e6ffd700 1 -- 172.21.15.33:0/1255083922 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80a2ee0 con 0x55880fb2b9d0 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f64e6ffd700 1 -- 172.21.15.33:0/1255083922 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f64f809edb0 con 0x55880fb2b9d0 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f6519c57b80 1 -- 172.21.15.33:0/1255083922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64fc021070 con 0x55880fb2b9d0 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f64e6ffd700 1 -- 172.21.15.33:0/1255083922 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64f809b6f0 con 0x55880fb2b9d0 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f6519c57b80 10 client.26243 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f6519c57b80 1 -- 172.21.15.33:0/1255083922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55880f8a4be0 con 0x55880fb2b9d0 2024-05-08T21:21:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.878+0000 7f64e6ffd700 1 -- 172.21.15.33:0/1255083922 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f64f807f080 con 0x55880fb2b9d0 2024-05-08T21:21:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.879+0000 7f64e6ffd700 1 -- 172.21.15.33:0/1255083922 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f64f80b3aa0 con 0x55880fb2b9d0 2024-05-08T21:21:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.879+0000 7f6519c57b80 10 client.26243 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.879+0000 7f6519c57b80 10 client.26243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.879+0000 7f6519c57b80 20 client.26243 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.879+0000 7f6519c57b80 1 --2- 172.21.15.33:0/1255083922 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55880fb93d50 0x55880fb96170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.879+0000 7f6519c57b80 4 client.26243 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:21:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.879+0000 7f6519c57b80 1 -- 172.21.15.33:0/1255083922 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55880fa6ff40 con 0x55880fb93d50 2024-05-08T21:21:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.879+0000 7f6506ffd700 1 --2- 172.21.15.33:0/1255083922 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55880fb93d50 0x55880fb96170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.879+0000 7f64e6ffd700 10 client.26243 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.879+0000 7f6506ffd700 1 --2- 172.21.15.33:0/1255083922 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55880fb93d50 0x55880fb96170 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7de6a5576b495b2e server_cookie=5fa0f12d7ff16857 in_seq=0 out_seq=0 2024-05-08T21:21:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.881+0000 7f64e6ffd700 1 -- 172.21.15.33:0/1255083922 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55880fa6ff40 con 0x55880fb93d50 2024-05-08T21:21:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.881+0000 7f64e6ffd700 10 client.26243 handle_command_reply: tid=0 2024-05-08T21:21:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.881+0000 7f6519c57b80 1 client.26243 shutdown 2024-05-08T21:21:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f6519c57b80 1 -- 172.21.15.33:0/1255083922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 msgr2=0x55880fb2abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f6519c57b80 1 --2- 172.21.15.33:0/1255083922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb2abc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f64f8097480 tx=0x7f64f8099530 comp rx=0 tx=0).stop 2024-05-08T21:21:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f6519c57b80 1 -- 172.21.15.33:0/1255083922 shutdown_connections 2024-05-08T21:21:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f6519c57b80 1 --2- 172.21.15.33:0/1255083922 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55880fb93d50 0x55880fb96170 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f6519c57b80 1 --2- 172.21.15.33:0/1255083922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55880fb2ccf0 0x55880fb3c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f6519c57b80 1 --2- 172.21.15.33:0/1255083922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55880fb2b9d0 0x55880fb2abc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f6519c57b80 1 --2- 172.21.15.33:0/1255083922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55880fb2c330 0x55880fb2b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f64e6ffd700 0 client.26243 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f6519c57b80 1 -- 172.21.15.33:0/1255083922 >> 172.21.15.33:0/1255083922 conn(0x55880fa85290 msgr2=0x55880fb2fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f6519c57b80 1 -- 172.21.15.33:0/1255083922 shutdown_connections 2024-05-08T21:21:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f6519c57b80 1 -- 172.21.15.33:0/1255083922 wait complete. 2024-05-08T21:21:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f6519c57b80 20 client.26243 trim_cache size 0 max 16384 2024-05-08T21:21:43.887 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:21:43.887 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:21:43.887 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "746e9193-5bfb-4e01-b393-80d07a14e740", 2024-05-08T21:21:43.887 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:21:43.887 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:21:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f650da9d700 1 -- 172.21.15.33:0/2821012381 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f64f00a1120 msgr2=0x7f64f00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f650da9d700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f64f00a1120 0x7f64f00a35e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009c60 tx=0x7f64fc009260 comp rx=0 tx=0).stop 2024-05-08T21:21:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f650da9d700 1 -- 172.21.15.33:0/2821012381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101940 msgr2=0x7f6508196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.882+0000 7f650da9d700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101940 0x7f6508196d50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f64f8006010 tx=0x7f64f8004730 comp rx=0 tx=0).stop 2024-05-08T21:21:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.883+0000 7f650da9d700 1 -- 172.21.15.33:0/2821012381 shutdown_connections 2024-05-08T21:21:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.883+0000 7f650da9d700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6508106ba0 0x7f650819e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.883+0000 7f650da9d700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101940 0x7f6508196d50 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.883+0000 7f650da9d700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f64f00a1120 0x7f64f00a35e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.883+0000 7f650da9d700 1 --2- 172.21.15.33:0/2821012381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6508104270 0x7f6508197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.883+0000 7f650da9d700 1 -- 172.21.15.33:0/2821012381 >> 172.21.15.33:0/2821012381 conn(0x7f65080fb290 msgr2=0x7f6508103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.883+0000 7f650da9d700 1 -- 172.21.15.33:0/2821012381 shutdown_connections 2024-05-08T21:21:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:43.883+0000 7f650da9d700 1 -- 172.21.15.33:0/2821012381 wait complete. 2024-05-08T21:21:43.901 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:43 smithi033 ceph-mon[42528]: pgmap v372: 129 pgs: 129 active+clean; 112 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 225 op/s 2024-05-08T21:21:43.903 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:21:44.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.075+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/1444885671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a54074fe0 msgr2=0x7f1a54073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.075+0000 7f1a5c2b4700 1 --2- 172.21.15.33:0/1444885671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a54074fe0 0x7f1a54073430 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009990 tx=0x7f1a50009ca0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.075+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/1444885671 shutdown_connections 2024-05-08T21:21:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.075+0000 7f1a5c2b4700 1 --2- 172.21.15.33:0/1444885671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a5410d450 0x7f1a5410f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.075+0000 7f1a5c2b4700 1 --2- 172.21.15.33:0/1444885671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a54074fe0 0x7f1a54073430 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.075+0000 7f1a5c2b4700 1 --2- 172.21.15.33:0/1444885671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a54073a00 0x7f1a54073e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.075+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/1444885671 >> 172.21.15.33:0/1444885671 conn(0x7f1a540fc3b0 msgr2=0x7f1a540fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.075+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/1444885671 shutdown_connections 2024-05-08T21:21:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.075+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/1444885671 wait complete. 2024-05-08T21:21:44.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.076+0000 7f1a5c2b4700 1 Processor -- start 2024-05-08T21:21:44.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.076+0000 7f1a5c2b4700 1 -- start start 2024-05-08T21:21:44.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.077+0000 7f1a5c2b4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a54073a00 0x7f1a5419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.077+0000 7f1a5c2b4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a54074fe0 0x7f1a5419d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.077+0000 7f1a5c2b4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a5410d450 0x7f1a541a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.077+0000 7f1a5c2b4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a541a4e50 con 0x7f1a5410d450 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.077+0000 7f1a5c2b4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a541a4f90 con 0x7f1a54074fe0 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.077+0000 7f1a5c2b4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a541a5120 con 0x7f1a54073a00 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.077+0000 7f1a5984f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a54074fe0 0x7f1a5419d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.077+0000 7f1a5984f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a54074fe0 0x7f1a5419d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37516/0 (socket says 172.21.15.33:37516) 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.077+0000 7f1a5984f700 1 -- 172.21.15.33:0/939115749 learned_addr learned my addr 172.21.15.33:0/939115749 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.077+0000 7f1a5a851700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a5410d450 0x7f1a541a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.078+0000 7f1a5a050700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a54073a00 0x7f1a5419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.078+0000 7f1a5984f700 1 -- 172.21.15.33:0/939115749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a54073a00 msgr2=0x7f1a5419d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.078+0000 7f1a5984f700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a54073a00 0x7f1a5419d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.078+0000 7f1a5984f700 1 -- 172.21.15.33:0/939115749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a5410d450 msgr2=0x7f1a541a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:44.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.078+0000 7f1a5984f700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a5410d450 0x7f1a541a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.078+0000 7f1a5984f700 1 -- 172.21.15.33:0/939115749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a500095f0 con 0x7f1a54074fe0 2024-05-08T21:21:44.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.078+0000 7f1a5a050700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a54073a00 0x7f1a5419d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:21:44.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.078+0000 7f1a5984f700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a54074fe0 0x7f1a5419d8b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400e970 tx=0x7f1a4400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:44.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.079+0000 7f1a4b7fe700 1 -- 172.21.15.33:0/939115749 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44009790 con 0x7f1a54074fe0 2024-05-08T21:21:44.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.079+0000 7f1a4b7fe700 1 -- 172.21.15.33:0/939115749 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a4400eec0 con 0x7f1a54074fe0 2024-05-08T21:21:44.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.079+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/939115749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a541a5400 con 0x7f1a54074fe0 2024-05-08T21:21:44.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.079+0000 7f1a4b7fe700 1 -- 172.21.15.33:0/939115749 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44003c10 con 0x7f1a54074fe0 2024-05-08T21:21:44.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.080+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/939115749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a541a5960 con 0x7f1a54074fe0 2024-05-08T21:21:44.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.083+0000 7f1a4b7fe700 1 -- 172.21.15.33:0/939115749 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1a44003db0 con 0x7f1a54074fe0 2024-05-08T21:21:44.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.085+0000 7f1a6846eb80 1 Processor -- start 2024-05-08T21:21:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.085+0000 7f1a6846eb80 1 -- start start 2024-05-08T21:21:44.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.085+0000 7f1a6846eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 0x55d495a55960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.085+0000 7f1a4b7fe700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1a400a11a0 0x7f1a400a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.085+0000 7f1a6846eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a55ea0 0x55d495a56320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.085+0000 7f1a6846eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a56860 0x55d495a56ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.085+0000 7f1a6846eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4959ac150 con 0x55d495a55ea0 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.085+0000 7f1a6846eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4959a9b10 con 0x55d495a56860 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.085+0000 7f1a6846eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4959a5400 con 0x55d495a55540 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5a050700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1a400a11a0 0x7f1a400a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a4b7fe700 1 -- 172.21.15.33:0/939115749 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1a44014070 con 0x7f1a54074fe0 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5984f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a55ea0 0x55d495a56320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5a851700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 0x55d495a55960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5984f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a55ea0 0x55d495a56320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52940/0 (socket says 172.21.15.33:52940) 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5984f700 1 -- 172.21.15.33:0/4108864705 learned_addr learned my addr 172.21.15.33:0/4108864705 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:44.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5a851700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 0x55d495a55960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:48210/0 (socket says 172.21.15.33:48210) 2024-05-08T21:21:44.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5a851700 1 --2- 172.21.15.33:0/4108864705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a56860 0x55d495a56ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5984f700 1 -- 172.21.15.33:0/4108864705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 msgr2=0x55d495a55960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5984f700 1 --2- 172.21.15.33:0/4108864705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 0x55d495a55960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5984f700 1 -- 172.21.15.33:0/4108864705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a56860 msgr2=0x55d495a56ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5984f700 1 --2- 172.21.15.33:0/4108864705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a56860 0x55d495a56ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5984f700 1 -- 172.21.15.33:0/4108864705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4958263b0 con 0x55d495a55ea0 2024-05-08T21:21:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5984f700 1 --2- 172.21.15.33:0/4108864705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a55ea0 0x55d495a56320 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1a440a1e80 tx=0x7f1a440963a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fea26110b4586b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.086+0000 7f1a5a050700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1a400a11a0 0x7f1a400a3660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000b2e0 tx=0x7f1a50005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.087+0000 7f1a48da4700 1 -- 172.21.15.33:0/4108864705 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a440a6dc0 con 0x55d495a55ea0 2024-05-08T21:21:44.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.087+0000 7f1a48da4700 1 -- 172.21.15.33:0/4108864705 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a4400ccd0 con 0x55d495a55ea0 2024-05-08T21:21:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.087+0000 7f1a48da4700 1 -- 172.21.15.33:0/4108864705 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a440a9750 con 0x55d495a55ea0 2024-05-08T21:21:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.087+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4108864705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a55ea0 msgr2=0x55d495a56320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.087+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/4108864705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a55ea0 0x55d495a56320 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1a440a1e80 tx=0x7f1a440963a0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.088+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4108864705 shutdown_connections 2024-05-08T21:21:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.088+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/4108864705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a56860 0x55d495a56ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.088+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/4108864705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 0x55d495a55960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.088+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/4108864705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a55ea0 0x55d495a56320 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.088+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4108864705 >> 172.21.15.33:0/4108864705 conn(0x55d4959b3210 msgr2=0x55d495a4fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.088+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4108864705 shutdown_connections 2024-05-08T21:21:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.088+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4108864705 wait complete. 2024-05-08T21:21:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.088+0000 7f1a6846eb80 1 -- start start 2024-05-08T21:21:44.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.088+0000 7f1a6846eb80 1 Processor -- start 2024-05-08T21:21:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.089+0000 7f1a6846eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55540 0x55d495a5ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.089+0000 7f1a6846eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55ea0 0x55d495a5f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.089+0000 7f1a6846eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a56860 0x55d495a66390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.089+0000 7f1a6846eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d495941560 con 0x55d495a56860 2024-05-08T21:21:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.089+0000 7f1a6846eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d495941200 con 0x55d495a55540 2024-05-08T21:21:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.089+0000 7f1a6846eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4958286a0 con 0x55d495a55ea0 2024-05-08T21:21:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.089+0000 7f1a6846eb80 10 client.0 _ll_register_callbacks cb 0x55d49594ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.089+0000 7f1a5a851700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a56860 0x55d495a66390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.089+0000 7f1a5a851700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a56860 0x55d495a66390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52956/0 (socket says 172.21.15.33:52956) 2024-05-08T21:21:44.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.089+0000 7f1a5a851700 1 -- 172.21.15.33:0/4221236911 learned_addr learned my addr 172.21.15.33:0/4221236911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.090+0000 7f1a5984f700 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55ea0 0x55d495a5f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.090+0000 7f1a5a050700 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55540 0x55d495a5ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.090+0000 7f1a5a851700 1 -- 172.21.15.33:0/4221236911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55ea0 msgr2=0x55d495a5f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.090+0000 7f1a5a851700 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55ea0 0x55d495a5f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.090+0000 7f1a5a851700 1 -- 172.21.15.33:0/4221236911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55540 msgr2=0x55d495a5ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.090+0000 7f1a5a851700 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55540 0x55d495a5ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.090+0000 7f1a5a851700 1 -- 172.21.15.33:0/4221236911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a541a5960 con 0x55d495a56860 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.090+0000 7f1a5a851700 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a56860 0x55d495a66390 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c008ce0 tx=0x7f1a4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf215f5cd374f920 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.090+0000 7f1a3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.090+0000 7f1a3dffb700 1 -- 172.21.15.33:0/4221236911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c00e030 con 0x55d495a56860 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.090+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4221236911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4958fe270 con 0x55d495a56860 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.090+0000 7f1a3dffb700 1 -- 172.21.15.33:0/4221236911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1a4c01abe0 con 0x55d495a56860 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.091+0000 7f1a3dffb700 1 -- 172.21.15.33:0/4221236911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a4c01ada0 con 0x55d495a56860 2024-05-08T21:21:44.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.091+0000 7f1a6846eb80 10 client.16383 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.091+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4221236911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d4957d2b60 con 0x55d495a56860 2024-05-08T21:21:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.091+0000 7f1a3dffb700 1 -- 172.21.15.33:0/4221236911 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d4958fe270 con 0x55d495a56860 2024-05-08T21:21:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.091+0000 7f1a3dffb700 1 -- 172.21.15.33:0/4221236911 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1a4c004500 con 0x55d495a56860 2024-05-08T21:21:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.091+0000 7f1a6846eb80 10 client.16383 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.091+0000 7f1a6846eb80 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.091+0000 7f1a6846eb80 20 client.16383 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.091+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d495a6ad50 0x55d495a6d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.091+0000 7f1a6846eb80 4 client.16383 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:21:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.091+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4221236911 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d495a24810 con 0x55d495a6ad50 2024-05-08T21:21:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.091+0000 7f1a5a050700 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d495a6ad50 0x55d495a6d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.092+0000 7f1a5a050700 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d495a6ad50 0x55d495a6d170 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3ff51ef2ebec880 server_cookie=b2aec95d9793d200 in_seq=0 out_seq=0 2024-05-08T21:21:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.092+0000 7f1a3dffb700 10 client.16383 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:44.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.092+0000 7f1a3dffb700 1 -- 172.21.15.33:0/4221236911 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d495a24810 con 0x55d495a6ad50 2024-05-08T21:21:44.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.092+0000 7f1a3dffb700 10 client.16383 handle_command_reply: tid=0 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.092+0000 7f1a6846eb80 1 client.16383 shutdown 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.093+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4221236911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a56860 msgr2=0x55d495a66390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.093+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a56860 0x55d495a66390 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c008ce0 tx=0x7f1a4c009bd0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.093+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4221236911 shutdown_connections 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.093+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d495a6ad50 0x55d495a6d170 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.093+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a56860 0x55d495a66390 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.093+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55540 0x55d495a5ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.093+0000 7f1a3dffb700 0 client.16383 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.093+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/4221236911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55ea0 0x55d495a5f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.093+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4221236911 >> 172.21.15.33:0/4221236911 conn(0x55d4959b3210 msgr2=0x55d495a4f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.093+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4221236911 shutdown_connections 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.093+0000 7f1a6846eb80 1 -- 172.21.15.33:0/4221236911 wait complete. 2024-05-08T21:21:44.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.093+0000 7f1a6846eb80 20 client.16383 trim_cache size 0 max 16384 2024-05-08T21:21:44.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.116+0000 7f1a6846eb80 1 Processor -- start 2024-05-08T21:21:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.116+0000 7f1a6846eb80 1 -- start start 2024-05-08T21:21:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a6846eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a55540 0x55d495a5e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a6846eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55ea0 0x55d495a662b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a6846eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a56860 0x55d495a59790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a6846eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d495a4cdc0 con 0x55d495a55540 2024-05-08T21:21:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a6846eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d495a667f0 con 0x55d495a55ea0 2024-05-08T21:21:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a6846eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d495a51570 con 0x55d495a56860 2024-05-08T21:21:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a5984f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55ea0 0x55d495a662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a5984f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55ea0 0x55d495a662b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37534/0 (socket says 172.21.15.33:37534) 2024-05-08T21:21:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a5984f700 1 -- 172.21.15.33:0/280156281 learned_addr learned my addr 172.21.15.33:0/280156281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a5a851700 1 --2- 172.21.15.33:0/280156281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a56860 0x55d495a59790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a5a050700 1 --2- 172.21.15.33:0/280156281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a55540 0x55d495a5e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a5984f700 1 -- 172.21.15.33:0/280156281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a56860 msgr2=0x55d495a59790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a5984f700 1 --2- 172.21.15.33:0/280156281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a56860 0x55d495a59790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a5984f700 1 -- 172.21.15.33:0/280156281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a55540 msgr2=0x55d495a5e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a5984f700 1 --2- 172.21.15.33:0/280156281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a55540 0x55d495a5e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.117+0000 7f1a5984f700 1 -- 172.21.15.33:0/280156281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d49596b560 con 0x55d495a55ea0 2024-05-08T21:21:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.118+0000 7f1a5984f700 1 --2- 172.21.15.33:0/280156281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55ea0 0x55d495a662b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1a440969a0 tx=0x7f1a44096300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89c15971c2d3cca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.118+0000 7f1a495a5700 1 -- 172.21.15.33:0/280156281 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44099f30 con 0x55d495a55ea0 2024-05-08T21:21:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.118+0000 7f1a495a5700 1 -- 172.21.15.33:0/280156281 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a440a23b0 con 0x55d495a55ea0 2024-05-08T21:21:44.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.118+0000 7f1a495a5700 1 -- 172.21.15.33:0/280156281 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4409a190 con 0x55d495a55ea0 2024-05-08T21:21:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.119+0000 7f1a6846eb80 1 -- 172.21.15.33:0/280156281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55ea0 msgr2=0x55d495a662b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.119+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/280156281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55ea0 0x55d495a662b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1a440969a0 tx=0x7f1a44096300 comp rx=0 tx=0).stop 2024-05-08T21:21:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.119+0000 7f1a6846eb80 1 -- 172.21.15.33:0/280156281 shutdown_connections 2024-05-08T21:21:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.119+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/280156281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a56860 0x55d495a59790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.119+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/280156281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a55540 0x55d495a5e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.119+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/280156281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55ea0 0x55d495a662b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.119+0000 7f1a6846eb80 1 -- 172.21.15.33:0/280156281 >> 172.21.15.33:0/280156281 conn(0x55d4959b3210 msgr2=0x55d495a56c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.119+0000 7f1a6846eb80 1 -- 172.21.15.33:0/280156281 shutdown_connections 2024-05-08T21:21:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.119+0000 7f1a6846eb80 1 -- 172.21.15.33:0/280156281 wait complete. 2024-05-08T21:21:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.119+0000 7f1a6846eb80 1 -- start start 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.119+0000 7f1a6846eb80 1 Processor -- start 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a6846eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 0x55d495a61cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a6846eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55ea0 0x55d495a62230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a6846eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a56860 0x55d495a65160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a6846eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d495a656a0 con 0x55d495a56860 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a6846eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d495a65810 con 0x55d495a55ea0 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a6846eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d495a659d0 con 0x55d495a55540 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a6846eb80 10 client.0 _ll_register_callbacks cb 0x55d49594ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a5a050700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 0x55d495a61cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a5a050700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 0x55d495a61cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:48234/0 (socket says 172.21.15.33:48234) 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a5a050700 1 -- 172.21.15.33:0/587842908 learned_addr learned my addr 172.21.15.33:0/587842908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a5a851700 1 --2- 172.21.15.33:0/587842908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a56860 0x55d495a65160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a5a050700 1 -- 172.21.15.33:0/587842908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55ea0 msgr2=0x55d495a62230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a5a050700 1 --2- 172.21.15.33:0/587842908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55ea0 0x55d495a62230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a5a050700 1 -- 172.21.15.33:0/587842908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a56860 msgr2=0x55d495a65160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a5a050700 1 --2- 172.21.15.33:0/587842908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a56860 0x55d495a65160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.120+0000 7f1a5a050700 1 -- 172.21.15.33:0/587842908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4958263b0 con 0x55d495a55540 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.121+0000 7f1a5a050700 1 --2- 172.21.15.33:0/587842908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 0x55d495a61cf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009990 tx=0x7f1a50009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed57a547f4ed6467 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.121+0000 7f1a3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.121+0000 7f1a3dffb700 1 -- 172.21.15.33:0/587842908 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5002c040 con 0x55d495a55540 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.121+0000 7f1a3dffb700 1 -- 172.21.15.33:0/587842908 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1a50005420 con 0x55d495a55540 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.121+0000 7f1a6846eb80 1 -- 172.21.15.33:0/587842908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a4400ccd0 con 0x55d495a55540 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.121+0000 7f1a3dffb700 1 -- 172.21.15.33:0/587842908 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a5001e070 con 0x55d495a55540 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.121+0000 7f1a6846eb80 10 client.26278 fetch_fsmap learned FSMap version 31 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.121+0000 7f1a6846eb80 1 -- 172.21.15.33:0/587842908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d49582acd0 con 0x55d495a55540 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.122+0000 7f1a3dffb700 1 -- 172.21.15.33:0/587842908 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1a4400ccd0 con 0x55d495a55540 2024-05-08T21:21:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.122+0000 7f1a3dffb700 1 -- 172.21.15.33:0/587842908 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 31) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1a5000ba70 con 0x55d495a55540 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.122+0000 7f1a6846eb80 10 client.26278 fetch_fsmap finished waiting for FSMap version 31 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.122+0000 7f1a6846eb80 10 client.26278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.122+0000 7f1a6846eb80 20 client.26278 populate_metadata read hostname 'smithi033' 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.122+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/587842908 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d495ac1df0 0x55d495ac4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.122+0000 7f1a6846eb80 4 client.26278 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.122+0000 7f1a6846eb80 1 -- 172.21.15.33:0/587842908 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d49599dec0 con 0x55d495ac1df0 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.122+0000 7f1a5984f700 1 --2- 172.21.15.33:0/587842908 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d495ac1df0 0x55d495ac4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a5984f700 1 --2- 172.21.15.33:0/587842908 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d495ac1df0 0x55d495ac4210 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc067767e8fc2e3e server_cookie=712c33d9d5514759 in_seq=0 out_seq=0 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a3dffb700 10 client.26278 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a3dffb700 1 -- 172.21.15.33:0/587842908 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d49599dec0 con 0x55d495ac1df0 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a3dffb700 10 client.26278 handle_command_reply: tid=0 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a6846eb80 1 client.26278 shutdown 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a6846eb80 1 -- 172.21.15.33:0/587842908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 msgr2=0x55d495a61cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/587842908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 0x55d495a61cf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009990 tx=0x7f1a50009ca0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a6846eb80 1 -- 172.21.15.33:0/587842908 shutdown_connections 2024-05-08T21:21:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/587842908 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d495ac1df0 0x55d495ac4210 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/587842908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d495a56860 0x55d495a65160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/587842908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d495a55540 0x55d495a61cf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a6846eb80 1 --2- 172.21.15.33:0/587842908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d495a55ea0 0x55d495a62230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a6846eb80 1 -- 172.21.15.33:0/587842908 >> 172.21.15.33:0/587842908 conn(0x55d4959b3210 msgr2=0x55d495a59790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.123+0000 7f1a3dffb700 0 client.26278 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:21:44.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.124+0000 7f1a6846eb80 1 -- 172.21.15.33:0/587842908 shutdown_connections 2024-05-08T21:21:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.124+0000 7f1a6846eb80 1 -- 172.21.15.33:0/587842908 wait complete. 2024-05-08T21:21:44.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.124+0000 7f1a6846eb80 20 client.26278 trim_cache size 0 max 16384 2024-05-08T21:21:44.130 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:21:44.130 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:21:44.130 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:21:44.130 INFO:teuthology.orchestra.run.smithi033.stdout: "746e9193-5bfb-4e01-b393-80d07a14e740": { 2024-05-08T21:21:44.130 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:21:44.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "746e9193-5bfb-4e01-b393-80d07a14e740", 2024-05-08T21:21:44.130 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:21:44.131 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:21:44.131 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:21:44.131 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:21:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.125+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/939115749 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1a400a11a0 msgr2=0x7f1a400a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.125+0000 7f1a5c2b4700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1a400a11a0 0x7f1a400a3660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000b2e0 tx=0x7f1a50005fb0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.125+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/939115749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a54074fe0 msgr2=0x7f1a5419d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:21:44.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.125+0000 7f1a5c2b4700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a54074fe0 0x7f1a5419d8b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400e970 tx=0x7f1a4400ec80 comp rx=0 tx=0).stop 2024-05-08T21:21:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.125+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/939115749 shutdown_connections 2024-05-08T21:21:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.125+0000 7f1a5c2b4700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a5410d450 0x7f1a541a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.125+0000 7f1a5c2b4700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a54073a00 0x7f1a5419d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.125+0000 7f1a5c2b4700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1a400a11a0 0x7f1a400a3660 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.125+0000 7f1a5c2b4700 1 --2- 172.21.15.33:0/939115749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a54074fe0 0x7f1a5419d8b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:21:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.125+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/939115749 >> 172.21.15.33:0/939115749 conn(0x7f1a540fc3b0 msgr2=0x7f1a541007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:21:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.125+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/939115749 shutdown_connections 2024-05-08T21:21:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:21:44.125+0000 7f1a5c2b4700 1 -- 172.21.15.33:0/939115749 wait complete. 2024-05-08T21:21:44.142 INFO:tasks.cephfs.filesystem:scrub status for tag:746e9193-5bfb-4e01-b393-80d07a14e740 - {'path': '/', 'tag': '746e9193-5bfb-4e01-b393-80d07a14e740', 'options': 'recursive,force'} 2024-05-08T21:21:44.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:21:44.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:44.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:44.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:44.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:44.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:44.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:44.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:44.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:44 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:21:44.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:44 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:21:44.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:44.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:44.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:44.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:44.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:44.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:44.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:44.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:44.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:44 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:45 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:21:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:45 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:45 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:45 smithi071 ceph-mon[47074]: pgmap v373: 129 pgs: 129 active+clean; 115 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2024-05-08T21:21:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:45 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:21:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:45 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:45 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:45 smithi082 ceph-mon[47512]: pgmap v373: 129 pgs: 129 active+clean; 115 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2024-05-08T21:21:45.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:45 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:21:45.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:45 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:45.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:45 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:21:45.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:45 smithi033 ceph-mon[42528]: pgmap v373: 129 pgs: 129 active+clean; 115 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 211 op/s 2024-05-08T21:21:47.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:47 smithi071 ceph-mon[47074]: pgmap v374: 129 pgs: 129 active+clean; 122 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 274 op/s 2024-05-08T21:21:47.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:47 smithi082 ceph-mon[47512]: pgmap v374: 129 pgs: 129 active+clean; 122 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 274 op/s 2024-05-08T21:21:47.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:47 smithi033 ceph-mon[42528]: pgmap v374: 129 pgs: 129 active+clean; 122 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 274 op/s 2024-05-08T21:21:49.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:49 smithi071 ceph-mon[47074]: pgmap v375: 129 pgs: 129 active+clean; 125 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-05-08T21:21:49.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:49 smithi082 ceph-mon[47512]: pgmap v375: 129 pgs: 129 active+clean; 125 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-05-08T21:21:49.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:49 smithi033 ceph-mon[42528]: pgmap v375: 129 pgs: 129 active+clean; 125 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-05-08T21:21:51.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:51 smithi071 ceph-mon[47074]: pgmap v376: 129 pgs: 129 active+clean; 135 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 273 op/s 2024-05-08T21:21:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:51 smithi082 ceph-mon[47512]: pgmap v376: 129 pgs: 129 active+clean; 135 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 273 op/s 2024-05-08T21:21:51.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:51 smithi033 ceph-mon[42528]: pgmap v376: 129 pgs: 129 active+clean; 135 MiB data, 955 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 273 op/s 2024-05-08T21:21:52.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:52 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:21:52.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:52 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:21:52.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:52 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:21:52.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:52 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:21:52.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:52 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:21:52.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:52 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:21:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:53 smithi071 ceph-mon[47074]: pgmap v377: 129 pgs: 129 active+clean; 139 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 229 op/s 2024-05-08T21:21:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:53 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:21:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:53 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:21:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:53 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:21:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:53 smithi082 ceph-mon[47512]: pgmap v377: 129 pgs: 129 active+clean; 139 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 229 op/s 2024-05-08T21:21:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:53 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:21:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:53 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:21:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:53 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:21:53.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:53 smithi033 ceph-mon[42528]: pgmap v377: 129 pgs: 129 active+clean; 139 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 229 op/s 2024-05-08T21:21:53.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:53 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:21:53.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:53 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:21:53.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:53 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:21:54.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:54 smithi071 ceph-mon[47074]: pgmap v378: 129 pgs: 129 active+clean; 140 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 225 op/s 2024-05-08T21:21:54.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:54 smithi082 ceph-mon[47512]: pgmap v378: 129 pgs: 129 active+clean; 140 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 225 op/s 2024-05-08T21:21:54.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:54 smithi033 ceph-mon[42528]: pgmap v378: 129 pgs: 129 active+clean; 140 MiB data, 999 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 225 op/s 2024-05-08T21:21:57.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:57 smithi071 ceph-mon[47074]: pgmap v379: 129 pgs: 129 active+clean; 148 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 257 op/s 2024-05-08T21:21:57.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:57 smithi082 ceph-mon[47512]: pgmap v379: 129 pgs: 129 active+clean; 148 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 257 op/s 2024-05-08T21:21:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:57 smithi033 ceph-mon[42528]: pgmap v379: 129 pgs: 129 active+clean; 148 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 257 op/s 2024-05-08T21:21:59.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:21:59 smithi071 ceph-mon[47074]: pgmap v380: 129 pgs: 129 active+clean; 149 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-05-08T21:21:59.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:21:59 smithi082 ceph-mon[47512]: pgmap v380: 129 pgs: 129 active+clean; 149 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-05-08T21:21:59.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:21:59 smithi033 ceph-mon[42528]: pgmap v380: 129 pgs: 129 active+clean; 149 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-05-08T21:22:01.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:01 smithi071 ceph-mon[47074]: pgmap v381: 129 pgs: 129 active+clean; 161 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 246 op/s 2024-05-08T21:22:01.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:01 smithi082 ceph-mon[47512]: pgmap v381: 129 pgs: 129 active+clean; 161 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 246 op/s 2024-05-08T21:22:01.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:01 smithi033 ceph-mon[42528]: pgmap v381: 129 pgs: 129 active+clean; 161 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 246 op/s 2024-05-08T21:22:03.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:03 smithi071 ceph-mon[47074]: pgmap v382: 129 pgs: 129 active+clean; 165 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 201 op/s 2024-05-08T21:22:03.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:03 smithi082 ceph-mon[47512]: pgmap v382: 129 pgs: 129 active+clean; 165 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 201 op/s 2024-05-08T21:22:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:03 smithi033 ceph-mon[42528]: pgmap v382: 129 pgs: 129 active+clean; 165 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 201 op/s 2024-05-08T21:22:04.105 INFO:tasks.workunit.client.0.smithi033.stderr:Note: switching to '12973fc04fd10d4af086901e10ffa8e48866b735'. 2024-05-08T21:22:04.105 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:22:04.105 INFO:tasks.workunit.client.0.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr: git switch -c 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr:Or undo this operation with: 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr: git switch - 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T21:22:04.106 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-05-08T21:22:04.109 INFO:tasks.workunit.client.0.smithi033.stderr:HEAD is now at 12973fc0 btrfs/022: Correct seq number in golden output 2024-05-08T21:22:04.122 INFO:tasks.workunit.client.0.smithi033.stderr:+ make -j4 2024-05-08T21:22:04.155 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize -cfi 2024-05-08T21:22:04.581 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize: putting auxiliary files in '.'. 2024-05-08T21:22:04.581 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize: copying file './config.guess' 2024-05-08T21:22:04.595 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize: copying file './config.sub' 2024-05-08T21:22:04.609 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize: copying file './install-sh' 2024-05-08T21:22:04.625 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize: copying file './ltmain.sh' 2024-05-08T21:22:04.703 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-05-08T21:22:04.703 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-05-08T21:22:04.733 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-05-08T21:22:04.760 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-05-08T21:22:04.786 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-05-08T21:22:04.813 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-05-08T21:22:04.824 INFO:tasks.workunit.client.0.smithi033.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-05-08T21:22:04.826 INFO:tasks.workunit.client.0.smithi033.stdout:cp include/install-sh . 2024-05-08T21:22:04.836 INFO:tasks.workunit.client.0.smithi033.stdout:aclocal -I m4 2024-05-08T21:22:05.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:05 smithi071 ceph-mon[47074]: pgmap v383: 129 pgs: 129 active+clean; 167 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-05-08T21:22:05.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:05 smithi082 ceph-mon[47512]: pgmap v383: 129 pgs: 129 active+clean; 167 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-05-08T21:22:05.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:05 smithi033 ceph-mon[42528]: pgmap v383: 129 pgs: 129 active+clean; 167 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-05-08T21:22:06.514 INFO:tasks.workunit.client.0.smithi033.stdout:autoheader 2024-05-08T21:22:07.110 INFO:tasks.workunit.client.0.smithi033.stdout:autoconf 2024-05-08T21:22:07.462 INFO:tasks.workunit.client.0.smithi033.stdout:./configure \ 2024-05-08T21:22:07.463 INFO:tasks.workunit.client.0.smithi033.stdout: --libexecdir=/usr/lib \ 2024-05-08T21:22:07.463 INFO:tasks.workunit.client.0.smithi033.stdout: --exec_prefix=/var/lib 2024-05-08T21:22:07.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:07 smithi071 ceph-mon[47074]: pgmap v384: 129 pgs: 129 active+clean; 170 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-05-08T21:22:07.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:07 smithi082 ceph-mon[47512]: pgmap v384: 129 pgs: 129 active+clean; 170 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-05-08T21:22:07.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:07 smithi033 ceph-mon[42528]: pgmap v384: 129 pgs: 129 active+clean; 170 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-05-08T21:22:07.889 INFO:tasks.workunit.client.0.smithi033.stdout:checking build system type... x86_64-pc-linux-gnu 2024-05-08T21:22:07.890 INFO:tasks.workunit.client.0.smithi033.stdout:checking host system type... x86_64-pc-linux-gnu 2024-05-08T21:22:07.892 INFO:tasks.workunit.client.0.smithi033.stdout:checking how to print strings... printf 2024-05-08T21:22:07.894 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc... gcc 2024-05-08T21:22:08.136 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether the C compiler works... yes 2024-05-08T21:22:08.136 INFO:tasks.workunit.client.0.smithi033.stdout:checking for C compiler default output file name... a.out 2024-05-08T21:22:08.313 INFO:tasks.workunit.client.0.smithi033.stdout:checking for suffix of executables... 2024-05-08T21:22:08.449 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether we are cross compiling... no 2024-05-08T21:22:08.531 INFO:tasks.workunit.client.0.smithi033.stdout:checking for suffix of object files... o 2024-05-08T21:22:08.577 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether we are using the GNU C compiler... yes 2024-05-08T21:22:08.619 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether gcc accepts -g... yes 2024-05-08T21:22:08.778 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-08T21:22:08.786 INFO:tasks.workunit.client.0.smithi033.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-05-08T21:22:08.792 INFO:tasks.workunit.client.0.smithi033.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-08T21:22:08.794 INFO:tasks.workunit.client.0.smithi033.stdout:checking for egrep... /usr/bin/grep -E 2024-05-08T21:22:08.796 INFO:tasks.workunit.client.0.smithi033.stdout:checking for fgrep... /usr/bin/grep -F 2024-05-08T21:22:08.801 INFO:tasks.workunit.client.0.smithi033.stdout:checking for ld used by gcc... /usr/bin/ld 2024-05-08T21:22:08.804 INFO:tasks.workunit.client.0.smithi033.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-05-08T21:22:08.813 INFO:tasks.workunit.client.0.smithi033.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-05-08T21:22:08.886 INFO:tasks.workunit.client.0.smithi033.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-05-08T21:22:08.886 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether ln -s works... yes 2024-05-08T21:22:08.894 INFO:tasks.workunit.client.0.smithi033.stdout:checking the maximum length of command line arguments... 1572864 2024-05-08T21:22:08.898 INFO:tasks.workunit.client.0.smithi033.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-05-08T21:22:08.898 INFO:tasks.workunit.client.0.smithi033.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-05-08T21:22:08.899 INFO:tasks.workunit.client.0.smithi033.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-05-08T21:22:08.900 INFO:tasks.workunit.client.0.smithi033.stdout:checking for objdump... objdump 2024-05-08T21:22:08.902 INFO:tasks.workunit.client.0.smithi033.stdout:checking how to recognize dependent libraries... pass_all 2024-05-08T21:22:08.903 INFO:tasks.workunit.client.0.smithi033.stdout:checking for dlltool... no 2024-05-08T21:22:08.903 INFO:tasks.workunit.client.0.smithi033.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-05-08T21:22:08.905 INFO:tasks.workunit.client.0.smithi033.stdout:checking for ar... ar 2024-05-08T21:22:09.018 INFO:tasks.workunit.client.0.smithi033.stdout:checking for archiver @FILE support... @ 2024-05-08T21:22:09.019 INFO:tasks.workunit.client.0.smithi033.stdout:checking for strip... strip 2024-05-08T21:22:09.020 INFO:tasks.workunit.client.0.smithi033.stdout:checking for ranlib... ranlib 2024-05-08T21:22:09.022 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gawk... gawk 2024-05-08T21:22:09.221 INFO:tasks.workunit.client.0.smithi033.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-05-08T21:22:09.223 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sysroot... no 2024-05-08T21:22:09.250 INFO:tasks.workunit.client.0.smithi033.stdout:checking for a working dd... /usr/bin/dd 2024-05-08T21:22:09.273 INFO:tasks.workunit.client.0.smithi033.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-05-08T21:22:09.326 INFO:tasks.workunit.client.0.smithi033.stdout:checking for mt... no 2024-05-08T21:22:09.350 INFO:tasks.workunit.client.0.smithi033.stdout:checking if : is a manifest tool... no 2024-05-08T21:22:09.466 INFO:tasks.workunit.client.0.smithi033.stdout:checking how to run the C preprocessor... gcc -E 2024-05-08T21:22:09.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:09 smithi071 ceph-mon[47074]: pgmap v385: 129 pgs: 129 active+clean; 173 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-05-08T21:22:09.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:09 smithi082 ceph-mon[47512]: pgmap v385: 129 pgs: 129 active+clean; 173 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-05-08T21:22:09.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:09 smithi033 ceph-mon[42528]: pgmap v385: 129 pgs: 129 active+clean; 173 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2024-05-08T21:22:09.787 INFO:tasks.workunit.client.0.smithi033.stdout:checking for ANSI C header files... yes 2024-05-08T21:22:09.852 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/types.h... yes 2024-05-08T21:22:09.937 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/stat.h... yes 2024-05-08T21:22:10.025 INFO:tasks.workunit.client.0.smithi033.stdout:checking for stdlib.h... yes 2024-05-08T21:22:10.105 INFO:tasks.workunit.client.0.smithi033.stdout:checking for string.h... yes 2024-05-08T21:22:10.183 INFO:tasks.workunit.client.0.smithi033.stdout:checking for memory.h... yes 2024-05-08T21:22:10.264 INFO:tasks.workunit.client.0.smithi033.stdout:checking for strings.h... yes 2024-05-08T21:22:10.345 INFO:tasks.workunit.client.0.smithi033.stdout:checking for inttypes.h... yes 2024-05-08T21:22:10.438 INFO:tasks.workunit.client.0.smithi033.stdout:checking for stdint.h... yes 2024-05-08T21:22:10.523 INFO:tasks.workunit.client.0.smithi033.stdout:checking for unistd.h... yes 2024-05-08T21:22:10.595 INFO:tasks.workunit.client.0.smithi033.stdout:checking for dlfcn.h... yes 2024-05-08T21:22:10.609 INFO:tasks.workunit.client.0.smithi033.stdout:checking for objdir... .libs 2024-05-08T21:22:10.818 INFO:tasks.workunit.client.0.smithi033.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-05-08T21:22:10.818 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-05-08T21:22:10.886 INFO:tasks.workunit.client.0.smithi033.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-05-08T21:22:10.970 INFO:tasks.workunit.client.0.smithi033.stdout:checking if gcc static flag -static works... no 2024-05-08T21:22:11.081 INFO:tasks.workunit.client.0.smithi033.stdout:checking if gcc supports -c -o file.o... yes 2024-05-08T21:22:11.081 INFO:tasks.workunit.client.0.smithi033.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-05-08T21:22:11.091 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-05-08T21:22:11.193 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether -lc should be explicitly linked in... no 2024-05-08T21:22:11.305 INFO:tasks.workunit.client.0.smithi033.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-05-08T21:22:11.305 INFO:tasks.workunit.client.0.smithi033.stdout:checking how to hardcode library paths into programs... immediate 2024-05-08T21:22:11.307 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether stripping libraries is possible... yes 2024-05-08T21:22:11.308 INFO:tasks.workunit.client.0.smithi033.stdout:checking if libtool supports shared libraries... yes 2024-05-08T21:22:11.308 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether to build shared libraries... yes 2024-05-08T21:22:11.309 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether to build static libraries... yes 2024-05-08T21:22:11.318 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc... (cached) gcc 2024-05-08T21:22:11.385 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-08T21:22:11.385 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-08T21:22:11.386 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-08T21:22:11.388 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gmake... /usr/bin/gmake 2024-05-08T21:22:11.388 INFO:tasks.workunit.client.0.smithi033.stderr:./configure: line 11956: test: $(SHELL): binary operator expected 2024-05-08T21:22:11.389 INFO:tasks.workunit.client.0.smithi033.stdout:checking for tar... /bin/tar 2024-05-08T21:22:11.390 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gzip... /bin/gzip 2024-05-08T21:22:11.409 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether gcc -MM is supported... yes 2024-05-08T21:22:11.412 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sort... /bin/sort 2024-05-08T21:22:11.414 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether ln -s works... yes 2024-05-08T21:22:11.420 INFO:tasks.workunit.client.0.smithi033.stdout:checking for rpm... /bin/rpm 2024-05-08T21:22:11.431 INFO:tasks.workunit.client.0.smithi033.stdout:checking for rpmbuild... no 2024-05-08T21:22:11.432 INFO:tasks.workunit.client.0.smithi033.stdout:checking for ANSI C header files... (cached) yes 2024-05-08T21:22:11.521 INFO:tasks.workunit.client.0.smithi033.stdout:checking assert.h usability... yes 2024-05-08T21:22:11.547 INFO:tasks.workunit.client.0.smithi033.stdout:checking assert.h presence... yes 2024-05-08T21:22:11.547 INFO:tasks.workunit.client.0.smithi033.stdout:checking for assert.h... yes 2024-05-08T21:22:11.569 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:11 smithi071 ceph-mon[47074]: pgmap v386: 129 pgs: 129 active+clean; 175 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 187 op/s 2024-05-08T21:22:11.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:11 smithi082 ceph-mon[47512]: pgmap v386: 129 pgs: 129 active+clean; 175 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 187 op/s 2024-05-08T21:22:11.642 INFO:tasks.workunit.client.0.smithi033.stdout:checking bstring.h usability... no 2024-05-08T21:22:11.694 INFO:tasks.workunit.client.0.smithi033.stdout:checking bstring.h presence... no 2024-05-08T21:22:11.694 INFO:tasks.workunit.client.0.smithi033.stdout:checking for bstring.h... no 2024-05-08T21:22:11.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:11 smithi033 ceph-mon[42528]: pgmap v386: 129 pgs: 129 active+clean; 175 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 187 op/s 2024-05-08T21:22:11.763 INFO:tasks.workunit.client.0.smithi033.stdout:checking libgen.h usability... yes 2024-05-08T21:22:11.789 INFO:tasks.workunit.client.0.smithi033.stdout:checking libgen.h presence... yes 2024-05-08T21:22:11.789 INFO:tasks.workunit.client.0.smithi033.stdout:checking for libgen.h... yes 2024-05-08T21:22:11.876 INFO:tasks.workunit.client.0.smithi033.stdout:checking dirent.h usability... yes 2024-05-08T21:22:11.903 INFO:tasks.workunit.client.0.smithi033.stdout:checking dirent.h presence... yes 2024-05-08T21:22:11.904 INFO:tasks.workunit.client.0.smithi033.stdout:checking for dirent.h... yes 2024-05-08T21:22:11.993 INFO:tasks.workunit.client.0.smithi033.stdout:checking errno.h usability... yes 2024-05-08T21:22:12.021 INFO:tasks.workunit.client.0.smithi033.stdout:checking errno.h presence... yes 2024-05-08T21:22:12.021 INFO:tasks.workunit.client.0.smithi033.stdout:checking for errno.h... yes 2024-05-08T21:22:12.110 INFO:tasks.workunit.client.0.smithi033.stdout:checking malloc.h usability... yes 2024-05-08T21:22:12.139 INFO:tasks.workunit.client.0.smithi033.stdout:checking malloc.h presence... yes 2024-05-08T21:22:12.139 INFO:tasks.workunit.client.0.smithi033.stdout:checking for malloc.h... yes 2024-05-08T21:22:12.246 INFO:tasks.workunit.client.0.smithi033.stdout:checking uuid.h usability... no 2024-05-08T21:22:12.299 INFO:tasks.workunit.client.0.smithi033.stdout:checking uuid.h presence... no 2024-05-08T21:22:12.299 INFO:tasks.workunit.client.0.smithi033.stdout:checking for uuid.h... no 2024-05-08T21:22:12.389 INFO:tasks.workunit.client.0.smithi033.stdout:checking uuid/uuid.h usability... yes 2024-05-08T21:22:12.434 INFO:tasks.workunit.client.0.smithi033.stdout:checking uuid/uuid.h presence... yes 2024-05-08T21:22:12.435 INFO:tasks.workunit.client.0.smithi033.stdout:checking for uuid/uuid.h... yes 2024-05-08T21:22:12.536 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/uuid.h usability... no 2024-05-08T21:22:12.587 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/uuid.h presence... no 2024-05-08T21:22:12.587 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/uuid.h... no 2024-05-08T21:22:12.667 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/file.h usability... yes 2024-05-08T21:22:12.694 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/file.h presence... yes 2024-05-08T21:22:12.695 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/file.h... yes 2024-05-08T21:22:12.785 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/fcntl.h usability... yes 2024-05-08T21:22:12.813 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/fcntl.h presence... yes 2024-05-08T21:22:12.813 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/fcntl.h... yes 2024-05-08T21:22:12.907 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/syssgi.h usability... no 2024-05-08T21:22:12.958 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/syssgi.h presence... no 2024-05-08T21:22:12.959 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/syssgi.h... no 2024-05-08T21:22:13.081 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/param.h usability... yes 2024-05-08T21:22:13.112 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/param.h presence... yes 2024-05-08T21:22:13.112 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/param.h... yes 2024-05-08T21:22:13.141 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/stat.h... (cached) yes 2024-05-08T21:22:13.226 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/statvfs.h usability... yes 2024-05-08T21:22:13.252 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/statvfs.h presence... yes 2024-05-08T21:22:13.252 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/statvfs.h... yes 2024-05-08T21:22:13.349 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/time.h usability... yes 2024-05-08T21:22:13.376 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/time.h presence... yes 2024-05-08T21:22:13.377 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/time.h... yes 2024-05-08T21:22:13.485 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/ioctl.h usability... yes 2024-05-08T21:22:13.510 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/ioctl.h presence... yes 2024-05-08T21:22:13.510 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/ioctl.h... yes 2024-05-08T21:22:13.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:13 smithi071 ceph-mon[47074]: pgmap v387: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-05-08T21:22:13.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:13 smithi082 ceph-mon[47512]: pgmap v387: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-05-08T21:22:13.606 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/wait.h usability... yes 2024-05-08T21:22:13.634 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/wait.h presence... yes 2024-05-08T21:22:13.634 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/wait.h... yes 2024-05-08T21:22:13.664 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/types.h... (cached) yes 2024-05-08T21:22:13.690 INFO:tasks.workunit.client.0.smithi033.stdout:checking for strings.h... (cached) yes 2024-05-08T21:22:13.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:13 smithi033 ceph-mon[42528]: pgmap v387: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-05-08T21:22:13.780 INFO:tasks.workunit.client.0.smithi033.stdout:checking err.h usability... yes 2024-05-08T21:22:13.806 INFO:tasks.workunit.client.0.smithi033.stdout:checking err.h presence... yes 2024-05-08T21:22:13.807 INFO:tasks.workunit.client.0.smithi033.stdout:checking for err.h... yes 2024-05-08T21:22:13.890 INFO:tasks.workunit.client.0.smithi033.stdout:checking linux/falloc.h usability... yes 2024-05-08T21:22:13.916 INFO:tasks.workunit.client.0.smithi033.stdout:checking linux/falloc.h presence... yes 2024-05-08T21:22:13.916 INFO:tasks.workunit.client.0.smithi033.stdout:checking for linux/falloc.h... yes 2024-05-08T21:22:14.023 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-05-08T21:22:14.072 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-05-08T21:22:14.072 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/fs/xfs_fsops.h... no 2024-05-08T21:22:14.143 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:22:14.151 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/fs/xfs_itable.h usability... no 2024-05-08T21:22:14.200 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/fs/xfs_itable.h presence... no 2024-05-08T21:22:14.200 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/fs/xfs_itable.h... no 2024-05-08T21:22:14.275 INFO:tasks.workunit.client.0.smithi033.stdout:checking xfs/platform_defs.h usability... no 2024-05-08T21:22:14.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.304+0000 7f52e817a700 1 -- 172.21.15.33:0/1084043719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e00ffd60 msgr2=0x7f52e0100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.304+0000 7f52e817a700 1 --2- 172.21.15.33:0/1084043719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e00ffd60 0x7f52e0100180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f52dc009920 tx=0x7f52dc009c30 comp rx=0 tx=0).stop 2024-05-08T21:22:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.304+0000 7f52e817a700 1 -- 172.21.15.33:0/1084043719 shutdown_connections 2024-05-08T21:22:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.304+0000 7f52e817a700 1 --2- 172.21.15.33:0/1084043719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e01018f0 0x7f52e01081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.304+0000 7f52e817a700 1 --2- 172.21.15.33:0/1084043719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e00ffd60 0x7f52e0100180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.304+0000 7f52e817a700 1 --2- 172.21.15.33:0/1084043719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52e0100f50 0x7f52e01013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.304+0000 7f52e817a700 1 -- 172.21.15.33:0/1084043719 >> 172.21.15.33:0/1084043719 conn(0x7f52e00fb290 msgr2=0x7f52e00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.304+0000 7f52e817a700 1 -- 172.21.15.33:0/1084043719 shutdown_connections 2024-05-08T21:22:14.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.304+0000 7f52e817a700 1 -- 172.21.15.33:0/1084043719 wait complete. 2024-05-08T21:22:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.304+0000 7f52e817a700 1 Processor -- start 2024-05-08T21:22:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.304+0000 7f52e817a700 1 -- start start 2024-05-08T21:22:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e817a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e00ffd60 0x7f52e0103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e817a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e0100f50 0x7f52e0102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e817a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52e01018f0 0x7f52e0102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e817a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e01032a0 con 0x7f52e01018f0 2024-05-08T21:22:14.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e817a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0103410 con 0x7f52e0100f50 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e817a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e01a0e90 con 0x7f52e00ffd60 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e5715700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e0100f50 0x7f52e0102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e5715700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e0100f50 0x7f52e0102660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44124/0 (socket says 172.21.15.33:44124) 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e5715700 1 -- 172.21.15.33:0/123021660 learned_addr learned my addr 172.21.15.33:0/123021660 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e6717700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52e01018f0 0x7f52e0102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e5f16700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e00ffd60 0x7f52e0103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e5715700 1 -- 172.21.15.33:0/123021660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e00ffd60 msgr2=0x7f52e0103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e5715700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e00ffd60 0x7f52e0103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e5715700 1 -- 172.21.15.33:0/123021660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52e01018f0 msgr2=0x7f52e0102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e5715700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52e01018f0 0x7f52e0102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e5715700 1 -- 172.21.15.33:0/123021660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc009520 con 0x7f52e0100f50 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52e5715700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e0100f50 0x7f52e0102660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f52d000e9a0 tx=0x7f52d000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.305+0000 7f52d6ffd700 1 -- 172.21.15.33:0/123021660 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d0009790 con 0x7f52e0100f50 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.306+0000 7f52d6ffd700 1 -- 172.21.15.33:0/123021660 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52d0010430 con 0x7f52e0100f50 2024-05-08T21:22:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.306+0000 7f52e817a700 1 -- 172.21.15.33:0/123021660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e01a10f0 con 0x7f52e0100f50 2024-05-08T21:22:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.306+0000 7f52d6ffd700 1 -- 172.21.15.33:0/123021660 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d000cea0 con 0x7f52e0100f50 2024-05-08T21:22:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.306+0000 7f52e817a700 1 -- 172.21.15.33:0/123021660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e01a1650 con 0x7f52e0100f50 2024-05-08T21:22:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.308+0000 7f52d6ffd700 1 -- 172.21.15.33:0/123021660 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f52d00105a0 con 0x7f52e0100f50 2024-05-08T21:22:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.308+0000 7f52d6ffd700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f52cc0a11a0 0x7f52cc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.309+0000 7f52d6ffd700 1 -- 172.21.15.33:0/123021660 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f52d0014070 con 0x7f52e0100f50 2024-05-08T21:22:14.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.309+0000 7f52e5f16700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f52cc0a11a0 0x7f52cc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.309+0000 7f52e5f16700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f52cc0a11a0 0x7f52cc0a3660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0098f0 tx=0x7f52dc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:14.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.310+0000 7f52f4334b80 1 Processor -- start 2024-05-08T21:22:14.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.310+0000 7f52f4334b80 1 -- start start 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52f4334b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340b540 0x559bf340b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52f4334b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340bea0 0x559bf340c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52f4334b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340c860 0x559bf340cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52f4334b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf3362150 con 0x559bf340bea0 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52f4334b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf335fb10 con 0x559bf340c860 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52f4334b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf335b400 con 0x559bf340b540 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e5715700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340bea0 0x559bf340c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e6717700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340c860 0x559bf340cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e5f16700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340b540 0x559bf340b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e6717700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340c860 0x559bf340cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44132/0 (socket says 172.21.15.33:44132) 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e6717700 1 -- 172.21.15.33:0/2907815192 learned_addr learned my addr 172.21.15.33:0/2907815192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e6717700 1 -- 172.21.15.33:0/2907815192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340b540 msgr2=0x559bf340b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:14.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e6717700 1 --2- 172.21.15.33:0/2907815192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340b540 0x559bf340b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e6717700 1 -- 172.21.15.33:0/2907815192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340bea0 msgr2=0x559bf340c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e6717700 1 --2- 172.21.15.33:0/2907815192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340bea0 0x559bf340c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e6717700 1 -- 172.21.15.33:0/2907815192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bf31dc3b0 con 0x559bf340c860 2024-05-08T21:22:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e5f16700 1 --2- 172.21.15.33:0/2907815192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340b540 0x559bf340b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e5715700 1 --2- 172.21.15.33:0/2907815192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340bea0 0x559bf340c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52e6717700 1 --2- 172.21.15.33:0/2907815192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340c860 0x559bf340cce0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52d8011600 tx=0x7f52d80099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac30d2f442129458 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.311+0000 7f52cbfff700 1 -- 172.21.15.33:0/2907815192 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d800e040 con 0x559bf340c860 2024-05-08T21:22:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.312+0000 7f52cbfff700 1 -- 172.21.15.33:0/2907815192 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52d800f040 con 0x559bf340c860 2024-05-08T21:22:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.312+0000 7f52cbfff700 1 -- 172.21.15.33:0/2907815192 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d80044c0 con 0x559bf340c860 2024-05-08T21:22:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.312+0000 7f52f4334b80 1 -- 172.21.15.33:0/2907815192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340c860 msgr2=0x559bf340cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.312+0000 7f52f4334b80 1 --2- 172.21.15.33:0/2907815192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340c860 0x559bf340cce0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52d8011600 tx=0x7f52d80099e0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.312+0000 7f52f4334b80 1 -- 172.21.15.33:0/2907815192 shutdown_connections 2024-05-08T21:22:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.312+0000 7f52f4334b80 1 --2- 172.21.15.33:0/2907815192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340c860 0x559bf340cce0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.312+0000 7f52f4334b80 1 --2- 172.21.15.33:0/2907815192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340b540 0x559bf340b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.312+0000 7f52f4334b80 1 --2- 172.21.15.33:0/2907815192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340bea0 0x559bf340c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.312+0000 7f52f4334b80 1 -- 172.21.15.33:0/2907815192 >> 172.21.15.33:0/2907815192 conn(0x559bf3369210 msgr2=0x559bf3405de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.312+0000 7f52f4334b80 1 -- 172.21.15.33:0/2907815192 shutdown_connections 2024-05-08T21:22:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.313+0000 7f52f4334b80 1 -- 172.21.15.33:0/2907815192 wait complete. 2024-05-08T21:22:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.313+0000 7f52f4334b80 1 -- start start 2024-05-08T21:22:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.313+0000 7f52f4334b80 1 Processor -- start 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52f4334b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 0x559bf3414de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52f4334b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 0x559bf3415320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52f4334b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340c860 0x559bf341c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52f4334b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf32f7560 con 0x559bf340b540 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52f4334b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf32f7200 con 0x559bf340bea0 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52f4334b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf31de6a0 con 0x559bf340c860 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52f4334b80 10 client.0 _ll_register_callbacks cb 0x559bf3300b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52e5f16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 0x559bf3414de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52e5715700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 0x559bf3415320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52e5f16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 0x559bf3414de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47340/0 (socket says 172.21.15.33:47340) 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52e5f16700 1 -- 172.21.15.33:0/3700358846 learned_addr learned my addr 172.21.15.33:0/3700358846 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52e6717700 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340c860 0x559bf341c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52e5f16700 1 -- 172.21.15.33:0/3700358846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340c860 msgr2=0x559bf341c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52e5f16700 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340c860 0x559bf341c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52e5f16700 1 -- 172.21.15.33:0/3700358846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 msgr2=0x559bf3415320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52e5f16700 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 0x559bf3415320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52e5f16700 1 -- 172.21.15.33:0/3700358846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e01a1650 con 0x559bf340b540 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52e5715700 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 0x559bf3415320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.314+0000 7f52e6717700 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340c860 0x559bf341c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52e5f16700 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 0x559bf3414de0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00f9d0 tx=0x7f52dc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed370b17bc9b8ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52c97fa700 1 -- 172.21.15.33:0/3700358846 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc02a050 con 0x559bf340b540 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52f4334b80 1 -- 172.21.15.33:0/3700358846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559bf32b4270 con 0x559bf340b540 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52c97fa700 1 -- 172.21.15.33:0/3700358846 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f52dc005040 con 0x559bf340b540 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52c97fa700 1 -- 172.21.15.33:0/3700358846 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52dc005200 con 0x559bf340b540 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52f4334b80 10 client.16401 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52f4334b80 1 -- 172.21.15.33:0/3700358846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f52d80112b0 con 0x559bf340b540 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52c97fa700 1 -- 172.21.15.33:0/3700358846 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559bf32b4270 con 0x559bf340b540 2024-05-08T21:22:14.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52c97fa700 1 -- 172.21.15.33:0/3700358846 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f52dc003d10 con 0x559bf340b540 2024-05-08T21:22:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52f4334b80 10 client.16401 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52f4334b80 10 client.16401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52f4334b80 20 client.16401 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.315+0000 7f52f4334b80 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559bf3420c40 0x559bf3423060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.316+0000 7f52f4334b80 4 client.16401 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.316+0000 7f52f4334b80 1 -- 172.21.15.33:0/3700358846 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bf33da810 con 0x559bf3420c40 2024-05-08T21:22:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.316+0000 7f52e5715700 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559bf3420c40 0x559bf3423060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.316+0000 7f52c97fa700 10 client.16401 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.316+0000 7f52e5715700 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559bf3420c40 0x559bf3423060 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35a9fe478131a server_cookie=964544bd62353cf6 in_seq=0 out_seq=0 2024-05-08T21:22:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.317+0000 7f52c97fa700 1 -- 172.21.15.33:0/3700358846 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559bf33da810 con 0x559bf3420c40 2024-05-08T21:22:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.317+0000 7f52c97fa700 10 client.16401 handle_command_reply: tid=0 2024-05-08T21:22:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52f4334b80 1 client.16401 shutdown 2024-05-08T21:22:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52f4334b80 1 -- 172.21.15.33:0/3700358846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 msgr2=0x559bf3414de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52f4334b80 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 0x559bf3414de0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00f9d0 tx=0x7f52dc009c30 comp rx=0 tx=0).stop 2024-05-08T21:22:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52f4334b80 1 -- 172.21.15.33:0/3700358846 shutdown_connections 2024-05-08T21:22:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52f4334b80 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559bf3420c40 0x559bf3423060 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52f4334b80 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340c860 0x559bf341c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52f4334b80 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 0x559bf3414de0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52c97fa700 0 client.16401 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52f4334b80 1 --2- 172.21.15.33:0/3700358846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 0x559bf3415320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52f4334b80 1 -- 172.21.15.33:0/3700358846 >> 172.21.15.33:0/3700358846 conn(0x559bf3369210 msgr2=0x559bf34055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52f4334b80 1 -- 172.21.15.33:0/3700358846 shutdown_connections 2024-05-08T21:22:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52f4334b80 1 -- 172.21.15.33:0/3700358846 wait complete. 2024-05-08T21:22:14.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.318+0000 7f52f4334b80 20 client.16401 trim_cache size 0 max 16384 2024-05-08T21:22:14.325 INFO:tasks.workunit.client.0.smithi033.stdout:checking xfs/platform_defs.h presence... no 2024-05-08T21:22:14.325 INFO:tasks.workunit.client.0.smithi033.stdout:checking for xfs/platform_defs.h... no 2024-05-08T21:22:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.338+0000 7f52f4334b80 1 Processor -- start 2024-05-08T21:22:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.338+0000 7f52f4334b80 1 -- start start 2024-05-08T21:22:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.338+0000 7f52f4334b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 0x559bf341ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.338+0000 7f52f4334b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 0x559bf341c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52f4334b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340c860 0x559bf34142f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52f4334b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf3402dc0 con 0x559bf340b540 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52f4334b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf3407570 con 0x559bf340bea0 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52f4334b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf3428270 con 0x559bf340c860 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52e5f16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 0x559bf341ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52e5715700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 0x559bf341c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52e5f16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 0x559bf341ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47344/0 (socket says 172.21.15.33:47344) 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52e5715700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 0x559bf341c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44156/0 (socket says 172.21.15.33:44156) 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52e5f16700 1 -- 172.21.15.33:0/52030872 learned_addr learned my addr 172.21.15.33:0/52030872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52e6717700 1 --2- 172.21.15.33:0/52030872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340c860 0x559bf34142f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52e5715700 1 -- 172.21.15.33:0/52030872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340c860 msgr2=0x559bf34142f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52e5715700 1 --2- 172.21.15.33:0/52030872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340c860 0x559bf34142f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52e5715700 1 -- 172.21.15.33:0/52030872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 msgr2=0x559bf341ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:14.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52e5715700 1 --2- 172.21.15.33:0/52030872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 0x559bf341ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52e5715700 1 -- 172.21.15.33:0/52030872 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bf3188b60 con 0x559bf340bea0 2024-05-08T21:22:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52e5715700 1 --2- 172.21.15.33:0/52030872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 0x559bf341c320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52d000e970 tx=0x7f52d0023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8039542066a10051 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.339+0000 7f52d4ff9700 1 -- 172.21.15.33:0/52030872 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d009aea0 con 0x559bf340bea0 2024-05-08T21:22:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.340+0000 7f52d4ff9700 1 -- 172.21.15.33:0/52030872 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52d0099e90 con 0x559bf340bea0 2024-05-08T21:22:14.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.340+0000 7f52d4ff9700 1 -- 172.21.15.33:0/52030872 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00a1e80 con 0x559bf340bea0 2024-05-08T21:22:14.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.340+0000 7f52f4334b80 1 -- 172.21.15.33:0/52030872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 msgr2=0x559bf341c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.340+0000 7f52f4334b80 1 --2- 172.21.15.33:0/52030872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 0x559bf341c320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52d000e970 tx=0x7f52d0023040 comp rx=0 tx=0).stop 2024-05-08T21:22:14.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.341+0000 7f52f4334b80 1 -- 172.21.15.33:0/52030872 shutdown_connections 2024-05-08T21:22:14.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.341+0000 7f52f4334b80 1 --2- 172.21.15.33:0/52030872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340c860 0x559bf34142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.341+0000 7f52f4334b80 1 --2- 172.21.15.33:0/52030872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340b540 0x559bf341ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.341+0000 7f52f4334b80 1 --2- 172.21.15.33:0/52030872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340bea0 0x559bf341c320 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.341+0000 7f52f4334b80 1 -- 172.21.15.33:0/52030872 >> 172.21.15.33:0/52030872 conn(0x559bf3369210 msgr2=0x559bf3402210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.341+0000 7f52f4334b80 1 -- 172.21.15.33:0/52030872 shutdown_connections 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.341+0000 7f52f4334b80 1 -- 172.21.15.33:0/52030872 wait complete. 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.341+0000 7f52f4334b80 1 -- start start 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52f4334b80 1 Processor -- start 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52f4334b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340b540 0x559bf3417ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52f4334b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340bea0 0x559bf34180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52f4334b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340c860 0x559bf341b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52f4334b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf341b5f0 con 0x559bf340c860 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52f4334b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf341b760 con 0x559bf340b540 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52f4334b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bf341b920 con 0x559bf340bea0 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52f4334b80 10 client.0 _ll_register_callbacks cb 0x559bf3300b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52e6717700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340c860 0x559bf341b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52e6717700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340c860 0x559bf341b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47352/0 (socket says 172.21.15.33:47352) 2024-05-08T21:22:14.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52e6717700 1 -- 172.21.15.33:0/4015905489 learned_addr learned my addr 172.21.15.33:0/4015905489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52e5f16700 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340b540 0x559bf3417ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.342+0000 7f52e5715700 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340bea0 0x559bf34180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.343+0000 7f52e5f16700 1 -- 172.21.15.33:0/4015905489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340bea0 msgr2=0x559bf34180e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.343+0000 7f52e5f16700 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340bea0 0x559bf34180e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.343+0000 7f52e5f16700 1 -- 172.21.15.33:0/4015905489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340c860 msgr2=0x559bf341b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.343+0000 7f52e5f16700 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340c860 0x559bf341b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.343+0000 7f52e5f16700 1 -- 172.21.15.33:0/4015905489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e01a10f0 con 0x559bf340b540 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.343+0000 7f52e5715700 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340bea0 0x559bf34180e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.343+0000 7f52e5f16700 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340b540 0x559bf3417ba0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00ffc0 tx=0x7f52dc0235f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f7028d77b5276b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.343+0000 7f52c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.343+0000 7f52c97fa700 1 -- 172.21.15.33:0/4015905489 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc02a040 con 0x559bf340b540 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.343+0000 7f52f4334b80 1 -- 172.21.15.33:0/4015905489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52d800f040 con 0x559bf340b540 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.343+0000 7f52c97fa700 1 -- 172.21.15.33:0/4015905489 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f52dc00fd00 con 0x559bf340b540 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.344+0000 7f52c97fa700 1 -- 172.21.15.33:0/4015905489 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52dc0236d0 con 0x559bf340b540 2024-05-08T21:22:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.344+0000 7f52f4334b80 10 client.26276 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.344+0000 7f52f4334b80 1 -- 172.21.15.33:0/4015905489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559bf3321560 con 0x559bf340b540 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.344+0000 7f52c97fa700 1 -- 172.21.15.33:0/4015905489 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52d800f040 con 0x559bf340b540 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.344+0000 7f52c97fa700 1 -- 172.21.15.33:0/4015905489 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f52dc020070 con 0x559bf340b540 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.344+0000 7f52f4334b80 10 client.26276 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.344+0000 7f52f4334b80 10 client.26276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.344+0000 7f52f4334b80 20 client.26276 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.344+0000 7f52f4334b80 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559bf347ffd0 0x559bf34823f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.344+0000 7f52f4334b80 4 client.26276 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.344+0000 7f52f4334b80 1 -- 172.21.15.33:0/4015905489 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559bf3353ec0 con 0x559bf347ffd0 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.344+0000 7f52e5715700 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559bf347ffd0 0x559bf34823f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52c97fa700 10 client.26276 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52e5715700 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559bf347ffd0 0x559bf34823f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5585ef8a4276489 server_cookie=226b891d7a4061a2 in_seq=0 out_seq=0 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52c97fa700 1 -- 172.21.15.33:0/4015905489 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559bf3353ec0 con 0x559bf347ffd0 2024-05-08T21:22:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52c97fa700 10 client.26276 handle_command_reply: tid=0 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52f4334b80 1 client.26276 shutdown 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52f4334b80 1 -- 172.21.15.33:0/4015905489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340b540 msgr2=0x559bf3417ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52f4334b80 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340b540 0x559bf3417ba0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00ffc0 tx=0x7f52dc0235f0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52f4334b80 1 -- 172.21.15.33:0/4015905489 shutdown_connections 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52f4334b80 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559bf347ffd0 0x559bf34823f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52f4334b80 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559bf340c860 0x559bf341b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52f4334b80 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559bf340b540 0x559bf3417ba0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52f4334b80 1 --2- 172.21.15.33:0/4015905489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559bf340bea0 0x559bf34180e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52c97fa700 0 client.26276 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52f4334b80 1 -- 172.21.15.33:0/4015905489 >> 172.21.15.33:0/4015905489 conn(0x559bf3369210 msgr2=0x559bf3412d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52f4334b80 1 -- 172.21.15.33:0/4015905489 shutdown_connections 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52f4334b80 1 -- 172.21.15.33:0/4015905489 wait complete. 2024-05-08T21:22:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.345+0000 7f52f4334b80 20 client.26276 trim_cache size 0 max 16384 2024-05-08T21:22:14.352 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:22:14.352 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:22:14.352 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:22:14.352 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:22:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.346+0000 7f52e817a700 1 -- 172.21.15.33:0/123021660 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f52cc0a11a0 msgr2=0x7f52cc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.346+0000 7f52e817a700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f52cc0a11a0 0x7f52cc0a3660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f52dc0098f0 tx=0x7f52dc019040 comp rx=0 tx=0).stop 2024-05-08T21:22:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.346+0000 7f52e817a700 1 -- 172.21.15.33:0/123021660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e0100f50 msgr2=0x7f52e0102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.346+0000 7f52e817a700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e0100f50 0x7f52e0102660 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f52d000e9a0 tx=0x7f52d000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.346+0000 7f52e817a700 1 -- 172.21.15.33:0/123021660 shutdown_connections 2024-05-08T21:22:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.346+0000 7f52e817a700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52e01018f0 0x7f52e0102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.346+0000 7f52e817a700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e00ffd60 0x7f52e0103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.346+0000 7f52e817a700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f52cc0a11a0 0x7f52cc0a3660 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.346+0000 7f52e817a700 1 --2- 172.21.15.33:0/123021660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e0100f50 0x7f52e0102660 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.346+0000 7f52e817a700 1 -- 172.21.15.33:0/123021660 >> 172.21.15.33:0/123021660 conn(0x7f52e00fb290 msgr2=0x7f52e0068160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.346+0000 7f52e817a700 1 -- 172.21.15.33:0/123021660 shutdown_connections 2024-05-08T21:22:14.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.346+0000 7f52e817a700 1 -- 172.21.15.33:0/123021660 wait complete. 2024-05-08T21:22:14.362 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:22:14.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:22:14.398 INFO:tasks.workunit.client.0.smithi033.stdout:checking btrfs/ioctl.h usability... no 2024-05-08T21:22:14.433 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:14 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:22:14.472 INFO:tasks.workunit.client.0.smithi033.stdout:checking btrfs/ioctl.h presence... no 2024-05-08T21:22:14.473 INFO:tasks.workunit.client.0.smithi033.stdout:checking for btrfs/ioctl.h... no 2024-05-08T21:22:14.546 INFO:tasks.workunit.client.0.smithi033.stdout:checking cifs/ioctl.h usability... no 2024-05-08T21:22:14.569 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:14 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:22:14.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.577+0000 7f34afc6b700 1 -- 172.21.15.33:0/1164782214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34a8102990 msgr2=0x7f34a81092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.577+0000 7f34afc6b700 1 --2- 172.21.15.33:0/1164782214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34a8102990 0x7f34a81092b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f34a000b0a0 tx=0x7f34a000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.577+0000 7f34afc6b700 1 -- 172.21.15.33:0/1164782214 shutdown_connections 2024-05-08T21:22:14.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.577+0000 7f34afc6b700 1 --2- 172.21.15.33:0/1164782214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34a8102990 0x7f34a81092b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.577+0000 7f34afc6b700 1 --2- 172.21.15.33:0/1164782214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34a8100e80 0x7f34a81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.577+0000 7f34afc6b700 1 --2- 172.21.15.33:0/1164782214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a8101fd0 0x7f34a8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.577+0000 7f34afc6b700 1 -- 172.21.15.33:0/1164782214 >> 172.21.15.33:0/1164782214 conn(0x7f34a80fc410 msgr2=0x7f34a80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:14.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.577+0000 7f34afc6b700 1 -- 172.21.15.33:0/1164782214 shutdown_connections 2024-05-08T21:22:14.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.578+0000 7f34afc6b700 1 -- 172.21.15.33:0/1164782214 wait complete. 2024-05-08T21:22:14.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.578+0000 7f34afc6b700 1 Processor -- start 2024-05-08T21:22:14.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.578+0000 7f34afc6b700 1 -- start start 2024-05-08T21:22:14.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.578+0000 7f34afc6b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a8100e80 0x7f34a8196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34afc6b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34a8101fd0 0x7f34a8197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34afc6b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34a8102990 0x7f34a819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34afc6b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a819e900 con 0x7f34a8102990 2024-05-08T21:22:14.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34afc6b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a819ea70 con 0x7f34a8100e80 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34afc6b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a819ec30 con 0x7f34a8101fd0 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34ada07700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a8100e80 0x7f34a8196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34ae208700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34a8102990 0x7f34a819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34ada07700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a8100e80 0x7f34a8196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44180/0 (socket says 172.21.15.33:44180) 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34ada07700 1 -- 172.21.15.33:0/3259965299 learned_addr learned my addr 172.21.15.33:0/3259965299 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34ad206700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34a8101fd0 0x7f34a8197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34ada07700 1 -- 172.21.15.33:0/3259965299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34a8101fd0 msgr2=0x7f34a8197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34ada07700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34a8101fd0 0x7f34a8197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34ada07700 1 -- 172.21.15.33:0/3259965299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34a8102990 msgr2=0x7f34a819e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34ada07700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34a8102990 0x7f34a819e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34ada07700 1 -- 172.21.15.33:0/3259965299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a4009520 con 0x7f34a8100e80 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34ae208700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34a8102990 0x7f34a819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f34ada07700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a8100e80 0x7f34a8196e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f349c00e9a0 tx=0x7f349c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.579+0000 7f349affd700 1 -- 172.21.15.33:0/3259965299 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c009790 con 0x7f34a8100e80 2024-05-08T21:22:14.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.580+0000 7f34afc6b700 1 -- 172.21.15.33:0/3259965299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a0009d00 con 0x7f34a8100e80 2024-05-08T21:22:14.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.580+0000 7f349affd700 1 -- 172.21.15.33:0/3259965299 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f349c010430 con 0x7f34a8100e80 2024-05-08T21:22:14.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.580+0000 7f349affd700 1 -- 172.21.15.33:0/3259965299 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c00cea0 con 0x7f34a8100e80 2024-05-08T21:22:14.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.580+0000 7f34afc6b700 1 -- 172.21.15.33:0/3259965299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a80735b0 con 0x7f34a8100e80 2024-05-08T21:22:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.581+0000 7f34afc6b700 1 -- 172.21.15.33:0/3259965299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a81082f0 con 0x7f34a8100e80 2024-05-08T21:22:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.582+0000 7f349affd700 1 -- 172.21.15.33:0/3259965299 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f349c0105a0 con 0x7f34a8100e80 2024-05-08T21:22:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.582+0000 7f349affd700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34940a1120 0x7f34940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.583+0000 7f34ad206700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34940a1120 0x7f34940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.583+0000 7f349affd700 1 -- 172.21.15.33:0/3259965299 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f349c014070 con 0x7f34a8100e80 2024-05-08T21:22:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.583+0000 7f34ad206700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34940a1120 0x7f34940a35e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009df0 tx=0x7f34a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:14.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.587+0000 7f349affd700 1 -- 172.21.15.33:0/3259965299 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f349c052d60 con 0x7f34a8100e80 2024-05-08T21:22:14.594 INFO:tasks.workunit.client.0.smithi033.stdout:checking cifs/ioctl.h presence... no 2024-05-08T21:22:14.594 INFO:tasks.workunit.client.0.smithi033.stdout:checking for cifs/ioctl.h... no 2024-05-08T21:22:14.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:14 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:22:14.672 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/mman.h usability... yes 2024-05-08T21:22:14.697 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/mman.h presence... yes 2024-05-08T21:22:14.697 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/mman.h... yes 2024-05-08T21:22:14.765 INFO:tasks.workunit.client.0.smithi033.stdout:checking for xfs/xfs_log_format.h... no 2024-05-08T21:22:14.894 INFO:tasks.workunit.client.0.smithi033.stdout:checking for uuid_compare... no 2024-05-08T21:22:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.907+0000 7f34afc6b700 1 -- 172.21.15.33:0/3259965299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34a8067480 con 0x7f34a8100e80 2024-05-08T21:22:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.908+0000 7f349affd700 1 -- 172.21.15.33:0/3259965299 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10907 (secure 0 0 0) 0x7f34a8067480 con 0x7f34a8100e80 2024-05-08T21:22:14.910 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:22:14.910 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:22:13.275360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:22:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.911+0000 7f34afc6b700 1 -- 172.21.15.33:0/3259965299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34940a1120 msgr2=0x7f34940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.911+0000 7f34afc6b700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34940a1120 0x7f34940a35e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009df0 tx=0x7f34a4009260 comp rx=0 tx=0).stop 2024-05-08T21:22:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.911+0000 7f34afc6b700 1 -- 172.21.15.33:0/3259965299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a8100e80 msgr2=0x7f34a8196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.911+0000 7f34afc6b700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a8100e80 0x7f34a8196e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f349c00e9a0 tx=0x7f349c00ecb0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.911+0000 7f34afc6b700 1 -- 172.21.15.33:0/3259965299 shutdown_connections 2024-05-08T21:22:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.911+0000 7f34afc6b700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34a8102990 0x7f34a819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.911+0000 7f34afc6b700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34940a1120 0x7f34940a35e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.911+0000 7f34afc6b700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a8100e80 0x7f34a8196e10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.911+0000 7f34afc6b700 1 --2- 172.21.15.33:0/3259965299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34a8101fd0 0x7f34a8197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:14.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.911+0000 7f34afc6b700 1 -- 172.21.15.33:0/3259965299 >> 172.21.15.33:0/3259965299 conn(0x7f34a80fc410 msgr2=0x7f34a8105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:14.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.911+0000 7f34afc6b700 1 -- 172.21.15.33:0/3259965299 shutdown_connections 2024-05-08T21:22:14.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:14.911+0000 7f34afc6b700 1 -- 172.21.15.33:0/3259965299 wait complete. 2024-05-08T21:22:14.914 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 32 2024-05-08T21:22:14.931 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:22:14.979 INFO:tasks.workunit.client.0.smithi033.stdout:checking for uuid_compare in -luuid... yes 2024-05-08T21:22:15.063 INFO:tasks.workunit.client.0.smithi033.stdout:checking for xfs/xfs.h... yes 2024-05-08T21:22:15.121 INFO:tasks.workunit.client.0.smithi033.stdout:checking for xfs/libxfs.h... no 2024-05-08T21:22:15.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.143+0000 7f044c6fb700 1 -- 172.21.15.33:0/3866405285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04441074c0 msgr2=0x7f04441078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.143+0000 7f044c6fb700 1 --2- 172.21.15.33:0/3866405285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04441074c0 0x7f04441078e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0440009990 tx=0x7f0440009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.144+0000 7f044c6fb700 1 -- 172.21.15.33:0/3866405285 shutdown_connections 2024-05-08T21:22:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.144+0000 7f044c6fb700 1 --2- 172.21.15.33:0/3866405285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0444109050 0x7f044410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.144+0000 7f044c6fb700 1 --2- 172.21.15.33:0/3866405285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04441074c0 0x7f04441078e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.144+0000 7f044c6fb700 1 --2- 172.21.15.33:0/3866405285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04441086b0 0x7f0444108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.144+0000 7f044c6fb700 1 -- 172.21.15.33:0/3866405285 >> 172.21.15.33:0/3866405285 conn(0x7f0444076490 msgr2=0x7f04440788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.144+0000 7f044c6fb700 1 -- 172.21.15.33:0/3866405285 shutdown_connections 2024-05-08T21:22:15.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.144+0000 7f044c6fb700 1 -- 172.21.15.33:0/3866405285 wait complete. 2024-05-08T21:22:15.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044c6fb700 1 Processor -- start 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044c6fb700 1 -- start start 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044c6fb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04441074c0 0x7f044419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044c6fb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04441086b0 0x7f044419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044c6fb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0444109050 0x7f04441a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044c6fb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04441a4f30 con 0x7f04441086b0 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044c6fb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04441a5070 con 0x7f0444109050 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044c6fb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04441a5200 con 0x7f04441074c0 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044a497700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04441074c0 0x7f044419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044ac98700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0444109050 0x7f04441a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044ac98700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0444109050 0x7f04441a4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44194/0 (socket says 172.21.15.33:44194) 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044a497700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04441074c0 0x7f044419d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59386/0 (socket says 172.21.15.33:59386) 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.145+0000 7f044ac98700 1 -- 172.21.15.33:0/1712286921 learned_addr learned my addr 172.21.15.33:0/1712286921 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.146+0000 7f0449c96700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04441086b0 0x7f044419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.146+0000 7f044ac98700 1 -- 172.21.15.33:0/1712286921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04441074c0 msgr2=0x7f044419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.146+0000 7f044ac98700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04441074c0 0x7f044419d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.146+0000 7f044ac98700 1 -- 172.21.15.33:0/1712286921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04441086b0 msgr2=0x7f044419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.146+0000 7f044ac98700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04441086b0 0x7f044419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.146+0000 7f044ac98700 1 -- 172.21.15.33:0/1712286921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04400095f0 con 0x7f0444109050 2024-05-08T21:22:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.146+0000 7f0449c96700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04441086b0 0x7f044419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:22:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.146+0000 7f044ac98700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0444109050 0x7f04441a4960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f043c00ef30 tx=0x7f043c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.146+0000 7f043b7fe700 1 -- 172.21.15.33:0/1712286921 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c018070 con 0x7f0444109050 2024-05-08T21:22:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.146+0000 7f043b7fe700 1 -- 172.21.15.33:0/1712286921 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f043c00f040 con 0x7f0444109050 2024-05-08T21:22:15.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.146+0000 7f043b7fe700 1 -- 172.21.15.33:0/1712286921 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c00cc70 con 0x7f0444109050 2024-05-08T21:22:15.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.147+0000 7f044c6fb700 1 -- 172.21.15.33:0/1712286921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04441a5480 con 0x7f0444109050 2024-05-08T21:22:15.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.147+0000 7f044c6fb700 1 -- 172.21.15.33:0/1712286921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04441a5990 con 0x7f0444109050 2024-05-08T21:22:15.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.150+0000 7f043b7fe700 1 -- 172.21.15.33:0/1712286921 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f043c007500 con 0x7f0444109050 2024-05-08T21:22:15.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.151+0000 7f043b7fe700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f04300a11a0 0x7f04300a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.151+0000 7f044a497700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f04300a11a0 0x7f04300a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.151+0000 7f043b7fe700 1 -- 172.21.15.33:0/1712286921 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f043c0cbfe0 con 0x7f0444109050 2024-05-08T21:22:15.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.151+0000 7f04588b5b80 1 Processor -- start 2024-05-08T21:22:15.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f04588b5b80 1 -- start start 2024-05-08T21:22:15.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f04588b5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98516540 0x55db98516960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f04588b5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98517320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f04588b5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98517860 0x55db98517ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f04588b5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9846d150 con 0x55db98517860 2024-05-08T21:22:15.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f04588b5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9846ab10 con 0x55db98516ea0 2024-05-08T21:22:15.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f04588b5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db98466400 con 0x55db98516540 2024-05-08T21:22:15.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f044a497700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f04300a11a0 0x7f04300a3660 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f044000b2e0 tx=0x7f0440005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f044a497700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98516540 0x55db98516960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f044a497700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98516540 0x55db98516960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59400/0 (socket says 172.21.15.33:59400) 2024-05-08T21:22:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f044a497700 1 -- 172.21.15.33:0/2005604798 learned_addr learned my addr 172.21.15.33:0/2005604798 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f0449c96700 1 --2- 172.21.15.33:0/2005604798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98517320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.152+0000 7f044ac98700 1 --2- 172.21.15.33:0/2005604798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98517860 0x55db98517ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.153+0000 7f0449c96700 1 -- 172.21.15.33:0/2005604798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98516540 msgr2=0x55db98516960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.153+0000 7f0449c96700 1 --2- 172.21.15.33:0/2005604798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98516540 0x55db98516960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.153+0000 7f0449c96700 1 -- 172.21.15.33:0/2005604798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98517860 msgr2=0x55db98517ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.153+0000 7f0449c96700 1 --2- 172.21.15.33:0/2005604798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98517860 0x55db98517ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.153+0000 7f0449c96700 1 -- 172.21.15.33:0/2005604798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db982e73b0 con 0x55db98516ea0 2024-05-08T21:22:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.153+0000 7f0449c96700 1 --2- 172.21.15.33:0/2005604798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98517320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f043400aa00 tx=0x7f043400ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bacdd912559803a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.153+0000 7f0438ff9700 1 -- 172.21.15.33:0/2005604798 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043400aea0 con 0x55db98516ea0 2024-05-08T21:22:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.153+0000 7f0438ff9700 1 -- 172.21.15.33:0/2005604798 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04340113f0 con 0x55db98516ea0 2024-05-08T21:22:15.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.153+0000 7f0438ff9700 1 -- 172.21.15.33:0/2005604798 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04340185c0 con 0x55db98516ea0 2024-05-08T21:22:15.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.154+0000 7f04588b5b80 1 -- 172.21.15.33:0/2005604798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 msgr2=0x55db98517320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.154+0000 7f04588b5b80 1 --2- 172.21.15.33:0/2005604798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98517320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f043400aa00 tx=0x7f043400ad10 comp rx=0 tx=0).stop 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.154+0000 7f04588b5b80 1 -- 172.21.15.33:0/2005604798 shutdown_connections 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.154+0000 7f04588b5b80 1 --2- 172.21.15.33:0/2005604798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98517860 0x55db98517ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.154+0000 7f04588b5b80 1 --2- 172.21.15.33:0/2005604798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98516540 0x55db98516960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.154+0000 7f04588b5b80 1 --2- 172.21.15.33:0/2005604798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98517320 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.154+0000 7f04588b5b80 1 -- 172.21.15.33:0/2005604798 >> 172.21.15.33:0/2005604798 conn(0x55db98474210 msgr2=0x55db98510de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.154+0000 7f04588b5b80 1 -- 172.21.15.33:0/2005604798 shutdown_connections 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.154+0000 7f04588b5b80 1 -- 172.21.15.33:0/2005604798 wait complete. 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f04588b5b80 1 -- start start 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f04588b5b80 1 Processor -- start 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f04588b5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516540 0x55db9851fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f04588b5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98520330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f04588b5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db98527370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f04588b5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db98402560 con 0x55db98516540 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f04588b5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db98402200 con 0x55db98516ea0 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f04588b5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db982e96a0 con 0x55db98517860 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f04588b5b80 10 client.0 _ll_register_callbacks cb 0x55db9840bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f044ac98700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db98527370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f0449c96700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98520330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f0449c96700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98520330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44206/0 (socket says 172.21.15.33:44206) 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f044ac98700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db98527370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59414/0 (socket says 172.21.15.33:59414) 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f0449c96700 1 -- 172.21.15.33:0/2265729680 learned_addr learned my addr 172.21.15.33:0/2265729680 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.155+0000 7f044a497700 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516540 0x55db9851fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.156+0000 7f044ac98700 1 -- 172.21.15.33:0/2265729680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 msgr2=0x55db98520330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.156+0000 7f044ac98700 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98520330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.156+0000 7f044ac98700 1 -- 172.21.15.33:0/2265729680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516540 msgr2=0x55db9851fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.156+0000 7f044ac98700 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516540 0x55db9851fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.156+0000 7f044ac98700 1 -- 172.21.15.33:0/2265729680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043400a600 con 0x55db98517860 2024-05-08T21:22:15.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.156+0000 7f0449c96700 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98520330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.156+0000 7f044ac98700 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db98527370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f043c0c9740 tx=0x7f043c0995d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a61d07b1df56c563 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.156+0000 7f042dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.156+0000 7f042dffb700 1 -- 172.21.15.33:0/2265729680 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c099090 con 0x55db98517860 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.156+0000 7f042dffb700 1 -- 172.21.15.33:0/2265729680 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f043c09cec0 con 0x55db98517860 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.156+0000 7f04588b5b80 1 -- 172.21.15.33:0/2265729680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db983bf270 con 0x55db98517860 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.156+0000 7f042dffb700 1 -- 172.21.15.33:0/2265729680 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f043c0a1510 con 0x55db98517860 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.157+0000 7f04588b5b80 10 client.26314 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.157+0000 7f04588b5b80 1 -- 172.21.15.33:0/2265729680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55db98293b60 con 0x55db98517860 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.157+0000 7f042dffb700 1 -- 172.21.15.33:0/2265729680 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55db983bf270 con 0x55db98517860 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.157+0000 7f042dffb700 1 -- 172.21.15.33:0/2265729680 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f043c0a1080 con 0x55db98517860 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.157+0000 7f04588b5b80 10 client.26314 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.157+0000 7f04588b5b80 10 client.26314 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.157+0000 7f04588b5b80 20 client.26314 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.157+0000 7f04588b5b80 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55db9852bc70 0x55db9852e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.157+0000 7f04588b5b80 4 client.26314 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.157+0000 7f04588b5b80 1 -- 172.21.15.33:0/2265729680 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db984e5810 con 0x55db9852bc70 2024-05-08T21:22:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.158+0000 7f044a497700 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55db9852bc70 0x55db9852e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.158+0000 7f044a497700 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55db9852bc70 0x55db9852e090 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6286110e957d06ff server_cookie=62bb8fe004d8e445 in_seq=0 out_seq=0 2024-05-08T21:22:15.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.158+0000 7f042dffb700 10 client.26314 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:22:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f042dffb700 1 -- 172.21.15.33:0/2265729680 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db984e5810 con 0x55db9852bc70 2024-05-08T21:22:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f042dffb700 10 client.26314 handle_command_reply: tid=0 2024-05-08T21:22:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f04588b5b80 1 client.26314 shutdown 2024-05-08T21:22:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f04588b5b80 1 -- 172.21.15.33:0/2265729680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 msgr2=0x55db98527370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f04588b5b80 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db98527370 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f043c0c9740 tx=0x7f043c0995d0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f04588b5b80 1 -- 172.21.15.33:0/2265729680 shutdown_connections 2024-05-08T21:22:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f04588b5b80 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55db9852bc70 0x55db9852e090 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f04588b5b80 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db98527370 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f04588b5b80 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516540 0x55db9851fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f04588b5b80 1 --2- 172.21.15.33:0/2265729680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98520330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f04588b5b80 1 -- 172.21.15.33:0/2265729680 >> 172.21.15.33:0/2265729680 conn(0x55db98474210 msgr2=0x55db985105a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f042dffb700 0 client.26314 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:22:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f04588b5b80 1 -- 172.21.15.33:0/2265729680 shutdown_connections 2024-05-08T21:22:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f04588b5b80 1 -- 172.21.15.33:0/2265729680 wait complete. 2024-05-08T21:22:15.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.159+0000 7f04588b5b80 20 client.26314 trim_cache size 0 max 16384 2024-05-08T21:22:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f04588b5b80 1 Processor -- start 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f04588b5b80 1 -- start start 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f04588b5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516540 0x55db9851f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f04588b5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98527330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f04588b5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db9851de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f04588b5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db985119d0 con 0x55db98516540 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f04588b5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db984e5810 con 0x55db98516ea0 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f04588b5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db983bf270 con 0x55db98517860 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f044a497700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516540 0x55db9851f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f044a497700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516540 0x55db9851f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47426/0 (socket says 172.21.15.33:47426) 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f0449c96700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98527330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f044a497700 1 -- 172.21.15.33:0/755997755 learned_addr learned my addr 172.21.15.33:0/755997755 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f044ac98700 1 --2- 172.21.15.33:0/755997755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db9851de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f0449c96700 1 -- 172.21.15.33:0/755997755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 msgr2=0x55db9851de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f0449c96700 1 --2- 172.21.15.33:0/755997755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db9851de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f0449c96700 1 -- 172.21.15.33:0/755997755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516540 msgr2=0x55db9851f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.177+0000 7f0449c96700 1 --2- 172.21.15.33:0/755997755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516540 0x55db9851f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.178+0000 7f0449c96700 1 -- 172.21.15.33:0/755997755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db9842c560 con 0x55db98516ea0 2024-05-08T21:22:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.178+0000 7f044a497700 1 --2- 172.21.15.33:0/755997755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516540 0x55db9851f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.178+0000 7f0449c96700 1 --2- 172.21.15.33:0/755997755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98527330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0434000c00 tx=0x7f0434013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab3d19b1c3d1fea5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.178+0000 7f04397fa700 1 -- 172.21.15.33:0/755997755 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043401f040 con 0x55db98516ea0 2024-05-08T21:22:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.178+0000 7f04397fa700 1 -- 172.21.15.33:0/755997755 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f043400aea0 con 0x55db98516ea0 2024-05-08T21:22:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.178+0000 7f04397fa700 1 -- 172.21.15.33:0/755997755 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434011ea0 con 0x55db98516ea0 2024-05-08T21:22:15.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.179+0000 7f04588b5b80 1 -- 172.21.15.33:0/755997755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 msgr2=0x55db98527330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.179+0000 7f04588b5b80 1 --2- 172.21.15.33:0/755997755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98527330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0434000c00 tx=0x7f0434013040 comp rx=0 tx=0).stop 2024-05-08T21:22:15.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.179+0000 7f04588b5b80 1 -- 172.21.15.33:0/755997755 shutdown_connections 2024-05-08T21:22:15.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.179+0000 7f04588b5b80 1 --2- 172.21.15.33:0/755997755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db9851de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.179+0000 7f04588b5b80 1 --2- 172.21.15.33:0/755997755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516540 0x55db9851f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.179+0000 7f04588b5b80 1 --2- 172.21.15.33:0/755997755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516ea0 0x55db98527330 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.179+0000 7f04588b5b80 1 -- 172.21.15.33:0/755997755 >> 172.21.15.33:0/755997755 conn(0x55db98474210 msgr2=0x55db98517c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.179+0000 7f04588b5b80 1 -- 172.21.15.33:0/755997755 shutdown_connections 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.179+0000 7f04588b5b80 1 -- 172.21.15.33:0/755997755 wait complete. 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.179+0000 7f04588b5b80 1 -- start start 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.179+0000 7f04588b5b80 1 Processor -- start 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f04588b5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516540 0x55db98521ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f04588b5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516ea0 0x55db98522400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f04588b5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db98580150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f04588b5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db985273e0 con 0x55db98516ea0 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f04588b5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db98527550 con 0x55db98516540 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f04588b5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db98527710 con 0x55db98517860 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f04588b5b80 10 client.0 _ll_register_callbacks cb 0x55db9840bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:15.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f044a497700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516540 0x55db98521ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f044a497700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516540 0x55db98521ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44230/0 (socket says 172.21.15.33:44230) 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f044a497700 1 -- 172.21.15.33:0/763682706 learned_addr learned my addr 172.21.15.33:0/763682706 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f044ac98700 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db98580150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f0449c96700 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516ea0 0x55db98522400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f044a497700 1 -- 172.21.15.33:0/763682706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 msgr2=0x55db98580150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f044a497700 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db98580150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f044a497700 1 -- 172.21.15.33:0/763682706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516ea0 msgr2=0x55db98522400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f044a497700 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516ea0 0x55db98522400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f044a497700 1 -- 172.21.15.33:0/763682706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0444110250 con 0x55db98516540 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f0449c96700 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516ea0 0x55db98522400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f044a497700 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516540 0x55db98521ec0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0440009990 tx=0x7f0440009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7faf11897ed0b15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f042dffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f042dffb700 1 -- 172.21.15.33:0/763682706 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440003880 con 0x55db98516540 2024-05-08T21:22:15.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f042dffb700 1 -- 172.21.15.33:0/763682706 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f04400039e0 con 0x55db98516540 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.180+0000 7f04588b5b80 1 -- 172.21.15.33:0/763682706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04340113f0 con 0x55db98516540 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.181+0000 7f042dffb700 1 -- 172.21.15.33:0/763682706 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0440003ba0 con 0x55db98516540 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.181+0000 7f04588b5b80 10 client.26300 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.181+0000 7f04588b5b80 1 -- 172.21.15.33:0/763682706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55db982e73b0 con 0x55db98516540 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.181+0000 7f042dffb700 1 -- 172.21.15.33:0/763682706 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04340113f0 con 0x55db98516540 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.181+0000 7f042dffb700 1 -- 172.21.15.33:0/763682706 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0440004500 con 0x55db98516540 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.181+0000 7f04588b5b80 10 client.26300 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.181+0000 7f04588b5b80 10 client.26300 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.181+0000 7f04588b5b80 20 client.26300 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.181+0000 7f04588b5b80 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55db98581f60 0x55db98584380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.181+0000 7f04588b5b80 4 client.26300 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.181+0000 7f04588b5b80 1 -- 172.21.15.33:0/763682706 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55db9845eec0 con 0x55db98581f60 2024-05-08T21:22:15.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.181+0000 7f0449c96700 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55db98581f60 0x55db98584380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.188 INFO:tasks.workunit.client.0.smithi033.stdout:checking whether xlog_assign_lsn is declared... no 2024-05-08T21:22:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f0449c96700 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55db98581f60 0x55db98584380 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5f604534553f471b server_cookie=cb2cfe5dc12ec06c in_seq=0 out_seq=0 2024-05-08T21:22:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f042dffb700 10 client.26300 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:22:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f042dffb700 1 -- 172.21.15.33:0/763682706 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55db9845eec0 con 0x55db98581f60 2024-05-08T21:22:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f042dffb700 10 client.26300 handle_command_reply: tid=0 2024-05-08T21:22:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f04588b5b80 1 client.26300 shutdown 2024-05-08T21:22:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f04588b5b80 1 -- 172.21.15.33:0/763682706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516540 msgr2=0x55db98521ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f04588b5b80 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516540 0x55db98521ec0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0440009990 tx=0x7f0440009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f04588b5b80 1 -- 172.21.15.33:0/763682706 shutdown_connections 2024-05-08T21:22:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f04588b5b80 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55db98581f60 0x55db98584380 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f04588b5b80 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55db98517860 0x55db98580150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f04588b5b80 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55db98516540 0x55db98521ec0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f042dffb700 0 client.26300 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:22:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f04588b5b80 1 --2- 172.21.15.33:0/763682706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55db98516ea0 0x55db98522400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f04588b5b80 1 -- 172.21.15.33:0/763682706 >> 172.21.15.33:0/763682706 conn(0x55db98474210 msgr2=0x55db985329f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f04588b5b80 1 -- 172.21.15.33:0/763682706 shutdown_connections 2024-05-08T21:22:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.182+0000 7f04588b5b80 1 -- 172.21.15.33:0/763682706 wait complete. 2024-05-08T21:22:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f04588b5b80 20 client.26300 trim_cache size 0 max 16384 2024-05-08T21:22:15.190 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:22:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f044c6fb700 1 -- 172.21.15.33:0/1712286921 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f04300a11a0 msgr2=0x7f04300a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f044c6fb700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f04300a11a0 0x7f04300a3660 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f044000b2e0 tx=0x7f0440005fd0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f044c6fb700 1 -- 172.21.15.33:0/1712286921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0444109050 msgr2=0x7f04441a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f044c6fb700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0444109050 0x7f04441a4960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f043c00ef30 tx=0x7f043c00c3a0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f044c6fb700 1 -- 172.21.15.33:0/1712286921 shutdown_connections 2024-05-08T21:22:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f044c6fb700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f04300a11a0 0x7f04300a3660 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f044c6fb700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0444109050 0x7f04441a4960 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f044c6fb700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04441074c0 0x7f044419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f044c6fb700 1 --2- 172.21.15.33:0/1712286921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04441086b0 0x7f044419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f044c6fb700 1 -- 172.21.15.33:0/1712286921 >> 172.21.15.33:0/1712286921 conn(0x7f0444076490 msgr2=0x7f044410bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f044c6fb700 1 -- 172.21.15.33:0/1712286921 shutdown_connections 2024-05-08T21:22:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.183+0000 7f044c6fb700 1 -- 172.21.15.33:0/1712286921 wait complete. 2024-05-08T21:22:15.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:22:15.240 INFO:tasks.workunit.client.0.smithi033.stdout:checking for xfs/xqm.h... yes 2024-05-08T21:22:15.338 INFO:tasks.workunit.client.0.smithi033.stdout:checking xfsctl from xfs/xfs.h... ok 2024-05-08T21:22:15.401 INFO:tasks.workunit.client.0.smithi033.stdout:checking xfs/handle.h usability... yes 2024-05-08T21:22:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.407+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1200266497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e80ffd60 msgr2=0x7fb8e8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.407+0000 7fb8ed7da700 1 --2- 172.21.15.33:0/1200266497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e80ffd60 0x7fb8e8100180 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80099c0 tx=0x7fb8d8009cd0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.407+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1200266497 shutdown_connections 2024-05-08T21:22:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.407+0000 7fb8ed7da700 1 --2- 172.21.15.33:0/1200266497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e81018f0 0x7fb8e81081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.407+0000 7fb8ed7da700 1 --2- 172.21.15.33:0/1200266497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e80ffd60 0x7fb8e8100180 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.407+0000 7fb8ed7da700 1 --2- 172.21.15.33:0/1200266497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e8100f50 0x7fb8e81013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.407+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1200266497 >> 172.21.15.33:0/1200266497 conn(0x7fb8e80fb290 msgr2=0x7fb8e80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.407+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1200266497 shutdown_connections 2024-05-08T21:22:15.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.407+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1200266497 wait complete. 2024-05-08T21:22:15.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.408+0000 7fb8ed7da700 1 Processor -- start 2024-05-08T21:22:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.408+0000 7fb8ed7da700 1 -- start start 2024-05-08T21:22:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.408+0000 7fb8ed7da700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e80ffd60 0x7fb8e8068ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.408+0000 7fb8ed7da700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8100f50 0x7fb8e819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.408+0000 7fb8ed7da700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e81018f0 0x7fb8e819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.408+0000 7fb8ed7da700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e819e840 con 0x7fb8e8100f50 2024-05-08T21:22:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.408+0000 7fb8ed7da700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e819e9b0 con 0x7fb8e80ffd60 2024-05-08T21:22:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.408+0000 7fb8ed7da700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e819eb70 con 0x7fb8e81018f0 2024-05-08T21:22:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.408+0000 7fb8e6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e80ffd60 0x7fb8e8068ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8e67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8100f50 0x7fb8e819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8e77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e81018f0 0x7fb8e819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8e6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e80ffd60 0x7fb8e8068ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44252/0 (socket says 172.21.15.33:44252) 2024-05-08T21:22:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8e77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e81018f0 0x7fb8e819e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59450/0 (socket says 172.21.15.33:59450) 2024-05-08T21:22:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8e6ffd700 1 -- 172.21.15.33:0/1924900468 learned_addr learned my addr 172.21.15.33:0/1924900468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8e77fe700 1 -- 172.21.15.33:0/1924900468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e80ffd60 msgr2=0x7fb8e8068ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8e77fe700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e80ffd60 0x7fb8e8068ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8e77fe700 1 -- 172.21.15.33:0/1924900468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8100f50 msgr2=0x7fb8e819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8e77fe700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8100f50 0x7fb8e819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8e77fe700 1 -- 172.21.15.33:0/1924900468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc009520 con 0x7fb8e81018f0 2024-05-08T21:22:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8e6ffd700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e80ffd60 0x7fb8e8068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8e77fe700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e81018f0 0x7fb8e819e0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000efc0 tx=0x7fb8e000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8cffff700 1 -- 172.21.15.33:0/1924900468 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0018070 con 0x7fb8e81018f0 2024-05-08T21:22:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8cffff700 1 -- 172.21.15.33:0/1924900468 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8e000f040 con 0x7fb8e81018f0 2024-05-08T21:22:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1924900468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d80095f0 con 0x7fb8e81018f0 2024-05-08T21:22:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8cffff700 1 -- 172.21.15.33:0/1924900468 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e000cba0 con 0x7fb8e81018f0 2024-05-08T21:22:15.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.409+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1924900468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e819f170 con 0x7fb8e81018f0 2024-05-08T21:22:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.412+0000 7fb8cffff700 1 -- 172.21.15.33:0/1924900468 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb8e0004000 con 0x7fb8e81018f0 2024-05-08T21:22:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.413+0000 7fb8cffff700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8d00a11a0 0x7fb8d00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.413+0000 7fb8cffff700 1 -- 172.21.15.33:0/1924900468 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fb8e00cc770 con 0x7fb8e81018f0 2024-05-08T21:22:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8f9994b80 1 Processor -- start 2024-05-08T21:22:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8f9994b80 1 -- start start 2024-05-08T21:22:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8e6ffd700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8d00a11a0 0x7fb8d00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8f9994b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a08960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8f9994b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a08ea0 0x55eb89a09320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8f9994b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a09860 0x55eb89a09ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8f9994b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8995f150 con 0x55eb89a08540 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8f9994b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8995cb10 con 0x55eb89a08ea0 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8f9994b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb89958400 con 0x55eb89a09860 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8e6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a08960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8e6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a08960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47464/0 (socket says 172.21.15.33:47464) 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8e6ffd700 1 -- 172.21.15.33:0/1847637360 learned_addr learned my addr 172.21.15.33:0/1847637360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8e67fc700 1 --2- 172.21.15.33:0/1847637360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a08ea0 0x55eb89a09320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.415+0000 7fb8e77fe700 1 --2- 172.21.15.33:0/1847637360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a09860 0x55eb89a09ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.416+0000 7fb8e67fc700 1 -- 172.21.15.33:0/1847637360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a09860 msgr2=0x55eb89a09ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.416+0000 7fb8e67fc700 1 --2- 172.21.15.33:0/1847637360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a09860 0x55eb89a09ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.416+0000 7fb8e67fc700 1 -- 172.21.15.33:0/1847637360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 msgr2=0x55eb89a08960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.416+0000 7fb8e67fc700 1 --2- 172.21.15.33:0/1847637360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a08960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.416+0000 7fb8e67fc700 1 -- 172.21.15.33:0/1847637360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb897d93b0 con 0x55eb89a08ea0 2024-05-08T21:22:15.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.416+0000 7fb8e6ffd700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8d00a11a0 0x7fb8d00a3660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80098e0 tx=0x7fb8d8004040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.416+0000 7fb8e67fc700 1 --2- 172.21.15.33:0/1847637360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a08ea0 0x55eb89a09320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009e10 tx=0x7fb8dc0106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbf4753bacab0974 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.416+0000 7fb8cd7fa700 1 -- 172.21.15.33:0/1847637360 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc010e10 con 0x55eb89a08ea0 2024-05-08T21:22:15.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.416+0000 7fb8cd7fa700 1 -- 172.21.15.33:0/1847637360 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8dc01cce0 con 0x55eb89a08ea0 2024-05-08T21:22:15.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.416+0000 7fb8cd7fa700 1 -- 172.21.15.33:0/1847637360 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc011bc0 con 0x55eb89a08ea0 2024-05-08T21:22:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.419+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1847637360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a08ea0 msgr2=0x55eb89a09320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.419+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/1847637360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a08ea0 0x55eb89a09320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009e10 tx=0x7fb8dc0106b0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.419+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1847637360 shutdown_connections 2024-05-08T21:22:15.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.419+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/1847637360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a09860 0x55eb89a09ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.420+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/1847637360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a08960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.420+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/1847637360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a08ea0 0x55eb89a09320 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.420+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1847637360 >> 172.21.15.33:0/1847637360 conn(0x55eb89966210 msgr2=0x55eb89a02de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.420+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1847637360 shutdown_connections 2024-05-08T21:22:15.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.420+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1847637360 wait complete. 2024-05-08T21:22:15.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.421+0000 7fb8f9994b80 1 -- start start 2024-05-08T21:22:15.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.421+0000 7fb8f9994b80 1 Processor -- start 2024-05-08T21:22:15.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.421+0000 7fb8f9994b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a11de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.421+0000 7fb8f9994b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a08ea0 0x55eb89a12320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.421+0000 7fb8f9994b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.421+0000 7fb8f9994b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb898f4560 con 0x55eb89a08540 2024-05-08T21:22:15.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.421+0000 7fb8f9994b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb898f4200 con 0x55eb89a09860 2024-05-08T21:22:15.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.421+0000 7fb8f9994b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb897db6a0 con 0x55eb89a08ea0 2024-05-08T21:22:15.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.421+0000 7fb8f9994b80 10 client.0 _ll_register_callbacks cb 0x55eb898fdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:15.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.421+0000 7fb8e67fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a08ea0 0x55eb89a12320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.421+0000 7fb8e67fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a08ea0 0x55eb89a12320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59474/0 (socket says 172.21.15.33:59474) 2024-05-08T21:22:15.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.421+0000 7fb8e67fc700 1 -- 172.21.15.33:0/1985035071 learned_addr learned my addr 172.21.15.33:0/1985035071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.422+0000 7fb8e77fe700 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.422+0000 7fb8e6ffd700 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a11de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.422+0000 7fb8e67fc700 1 -- 172.21.15.33:0/1985035071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 msgr2=0x55eb89a19360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.422+0000 7fb8e67fc700 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.422+0000 7fb8e67fc700 1 -- 172.21.15.33:0/1985035071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 msgr2=0x55eb89a11de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.422+0000 7fb8e67fc700 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a11de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.422+0000 7fb8e67fc700 1 -- 172.21.15.33:0/1985035071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc009a40 con 0x55eb89a08ea0 2024-05-08T21:22:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.422+0000 7fb8e67fc700 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a08ea0 0x55eb89a12320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009de0 tx=0x7fb8dc01cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e8435d359c5275e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.422+0000 7fb8caff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.423+0000 7fb8caff5700 1 -- 172.21.15.33:0/1985035071 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc029070 con 0x55eb89a08ea0 2024-05-08T21:22:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.423+0000 7fb8caff5700 1 -- 172.21.15.33:0/1985035071 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fb8dc011940 con 0x55eb89a08ea0 2024-05-08T21:22:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.423+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1985035071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb898b1270 con 0x55eb89a08ea0 2024-05-08T21:22:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.424+0000 7fb8caff5700 1 -- 172.21.15.33:0/1985035071 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8dc01cb60 con 0x55eb89a08ea0 2024-05-08T21:22:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.424+0000 7fb8f9994b80 10 client.26332 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:15.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.424+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1985035071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55eb89785b60 con 0x55eb89a08ea0 2024-05-08T21:22:15.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.424+0000 7fb8caff5700 1 -- 172.21.15.33:0/1985035071 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8dc035030 con 0x55eb89a08ea0 2024-05-08T21:22:15.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.425+0000 7fb8caff5700 1 -- 172.21.15.33:0/1985035071 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fb8dc023020 con 0x55eb89a08ea0 2024-05-08T21:22:15.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.425+0000 7fb8f9994b80 10 client.26332 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:15.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.425+0000 7fb8f9994b80 10 client.26332 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:22:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.426+0000 7fb8f9994b80 20 client.26332 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.426+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55eb89a1da30 0x55eb89a1fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.426+0000 7fb8f9994b80 4 client.26332 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.426+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1985035071 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb899d7810 con 0x55eb89a1da30 2024-05-08T21:22:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.426+0000 7fb8e6ffd700 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55eb89a1da30 0x55eb89a1fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.426+0000 7fb8e6ffd700 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55eb89a1da30 0x55eb89a1fe50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46d4d411e20b90cf server_cookie=cbea7b96ef09604e in_seq=0 out_seq=0 2024-05-08T21:22:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.426+0000 7fb8caff5700 10 client.26332 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:22:15.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.427+0000 7fb8caff5700 1 -- 172.21.15.33:0/1985035071 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb899d7810 con 0x55eb89a1da30 2024-05-08T21:22:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.427+0000 7fb8caff5700 10 client.26332 handle_command_reply: tid=0 2024-05-08T21:22:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.427+0000 7fb8f9994b80 1 client.26332 shutdown 2024-05-08T21:22:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.427+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1985035071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a08ea0 msgr2=0x55eb89a12320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.427+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a08ea0 0x55eb89a12320 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009de0 tx=0x7fb8dc01cf80 comp rx=0 tx=0).stop 2024-05-08T21:22:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.428+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1985035071 shutdown_connections 2024-05-08T21:22:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.428+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.428+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a11de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.428+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55eb89a1da30 0x55eb89a1fe50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.428+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/1985035071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a08ea0 0x55eb89a12320 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009de0 tx=0x7fb8dc01cf80 comp rx=0 tx=0).stop 2024-05-08T21:22:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.428+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1985035071 >> 172.21.15.33:0/1985035071 conn(0x55eb89966210 msgr2=0x55eb89a025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.428+0000 7fb8caff5700 0 client.26332 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:22:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.429+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1985035071 shutdown_connections 2024-05-08T21:22:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.429+0000 7fb8f9994b80 1 -- 172.21.15.33:0/1985035071 wait complete. 2024-05-08T21:22:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.429+0000 7fb8f9994b80 20 client.26332 trim_cache size 0 max 16384 2024-05-08T21:22:15.440 INFO:tasks.workunit.client.0.smithi033.stdout:checking xfs/handle.h presence... yes 2024-05-08T21:22:15.441 INFO:tasks.workunit.client.0.smithi033.stdout:checking for xfs/handle.h... yes 2024-05-08T21:22:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.455+0000 7fb8f9994b80 1 Processor -- start 2024-05-08T21:22:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.455+0000 7fb8f9994b80 1 -- start start 2024-05-08T21:22:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.455+0000 7fb8f9994b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a08540 0x55eb89a19a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.455+0000 7fb8f9994b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.455+0000 7fb8f9994b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a112f0 0x55eb89a0fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.455+0000 7fb8f9994b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb899ffe30 con 0x55eb89a112f0 2024-05-08T21:22:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.455+0000 7fb8f9994b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb89a117b0 con 0x55eb89a09860 2024-05-08T21:22:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.455+0000 7fb8f9994b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb89a045f0 con 0x55eb89a08540 2024-05-08T21:22:15.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:15 smithi033 ceph-mon[42528]: pgmap v388: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-08T21:22:15.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:22:15.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:15 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3259965299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:22:15.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:15.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:15.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:15.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:15 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.455+0000 7fb8e77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a112f0 0x55eb89a0fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.455+0000 7fb8e77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a112f0 0x55eb89a0fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47494/0 (socket says 172.21.15.33:47494) 2024-05-08T21:22:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.455+0000 7fb8e77fe700 1 -- 172.21.15.33:0/4280796035 learned_addr learned my addr 172.21.15.33:0/4280796035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.456+0000 7fb8e6ffd700 1 --2- 172.21.15.33:0/4280796035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a08540 0x55eb89a19a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.456+0000 7fb8e67fc700 1 --2- 172.21.15.33:0/4280796035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.456+0000 7fb8e77fe700 1 -- 172.21.15.33:0/4280796035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a08540 msgr2=0x55eb89a19a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.456+0000 7fb8e77fe700 1 --2- 172.21.15.33:0/4280796035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a08540 0x55eb89a19a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.456+0000 7fb8e77fe700 1 -- 172.21.15.33:0/4280796035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 msgr2=0x55eb89a19320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.456+0000 7fb8e77fe700 1 --2- 172.21.15.33:0/4280796035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.456+0000 7fb8e77fe700 1 -- 172.21.15.33:0/4280796035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb8991e560 con 0x55eb89a112f0 2024-05-08T21:22:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.456+0000 7fb8e67fc700 1 --2- 172.21.15.33:0/4280796035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:22:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.456+0000 7fb8e77fe700 1 --2- 172.21.15.33:0/4280796035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a112f0 0x55eb89a0fdd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb8e009a350 tx=0x7fb8e0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5203cb3ba65238c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.457+0000 7fb8cdffb700 1 -- 172.21.15.33:0/4280796035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0083ea0 con 0x55eb89a112f0 2024-05-08T21:22:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.457+0000 7fb8cdffb700 1 -- 172.21.15.33:0/4280796035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8e00092e0 con 0x55eb89a112f0 2024-05-08T21:22:15.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.457+0000 7fb8cdffb700 1 -- 172.21.15.33:0/4280796035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e009e510 con 0x55eb89a112f0 2024-05-08T21:22:15.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.458+0000 7fb8f9994b80 1 -- 172.21.15.33:0/4280796035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a112f0 msgr2=0x55eb89a0fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.458+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/4280796035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a112f0 0x55eb89a0fdd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb8e009a350 tx=0x7fb8e0013000 comp rx=0 tx=0).stop 2024-05-08T21:22:15.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.459+0000 7fb8f9994b80 1 -- 172.21.15.33:0/4280796035 shutdown_connections 2024-05-08T21:22:15.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.459+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/4280796035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a112f0 0x55eb89a0fdd0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.459+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/4280796035 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a08540 0x55eb89a19a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.459+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/4280796035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.459+0000 7fb8f9994b80 1 -- 172.21.15.33:0/4280796035 >> 172.21.15.33:0/4280796035 conn(0x55eb89966210 msgr2=0x55eb89a09c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.459+0000 7fb8f9994b80 1 -- 172.21.15.33:0/4280796035 shutdown_connections 2024-05-08T21:22:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.459+0000 7fb8f9994b80 1 -- 172.21.15.33:0/4280796035 wait complete. 2024-05-08T21:22:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.460+0000 7fb8f9994b80 1 -- start start 2024-05-08T21:22:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.460+0000 7fb8f9994b80 1 Processor -- start 2024-05-08T21:22:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.460+0000 7fb8f9994b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a18bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.460+0000 7fb8f9994b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.460+0000 7fb8f9994b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a112f0 0x55eb89a156c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.460+0000 7fb8f9994b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb89a15ce0 con 0x55eb89a08540 2024-05-08T21:22:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.460+0000 7fb8f9994b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb89a15e50 con 0x55eb89a09860 2024-05-08T21:22:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.460+0000 7fb8f9994b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb89a16010 con 0x55eb89a112f0 2024-05-08T21:22:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.460+0000 7fb8f9994b80 10 client.0 _ll_register_callbacks cb 0x55eb898fdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:15.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.461+0000 7fb8e67fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.461+0000 7fb8e67fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44286/0 (socket says 172.21.15.33:44286) 2024-05-08T21:22:15.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.461+0000 7fb8e67fc700 1 -- 172.21.15.33:0/2550589063 learned_addr learned my addr 172.21.15.33:0/2550589063 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.461+0000 7fb8e6ffd700 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a18bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.461+0000 7fb8e77fe700 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a112f0 0x55eb89a156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.461+0000 7fb8e77fe700 1 -- 172.21.15.33:0/2550589063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 msgr2=0x55eb89a19110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.461+0000 7fb8e77fe700 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.461+0000 7fb8e77fe700 1 -- 172.21.15.33:0/2550589063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 msgr2=0x55eb89a18bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.461+0000 7fb8e77fe700 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a18bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.461+0000 7fb8e77fe700 1 -- 172.21.15.33:0/2550589063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e819f170 con 0x55eb89a112f0 2024-05-08T21:22:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.462+0000 7fb8e77fe700 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a112f0 0x55eb89a156c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb8e009a5e0 tx=0x7fb8e0004c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f664cefccf36c03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.462+0000 7fb8caff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.462+0000 7fb8caff5700 1 -- 172.21.15.33:0/2550589063 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e007d070 con 0x55eb89a112f0 2024-05-08T21:22:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.462+0000 7fb8caff5700 1 -- 172.21.15.33:0/2550589063 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fb8e009ad50 con 0x55eb89a112f0 2024-05-08T21:22:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.462+0000 7fb8f9994b80 1 -- 172.21.15.33:0/2550589063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8dc01cce0 con 0x55eb89a112f0 2024-05-08T21:22:15.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.462+0000 7fb8caff5700 1 -- 172.21.15.33:0/2550589063 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8e0083ea0 con 0x55eb89a112f0 2024-05-08T21:22:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.464+0000 7fb8f9994b80 10 client.26335 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.464+0000 7fb8f9994b80 1 -- 172.21.15.33:0/2550589063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55eb897ddcd0 con 0x55eb89a112f0 2024-05-08T21:22:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.464+0000 7fb8caff5700 1 -- 172.21.15.33:0/2550589063 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8dc01cce0 con 0x55eb89a112f0 2024-05-08T21:22:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.464+0000 7fb8caff5700 1 -- 172.21.15.33:0/2550589063 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fb8e0080030 con 0x55eb89a112f0 2024-05-08T21:22:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.464+0000 7fb8f9994b80 10 client.26335 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.464+0000 7fb8f9994b80 10 client.26335 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:22:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.464+0000 7fb8f9994b80 20 client.26335 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:15.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.464+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55eb89a74f20 0x55eb89a77310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.464+0000 7fb8f9994b80 4 client.26335 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:22:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.464+0000 7fb8f9994b80 1 -- 172.21.15.33:0/2550589063 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eb89950ec0 con 0x55eb89a74f20 2024-05-08T21:22:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.465+0000 7fb8e6ffd700 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55eb89a74f20 0x55eb89a77310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.465+0000 7fb8e6ffd700 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55eb89a74f20 0x55eb89a77310 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3cd5a3a9e8820ed7 server_cookie=1b68108aac8c46f2 in_seq=0 out_seq=0 2024-05-08T21:22:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.465+0000 7fb8caff5700 10 client.26335 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:22:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.465+0000 7fb8caff5700 1 -- 172.21.15.33:0/2550589063 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eb89950ec0 con 0x55eb89a74f20 2024-05-08T21:22:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.465+0000 7fb8caff5700 10 client.26335 handle_command_reply: tid=0 2024-05-08T21:22:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8f9994b80 1 client.26335 shutdown 2024-05-08T21:22:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8f9994b80 1 -- 172.21.15.33:0/2550589063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a112f0 msgr2=0x55eb89a156c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a112f0 0x55eb89a156c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb8e009a5e0 tx=0x7fb8e0004c50 comp rx=0 tx=0).stop 2024-05-08T21:22:15.468 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:22:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8f9994b80 1 -- 172.21.15.33:0/2550589063 shutdown_connections 2024-05-08T21:22:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55eb89a74f20 0x55eb89a77310 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eb89a112f0 0x55eb89a156c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8caff5700 0 client.26335 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:22:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb89a08540 0x55eb89a18bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8f9994b80 1 --2- 172.21.15.33:0/2550589063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eb89a09860 0x55eb89a19110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8f9994b80 1 -- 172.21.15.33:0/2550589063 >> 172.21.15.33:0/2550589063 conn(0x55eb89966210 msgr2=0x55eb89a0c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8f9994b80 1 -- 172.21.15.33:0/2550589063 shutdown_connections 2024-05-08T21:22:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8f9994b80 1 -- 172.21.15.33:0/2550589063 wait complete. 2024-05-08T21:22:15.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.466+0000 7fb8f9994b80 20 client.26335 trim_cache size 0 max 16384 2024-05-08T21:22:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.470+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1924900468 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8d00a11a0 msgr2=0x7fb8d00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.470+0000 7fb8ed7da700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8d00a11a0 0x7fb8d00a3660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80098e0 tx=0x7fb8d8004040 comp rx=0 tx=0).stop 2024-05-08T21:22:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.470+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1924900468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e81018f0 msgr2=0x7fb8e819e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.470+0000 7fb8ed7da700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e81018f0 0x7fb8e819e0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000efc0 tx=0x7fb8e000c3a0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.471+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1924900468 shutdown_connections 2024-05-08T21:22:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.471+0000 7fb8ed7da700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e81018f0 0x7fb8e819e0a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.471+0000 7fb8ed7da700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb8d00a11a0 0x7fb8d00a3660 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.471+0000 7fb8ed7da700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8e80ffd60 0x7fb8e8068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.471+0000 7fb8ed7da700 1 --2- 172.21.15.33:0/1924900468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e8100f50 0x7fb8e819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.471+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1924900468 >> 172.21.15.33:0/1924900468 conn(0x7fb8e80fb290 msgr2=0x7fb8e81043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.471+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1924900468 shutdown_connections 2024-05-08T21:22:15.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.471+0000 7fb8ed7da700 1 -- 172.21.15.33:0/1924900468 wait complete. 2024-05-08T21:22:15.494 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:22:15.542 INFO:tasks.workunit.client.0.smithi033.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-05-08T21:22:15.569 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:15 smithi071 ceph-mon[47074]: pgmap v388: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-08T21:22:15.569 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:22:15.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:15 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3259965299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:22:15.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:15.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:15.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:15.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:15 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:15.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:15 smithi082 ceph-mon[47512]: pgmap v388: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-08T21:22:15.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:22:15.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:15 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3259965299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:22:15.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:15.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:15.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:15.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:15 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:15.651 INFO:tasks.workunit.client.0.smithi033.stdout:checking attr/attributes.h usability... yes 2024-05-08T21:22:15.678 INFO:tasks.workunit.client.0.smithi033.stdout:checking attr/attributes.h presence... yes 2024-05-08T21:22:15.678 INFO:tasks.workunit.client.0.smithi033.stdout:checking for attr/attributes.h... yes 2024-05-08T21:22:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.695+0000 7ff49986c700 1 -- 172.21.15.33:0/2307986270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4940ffd00 msgr2=0x7ff494100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.695+0000 7ff49986c700 1 --2- 172.21.15.33:0/2307986270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4940ffd00 0x7ff494100120 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff4840098f0 tx=0x7ff484009c00 comp rx=0 tx=0).stop 2024-05-08T21:22:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.695+0000 7ff49986c700 1 -- 172.21.15.33:0/2307986270 shutdown_connections 2024-05-08T21:22:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.695+0000 7ff49986c700 1 --2- 172.21.15.33:0/2307986270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff494101890 0x7ff494108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.695+0000 7ff49986c700 1 --2- 172.21.15.33:0/2307986270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4940ffd00 0x7ff494100120 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.695+0000 7ff49986c700 1 --2- 172.21.15.33:0/2307986270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff494100ef0 0x7ff494101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.695+0000 7ff49986c700 1 -- 172.21.15.33:0/2307986270 >> 172.21.15.33:0/2307986270 conn(0x7ff4940fb250 msgr2=0x7ff4940fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.695+0000 7ff49986c700 1 -- 172.21.15.33:0/2307986270 shutdown_connections 2024-05-08T21:22:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.695+0000 7ff49986c700 1 -- 172.21.15.33:0/2307986270 wait complete. 2024-05-08T21:22:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.696+0000 7ff49986c700 1 Processor -- start 2024-05-08T21:22:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.696+0000 7ff49986c700 1 -- start start 2024-05-08T21:22:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.696+0000 7ff49986c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4940ffd00 0x7ff49419d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.696+0000 7ff49986c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff494100ef0 0x7ff49419d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.696+0000 7ff49986c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff494101890 0x7ff4941a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.696+0000 7ff49986c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4941a4e40 con 0x7ff4940ffd00 2024-05-08T21:22:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.696+0000 7ff49986c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4941a4f80 con 0x7ff494100ef0 2024-05-08T21:22:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.696+0000 7ff49986c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4941a5110 con 0x7ff494101890 2024-05-08T21:22:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.697+0000 7ff4927fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff494100ef0 0x7ff49419d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.697+0000 7ff492ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4940ffd00 0x7ff49419d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.697+0000 7ff492ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4940ffd00 0x7ff49419d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47526/0 (socket says 172.21.15.33:47526) 2024-05-08T21:22:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.697+0000 7ff4927fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff494100ef0 0x7ff49419d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44300/0 (socket says 172.21.15.33:44300) 2024-05-08T21:22:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.697+0000 7ff492ffd700 1 -- 172.21.15.33:0/3551620796 learned_addr learned my addr 172.21.15.33:0/3551620796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.697+0000 7ff4937fe700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff494101890 0x7ff4941a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.697+0000 7ff4927fc700 1 -- 172.21.15.33:0/3551620796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff494101890 msgr2=0x7ff4941a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.697+0000 7ff4927fc700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff494101890 0x7ff4941a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.698+0000 7ff4927fc700 1 -- 172.21.15.33:0/3551620796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4940ffd00 msgr2=0x7ff49419d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.698+0000 7ff4927fc700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4940ffd00 0x7ff49419d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.698+0000 7ff4927fc700 1 -- 172.21.15.33:0/3551620796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4880095f0 con 0x7ff494100ef0 2024-05-08T21:22:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.698+0000 7ff4927fc700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff494100ef0 0x7ff49419d850 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff48800eb90 tx=0x7ff48800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.698+0000 7ff49886a700 1 -- 172.21.15.33:0/3551620796 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48800cc30 con 0x7ff494100ef0 2024-05-08T21:22:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.698+0000 7ff49886a700 1 -- 172.21.15.33:0/3551620796 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4880044b0 con 0x7ff494100ef0 2024-05-08T21:22:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.698+0000 7ff49986c700 1 -- 172.21.15.33:0/3551620796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c009c40 con 0x7ff494100ef0 2024-05-08T21:22:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.698+0000 7ff49986c700 1 -- 172.21.15.33:0/3551620796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff484009520 con 0x7ff494100ef0 2024-05-08T21:22:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.698+0000 7ff49886a700 1 -- 172.21.15.33:0/3551620796 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488003dd0 con 0x7ff494100ef0 2024-05-08T21:22:15.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.701+0000 7ff49886a700 1 -- 172.21.15.33:0/3551620796 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff488018430 con 0x7ff494100ef0 2024-05-08T21:22:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.702+0000 7ff49886a700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff47c0a1120 0x7ff47c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.702+0000 7ff492ffd700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff47c0a1120 0x7ff47c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.702+0000 7ff49886a700 1 -- 172.21.15.33:0/3551620796 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff488014070 con 0x7ff494100ef0 2024-05-08T21:22:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.703+0000 7ff492ffd700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff47c0a1120 0x7ff47c0a35e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff484009fd0 tx=0x7ff484019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.703+0000 7ff4a5a26b80 1 Processor -- start 2024-05-08T21:22:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff4a5a26b80 1 -- start start 2024-05-08T21:22:15.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff4a5a26b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfc540 0x55c480dfc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff4a5a26b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfcea0 0x55c480dfd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff4a5a26b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480dfdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff4a5a26b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480d53150 con 0x55c480dfcea0 2024-05-08T21:22:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff4a5a26b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480d50b10 con 0x55c480dfd860 2024-05-08T21:22:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff4a5a26b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480d4c400 con 0x55c480dfc540 2024-05-08T21:22:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff4937fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480dfdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff4927fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfcea0 0x55c480dfd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff492ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfc540 0x55c480dfc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff4937fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480dfdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44314/0 (socket says 172.21.15.33:44314) 2024-05-08T21:22:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff4937fe700 1 -- 172.21.15.33:0/1339002837 learned_addr learned my addr 172.21.15.33:0/1339002837 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.704+0000 7ff492ffd700 1 --2- 172.21.15.33:0/1339002837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfc540 0x55c480dfc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59506/0 (socket says 172.21.15.33:59506) 2024-05-08T21:22:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.705+0000 7ff4937fe700 1 -- 172.21.15.33:0/1339002837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfc540 msgr2=0x55c480dfc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.705+0000 7ff4937fe700 1 --2- 172.21.15.33:0/1339002837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfc540 0x55c480dfc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.705+0000 7ff4937fe700 1 -- 172.21.15.33:0/1339002837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfcea0 msgr2=0x55c480dfd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.705+0000 7ff4937fe700 1 --2- 172.21.15.33:0/1339002837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfcea0 0x55c480dfd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.705+0000 7ff4937fe700 1 -- 172.21.15.33:0/1339002837 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c480bcd3b0 con 0x55c480dfd860 2024-05-08T21:22:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.705+0000 7ff492ffd700 1 --2- 172.21.15.33:0/1339002837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfc540 0x55c480dfc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.705+0000 7ff4937fe700 1 --2- 172.21.15.33:0/1339002837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480dfdce0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00e760 tx=0x7ff48c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af21a57ed786bcb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.705+0000 7ff479ffb700 1 -- 172.21.15.33:0/1339002837 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00e040 con 0x55c480dfd860 2024-05-08T21:22:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.705+0000 7ff479ffb700 1 -- 172.21.15.33:0/1339002837 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff48c00f040 con 0x55c480dfd860 2024-05-08T21:22:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.705+0000 7ff479ffb700 1 -- 172.21.15.33:0/1339002837 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c007500 con 0x55c480dfd860 2024-05-08T21:22:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.706+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/1339002837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 msgr2=0x55c480dfdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.706+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/1339002837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480dfdce0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00e760 tx=0x7ff48c009920 comp rx=0 tx=0).stop 2024-05-08T21:22:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.706+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/1339002837 shutdown_connections 2024-05-08T21:22:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.706+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/1339002837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480dfdce0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.706+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/1339002837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfc540 0x55c480dfc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.706+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/1339002837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfcea0 0x55c480dfd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.706+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/1339002837 >> 172.21.15.33:0/1339002837 conn(0x55c480d5a210 msgr2=0x55c480df6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.706+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/1339002837 shutdown_connections 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.706+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/1339002837 wait complete. 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.707+0000 7ff4a5a26b80 1 -- start start 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.707+0000 7ff4a5a26b80 1 Processor -- start 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.707+0000 7ff4a5a26b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e05dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.707+0000 7ff4a5a26b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e06310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.707+0000 7ff4a5a26b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e0d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.707+0000 7ff4a5a26b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480ce8560 con 0x55c480dfc540 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.707+0000 7ff4a5a26b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480ce8200 con 0x55c480dfd860 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.707+0000 7ff4a5a26b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480bcf6a0 con 0x55c480dfcea0 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.707+0000 7ff4a5a26b80 10 client.0 _ll_register_callbacks cb 0x55c480cf1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.707+0000 7ff492ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e05dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.708+0000 7ff492ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e05dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47556/0 (socket says 172.21.15.33:47556) 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.708+0000 7ff492ffd700 1 -- 172.21.15.33:0/3827269640 learned_addr learned my addr 172.21.15.33:0/3827269640 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.708+0000 7ff4937fe700 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e0d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.708+0000 7ff4927fc700 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e06310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.708+0000 7ff492ffd700 1 -- 172.21.15.33:0/3827269640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 msgr2=0x55c480e06310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.708+0000 7ff492ffd700 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e06310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.708+0000 7ff492ffd700 1 -- 172.21.15.33:0/3827269640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 msgr2=0x55c480e0d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.708+0000 7ff492ffd700 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e0d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.708+0000 7ff492ffd700 1 -- 172.21.15.33:0/3827269640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff494069810 con 0x55c480dfc540 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.708+0000 7ff4937fe700 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e0d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.708+0000 7ff492ffd700 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e05dd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff48400fb00 tx=0x7ff484009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d5cc42cb54b510 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.708+0000 7ff472ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff472ffd700 1 -- 172.21.15.33:0/3827269640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48402b040 con 0x55c480dfc540 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff472ffd700 1 -- 172.21.15.33:0/3827269640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff484005240 con 0x55c480dfc540 2024-05-08T21:22:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3827269640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c480ca5270 con 0x55c480dfc540 2024-05-08T21:22:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff472ffd700 1 -- 172.21.15.33:0/3827269640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff48400bbe0 con 0x55c480dfc540 2024-05-08T21:22:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff4a5a26b80 10 client.16455 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3827269640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7ff48c00e410 con 0x55c480dfc540 2024-05-08T21:22:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff472ffd700 1 -- 172.21.15.33:0/3827269640 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c480ca5270 con 0x55c480dfc540 2024-05-08T21:22:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff472ffd700 1 -- 172.21.15.33:0/3827269640 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff484025430 con 0x55c480dfc540 2024-05-08T21:22:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff4a5a26b80 10 client.16455 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff4a5a26b80 10 client.16455 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:22:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff4a5a26b80 20 client.16455 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c480e11bf0 0x55c480e14010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff4a5a26b80 4 client.16455 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.709+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3827269640 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c480dcb810 con 0x55c480e11bf0 2024-05-08T21:22:15.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.710+0000 7ff4927fc700 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c480e11bf0 0x55c480e14010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.710+0000 7ff472ffd700 10 client.16455 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:22:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.710+0000 7ff4927fc700 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c480e11bf0 0x55c480e14010 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e497e6c84f04ca6d server_cookie=c55375d9bf71b844 in_seq=0 out_seq=0 2024-05-08T21:22:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff472ffd700 1 -- 172.21.15.33:0/3827269640 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c480dcb810 con 0x55c480e11bf0 2024-05-08T21:22:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff472ffd700 10 client.16455 handle_command_reply: tid=0 2024-05-08T21:22:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff4a5a26b80 1 client.16455 shutdown 2024-05-08T21:22:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3827269640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 msgr2=0x55c480e05dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e05dd0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff48400fb00 tx=0x7ff484009c00 comp rx=0 tx=0).stop 2024-05-08T21:22:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3827269640 shutdown_connections 2024-05-08T21:22:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c480e11bf0 0x55c480e14010 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e0d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e05dd0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/3827269640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e06310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3827269640 >> 172.21.15.33:0/3827269640 conn(0x55c480d5a210 msgr2=0x55c480df65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff472ffd700 0 client.16455 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:22:15.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3827269640 shutdown_connections 2024-05-08T21:22:15.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.711+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3827269640 wait complete. 2024-05-08T21:22:15.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.712+0000 7ff4a5a26b80 20 client.16455 trim_cache size 0 max 16384 2024-05-08T21:22:15.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.729+0000 7ff4a5a26b80 1 Processor -- start 2024-05-08T21:22:15.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.729+0000 7ff4a5a26b80 1 -- start start 2024-05-08T21:22:15.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.729+0000 7ff4a5a26b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e03d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.729+0000 7ff4a5a26b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e0d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.729+0000 7ff4a5a26b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e00730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4a5a26b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480e048d0 con 0x55c480dfc540 2024-05-08T21:22:15.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4a5a26b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480df1800 con 0x55c480dfd860 2024-05-08T21:22:15.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4a5a26b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480df7b60 con 0x55c480dfcea0 2024-05-08T21:22:15.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4927fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e0d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff492ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e03d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4927fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e0d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59516/0 (socket says 172.21.15.33:59516) 2024-05-08T21:22:15.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4927fc700 1 -- 172.21.15.33:0/2464903832 learned_addr learned my addr 172.21.15.33:0/2464903832 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4937fe700 1 --2- 172.21.15.33:0/2464903832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e00730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4927fc700 1 -- 172.21.15.33:0/2464903832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 msgr2=0x55c480e00730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4927fc700 1 --2- 172.21.15.33:0/2464903832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e00730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4927fc700 1 -- 172.21.15.33:0/2464903832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 msgr2=0x55c480e03d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4927fc700 1 --2- 172.21.15.33:0/2464903832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e03d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4927fc700 1 -- 172.21.15.33:0/2464903832 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c480b79b60 con 0x55c480dfcea0 2024-05-08T21:22:15.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4937fe700 1 --2- 172.21.15.33:0/2464903832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e00730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:22:15.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff4927fc700 1 --2- 172.21.15.33:0/2464903832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e0d2e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff4880ca740 tx=0x7ff488000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c615c7d9e7936d0b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.730+0000 7ff47a7fc700 1 -- 172.21.15.33:0/2464903832 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48809dec0 con 0x55c480dfcea0 2024-05-08T21:22:15.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.731+0000 7ff47a7fc700 1 -- 172.21.15.33:0/2464903832 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4880a2510 con 0x55c480dfcea0 2024-05-08T21:22:15.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.731+0000 7ff47a7fc700 1 -- 172.21.15.33:0/2464903832 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48809a2f0 con 0x55c480dfcea0 2024-05-08T21:22:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.731+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/2464903832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 msgr2=0x55c480e0d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.731+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/2464903832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e0d2e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff4880ca740 tx=0x7ff488000bc0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.731+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/2464903832 shutdown_connections 2024-05-08T21:22:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.731+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/2464903832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e00730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.731+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/2464903832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e03d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.731+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/2464903832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e0d2e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.731+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/2464903832 >> 172.21.15.33:0/2464903832 conn(0x55c480d5a210 msgr2=0x55c480e052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.731+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/2464903832 shutdown_connections 2024-05-08T21:22:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.731+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/2464903832 wait complete. 2024-05-08T21:22:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.732+0000 7ff4a5a26b80 1 -- start start 2024-05-08T21:22:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.732+0000 7ff4a5a26b80 1 Processor -- start 2024-05-08T21:22:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff4a5a26b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e08740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff4a5a26b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e08c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff4a5a26b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e0bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff4a5a26b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480e0c190 con 0x55c480dfc540 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff4a5a26b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480e0c300 con 0x55c480dfd860 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff4a5a26b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c480e0c4c0 con 0x55c480dfcea0 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff4a5a26b80 10 client.0 _ll_register_callbacks cb 0x55c480cf1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff492ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e08740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff492ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e08740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47576/0 (socket says 172.21.15.33:47576) 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff492ffd700 1 -- 172.21.15.33:0/3917122317 learned_addr learned my addr 172.21.15.33:0/3917122317 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff4927fc700 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e08c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff4937fe700 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e0bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff492ffd700 1 -- 172.21.15.33:0/3917122317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 msgr2=0x55c480e08c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff492ffd700 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e08c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff492ffd700 1 -- 172.21.15.33:0/3917122317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 msgr2=0x55c480e0bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff492ffd700 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e0bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff492ffd700 1 -- 172.21.15.33:0/3917122317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff484009520 con 0x55c480dfc540 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff4927fc700 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e08c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.733+0000 7ff4937fe700 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e0bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.734+0000 7ff492ffd700 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e08740 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff48400f9d0 tx=0x7ff484025f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd79a9e6a728f06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.734+0000 7ff472ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.734+0000 7ff472ffd700 1 -- 172.21.15.33:0/3917122317 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48402b070 con 0x55c480dfc540 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.734+0000 7ff472ffd700 1 -- 172.21.15.33:0/3917122317 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff4840050d0 con 0x55c480dfc540 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.734+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3917122317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff48c00f040 con 0x55c480dfc540 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.734+0000 7ff472ffd700 1 -- 172.21.15.33:0/3917122317 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff48400fba0 con 0x55c480dfc540 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.734+0000 7ff4a5a26b80 10 client.16461 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.734+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3917122317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55c480d12560 con 0x55c480dfc540 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.735+0000 7ff472ffd700 1 -- 172.21.15.33:0/3917122317 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff48c00f040 con 0x55c480dfc540 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.735+0000 7ff472ffd700 1 -- 172.21.15.33:0/3917122317 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff484037030 con 0x55c480dfc540 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.735+0000 7ff4a5a26b80 10 client.16461 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.735+0000 7ff4a5a26b80 10 client.16461 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.735+0000 7ff4a5a26b80 20 client.16461 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.735+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c480e683e0 0x55c480e6a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.735+0000 7ff4a5a26b80 4 client.16461 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.735+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3917122317 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c480d44ec0 con 0x55c480e683e0 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.735+0000 7ff4927fc700 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c480e683e0 0x55c480e6a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff472ffd700 10 client.16461 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff4927fc700 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c480e683e0 0x55c480e6a800 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d24b5f6b5bc6b22 server_cookie=b05b63efbe9133d1 in_seq=0 out_seq=0 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff472ffd700 1 -- 172.21.15.33:0/3917122317 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c480d44ec0 con 0x55c480e683e0 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff472ffd700 10 client.16461 handle_command_reply: tid=0 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff4a5a26b80 1 client.16461 shutdown 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3917122317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 msgr2=0x55c480e08740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e08740 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff48400f9d0 tx=0x7ff484025f70 comp rx=0 tx=0).stop 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3917122317 shutdown_connections 2024-05-08T21:22:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c480e683e0 0x55c480e6a800 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c480dfd860 0x55c480e0bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c480dfc540 0x55c480e08740 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff4a5a26b80 1 --2- 172.21.15.33:0/3917122317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c480dfcea0 0x55c480e08c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3917122317 >> 172.21.15.33:0/3917122317 conn(0x55c480d5a210 msgr2=0x55c480e03cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff472ffd700 0 client.16461 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:22:15.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3917122317 shutdown_connections 2024-05-08T21:22:15.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.736+0000 7ff4a5a26b80 1 -- 172.21.15.33:0/3917122317 wait complete. 2024-05-08T21:22:15.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.737+0000 7ff4a5a26b80 20 client.16461 trim_cache size 0 max 16384 2024-05-08T21:22:15.743 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:22:15.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.737+0000 7ff49986c700 1 -- 172.21.15.33:0/3551620796 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff47c0a1120 msgr2=0x7ff47c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.737+0000 7ff49986c700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff47c0a1120 0x7ff47c0a35e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff484009fd0 tx=0x7ff484019040 comp rx=0 tx=0).stop 2024-05-08T21:22:15.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.737+0000 7ff49986c700 1 -- 172.21.15.33:0/3551620796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff494100ef0 msgr2=0x7ff49419d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.737+0000 7ff49986c700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff494100ef0 0x7ff49419d850 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff48800eb90 tx=0x7ff48800eea0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.737+0000 7ff49986c700 1 -- 172.21.15.33:0/3551620796 shutdown_connections 2024-05-08T21:22:15.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.737+0000 7ff49986c700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff494101890 0x7ff4941a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.737+0000 7ff49986c700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4940ffd00 0x7ff49419d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.737+0000 7ff49986c700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff47c0a1120 0x7ff47c0a35e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.737+0000 7ff49986c700 1 --2- 172.21.15.33:0/3551620796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff494100ef0 0x7ff49419d850 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.737+0000 7ff49986c700 1 -- 172.21.15.33:0/3551620796 >> 172.21.15.33:0/3551620796 conn(0x7ff4940fb250 msgr2=0x7ff494102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.738+0000 7ff49986c700 1 -- 172.21.15.33:0/3551620796 shutdown_connections 2024-05-08T21:22:15.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.738+0000 7ff49986c700 1 -- 172.21.15.33:0/3551620796 wait complete. 2024-05-08T21:22:15.758 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:22:15.790 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/attributes.h usability... no 2024-05-08T21:22:15.842 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/attributes.h presence... no 2024-05-08T21:22:15.843 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/attributes.h... no 2024-05-08T21:22:15.925 INFO:tasks.workunit.client.0.smithi033.stdout:checking for attr_list in -lattr... yes 2024-05-08T21:22:15.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.944+0000 7f0ac066d700 1 -- 172.21.15.33:0/2922893910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab8101330 msgr2=0x7f0ab8101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.944+0000 7f0ac066d700 1 --2- 172.21.15.33:0/2922893910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab8101330 0x7f0ab8101750 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40099c0 tx=0x7f0ab4009cd0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.945+0000 7f0ac066d700 1 -- 172.21.15.33:0/2922893910 shutdown_connections 2024-05-08T21:22:15.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.945+0000 7f0ac066d700 1 --2- 172.21.15.33:0/2922893910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ab8100940 0x7f0ab81081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.945+0000 7f0ac066d700 1 --2- 172.21.15.33:0/2922893910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab8101330 0x7f0ab8101750 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.945+0000 7f0ac066d700 1 --2- 172.21.15.33:0/2922893910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab80fff80 0x7f0ab8100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.945+0000 7f0ac066d700 1 -- 172.21.15.33:0/2922893910 >> 172.21.15.33:0/2922893910 conn(0x7f0ab80faa50 msgr2=0x7f0ab80fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.945+0000 7f0ac066d700 1 -- 172.21.15.33:0/2922893910 shutdown_connections 2024-05-08T21:22:15.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.945+0000 7f0ac066d700 1 -- 172.21.15.33:0/2922893910 wait complete. 2024-05-08T21:22:15.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.945+0000 7f0ac066d700 1 Processor -- start 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.945+0000 7f0ac066d700 1 -- start start 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0ac066d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab80fff80 0x7f0ab8103ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0ac066d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab8100940 0x7f0ab8102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0ac066d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ab8101330 0x7f0ab81028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0ac066d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8103050 con 0x7f0ab8100940 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0ac066d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81031c0 con 0x7f0ab8101330 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0ac066d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81a0d50 con 0x7f0ab80fff80 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abdc08700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab8100940 0x7f0ab8102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abe409700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab80fff80 0x7f0ab8103ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abdc08700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab8100940 0x7f0ab8102370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47584/0 (socket says 172.21.15.33:47584) 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abe409700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab80fff80 0x7f0ab8103ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59546/0 (socket says 172.21.15.33:59546) 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abdc08700 1 -- 172.21.15.33:0/587058935 learned_addr learned my addr 172.21.15.33:0/587058935 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abec0a700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ab8101330 0x7f0ab81028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abe409700 1 -- 172.21.15.33:0/587058935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ab8101330 msgr2=0x7f0ab81028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abe409700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ab8101330 0x7f0ab81028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abe409700 1 -- 172.21.15.33:0/587058935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab8100940 msgr2=0x7f0ab8102370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abe409700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab8100940 0x7f0ab8102370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abe409700 1 -- 172.21.15.33:0/587058935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40095f0 con 0x7f0ab80fff80 2024-05-08T21:22:15.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abdc08700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab8100940 0x7f0ab8102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:15.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.946+0000 7f0abe409700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab80fff80 0x7f0ab8103ce0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400bf10 tx=0x7f0ab400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.947+0000 7f0aaf7fe700 1 -- 172.21.15.33:0/587058935 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab401d070 con 0x7f0ab80fff80 2024-05-08T21:22:15.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.947+0000 7f0aaf7fe700 1 -- 172.21.15.33:0/587058935 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ab4005020 con 0x7f0ab80fff80 2024-05-08T21:22:15.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.947+0000 7f0ac066d700 1 -- 172.21.15.33:0/587058935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab81a0f80 con 0x7f0ab80fff80 2024-05-08T21:22:15.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.947+0000 7f0aaf7fe700 1 -- 172.21.15.33:0/587058935 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4004a00 con 0x7f0ab80fff80 2024-05-08T21:22:15.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.947+0000 7f0ac066d700 1 -- 172.21.15.33:0/587058935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab81a12f0 con 0x7f0ab80fff80 2024-05-08T21:22:15.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.950+0000 7f0aaf7fe700 1 -- 172.21.15.33:0/587058935 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0ab400fad0 con 0x7f0ab80fff80 2024-05-08T21:22:15.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.950+0000 7f0aaf7fe700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0aa40a11a0 0x7f0aa40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.950+0000 7f0aaf7fe700 1 -- 172.21.15.33:0/587058935 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f0ab40cd5d0 con 0x7f0ab80fff80 2024-05-08T21:22:15.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.950+0000 7f0abdc08700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0aa40a11a0 0x7f0aa40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.951+0000 7f0abdc08700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0aa40a11a0 0x7f0aa40a3660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8005950 tx=0x7f0aa8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.951+0000 7f0acc827b80 1 Processor -- start 2024-05-08T21:22:15.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.951+0000 7f0acc827b80 1 -- start start 2024-05-08T21:22:15.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0acc827b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 0x55d7f49ef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0acc827b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49f0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0acc827b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 0x55d7f49f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0acc827b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f4946150 con 0x55d7f49ef540 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0acc827b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f4943b10 con 0x55d7f49f0860 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0acc827b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f493f400 con 0x55d7f49efea0 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abe409700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 0x55d7f49ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abe409700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 0x55d7f49ef960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47586/0 (socket says 172.21.15.33:47586) 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abe409700 1 -- 172.21.15.33:0/762497157 learned_addr learned my addr 172.21.15.33:0/762497157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abec0a700 1 --2- 172.21.15.33:0/762497157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 0x55d7f49f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abdc08700 1 --2- 172.21.15.33:0/762497157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49f0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abe409700 1 -- 172.21.15.33:0/762497157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 msgr2=0x55d7f49f0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abe409700 1 --2- 172.21.15.33:0/762497157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49f0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abe409700 1 -- 172.21.15.33:0/762497157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 msgr2=0x55d7f49f0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abe409700 1 --2- 172.21.15.33:0/762497157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 0x55d7f49f0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abe409700 1 -- 172.21.15.33:0/762497157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7f47c03b0 con 0x55d7f49ef540 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abdc08700 1 --2- 172.21.15.33:0/762497157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49f0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abec0a700 1 --2- 172.21.15.33:0/762497157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 0x55d7f49f0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.952+0000 7f0abe409700 1 --2- 172.21.15.33:0/762497157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 0x55d7f49ef960 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4096c30 tx=0x7f0ab409a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90704cb937bb382d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.953+0000 7f0aacff9700 1 -- 172.21.15.33:0/762497157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400fd80 con 0x55d7f49ef540 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.953+0000 7f0aacff9700 1 -- 172.21.15.33:0/762497157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ab409ae30 con 0x55d7f49ef540 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.953+0000 7f0aacff9700 1 -- 172.21.15.33:0/762497157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4098ad0 con 0x55d7f49ef540 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.953+0000 7f0acc827b80 1 -- 172.21.15.33:0/762497157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 msgr2=0x55d7f49ef960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.953+0000 7f0acc827b80 1 --2- 172.21.15.33:0/762497157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 0x55d7f49ef960 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4096c30 tx=0x7f0ab409a570 comp rx=0 tx=0).stop 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 -- 172.21.15.33:0/762497157 shutdown_connections 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 --2- 172.21.15.33:0/762497157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 0x55d7f49f0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 --2- 172.21.15.33:0/762497157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 0x55d7f49ef960 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 --2- 172.21.15.33:0/762497157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49f0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 -- 172.21.15.33:0/762497157 >> 172.21.15.33:0/762497157 conn(0x55d7f494d210 msgr2=0x55d7f49e9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 -- 172.21.15.33:0/762497157 shutdown_connections 2024-05-08T21:22:15.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 -- 172.21.15.33:0/762497157 wait complete. 2024-05-08T21:22:15.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 -- start start 2024-05-08T21:22:15.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 Processor -- start 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 0x55d7f49f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49f8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49f0860 0x55d7f49fff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f48db560 con 0x55d7f49f0860 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f48db200 con 0x55d7f49ef540 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f47c26a0 con 0x55d7f49efea0 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.954+0000 7f0acc827b80 10 client.0 _ll_register_callbacks cb 0x55d7f48e4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abec0a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49f0860 0x55d7f49fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abe409700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 0x55d7f49f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abe409700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 0x55d7f49f8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44376/0 (socket says 172.21.15.33:44376) 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abe409700 1 -- 172.21.15.33:0/882147088 learned_addr learned my addr 172.21.15.33:0/882147088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abec0a700 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49f0860 0x55d7f49fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47592/0 (socket says 172.21.15.33:47592) 2024-05-08T21:22:15.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abdc08700 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49f8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abe409700 1 -- 172.21.15.33:0/882147088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 msgr2=0x55d7f49f8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abe409700 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49f8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abe409700 1 -- 172.21.15.33:0/882147088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49f0860 msgr2=0x55d7f49fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abe409700 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49f0860 0x55d7f49fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abe409700 1 -- 172.21.15.33:0/882147088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab0011230 con 0x55d7f49ef540 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abdc08700 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0abe409700 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 0x55d7f49f8a10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400b580 tx=0x7f0ab409e490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c31fc52ba91799b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0aa1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0aa1ffb700 1 -- 172.21.15.33:0/882147088 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab409a730 con 0x55d7f49ef540 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0acc827b80 1 -- 172.21.15.33:0/882147088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7f4898270 con 0x55d7f49ef540 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.955+0000 7f0aa1ffb700 1 -- 172.21.15.33:0/882147088 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f0ab4098ad0 con 0x55d7f49ef540 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.956+0000 7f0aa1ffb700 1 -- 172.21.15.33:0/882147088 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ab4005440 con 0x55d7f49ef540 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.956+0000 7f0acc827b80 10 client.26345 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.956+0000 7f0acc827b80 1 -- 172.21.15.33:0/882147088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f0ab81a12f0 con 0x55d7f49ef540 2024-05-08T21:22:15.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.956+0000 7f0aa1ffb700 1 -- 172.21.15.33:0/882147088 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ab40a6020 con 0x55d7f49ef540 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.956+0000 7f0aa1ffb700 1 -- 172.21.15.33:0/882147088 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0ab40b5070 con 0x55d7f49ef540 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.956+0000 7f0acc827b80 10 client.26345 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.956+0000 7f0acc827b80 10 client.26345 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.956+0000 7f0acc827b80 20 client.26345 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.956+0000 7f0acc827b80 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d7f4a042f0 0x55d7f4a06710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.956+0000 7f0acc827b80 4 client.26345 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.956+0000 7f0acc827b80 1 -- 172.21.15.33:0/882147088 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7f49be810 con 0x55d7f4a042f0 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.956+0000 7f0abdc08700 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d7f4a042f0 0x55d7f4a06710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.957+0000 7f0abdc08700 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d7f4a042f0 0x55d7f4a06710 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=972ae6be267c952c server_cookie=7260dcf32111f4cf in_seq=0 out_seq=0 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.957+0000 7f0aa1ffb700 10 client.26345 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.957+0000 7f0aa1ffb700 1 -- 172.21.15.33:0/882147088 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d7f49be810 con 0x55d7f4a042f0 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.957+0000 7f0aa1ffb700 10 client.26345 handle_command_reply: tid=0 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.957+0000 7f0acc827b80 1 client.26345 shutdown 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.957+0000 7f0acc827b80 1 -- 172.21.15.33:0/882147088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 msgr2=0x55d7f49f8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.957+0000 7f0acc827b80 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 0x55d7f49f8a10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400b580 tx=0x7f0ab409e490 comp rx=0 tx=0).stop 2024-05-08T21:22:15.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.958+0000 7f0acc827b80 1 -- 172.21.15.33:0/882147088 shutdown_connections 2024-05-08T21:22:15.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.958+0000 7f0acc827b80 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d7f4a042f0 0x55d7f4a06710 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.958+0000 7f0acc827b80 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49f0860 0x55d7f49fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.958+0000 7f0aa1ffb700 0 client.26345 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:22:15.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.958+0000 7f0acc827b80 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 0x55d7f49f8a10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.958+0000 7f0acc827b80 1 --2- 172.21.15.33:0/882147088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49f8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.958+0000 7f0acc827b80 1 -- 172.21.15.33:0/882147088 >> 172.21.15.33:0/882147088 conn(0x55d7f494d210 msgr2=0x55d7f49e95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.958+0000 7f0acc827b80 1 -- 172.21.15.33:0/882147088 shutdown_connections 2024-05-08T21:22:15.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.958+0000 7f0acc827b80 1 -- 172.21.15.33:0/882147088 wait complete. 2024-05-08T21:22:15.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.958+0000 7f0acc827b80 20 client.26345 trim_cache size 0 max 16384 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.973+0000 7f0acc827b80 1 Processor -- start 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0acc827b80 1 -- start start 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0acc827b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 0x55d7f49ef920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0acc827b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49fff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0acc827b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 0x55d7f49f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0acc827b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f4a040e0 con 0x55d7f49ef540 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0acc827b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f49e4800 con 0x55d7f49f0860 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0acc827b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f4a0b900 con 0x55d7f49efea0 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0abe409700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 0x55d7f49ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0abec0a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 0x55d7f49f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0abec0a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 0x55d7f49f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44378/0 (socket says 172.21.15.33:44378) 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0abe409700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 0x55d7f49ef920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47604/0 (socket says 172.21.15.33:47604) 2024-05-08T21:22:15.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0abec0a700 1 -- 172.21.15.33:0/2352776882 learned_addr learned my addr 172.21.15.33:0/2352776882 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0abdc08700 1 --2- 172.21.15.33:0/2352776882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0abec0a700 1 -- 172.21.15.33:0/2352776882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 msgr2=0x55d7f49fff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0abec0a700 1 --2- 172.21.15.33:0/2352776882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49fff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0abec0a700 1 -- 172.21.15.33:0/2352776882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 msgr2=0x55d7f49ef920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0abec0a700 1 --2- 172.21.15.33:0/2352776882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 0x55d7f49ef920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.974+0000 7f0abec0a700 1 -- 172.21.15.33:0/2352776882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7f476cb60 con 0x55d7f49f0860 2024-05-08T21:22:15.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.975+0000 7f0abec0a700 1 --2- 172.21.15.33:0/2352776882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 0x55d7f49f7f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0003b30 tx=0x7f0ab0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1572cf737358698a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.975+0000 7f0aad7fa700 1 -- 172.21.15.33:0/2352776882 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000e040 con 0x55d7f49f0860 2024-05-08T21:22:15.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.975+0000 7f0aad7fa700 1 -- 172.21.15.33:0/2352776882 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ab000f040 con 0x55d7f49f0860 2024-05-08T21:22:15.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.975+0000 7f0aad7fa700 1 -- 172.21.15.33:0/2352776882 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0016ac0 con 0x55d7f49f0860 2024-05-08T21:22:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.976+0000 7f0acc827b80 1 -- 172.21.15.33:0/2352776882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 msgr2=0x55d7f49f7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.976+0000 7f0acc827b80 1 --2- 172.21.15.33:0/2352776882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 0x55d7f49f7f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0003b30 tx=0x7f0ab0009920 comp rx=0 tx=0).stop 2024-05-08T21:22:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.976+0000 7f0acc827b80 1 -- 172.21.15.33:0/2352776882 shutdown_connections 2024-05-08T21:22:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.976+0000 7f0acc827b80 1 --2- 172.21.15.33:0/2352776882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49f0860 0x55d7f49f7f20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.976+0000 7f0acc827b80 1 --2- 172.21.15.33:0/2352776882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49ef540 0x55d7f49ef920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.976+0000 7f0acc827b80 1 --2- 172.21.15.33:0/2352776882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.976+0000 7f0acc827b80 1 -- 172.21.15.33:0/2352776882 >> 172.21.15.33:0/2352776882 conn(0x55d7f494d210 msgr2=0x55d7f49f0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.976+0000 7f0acc827b80 1 -- 172.21.15.33:0/2352776882 shutdown_connections 2024-05-08T21:22:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.976+0000 7f0acc827b80 1 -- 172.21.15.33:0/2352776882 wait complete. 2024-05-08T21:22:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.976+0000 7f0acc827b80 1 -- start start 2024-05-08T21:22:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0acc827b80 1 Processor -- start 2024-05-08T21:22:15.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0acc827b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 0x55d7f49fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0acc827b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49fc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0acc827b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49f0860 0x55d7f49ff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0acc827b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f49ff5a0 con 0x55d7f49f0860 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0acc827b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f49ff710 con 0x55d7f49ef540 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0acc827b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7f49ff8d0 con 0x55d7f49efea0 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0acc827b80 10 client.0 _ll_register_callbacks cb 0x55d7f48e4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0abe409700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 0x55d7f49fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0abe409700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 0x55d7f49fbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44388/0 (socket says 172.21.15.33:44388) 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0abe409700 1 -- 172.21.15.33:0/3050677593 learned_addr learned my addr 172.21.15.33:0/3050677593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0abec0a700 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49f0860 0x55d7f49ff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0abdc08700 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49fc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0abe409700 1 -- 172.21.15.33:0/3050677593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 msgr2=0x55d7f49fc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0abe409700 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49fc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0abe409700 1 -- 172.21.15.33:0/3050677593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49f0860 msgr2=0x55d7f49ff060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:15.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0abe409700 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49f0860 0x55d7f49ff060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0abe409700 1 -- 172.21.15.33:0/3050677593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab00092f0 con 0x55d7f49ef540 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.977+0000 7f0abec0a700 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49f0860 0x55d7f49ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0abe409700 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 0x55d7f49fbb00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4096c30 tx=0x7f0ab409a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e31a1203fcc4fc3e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0aa1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0aa1ffb700 1 -- 172.21.15.33:0/3050677593 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40a2b20 con 0x55d7f49ef540 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0aa1ffb700 1 -- 172.21.15.33:0/3050677593 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f0ab40a7080 con 0x55d7f49ef540 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0acc827b80 1 -- 172.21.15.33:0/3050677593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ab409ae30 con 0x55d7f49ef540 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0aa1ffb700 1 -- 172.21.15.33:0/3050677593 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ab4005190 con 0x55d7f49ef540 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0acc827b80 10 client.26351 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0acc827b80 1 -- 172.21.15.33:0/3050677593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d7f4905560 con 0x55d7f49ef540 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0aa1ffb700 1 -- 172.21.15.33:0/3050677593 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ab409ae30 con 0x55d7f49ef540 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0aa1ffb700 1 -- 172.21.15.33:0/3050677593 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0ab4096e40 con 0x55d7f49ef540 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0acc827b80 10 client.26351 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:15.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0acc827b80 10 client.26351 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0acc827b80 20 client.26351 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0acc827b80 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d7f4a5afe0 0x55d7f4a5d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0acc827b80 4 client.26351 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.978+0000 7f0acc827b80 1 -- 172.21.15.33:0/3050677593 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d7f4937ec0 con 0x55d7f4a5afe0 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0abdc08700 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d7f4a5afe0 0x55d7f4a5d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0aa1ffb700 10 client.26351 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0abdc08700 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d7f4a5afe0 0x55d7f4a5d3d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb214f15831ca009 server_cookie=91111598acffa7ed in_seq=0 out_seq=0 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0aa1ffb700 1 -- 172.21.15.33:0/3050677593 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d7f4937ec0 con 0x55d7f4a5afe0 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0aa1ffb700 10 client.26351 handle_command_reply: tid=0 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0acc827b80 1 client.26351 shutdown 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0acc827b80 1 -- 172.21.15.33:0/3050677593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 msgr2=0x55d7f49fbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0acc827b80 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 0x55d7f49fbb00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4096c30 tx=0x7f0ab409a370 comp rx=0 tx=0).stop 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0acc827b80 1 -- 172.21.15.33:0/3050677593 shutdown_connections 2024-05-08T21:22:15.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0acc827b80 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d7f4a5afe0 0x55d7f4a5d3d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0acc827b80 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7f49f0860 0x55d7f49ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0acc827b80 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7f49ef540 0x55d7f49fbb00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0acc827b80 1 --2- 172.21.15.33:0/3050677593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7f49efea0 0x55d7f49fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0acc827b80 1 -- 172.21.15.33:0/3050677593 >> 172.21.15.33:0/3050677593 conn(0x55d7f494d210 msgr2=0x55d7f49f36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.979+0000 7f0aa1ffb700 0 client.26351 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:22:15.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0acc827b80 1 -- 172.21.15.33:0/3050677593 shutdown_connections 2024-05-08T21:22:15.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0acc827b80 1 -- 172.21.15.33:0/3050677593 wait complete. 2024-05-08T21:22:15.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0acc827b80 20 client.26351 trim_cache size 0 max 16384 2024-05-08T21:22:15.987 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:22:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0ac066d700 1 -- 172.21.15.33:0/587058935 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0aa40a11a0 msgr2=0x7f0aa40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0ac066d700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0aa40a11a0 0x7f0aa40a3660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8005950 tx=0x7f0aa8009260 comp rx=0 tx=0).stop 2024-05-08T21:22:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0ac066d700 1 -- 172.21.15.33:0/587058935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab80fff80 msgr2=0x7f0ab8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0ac066d700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab80fff80 0x7f0ab8103ce0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ab400bf10 tx=0x7f0ab400f690 comp rx=0 tx=0).stop 2024-05-08T21:22:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0ac066d700 1 -- 172.21.15.33:0/587058935 shutdown_connections 2024-05-08T21:22:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0ac066d700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0aa40a11a0 0x7f0aa40a3660 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0ac066d700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ab8101330 0x7f0ab81028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0ac066d700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab80fff80 0x7f0ab8103ce0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0ac066d700 1 --2- 172.21.15.33:0/587058935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ab8100940 0x7f0ab8102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.980+0000 7f0ac066d700 1 -- 172.21.15.33:0/587058935 >> 172.21.15.33:0/587058935 conn(0x7f0ab80faa50 msgr2=0x7f0ab80feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:15.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.981+0000 7f0ac066d700 1 -- 172.21.15.33:0/587058935 shutdown_connections 2024-05-08T21:22:15.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:15.981+0000 7f0ac066d700 1 -- 172.21.15.33:0/587058935 wait complete. 2024-05-08T21:22:15.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:22:16.007 INFO:tasks.workunit.client.0.smithi033.stdout:checking for attr_set in -lattr... yes 2024-05-08T21:22:16.073 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/acl.h usability... yes 2024-05-08T21:22:16.102 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/acl.h presence... yes 2024-05-08T21:22:16.103 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/acl.h... yes 2024-05-08T21:22:16.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.148+0000 7fcb4618f700 1 -- 172.21.15.33:0/813858171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb40100f80 msgr2=0x7fcb401013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:16.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.148+0000 7fcb4618f700 1 --2- 172.21.15.33:0/813858171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb40100f80 0x7fcb401013e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcb280099c0 tx=0x7fcb28009cd0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.148+0000 7fcb4618f700 1 -- 172.21.15.33:0/813858171 shutdown_connections 2024-05-08T21:22:16.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.148+0000 7fcb4618f700 1 --2- 172.21.15.33:0/813858171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb40101920 0x7fcb401081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.148+0000 7fcb4618f700 1 --2- 172.21.15.33:0/813858171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb400ffd90 0x7fcb401001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.148+0000 7fcb4618f700 1 --2- 172.21.15.33:0/813858171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb40100f80 0x7fcb401013e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.148+0000 7fcb4618f700 1 -- 172.21.15.33:0/813858171 >> 172.21.15.33:0/813858171 conn(0x7fcb400fb300 msgr2=0x7fcb400fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:16.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.148+0000 7fcb4618f700 1 -- 172.21.15.33:0/813858171 shutdown_connections 2024-05-08T21:22:16.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.148+0000 7fcb4618f700 1 -- 172.21.15.33:0/813858171 wait complete. 2024-05-08T21:22:16.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.149+0000 7fcb4618f700 1 Processor -- start 2024-05-08T21:22:16.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.149+0000 7fcb4618f700 1 -- start start 2024-05-08T21:22:16.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.149+0000 7fcb4618f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb400ffd90 0x7fcb40195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.149+0000 7fcb4618f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb40100f80 0x7fcb40196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.149+0000 7fcb4618f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb40101920 0x7fcb4019d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.149+0000 7fcb4618f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4019d720 con 0x7fcb40100f80 2024-05-08T21:22:16.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.149+0000 7fcb4618f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4019d860 con 0x7fcb400ffd90 2024-05-08T21:22:16.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.149+0000 7fcb4618f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4019d9f0 con 0x7fcb40101920 2024-05-08T21:22:16.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb36ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb40100f80 0x7fcb40196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb3f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb400ffd90 0x7fcb40195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb36ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb40100f80 0x7fcb40196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47636/0 (socket says 172.21.15.33:47636) 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb36ffd700 1 -- 172.21.15.33:0/2117168727 learned_addr learned my addr 172.21.15.33:0/2117168727 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb3ffff700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb40101920 0x7fcb4019d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb3f7fe700 1 -- 172.21.15.33:0/2117168727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb40101920 msgr2=0x7fcb4019d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb3f7fe700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb40101920 0x7fcb4019d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb3f7fe700 1 -- 172.21.15.33:0/2117168727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb40100f80 msgr2=0x7fcb40196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb3f7fe700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb40100f80 0x7fcb40196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb3f7fe700 1 -- 172.21.15.33:0/2117168727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb280095f0 con 0x7fcb400ffd90 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb3f7fe700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb400ffd90 0x7fcb40195bf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000e9c0 tx=0x7fcb3000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb3d7fa700 1 -- 172.21.15.33:0/2117168727 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30004220 con 0x7fcb400ffd90 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb3d7fa700 1 -- 172.21.15.33:0/2117168727 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb30004380 con 0x7fcb400ffd90 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb4618f700 1 -- 172.21.15.33:0/2117168727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4019dcd0 con 0x7fcb400ffd90 2024-05-08T21:22:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb3d7fa700 1 -- 172.21.15.33:0/2117168727 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30003b40 con 0x7fcb400ffd90 2024-05-08T21:22:16.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.150+0000 7fcb4618f700 1 -- 172.21.15.33:0/2117168727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4019e190 con 0x7fcb400ffd90 2024-05-08T21:22:16.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.153+0000 7fcb3d7fa700 1 -- 172.21.15.33:0/2117168727 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcb30003ca0 con 0x7fcb400ffd90 2024-05-08T21:22:16.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.154+0000 7fcb3d7fa700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcb2c0a1120 0x7fcb2c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.154+0000 7fcb36ffd700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcb2c0a1120 0x7fcb2c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.154+0000 7fcb3d7fa700 1 -- 172.21.15.33:0/2117168727 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcb30014070 con 0x7fcb400ffd90 2024-05-08T21:22:16.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.154+0000 7fcb36ffd700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcb2c0a1120 0x7fcb2c0a35e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800b580 tx=0x7fcb28003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:16.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb52349b80 1 Processor -- start 2024-05-08T21:22:16.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb52349b80 1 -- start start 2024-05-08T21:22:16.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb52349b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc53750 0x55f11fc53b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb52349b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc540b0 0x55f11fc54530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb52349b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc54a70 0x55f11fc54ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb52349b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11fba8150 con 0x55f11fc53750 2024-05-08T21:22:16.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb52349b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11fba5b10 con 0x55f11fc540b0 2024-05-08T21:22:16.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb52349b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11fba1400 con 0x55f11fc54a70 2024-05-08T21:22:16.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb3f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc53750 0x55f11fc53b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb3f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc53750 0x55f11fc53b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47642/0 (socket says 172.21.15.33:47642) 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb3f7fe700 1 -- 172.21.15.33:0/2610043445 learned_addr learned my addr 172.21.15.33:0/2610043445 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb36ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc540b0 0x55f11fc54530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.155+0000 7fcb3ffff700 1 --2- 172.21.15.33:0/2610043445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc54a70 0x55f11fc54ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.156+0000 7fcb36ffd700 1 -- 172.21.15.33:0/2610043445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc54a70 msgr2=0x55f11fc54ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.156+0000 7fcb36ffd700 1 --2- 172.21.15.33:0/2610043445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc54a70 0x55f11fc54ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.156+0000 7fcb36ffd700 1 -- 172.21.15.33:0/2610043445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc53750 msgr2=0x55f11fc53b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.156+0000 7fcb36ffd700 1 --2- 172.21.15.33:0/2610043445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc53750 0x55f11fc53b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.156+0000 7fcb36ffd700 1 -- 172.21.15.33:0/2610043445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f11fa223b0 con 0x55f11fc540b0 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.156+0000 7fcb36ffd700 1 --2- 172.21.15.33:0/2610043445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc540b0 0x55f11fc54530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcb28000c00 tx=0x7fcb2801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3199f6fa9fe44a16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.156+0000 7fcb35ffb700 1 -- 172.21.15.33:0/2610043445 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28029070 con 0x55f11fc540b0 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.156+0000 7fcb35ffb700 1 -- 172.21.15.33:0/2610043445 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb2801e040 con 0x55f11fc540b0 2024-05-08T21:22:16.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.156+0000 7fcb35ffb700 1 -- 172.21.15.33:0/2610043445 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800f6c0 con 0x55f11fc540b0 2024-05-08T21:22:16.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.157+0000 7fcb52349b80 1 -- 172.21.15.33:0/2610043445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc540b0 msgr2=0x55f11fc54530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:16.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.157+0000 7fcb52349b80 1 --2- 172.21.15.33:0/2610043445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc540b0 0x55f11fc54530 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcb28000c00 tx=0x7fcb2801f000 comp rx=0 tx=0).stop 2024-05-08T21:22:16.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.157+0000 7fcb52349b80 1 -- 172.21.15.33:0/2610043445 shutdown_connections 2024-05-08T21:22:16.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.157+0000 7fcb52349b80 1 --2- 172.21.15.33:0/2610043445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc54a70 0x55f11fc54ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.157+0000 7fcb52349b80 1 --2- 172.21.15.33:0/2610043445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc53750 0x55f11fc53b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.157+0000 7fcb52349b80 1 --2- 172.21.15.33:0/2610043445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc540b0 0x55f11fc54530 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.157+0000 7fcb52349b80 1 -- 172.21.15.33:0/2610043445 >> 172.21.15.33:0/2610043445 conn(0x55f11fbaf210 msgr2=0x55f11fc4bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:16.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.157+0000 7fcb52349b80 1 -- 172.21.15.33:0/2610043445 shutdown_connections 2024-05-08T21:22:16.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.157+0000 7fcb52349b80 1 -- 172.21.15.33:0/2610043445 wait complete. 2024-05-08T21:22:16.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.158+0000 7fcb52349b80 1 -- start start 2024-05-08T21:22:16.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.158+0000 7fcb52349b80 1 Processor -- start 2024-05-08T21:22:16.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.158+0000 7fcb52349b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc53750 0x55f11fc5cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.158+0000 7fcb52349b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 0x55f11fc5d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.158+0000 7fcb52349b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc54a70 0x55f11fc64550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.158+0000 7fcb52349b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11fb3d560 con 0x55f11fc54a70 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.158+0000 7fcb52349b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11fb3d200 con 0x55f11fc53750 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.158+0000 7fcb52349b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11fa246a0 con 0x55f11fc540b0 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.158+0000 7fcb52349b80 10 client.0 _ll_register_callbacks cb 0x55f11fb46b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.158+0000 7fcb36ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 0x55f11fc5d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.159+0000 7fcb3ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc54a70 0x55f11fc64550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.159+0000 7fcb36ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 0x55f11fc5d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59638/0 (socket says 172.21.15.33:59638) 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.159+0000 7fcb36ffd700 1 -- 172.21.15.33:0/3989920320 learned_addr learned my addr 172.21.15.33:0/3989920320 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.159+0000 7fcb3ffff700 1 -- 172.21.15.33:0/3989920320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 msgr2=0x55f11fc5d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.159+0000 7fcb3ffff700 1 --2- 172.21.15.33:0/3989920320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 0x55f11fc5d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.159+0000 7fcb3ffff700 1 -- 172.21.15.33:0/3989920320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc53750 msgr2=0x55f11fc5cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.159+0000 7fcb3ffff700 1 --2- 172.21.15.33:0/3989920320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc53750 0x55f11fc5cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.159+0000 7fcb3ffff700 1 -- 172.21.15.33:0/3989920320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb40108b20 con 0x55f11fc54a70 2024-05-08T21:22:16.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.159+0000 7fcb3ffff700 1 --2- 172.21.15.33:0/3989920320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc54a70 0x55f11fc64550 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcb38011e60 tx=0x7fcb38014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109d1656afd70b32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:16.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.160+0000 7fcb1effd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:22:16.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.160+0000 7fcb1effd700 1 -- 172.21.15.33:0/3989920320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800e040 con 0x55f11fc54a70 2024-05-08T21:22:16.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.160+0000 7fcb1effd700 1 -- 172.21.15.33:0/3989920320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcb380093a0 con 0x55f11fc54a70 2024-05-08T21:22:16.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.160+0000 7fcb52349b80 1 -- 172.21.15.33:0/3989920320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f11fafa270 con 0x55f11fc54a70 2024-05-08T21:22:16.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.160+0000 7fcb1effd700 1 -- 172.21.15.33:0/3989920320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb38004db0 con 0x55f11fc54a70 2024-05-08T21:22:16.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.160+0000 7fcb52349b80 10 client.16491 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:16.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.160+0000 7fcb52349b80 1 -- 172.21.15.33:0/3989920320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f11f9ceb60 con 0x55f11fc54a70 2024-05-08T21:22:16.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.160+0000 7fcb1effd700 1 -- 172.21.15.33:0/3989920320 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f11fafa270 con 0x55f11fc54a70 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.161+0000 7fcb1effd700 1 -- 172.21.15.33:0/3989920320 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fcb3801c030 con 0x55f11fc54a70 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.161+0000 7fcb52349b80 10 client.16491 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.161+0000 7fcb52349b80 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.161+0000 7fcb52349b80 20 client.16491 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.161+0000 7fcb52349b80 1 --2- 172.21.15.33:0/3989920320 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f11fc68fc0 0x55f11fc6b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.161+0000 7fcb52349b80 4 client.16491 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.161+0000 7fcb52349b80 1 -- 172.21.15.33:0/3989920320 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f11fc20810 con 0x55f11fc68fc0 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.161+0000 7fcb3f7fe700 1 --2- 172.21.15.33:0/3989920320 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f11fc68fc0 0x55f11fc6b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.161+0000 7fcb1effd700 10 client.16491 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.161+0000 7fcb3f7fe700 1 --2- 172.21.15.33:0/3989920320 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f11fc68fc0 0x55f11fc6b3e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9977747dfd268f62 server_cookie=7c1743e6c67a9d4e in_seq=0 out_seq=0 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb1effd700 1 -- 172.21.15.33:0/3989920320 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f11fc20810 con 0x55f11fc68fc0 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb1effd700 10 client.16491 handle_command_reply: tid=0 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb52349b80 1 client.16491 shutdown 2024-05-08T21:22:16.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb52349b80 1 -- 172.21.15.33:0/3989920320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc54a70 msgr2=0x55f11fc64550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:16.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb52349b80 1 --2- 172.21.15.33:0/3989920320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc54a70 0x55f11fc64550 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcb38011e60 tx=0x7fcb38014600 comp rx=0 tx=0).stop 2024-05-08T21:22:16.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb52349b80 1 -- 172.21.15.33:0/3989920320 shutdown_connections 2024-05-08T21:22:16.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb52349b80 1 --2- 172.21.15.33:0/3989920320 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f11fc68fc0 0x55f11fc6b3e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb52349b80 1 --2- 172.21.15.33:0/3989920320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc54a70 0x55f11fc64550 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb52349b80 1 --2- 172.21.15.33:0/3989920320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc53750 0x55f11fc5cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb1effd700 0 client.16491 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:16.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb52349b80 1 --2- 172.21.15.33:0/3989920320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 0x55f11fc5d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb52349b80 1 -- 172.21.15.33:0/3989920320 >> 172.21.15.33:0/3989920320 conn(0x55f11fbaf210 msgr2=0x55f11fc4b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:16.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb52349b80 1 -- 172.21.15.33:0/3989920320 shutdown_connections 2024-05-08T21:22:16.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb52349b80 1 -- 172.21.15.33:0/3989920320 wait complete. 2024-05-08T21:22:16.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.162+0000 7fcb52349b80 20 client.16491 trim_cache size 0 max 16384 2024-05-08T21:22:16.171 INFO:tasks.workunit.client.0.smithi033.stdout:checking acl/libacl.h usability... yes 2024-05-08T21:22:16.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.178+0000 7fcb52349b80 1 Processor -- start 2024-05-08T21:22:16.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.178+0000 7fcb52349b80 1 -- start start 2024-05-08T21:22:16.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.178+0000 7fcb52349b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc53750 0x55f11fc64c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.178+0000 7fcb52349b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 0x55f11fc54e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.178+0000 7fcb52349b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc54a70 0x55f11fc5b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.178+0000 7fcb52349b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11fc5e1c0 con 0x55f11fc53750 2024-05-08T21:22:16.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.178+0000 7fcb52349b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11fc55390 con 0x55f11fc54a70 2024-05-08T21:22:16.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.178+0000 7fcb52349b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11fc5ad20 con 0x55f11fc540b0 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb36ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 0x55f11fc54e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb3f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc53750 0x55f11fc64c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb36ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 0x55f11fc54e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59650/0 (socket says 172.21.15.33:59650) 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb36ffd700 1 -- 172.21.15.33:0/1839969124 learned_addr learned my addr 172.21.15.33:0/1839969124 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb3ffff700 1 --2- 172.21.15.33:0/1839969124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc54a70 0x55f11fc5b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb36ffd700 1 -- 172.21.15.33:0/1839969124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc54a70 msgr2=0x55f11fc5b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb36ffd700 1 --2- 172.21.15.33:0/1839969124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc54a70 0x55f11fc5b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb36ffd700 1 -- 172.21.15.33:0/1839969124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc53750 msgr2=0x55f11fc64c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb36ffd700 1 --2- 172.21.15.33:0/1839969124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc53750 0x55f11fc64c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb36ffd700 1 -- 172.21.15.33:0/1839969124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f11fb67560 con 0x55f11fc540b0 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb36ffd700 1 --2- 172.21.15.33:0/1839969124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 0x55f11fc54e50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcb28005ae0 tx=0x7fcb2800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f35c757b28dbb251 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb367fc700 1 -- 172.21.15.33:0/1839969124 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28029040 con 0x55f11fc540b0 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.179+0000 7fcb367fc700 1 -- 172.21.15.33:0/1839969124 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb28003bf0 con 0x55f11fc540b0 2024-05-08T21:22:16.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.180+0000 7fcb367fc700 1 -- 172.21.15.33:0/1839969124 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800fb20 con 0x55f11fc540b0 2024-05-08T21:22:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.180+0000 7fcb52349b80 1 -- 172.21.15.33:0/1839969124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 msgr2=0x55f11fc54e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.180+0000 7fcb52349b80 1 --2- 172.21.15.33:0/1839969124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 0x55f11fc54e50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcb28005ae0 tx=0x7fcb2800b8d0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.180+0000 7fcb52349b80 1 -- 172.21.15.33:0/1839969124 shutdown_connections 2024-05-08T21:22:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.180+0000 7fcb52349b80 1 --2- 172.21.15.33:0/1839969124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc54a70 0x55f11fc5b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.180+0000 7fcb52349b80 1 --2- 172.21.15.33:0/1839969124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc53750 0x55f11fc64c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.180+0000 7fcb52349b80 1 --2- 172.21.15.33:0/1839969124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc540b0 0x55f11fc54e50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.180+0000 7fcb52349b80 1 -- 172.21.15.33:0/1839969124 >> 172.21.15.33:0/1839969124 conn(0x55f11fbaf210 msgr2=0x55f11fc48210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.180+0000 7fcb52349b80 1 -- 172.21.15.33:0/1839969124 shutdown_connections 2024-05-08T21:22:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.180+0000 7fcb52349b80 1 -- 172.21.15.33:0/1839969124 wait complete. 2024-05-08T21:22:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.181+0000 7fcb52349b80 1 -- start start 2024-05-08T21:22:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.181+0000 7fcb52349b80 1 Processor -- start 2024-05-08T21:22:16.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.181+0000 7fcb52349b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc53750 0x55f11fc500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.181+0000 7fcb52349b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc540b0 0x55f11fc50630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.181+0000 7fcb52349b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc54a70 0x55f11fc51640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.181+0000 7fcb52349b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11fc51b80 con 0x55f11fc54a70 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.181+0000 7fcb52349b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11fc51cf0 con 0x55f11fc540b0 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.181+0000 7fcb52349b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11fc51eb0 con 0x55f11fc53750 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.181+0000 7fcb52349b80 10 client.0 _ll_register_callbacks cb 0x55f11fb46b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb3f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc53750 0x55f11fc500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb36ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc540b0 0x55f11fc50630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb3f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc53750 0x55f11fc500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59654/0 (socket says 172.21.15.33:59654) 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb3ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc54a70 0x55f11fc51640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb3f7fe700 1 -- 172.21.15.33:0/1078288220 learned_addr learned my addr 172.21.15.33:0/1078288220 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb3f7fe700 1 -- 172.21.15.33:0/1078288220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc540b0 msgr2=0x55f11fc50630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb3f7fe700 1 --2- 172.21.15.33:0/1078288220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc540b0 0x55f11fc50630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb3f7fe700 1 -- 172.21.15.33:0/1078288220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc54a70 msgr2=0x55f11fc51640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:16.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb3f7fe700 1 --2- 172.21.15.33:0/1078288220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc54a70 0x55f11fc51640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb3f7fe700 1 -- 172.21.15.33:0/1078288220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f11fa223b0 con 0x55f11fc53750 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb3f7fe700 1 --2- 172.21.15.33:0/1078288220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc53750 0x55f11fc500f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb30099c40 tx=0x7fcb30005ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58d3981af4012a84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb1effd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb1effd700 1 -- 172.21.15.33:0/1078288220 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30099f40 con 0x55f11fc53750 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb52349b80 1 -- 172.21.15.33:0/1078288220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb2801e040 con 0x55f11fc53750 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb1effd700 1 -- 172.21.15.33:0/1078288220 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcb3005fe40 con 0x55f11fc53750 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb1effd700 1 -- 172.21.15.33:0/1078288220 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb3009dd70 con 0x55f11fc53750 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb52349b80 10 client.26389 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.182+0000 7fcb52349b80 1 -- 172.21.15.33:0/1078288220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f11fa26cd0 con 0x55f11fc53750 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.183+0000 7fcb1effd700 1 -- 172.21.15.33:0/1078288220 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb2801e040 con 0x55f11fc53750 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.183+0000 7fcb1effd700 1 -- 172.21.15.33:0/1078288220 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fcb3009a1a0 con 0x55f11fc53750 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.183+0000 7fcb52349b80 10 client.26389 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.183+0000 7fcb52349b80 10 client.26389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:16.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.183+0000 7fcb52349b80 20 client.26389 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.183+0000 7fcb52349b80 1 --2- 172.21.15.33:0/1078288220 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f11fcbd880 0x55f11fcbfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.183+0000 7fcb52349b80 4 client.26389 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.183+0000 7fcb52349b80 1 -- 172.21.15.33:0/1078288220 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f11fb99ec0 con 0x55f11fcbd880 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.183+0000 7fcb36ffd700 1 --2- 172.21.15.33:0/1078288220 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f11fcbd880 0x55f11fcbfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.183+0000 7fcb1effd700 10 client.26389 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.183+0000 7fcb36ffd700 1 --2- 172.21.15.33:0/1078288220 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f11fcbd880 0x55f11fcbfca0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60977b60efedb6b7 server_cookie=b0776b181e570c6 in_seq=0 out_seq=0 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb1effd700 1 -- 172.21.15.33:0/1078288220 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f11fb99ec0 con 0x55f11fcbd880 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb1effd700 10 client.26389 handle_command_reply: tid=0 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb52349b80 1 client.26389 shutdown 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb52349b80 1 -- 172.21.15.33:0/1078288220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc53750 msgr2=0x55f11fc500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb52349b80 1 --2- 172.21.15.33:0/1078288220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc53750 0x55f11fc500f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb30099c40 tx=0x7fcb30005ba0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb52349b80 1 -- 172.21.15.33:0/1078288220 shutdown_connections 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb52349b80 1 --2- 172.21.15.33:0/1078288220 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f11fcbd880 0x55f11fcbfca0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb52349b80 1 --2- 172.21.15.33:0/1078288220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f11fc54a70 0x55f11fc51640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb52349b80 1 --2- 172.21.15.33:0/1078288220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f11fc53750 0x55f11fc500f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb1effd700 0 client.26389 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:16.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb52349b80 1 --2- 172.21.15.33:0/1078288220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f11fc540b0 0x55f11fc50630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb52349b80 1 -- 172.21.15.33:0/1078288220 >> 172.21.15.33:0/1078288220 conn(0x55f11fbaf210 msgr2=0x55f11fc5c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:16.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb52349b80 1 -- 172.21.15.33:0/1078288220 shutdown_connections 2024-05-08T21:22:16.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb52349b80 1 -- 172.21.15.33:0/1078288220 wait complete. 2024-05-08T21:22:16.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.184+0000 7fcb52349b80 20 client.26389 trim_cache size 0 max 16384 2024-05-08T21:22:16.191 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:22:16.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.185+0000 7fcb4618f700 1 -- 172.21.15.33:0/2117168727 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcb2c0a1120 msgr2=0x7fcb2c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:16.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.185+0000 7fcb4618f700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcb2c0a1120 0x7fcb2c0a35e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800b580 tx=0x7fcb28003680 comp rx=0 tx=0).stop 2024-05-08T21:22:16.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.185+0000 7fcb4618f700 1 -- 172.21.15.33:0/2117168727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb400ffd90 msgr2=0x7fcb40195bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:16.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.185+0000 7fcb4618f700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb400ffd90 0x7fcb40195bf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000e9c0 tx=0x7fcb3000ecd0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.185+0000 7fcb4618f700 1 -- 172.21.15.33:0/2117168727 shutdown_connections 2024-05-08T21:22:16.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.185+0000 7fcb4618f700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb40101920 0x7fcb4019d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.185+0000 7fcb4618f700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcb2c0a1120 0x7fcb2c0a35e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.185+0000 7fcb4618f700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb400ffd90 0x7fcb40195bf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.185+0000 7fcb4618f700 1 --2- 172.21.15.33:0/2117168727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb40100f80 0x7fcb40196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:16.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.185+0000 7fcb4618f700 1 -- 172.21.15.33:0/2117168727 >> 172.21.15.33:0/2117168727 conn(0x7fcb400fb300 msgr2=0x7fcb401043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:16.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.185+0000 7fcb4618f700 1 -- 172.21.15.33:0/2117168727 shutdown_connections 2024-05-08T21:22:16.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:16.185+0000 7fcb4618f700 1 -- 172.21.15.33:0/2117168727 wait complete. 2024-05-08T21:22:16.201 INFO:tasks.workunit.client.0.smithi033.stdout:checking acl/libacl.h presence... yes 2024-05-08T21:22:16.202 INFO:tasks.workunit.client.0.smithi033.stdout:checking for acl/libacl.h... yes 2024-05-08T21:22:16.280 INFO:tasks.workunit.client.0.smithi033.stdout:checking for acl_init in -lacl... yes 2024-05-08T21:22:16.343 INFO:tasks.workunit.client.0.smithi033.stdout:checking gdbm-ndbm.h usability... no 2024-05-08T21:22:16.391 INFO:tasks.workunit.client.0.smithi033.stdout:checking gdbm-ndbm.h presence... no 2024-05-08T21:22:16.392 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gdbm-ndbm.h... no 2024-05-08T21:22:16.481 INFO:tasks.workunit.client.0.smithi033.stdout:checking gdbm/ndbm.h usability... no 2024-05-08T21:22:16.527 INFO:tasks.workunit.client.0.smithi033.stdout:checking gdbm/ndbm.h presence... no 2024-05-08T21:22:16.527 INFO:tasks.workunit.client.0.smithi033.stdout:checking for gdbm/ndbm.h... no 2024-05-08T21:22:16.569 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:22:16.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:22:16.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:16.589 INFO:tasks.workunit.client.0.smithi033.stdout:checking ndbm.h usability... no 2024-05-08T21:22:16.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:22:16.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:22:16.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:16.636 INFO:tasks.workunit.client.0.smithi033.stdout:checking ndbm.h presence... no 2024-05-08T21:22:16.636 INFO:tasks.workunit.client.0.smithi033.stdout:checking for ndbm.h... no 2024-05-08T21:22:16.689 INFO:tasks.workunit.client.0.smithi033.stdout:checking libaio.h usability... yes 2024-05-08T21:22:16.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:22:16.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:22:16.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:22:16.719 INFO:tasks.workunit.client.0.smithi033.stdout:checking libaio.h presence... yes 2024-05-08T21:22:16.719 INFO:tasks.workunit.client.0.smithi033.stdout:checking for libaio.h... yes 2024-05-08T21:22:16.809 INFO:tasks.workunit.client.0.smithi033.stdout:checking for dm_make_handle in -ldm... no 2024-05-08T21:22:16.810 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T21:22:16.810 INFO:tasks.workunit.client.0.smithi033.stdout:WARNING: could not find a valid DMAPI base library. 2024-05-08T21:22:16.810 INFO:tasks.workunit.client.0.smithi033.stdout:If you want DMAPI support please install the data migration 2024-05-08T21:22:16.810 INFO:tasks.workunit.client.0.smithi033.stdout:API (dmapi) library package. Alternatively, run "make install" 2024-05-08T21:22:16.810 INFO:tasks.workunit.client.0.smithi033.stdout:from the dmapi source. 2024-05-08T21:22:16.810 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-05-08T21:22:16.861 INFO:tasks.workunit.client.0.smithi033.stdout:checking linux/fiemap.h usability... yes 2024-05-08T21:22:16.886 INFO:tasks.workunit.client.0.smithi033.stdout:checking linux/fiemap.h presence... yes 2024-05-08T21:22:16.887 INFO:tasks.workunit.client.0.smithi033.stdout:checking for linux/fiemap.h... yes 2024-05-08T21:22:16.969 INFO:tasks.workunit.client.0.smithi033.stdout:checking for fallocate... yes 2024-05-08T21:22:17.049 INFO:tasks.workunit.client.0.smithi033.stdout:checking for open_by_handle_at... yes 2024-05-08T21:22:17.108 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/prctl.h usability... yes 2024-05-08T21:22:17.132 INFO:tasks.workunit.client.0.smithi033.stdout:checking sys/prctl.h presence... yes 2024-05-08T21:22:17.133 INFO:tasks.workunit.client.0.smithi033.stdout:checking for sys/prctl.h... yes 2024-05-08T21:22:17.194 INFO:tasks.workunit.client.0.smithi033.stdout:checking linux/fs.h usability... yes 2024-05-08T21:22:17.203 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:22:17.203 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:22:17.221 INFO:tasks.workunit.client.0.smithi033.stdout:checking linux/fs.h presence... yes 2024-05-08T21:22:17.222 INFO:tasks.workunit.client.0.smithi033.stdout:checking for linux/fs.h... yes 2024-05-08T21:22:17.294 INFO:tasks.workunit.client.0.smithi033.stdout:checking btrfsutil.h usability... no 2024-05-08T21:22:17.338 INFO:tasks.workunit.client.0.smithi033.stdout:checking btrfsutil.h presence... no 2024-05-08T21:22:17.339 INFO:tasks.workunit.client.0.smithi033.stdout:checking for btrfsutil.h... no 2024-05-08T21:22:17.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.361+0000 7f8e896c9700 1 -- 172.21.15.33:0/1755605512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e840fef30 msgr2=0x7f8e840ff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.361+0000 7f8e896c9700 1 --2- 172.21.15.33:0/1755605512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e840fef30 0x7f8e840ff390 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009990 tx=0x7f8e78009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.361+0000 7f8e896c9700 1 -- 172.21.15.33:0/1755605512 shutdown_connections 2024-05-08T21:22:17.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.361+0000 7f8e896c9700 1 --2- 172.21.15.33:0/1755605512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e840ff8d0 0x7f8e840ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.361+0000 7f8e896c9700 1 --2- 172.21.15.33:0/1755605512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e841013b0 0x7f8e841017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.361+0000 7f8e896c9700 1 --2- 172.21.15.33:0/1755605512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e840fef30 0x7f8e840ff390 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.361+0000 7f8e896c9700 1 -- 172.21.15.33:0/1755605512 >> 172.21.15.33:0/1755605512 conn(0x7f8e840faa90 msgr2=0x7f8e840fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:17.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.361+0000 7f8e896c9700 1 -- 172.21.15.33:0/1755605512 shutdown_connections 2024-05-08T21:22:17.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.362+0000 7f8e896c9700 1 -- 172.21.15.33:0/1755605512 wait complete. 2024-05-08T21:22:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.362+0000 7f8e896c9700 1 Processor -- start 2024-05-08T21:22:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e896c9700 1 -- start start 2024-05-08T21:22:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e896c9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e840fef30 0x7f8e84198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e896c9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e840ff8d0 0x7f8e84199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e896c9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e841013b0 0x7f8e841a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e896c9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e841a0a20 con 0x7f8e840ff8d0 2024-05-08T21:22:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e896c9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e841a0b60 con 0x7f8e840fef30 2024-05-08T21:22:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e896c9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e841a0cf0 con 0x7f8e841013b0 2024-05-08T21:22:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e827fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e840ff8d0 0x7f8e84199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e827fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e840ff8d0 0x7f8e84199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47672/0 (socket says 172.21.15.33:47672) 2024-05-08T21:22:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e827fc700 1 -- 172.21.15.33:0/4216931537 learned_addr learned my addr 172.21.15.33:0/4216931537 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e837fe700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e841013b0 0x7f8e841a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e82ffd700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e840fef30 0x7f8e84198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e827fc700 1 -- 172.21.15.33:0/4216931537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e841013b0 msgr2=0x7f8e841a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e827fc700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e841013b0 0x7f8e841a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e827fc700 1 -- 172.21.15.33:0/4216931537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e840fef30 msgr2=0x7f8e84198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e827fc700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e840fef30 0x7f8e84198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e827fc700 1 -- 172.21.15.33:0/4216931537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e74009520 con 0x7f8e840ff8d0 2024-05-08T21:22:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e837fe700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e841013b0 0x7f8e841a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.363+0000 7f8e82ffd700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e840fef30 0x7f8e84198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:22:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.364+0000 7f8e827fc700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e840ff8d0 0x7f8e84199430 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8e78003ec0 tx=0x7f8e78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.364+0000 7f8e6bfff700 1 -- 172.21.15.33:0/4216931537 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7801d070 con 0x7f8e840ff8d0 2024-05-08T21:22:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.364+0000 7f8e896c9700 1 -- 172.21.15.33:0/4216931537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e780095f0 con 0x7f8e840ff8d0 2024-05-08T21:22:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.364+0000 7f8e6bfff700 1 -- 172.21.15.33:0/4216931537 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e7800bdb0 con 0x7f8e840ff8d0 2024-05-08T21:22:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.364+0000 7f8e6bfff700 1 -- 172.21.15.33:0/4216931537 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800fda0 con 0x7f8e840ff8d0 2024-05-08T21:22:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.364+0000 7f8e896c9700 1 -- 172.21.15.33:0/4216931537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e841a11b0 con 0x7f8e840ff8d0 2024-05-08T21:22:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.366+0000 7f8e6bfff700 1 -- 172.21.15.33:0/4216931537 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8e7800f460 con 0x7f8e840ff8d0 2024-05-08T21:22:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.367+0000 7f8e6bfff700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8e6c0a1120 0x7f8e6c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.367+0000 7f8e82ffd700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8e6c0a1120 0x7f8e6c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.367+0000 7f8e6bfff700 1 -- 172.21.15.33:0/4216931537 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8e780cdf80 con 0x7f8e840ff8d0 2024-05-08T21:22:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.368+0000 7f8e82ffd700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8e6c0a1120 0x7f8e6c0a35e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8e84069440 tx=0x7f8e74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.368+0000 7f8e95883b80 1 Processor -- start 2024-05-08T21:22:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.368+0000 7f8e95883b80 1 -- start start 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e95883b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1545c0 0x55a84c1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e95883b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c154f20 0x55a84c1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e95883b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 0x55a84c155d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e95883b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c0ab1d0 con 0x55a84c154f20 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e95883b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c0a8b90 con 0x55a84c1545c0 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e95883b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c0a4480 con 0x55a84c1558e0 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e827fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c154f20 0x55a84c1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e837fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 0x55a84c155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e82ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1545c0 0x55a84c1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e827fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c154f20 0x55a84c1553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47674/0 (socket says 172.21.15.33:47674) 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e837fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 0x55a84c155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59678/0 (socket says 172.21.15.33:59678) 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e82ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1545c0 0x55a84c1549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44494/0 (socket says 172.21.15.33:44494) 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e827fc700 1 -- 172.21.15.33:0/2936352218 learned_addr learned my addr 172.21.15.33:0/2936352218 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e837fe700 1 -- 172.21.15.33:0/2936352218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1545c0 msgr2=0x55a84c1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e837fe700 1 --2- 172.21.15.33:0/2936352218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1545c0 0x55a84c1549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e837fe700 1 -- 172.21.15.33:0/2936352218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c154f20 msgr2=0x55a84c1553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e837fe700 1 --2- 172.21.15.33:0/2936352218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c154f20 0x55a84c1553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e837fe700 1 -- 172.21.15.33:0/2936352218 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a84bf25430 con 0x55a84c1558e0 2024-05-08T21:22:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e827fc700 1 --2- 172.21.15.33:0/2936352218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c154f20 0x55a84c1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e837fe700 1 --2- 172.21.15.33:0/2936352218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 0x55a84c155d60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c003de0 tx=0x7f8e7c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b40e7a3b74c4f6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.369+0000 7f8e697fa700 1 -- 172.21.15.33:0/2936352218 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004bb0 con 0x55a84c1558e0 2024-05-08T21:22:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.370+0000 7f8e697fa700 1 -- 172.21.15.33:0/2936352218 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e7c00f040 con 0x55a84c1558e0 2024-05-08T21:22:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.370+0000 7f8e697fa700 1 -- 172.21.15.33:0/2936352218 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c007bf0 con 0x55a84c1558e0 2024-05-08T21:22:17.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.370+0000 7f8e95883b80 1 -- 172.21.15.33:0/2936352218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 msgr2=0x55a84c155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.370+0000 7f8e95883b80 1 --2- 172.21.15.33:0/2936352218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 0x55a84c155d60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c003de0 tx=0x7f8e7c009920 comp rx=0 tx=0).stop 2024-05-08T21:22:17.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.370+0000 7f8e95883b80 1 -- 172.21.15.33:0/2936352218 shutdown_connections 2024-05-08T21:22:17.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.370+0000 7f8e95883b80 1 --2- 172.21.15.33:0/2936352218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 0x55a84c155d60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.370+0000 7f8e95883b80 1 --2- 172.21.15.33:0/2936352218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1545c0 0x55a84c1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.370+0000 7f8e95883b80 1 --2- 172.21.15.33:0/2936352218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c154f20 0x55a84c1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.370+0000 7f8e95883b80 1 -- 172.21.15.33:0/2936352218 >> 172.21.15.33:0/2936352218 conn(0x55a84c0b2290 msgr2=0x55a84c14ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.370+0000 7f8e95883b80 1 -- 172.21.15.33:0/2936352218 shutdown_connections 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.370+0000 7f8e95883b80 1 -- 172.21.15.33:0/2936352218 wait complete. 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.371+0000 7f8e95883b80 1 -- start start 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.371+0000 7f8e95883b80 1 Processor -- start 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.371+0000 7f8e95883b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1545c0 0x55a84c15de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.371+0000 7f8e95883b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c154f20 0x55a84c15e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.371+0000 7f8e95883b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1558e0 0x55a84c1653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.371+0000 7f8e95883b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c0405e0 con 0x55a84c1558e0 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.371+0000 7f8e95883b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c040280 con 0x55a84c1545c0 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.371+0000 7f8e95883b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84bf27720 con 0x55a84c154f20 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e95883b80 10 client.0 _ll_register_callbacks cb 0x55a84c049bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1558e0 0x55a84c1653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1558e0 0x55a84c1653d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47686/0 (socket says 172.21.15.33:47686) 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e837fe700 1 -- 172.21.15.33:0/2537286163 learned_addr learned my addr 172.21.15.33:0/2537286163 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e82ffd700 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1545c0 0x55a84c15de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e827fc700 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c154f20 0x55a84c15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e82ffd700 1 -- 172.21.15.33:0/2537286163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c154f20 msgr2=0x55a84c15e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e82ffd700 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c154f20 0x55a84c15e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e82ffd700 1 -- 172.21.15.33:0/2537286163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1558e0 msgr2=0x55a84c1653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e82ffd700 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1558e0 0x55a84c1653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e82ffd700 1 -- 172.21.15.33:0/2537286163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e841a11b0 con 0x55a84c1545c0 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e837fe700 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1558e0 0x55a84c1653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e82ffd700 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1545c0 0x55a84c15de50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009fd0 tx=0x7f8e740047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fa72422adcebd3d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e66ff5700 1 -- 172.21.15.33:0/2537286163 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74020900 con 0x55a84c1545c0 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e66ff5700 1 -- 172.21.15.33:0/2537286163 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8e74018070 con 0x55a84c1545c0 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.372+0000 7f8e95883b80 1 -- 172.21.15.33:0/2537286163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a84bffd2f0 con 0x55a84c1545c0 2024-05-08T21:22:17.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.373+0000 7f8e66ff5700 1 -- 172.21.15.33:0/2537286163 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e74020a60 con 0x55a84c1545c0 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.373+0000 7f8e95883b80 10 client.26378 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.373+0000 7f8e95883b80 1 -- 172.21.15.33:0/2537286163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f8e7c003a60 con 0x55a84c1545c0 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.373+0000 7f8e66ff5700 1 -- 172.21.15.33:0/2537286163 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a84bffd2f0 con 0x55a84c1545c0 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.373+0000 7f8e66ff5700 1 -- 172.21.15.33:0/2537286163 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f8e74029c60 con 0x55a84c1545c0 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.373+0000 7f8e95883b80 10 client.26378 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.373+0000 7f8e95883b80 10 client.26378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.373+0000 7f8e95883b80 20 client.26378 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.373+0000 7f8e95883b80 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a84c169cc0 0x55a84c16c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.373+0000 7f8e95883b80 4 client.26378 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.373+0000 7f8e95883b80 1 -- 172.21.15.33:0/2537286163 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a84c123890 con 0x55a84c169cc0 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.373+0000 7f8e827fc700 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a84c169cc0 0x55a84c16c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.374+0000 7f8e66ff5700 10 client.26378 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.374+0000 7f8e827fc700 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a84c169cc0 0x55a84c16c0e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b032be65661bbbc4 server_cookie=fea6a17060cb135d in_seq=0 out_seq=0 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.374+0000 7f8e66ff5700 1 -- 172.21.15.33:0/2537286163 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a84c123890 con 0x55a84c169cc0 2024-05-08T21:22:17.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.374+0000 7f8e66ff5700 10 client.26378 handle_command_reply: tid=0 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.374+0000 7f8e95883b80 1 client.26378 shutdown 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.375+0000 7f8e95883b80 1 -- 172.21.15.33:0/2537286163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1545c0 msgr2=0x55a84c15de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.375+0000 7f8e95883b80 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1545c0 0x55a84c15de50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009fd0 tx=0x7f8e740047c0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.375+0000 7f8e95883b80 1 -- 172.21.15.33:0/2537286163 shutdown_connections 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.375+0000 7f8e95883b80 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a84c169cc0 0x55a84c16c0e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.375+0000 7f8e95883b80 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1558e0 0x55a84c1653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.375+0000 7f8e95883b80 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1545c0 0x55a84c15de50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.375+0000 7f8e95883b80 1 --2- 172.21.15.33:0/2537286163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c154f20 0x55a84c15e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.375+0000 7f8e66ff5700 0 client.26378 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.375+0000 7f8e95883b80 1 -- 172.21.15.33:0/2537286163 >> 172.21.15.33:0/2537286163 conn(0x55a84c0b2290 msgr2=0x55a84c14e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.375+0000 7f8e95883b80 1 -- 172.21.15.33:0/2537286163 shutdown_connections 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.375+0000 7f8e95883b80 1 -- 172.21.15.33:0/2537286163 wait complete. 2024-05-08T21:22:17.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.375+0000 7f8e95883b80 20 client.26378 trim_cache size 0 max 16384 2024-05-08T21:22:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e95883b80 1 Processor -- start 2024-05-08T21:22:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e95883b80 1 -- start start 2024-05-08T21:22:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e95883b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 0x55a84c15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e95883b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c154f20 0x55a84c165390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e95883b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 0x55a84c15e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e95883b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c166550 con 0x55a84c1545c0 2024-05-08T21:22:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e95883b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c1505f0 con 0x55a84c154f20 2024-05-08T21:22:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e95883b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c1712f0 con 0x55a84c1558e0 2024-05-08T21:22:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e82ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 0x55a84c15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e82ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 0x55a84c15dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47700/0 (socket says 172.21.15.33:47700) 2024-05-08T21:22:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e82ffd700 1 -- 172.21.15.33:0/1275872298 learned_addr learned my addr 172.21.15.33:0/1275872298 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e837fe700 1 --2- 172.21.15.33:0/1275872298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 0x55a84c15e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e827fc700 1 --2- 172.21.15.33:0/1275872298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c154f20 0x55a84c165390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e837fe700 1 -- 172.21.15.33:0/1275872298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c154f20 msgr2=0x55a84c165390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e837fe700 1 --2- 172.21.15.33:0/1275872298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c154f20 0x55a84c165390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e837fe700 1 -- 172.21.15.33:0/1275872298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 msgr2=0x55a84c15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e837fe700 1 --2- 172.21.15.33:0/1275872298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 0x55a84c15dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e837fe700 1 -- 172.21.15.33:0/1275872298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a84c14b560 con 0x55a84c1558e0 2024-05-08T21:22:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.397+0000 7f8e82ffd700 1 --2- 172.21.15.33:0/1275872298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 0x55a84c15dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.398+0000 7f8e837fe700 1 --2- 172.21.15.33:0/1275872298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 0x55a84c15e140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009b30 tx=0x7f8e7c00afd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d4cb10fd6c71064 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.398+0000 7f8e69ffb700 1 -- 172.21.15.33:0/1275872298 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004ea0 con 0x55a84c1558e0 2024-05-08T21:22:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.398+0000 7f8e69ffb700 1 -- 172.21.15.33:0/1275872298 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e7c0090d0 con 0x55a84c1558e0 2024-05-08T21:22:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.398+0000 7f8e69ffb700 1 -- 172.21.15.33:0/1275872298 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c00e4f0 con 0x55a84c1558e0 2024-05-08T21:22:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.399+0000 7f8e95883b80 1 -- 172.21.15.33:0/1275872298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 msgr2=0x55a84c15e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.399+0000 7f8e95883b80 1 --2- 172.21.15.33:0/1275872298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 0x55a84c15e140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009b30 tx=0x7f8e7c00afd0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.399+0000 7f8e95883b80 1 -- 172.21.15.33:0/1275872298 shutdown_connections 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.399+0000 7f8e95883b80 1 --2- 172.21.15.33:0/1275872298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c1558e0 0x55a84c15e140 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.399+0000 7f8e95883b80 1 --2- 172.21.15.33:0/1275872298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 0x55a84c15dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.399+0000 7f8e95883b80 1 --2- 172.21.15.33:0/1275872298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c154f20 0x55a84c165390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.399+0000 7f8e95883b80 1 -- 172.21.15.33:0/1275872298 >> 172.21.15.33:0/1275872298 conn(0x55a84c0b2290 msgr2=0x55a84c15d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.399+0000 7f8e95883b80 1 -- 172.21.15.33:0/1275872298 shutdown_connections 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.399+0000 7f8e95883b80 1 -- 172.21.15.33:0/1275872298 wait complete. 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.399+0000 7f8e95883b80 1 -- start start 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.399+0000 7f8e95883b80 1 Processor -- start 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e95883b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 0x55a84c1651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e95883b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c154f20 0x55a84c161ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e95883b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1558e0 0x55a84c1621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e95883b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c162800 con 0x55a84c1545c0 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e95883b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c1c01b0 con 0x55a84c1558e0 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e95883b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84c1c02f0 con 0x55a84c154f20 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e95883b80 10 client.0 _ll_register_callbacks cb 0x55a84c049bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e82ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 0x55a84c1651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e827fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c154f20 0x55a84c161ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e82ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 0x55a84c1651b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47714/0 (socket says 172.21.15.33:47714) 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e82ffd700 1 -- 172.21.15.33:0/1022046771 learned_addr learned my addr 172.21.15.33:0/1022046771 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e837fe700 1 --2- 172.21.15.33:0/1022046771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1558e0 0x55a84c1621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e827fc700 1 -- 172.21.15.33:0/1022046771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1558e0 msgr2=0x55a84c1621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e827fc700 1 --2- 172.21.15.33:0/1022046771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1558e0 0x55a84c1621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e827fc700 1 -- 172.21.15.33:0/1022046771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 msgr2=0x55a84c1651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e827fc700 1 --2- 172.21.15.33:0/1022046771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 0x55a84c1651b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e827fc700 1 -- 172.21.15.33:0/1022046771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7c003a60 con 0x55a84c154f20 2024-05-08T21:22:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e827fc700 1 --2- 172.21.15.33:0/1022046771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c154f20 0x55a84c161ca0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8e78097b10 tx=0x7f8e78005ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dae130c2010ecf15 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e66ff5700 1 -- 172.21.15.33:0/1022046771 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7809eed0 con 0x55a84c154f20 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e66ff5700 1 -- 172.21.15.33:0/1022046771 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8e780a3490 con 0x55a84c154f20 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.400+0000 7f8e95883b80 1 -- 172.21.15.33:0/1022046771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e7c00f040 con 0x55a84c154f20 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.401+0000 7f8e66ff5700 1 -- 172.21.15.33:0/1022046771 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e780cb7d0 con 0x55a84c154f20 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.401+0000 7f8e95883b80 10 client.26407 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.401+0000 7f8e95883b80 1 -- 172.21.15.33:0/1022046771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a84bf25430 con 0x55a84c154f20 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.401+0000 7f8e66ff5700 1 -- 172.21.15.33:0/1022046771 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e7c00f040 con 0x55a84c154f20 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.401+0000 7f8e66ff5700 1 -- 172.21.15.33:0/1022046771 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f8e7809a590 con 0x55a84c154f20 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.401+0000 7f8e95883b80 10 client.26407 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.401+0000 7f8e95883b80 10 client.26407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.401+0000 7f8e95883b80 20 client.26407 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.401+0000 7f8e95883b80 1 --2- 172.21.15.33:0/1022046771 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a84c1c1a10 0x55a84c1c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.401+0000 7f8e95883b80 4 client.26407 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:22:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.401+0000 7f8e95883b80 1 -- 172.21.15.33:0/1022046771 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a84c09cf40 con 0x55a84c1c1a10 2024-05-08T21:22:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.401+0000 7f8e82ffd700 1 --2- 172.21.15.33:0/1022046771 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a84c1c1a10 0x55a84c1c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.402+0000 7f8e66ff5700 10 client.26407 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.402+0000 7f8e82ffd700 1 --2- 172.21.15.33:0/1022046771 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a84c1c1a10 0x55a84c1c3e30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe2530567e801354 server_cookie=f54ac68aab3a52ed in_seq=0 out_seq=0 2024-05-08T21:22:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.402+0000 7f8e66ff5700 1 -- 172.21.15.33:0/1022046771 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a84c09cf40 con 0x55a84c1c1a10 2024-05-08T21:22:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.402+0000 7f8e66ff5700 10 client.26407 handle_command_reply: tid=0 2024-05-08T21:22:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.402+0000 7f8e95883b80 1 client.26407 shutdown 2024-05-08T21:22:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.403+0000 7f8e95883b80 1 -- 172.21.15.33:0/1022046771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c154f20 msgr2=0x55a84c161ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.403+0000 7f8e95883b80 1 --2- 172.21.15.33:0/1022046771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c154f20 0x55a84c161ca0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8e78097b10 tx=0x7f8e78005ac0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.403+0000 7f8e95883b80 1 -- 172.21.15.33:0/1022046771 shutdown_connections 2024-05-08T21:22:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.403+0000 7f8e95883b80 1 --2- 172.21.15.33:0/1022046771 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a84c1c1a10 0x55a84c1c3e30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.403+0000 7f8e95883b80 1 --2- 172.21.15.33:0/1022046771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84c1558e0 0x55a84c1621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.403+0000 7f8e95883b80 1 --2- 172.21.15.33:0/1022046771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a84c1545c0 0x55a84c1651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.403+0000 7f8e66ff5700 0 client.26407 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.403+0000 7f8e95883b80 1 --2- 172.21.15.33:0/1022046771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a84c154f20 0x55a84c161ca0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.403+0000 7f8e95883b80 1 -- 172.21.15.33:0/1022046771 >> 172.21.15.33:0/1022046771 conn(0x55a84c0b2290 msgr2=0x55a84c15dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.403+0000 7f8e95883b80 1 -- 172.21.15.33:0/1022046771 shutdown_connections 2024-05-08T21:22:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.403+0000 7f8e95883b80 1 -- 172.21.15.33:0/1022046771 wait complete. 2024-05-08T21:22:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.403+0000 7f8e95883b80 20 client.26407 trim_cache size 0 max 16384 2024-05-08T21:22:17.410 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:22:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:22:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0ec3d6de-25b9-45c8-94d1-c7fdb51ff6de", 2024-05-08T21:22:17.410 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:22:17.410 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:22:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.404+0000 7f8e896c9700 1 -- 172.21.15.33:0/4216931537 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8e6c0a1120 msgr2=0x7f8e6c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.404+0000 7f8e896c9700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8e6c0a1120 0x7f8e6c0a35e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8e84069440 tx=0x7f8e74009260 comp rx=0 tx=0).stop 2024-05-08T21:22:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.404+0000 7f8e896c9700 1 -- 172.21.15.33:0/4216931537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e840ff8d0 msgr2=0x7f8e84199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.404+0000 7f8e896c9700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e840ff8d0 0x7f8e84199430 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8e78003ec0 tx=0x7f8e78004730 comp rx=0 tx=0).stop 2024-05-08T21:22:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.404+0000 7f8e896c9700 1 -- 172.21.15.33:0/4216931537 shutdown_connections 2024-05-08T21:22:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.404+0000 7f8e896c9700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e841013b0 0x7f8e841a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.404+0000 7f8e896c9700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8e6c0a1120 0x7f8e6c0a35e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.404+0000 7f8e896c9700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e840fef30 0x7f8e84198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.404+0000 7f8e896c9700 1 --2- 172.21.15.33:0/4216931537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e840ff8d0 0x7f8e84199430 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.404+0000 7f8e896c9700 1 -- 172.21.15.33:0/4216931537 >> 172.21.15.33:0/4216931537 conn(0x7f8e840faa90 msgr2=0x7f8e840fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.405+0000 7f8e896c9700 1 -- 172.21.15.33:0/4216931537 shutdown_connections 2024-05-08T21:22:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.405+0000 7f8e896c9700 1 -- 172.21.15.33:0/4216931537 wait complete. 2024-05-08T21:22:17.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:22:17.425 INFO:tasks.workunit.client.0.smithi033.stdout:checking for copy_file_range... yes 2024-05-08T21:22:17.487 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:17 smithi033 ceph-mon[42528]: pgmap v389: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1011 KiB/s wr, 135 op/s 2024-05-08T21:22:17.570 INFO:tasks.workunit.client.0.smithi033.stdout:checking for renameat2... yes 2024-05-08T21:22:17.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:17 smithi071 ceph-mon[47074]: pgmap v389: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1011 KiB/s wr, 135 op/s 2024-05-08T21:22:17.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:17 smithi082 ceph-mon[47512]: pgmap v389: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1011 KiB/s wr, 135 op/s 2024-05-08T21:22:17.616 INFO:tasks.workunit.client.0.smithi033.stdout:configure: creating ./config.status 2024-05-08T21:22:17.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.648+0000 7f4db33a2700 1 -- 172.21.15.33:0/2850836907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dac0fef90 msgr2=0x7f4dac0ff3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.648+0000 7f4db33a2700 1 --2- 172.21.15.33:0/2850836907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dac0fef90 0x7f4dac0ff3b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009990 tx=0x7f4da8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.648+0000 7f4db33a2700 1 -- 172.21.15.33:0/2850836907 shutdown_connections 2024-05-08T21:22:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.648+0000 7f4db33a2700 1 --2- 172.21.15.33:0/2850836907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dac102900 0x7f4dac1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.648+0000 7f4db33a2700 1 --2- 172.21.15.33:0/2850836907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dac0fef90 0x7f4dac0ff3b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.648+0000 7f4db33a2700 1 --2- 172.21.15.33:0/2850836907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dac101f60 0x7f4dac1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.648+0000 7f4db33a2700 1 -- 172.21.15.33:0/2850836907 >> 172.21.15.33:0/2850836907 conn(0x7f4dac0fa8c0 msgr2=0x7f4dac0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.648+0000 7f4db33a2700 1 -- 172.21.15.33:0/2850836907 shutdown_connections 2024-05-08T21:22:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.648+0000 7f4db33a2700 1 -- 172.21.15.33:0/2850836907 wait complete. 2024-05-08T21:22:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.649+0000 7f4db33a2700 1 Processor -- start 2024-05-08T21:22:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.649+0000 7f4db33a2700 1 -- start start 2024-05-08T21:22:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.649+0000 7f4db33a2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dac0fef90 0x7f4dac198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.649+0000 7f4db33a2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dac101f60 0x7f4dac199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.649+0000 7f4db33a2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dac102900 0x7f4dac1a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.649+0000 7f4db33a2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac1a0820 con 0x7f4dac101f60 2024-05-08T21:22:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.649+0000 7f4db33a2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac06a9f0 con 0x7f4dac102900 2024-05-08T21:22:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.649+0000 7f4db33a2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac06abb0 con 0x7f4dac0fef90 2024-05-08T21:22:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.649+0000 7f4db2ba1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dac102900 0x7f4dac1a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.649+0000 7f4db2ba1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dac102900 0x7f4dac1a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:44548/0 (socket says 172.21.15.33:44548) 2024-05-08T21:22:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4db2ba1700 1 -- 172.21.15.33:0/2422396628 learned_addr learned my addr 172.21.15.33:0/2422396628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.649+0000 7f4db1b9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dac101f60 0x7f4dac199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4db23a0700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dac0fef90 0x7f4dac198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4db2ba1700 1 -- 172.21.15.33:0/2422396628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dac0fef90 msgr2=0x7f4dac198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4db2ba1700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dac0fef90 0x7f4dac198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4db2ba1700 1 -- 172.21.15.33:0/2422396628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dac101f60 msgr2=0x7f4dac199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4db2ba1700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dac101f60 0x7f4dac199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4db2ba1700 1 -- 172.21.15.33:0/2422396628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da80095f0 con 0x7f4dac102900 2024-05-08T21:22:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4db2ba1700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dac102900 0x7f4dac1a0250 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4da400b970 tx=0x7f4da400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4da37fe700 1 -- 172.21.15.33:0/2422396628 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da400c930 con 0x7f4dac102900 2024-05-08T21:22:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4db33a2700 1 -- 172.21.15.33:0/2422396628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dac06ae90 con 0x7f4dac102900 2024-05-08T21:22:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4da37fe700 1 -- 172.21.15.33:0/2422396628 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4da4014440 con 0x7f4dac102900 2024-05-08T21:22:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4da37fe700 1 -- 172.21.15.33:0/2422396628 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da40075d0 con 0x7f4dac102900 2024-05-08T21:22:17.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.650+0000 7f4db33a2700 1 -- 172.21.15.33:0/2422396628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dac06b420 con 0x7f4dac102900 2024-05-08T21:22:17.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.654+0000 7f4da37fe700 1 -- 172.21.15.33:0/2422396628 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4da40145b0 con 0x7f4dac102900 2024-05-08T21:22:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.654+0000 7f4da37fe700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4d980a1240 0x7f4d980a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.655+0000 7f4da37fe700 1 -- 172.21.15.33:0/2422396628 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f4da4006e40 con 0x7f4dac102900 2024-05-08T21:22:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.655+0000 7f4db23a0700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4d980a1240 0x7f4d980a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.655+0000 7f4db23a0700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4d980a1240 0x7f4d980a3700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4da80098e0 tx=0x7f4da8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:17.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.656+0000 7f4dc07beb80 1 Processor -- start 2024-05-08T21:22:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.656+0000 7f4dc07beb80 1 -- start start 2024-05-08T21:22:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.656+0000 7f4dc07beb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269945250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.656+0000 7f4dc07beb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269945790 0x55d269945c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.656+0000 7f4dc07beb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269946220 0x55d2699466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.656+0000 7f4dc07beb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d269893150 con 0x55d269945790 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.656+0000 7f4dc07beb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d269890b10 con 0x55d269946220 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.656+0000 7f4dc07beb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26988c400 con 0x55d269944e30 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4db1b9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269945790 0x55d269945c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4db23a0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269945250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4db1b9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269945790 0x55d269945c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47750/0 (socket says 172.21.15.33:47750) 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4db1b9f700 1 -- 172.21.15.33:0/3614619665 learned_addr learned my addr 172.21.15.33:0/3614619665 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4db2ba1700 1 --2- 172.21.15.33:0/3614619665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269946220 0x55d2699466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4db23a0700 1 -- 172.21.15.33:0/3614619665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269946220 msgr2=0x55d2699466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4db23a0700 1 --2- 172.21.15.33:0/3614619665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269946220 0x55d2699466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4db23a0700 1 -- 172.21.15.33:0/3614619665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269945790 msgr2=0x55d269945c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4db23a0700 1 --2- 172.21.15.33:0/3614619665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269945790 0x55d269945c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4db23a0700 1 -- 172.21.15.33:0/3614619665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d26970d3b0 con 0x55d269944e30 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4db1b9f700 1 --2- 172.21.15.33:0/3614619665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269945790 0x55d269945c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4db23a0700 1 --2- 172.21.15.33:0/3614619665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269945250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0fcd80 tx=0x7f4da801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6609c9f2012ce6dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4da0ff9700 1 -- 172.21.15.33:0/3614619665 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8003900 con 0x55d269944e30 2024-05-08T21:22:17.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.657+0000 7f4da0ff9700 1 -- 172.21.15.33:0/3614619665 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4da801e040 con 0x55d269944e30 2024-05-08T21:22:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.658+0000 7f4da0ff9700 1 -- 172.21.15.33:0/3614619665 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8004760 con 0x55d269944e30 2024-05-08T21:22:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.658+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3614619665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 msgr2=0x55d269945250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.658+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/3614619665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269945250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0fcd80 tx=0x7f4da801f000 comp rx=0 tx=0).stop 2024-05-08T21:22:17.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.659+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3614619665 shutdown_connections 2024-05-08T21:22:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.659+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/3614619665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269946220 0x55d2699466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.659+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/3614619665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269945250 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.659+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/3614619665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269945790 0x55d269945c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.659+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3614619665 >> 172.21.15.33:0/3614619665 conn(0x55d26989a210 msgr2=0x55d269937000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.659+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3614619665 shutdown_connections 2024-05-08T21:22:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.660+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3614619665 wait complete. 2024-05-08T21:22:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.660+0000 7f4dc07beb80 1 -- start start 2024-05-08T21:22:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.660+0000 7f4dc07beb80 1 Processor -- start 2024-05-08T21:22:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.661+0000 7f4dc07beb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269944e30 0x55d26993dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.661+0000 7f4dc07beb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269945790 0x55d26993e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.661+0000 7f4dc07beb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269946220 0x55d2699431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.661+0000 7f4dc07beb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d269828560 con 0x55d269945790 2024-05-08T21:22:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.661+0000 7f4dc07beb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d269828200 con 0x55d269944e30 2024-05-08T21:22:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.661+0000 7f4dc07beb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26970f6a0 con 0x55d269946220 2024-05-08T21:22:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.661+0000 7f4dc07beb80 10 client.0 _ll_register_callbacks cb 0x55d269831b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.661+0000 7f4db2ba1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269946220 0x55d2699431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.661+0000 7f4db2ba1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269946220 0x55d2699431d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59750/0 (socket says 172.21.15.33:59750) 2024-05-08T21:22:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.661+0000 7f4db2ba1700 1 -- 172.21.15.33:0/1111460674 learned_addr learned my addr 172.21.15.33:0/1111460674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.661+0000 7f4db23a0700 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269944e30 0x55d26993dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.661+0000 7f4db1b9f700 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269945790 0x55d26993e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.662+0000 7f4db2ba1700 1 -- 172.21.15.33:0/1111460674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269944e30 msgr2=0x55d26993dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.662+0000 7f4db2ba1700 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269944e30 0x55d26993dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.662+0000 7f4db2ba1700 1 -- 172.21.15.33:0/1111460674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269945790 msgr2=0x55d26993e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.662+0000 7f4db2ba1700 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269945790 0x55d26993e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.662+0000 7f4db2ba1700 1 -- 172.21.15.33:0/1111460674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c009980 con 0x55d269946220 2024-05-08T21:22:17.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.662+0000 7f4db2ba1700 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269946220 0x55d2699431d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4da4099570 tx=0x7f4da4099b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19e103819fd6e5c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:17.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.662+0000 7f4d95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:17.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.662+0000 7f4d95ffb700 1 -- 172.21.15.33:0/1111460674 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da40995e0 con 0x55d269946220 2024-05-08T21:22:17.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.662+0000 7f4d95ffb700 1 -- 172.21.15.33:0/1111460674 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f4da409d410 con 0x55d269946220 2024-05-08T21:22:17.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.662+0000 7f4dc07beb80 1 -- 172.21.15.33:0/1111460674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2697e5270 con 0x55d269946220 2024-05-08T21:22:17.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.663+0000 7f4d95ffb700 1 -- 172.21.15.33:0/1111460674 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4da40a1a60 con 0x55d269946220 2024-05-08T21:22:17.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.663+0000 7f4dc07beb80 10 client.26416 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:17.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.663+0000 7f4dc07beb80 1 -- 172.21.15.33:0/1111460674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f4dac0ffd50 con 0x55d269946220 2024-05-08T21:22:17.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.663+0000 7f4d95ffb700 1 -- 172.21.15.33:0/1111460674 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d2697e5270 con 0x55d269946220 2024-05-08T21:22:17.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.663+0000 7f4d95ffb700 1 -- 172.21.15.33:0/1111460674 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f4da40840e0 con 0x55d269946220 2024-05-08T21:22:17.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.663+0000 7f4dc07beb80 10 client.26416 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:17.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.663+0000 7f4dc07beb80 10 client.26416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.663+0000 7f4dc07beb80 20 client.26416 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.663+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d269962df0 0x55d269965210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.663+0000 7f4dc07beb80 4 client.26416 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.663+0000 7f4dc07beb80 1 -- 172.21.15.33:0/1111460674 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d26990b810 con 0x55d269962df0 2024-05-08T21:22:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.663+0000 7f4db23a0700 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d269962df0 0x55d269965210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.664+0000 7f4db23a0700 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d269962df0 0x55d269965210 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9db2970a78ec75d6 server_cookie=46ef78e10c17f3f in_seq=0 out_seq=0 2024-05-08T21:22:17.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.664+0000 7f4d95ffb700 10 client.26416 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.665+0000 7f4d95ffb700 1 -- 172.21.15.33:0/1111460674 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d26990b810 con 0x55d269962df0 2024-05-08T21:22:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.665+0000 7f4d95ffb700 10 client.26416 handle_command_reply: tid=0 2024-05-08T21:22:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.665+0000 7f4dc07beb80 1 client.26416 shutdown 2024-05-08T21:22:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.666+0000 7f4dc07beb80 1 -- 172.21.15.33:0/1111460674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269946220 msgr2=0x55d2699431d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.666+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269946220 0x55d2699431d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4da4099570 tx=0x7f4da4099b20 comp rx=0 tx=0).stop 2024-05-08T21:22:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.666+0000 7f4dc07beb80 1 -- 172.21.15.33:0/1111460674 shutdown_connections 2024-05-08T21:22:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.666+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d269962df0 0x55d269965210 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.666+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269946220 0x55d2699431d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.666+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269944e30 0x55d26993dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.666+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/1111460674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269945790 0x55d26993e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.666+0000 7f4dc07beb80 1 -- 172.21.15.33:0/1111460674 >> 172.21.15.33:0/1111460674 conn(0x55d26989a210 msgr2=0x55d269948e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.666+0000 7f4d95ffb700 0 client.26416 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.666+0000 7f4dc07beb80 1 -- 172.21.15.33:0/1111460674 shutdown_connections 2024-05-08T21:22:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.666+0000 7f4dc07beb80 1 -- 172.21.15.33:0/1111460674 wait complete. 2024-05-08T21:22:17.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.667+0000 7f4dc07beb80 20 client.26416 trim_cache size 0 max 16384 2024-05-08T21:22:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.688+0000 7f4dc07beb80 1 Processor -- start 2024-05-08T21:22:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.688+0000 7f4dc07beb80 1 -- start start 2024-05-08T21:22:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.688+0000 7f4dc07beb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269946600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.688+0000 7f4dc07beb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269945790 0x55d269969b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.688+0000 7f4dc07beb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269946220 0x55d26994c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.688+0000 7f4dc07beb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26992f0d0 con 0x55d269946220 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.688+0000 7f4dc07beb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d269944350 con 0x55d269945790 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.688+0000 7f4dc07beb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d26993d7d0 con 0x55d269944e30 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db2ba1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269946220 0x55d26994c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db23a0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269946600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db2ba1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269946220 0x55d26994c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47762/0 (socket says 172.21.15.33:47762) 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db2ba1700 1 -- 172.21.15.33:0/228501451 learned_addr learned my addr 172.21.15.33:0/228501451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db1b9f700 1 --2- 172.21.15.33:0/228501451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269945790 0x55d269969b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db23a0700 1 -- 172.21.15.33:0/228501451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269945790 msgr2=0x55d269969b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db23a0700 1 --2- 172.21.15.33:0/228501451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269945790 0x55d269969b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db23a0700 1 -- 172.21.15.33:0/228501451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269946220 msgr2=0x55d26994c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db23a0700 1 --2- 172.21.15.33:0/228501451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269946220 0x55d26994c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db23a0700 1 -- 172.21.15.33:0/228501451 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2696b9b60 con 0x55d269944e30 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db2ba1700 1 --2- 172.21.15.33:0/228501451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269946220 0x55d26994c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db1b9f700 1 --2- 172.21.15.33:0/228501451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269945790 0x55d269969b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4db23a0700 1 --2- 172.21.15.33:0/228501451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269946600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009990 tx=0x7f4da8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd15892c2d6f421a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.689+0000 7f4da17fa700 1 -- 172.21.15.33:0/228501451 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da80059b0 con 0x55d269944e30 2024-05-08T21:22:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.690+0000 7f4da17fa700 1 -- 172.21.15.33:0/228501451 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4da8035040 con 0x55d269944e30 2024-05-08T21:22:17.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.690+0000 7f4da17fa700 1 -- 172.21.15.33:0/228501451 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8003960 con 0x55d269944e30 2024-05-08T21:22:17.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.690+0000 7f4dc07beb80 1 -- 172.21.15.33:0/228501451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 msgr2=0x55d269946600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.690+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/228501451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269946600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009990 tx=0x7f4da8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.690+0000 7f4dc07beb80 1 -- 172.21.15.33:0/228501451 shutdown_connections 2024-05-08T21:22:17.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.690+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/228501451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269946220 0x55d26994c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.690+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/228501451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269946600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.690+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/228501451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269945790 0x55d269969b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.690+0000 7f4dc07beb80 1 -- 172.21.15.33:0/228501451 >> 172.21.15.33:0/228501451 conn(0x55d26989a210 msgr2=0x55d26993ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.690+0000 7f4dc07beb80 1 -- 172.21.15.33:0/228501451 shutdown_connections 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4dc07beb80 1 -- 172.21.15.33:0/228501451 wait complete. 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4dc07beb80 1 -- start start 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4dc07beb80 1 Processor -- start 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4dc07beb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269936ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4dc07beb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269945790 0x55d2699370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4dc07beb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269946220 0x55d2699b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4dc07beb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2699b7a30 con 0x55d269946220 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4dc07beb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2699b7ba0 con 0x55d269945790 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4dc07beb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2699b7d60 con 0x55d269944e30 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4dc07beb80 10 client.0 _ll_register_callbacks cb 0x55d269831b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:17.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4db23a0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269936ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4db23a0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269936ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:59774/0 (socket says 172.21.15.33:59774) 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.691+0000 7f4db23a0700 1 -- 172.21.15.33:0/3756555083 learned_addr learned my addr 172.21.15.33:0/3756555083 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4db1b9f700 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269945790 0x55d2699370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4db2ba1700 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269946220 0x55d2699b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4db23a0700 1 -- 172.21.15.33:0/3756555083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269945790 msgr2=0x55d2699370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4db23a0700 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269945790 0x55d2699370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4db23a0700 1 -- 172.21.15.33:0/3756555083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269946220 msgr2=0x55d2699b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4db23a0700 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269946220 0x55d2699b72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4db23a0700 1 -- 172.21.15.33:0/3756555083 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c009cc0 con 0x55d269944e30 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4db23a0700 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269936ba0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4da8003e60 tx=0x7f4da803b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eba360f8b7feced server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4d95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4d95ffb700 1 -- 172.21.15.33:0/3756555083 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8003890 con 0x55d269944e30 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4d95ffb700 1 -- 172.21.15.33:0/3756555083 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f4da800b740 con 0x55d269944e30 2024-05-08T21:22:17.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3756555083 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4da801e040 con 0x55d269944e30 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4d95ffb700 1 -- 172.21.15.33:0/3756555083 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4da80039f0 con 0x55d269944e30 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4dc07beb80 10 client.26422 fetch_fsmap learned FSMap version 32 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.692+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3756555083 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d26970d3b0 con 0x55d269944e30 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.693+0000 7f4d95ffb700 1 -- 172.21.15.33:0/3756555083 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4da801e040 con 0x55d269944e30 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.693+0000 7f4d95ffb700 1 -- 172.21.15.33:0/3756555083 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f4da804a030 con 0x55d269944e30 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.693+0000 7f4dc07beb80 10 client.26422 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.693+0000 7f4dc07beb80 10 client.26422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.693+0000 7f4dc07beb80 20 client.26422 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.693+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d2699b9430 0x55d2699bb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.693+0000 7f4dc07beb80 4 client.26422 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.693+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3756555083 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d269884ec0 con 0x55d2699b9430 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.693+0000 7f4db1b9f700 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d2699b9430 0x55d2699bb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.693+0000 7f4d95ffb700 10 client.26422 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.693+0000 7f4db1b9f700 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d2699b9430 0x55d2699bb850 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a2e4f1903e0bcb server_cookie=fa9aac1016c60177 in_seq=0 out_seq=0 2024-05-08T21:22:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4d95ffb700 1 -- 172.21.15.33:0/3756555083 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d269884ec0 con 0x55d2699b9430 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4d95ffb700 10 client.26422 handle_command_reply: tid=0 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4dc07beb80 1 client.26422 shutdown 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3756555083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 msgr2=0x55d269936ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269936ba0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4da8003e60 tx=0x7f4da803b430 comp rx=0 tx=0).stop 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3756555083 shutdown_connections 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d2699b9430 0x55d2699bb850 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d269946220 0x55d2699b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d269944e30 0x55d269936ba0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4dc07beb80 1 --2- 172.21.15.33:0/3756555083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d269945790 0x55d2699370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4d95ffb700 0 client.26422 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3756555083 >> 172.21.15.33:0/3756555083 conn(0x55d26989a210 msgr2=0x55d26993ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3756555083 shutdown_connections 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4dc07beb80 1 -- 172.21.15.33:0/3756555083 wait complete. 2024-05-08T21:22:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.694+0000 7f4dc07beb80 20 client.26422 trim_cache size 0 max 16384 2024-05-08T21:22:17.702 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:22:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:22:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:22:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: "0ec3d6de-25b9-45c8-94d1-c7fdb51ff6de": { 2024-05-08T21:22:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:22:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0ec3d6de-25b9-45c8-94d1-c7fdb51ff6de", 2024-05-08T21:22:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:22:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:22:17.702 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:22:17.702 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:22:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.695+0000 7f4db33a2700 1 -- 172.21.15.33:0/2422396628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4d980a1240 msgr2=0x7f4d980a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.695+0000 7f4db33a2700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4d980a1240 0x7f4d980a3700 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4da80098e0 tx=0x7f4da8005fd0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.695+0000 7f4db33a2700 1 -- 172.21.15.33:0/2422396628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dac102900 msgr2=0x7f4dac1a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.695+0000 7f4db33a2700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dac102900 0x7f4dac1a0250 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4da400b970 tx=0x7f4da400bc80 comp rx=0 tx=0).stop 2024-05-08T21:22:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.695+0000 7f4db33a2700 1 -- 172.21.15.33:0/2422396628 shutdown_connections 2024-05-08T21:22:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.695+0000 7f4db33a2700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f4d980a1240 0x7f4d980a3700 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.695+0000 7f4db33a2700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4dac102900 0x7f4dac1a0250 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.695+0000 7f4db33a2700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dac0fef90 0x7f4dac198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.695+0000 7f4db33a2700 1 --2- 172.21.15.33:0/2422396628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dac101f60 0x7f4dac199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.695+0000 7f4db33a2700 1 -- 172.21.15.33:0/2422396628 >> 172.21.15.33:0/2422396628 conn(0x7f4dac0fa8c0 msgr2=0x7f4dac105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.695+0000 7f4db33a2700 1 -- 172.21.15.33:0/2422396628 shutdown_connections 2024-05-08T21:22:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:17.695+0000 7f4db33a2700 1 -- 172.21.15.33:0/2422396628 wait complete. 2024-05-08T21:22:17.711 INFO:tasks.cephfs.filesystem:scrub status for tag:0ec3d6de-25b9-45c8-94d1-c7fdb51ff6de - {'path': '/', 'tag': '0ec3d6de-25b9-45c8-94d1-c7fdb51ff6de', 'options': 'recursive,force'} 2024-05-08T21:22:18.149 INFO:tasks.workunit.client.0.smithi033.stdout:config.status: creating include/builddefs 2024-05-08T21:22:18.200 INFO:tasks.workunit.client.0.smithi033.stdout:config.status: creating include/config.h 2024-05-08T21:22:18.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:22:18.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:18.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:18.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:18.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:18.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:18.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:18.571 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:18 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:18.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:18 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:22:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:18.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:18.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:18.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:18.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:18 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:19.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:19 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:19.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:19 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:19.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:19 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:19.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:19 smithi071 ceph-mon[47074]: pgmap v390: 129 pgs: 129 active+clean; 180 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-05-08T21:22:19.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:19 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:22:19.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:19 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:19.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:19 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:19.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:19 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:19.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:19 smithi082 ceph-mon[47512]: pgmap v390: 129 pgs: 129 active+clean; 180 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-05-08T21:22:19.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:19 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:22:19.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:19 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:19.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:19 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:19.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:19 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:19.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:19 smithi033 ceph-mon[42528]: pgmap v390: 129 pgs: 129 active+clean; 180 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-05-08T21:22:19.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:19 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:22:21.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:21 smithi071 ceph-mon[47074]: pgmap v391: 129 pgs: 129 active+clean; 180 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 826 KiB/s wr, 131 op/s 2024-05-08T21:22:21.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:21 smithi082 ceph-mon[47512]: pgmap v391: 129 pgs: 129 active+clean; 180 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 826 KiB/s wr, 131 op/s 2024-05-08T21:22:21.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:21 smithi033 ceph-mon[42528]: pgmap v391: 129 pgs: 129 active+clean; 180 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 826 KiB/s wr, 131 op/s 2024-05-08T21:22:22.457 INFO:tasks.workunit.client.0.smithi033.stdout:config.status: executing libtool commands 2024-05-08T21:22:22.725 INFO:tasks.workunit.client.0.smithi033.stdout:Building include 2024-05-08T21:22:22.726 INFO:tasks.workunit.client.0.smithi033.stdout:Building lib 2024-05-08T21:22:22.726 INFO:tasks.workunit.client.0.smithi033.stdout:Building tests 2024-05-08T21:22:22.755 INFO:tasks.workunit.client.0.smithi033.stdout: [LTDEP] 2024-05-08T21:22:23.229 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] dataascii.lo 2024-05-08T21:22:23.229 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] databin.lo 2024-05-08T21:22:23.230 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] datapid.lo 2024-05-08T21:22:23.231 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] file_lock.lo 2024-05-08T21:22:23.516 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] forker.lo 2024-05-08T21:22:23.526 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] pattern.lo 2024-05-08T21:22:23.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:23 smithi071 ceph-mon[47074]: pgmap v392: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 613 KiB/s wr, 128 op/s 2024-05-08T21:22:23.600 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] open_flags.lo 2024-05-08T21:22:23.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:23 smithi082 ceph-mon[47512]: pgmap v392: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 613 KiB/s wr, 128 op/s 2024-05-08T21:22:23.627 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] random_range.lo 2024-05-08T21:22:23.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:23 smithi033 ceph-mon[42528]: pgmap v392: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 613 KiB/s wr, 128 op/s 2024-05-08T21:22:23.817 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] string_to_tokens.lo 2024-05-08T21:22:23.857 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] str_to_bytes.lo 2024-05-08T21:22:23.955 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] tlibio.lo 2024-05-08T21:22:24.065 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] write_log.lo 2024-05-08T21:22:24.091 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] random.lo 2024-05-08T21:22:24.600 INFO:tasks.workunit.client.0.smithi033.stdout: [LD] 2024-05-08T21:22:24.839 INFO:tasks.workunit.client.0.smithi033.stdout:Building ltp 2024-05-08T21:22:24.839 INFO:tasks.workunit.client.0.smithi033.stdout:Building src 2024-05-08T21:22:24.839 INFO:tasks.workunit.client.0.smithi033.stdout:Building m4 2024-05-08T21:22:24.840 INFO:tasks.workunit.client.0.smithi033.stdout:Building common 2024-05-08T21:22:24.876 INFO:tasks.workunit.client.0.smithi033.stdout: [DEP] 2024-05-08T21:22:24.911 INFO:tasks.workunit.client.0.smithi033.stdout: [DEP] 2024-05-08T21:22:25.569 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:25 smithi071 ceph-mon[47074]: pgmap v393: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 514 KiB/s wr, 116 op/s 2024-05-08T21:22:25.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:25 smithi082 ceph-mon[47512]: pgmap v393: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 514 KiB/s wr, 116 op/s 2024-05-08T21:22:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:25 smithi033 ceph-mon[42528]: pgmap v393: 129 pgs: 129 active+clean; 181 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 514 KiB/s wr, 116 op/s 2024-05-08T21:22:26.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:26 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:22:26.594 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] doio 2024-05-08T21:22:26.594 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] fsstress 2024-05-08T21:22:26.594 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] fsx 2024-05-08T21:22:26.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:26 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:22:26.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:26 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:22:27.072 INFO:tasks.workunit.client.0.smithi033.stderr:fsstress.c: In function 'attr_list_path': 2024-05-08T21:22:27.072 INFO:tasks.workunit.client.0.smithi033.stderr:fsstress.c:859:2: warning: 'attr_list' is deprecated: Use listxattr or llistxattr instead [-Wdeprecated-declarations] 2024-05-08T21:22:27.072 INFO:tasks.workunit.client.0.smithi033.stderr: rval = attr_list(name->path, buffer, buffersize, flags, cursor); 2024-05-08T21:22:27.072 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~ 2024-05-08T21:22:27.072 INFO:tasks.workunit.client.0.smithi033.stderr:In file included from ../src/global.h:25, 2024-05-08T21:22:27.072 INFO:tasks.workunit.client.0.smithi033.stderr: from fsstress.c:12: 2024-05-08T21:22:27.072 INFO:tasks.workunit.client.0.smithi033.stderr:/usr/include/attr/attributes.h:165:12: note: declared here 2024-05-08T21:22:27.072 INFO:tasks.workunit.client.0.smithi033.stderr: extern int attr_list(const char *__path, char *__buffer, const int __buffersize, 2024-05-08T21:22:27.072 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~ 2024-05-08T21:22:27.072 INFO:tasks.workunit.client.0.smithi033.stderr:fsstress.c: In function 'attr_remove_path': 2024-05-08T21:22:27.072 INFO:tasks.workunit.client.0.smithi033.stderr:fsstress.c:878:2: warning: 'attr_remove' is deprecated: Use removexattr or lremovexattr instead [-Wdeprecated-declarations] 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr: rval = attr_remove(name->path, attrname, flags); 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~ 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr:In file included from ../src/global.h:25, 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr: from fsstress.c:12: 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr:/usr/include/attr/attributes.h:152:12: note: declared here 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr: extern int attr_remove (const char *__path, const char *__attrname, 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~ 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr:fsstress.c: In function 'attr_set_path': 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr:fsstress.c:898:2: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr: rval = attr_set(name->path, attrname, attrvalue, valuelength, flags); 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~ 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr:In file included from ../src/global.h:25, 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr: from fsstress.c:12: 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr: extern int attr_set (const char *__path, const char *__attrname, 2024-05-08T21:22:27.073 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~ 2024-05-08T21:22:27.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:27 smithi071 ceph-mon[47074]: pgmap v394: 129 pgs: 129 active+clean; 182 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 508 KiB/s wr, 103 op/s 2024-05-08T21:22:27.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:27 smithi082 ceph-mon[47512]: pgmap v394: 129 pgs: 129 active+clean; 182 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 508 KiB/s wr, 103 op/s 2024-05-08T21:22:27.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:27 smithi033 ceph-mon[42528]: pgmap v394: 129 pgs: 129 active+clean; 182 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 508 KiB/s wr, 103 op/s 2024-05-08T21:22:27.736 INFO:tasks.workunit.client.0.smithi033.stderr:fsx.c: In function 'main': 2024-05-08T21:22:27.736 INFO:tasks.workunit.client.0.smithi033.stderr:fsx.c:2662:5: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-05-08T21:22:27.736 INFO:tasks.workunit.client.0.smithi033.stderr: strncpy(opsfile, optarg, sizeof(opsfile)); 2024-05-08T21:22:27.736 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-08T21:22:27.736 INFO:tasks.workunit.client.0.smithi033.stderr:fsx.c:2632:4: warning: 'strncpy' specified bound 1024 equals destination size [-Wstringop-truncation] 2024-05-08T21:22:27.736 INFO:tasks.workunit.client.0.smithi033.stderr: strncpy(dname, optarg, sizeof(dname)); 2024-05-08T21:22:27.736 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-08T21:22:27.737 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] growfiles 2024-05-08T21:22:27.904 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] iogen 2024-05-08T21:22:28.587 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-stress 2024-05-08T21:22:28.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:28 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:22:28.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:28 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:22:28.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:28 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:22:28.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:28 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:22:28.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:28 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:22:28.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:28 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:22:28.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:28 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:22:28.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:28 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:22:28.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:28 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:22:28.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:28 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:22:28.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:28 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:22:28.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:28 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:22:28.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:28 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:22:28.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:28 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:22:28.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:28 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:22:29.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:29 smithi082 ceph-mon[47512]: pgmap v395: 129 pgs: 129 active+clean; 182 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 517 KiB/s wr, 94 op/s 2024-05-08T21:22:29.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:29 smithi033 ceph-mon[42528]: pgmap v395: 129 pgs: 129 active+clean; 182 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 517 KiB/s wr, 94 op/s 2024-05-08T21:22:29.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:29 smithi071 ceph-mon[47074]: pgmap v395: 129 pgs: 129 active+clean; 182 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 517 KiB/s wr, 94 op/s 2024-05-08T21:22:30.377 INFO:tasks.workunit.client.0.smithi033.stdout: [DEP] 2024-05-08T21:22:30.378 INFO:tasks.workunit.client.0.smithi033.stdout: [DEP] 2024-05-08T21:22:30.383 INFO:tasks.workunit.client.0.smithi033.stdout: [DEP] 2024-05-08T21:22:31.209 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] dirstress 2024-05-08T21:22:31.229 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] fill2 2024-05-08T21:22:31.232 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] fill 2024-05-08T21:22:31.237 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] getpagesize 2024-05-08T21:22:31.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:31 smithi082 ceph-mon[47512]: pgmap v396: 129 pgs: 129 active+clean; 183 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 398 KiB/s wr, 88 op/s 2024-05-08T21:22:31.694 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] holes 2024-05-08T21:22:31.706 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] lstat64 2024-05-08T21:22:31.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:31 smithi033 ceph-mon[42528]: pgmap v396: 129 pgs: 129 active+clean; 183 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 398 KiB/s wr, 88 op/s 2024-05-08T21:22:31.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:31 smithi071 ceph-mon[47074]: pgmap v396: 129 pgs: 129 active+clean; 183 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 398 KiB/s wr, 88 op/s 2024-05-08T21:22:31.827 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] nametest 2024-05-08T21:22:31.859 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] permname 2024-05-08T21:22:32.166 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] randholes 2024-05-08T21:22:32.204 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] runas 2024-05-08T21:22:32.374 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] truncfile 2024-05-08T21:22:32.405 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] usemem 2024-05-08T21:22:32.692 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] mmapcat 2024-05-08T21:22:32.789 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] append_reader 2024-05-08T21:22:32.806 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] append_writer 2024-05-08T21:22:32.866 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] dirperf 2024-05-08T21:22:33.107 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] metaperf 2024-05-08T21:22:33.221 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] devzero 2024-05-08T21:22:33.241 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] feature 2024-05-08T21:22:33.406 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] alloc 2024-05-08T21:22:33.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:33 smithi082 ceph-mon[47512]: pgmap v397: 129 pgs: 129 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 391 KiB/s wr, 65 op/s 2024-05-08T21:22:33.675 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] fault 2024-05-08T21:22:33.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:33 smithi033 ceph-mon[42528]: pgmap v397: 129 pgs: 129 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 391 KiB/s wr, 65 op/s 2024-05-08T21:22:33.712 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] fstest 2024-05-08T21:22:33.785 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_access_root 2024-05-08T21:22:33.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:33 smithi071 ceph-mon[47074]: pgmap v397: 129 pgs: 129 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 391 KiB/s wr, 65 op/s 2024-05-08T21:22:33.931 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] godown 2024-05-08T21:22:34.145 INFO:tasks.workunit.client.0.smithi033.stderr:fstest.c: In function 'run_child': 2024-05-08T21:22:34.145 INFO:tasks.workunit.client.0.smithi033.stderr:fstest.c:95:20: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-05-08T21:22:34.145 INFO:tasks.workunit.client.0.smithi033.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2024-05-08T21:22:34.145 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~ 2024-05-08T21:22:34.145 INFO:tasks.workunit.client.0.smithi033.stderr:fstest.c:95:17: note: directive argument in the range [0, 2147483647] 2024-05-08T21:22:34.145 INFO:tasks.workunit.client.0.smithi033.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2024-05-08T21:22:34.145 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~ 2024-05-08T21:22:34.145 INFO:tasks.workunit.client.0.smithi033.stderr:fstest.c:95:2: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-05-08T21:22:34.146 INFO:tasks.workunit.client.0.smithi033.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2024-05-08T21:22:34.146 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-08T21:22:34.146 INFO:tasks.workunit.client.0.smithi033.stderr:fstest.c:166:20: warning: '/file' directive writing 5 bytes into a region of size between 1 and 1024 [-Wformat-overflow=] 2024-05-08T21:22:34.146 INFO:tasks.workunit.client.0.smithi033.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2024-05-08T21:22:34.146 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~ 2024-05-08T21:22:34.146 INFO:tasks.workunit.client.0.smithi033.stderr:fstest.c:166:17: note: directive argument in the range [0, 2147483647] 2024-05-08T21:22:34.146 INFO:tasks.workunit.client.0.smithi033.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2024-05-08T21:22:34.146 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~ 2024-05-08T21:22:34.146 INFO:tasks.workunit.client.0.smithi033.stderr:fstest.c:166:2: note: 'sprintf' output between 7 and 1039 bytes into a destination of size 1024 2024-05-08T21:22:34.146 INFO:tasks.workunit.client.0.smithi033.stderr: sprintf(fname, "%s/file%d", dir, fnum); 2024-05-08T21:22:34.146 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-08T21:22:34.147 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] resvtest 2024-05-08T21:22:34.201 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] writemod 2024-05-08T21:22:34.282 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] writev_on_pagefault 2024-05-08T21:22:34.401 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] makeextents 2024-05-08T21:22:34.611 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] itrash 2024-05-08T21:22:34.672 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] rename 2024-05-08T21:22:34.696 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] multi_open_unlink 2024-05-08T21:22:34.900 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] dmiperf 2024-05-08T21:22:35.036 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] unwritten_sync 2024-05-08T21:22:35.069 INFO:tasks.workunit.client.0.smithi033.stderr:multi_open_unlink.c: In function 'main': 2024-05-08T21:22:35.071 INFO:tasks.workunit.client.0.smithi033.stderr:multi_open_unlink.c:101:4: warning: 'attr_set' is deprecated: Use setxattr or lsetxattr instead [-Wdeprecated-declarations] 2024-05-08T21:22:35.071 INFO:tasks.workunit.client.0.smithi033.stderr: sts = attr_set(path, attrname, attrvalue, value_size, flags); 2024-05-08T21:22:35.071 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~ 2024-05-08T21:22:35.071 INFO:tasks.workunit.client.0.smithi033.stderr:In file included from multi_open_unlink.c:15: 2024-05-08T21:22:35.071 INFO:tasks.workunit.client.0.smithi033.stderr:/usr/include/attr/attributes.h:139:12: note: declared here 2024-05-08T21:22:35.071 INFO:tasks.workunit.client.0.smithi033.stderr: extern int attr_set (const char *__path, const char *__attrname, 2024-05-08T21:22:35.071 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~ 2024-05-08T21:22:35.074 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] genhashnames 2024-05-08T21:22:35.168 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_holes 2024-05-08T21:22:35.293 INFO:tasks.workunit.client.0.smithi033.stderr:dmiperf.c: In function 'mkfile': 2024-05-08T21:22:35.293 INFO:tasks.workunit.client.0.smithi033.stderr:dmiperf.c:192:2: warning: 'attr_setf' is deprecated: Use fsetxattr instead [-Wdeprecated-declarations] 2024-05-08T21:22:35.293 INFO:tasks.workunit.client.0.smithi033.stderr: if (attr_setf(fd, DMFATTRNAME, attr, DMFATTRLEN, ATTR_ROOT) < 0) { 2024-05-08T21:22:35.293 INFO:tasks.workunit.client.0.smithi033.stderr: ^~ 2024-05-08T21:22:35.293 INFO:tasks.workunit.client.0.smithi033.stderr:In file included from dmiperf.c:20: 2024-05-08T21:22:35.294 INFO:tasks.workunit.client.0.smithi033.stderr:/usr/include/attr/attributes.h:143:12: note: declared here 2024-05-08T21:22:35.294 INFO:tasks.workunit.client.0.smithi033.stderr: extern int attr_setf (int __fd, const char *__attrname, 2024-05-08T21:22:35.294 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~ 2024-05-08T21:22:35.430 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_mmap_writev 2024-05-08T21:22:35.530 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_truncate_cmtime 2024-05-08T21:22:35.532 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] dirhash_collide 2024-05-08T21:22:35.565 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_rename_overwrite 2024-05-08T21:22:35.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:35 smithi082 ceph-mon[47512]: pgmap v398: 129 pgs: 129 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 342 KiB/s wr, 58 op/s 2024-05-08T21:22:35.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:35 smithi033 ceph-mon[42528]: pgmap v398: 129 pgs: 129 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 342 KiB/s wr, 58 op/s 2024-05-08T21:22:35.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:35 smithi071 ceph-mon[47074]: pgmap v398: 129 pgs: 129 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 342 KiB/s wr, 58 op/s 2024-05-08T21:22:35.824 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] holetest 2024-05-08T21:22:35.969 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_truncate_self 2024-05-08T21:22:35.982 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_mmap_dio 2024-05-08T21:22:36.019 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] af_unix 2024-05-08T21:22:36.358 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_mmap_stale_pmd 2024-05-08T21:22:36.372 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_mmap_cow_race 2024-05-08T21:22:36.438 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_mmap_fallocate 2024-05-08T21:22:36.459 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] fsync-err 2024-05-08T21:22:36.796 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_mmap_write_ro 2024-05-08T21:22:36.841 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_ext4_dax_journal_corruption 2024-05-08T21:22:36.864 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_ext4_dax_inline_corruption 2024-05-08T21:22:36.945 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_ofd_locks 2024-05-08T21:22:37.220 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_locks_execve 2024-05-08T21:22:37.305 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_mmap_collision 2024-05-08T21:22:37.306 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] mmap-write-concurrent 2024-05-08T21:22:37.444 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] xfsctl 2024-05-08T21:22:37.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:37 smithi082 ceph-mon[47512]: pgmap v399: 129 pgs: 129 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 335 KiB/s wr, 64 op/s 2024-05-08T21:22:37.671 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] bstat 2024-05-08T21:22:37.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:37 smithi033 ceph-mon[42528]: pgmap v399: 129 pgs: 129 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 335 KiB/s wr, 64 op/s 2024-05-08T21:22:37.767 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_mtab 2024-05-08T21:22:37.805 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] getdevicesize 2024-05-08T21:22:37.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:37 smithi071 ceph-mon[47074]: pgmap v399: 129 pgs: 129 active+clean; 184 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 335 KiB/s wr, 64 op/s 2024-05-08T21:22:37.940 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] preallo_rw_pattern_reader 2024-05-08T21:22:38.201 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] preallo_rw_pattern_writer 2024-05-08T21:22:38.205 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] ftrunc 2024-05-08T21:22:38.240 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] trunc 2024-05-08T21:22:38.394 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] fs_perms 2024-05-08T21:22:38.669 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] testx 2024-05-08T21:22:38.707 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] looptest 2024-05-08T21:22:38.708 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] locktest 2024-05-08T21:22:38.840 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] unwritten_mmap 2024-05-08T21:22:39.053 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] bulkstat_unlink_test 2024-05-08T21:22:39.210 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] bulkstat_unlink_test_modified 2024-05-08T21:22:39.278 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_dir_offset 2024-05-08T21:22:39.386 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_futimens 2024-05-08T21:22:39.587 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_immutable 2024-05-08T21:22:39.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:39 smithi082 ceph-mon[47512]: pgmap v400: 129 pgs: 129 active+clean; 185 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 337 KiB/s wr, 68 op/s 2024-05-08T21:22:39.700 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] stale_handle 2024-05-08T21:22:39.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:39 smithi033 ceph-mon[42528]: pgmap v400: 129 pgs: 129 active+clean; 185 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 337 KiB/s wr, 68 op/s 2024-05-08T21:22:39.722 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] pwrite_mmap_blocked 2024-05-08T21:22:39.782 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_dir_offset2 2024-05-08T21:22:39.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:39 smithi071 ceph-mon[47074]: pgmap v400: 129 pgs: 129 active+clean; 185 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 337 KiB/s wr, 68 op/s 2024-05-08T21:22:40.256 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] seek_sanity_test 2024-05-08T21:22:40.258 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] seek_copy_test 2024-05-08T21:22:40.259 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_readdir_1 2024-05-08T21:22:40.700 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_readdir_2 2024-05-08T21:22:40.801 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] fsync-tester 2024-05-08T21:22:40.805 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] nsexec 2024-05-08T21:22:41.115 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] cloner 2024-05-08T21:22:41.155 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] renameat2 2024-05-08T21:22:41.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:41 smithi082 ceph-mon[47512]: pgmap v401: 129 pgs: 129 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 401 KiB/s wr, 91 op/s 2024-05-08T21:22:41.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:41 smithi033 ceph-mon[42528]: pgmap v401: 129 pgs: 129 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 401 KiB/s wr, 91 op/s 2024-05-08T21:22:41.798 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_getcwd 2024-05-08T21:22:41.798 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] e4compact 2024-05-08T21:22:41.799 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] test-nextquota 2024-05-08T21:22:41.802 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] punch-alternating 2024-05-08T21:22:41.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:41 smithi071 ceph-mon[47074]: pgmap v401: 129 pgs: 129 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 401 KiB/s wr, 91 op/s 2024-05-08T21:22:42.224 INFO:tasks.workunit.client.0.smithi033.stderr:t_getcwd.c: In function 'do_rename': 2024-05-08T21:22:42.224 INFO:tasks.workunit.client.0.smithi033.stderr:t_getcwd.c:40:2: warning: 'strncpy' specified bound 256 equals destination size [-Wstringop-truncation] 2024-05-08T21:22:42.224 INFO:tasks.workunit.client.0.smithi033.stderr: strncpy(c_name, prefix, BUF_SIZE); 2024-05-08T21:22:42.224 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-08T21:22:42.351 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] attr-list-by-handle-cursor-test 2024-05-08T21:22:42.414 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] listxattr 2024-05-08T21:22:42.415 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] dio-interleaved 2024-05-08T21:22:42.436 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_dir_type 2024-05-08T21:22:42.868 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] dio-invalidate-cache 2024-05-08T21:22:42.894 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] stat_test 2024-05-08T21:22:42.894 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_encrypted_d_revalidate 2024-05-08T21:22:42.927 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] attr_replace_test 2024-05-08T21:22:43.460 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] swapon 2024-05-08T21:22:43.460 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] mkswap 2024-05-08T21:22:43.461 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_attr_corruption 2024-05-08T21:22:43.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:43 smithi082 ceph-mon[47512]: pgmap v402: 129 pgs: 129 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 299 KiB/s wr, 95 op/s 2024-05-08T21:22:43.648 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_open_tmpfiles 2024-05-08T21:22:43.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:43 smithi033 ceph-mon[42528]: pgmap v402: 129 pgs: 129 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 299 KiB/s wr, 95 op/s 2024-05-08T21:22:43.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:43 smithi071 ceph-mon[47074]: pgmap v402: 129 pgs: 129 active+clean; 186 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 299 KiB/s wr, 95 op/s 2024-05-08T21:22:43.851 INFO:tasks.workunit.client.0.smithi033.stderr:swapon.c: In function 'main': 2024-05-08T21:22:43.851 INFO:tasks.workunit.client.0.smithi033.stderr:swapon.c:135:3: warning: 'p' may be used uninitialized in this function [-Wmaybe-uninitialized] 2024-05-08T21:22:43.851 INFO:tasks.workunit.client.0.smithi033.stderr: memcpy(p, buf, BUF_SIZE); 2024-05-08T21:22:43.851 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-08T21:22:43.914 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] fscrypt-crypt-util 2024-05-08T21:22:43.917 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] bulkstat_null_ocount 2024-05-08T21:22:43.928 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] splice-test 2024-05-08T21:22:44.146 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] fiemap-tester 2024-05-08T21:22:44.402 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] t_stripealign 2024-05-08T21:22:44.416 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] open_by_handle 2024-05-08T21:22:44.733 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] fssum 2024-05-08T21:22:44.782 INFO:tasks.workunit.client.0.smithi033.stdout:Building log-writes 2024-05-08T21:22:44.820 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] replay-log 2024-05-08T21:22:44.913 INFO:tasks.workunit.client.0.smithi033.stdout:Building perf 2024-05-08T21:22:44.941 INFO:tasks.workunit.client.0.smithi033.stdout:Building aio-dio-regress 2024-05-08T21:22:45.004 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-dio-append-write-fallocate-race 2024-05-08T21:22:45.004 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-dio-fcntl-race 2024-05-08T21:22:45.364 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-dio-subblock-eof-read 2024-05-08T21:22:45.370 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-dio-append-write-read-race 2024-05-08T21:22:45.399 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-dio-invalidate-readahead 2024-05-08T21:22:45.568 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-dio-hole-filling-race 2024-05-08T21:22:45.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:45 smithi033 ceph-mon[42528]: pgmap v403: 129 pgs: 129 active+clean; 187 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 327 KiB/s wr, 104 op/s 2024-05-08T21:22:45.731 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-05-08T21:22:45.782 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-last-ref-held-by-io 2024-05-08T21:22:45.801 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-dio-cycle-write 2024-05-08T21:22:45.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:45 smithi071 ceph-mon[47074]: pgmap v403: 129 pgs: 129 active+clean; 187 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 327 KiB/s wr, 104 op/s 2024-05-08T21:22:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:45 smithi082 ceph-mon[47512]: pgmap v403: 129 pgs: 129 active+clean; 187 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 327 KiB/s wr, 104 op/s 2024-05-08T21:22:45.944 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-dio-extend-stat 2024-05-08T21:22:46.082 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-05-08T21:22:46.188 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aiocp 2024-05-08T21:22:46.214 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-dio-invalidate-failure 2024-05-08T21:22:46.339 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aiodio_sparse2 2024-05-08T21:22:46.422 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-dio-cow-race 2024-05-08T21:22:46.602 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-dio-write-verify 2024-05-08T21:22:46.669 INFO:tasks.workunit.client.0.smithi033.stdout: [CC] aio-dio-eof-race 2024-05-08T21:22:46.726 INFO:tasks.workunit.client.0.smithi033.stderr:aiodio_sparse2.c: In function 'main': 2024-05-08T21:22:46.726 INFO:tasks.workunit.client.0.smithi033.stderr:aiodio_sparse2.c:404:2: warning: 'strncpy' specified bound 4096 equals destination size [-Wstringop-truncation] 2024-05-08T21:22:46.726 INFO:tasks.workunit.client.0.smithi033.stderr: strncpy(filename, argv[argc-1], PATH_MAX); 2024-05-08T21:22:46.726 INFO:tasks.workunit.client.0.smithi033.stderr: ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-05-08T21:22:47.020 INFO:tasks.workunit.client.0.smithi033.stderr:+ cd .. 2024-05-08T21:22:47.021 INFO:tasks.workunit.client.0.smithi033.stderr:+ cp xfstests-dev/ltp/fsx . 2024-05-08T21:22:47.028 INFO:tasks.workunit.client.0.smithi033.stderr:+ OPTIONS=-z 2024-05-08T21:22:47.028 INFO:tasks.workunit.client.0.smithi033.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-05-08T21:22:47.029 INFO:tasks.workunit.client.0.smithi033.stdout:Seed set to 1 2024-05-08T21:22:47.063 INFO:tasks.workunit.client.0.smithi033.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-08T21:22:47.064 INFO:tasks.workunit.client.0.smithi033.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-08T21:22:47.064 INFO:tasks.workunit.client.0.smithi033.stderr:main: filesystem does not support clone range, disabling! 2024-05-08T21:22:47.064 INFO:tasks.workunit.client.0.smithi033.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-08T21:22:47.080 INFO:tasks.workunit.client.0.smithi033.stdout:truncating to largest ever: 0x366e7 2024-05-08T21:22:47.118 INFO:tasks.workunit.client.0.smithi033.stdout:copying to largest ever: 0xead53 2024-05-08T21:22:47.172 INFO:tasks.workunit.client.0.smithi033.stdout:truncating to largest ever: 0xfc5c3 2024-05-08T21:22:47.283 INFO:tasks.workunit.client.0.smithi033.stdout:truncating to largest ever: 0xfca47 2024-05-08T21:22:47.641 INFO:tasks.workunit.client.0.smithi033.stdout:copying to largest ever: 0xff398 2024-05-08T21:22:47.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:47 smithi033 ceph-mon[42528]: pgmap v404: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 360 KiB/s wr, 114 op/s 2024-05-08T21:22:47.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:22:47.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:47 smithi071 ceph-mon[47074]: pgmap v404: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 360 KiB/s wr, 114 op/s 2024-05-08T21:22:47.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:47 smithi082 ceph-mon[47512]: pgmap v404: 129 pgs: 129 active+clean; 188 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 360 KiB/s wr, 114 op/s 2024-05-08T21:22:47.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.876+0000 7f21c1321700 1 -- 172.21.15.33:0/612521792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc101360 msgr2=0x7f21bc101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:47.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.876+0000 7f21c1321700 1 --2- 172.21.15.33:0/612521792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc101360 0x7f21bc101780 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f21ac009990 tx=0x7f21ac009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.876+0000 7f21c1321700 1 -- 172.21.15.33:0/612521792 shutdown_connections 2024-05-08T21:22:47.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.876+0000 7f21c1321700 1 --2- 172.21.15.33:0/612521792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21bc1009c0 0x7f21bc108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.876+0000 7f21c1321700 1 --2- 172.21.15.33:0/612521792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc101360 0x7f21bc101780 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.876+0000 7f21c1321700 1 --2- 172.21.15.33:0/612521792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21bc100000 0x7f21bc100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.876+0000 7f21c1321700 1 -- 172.21.15.33:0/612521792 >> 172.21.15.33:0/612521792 conn(0x7f21bc0faa50 msgr2=0x7f21bc0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:47.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.876+0000 7f21c1321700 1 -- 172.21.15.33:0/612521792 shutdown_connections 2024-05-08T21:22:47.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.876+0000 7f21c1321700 1 -- 172.21.15.33:0/612521792 wait complete. 2024-05-08T21:22:47.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.877+0000 7f21c1321700 1 Processor -- start 2024-05-08T21:22:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.877+0000 7f21c1321700 1 -- start start 2024-05-08T21:22:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.877+0000 7f21c1321700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21bc100000 0x7f21bc196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.877+0000 7f21c1321700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21bc1009c0 0x7f21bc197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.877+0000 7f21c1321700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc101360 0x7f21bc19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.877+0000 7f21c1321700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc19e8f0 con 0x7f21bc101360 2024-05-08T21:22:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.877+0000 7f21c1321700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc19ea30 con 0x7f21bc100000 2024-05-08T21:22:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21c1321700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc19ebc0 con 0x7f21bc1009c0 2024-05-08T21:22:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc101360 0x7f21bc19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21ba7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21bc1009c0 0x7f21bc197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21baffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21bc100000 0x7f21bc196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc101360 0x7f21bc19e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42428/0 (socket says 172.21.15.33:42428) 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21ba7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21bc1009c0 0x7f21bc197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:40120/0 (socket says 172.21.15.33:40120) 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21bb7fe700 1 -- 172.21.15.33:0/992152892 learned_addr learned my addr 172.21.15.33:0/992152892 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21baffd700 1 -- 172.21.15.33:0/992152892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21bc1009c0 msgr2=0x7f21bc197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21baffd700 1 --2- 172.21.15.33:0/992152892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21bc1009c0 0x7f21bc197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21baffd700 1 -- 172.21.15.33:0/992152892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc101360 msgr2=0x7f21bc19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21baffd700 1 --2- 172.21.15.33:0/992152892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc101360 0x7f21bc19e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21baffd700 1 -- 172.21.15.33:0/992152892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0095f0 con 0x7f21bc100000 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.878+0000 7f21baffd700 1 --2- 172.21.15.33:0/992152892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21bc100000 0x7f21bc196dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f21ac000c00 tx=0x7f21ac004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.879+0000 7f21a3fff700 1 -- 172.21.15.33:0/992152892 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac021070 con 0x7f21bc100000 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.879+0000 7f21a3fff700 1 -- 172.21.15.33:0/992152892 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21ac003d20 con 0x7f21bc100000 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.879+0000 7f21c1321700 1 -- 172.21.15.33:0/992152892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc102230 con 0x7f21bc100000 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.879+0000 7f21a3fff700 1 -- 172.21.15.33:0/992152892 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac004a00 con 0x7f21bc100000 2024-05-08T21:22:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.879+0000 7f21c1321700 1 -- 172.21.15.33:0/992152892 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc102700 con 0x7f21bc100000 2024-05-08T21:22:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.881+0000 7f21a3fff700 1 -- 172.21.15.33:0/992152892 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f21ac0051c0 con 0x7f21bc100000 2024-05-08T21:22:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.882+0000 7f21a3fff700 1 --2- 172.21.15.33:0/992152892 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21a40a11a0 0x7f21a40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.882+0000 7f21ba7fc700 1 --2- 172.21.15.33:0/992152892 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21a40a11a0 0x7f21a40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.882+0000 7f21a3fff700 1 -- 172.21.15.33:0/992152892 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f21ac0d22a0 con 0x7f21bc100000 2024-05-08T21:22:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.883+0000 7f21ba7fc700 1 --2- 172.21.15.33:0/992152892 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21a40a11a0 0x7f21a40a3660 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f21bc101080 tx=0x7f21b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:47.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.883+0000 7f21cd4dbb80 1 Processor -- start 2024-05-08T21:22:47.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.883+0000 7f21cd4dbb80 1 -- start start 2024-05-08T21:22:47.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.883+0000 7f21cd4dbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c84960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.883+0000 7f21cd4dbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c85320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.883+0000 7f21cd4dbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c85ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.883+0000 7f21cd4dbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e3bdb150 con 0x55b6e3c85860 2024-05-08T21:22:47.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.883+0000 7f21cd4dbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e3bd8b10 con 0x55b6e3c84540 2024-05-08T21:22:47.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.883+0000 7f21cd4dbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e3bd4400 con 0x55b6e3c84ea0 2024-05-08T21:22:47.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c85ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21baffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c84960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21baffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c84960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:50972/0 (socket says 172.21.15.33:50972) 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21baffd700 1 -- 172.21.15.33:0/587195432 learned_addr learned my addr 172.21.15.33:0/587195432 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c85ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42436/0 (socket says 172.21.15.33:42436) 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21ba7fc700 1 --2- 172.21.15.33:0/587195432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c85320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21baffd700 1 -- 172.21.15.33:0/587195432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 msgr2=0x55b6e3c85320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21baffd700 1 --2- 172.21.15.33:0/587195432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c85320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21baffd700 1 -- 172.21.15.33:0/587195432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 msgr2=0x55b6e3c85ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21baffd700 1 --2- 172.21.15.33:0/587195432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c85ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21baffd700 1 -- 172.21.15.33:0/587195432 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6e3a553b0 con 0x55b6e3c84540 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21baffd700 1 --2- 172.21.15.33:0/587195432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c84960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f21ac09f350 tx=0x7f21ac009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec41f16a099ccb1e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21a17fa700 1 -- 172.21.15.33:0/587195432 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac0cfea0 con 0x55b6e3c84540 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21a17fa700 1 -- 172.21.15.33:0/587195432 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21ac08a0d0 con 0x55b6e3c84540 2024-05-08T21:22:47.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.884+0000 7f21a17fa700 1 -- 172.21.15.33:0/587195432 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac09d760 con 0x55b6e3c84540 2024-05-08T21:22:47.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.885+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/587195432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 msgr2=0x55b6e3c84960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:47.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.885+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/587195432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c84960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f21ac09f350 tx=0x7f21ac009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.885+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/587195432 shutdown_connections 2024-05-08T21:22:47.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.885+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/587195432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c85ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.885+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/587195432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c84960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.885+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/587195432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c85320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.885+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/587195432 >> 172.21.15.33:0/587195432 conn(0x55b6e3be2210 msgr2=0x55b6e3c7ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:47.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.885+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/587195432 shutdown_connections 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.885+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/587195432 wait complete. 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21cd4dbb80 1 -- start start 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21cd4dbb80 1 Processor -- start 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21cd4dbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c8db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21cd4dbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c8e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21cd4dbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c85860 0x55b6e3c95080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21cd4dbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e3b70560 con 0x55b6e3c84ea0 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21cd4dbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e3b70200 con 0x55b6e3c84540 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21cd4dbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e3a576a0 con 0x55b6e3c85860 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21cd4dbb80 10 client.0 _ll_register_callbacks cb 0x55b6e3b79b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21ba7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c8e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c85860 0x55b6e3c95080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c85860 0x55b6e3c95080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:40144/0 (socket says 172.21.15.33:40144) 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21ba7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c8e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42438/0 (socket says 172.21.15.33:42438) 2024-05-08T21:22:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21bb7fe700 1 -- 172.21.15.33:0/2311983215 learned_addr learned my addr 172.21.15.33:0/2311983215 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.886+0000 7f21baffd700 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c8db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f21bb7fe700 1 -- 172.21.15.33:0/2311983215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 msgr2=0x55b6e3c8db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f21bb7fe700 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c8db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f21bb7fe700 1 -- 172.21.15.33:0/2311983215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c84ea0 msgr2=0x55b6e3c8e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f21bb7fe700 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c8e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f21bb7fe700 1 -- 172.21.15.33:0/2311983215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc102700 con 0x55b6e3c85860 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f21ba7fc700 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c8e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f21bb7fe700 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c85860 0x55b6e3c95080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f21b4014e10 tx=0x7f21b400e610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a0da91c057e9fa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f219eff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f219eff5700 1 -- 172.21.15.33:0/2311983215 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400e040 con 0x55b6e3c85860 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/2311983215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b6e3b2d270 con 0x55b6e3c85860 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f219eff5700 1 -- 172.21.15.33:0/2311983215 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f21b40090d0 con 0x55b6e3c85860 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f219eff5700 1 -- 172.21.15.33:0/2311983215 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21b4009290 con 0x55b6e3c85860 2024-05-08T21:22:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f21cd4dbb80 10 client.26434 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.887+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/2311983215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b6e3a01b60 con 0x55b6e3c85860 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.888+0000 7f219eff5700 1 -- 172.21.15.33:0/2311983215 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b6e3b2d270 con 0x55b6e3c85860 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.888+0000 7f219eff5700 1 -- 172.21.15.33:0/2311983215 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f21b401c030 con 0x55b6e3c85860 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.888+0000 7f21cd4dbb80 10 client.26434 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.888+0000 7f21cd4dbb80 10 client.26434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.888+0000 7f21cd4dbb80 20 client.26434 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.888+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b6e3c999e0 0x55b6e3c9be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.888+0000 7f21cd4dbb80 4 client.26434 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.888+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/2311983215 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b6e3c53810 con 0x55b6e3c999e0 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.888+0000 7f21baffd700 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b6e3c999e0 0x55b6e3c9be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.889+0000 7f21baffd700 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b6e3c999e0 0x55b6e3c9be00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c975a5ceefde39f9 server_cookie=59d04340089e9339 in_seq=0 out_seq=0 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.889+0000 7f219eff5700 10 client.26434 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.889+0000 7f219eff5700 1 -- 172.21.15.33:0/2311983215 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b6e3c53810 con 0x55b6e3c999e0 2024-05-08T21:22:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.889+0000 7f219eff5700 10 client.26434 handle_command_reply: tid=0 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.889+0000 7f21cd4dbb80 1 client.26434 shutdown 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.889+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/2311983215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c85860 msgr2=0x55b6e3c95080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.889+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c85860 0x55b6e3c95080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f21b4014e10 tx=0x7f21b400e610 comp rx=0 tx=0).stop 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.890+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/2311983215 shutdown_connections 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.890+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b6e3c999e0 0x55b6e3c9be00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.890+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c85860 0x55b6e3c95080 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.890+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c8db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.890+0000 7f219eff5700 0 client.26434 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.890+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/2311983215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c8e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.890+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/2311983215 >> 172.21.15.33:0/2311983215 conn(0x55b6e3be2210 msgr2=0x55b6e3c7e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.890+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/2311983215 shutdown_connections 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.890+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/2311983215 wait complete. 2024-05-08T21:22:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.890+0000 7f21cd4dbb80 20 client.26434 trim_cache size 0 max 16384 2024-05-08T21:22:47.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.909+0000 7f21cd4dbb80 1 Processor -- start 2024-05-08T21:22:47.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.909+0000 7f21cd4dbb80 1 -- start start 2024-05-08T21:22:47.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.909+0000 7f21cd4dbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c8d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.909+0000 7f21cd4dbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c95040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.909+0000 7f21cd4dbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c8bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.909+0000 7f21cd4dbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e3c8d670 con 0x55b6e3c85860 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.909+0000 7f21cd4dbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e3c96d90 con 0x55b6e3c84540 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.909+0000 7f21cd4dbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e3c7bdc0 con 0x55b6e3c84ea0 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21baffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c8d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21baffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c8d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:50998/0 (socket says 172.21.15.33:50998) 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21baffd700 1 -- 172.21.15.33:0/715800977 learned_addr learned my addr 172.21.15.33:0/715800977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21ba7fc700 1 --2- 172.21.15.33:0/715800977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c95040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21bb7fe700 1 --2- 172.21.15.33:0/715800977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c8bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21baffd700 1 -- 172.21.15.33:0/715800977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 msgr2=0x55b6e3c95040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21baffd700 1 --2- 172.21.15.33:0/715800977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c95040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21baffd700 1 -- 172.21.15.33:0/715800977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 msgr2=0x55b6e3c8bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21baffd700 1 --2- 172.21.15.33:0/715800977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c8bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21baffd700 1 -- 172.21.15.33:0/715800977 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b6e3b9a560 con 0x55b6e3c84540 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21bb7fe700 1 --2- 172.21.15.33:0/715800977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c8bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:22:47.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21baffd700 1 --2- 172.21.15.33:0/715800977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c8d010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f21ac09f350 tx=0x7f21ac005ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92349bbd2ee91d25 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21a1ffb700 1 -- 172.21.15.33:0/715800977 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac0cfea0 con 0x55b6e3c84540 2024-05-08T21:22:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21a1ffb700 1 -- 172.21.15.33:0/715800977 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21ac0a3270 con 0x55b6e3c84540 2024-05-08T21:22:47.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.910+0000 7f21a1ffb700 1 -- 172.21.15.33:0/715800977 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac0d2030 con 0x55b6e3c84540 2024-05-08T21:22:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.911+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/715800977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 msgr2=0x55b6e3c8d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.911+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/715800977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c8d010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f21ac09f350 tx=0x7f21ac005ac0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.911+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/715800977 shutdown_connections 2024-05-08T21:22:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.911+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/715800977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c8bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.911+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/715800977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c8d010 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.911+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/715800977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c95040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.911+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/715800977 >> 172.21.15.33:0/715800977 conn(0x55b6e3be2210 msgr2=0x55b6e3c85c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.911+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/715800977 shutdown_connections 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.911+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/715800977 wait complete. 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21cd4dbb80 1 -- start start 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21cd4dbb80 1 Processor -- start 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21cd4dbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c90c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21cd4dbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c91180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21cd4dbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c94150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21cd4dbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e3c94690 con 0x55b6e3c85860 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21cd4dbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e3c94800 con 0x55b6e3c84540 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21cd4dbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6e3c949c0 con 0x55b6e3c84ea0 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21cd4dbb80 10 client.0 _ll_register_callbacks cb 0x55b6e3b79b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c94150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c94150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42448/0 (socket says 172.21.15.33:42448) 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21baffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c90c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21bb7fe700 1 -- 172.21.15.33:0/1558105271 learned_addr learned my addr 172.21.15.33:0/1558105271 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.912+0000 7f21ba7fc700 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c91180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21baffd700 1 -- 172.21.15.33:0/1558105271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 msgr2=0x55b6e3c91180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21baffd700 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c91180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21baffd700 1 -- 172.21.15.33:0/1558105271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 msgr2=0x55b6e3c94150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21baffd700 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c94150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21baffd700 1 -- 172.21.15.33:0/1558105271 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b6e3a553b0 con 0x55b6e3c84540 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21bb7fe700 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c94150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21baffd700 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c90c40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55b6e3c962c0 tx=0x7f21ac09b0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a41f1ced5cf3c9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f219eff5700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f219eff5700 1 -- 172.21.15.33:0/1558105271 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac09fac0 con 0x55b6e3c84540 2024-05-08T21:22:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/1558105271 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21ac08a0d0 con 0x55b6e3c84540 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f219eff5700 1 -- 172.21.15.33:0/1558105271 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f21ac09d760 con 0x55b6e3c84540 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f219eff5700 1 -- 172.21.15.33:0/1558105271 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21ac09e8b0 con 0x55b6e3c84540 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21cd4dbb80 10 client.26420 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/1558105271 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b6e3a59cd0 con 0x55b6e3c84540 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f219eff5700 1 -- 172.21.15.33:0/1558105271 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f21ac084020 con 0x55b6e3c84540 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f219eff5700 1 -- 172.21.15.33:0/1558105271 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f21ac09b1b0 con 0x55b6e3c84540 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21cd4dbb80 10 client.26420 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21cd4dbb80 10 client.26420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21cd4dbb80 20 client.26420 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b6e3c999e0 0x55b6e3cf2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21cd4dbb80 4 client.26420 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.913+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/1558105271 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b6e3bccec0 con 0x55b6e3c999e0 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f21ba7fc700 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b6e3c999e0 0x55b6e3cf2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f219eff5700 10 client.26420 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f21ba7fc700 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b6e3c999e0 0x55b6e3cf2830 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7aa6c74ccc60268 server_cookie=a6c46e6739495706 in_seq=0 out_seq=0 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f219eff5700 1 -- 172.21.15.33:0/1558105271 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b6e3bccec0 con 0x55b6e3c999e0 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f219eff5700 10 client.26420 handle_command_reply: tid=0 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f21cd4dbb80 1 client.26420 shutdown 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/1558105271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 msgr2=0x55b6e3c90c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c90c40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55b6e3c962c0 tx=0x7f21ac09b0d0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/1558105271 shutdown_connections 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55b6e3c999e0 0x55b6e3cf2830 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6e3c85860 0x55b6e3c94150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e3c84540 0x55b6e3c90c40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f21cd4dbb80 1 --2- 172.21.15.33:0/1558105271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b6e3c84ea0 0x55b6e3c91180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f219eff5700 0 client.26420 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.914+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/1558105271 >> 172.21.15.33:0/1558105271 conn(0x55b6e3be2210 msgr2=0x55b6e3c88790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/1558105271 shutdown_connections 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21cd4dbb80 1 -- 172.21.15.33:0/1558105271 wait complete. 2024-05-08T21:22:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21cd4dbb80 20 client.26420 trim_cache size 0 max 16384 2024-05-08T21:22:47.923 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:22:47.923 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:22:47.923 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:22:47.923 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:22:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21c1321700 1 -- 172.21.15.33:0/992152892 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21a40a11a0 msgr2=0x7f21a40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21c1321700 1 --2- 172.21.15.33:0/992152892 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21a40a11a0 0x7f21a40a3660 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f21bc101080 tx=0x7f21b000b410 comp rx=0 tx=0).stop 2024-05-08T21:22:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21c1321700 1 -- 172.21.15.33:0/992152892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21bc100000 msgr2=0x7f21bc196dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21c1321700 1 --2- 172.21.15.33:0/992152892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21bc100000 0x7f21bc196dc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f21ac000c00 tx=0x7f21ac004730 comp rx=0 tx=0).stop 2024-05-08T21:22:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21c1321700 1 -- 172.21.15.33:0/992152892 shutdown_connections 2024-05-08T21:22:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21c1321700 1 --2- 172.21.15.33:0/992152892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc101360 0x7f21bc19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21c1321700 1 --2- 172.21.15.33:0/992152892 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f21a40a11a0 0x7f21a40a3660 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21c1321700 1 --2- 172.21.15.33:0/992152892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21bc100000 0x7f21bc196dc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21c1321700 1 --2- 172.21.15.33:0/992152892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21bc1009c0 0x7f21bc197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:47.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21c1321700 1 -- 172.21.15.33:0/992152892 >> 172.21.15.33:0/992152892 conn(0x7f21bc0faa50 msgr2=0x7f21bc0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:47.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21c1321700 1 -- 172.21.15.33:0/992152892 shutdown_connections 2024-05-08T21:22:47.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:47.915+0000 7f21c1321700 1 -- 172.21.15.33:0/992152892 wait complete. 2024-05-08T21:22:47.932 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:22:47.932 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:22:48.022 INFO:tasks.workunit.client.0.smithi033.stdout:fallocating to largest ever: 0x100000 2024-05-08T21:22:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.085+0000 7fc4e493d700 1 -- 172.21.15.33:0/3760702679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00ffda0 msgr2=0x7fc4e01001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.085+0000 7fc4e493d700 1 --2- 172.21.15.33:0/3760702679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00ffda0 0x7fc4e01001c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00099c0 tx=0x7fc4d0009cd0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.086+0000 7fc4e493d700 1 -- 172.21.15.33:0/3760702679 shutdown_connections 2024-05-08T21:22:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.086+0000 7fc4e493d700 1 --2- 172.21.15.33:0/3760702679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4e0101930 0x7fc4e01081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.086+0000 7fc4e493d700 1 --2- 172.21.15.33:0/3760702679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00ffda0 0x7fc4e01001c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.086+0000 7fc4e493d700 1 --2- 172.21.15.33:0/3760702679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e0100f90 0x7fc4e01013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.086+0000 7fc4e493d700 1 -- 172.21.15.33:0/3760702679 >> 172.21.15.33:0/3760702679 conn(0x7fc4e00fb310 msgr2=0x7fc4e00fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.086+0000 7fc4e493d700 1 -- 172.21.15.33:0/3760702679 shutdown_connections 2024-05-08T21:22:48.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.086+0000 7fc4e493d700 1 -- 172.21.15.33:0/3760702679 wait complete. 2024-05-08T21:22:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4e493d700 1 Processor -- start 2024-05-08T21:22:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4e493d700 1 -- start start 2024-05-08T21:22:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4e493d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e00ffda0 0x7fc4e0196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4e493d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4e0100f90 0x7fc4e0197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4e493d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e0101930 0x7fc4e019e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4e493d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e019e860 con 0x7fc4e0101930 2024-05-08T21:22:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4e493d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e019e9d0 con 0x7fc4e00ffda0 2024-05-08T21:22:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4e493d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e019eb90 con 0x7fc4e0100f90 2024-05-08T21:22:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4de59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e00ffda0 0x7fc4e0196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4ded9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e0101930 0x7fc4e019e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4de59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e00ffda0 0x7fc4e0196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51034/0 (socket says 172.21.15.33:51034) 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4de59c700 1 -- 172.21.15.33:0/2531246819 learned_addr learned my addr 172.21.15.33:0/2531246819 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4ddd9b700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4e0100f90 0x7fc4e0197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4de59c700 1 -- 172.21.15.33:0/2531246819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4e0100f90 msgr2=0x7fc4e0197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4de59c700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4e0100f90 0x7fc4e0197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4de59c700 1 -- 172.21.15.33:0/2531246819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e0101930 msgr2=0x7fc4e019e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4de59c700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e0101930 0x7fc4e019e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.087+0000 7fc4de59c700 1 -- 172.21.15.33:0/2531246819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d00095f0 con 0x7fc4e00ffda0 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.088+0000 7fc4de59c700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e00ffda0 0x7fc4e0196dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0006010 tx=0x7fc4d000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.088+0000 7fc4cf7fe700 1 -- 172.21.15.33:0/2531246819 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d001d070 con 0x7fc4e00ffda0 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.088+0000 7fc4e493d700 1 -- 172.21.15.33:0/2531246819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e019ee10 con 0x7fc4e00ffda0 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.088+0000 7fc4cf7fe700 1 -- 172.21.15.33:0/2531246819 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4d0005030 con 0x7fc4e00ffda0 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.088+0000 7fc4cf7fe700 1 -- 172.21.15.33:0/2531246819 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0004ab0 con 0x7fc4e00ffda0 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.088+0000 7fc4e493d700 1 -- 172.21.15.33:0/2531246819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e019f180 con 0x7fc4e00ffda0 2024-05-08T21:22:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.089+0000 7fc4e493d700 1 -- 172.21.15.33:0/2531246819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e0067480 con 0x7fc4e00ffda0 2024-05-08T21:22:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.090+0000 7fc4cf7fe700 1 -- 172.21.15.33:0/2531246819 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc4d00051a0 con 0x7fc4e00ffda0 2024-05-08T21:22:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.091+0000 7fc4cf7fe700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc4c80a1120 0x7fc4c80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.091+0000 7fc4ddd9b700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc4c80a1120 0x7fc4c80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.091+0000 7fc4cf7fe700 1 -- 172.21.15.33:0/2531246819 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc4d00cd7f0 con 0x7fc4e00ffda0 2024-05-08T21:22:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.091+0000 7fc4ddd9b700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc4c80a1120 0x7fc4c80a35e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009c60 tx=0x7fc4d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.096+0000 7fc4cf7fe700 1 -- 172.21.15.33:0/2531246819 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc4d0096d70 con 0x7fc4e00ffda0 2024-05-08T21:22:48.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.407+0000 7fc4e493d700 1 -- 172.21.15.33:0/2531246819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4e019f500 con 0x7fc4e00ffda0 2024-05-08T21:22:48.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.408+0000 7fc4cf7fe700 1 -- 172.21.15.33:0/2531246819 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10914 (secure 0 0 0) 0x7fc4d0026090 con 0x7fc4e00ffda0 2024-05-08T21:22:48.410 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:22:48.410 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:22:25.305999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:22:48.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.411+0000 7fc4e493d700 1 -- 172.21.15.33:0/2531246819 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc4c80a1120 msgr2=0x7fc4c80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.411+0000 7fc4e493d700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc4c80a1120 0x7fc4c80a35e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009c60 tx=0x7fc4d4009260 comp rx=0 tx=0).stop 2024-05-08T21:22:48.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.411+0000 7fc4e493d700 1 -- 172.21.15.33:0/2531246819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e00ffda0 msgr2=0x7fc4e0196dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.411+0000 7fc4e493d700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e00ffda0 0x7fc4e0196dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0006010 tx=0x7fc4d000f690 comp rx=0 tx=0).stop 2024-05-08T21:22:48.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.411+0000 7fc4e493d700 1 -- 172.21.15.33:0/2531246819 shutdown_connections 2024-05-08T21:22:48.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.411+0000 7fc4e493d700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e0101930 0x7fc4e019e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.411+0000 7fc4e493d700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc4c80a1120 0x7fc4c80a35e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.411+0000 7fc4e493d700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e00ffda0 0x7fc4e0196dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.411+0000 7fc4e493d700 1 --2- 172.21.15.33:0/2531246819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4e0100f90 0x7fc4e0197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.411+0000 7fc4e493d700 1 -- 172.21.15.33:0/2531246819 >> 172.21.15.33:0/2531246819 conn(0x7fc4e00fb310 msgr2=0x7fc4e0069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.411+0000 7fc4e493d700 1 -- 172.21.15.33:0/2531246819 shutdown_connections 2024-05-08T21:22:48.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.411+0000 7fc4e493d700 1 -- 172.21.15.33:0/2531246819 wait complete. 2024-05-08T21:22:48.414 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 34 2024-05-08T21:22:48.427 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:22:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.583+0000 7f4145d5d700 1 -- 172.21.15.33:0/3429735248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140104270 msgr2=0x7f4140106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.583+0000 7f4145d5d700 1 --2- 172.21.15.33:0/3429735248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140104270 0x7f4140106660 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4134009990 tx=0x7f4134009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.583+0000 7f4145d5d700 1 -- 172.21.15.33:0/3429735248 shutdown_connections 2024-05-08T21:22:48.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.583+0000 7f4145d5d700 1 --2- 172.21.15.33:0/3429735248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4140106ba0 0x7f4140109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.583+0000 7f4145d5d700 1 --2- 172.21.15.33:0/3429735248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4140101940 0x7f4140103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.583+0000 7f4145d5d700 1 --2- 172.21.15.33:0/3429735248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140104270 0x7f4140106660 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.583+0000 7f4145d5d700 1 -- 172.21.15.33:0/3429735248 >> 172.21.15.33:0/3429735248 conn(0x7f41400fb330 msgr2=0x7f41400fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.583+0000 7f4145d5d700 1 -- 172.21.15.33:0/3429735248 shutdown_connections 2024-05-08T21:22:48.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.583+0000 7f4145d5d700 1 -- 172.21.15.33:0/3429735248 wait complete. 2024-05-08T21:22:48.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f4145d5d700 1 Processor -- start 2024-05-08T21:22:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f4145d5d700 1 -- start start 2024-05-08T21:22:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f4145d5d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4140101940 0x7f4140198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f4145d5d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4140104270 0x7f4140199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f4145d5d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140106ba0 0x7f41401a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f4145d5d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41401a0a50 con 0x7f4140106ba0 2024-05-08T21:22:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f4145d5d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41401a0b90 con 0x7f4140101940 2024-05-08T21:22:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f4145d5d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41401a0d20 con 0x7f4140104270 2024-05-08T21:22:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f413ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140106ba0 0x7f41401a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f413f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4140101940 0x7f4140198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f413ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140106ba0 0x7f41401a0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42488/0 (socket says 172.21.15.33:42488) 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f413ffff700 1 -- 172.21.15.33:0/3734246936 learned_addr learned my addr 172.21.15.33:0/3734246936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.584+0000 7f413effd700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4140104270 0x7f4140199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.585+0000 7f413f7fe700 1 -- 172.21.15.33:0/3734246936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4140104270 msgr2=0x7f4140199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.585+0000 7f413f7fe700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4140104270 0x7f4140199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.585+0000 7f413f7fe700 1 -- 172.21.15.33:0/3734246936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140106ba0 msgr2=0x7f41401a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.585+0000 7f413f7fe700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140106ba0 0x7f41401a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.585+0000 7f413f7fe700 1 -- 172.21.15.33:0/3734246936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4130009520 con 0x7f4140101940 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.585+0000 7f413f7fe700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4140101940 0x7f4140198f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f413000e9f0 tx=0x7f413000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.585+0000 7f413cff9700 1 -- 172.21.15.33:0/3734246936 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413000cb70 con 0x7f4140101940 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.585+0000 7f413cff9700 1 -- 172.21.15.33:0/3734246936 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f413000ee90 con 0x7f4140101940 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.585+0000 7f4145d5d700 1 -- 172.21.15.33:0/3734246936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41340095f0 con 0x7f4140101940 2024-05-08T21:22:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.585+0000 7f413cff9700 1 -- 172.21.15.33:0/3734246936 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130003bc0 con 0x7f4140101940 2024-05-08T21:22:48.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.585+0000 7f4145d5d700 1 -- 172.21.15.33:0/3734246936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41401a12e0 con 0x7f4140101940 2024-05-08T21:22:48.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.588+0000 7f413cff9700 1 -- 172.21.15.33:0/3734246936 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f4130003e20 con 0x7f4140101940 2024-05-08T21:22:48.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.589+0000 7f413cff9700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f41280a11a0 0x7f41280a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.589+0000 7f413cff9700 1 -- 172.21.15.33:0/3734246936 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f4130014070 con 0x7f4140101940 2024-05-08T21:22:48.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.589+0000 7f413effd700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f41280a11a0 0x7f41280a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.590+0000 7f413effd700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f41280a11a0 0x7f41280a3660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4134006010 tx=0x7f413400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.590+0000 7f4151f17b80 1 Processor -- start 2024-05-08T21:22:48.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.590+0000 7f4151f17b80 1 -- start start 2024-05-08T21:22:48.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.590+0000 7f4151f17b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084e540 0x563ea084e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.590+0000 7f4151f17b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea084f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.590+0000 7f4151f17b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084f860 0x563ea084fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.590+0000 7f4151f17b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea07a5150 con 0x563ea084eea0 2024-05-08T21:22:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.590+0000 7f4151f17b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea07a2b10 con 0x563ea084f860 2024-05-08T21:22:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.590+0000 7f4151f17b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea079e400 con 0x563ea084e540 2024-05-08T21:22:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f413f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084e540 0x563ea084e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f413effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea084f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f413f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084e540 0x563ea084e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:40212/0 (socket says 172.21.15.33:40212) 2024-05-08T21:22:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f413f7fe700 1 -- 172.21.15.33:0/2287548196 learned_addr learned my addr 172.21.15.33:0/2287548196 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f413ffff700 1 --2- 172.21.15.33:0/2287548196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084f860 0x563ea084fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f413f7fe700 1 -- 172.21.15.33:0/2287548196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084f860 msgr2=0x563ea084fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f413f7fe700 1 --2- 172.21.15.33:0/2287548196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084f860 0x563ea084fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f413f7fe700 1 -- 172.21.15.33:0/2287548196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 msgr2=0x563ea084f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f413f7fe700 1 --2- 172.21.15.33:0/2287548196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea084f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f413f7fe700 1 -- 172.21.15.33:0/2287548196 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ea061f3b0 con 0x563ea084e540 2024-05-08T21:22:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f413f7fe700 1 --2- 172.21.15.33:0/2287548196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084e540 0x563ea084e960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41300a1e80 tx=0x7f4130003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d5c2b9f46f8b161 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f4125ffb700 1 -- 172.21.15.33:0/2287548196 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300b1070 con 0x563ea084e540 2024-05-08T21:22:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f4125ffb700 1 -- 172.21.15.33:0/2287548196 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4130099890 con 0x563ea084e540 2024-05-08T21:22:48.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.591+0000 7f4125ffb700 1 -- 172.21.15.33:0/2287548196 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300982d0 con 0x563ea084e540 2024-05-08T21:22:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.592+0000 7f4151f17b80 1 -- 172.21.15.33:0/2287548196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084e540 msgr2=0x563ea084e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.592+0000 7f4151f17b80 1 --2- 172.21.15.33:0/2287548196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084e540 0x563ea084e960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f41300a1e80 tx=0x7f4130003680 comp rx=0 tx=0).stop 2024-05-08T21:22:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.592+0000 7f4151f17b80 1 -- 172.21.15.33:0/2287548196 shutdown_connections 2024-05-08T21:22:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.592+0000 7f4151f17b80 1 --2- 172.21.15.33:0/2287548196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084f860 0x563ea084fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.592+0000 7f4151f17b80 1 --2- 172.21.15.33:0/2287548196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084e540 0x563ea084e960 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.592+0000 7f4151f17b80 1 --2- 172.21.15.33:0/2287548196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea084f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.592+0000 7f4151f17b80 1 -- 172.21.15.33:0/2287548196 >> 172.21.15.33:0/2287548196 conn(0x563ea07ac210 msgr2=0x563ea0848de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.592+0000 7f4151f17b80 1 -- 172.21.15.33:0/2287548196 shutdown_connections 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.592+0000 7f4151f17b80 1 -- 172.21.15.33:0/2287548196 wait complete. 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.593+0000 7f4151f17b80 1 -- start start 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.593+0000 7f4151f17b80 1 Processor -- start 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.593+0000 7f4151f17b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea0857e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.593+0000 7f4151f17b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea0858390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.593+0000 7f4151f17b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084f860 0x563ea085f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.593+0000 7f4151f17b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea073a560 con 0x563ea084eea0 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.593+0000 7f4151f17b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea073a200 con 0x563ea084e540 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.593+0000 7f4151f17b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea06216a0 con 0x563ea084f860 2024-05-08T21:22:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.593+0000 7f4151f17b80 10 client.0 _ll_register_callbacks cb 0x563ea0743b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.593+0000 7f413effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea0858390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.593+0000 7f413f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea0857e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f413f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea0857e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51074/0 (socket says 172.21.15.33:51074) 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f413effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea0858390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42496/0 (socket says 172.21.15.33:42496) 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f413ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084f860 0x563ea085f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f413f7fe700 1 -- 172.21.15.33:0/828949668 learned_addr learned my addr 172.21.15.33:0/828949668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f413f7fe700 1 -- 172.21.15.33:0/828949668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084f860 msgr2=0x563ea085f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f413f7fe700 1 --2- 172.21.15.33:0/828949668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084f860 0x563ea085f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f413f7fe700 1 -- 172.21.15.33:0/828949668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 msgr2=0x563ea0858390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f413f7fe700 1 --2- 172.21.15.33:0/828949668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea0858390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f413f7fe700 1 -- 172.21.15.33:0/828949668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41401a12e0 con 0x563ea084e540 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f413f7fe700 1 --2- 172.21.15.33:0/828949668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea0857e50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f41300969a0 tx=0x7f4130099fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51b1147abaac36f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f411effd700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f411effd700 1 -- 172.21.15.33:0/828949668 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300b1070 con 0x563ea084e540 2024-05-08T21:22:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f411effd700 1 -- 172.21.15.33:0/828949668 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f41300980e0 con 0x563ea084e540 2024-05-08T21:22:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f4151f17b80 1 -- 172.21.15.33:0/828949668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ea06f7270 con 0x563ea084e540 2024-05-08T21:22:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f411effd700 1 -- 172.21.15.33:0/828949668 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f413009a630 con 0x563ea084e540 2024-05-08T21:22:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f4151f17b80 10 client.26438 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.594+0000 7f4151f17b80 1 -- 172.21.15.33:0/828949668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563ea05cbb60 con 0x563ea084e540 2024-05-08T21:22:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.595+0000 7f411effd700 1 -- 172.21.15.33:0/828949668 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563ea06f7270 con 0x563ea084e540 2024-05-08T21:22:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.595+0000 7f411effd700 1 -- 172.21.15.33:0/828949668 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f41300040d0 con 0x563ea084e540 2024-05-08T21:22:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.595+0000 7f4151f17b80 10 client.26438 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.595+0000 7f4151f17b80 10 client.26438 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:22:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.595+0000 7f4151f17b80 20 client.26438 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.595+0000 7f4151f17b80 1 --2- 172.21.15.33:0/828949668 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563ea08638d0 0x563ea0865cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.595+0000 7f4151f17b80 4 client.26438 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.595+0000 7f4151f17b80 1 -- 172.21.15.33:0/828949668 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ea081d810 con 0x563ea08638d0 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.595+0000 7f413effd700 1 --2- 172.21.15.33:0/828949668 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563ea08638d0 0x563ea0865cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.596+0000 7f413effd700 1 --2- 172.21.15.33:0/828949668 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563ea08638d0 0x563ea0865cf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1e3890fbaec46d2 server_cookie=95fc9af59e52d0db in_seq=0 out_seq=0 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.596+0000 7f411effd700 10 client.26438 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.596+0000 7f411effd700 1 -- 172.21.15.33:0/828949668 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ea081d810 con 0x563ea08638d0 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.596+0000 7f411effd700 10 client.26438 handle_command_reply: tid=0 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.596+0000 7f4151f17b80 1 client.26438 shutdown 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.597+0000 7f4151f17b80 1 -- 172.21.15.33:0/828949668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 msgr2=0x563ea0857e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.597+0000 7f4151f17b80 1 --2- 172.21.15.33:0/828949668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea0857e50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f41300969a0 tx=0x7f4130099fc0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.597+0000 7f4151f17b80 1 -- 172.21.15.33:0/828949668 shutdown_connections 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.597+0000 7f4151f17b80 1 --2- 172.21.15.33:0/828949668 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563ea08638d0 0x563ea0865cf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.597+0000 7f411effd700 0 client.26438 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:22:48.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.597+0000 7f4151f17b80 1 --2- 172.21.15.33:0/828949668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084f860 0x563ea085f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.597+0000 7f4151f17b80 1 --2- 172.21.15.33:0/828949668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea0857e50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.597+0000 7f4151f17b80 1 --2- 172.21.15.33:0/828949668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea0858390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.597+0000 7f4151f17b80 1 -- 172.21.15.33:0/828949668 >> 172.21.15.33:0/828949668 conn(0x563ea07ac210 msgr2=0x563ea08485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.597+0000 7f4151f17b80 1 -- 172.21.15.33:0/828949668 shutdown_connections 2024-05-08T21:22:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.597+0000 7f4151f17b80 1 -- 172.21.15.33:0/828949668 wait complete. 2024-05-08T21:22:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.597+0000 7f4151f17b80 20 client.26438 trim_cache size 0 max 16384 2024-05-08T21:22:48.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.612+0000 7f4151f17b80 1 Processor -- start 2024-05-08T21:22:48.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.612+0000 7f4151f17b80 1 -- start start 2024-05-08T21:22:48.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.612+0000 7f4151f17b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea0865cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.612+0000 7f4151f17b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea085f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.612+0000 7f4151f17b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084f860 0x563ea084fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.612+0000 7f4151f17b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea0858e80 con 0x563ea084eea0 2024-05-08T21:22:48.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.612+0000 7f4151f17b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea0845dc0 con 0x563ea084e540 2024-05-08T21:22:48.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.612+0000 7f4151f17b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea0859040 con 0x563ea084f860 2024-05-08T21:22:48.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f413f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea0865cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f413effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea085f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f413f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea0865cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51076/0 (socket says 172.21.15.33:51076) 2024-05-08T21:22:48.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f413ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084f860 0x563ea084fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f413f7fe700 1 -- 172.21.15.33:0/2469040635 learned_addr learned my addr 172.21.15.33:0/2469040635 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:48.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f413f7fe700 1 -- 172.21.15.33:0/2469040635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084f860 msgr2=0x563ea084fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f413f7fe700 1 --2- 172.21.15.33:0/2469040635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084f860 0x563ea084fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f413f7fe700 1 -- 172.21.15.33:0/2469040635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 msgr2=0x563ea085f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f413f7fe700 1 --2- 172.21.15.33:0/2469040635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea085f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f413f7fe700 1 -- 172.21.15.33:0/2469040635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ea0764560 con 0x563ea084e540 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f413f7fe700 1 --2- 172.21.15.33:0/2469040635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea0865cf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f413000e9c0 tx=0x7f41300984e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969b900d23b0b0c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f41267fc700 1 -- 172.21.15.33:0/2469040635 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300b8300 con 0x563ea084e540 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f41267fc700 1 -- 172.21.15.33:0/2469040635 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f413009a630 con 0x563ea084e540 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.613+0000 7f41267fc700 1 -- 172.21.15.33:0/2469040635 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130099420 con 0x563ea084e540 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.614+0000 7f4151f17b80 1 -- 172.21.15.33:0/2469040635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 msgr2=0x563ea0865cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.614+0000 7f4151f17b80 1 --2- 172.21.15.33:0/2469040635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea0865cf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f413000e9c0 tx=0x7f41300984e0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.614+0000 7f4151f17b80 1 -- 172.21.15.33:0/2469040635 shutdown_connections 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.614+0000 7f4151f17b80 1 --2- 172.21.15.33:0/2469040635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084f860 0x563ea084fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.614+0000 7f4151f17b80 1 --2- 172.21.15.33:0/2469040635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea0865cf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.614+0000 7f4151f17b80 1 --2- 172.21.15.33:0/2469040635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084eea0 0x563ea085f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.614+0000 7f4151f17b80 1 -- 172.21.15.33:0/2469040635 >> 172.21.15.33:0/2469040635 conn(0x563ea07ac210 msgr2=0x563ea0845210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.614+0000 7f4151f17b80 1 -- 172.21.15.33:0/2469040635 shutdown_connections 2024-05-08T21:22:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.614+0000 7f4151f17b80 1 -- 172.21.15.33:0/2469040635 wait complete. 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f4151f17b80 1 -- start start 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f4151f17b80 1 Processor -- start 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f4151f17b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea085a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f4151f17b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084eea0 0x563ea085ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f4151f17b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084f860 0x563ea08b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f4151f17b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea08b82f0 con 0x563ea084f860 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f4151f17b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea08b8430 con 0x563ea084e540 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f4151f17b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ea08b8570 con 0x563ea084eea0 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f4151f17b80 10 client.0 _ll_register_callbacks cb 0x563ea0743b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f413effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084eea0 0x563ea085ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f413effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084eea0 0x563ea085ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:40240/0 (socket says 172.21.15.33:40240) 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f413f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea085a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f413effd700 1 -- 172.21.15.33:0/3950651964 learned_addr learned my addr 172.21.15.33:0/3950651964 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:48.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.615+0000 7f413ffff700 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084f860 0x563ea08b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f413effd700 1 -- 172.21.15.33:0/3950651964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 msgr2=0x563ea085a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f413effd700 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea085a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f413effd700 1 -- 172.21.15.33:0/3950651964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084f860 msgr2=0x563ea08b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f413effd700 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084f860 0x563ea08b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f413effd700 1 -- 172.21.15.33:0/3950651964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ea05cbb60 con 0x563ea084eea0 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f413f7fe700 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea085a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f413effd700 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084eea0 0x563ea085ab40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f41340098e0 tx=0x7f4134009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbecd751c4a8a5c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f411effd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f411effd700 1 -- 172.21.15.33:0/3950651964 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134003d80 con 0x563ea084eea0 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f411effd700 1 -- 172.21.15.33:0/3950651964 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f413400b710 con 0x563ea084eea0 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f4151f17b80 1 -- 172.21.15.33:0/3950651964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4130099890 con 0x563ea084eea0 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f411effd700 1 -- 172.21.15.33:0/3950651964 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f413400b8d0 con 0x563ea084eea0 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f4151f17b80 10 client.26464 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:48.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.616+0000 7f4151f17b80 1 -- 172.21.15.33:0/3950651964 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563ea0623cd0 con 0x563ea084eea0 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.617+0000 7f411effd700 1 -- 172.21.15.33:0/3950651964 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4130099890 con 0x563ea084eea0 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.617+0000 7f411effd700 1 -- 172.21.15.33:0/3950651964 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f4134036750 con 0x563ea084eea0 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.617+0000 7f4151f17b80 10 client.26464 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.617+0000 7f4151f17b80 10 client.26464 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.617+0000 7f4151f17b80 20 client.26464 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.617+0000 7f4151f17b80 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563ea08638d0 0x563ea08bc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.617+0000 7f4151f17b80 4 client.26464 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.617+0000 7f4151f17b80 1 -- 172.21.15.33:0/3950651964 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563ea0796ec0 con 0x563ea08638d0 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.617+0000 7f413f7fe700 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563ea08638d0 0x563ea08bc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.617+0000 7f413f7fe700 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563ea08638d0 0x563ea08bc040 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=766a07b8fc9ba82f server_cookie=b49256f694d00d2a in_seq=0 out_seq=0 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.617+0000 7f411effd700 10 client.26464 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f411effd700 1 -- 172.21.15.33:0/3950651964 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563ea0796ec0 con 0x563ea08638d0 2024-05-08T21:22:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f411effd700 10 client.26464 handle_command_reply: tid=0 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f4151f17b80 1 client.26464 shutdown 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f4151f17b80 1 -- 172.21.15.33:0/3950651964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084eea0 msgr2=0x563ea085ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f4151f17b80 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084eea0 0x563ea085ab40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f41340098e0 tx=0x7f4134009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f4151f17b80 1 -- 172.21.15.33:0/3950651964 shutdown_connections 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f4151f17b80 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ea084f860 0x563ea08b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f4151f17b80 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ea084e540 0x563ea085a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f4151f17b80 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563ea08638d0 0x563ea08bc040 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f4151f17b80 1 --2- 172.21.15.33:0/3950651964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563ea084eea0 0x563ea085ab40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f411effd700 0 client.26464 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f4151f17b80 1 -- 172.21.15.33:0/3950651964 >> 172.21.15.33:0/3950651964 conn(0x563ea07ac210 msgr2=0x563ea084fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f4151f17b80 1 -- 172.21.15.33:0/3950651964 shutdown_connections 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f4151f17b80 1 -- 172.21.15.33:0/3950651964 wait complete. 2024-05-08T21:22:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.618+0000 7f4151f17b80 20 client.26464 trim_cache size 0 max 16384 2024-05-08T21:22:48.626 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:22:48.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.619+0000 7f4145d5d700 1 -- 172.21.15.33:0/3734246936 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f41280a11a0 msgr2=0x7f41280a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.619+0000 7f4145d5d700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f41280a11a0 0x7f41280a3660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4134006010 tx=0x7f413400b610 comp rx=0 tx=0).stop 2024-05-08T21:22:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.619+0000 7f4145d5d700 1 -- 172.21.15.33:0/3734246936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4140101940 msgr2=0x7f4140198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.619+0000 7f4145d5d700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4140101940 0x7f4140198f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f413000e9f0 tx=0x7f413000ed00 comp rx=0 tx=0).stop 2024-05-08T21:22:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.619+0000 7f4145d5d700 1 -- 172.21.15.33:0/3734246936 shutdown_connections 2024-05-08T21:22:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.619+0000 7f4145d5d700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140106ba0 0x7f41401a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.619+0000 7f4145d5d700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f41280a11a0 0x7f41280a3660 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.619+0000 7f4145d5d700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4140101940 0x7f4140198f20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.619+0000 7f4145d5d700 1 --2- 172.21.15.33:0/3734246936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4140104270 0x7f4140199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.619+0000 7f4145d5d700 1 -- 172.21.15.33:0/3734246936 >> 172.21.15.33:0/3734246936 conn(0x7f41400fb330 msgr2=0x7f41400ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.619+0000 7f4145d5d700 1 -- 172.21.15.33:0/3734246936 shutdown_connections 2024-05-08T21:22:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.619+0000 7f4145d5d700 1 -- 172.21.15.33:0/3734246936 wait complete. 2024-05-08T21:22:48.636 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:22:48.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.789+0000 7fc5da09d700 1 -- 172.21.15.33:0/4086497238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d4101330 msgr2=0x7fc5d4101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.789+0000 7fc5da09d700 1 --2- 172.21.15.33:0/4086497238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d4101330 0x7fc5d4101750 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009990 tx=0x7fc5c4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.789+0000 7fc5da09d700 1 -- 172.21.15.33:0/4086497238 shutdown_connections 2024-05-08T21:22:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.789+0000 7fc5da09d700 1 --2- 172.21.15.33:0/4086497238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d4100940 0x7fc5d41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.789+0000 7fc5da09d700 1 --2- 172.21.15.33:0/4086497238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d4101330 0x7fc5d4101750 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.789+0000 7fc5da09d700 1 --2- 172.21.15.33:0/4086497238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d40fff80 0x7fc5d4100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.789+0000 7fc5da09d700 1 -- 172.21.15.33:0/4086497238 >> 172.21.15.33:0/4086497238 conn(0x7fc5d40faa50 msgr2=0x7fc5d40fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.789+0000 7fc5da09d700 1 -- 172.21.15.33:0/4086497238 shutdown_connections 2024-05-08T21:22:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.789+0000 7fc5da09d700 1 -- 172.21.15.33:0/4086497238 wait complete. 2024-05-08T21:22:48.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.790+0000 7fc5da09d700 1 Processor -- start 2024-05-08T21:22:48.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.790+0000 7fc5da09d700 1 -- start start 2024-05-08T21:22:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.790+0000 7fc5da09d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d40fff80 0x7fc5d40ff940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.790+0000 7fc5da09d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d4100940 0x7fc5d419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.790+0000 7fc5da09d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d4101330 0x7fc5d419e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.790+0000 7fc5da09d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d419e870 con 0x7fc5d4100940 2024-05-08T21:22:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.790+0000 7fc5da09d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d419e9e0 con 0x7fc5d40fff80 2024-05-08T21:22:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.790+0000 7fc5da09d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d419eba0 con 0x7fc5d4101330 2024-05-08T21:22:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.790+0000 7fc5d2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d4100940 0x7fc5d419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.790+0000 7fc5d37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d40fff80 0x7fc5d40ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5d2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d4100940 0x7fc5d419bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42526/0 (socket says 172.21.15.33:42526) 2024-05-08T21:22:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5d2ffd700 1 -- 172.21.15.33:0/1791051943 learned_addr learned my addr 172.21.15.33:0/1791051943 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5d3fff700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d4101330 0x7fc5d419e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5d37fe700 1 -- 172.21.15.33:0/1791051943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d4101330 msgr2=0x7fc5d419e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5d37fe700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d4101330 0x7fc5d419e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5d37fe700 1 -- 172.21.15.33:0/1791051943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d4100940 msgr2=0x7fc5d419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5d37fe700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d4100940 0x7fc5d419bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5d37fe700 1 -- 172.21.15.33:0/1791051943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c8009520 con 0x7fc5d40fff80 2024-05-08T21:22:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5d37fe700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d40fff80 0x7fc5d40ff940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4000c00 tx=0x7fc5c4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5d0ff9700 1 -- 172.21.15.33:0/1791051943 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c401d070 con 0x7fc5d40fff80 2024-05-08T21:22:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5da09d700 1 -- 172.21.15.33:0/1791051943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc009c40 con 0x7fc5d40fff80 2024-05-08T21:22:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5d0ff9700 1 -- 172.21.15.33:0/1791051943 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5c4005dc0 con 0x7fc5d40fff80 2024-05-08T21:22:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5d0ff9700 1 -- 172.21.15.33:0/1791051943 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4021df0 con 0x7fc5d40fff80 2024-05-08T21:22:48.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.791+0000 7fc5da09d700 1 -- 172.21.15.33:0/1791051943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c40095f0 con 0x7fc5d40fff80 2024-05-08T21:22:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.794+0000 7fc5d0ff9700 1 -- 172.21.15.33:0/1791051943 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc5c4021580 con 0x7fc5d40fff80 2024-05-08T21:22:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.794+0000 7fc5d0ff9700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5bc0a1120 0x7fc5bc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.795+0000 7fc5d0ff9700 1 -- 172.21.15.33:0/1791051943 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc5c40cde60 con 0x7fc5d40fff80 2024-05-08T21:22:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.795+0000 7fc5d2ffd700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5bc0a1120 0x7fc5bc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.795+0000 7fc5d2ffd700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5bc0a1120 0x7fc5bc0a35e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8005950 tx=0x7fc5c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5e6257b80 1 Processor -- start 2024-05-08T21:22:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5e6257b80 1 -- start start 2024-05-08T21:22:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5e6257b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fd540 0x563a406fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5e6257b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a406fe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5e6257b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fe860 0x563a406fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5e6257b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a40654150 con 0x563a406fd540 2024-05-08T21:22:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5e6257b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a40651b10 con 0x563a406fdea0 2024-05-08T21:22:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5e6257b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4064d400 con 0x563a406fe860 2024-05-08T21:22:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5d3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fe860 0x563a406fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5d37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fd540 0x563a406fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a406fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5d3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fe860 0x563a406fece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:40284/0 (socket says 172.21.15.33:40284) 2024-05-08T21:22:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.796+0000 7fc5d3fff700 1 -- 172.21.15.33:0/3224831785 learned_addr learned my addr 172.21.15.33:0/3224831785 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.797+0000 7fc5d2ffd700 1 -- 172.21.15.33:0/3224831785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fe860 msgr2=0x563a406fece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.797+0000 7fc5d2ffd700 1 --2- 172.21.15.33:0/3224831785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fe860 0x563a406fece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.797+0000 7fc5d2ffd700 1 -- 172.21.15.33:0/3224831785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fd540 msgr2=0x563a406fd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.797+0000 7fc5d2ffd700 1 --2- 172.21.15.33:0/3224831785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fd540 0x563a406fd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.797+0000 7fc5d2ffd700 1 -- 172.21.15.33:0/3224831785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a404ce3b0 con 0x563a406fdea0 2024-05-08T21:22:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.797+0000 7fc5d2ffd700 1 --2- 172.21.15.33:0/3224831785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a406fe320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e630 tx=0x7fc5c8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40cd3d0bc193193 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.797+0000 7fc5b9ffb700 1 -- 172.21.15.33:0/3224831785 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8004020 con 0x563a406fdea0 2024-05-08T21:22:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.797+0000 7fc5b9ffb700 1 -- 172.21.15.33:0/3224831785 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5c8021070 con 0x563a406fdea0 2024-05-08T21:22:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.797+0000 7fc5b9ffb700 1 -- 172.21.15.33:0/3224831785 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80266e0 con 0x563a406fdea0 2024-05-08T21:22:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.798+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3224831785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 msgr2=0x563a406fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.798+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/3224831785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a406fe320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e630 tx=0x7fc5c8014000 comp rx=0 tx=0).stop 2024-05-08T21:22:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.798+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3224831785 shutdown_connections 2024-05-08T21:22:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.798+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/3224831785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fe860 0x563a406fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.798+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/3224831785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fd540 0x563a406fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.798+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/3224831785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a406fe320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.798+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3224831785 >> 172.21.15.33:0/3224831785 conn(0x563a4065b210 msgr2=0x563a406f7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.798+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3224831785 shutdown_connections 2024-05-08T21:22:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.798+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3224831785 wait complete. 2024-05-08T21:22:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.798+0000 7fc5e6257b80 1 -- start start 2024-05-08T21:22:48.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5e6257b80 1 Processor -- start 2024-05-08T21:22:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5e6257b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fd540 0x563a40706e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5e6257b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a40707390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5e6257b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fe860 0x563a4070e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5e6257b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a405e9560 con 0x563a406fd540 2024-05-08T21:22:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5e6257b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a405e9200 con 0x563a406fdea0 2024-05-08T21:22:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5e6257b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a404d06a0 con 0x563a406fe860 2024-05-08T21:22:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5e6257b80 10 client.0 _ll_register_callbacks cb 0x563a405f2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a40707390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5d37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fd540 0x563a40706e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5d37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fd540 0x563a40706e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42538/0 (socket says 172.21.15.33:42538) 2024-05-08T21:22:48.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a40707390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51124/0 (socket says 172.21.15.33:51124) 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5d37fe700 1 -- 172.21.15.33:0/3474737163 learned_addr learned my addr 172.21.15.33:0/3474737163 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5d3fff700 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fe860 0x563a4070e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5d2ffd700 1 -- 172.21.15.33:0/3474737163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fe860 msgr2=0x563a4070e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5d2ffd700 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fe860 0x563a4070e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5d2ffd700 1 -- 172.21.15.33:0/3474737163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fd540 msgr2=0x563a40706e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5d2ffd700 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fd540 0x563a40706e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5d2ffd700 1 -- 172.21.15.33:0/3474737163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c8017070 con 0x563a406fdea0 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.799+0000 7fc5d3fff700 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fe860 0x563a4070e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.800+0000 7fc5d2ffd700 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a40707390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e600 tx=0x7fc5c8003720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a2e7b227a72ee0a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.800+0000 7fc5b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.800+0000 7fc5b2ffd700 1 -- 172.21.15.33:0/3474737163 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c802a040 con 0x563a406fdea0 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.800+0000 7fc5b2ffd700 1 -- 172.21.15.33:0/3474737163 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc5c8004020 con 0x563a406fdea0 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.800+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3474737163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a405a6270 con 0x563a406fdea0 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.800+0000 7fc5b2ffd700 1 -- 172.21.15.33:0/3474737163 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5c80041e0 con 0x563a406fdea0 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.800+0000 7fc5e6257b80 10 client.26456 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:48.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.800+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3474737163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563a4047ab60 con 0x563a406fdea0 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.800+0000 7fc5b2ffd700 1 -- 172.21.15.33:0/3474737163 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563a405a6270 con 0x563a406fdea0 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.801+0000 7fc5b2ffd700 1 -- 172.21.15.33:0/3474737163 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fc5c802f430 con 0x563a406fdea0 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.801+0000 7fc5e6257b80 10 client.26456 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.801+0000 7fc5e6257b80 10 client.26456 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.801+0000 7fc5e6257b80 20 client.26456 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.801+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x563a40712b50 0x563a40714f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.801+0000 7fc5e6257b80 4 client.26456 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.801+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3474737163 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a406cc810 con 0x563a40712b50 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.801+0000 7fc5d37fe700 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x563a40712b50 0x563a40714f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.801+0000 7fc5b2ffd700 10 client.26456 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.801+0000 7fc5d37fe700 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x563a40712b50 0x563a40714f70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc8103930222100a server_cookie=dec1a06b7912d68e in_seq=0 out_seq=0 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5b2ffd700 1 -- 172.21.15.33:0/3474737163 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a406cc810 con 0x563a40712b50 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5b2ffd700 10 client.26456 handle_command_reply: tid=0 2024-05-08T21:22:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5e6257b80 1 client.26456 shutdown 2024-05-08T21:22:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3474737163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 msgr2=0x563a40707390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a40707390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800e600 tx=0x7fc5c8003720 comp rx=0 tx=0).stop 2024-05-08T21:22:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3474737163 shutdown_connections 2024-05-08T21:22:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x563a40712b50 0x563a40714f70 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fe860 0x563a4070e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fd540 0x563a40706e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/3474737163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a40707390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5b2ffd700 0 client.26456 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:22:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3474737163 >> 172.21.15.33:0/3474737163 conn(0x563a4065b210 msgr2=0x563a406f75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3474737163 shutdown_connections 2024-05-08T21:22:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.802+0000 7fc5e6257b80 1 -- 172.21.15.33:0/3474737163 wait complete. 2024-05-08T21:22:48.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.803+0000 7fc5e6257b80 20 client.26456 trim_cache size 0 max 16384 2024-05-08T21:22:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5e6257b80 1 Processor -- start 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5e6257b80 1 -- start start 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5e6257b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 0x563a40714f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5e6257b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fdea0 0x563a407063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5e6257b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fe860 0x563a4070e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5e6257b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a407069c0 con 0x563a406fdea0 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5e6257b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a406f4dc0 con 0x563a406fe860 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5e6257b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4071a140 con 0x563a406fd540 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5d37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 0x563a40714f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5d3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fe860 0x563a4070e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5d2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fdea0 0x563a407063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5d37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 0x563a40714f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:40288/0 (socket says 172.21.15.33:40288) 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5d3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fe860 0x563a4070e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51126/0 (socket says 172.21.15.33:51126) 2024-05-08T21:22:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5d37fe700 1 -- 172.21.15.33:0/2748805155 learned_addr learned my addr 172.21.15.33:0/2748805155 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5d3fff700 1 -- 172.21.15.33:0/2748805155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 msgr2=0x563a40714f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5d3fff700 1 --2- 172.21.15.33:0/2748805155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 0x563a40714f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5d3fff700 1 -- 172.21.15.33:0/2748805155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fdea0 msgr2=0x563a407063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5d3fff700 1 --2- 172.21.15.33:0/2748805155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fdea0 0x563a407063d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.819+0000 7fc5d3fff700 1 -- 172.21.15.33:0/2748805155 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a40613560 con 0x563a406fe860 2024-05-08T21:22:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.820+0000 7fc5d37fe700 1 --2- 172.21.15.33:0/2748805155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 0x563a40714f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.820+0000 7fc5d3fff700 1 --2- 172.21.15.33:0/2748805155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fe860 0x563a4070e390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009fd0 tx=0x7fc5cc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4515b60f14719acc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.820+0000 7fc5ba7fc700 1 -- 172.21.15.33:0/2748805155 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc00e040 con 0x563a406fe860 2024-05-08T21:22:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.820+0000 7fc5ba7fc700 1 -- 172.21.15.33:0/2748805155 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5cc00f040 con 0x563a406fe860 2024-05-08T21:22:48.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.820+0000 7fc5ba7fc700 1 -- 172.21.15.33:0/2748805155 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc007d00 con 0x563a406fe860 2024-05-08T21:22:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.821+0000 7fc5e6257b80 1 -- 172.21.15.33:0/2748805155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fe860 msgr2=0x563a4070e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.821+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/2748805155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fe860 0x563a4070e390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009fd0 tx=0x7fc5cc009920 comp rx=0 tx=0).stop 2024-05-08T21:22:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.821+0000 7fc5e6257b80 1 -- 172.21.15.33:0/2748805155 shutdown_connections 2024-05-08T21:22:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.821+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/2748805155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fe860 0x563a4070e390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.821+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/2748805155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 0x563a40714f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.821+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/2748805155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fdea0 0x563a407063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.821+0000 7fc5e6257b80 1 -- 172.21.15.33:0/2748805155 >> 172.21.15.33:0/2748805155 conn(0x563a4065b210 msgr2=0x563a406fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.821+0000 7fc5e6257b80 1 -- 172.21.15.33:0/2748805155 shutdown_connections 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.821+0000 7fc5e6257b80 1 -- 172.21.15.33:0/2748805155 wait complete. 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.821+0000 7fc5e6257b80 1 -- start start 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5e6257b80 1 Processor -- start 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5e6257b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 0x563a40709430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5e6257b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a40709970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5e6257b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fe860 0x563a4070c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5e6257b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4070ced0 con 0x563a406fe860 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5e6257b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4070d040 con 0x563a406fdea0 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5e6257b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4070d200 con 0x563a406fd540 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5e6257b80 10 client.0 _ll_register_callbacks cb 0x563a405f2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5d37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 0x563a40709430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a40709970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5d2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a40709970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51134/0 (socket says 172.21.15.33:51134) 2024-05-08T21:22:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5d37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 0x563a40709430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:40298/0 (socket says 172.21.15.33:40298) 2024-05-08T21:22:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5d2ffd700 1 -- 172.21.15.33:0/1899039014 learned_addr learned my addr 172.21.15.33:0/1899039014 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5d3fff700 1 --2- 172.21.15.33:0/1899039014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fe860 0x563a4070c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5d2ffd700 1 -- 172.21.15.33:0/1899039014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 msgr2=0x563a40709430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5d2ffd700 1 --2- 172.21.15.33:0/1899039014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 0x563a40709430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5d2ffd700 1 -- 172.21.15.33:0/1899039014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fe860 msgr2=0x563a4070c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.822+0000 7fc5d2ffd700 1 --2- 172.21.15.33:0/1899039014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fe860 0x563a4070c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.823+0000 7fc5d2ffd700 1 -- 172.21.15.33:0/1899039014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c40095f0 con 0x563a406fdea0 2024-05-08T21:22:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.823+0000 7fc5d2ffd700 1 --2- 172.21.15.33:0/1899039014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a40709970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800ec20 tx=0x7fc5c802f660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=280680446584cda8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.823+0000 7fc5b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.823+0000 7fc5b2ffd700 1 -- 172.21.15.33:0/1899039014 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8042070 con 0x563a406fdea0 2024-05-08T21:22:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.823+0000 7fc5b2ffd700 1 -- 172.21.15.33:0/1899039014 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc5c802fa80 con 0x563a406fdea0 2024-05-08T21:22:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.823+0000 7fc5e6257b80 1 -- 172.21.15.33:0/1899039014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5c8021070 con 0x563a406fdea0 2024-05-08T21:22:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.823+0000 7fc5b2ffd700 1 -- 172.21.15.33:0/1899039014 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5c802a040 con 0x563a406fdea0 2024-05-08T21:22:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.823+0000 7fc5e6257b80 10 client.26462 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.823+0000 7fc5e6257b80 1 -- 172.21.15.33:0/1899039014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fc5cc00e9a0 con 0x563a406fdea0 2024-05-08T21:22:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.823+0000 7fc5b2ffd700 1 -- 172.21.15.33:0/1899039014 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5c8021070 con 0x563a406fdea0 2024-05-08T21:22:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.823+0000 7fc5b2ffd700 1 -- 172.21.15.33:0/1899039014 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fc5c8045020 con 0x563a406fdea0 2024-05-08T21:22:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.824+0000 7fc5e6257b80 10 client.26462 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.824+0000 7fc5e6257b80 10 client.26462 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:22:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.824+0000 7fc5e6257b80 20 client.26462 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.824+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/1899039014 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x563a40766ff0 0x563a40769410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.824+0000 7fc5e6257b80 4 client.26462 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:22:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.824+0000 7fc5e6257b80 1 -- 172.21.15.33:0/1899039014 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563a40645ec0 con 0x563a40766ff0 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.824+0000 7fc5d37fe700 1 --2- 172.21.15.33:0/1899039014 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x563a40766ff0 0x563a40769410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.824+0000 7fc5b2ffd700 10 client.26462 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.824+0000 7fc5d37fe700 1 --2- 172.21.15.33:0/1899039014 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x563a40766ff0 0x563a40769410 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=850073e9ad7451f7 server_cookie=72a8f350195a02fb in_seq=0 out_seq=0 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.824+0000 7fc5b2ffd700 1 -- 172.21.15.33:0/1899039014 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563a40645ec0 con 0x563a40766ff0 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.824+0000 7fc5b2ffd700 10 client.26462 handle_command_reply: tid=0 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.824+0000 7fc5e6257b80 1 client.26462 shutdown 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.825+0000 7fc5e6257b80 1 -- 172.21.15.33:0/1899039014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 msgr2=0x563a40709970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.825+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/1899039014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a40709970 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800ec20 tx=0x7fc5c802f660 comp rx=0 tx=0).stop 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.825+0000 7fc5e6257b80 1 -- 172.21.15.33:0/1899039014 shutdown_connections 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.825+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/1899039014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563a406fe860 0x563a4070c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.825+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/1899039014 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x563a40766ff0 0x563a40769410 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.825+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/1899039014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a406fd540 0x563a40709430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.825+0000 7fc5e6257b80 1 --2- 172.21.15.33:0/1899039014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a406fdea0 0x563a40709970 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.825+0000 7fc5b2ffd700 0 client.26462 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:22:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.825+0000 7fc5e6257b80 1 -- 172.21.15.33:0/1899039014 >> 172.21.15.33:0/1899039014 conn(0x563a4065b210 msgr2=0x563a40704e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.825+0000 7fc5e6257b80 1 -- 172.21.15.33:0/1899039014 shutdown_connections 2024-05-08T21:22:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.825+0000 7fc5e6257b80 1 -- 172.21.15.33:0/1899039014 wait complete. 2024-05-08T21:22:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.825+0000 7fc5e6257b80 20 client.26462 trim_cache size 0 max 16384 2024-05-08T21:22:48.833 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:22:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.826+0000 7fc5da09d700 1 -- 172.21.15.33:0/1791051943 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5bc0a1120 msgr2=0x7fc5bc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.826+0000 7fc5da09d700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5bc0a1120 0x7fc5bc0a35e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8005950 tx=0x7fc5c8009260 comp rx=0 tx=0).stop 2024-05-08T21:22:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.826+0000 7fc5da09d700 1 -- 172.21.15.33:0/1791051943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d40fff80 msgr2=0x7fc5d40ff940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.826+0000 7fc5da09d700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d40fff80 0x7fc5d40ff940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4000c00 tx=0x7fc5c4005740 comp rx=0 tx=0).stop 2024-05-08T21:22:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.826+0000 7fc5da09d700 1 -- 172.21.15.33:0/1791051943 shutdown_connections 2024-05-08T21:22:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.826+0000 7fc5da09d700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5d4101330 0x7fc5d419e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.826+0000 7fc5da09d700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5bc0a1120 0x7fc5bc0a35e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.826+0000 7fc5da09d700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5d40fff80 0x7fc5d40ff940 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.826+0000 7fc5da09d700 1 --2- 172.21.15.33:0/1791051943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d4100940 0x7fc5d419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.826+0000 7fc5da09d700 1 -- 172.21.15.33:0/1791051943 >> 172.21.15.33:0/1791051943 conn(0x7fc5d40faa50 msgr2=0x7fc5d41043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.826+0000 7fc5da09d700 1 -- 172.21.15.33:0/1791051943 shutdown_connections 2024-05-08T21:22:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.826+0000 7fc5da09d700 1 -- 172.21.15.33:0/1791051943 wait complete. 2024-05-08T21:22:48.842 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:22:49.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.997+0000 7f8367996700 1 -- 172.21.15.33:0/3442125144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83600ff700 msgr2=0x7f83600ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.997+0000 7f8367996700 1 --2- 172.21.15.33:0/3442125144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83600ff700 0x7f83600ffb20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f835c009990 tx=0x7f835c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.997+0000 7f8367996700 1 -- 172.21.15.33:0/3442125144 shutdown_connections 2024-05-08T21:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.997+0000 7f8367996700 1 --2- 172.21.15.33:0/3442125144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100a20 0x7f8360100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.997+0000 7f8367996700 1 --2- 172.21.15.33:0/3442125144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83600ff700 0x7f83600ffb20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.997+0000 7f8367996700 1 --2- 172.21.15.33:0/3442125144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8360100060 0x7f83601004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.997+0000 7f8367996700 1 -- 172.21.15.33:0/3442125144 >> 172.21.15.33:0/3442125144 conn(0x7f83600fb300 msgr2=0x7f83600fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.997+0000 7f8367996700 1 -- 172.21.15.33:0/3442125144 shutdown_connections 2024-05-08T21:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.997+0000 7f8367996700 1 -- 172.21.15.33:0/3442125144 wait complete. 2024-05-08T21:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.998+0000 7f8367996700 1 Processor -- start 2024-05-08T21:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.998+0000 7f8367996700 1 -- start start 2024-05-08T21:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.998+0000 7f8367996700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83600ff700 0x7f8360198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.998+0000 7f8367996700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100060 0x7f8360199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.998+0000 7f8367996700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8360100a20 0x7f83601a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.998+0000 7f8367996700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83601a0a40 con 0x7f8360100060 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.998+0000 7f8367996700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83601a0b80 con 0x7f83600ff700 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.998+0000 7f8367996700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83601a0d10 con 0x7f8360100a20 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8365732700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83600ff700 0x7f8360198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8365f33700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8360100a20 0x7f83601a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8365732700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83600ff700 0x7f8360198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51148/0 (socket says 172.21.15.33:51148) 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8365f33700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8360100a20 0x7f83601a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:40306/0 (socket says 172.21.15.33:40306) 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8365732700 1 -- 172.21.15.33:0/383342779 learned_addr learned my addr 172.21.15.33:0/383342779 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8364f31700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100060 0x7f8360199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8365f33700 1 -- 172.21.15.33:0/383342779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83600ff700 msgr2=0x7f8360198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8365f33700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83600ff700 0x7f8360198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8365f33700 1 -- 172.21.15.33:0/383342779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100060 msgr2=0x7f8360199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8365f33700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100060 0x7f8360199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8365f33700 1 -- 172.21.15.33:0/383342779 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c0095f0 con 0x7f8360100a20 2024-05-08T21:22:49.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8365732700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83600ff700 0x7f8360198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:49.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8365f33700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8360100a20 0x7f83601a0470 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f835800efc0 tx=0x7f835800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f83567fc700 1 -- 172.21.15.33:0/383342779 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358018070 con 0x7f8360100a20 2024-05-08T21:22:49.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f83567fc700 1 -- 172.21.15.33:0/383342779 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f835800f040 con 0x7f8360100a20 2024-05-08T21:22:49.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f8367996700 1 -- 172.21.15.33:0/383342779 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83601a0f90 con 0x7f8360100a20 2024-05-08T21:22:49.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:48.999+0000 7f83567fc700 1 -- 172.21.15.33:0/383342779 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835800cba0 con 0x7f8360100a20 2024-05-08T21:22:49.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.000+0000 7f8367996700 1 -- 172.21.15.33:0/383342779 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83601a1410 con 0x7f8360100a20 2024-05-08T21:22:49.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.002+0000 7f83567fc700 1 -- 172.21.15.33:0/383342779 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8358003a00 con 0x7f8360100a20 2024-05-08T21:22:49.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.003+0000 7f83567fc700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f834c0a11a0 0x7f834c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.003+0000 7f8365732700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f834c0a11a0 0x7f834c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.003+0000 7f83567fc700 1 -- 172.21.15.33:0/383342779 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f83580cc700 con 0x7f8360100a20 2024-05-08T21:22:49.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.004+0000 7f8365732700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f834c0a11a0 0x7f834c0a3660 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f835c00b2e0 tx=0x7f835c00b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.004+0000 7f8373b50b80 1 Processor -- start 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.004+0000 7f8373b50b80 1 -- start start 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.004+0000 7f8373b50b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 0x55f819dbf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8373b50b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dbfea0 0x55f819dc0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8373b50b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dc0860 0x55f819dc0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8373b50b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f819d16150 con 0x55f819dbfea0 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8373b50b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f819d13b10 con 0x55f819dc0860 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8373b50b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f819d0f400 con 0x55f819dbf540 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8365732700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 0x55f819dbf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8365732700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 0x55f819dbf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:40318/0 (socket says 172.21.15.33:40318) 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8364f31700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dbfea0 0x55f819dc0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8365732700 1 -- 172.21.15.33:0/3016513843 learned_addr learned my addr 172.21.15.33:0/3016513843 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8365f33700 1 --2- 172.21.15.33:0/3016513843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dc0860 0x55f819dc0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8365f33700 1 -- 172.21.15.33:0/3016513843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 msgr2=0x55f819dbf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8365f33700 1 --2- 172.21.15.33:0/3016513843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 0x55f819dbf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8365f33700 1 -- 172.21.15.33:0/3016513843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dbfea0 msgr2=0x55f819dc0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8365f33700 1 --2- 172.21.15.33:0/3016513843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dbfea0 0x55f819dc0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8365f33700 1 -- 172.21.15.33:0/3016513843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f819b903b0 con 0x55f819dc0860 2024-05-08T21:22:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8365732700 1 --2- 172.21.15.33:0/3016513843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 0x55f819dbf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.005+0000 7f8365f33700 1 --2- 172.21.15.33:0/3016513843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dc0860 0x55f819dc0ce0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f83580c9e10 tx=0x7f83580993e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77816c93c632041 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.006+0000 7f834b7fe700 1 -- 172.21.15.33:0/3016513843 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835801b320 con 0x55f819dc0860 2024-05-08T21:22:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.006+0000 7f834b7fe700 1 -- 172.21.15.33:0/3016513843 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8358099f20 con 0x55f819dc0860 2024-05-08T21:22:49.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.006+0000 7f834b7fe700 1 -- 172.21.15.33:0/3016513843 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83580cc420 con 0x55f819dc0860 2024-05-08T21:22:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.006+0000 7f8373b50b80 1 -- 172.21.15.33:0/3016513843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dc0860 msgr2=0x55f819dc0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.006+0000 7f8373b50b80 1 --2- 172.21.15.33:0/3016513843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dc0860 0x55f819dc0ce0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f83580c9e10 tx=0x7f83580993e0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.007+0000 7f8373b50b80 1 -- 172.21.15.33:0/3016513843 shutdown_connections 2024-05-08T21:22:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.007+0000 7f8373b50b80 1 --2- 172.21.15.33:0/3016513843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dc0860 0x55f819dc0ce0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.007+0000 7f8373b50b80 1 --2- 172.21.15.33:0/3016513843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 0x55f819dbf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.007+0000 7f8373b50b80 1 --2- 172.21.15.33:0/3016513843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dbfea0 0x55f819dc0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.007+0000 7f8373b50b80 1 -- 172.21.15.33:0/3016513843 >> 172.21.15.33:0/3016513843 conn(0x55f819d1d210 msgr2=0x55f819db9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.007+0000 7f8373b50b80 1 -- 172.21.15.33:0/3016513843 shutdown_connections 2024-05-08T21:22:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.007+0000 7f8373b50b80 1 -- 172.21.15.33:0/3016513843 wait complete. 2024-05-08T21:22:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.007+0000 7f8373b50b80 1 -- start start 2024-05-08T21:22:49.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.007+0000 7f8373b50b80 1 Processor -- start 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8373b50b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dbf540 0x55f819dc8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8373b50b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 0x55f819dc93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8373b50b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dc0860 0x55f819dd03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8373b50b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f819cab560 con 0x55f819dbf540 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8373b50b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f819cab200 con 0x55f819dbfea0 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8373b50b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f819b926a0 con 0x55f819dc0860 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8373b50b80 10 client.0 _ll_register_callbacks cb 0x55f819cb4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8365732700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dbf540 0x55f819dc8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8365f33700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dc0860 0x55f819dd03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8365732700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dbf540 0x55f819dc8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42588/0 (socket says 172.21.15.33:42588) 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8365732700 1 -- 172.21.15.33:0/335238787 learned_addr learned my addr 172.21.15.33:0/335238787 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8364f31700 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 0x55f819dc93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8365732700 1 -- 172.21.15.33:0/335238787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dc0860 msgr2=0x55f819dd03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8365732700 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dc0860 0x55f819dd03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8365732700 1 -- 172.21.15.33:0/335238787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 msgr2=0x55f819dc93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8365732700 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 0x55f819dc93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8365732700 1 -- 172.21.15.33:0/335238787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836010be90 con 0x55f819dbf540 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8365f33700 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dc0860 0x55f819dd03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.008+0000 7f8364f31700 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 0x55f819dc93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8365732700 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dbf540 0x55f819dc8e60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f835c003f60 tx=0x7f835c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c791b09d3d89fcf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8348ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8348ff9700 1 -- 172.21.15.33:0/335238787 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c02b050 con 0x55f819dbf540 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8348ff9700 1 -- 172.21.15.33:0/335238787 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f835c004700 con 0x55f819dbf540 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8373b50b80 1 -- 172.21.15.33:0/335238787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f819c68270 con 0x55f819dbf540 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8348ff9700 1 -- 172.21.15.33:0/335238787 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f835c004900 con 0x55f819dbf540 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8373b50b80 10 client.16605 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8373b50b80 1 -- 172.21.15.33:0/335238787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f83601a1410 con 0x55f819dbf540 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8348ff9700 1 -- 172.21.15.33:0/335238787 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f819c68270 con 0x55f819dbf540 2024-05-08T21:22:49.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8348ff9700 1 -- 172.21.15.33:0/335238787 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f835c00f9f0 con 0x55f819dbf540 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8373b50b80 10 client.16605 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8373b50b80 10 client.16605 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.009+0000 7f8373b50b80 20 client.16605 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.010+0000 7f8373b50b80 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f819dd4cc0 0x55f819dd70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.010+0000 7f8373b50b80 4 client.16605 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.010+0000 7f8373b50b80 1 -- 172.21.15.33:0/335238787 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f819d8e810 con 0x55f819dd4cc0 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.010+0000 7f8364f31700 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f819dd4cc0 0x55f819dd70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.010+0000 7f8364f31700 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f819dd4cc0 0x55f819dd70e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a0835d179eb887a server_cookie=6cf16f2a89ae99c7 in_seq=0 out_seq=0 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.010+0000 7f8348ff9700 10 client.16605 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8348ff9700 1 -- 172.21.15.33:0/335238787 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f819d8e810 con 0x55f819dd4cc0 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8348ff9700 10 client.16605 handle_command_reply: tid=0 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8373b50b80 1 client.16605 shutdown 2024-05-08T21:22:49.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8373b50b80 1 -- 172.21.15.33:0/335238787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dbf540 msgr2=0x55f819dc8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8373b50b80 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dbf540 0x55f819dc8e60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f835c003f60 tx=0x7f835c01a050 comp rx=0 tx=0).stop 2024-05-08T21:22:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8373b50b80 1 -- 172.21.15.33:0/335238787 shutdown_connections 2024-05-08T21:22:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8373b50b80 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f819dd4cc0 0x55f819dd70e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8373b50b80 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dc0860 0x55f819dd03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8373b50b80 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dbf540 0x55f819dc8e60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8348ff9700 0 client.16605 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:22:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8373b50b80 1 --2- 172.21.15.33:0/335238787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 0x55f819dc93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8373b50b80 1 -- 172.21.15.33:0/335238787 >> 172.21.15.33:0/335238787 conn(0x55f819d1d210 msgr2=0x55f819db95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8373b50b80 1 -- 172.21.15.33:0/335238787 shutdown_connections 2024-05-08T21:22:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8373b50b80 1 -- 172.21.15.33:0/335238787 wait complete. 2024-05-08T21:22:49.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.011+0000 7f8373b50b80 20 client.16605 trim_cache size 0 max 16384 2024-05-08T21:22:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8373b50b80 1 Processor -- start 2024-05-08T21:22:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8373b50b80 1 -- start start 2024-05-08T21:22:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8373b50b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbf540 0x55f819dd0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8373b50b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbfea0 0x55f819dd03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8373b50b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dc0860 0x55f819dc8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8373b50b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f819db6dc0 con 0x55f819dc0860 2024-05-08T21:22:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8373b50b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f819ddc2f0 con 0x55f819dbf540 2024-05-08T21:22:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8373b50b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f819dbb830 con 0x55f819dbfea0 2024-05-08T21:22:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8365732700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbf540 0x55f819dd0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8365732700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbf540 0x55f819dd0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51174/0 (socket says 172.21.15.33:51174) 2024-05-08T21:22:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8365732700 1 -- 172.21.15.33:0/2378024395 learned_addr learned my addr 172.21.15.33:0/2378024395 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8364f31700 1 --2- 172.21.15.33:0/2378024395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbfea0 0x55f819dd03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.027+0000 7f8365f33700 1 --2- 172.21.15.33:0/2378024395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dc0860 0x55f819dc8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.028+0000 7f8365732700 1 -- 172.21.15.33:0/2378024395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbfea0 msgr2=0x55f819dd03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.028+0000 7f8365732700 1 --2- 172.21.15.33:0/2378024395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbfea0 0x55f819dd03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.028+0000 7f8365732700 1 -- 172.21.15.33:0/2378024395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dc0860 msgr2=0x55f819dc8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.028+0000 7f8365732700 1 --2- 172.21.15.33:0/2378024395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dc0860 0x55f819dc8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.028+0000 7f8365732700 1 -- 172.21.15.33:0/2378024395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f819b3cb60 con 0x55f819dbf540 2024-05-08T21:22:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.028+0000 7f8365f33700 1 --2- 172.21.15.33:0/2378024395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dc0860 0x55f819dc8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:22:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.028+0000 7f8365732700 1 --2- 172.21.15.33:0/2378024395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbf540 0x55f819dd0a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f835c006010 tx=0x7f835c0094d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fa35e2ffe6d950b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.028+0000 7f834bfff700 1 -- 172.21.15.33:0/2378024395 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c02b070 con 0x55f819dbf540 2024-05-08T21:22:49.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.028+0000 7f834bfff700 1 -- 172.21.15.33:0/2378024395 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f835c02c040 con 0x55f819dbf540 2024-05-08T21:22:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.028+0000 7f834bfff700 1 -- 172.21.15.33:0/2378024395 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c0249f0 con 0x55f819dbf540 2024-05-08T21:22:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.029+0000 7f8373b50b80 1 -- 172.21.15.33:0/2378024395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbf540 msgr2=0x55f819dd0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.029+0000 7f8373b50b80 1 --2- 172.21.15.33:0/2378024395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbf540 0x55f819dd0a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f835c006010 tx=0x7f835c0094d0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.029+0000 7f8373b50b80 1 -- 172.21.15.33:0/2378024395 shutdown_connections 2024-05-08T21:22:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.029+0000 7f8373b50b80 1 --2- 172.21.15.33:0/2378024395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dc0860 0x55f819dc8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.029+0000 7f8373b50b80 1 --2- 172.21.15.33:0/2378024395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbf540 0x55f819dd0a90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.029+0000 7f8373b50b80 1 --2- 172.21.15.33:0/2378024395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbfea0 0x55f819dd03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.029+0000 7f8373b50b80 1 -- 172.21.15.33:0/2378024395 >> 172.21.15.33:0/2378024395 conn(0x55f819d1d210 msgr2=0x55f819dc6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.029+0000 7f8373b50b80 1 -- 172.21.15.33:0/2378024395 shutdown_connections 2024-05-08T21:22:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.029+0000 7f8373b50b80 1 -- 172.21.15.33:0/2378024395 wait complete. 2024-05-08T21:22:49.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.029+0000 7f8373b50b80 1 -- start start 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8373b50b80 1 Processor -- start 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8373b50b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 0x55f819dcbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8373b50b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 0x55f819dcc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8373b50b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dc0860 0x55f819dcf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8373b50b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f819dcf720 con 0x55f819dc0860 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8373b50b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f819dcf890 con 0x55f819dbfea0 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8373b50b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f819dcfa50 con 0x55f819dbf540 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8373b50b80 10 client.0 _ll_register_callbacks cb 0x55f819cb4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8364f31700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 0x55f819dcc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8365732700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 0x55f819dcbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8365f33700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dc0860 0x55f819dcf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8364f31700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 0x55f819dcc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51182/0 (socket says 172.21.15.33:51182) 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8364f31700 1 -- 172.21.15.33:0/1798612332 learned_addr learned my addr 172.21.15.33:0/1798612332 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8365732700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 0x55f819dcbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:40354/0 (socket says 172.21.15.33:40354) 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8365732700 1 -- 172.21.15.33:0/1798612332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 msgr2=0x55f819dcc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8365732700 1 --2- 172.21.15.33:0/1798612332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 0x55f819dcc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.030+0000 7f8365732700 1 -- 172.21.15.33:0/1798612332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dc0860 msgr2=0x55f819dcf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8365732700 1 --2- 172.21.15.33:0/1798612332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dc0860 0x55f819dcf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8365732700 1 -- 172.21.15.33:0/1798612332 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83601a1410 con 0x55f819dbf540 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8364f31700 1 --2- 172.21.15.33:0/1798612332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 0x55f819dcc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8365732700 1 --2- 172.21.15.33:0/1798612332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 0x55f819dcbc80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f835c009350 tx=0x7f835c024490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cfa2ade0441b633 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8348ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8348ff9700 1 -- 172.21.15.33:0/1798612332 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c02b040 con 0x55f819dbf540 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8348ff9700 1 -- 172.21.15.33:0/1798612332 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f835c00f870 con 0x55f819dbf540 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8373b50b80 1 -- 172.21.15.33:0/1798612332 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8358099f20 con 0x55f819dbf540 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8348ff9700 1 -- 172.21.15.33:0/1798612332 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f835c024790 con 0x55f819dbf540 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8373b50b80 10 client.26500 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:49.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8373b50b80 1 -- 172.21.15.33:0/1798612332 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f819cd5560 con 0x55f819dbf540 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8348ff9700 1 -- 172.21.15.33:0/1798612332 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8358099f20 con 0x55f819dbf540 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.031+0000 7f8348ff9700 1 -- 172.21.15.33:0/1798612332 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f835c03a4e0 con 0x55f819dbf540 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.032+0000 7f8373b50b80 10 client.26500 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.032+0000 7f8373b50b80 10 client.26500 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.032+0000 7f8373b50b80 20 client.26500 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.032+0000 7f8373b50b80 1 --2- 172.21.15.33:0/1798612332 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f819e2b840 0x55f819e2dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.032+0000 7f8373b50b80 4 client.26500 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.032+0000 7f8373b50b80 1 -- 172.21.15.33:0/1798612332 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f819d07ec0 con 0x55f819e2b840 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.032+0000 7f8364f31700 1 --2- 172.21.15.33:0/1798612332 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f819e2b840 0x55f819e2dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.032+0000 7f8364f31700 1 --2- 172.21.15.33:0/1798612332 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f819e2b840 0x55f819e2dc30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77da855bf76a63c7 server_cookie=7ad5e0cdf3de2bca in_seq=0 out_seq=0 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.032+0000 7f8348ff9700 10 client.26500 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.032+0000 7f8348ff9700 1 -- 172.21.15.33:0/1798612332 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f819d07ec0 con 0x55f819e2b840 2024-05-08T21:22:49.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.032+0000 7f8348ff9700 10 client.26500 handle_command_reply: tid=0 2024-05-08T21:22:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.032+0000 7f8373b50b80 1 client.26500 shutdown 2024-05-08T21:22:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8373b50b80 1 -- 172.21.15.33:0/1798612332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 msgr2=0x55f819dcbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8373b50b80 1 --2- 172.21.15.33:0/1798612332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 0x55f819dcbc80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f835c009350 tx=0x7f835c024490 comp rx=0 tx=0).stop 2024-05-08T21:22:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8373b50b80 1 -- 172.21.15.33:0/1798612332 shutdown_connections 2024-05-08T21:22:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8373b50b80 1 --2- 172.21.15.33:0/1798612332 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f819e2b840 0x55f819e2dc30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8373b50b80 1 --2- 172.21.15.33:0/1798612332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f819dc0860 0x55f819dcf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8373b50b80 1 --2- 172.21.15.33:0/1798612332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f819dbf540 0x55f819dcbc80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8373b50b80 1 --2- 172.21.15.33:0/1798612332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f819dbfea0 0x55f819dcc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8348ff9700 0 client.26500 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:22:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8373b50b80 1 -- 172.21.15.33:0/1798612332 >> 172.21.15.33:0/1798612332 conn(0x55f819d1d210 msgr2=0x55f819dc8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8373b50b80 1 -- 172.21.15.33:0/1798612332 shutdown_connections 2024-05-08T21:22:49.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8373b50b80 1 -- 172.21.15.33:0/1798612332 wait complete. 2024-05-08T21:22:49.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8373b50b80 20 client.26500 trim_cache size 0 max 16384 2024-05-08T21:22:49.041 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:22:49.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8367996700 1 -- 172.21.15.33:0/383342779 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f834c0a11a0 msgr2=0x7f834c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8367996700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f834c0a11a0 0x7f834c0a3660 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f835c00b2e0 tx=0x7f835c00b610 comp rx=0 tx=0).stop 2024-05-08T21:22:49.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8367996700 1 -- 172.21.15.33:0/383342779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8360100a20 msgr2=0x7f83601a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8367996700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8360100a20 0x7f83601a0470 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f835800efc0 tx=0x7f835800c3a0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8367996700 1 -- 172.21.15.33:0/383342779 shutdown_connections 2024-05-08T21:22:49.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8367996700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8360100a20 0x7f83601a0470 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.033+0000 7f8367996700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f834c0a11a0 0x7f834c0a3660 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.034+0000 7f8367996700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83600ff700 0x7f8360198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.034+0000 7f8367996700 1 --2- 172.21.15.33:0/383342779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8360100060 0x7f8360199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.034+0000 7f8367996700 1 -- 172.21.15.33:0/383342779 >> 172.21.15.33:0/383342779 conn(0x7f83600fb300 msgr2=0x7f83600fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.034+0000 7f8367996700 1 -- 172.21.15.33:0/383342779 shutdown_connections 2024-05-08T21:22:49.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.034+0000 7f8367996700 1 -- 172.21.15.33:0/383342779 wait complete. 2024-05-08T21:22:49.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:22:49.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.207+0000 7f1567217700 1 -- 172.21.15.33:0/3694537928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1560100f50 msgr2=0x7f15601013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.207+0000 7f1567217700 1 --2- 172.21.15.33:0/3694537928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1560100f50 0x7f15601013b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f154c009990 tx=0x7f154c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.207+0000 7f1567217700 1 -- 172.21.15.33:0/3694537928 shutdown_connections 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.207+0000 7f1567217700 1 --2- 172.21.15.33:0/3694537928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15601018f0 0x7f15601081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.207+0000 7f1567217700 1 --2- 172.21.15.33:0/3694537928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15600ffd60 0x7f1560100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.207+0000 7f1567217700 1 --2- 172.21.15.33:0/3694537928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1560100f50 0x7f15601013b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.207+0000 7f1567217700 1 -- 172.21.15.33:0/3694537928 >> 172.21.15.33:0/3694537928 conn(0x7f15600fb290 msgr2=0x7f15600fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.207+0000 7f1567217700 1 -- 172.21.15.33:0/3694537928 shutdown_connections 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.207+0000 7f1567217700 1 -- 172.21.15.33:0/3694537928 wait complete. 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.207+0000 7f1567217700 1 Processor -- start 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.207+0000 7f1567217700 1 -- start start 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f1567217700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15600ffd60 0x7f1560068ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f1567217700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1560100f50 0x7f156019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f1567217700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15601018f0 0x7f156019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f1567217700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156019e850 con 0x7f15600ffd60 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f1567217700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156019e9c0 con 0x7f1560100f50 2024-05-08T21:22:49.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f1567217700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156019eb80 con 0x7f15601018f0 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f1564fb3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15600ffd60 0x7f1560068ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f155ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1560100f50 0x7f156019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f1564fb3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15600ffd60 0x7f1560068ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42624/0 (socket says 172.21.15.33:42624) 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f1564fb3700 1 -- 172.21.15.33:0/909526325 learned_addr learned my addr 172.21.15.33:0/909526325 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f15657b4700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15601018f0 0x7f156019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f155ffff700 1 -- 172.21.15.33:0/909526325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15601018f0 msgr2=0x7f156019e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f155ffff700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15601018f0 0x7f156019e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f155ffff700 1 -- 172.21.15.33:0/909526325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15600ffd60 msgr2=0x7f1560068ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f155ffff700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15600ffd60 0x7f1560068ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f155ffff700 1 -- 172.21.15.33:0/909526325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0095f0 con 0x7f1560100f50 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f155ffff700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1560100f50 0x7f156019ba60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f154c005950 tx=0x7f154c00b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.208+0000 7f155dffb700 1 -- 172.21.15.33:0/909526325 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c01d070 con 0x7f1560100f50 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.209+0000 7f155dffb700 1 -- 172.21.15.33:0/909526325 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f154c00bed0 con 0x7f1560100f50 2024-05-08T21:22:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.209+0000 7f1567217700 1 -- 172.21.15.33:0/909526325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156019ee00 con 0x7f1560100f50 2024-05-08T21:22:49.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.209+0000 7f155dffb700 1 -- 172.21.15.33:0/909526325 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c004ea0 con 0x7f1560100f50 2024-05-08T21:22:49.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.209+0000 7f1567217700 1 -- 172.21.15.33:0/909526325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156019f1c0 con 0x7f1560100f50 2024-05-08T21:22:49.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.211+0000 7f155dffb700 1 -- 172.21.15.33:0/909526325 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f154c004890 con 0x7f1560100f50 2024-05-08T21:22:49.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.212+0000 7f155dffb700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15500a1120 0x7f15500a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.213+0000 7f1564fb3700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15500a1120 0x7f15500a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.213+0000 7f155dffb700 1 -- 172.21.15.33:0/909526325 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f154c0cd890 con 0x7f1560100f50 2024-05-08T21:22:49.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.213+0000 7f15733d1b80 1 Processor -- start 2024-05-08T21:22:49.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.213+0000 7f1564fb3700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15500a1120 0x7f15500a35e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1554005950 tx=0x7f1554009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.213+0000 7f15733d1b80 1 -- start start 2024-05-08T21:22:49.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.213+0000 7f15733d1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c2540 0x55d40a2c2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.213+0000 7f15733d1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 0x55d40a2c3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.213+0000 7f15733d1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c3860 0x55d40a2c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.213+0000 7f15733d1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40a219150 con 0x55d40a2c2540 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.213+0000 7f15733d1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40a216b10 con 0x55d40a2c2ea0 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.213+0000 7f15733d1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40a212400 con 0x55d40a2c3860 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.213+0000 7f1564fb3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c2540 0x55d40a2c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f1564fb3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c2540 0x55d40a2c2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42628/0 (socket says 172.21.15.33:42628) 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f15657b4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c3860 0x55d40a2c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f1564fb3700 1 -- 172.21.15.33:0/2649996906 learned_addr learned my addr 172.21.15.33:0/2649996906 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f155ffff700 1 --2- 172.21.15.33:0/2649996906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 0x55d40a2c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f15657b4700 1 -- 172.21.15.33:0/2649996906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 msgr2=0x55d40a2c3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f15657b4700 1 --2- 172.21.15.33:0/2649996906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 0x55d40a2c3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f15657b4700 1 -- 172.21.15.33:0/2649996906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c2540 msgr2=0x55d40a2c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f15657b4700 1 --2- 172.21.15.33:0/2649996906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c2540 0x55d40a2c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f15657b4700 1 -- 172.21.15.33:0/2649996906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d40a0933b0 con 0x55d40a2c3860 2024-05-08T21:22:49.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f155ffff700 1 --2- 172.21.15.33:0/2649996906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 0x55d40a2c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:49.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f1564fb3700 1 --2- 172.21.15.33:0/2649996906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c2540 0x55d40a2c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:49.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f15657b4700 1 --2- 172.21.15.33:0/2649996906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c3860 0x55d40a2c3ce0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f15580115b0 tx=0x7f1558009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4e00d3c0f8ce1c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f154affd700 1 -- 172.21.15.33:0/2649996906 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155800e040 con 0x55d40a2c3860 2024-05-08T21:22:49.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f154affd700 1 -- 172.21.15.33:0/2649996906 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f155800f040 con 0x55d40a2c3860 2024-05-08T21:22:49.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.214+0000 7f154affd700 1 -- 172.21.15.33:0/2649996906 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558004330 con 0x55d40a2c3860 2024-05-08T21:22:49.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.215+0000 7f15733d1b80 1 -- 172.21.15.33:0/2649996906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c3860 msgr2=0x55d40a2c3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.215+0000 7f15733d1b80 1 --2- 172.21.15.33:0/2649996906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c3860 0x55d40a2c3ce0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f15580115b0 tx=0x7f1558009920 comp rx=0 tx=0).stop 2024-05-08T21:22:49.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.215+0000 7f15733d1b80 1 -- 172.21.15.33:0/2649996906 shutdown_connections 2024-05-08T21:22:49.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.215+0000 7f15733d1b80 1 --2- 172.21.15.33:0/2649996906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c3860 0x55d40a2c3ce0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.215+0000 7f15733d1b80 1 --2- 172.21.15.33:0/2649996906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c2540 0x55d40a2c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.215+0000 7f15733d1b80 1 --2- 172.21.15.33:0/2649996906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 0x55d40a2c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.215+0000 7f15733d1b80 1 -- 172.21.15.33:0/2649996906 >> 172.21.15.33:0/2649996906 conn(0x55d40a220210 msgr2=0x55d40a2bcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.215+0000 7f15733d1b80 1 -- 172.21.15.33:0/2649996906 shutdown_connections 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.215+0000 7f15733d1b80 1 -- 172.21.15.33:0/2649996906 wait complete. 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.216+0000 7f15733d1b80 1 -- start start 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.216+0000 7f15733d1b80 1 Processor -- start 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.216+0000 7f15733d1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 0x55d40a2cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.216+0000 7f15733d1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 0x55d40a2cc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.216+0000 7f15733d1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.216+0000 7f15733d1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40a1ae560 con 0x55d40a2c3860 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.216+0000 7f15733d1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40a1ae200 con 0x55d40a2c2540 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.216+0000 7f15733d1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40a0956a0 con 0x55d40a2c2ea0 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.216+0000 7f15733d1b80 10 client.0 _ll_register_callbacks cb 0x55d40a1b7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.216+0000 7f1564fb3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 0x55d40a2cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f15657b4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f1564fb3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 0x55d40a2cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51208/0 (socket says 172.21.15.33:51208) 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f1564fb3700 1 -- 172.21.15.33:0/3141254347 learned_addr learned my addr 172.21.15.33:0/3141254347 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f155ffff700 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 0x55d40a2cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f1564fb3700 1 -- 172.21.15.33:0/3141254347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 msgr2=0x55d40a2cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f1564fb3700 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 0x55d40a2cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f1564fb3700 1 -- 172.21.15.33:0/3141254347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 msgr2=0x55d40a2d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f1564fb3700 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f1564fb3700 1 -- 172.21.15.33:0/3141254347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156019f1c0 con 0x55d40a2c2540 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f155ffff700 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 0x55d40a2cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f1564fb3700 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 0x55d40a2cbe00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f155400ec80 tx=0x7f15540047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be6ff7b8582ccf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f153ffff700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f153ffff700 1 -- 172.21.15.33:0/3141254347 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554022070 con 0x55d40a2c2540 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.217+0000 7f153ffff700 1 -- 172.21.15.33:0/3141254347 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f155401dae0 con 0x55d40a2c2540 2024-05-08T21:22:49.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.218+0000 7f15733d1b80 1 -- 172.21.15.33:0/3141254347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d40a16b270 con 0x55d40a2c2540 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.218+0000 7f153ffff700 1 -- 172.21.15.33:0/3141254347 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f155401dca0 con 0x55d40a2c2540 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.218+0000 7f15733d1b80 10 client.26492 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.218+0000 7f15733d1b80 1 -- 172.21.15.33:0/3141254347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f1558011230 con 0x55d40a2c2540 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.218+0000 7f153ffff700 1 -- 172.21.15.33:0/3141254347 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d40a16b270 con 0x55d40a2c2540 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.218+0000 7f153ffff700 1 -- 172.21.15.33:0/3141254347 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f1554031420 con 0x55d40a2c2540 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.218+0000 7f15733d1b80 10 client.26492 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.218+0000 7f15733d1b80 10 client.26492 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.218+0000 7f15733d1b80 20 client.26492 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.218+0000 7f15733d1b80 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d40a2d7c40 0x55d40a2da060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.218+0000 7f15733d1b80 4 client.26492 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.218+0000 7f15733d1b80 1 -- 172.21.15.33:0/3141254347 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d40a291810 con 0x55d40a2d7c40 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.219+0000 7f155ffff700 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d40a2d7c40 0x55d40a2da060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.219+0000 7f153ffff700 10 client.26492 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:22:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.219+0000 7f155ffff700 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d40a2d7c40 0x55d40a2da060 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0d5572709478cf9 server_cookie=52211bc12ec723a1 in_seq=0 out_seq=0 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f153ffff700 1 -- 172.21.15.33:0/3141254347 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d40a291810 con 0x55d40a2d7c40 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f153ffff700 10 client.26492 handle_command_reply: tid=0 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f15733d1b80 1 client.26492 shutdown 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f15733d1b80 1 -- 172.21.15.33:0/3141254347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 msgr2=0x55d40a2cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f15733d1b80 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 0x55d40a2cbe00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f155400ec80 tx=0x7f15540047c0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f15733d1b80 1 -- 172.21.15.33:0/3141254347 shutdown_connections 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f15733d1b80 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d40a2d7c40 0x55d40a2da060 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f15733d1b80 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f15733d1b80 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 0x55d40a2cbe00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f153ffff700 0 client.26492 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f15733d1b80 1 --2- 172.21.15.33:0/3141254347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 0x55d40a2cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f15733d1b80 1 -- 172.21.15.33:0/3141254347 >> 172.21.15.33:0/3141254347 conn(0x55d40a220210 msgr2=0x55d40a2bc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f15733d1b80 1 -- 172.21.15.33:0/3141254347 shutdown_connections 2024-05-08T21:22:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f15733d1b80 1 -- 172.21.15.33:0/3141254347 wait complete. 2024-05-08T21:22:49.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.220+0000 7f15733d1b80 20 client.26492 trim_cache size 0 max 16384 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f15733d1b80 1 Processor -- start 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f15733d1b80 1 -- start start 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f15733d1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2540 0x55d40a2da030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f15733d1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 0x55d40a2d39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f15733d1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f15733d1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40a2ccfa0 con 0x55d40a2c3860 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f15733d1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40a2df250 con 0x55d40a2c2ea0 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f15733d1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40a2d7940 con 0x55d40a2c2540 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f1564fb3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2540 0x55d40a2da030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f155ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 0x55d40a2d39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f15657b4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f1564fb3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2540 0x55d40a2da030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:40390/0 (socket says 172.21.15.33:40390) 2024-05-08T21:22:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f155ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 0x55d40a2d39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51212/0 (socket says 172.21.15.33:51212) 2024-05-08T21:22:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.237+0000 7f1564fb3700 1 -- 172.21.15.33:0/4019656795 learned_addr learned my addr 172.21.15.33:0/4019656795 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.238+0000 7f155ffff700 1 -- 172.21.15.33:0/4019656795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2540 msgr2=0x55d40a2da030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.238+0000 7f155ffff700 1 --2- 172.21.15.33:0/4019656795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2540 0x55d40a2da030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.238+0000 7f155ffff700 1 -- 172.21.15.33:0/4019656795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 msgr2=0x55d40a2d32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.238+0000 7f155ffff700 1 --2- 172.21.15.33:0/4019656795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.238+0000 7f155ffff700 1 -- 172.21.15.33:0/4019656795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d40a03fb60 con 0x55d40a2c2ea0 2024-05-08T21:22:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.238+0000 7f1564fb3700 1 --2- 172.21.15.33:0/4019656795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2540 0x55d40a2da030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.238+0000 7f15657b4700 1 --2- 172.21.15.33:0/4019656795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.238+0000 7f155ffff700 1 --2- 172.21.15.33:0/4019656795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 0x55d40a2d39e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f154c09a940 tx=0x7f154c0ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba96c5eb1705bad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.238+0000 7f154b7fe700 1 -- 172.21.15.33:0/4019656795 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c09e7e0 con 0x55d40a2c2ea0 2024-05-08T21:22:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.238+0000 7f154b7fe700 1 -- 172.21.15.33:0/4019656795 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f154c0a2da0 con 0x55d40a2c2ea0 2024-05-08T21:22:49.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.238+0000 7f154b7fe700 1 -- 172.21.15.33:0/4019656795 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c09a100 con 0x55d40a2c2ea0 2024-05-08T21:22:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.239+0000 7f15733d1b80 1 -- 172.21.15.33:0/4019656795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 msgr2=0x55d40a2d39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.239+0000 7f15733d1b80 1 --2- 172.21.15.33:0/4019656795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 0x55d40a2d39e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f154c09a940 tx=0x7f154c0ba040 comp rx=0 tx=0).stop 2024-05-08T21:22:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.239+0000 7f15733d1b80 1 -- 172.21.15.33:0/4019656795 shutdown_connections 2024-05-08T21:22:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.239+0000 7f15733d1b80 1 --2- 172.21.15.33:0/4019656795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.239+0000 7f15733d1b80 1 --2- 172.21.15.33:0/4019656795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2540 0x55d40a2da030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.239+0000 7f15733d1b80 1 --2- 172.21.15.33:0/4019656795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2ea0 0x55d40a2d39e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.239+0000 7f15733d1b80 1 -- 172.21.15.33:0/4019656795 >> 172.21.15.33:0/4019656795 conn(0x55d40a220210 msgr2=0x55d40a2c3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.239+0000 7f15733d1b80 1 -- 172.21.15.33:0/4019656795 shutdown_connections 2024-05-08T21:22:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.239+0000 7f15733d1b80 1 -- 172.21.15.33:0/4019656795 wait complete. 2024-05-08T21:22:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.239+0000 7f15733d1b80 1 -- start start 2024-05-08T21:22:49.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.239+0000 7f15733d1b80 1 Processor -- start 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f15733d1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 0x55d40a2ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f15733d1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 0x55d40a2cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f15733d1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f15733d1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40a2d2460 con 0x55d40a2c3860 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f15733d1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40a2d25d0 con 0x55d40a2c2540 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f15733d1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d40a2d2790 con 0x55d40a2c2ea0 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f15733d1b80 10 client.0 _ll_register_callbacks cb 0x55d40a1b7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f155ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 0x55d40a2cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f1564fb3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 0x55d40a2ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f155ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 0x55d40a2cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:40394/0 (socket says 172.21.15.33:40394) 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f155ffff700 1 -- 172.21.15.33:0/784582721 learned_addr learned my addr 172.21.15.33:0/784582721 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f1564fb3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 0x55d40a2ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51214/0 (socket says 172.21.15.33:51214) 2024-05-08T21:22:49.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f15657b4700 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f155ffff700 1 -- 172.21.15.33:0/784582721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 msgr2=0x55d40a2ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f155ffff700 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 0x55d40a2ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f155ffff700 1 -- 172.21.15.33:0/784582721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 msgr2=0x55d40a2d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f155ffff700 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.240+0000 7f155ffff700 1 -- 172.21.15.33:0/784582721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1558011230 con 0x55d40a2c2ea0 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f1564fb3700 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 0x55d40a2ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f155ffff700 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 0x55d40a2cf020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f154c09b010 tx=0x7f154c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=950cf3a5b66ec46b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f153ffff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f153ffff700 1 -- 172.21.15.33:0/784582721 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c099ea0 con 0x55d40a2c2ea0 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f153ffff700 1 -- 172.21.15.33:0/784582721 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f154c09a000 con 0x55d40a2c2ea0 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f15733d1b80 1 -- 172.21.15.33:0/784582721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f155800f040 con 0x55d40a2c2ea0 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f153ffff700 1 -- 172.21.15.33:0/784582721 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f154c0a2910 con 0x55d40a2c2ea0 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f15733d1b80 10 client.26518 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f15733d1b80 1 -- 172.21.15.33:0/784582721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d40a0933b0 con 0x55d40a2c2ea0 2024-05-08T21:22:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f153ffff700 1 -- 172.21.15.33:0/784582721 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f155800f040 con 0x55d40a2c2ea0 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f153ffff700 1 -- 172.21.15.33:0/784582721 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f154c099910 con 0x55d40a2c2ea0 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f15733d1b80 10 client.26518 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f15733d1b80 10 client.26518 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f15733d1b80 20 client.26518 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.241+0000 7f15733d1b80 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d40a32e4a0 0x55d40a3308c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f15733d1b80 4 client.26518 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f15733d1b80 1 -- 172.21.15.33:0/784582721 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d40a20aec0 con 0x55d40a32e4a0 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f1564fb3700 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d40a32e4a0 0x55d40a3308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f1564fb3700 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d40a32e4a0 0x55d40a3308c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=553b3b61f283ae5a server_cookie=132b3b5aa9deccf9 in_seq=0 out_seq=0 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f153ffff700 10 client.26518 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f153ffff700 1 -- 172.21.15.33:0/784582721 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d40a20aec0 con 0x55d40a32e4a0 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f153ffff700 10 client.26518 handle_command_reply: tid=0 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f15733d1b80 1 client.26518 shutdown 2024-05-08T21:22:49.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f15733d1b80 1 -- 172.21.15.33:0/784582721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 msgr2=0x55d40a2cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f15733d1b80 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 0x55d40a2cf020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f154c09b010 tx=0x7f154c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f15733d1b80 1 -- 172.21.15.33:0/784582721 shutdown_connections 2024-05-08T21:22:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f15733d1b80 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d40a32e4a0 0x55d40a3308c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f15733d1b80 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40a2c3860 0x55d40a2d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f15733d1b80 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d40a2c2540 0x55d40a2ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f153ffff700 0 client.26518 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:22:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.242+0000 7f15733d1b80 1 --2- 172.21.15.33:0/784582721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d40a2c2ea0 0x55d40a2cf020 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.243+0000 7f15733d1b80 1 -- 172.21.15.33:0/784582721 >> 172.21.15.33:0/784582721 conn(0x55d40a220210 msgr2=0x55d40a2cb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.243+0000 7f15733d1b80 1 -- 172.21.15.33:0/784582721 shutdown_connections 2024-05-08T21:22:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.243+0000 7f15733d1b80 1 -- 172.21.15.33:0/784582721 wait complete. 2024-05-08T21:22:49.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.243+0000 7f15733d1b80 20 client.26518 trim_cache size 0 max 16384 2024-05-08T21:22:49.251 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:22:49.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.243+0000 7f1567217700 1 -- 172.21.15.33:0/909526325 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15500a1120 msgr2=0x7f15500a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.243+0000 7f1567217700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15500a1120 0x7f15500a35e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1554005950 tx=0x7f1554009260 comp rx=0 tx=0).stop 2024-05-08T21:22:49.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.243+0000 7f1567217700 1 -- 172.21.15.33:0/909526325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1560100f50 msgr2=0x7f156019ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.243+0000 7f1567217700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1560100f50 0x7f156019ba60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f154c005950 tx=0x7f154c00b9c0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.244+0000 7f1567217700 1 -- 172.21.15.33:0/909526325 shutdown_connections 2024-05-08T21:22:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.244+0000 7f1567217700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15601018f0 0x7f156019e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.244+0000 7f1567217700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15600ffd60 0x7f1560068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.244+0000 7f1567217700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15500a1120 0x7f15500a35e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.244+0000 7f1567217700 1 --2- 172.21.15.33:0/909526325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1560100f50 0x7f156019ba60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.244+0000 7f1567217700 1 -- 172.21.15.33:0/909526325 >> 172.21.15.33:0/909526325 conn(0x7f15600fb290 msgr2=0x7f1560104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.244+0000 7f1567217700 1 -- 172.21.15.33:0/909526325 shutdown_connections 2024-05-08T21:22:49.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.244+0000 7f1567217700 1 -- 172.21.15.33:0/909526325 wait complete. 2024-05-08T21:22:49.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:22:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.416+0000 7f9d70952700 1 -- 172.21.15.33:0/2005389370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6c100f50 msgr2=0x7f9d6c1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.416+0000 7f9d70952700 1 --2- 172.21.15.33:0/2005389370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6c100f50 0x7f9d6c1013b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d600099c0 tx=0x7f9d60009cd0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.416+0000 7f9d70952700 1 -- 172.21.15.33:0/2005389370 shutdown_connections 2024-05-08T21:22:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.416+0000 7f9d70952700 1 --2- 172.21.15.33:0/2005389370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c1018f0 0x7f9d6c1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.416+0000 7f9d70952700 1 --2- 172.21.15.33:0/2005389370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c0ffd60 0x7f9d6c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.416+0000 7f9d70952700 1 --2- 172.21.15.33:0/2005389370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6c100f50 0x7f9d6c1013b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.416+0000 7f9d70952700 1 -- 172.21.15.33:0/2005389370 >> 172.21.15.33:0/2005389370 conn(0x7f9d6c0fb290 msgr2=0x7f9d6c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.416+0000 7f9d70952700 1 -- 172.21.15.33:0/2005389370 shutdown_connections 2024-05-08T21:22:49.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.416+0000 7f9d70952700 1 -- 172.21.15.33:0/2005389370 wait complete. 2024-05-08T21:22:49.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d70952700 1 Processor -- start 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d70952700 1 -- start start 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d70952700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c0ffd60 0x7f9d6c196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d70952700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c100f50 0x7f9d6c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d70952700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6c1018f0 0x7f9d6c19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d70952700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19e820 con 0x7f9d6c100f50 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d70952700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19e990 con 0x7f9d6c1018f0 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d70952700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19eb50 con 0x7f9d6c0ffd60 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d69d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c100f50 0x7f9d6c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d6ad9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6c1018f0 0x7f9d6c19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d6a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c0ffd60 0x7f9d6c196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d69d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c100f50 0x7f9d6c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42658/0 (socket says 172.21.15.33:42658) 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.417+0000 7f9d69d9b700 1 -- 172.21.15.33:0/1002596100 learned_addr learned my addr 172.21.15.33:0/1002596100 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.418+0000 7f9d6a59c700 1 -- 172.21.15.33:0/1002596100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6c1018f0 msgr2=0x7f9d6c19e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.418+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6c1018f0 0x7f9d6c19e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.418+0000 7f9d6a59c700 1 -- 172.21.15.33:0/1002596100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c100f50 msgr2=0x7f9d6c197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.418+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c100f50 0x7f9d6c197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.418+0000 7f9d6a59c700 1 -- 172.21.15.33:0/1002596100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d600095f0 con 0x7f9d6c0ffd60 2024-05-08T21:22:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.418+0000 7f9d69d9b700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c100f50 0x7f9d6c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.418+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c0ffd60 0x7f9d6c196d00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00e9c0 tx=0x7f9d5c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.418+0000 7f9d5b7fe700 1 -- 172.21.15.33:0/1002596100 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c004220 con 0x7f9d6c0ffd60 2024-05-08T21:22:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.418+0000 7f9d70952700 1 -- 172.21.15.33:0/1002596100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c19ee30 con 0x7f9d6c0ffd60 2024-05-08T21:22:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.418+0000 7f9d5b7fe700 1 -- 172.21.15.33:0/1002596100 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d5c004380 con 0x7f9d6c0ffd60 2024-05-08T21:22:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.418+0000 7f9d5b7fe700 1 -- 172.21.15.33:0/1002596100 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c003b40 con 0x7f9d6c0ffd60 2024-05-08T21:22:49.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.418+0000 7f9d70952700 1 -- 172.21.15.33:0/1002596100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c19f2b0 con 0x7f9d6c0ffd60 2024-05-08T21:22:49.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.421+0000 7f9d5b7fe700 1 -- 172.21.15.33:0/1002596100 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9d5c003ca0 con 0x7f9d6c0ffd60 2024-05-08T21:22:49.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.422+0000 7f9d5b7fe700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9d540a1120 0x7f9d540a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.422+0000 7f9d5b7fe700 1 -- 172.21.15.33:0/1002596100 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9d5c014070 con 0x7f9d6c0ffd60 2024-05-08T21:22:49.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.423+0000 7f9d7cb0cb80 1 Processor -- start 2024-05-08T21:22:49.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.423+0000 7f9d69d9b700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9d540a1120 0x7f9d540a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.423+0000 7f9d7cb0cb80 1 -- start start 2024-05-08T21:22:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.423+0000 7f9d7cb0cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 0x55e2cf7ec960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.423+0000 7f9d7cb0cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7ed320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.423+0000 7f9d7cb0cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7edce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.423+0000 7f9d7cb0cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cf743150 con 0x55e2cf7ec540 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.423+0000 7f9d7cb0cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cf740b10 con 0x55e2cf7ecea0 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.423+0000 7f9d7cb0cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cf73c400 con 0x55e2cf7ed860 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d6a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 0x55e2cf7ec960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d6a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 0x55e2cf7ec960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42666/0 (socket says 172.21.15.33:42666) 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d6a59c700 1 -- 172.21.15.33:0/324547426 learned_addr learned my addr 172.21.15.33:0/324547426 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d6ad9d700 1 --2- 172.21.15.33:0/324547426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d69d9b700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9d540a1120 0x7f9d540a35e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c1981b0 tx=0x7f9d6001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d69d9b700 1 --2- 172.21.15.33:0/324547426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7ed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d6a59c700 1 -- 172.21.15.33:0/324547426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 msgr2=0x55e2cf7edce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/324547426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7edce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d6a59c700 1 -- 172.21.15.33:0/324547426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ecea0 msgr2=0x55e2cf7ed320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/324547426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7ed320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d6a59c700 1 -- 172.21.15.33:0/324547426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2cf5bd3b0 con 0x55e2cf7ec540 2024-05-08T21:22:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d69d9b700 1 --2- 172.21.15.33:0/324547426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7ed320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:22:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/324547426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 0x55e2cf7ec960 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c096440 tx=0x7f9d5c099b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7381fda7e862094a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d58ff9700 1 -- 172.21.15.33:0/324547426 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c09a640 con 0x55e2cf7ec540 2024-05-08T21:22:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d58ff9700 1 -- 172.21.15.33:0/324547426 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d5c099f40 con 0x55e2cf7ec540 2024-05-08T21:22:49.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.424+0000 7f9d58ff9700 1 -- 172.21.15.33:0/324547426 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c09a1a0 con 0x55e2cf7ec540 2024-05-08T21:22:49.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.425+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/324547426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 msgr2=0x55e2cf7ec960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.425+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/324547426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 0x55e2cf7ec960 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c096440 tx=0x7f9d5c099b00 comp rx=0 tx=0).stop 2024-05-08T21:22:49.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.425+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/324547426 shutdown_connections 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.425+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/324547426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7edce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.426+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/324547426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 0x55e2cf7ec960 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.426+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/324547426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7ed320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.426+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/324547426 >> 172.21.15.33:0/324547426 conn(0x55e2cf74a210 msgr2=0x55e2cf7e6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.426+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/324547426 shutdown_connections 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.426+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/324547426 wait complete. 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.426+0000 7f9d7cb0cb80 1 -- start start 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.426+0000 7f9d7cb0cb80 1 Processor -- start 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d7cb0cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d7cb0cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7f6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d7cb0cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d7cb0cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cf6d8560 con 0x55e2cf7ec540 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d7cb0cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cf6d8200 con 0x55e2cf7ecea0 2024-05-08T21:22:49.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d7cb0cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cf5bf6a0 con 0x55e2cf7ed860 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d7cb0cb80 10 client.0 _ll_register_callbacks cb 0x55e2cf6e1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d6a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d6ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d6a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42680/0 (socket says 172.21.15.33:42680) 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d69d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d6a59c700 1 -- 172.21.15.33:0/1314844321 learned_addr learned my addr 172.21.15.33:0/1314844321 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d6ad9d700 1 -- 172.21.15.33:0/1314844321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ecea0 msgr2=0x55e2cf7f6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d6ad9d700 1 --2- 172.21.15.33:0/1314844321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7f6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d6ad9d700 1 -- 172.21.15.33:0/1314844321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 msgr2=0x55e2cf7f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d6ad9d700 1 --2- 172.21.15.33:0/1314844321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d6ad9d700 1 -- 172.21.15.33:0/1314844321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c19f2b0 con 0x55e2cf7ed860 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.427+0000 7f9d6ad9d700 1 --2- 172.21.15.33:0/1314844321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7fd3b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9d64011e60 tx=0x7f9d64014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e02d3d33f37ed14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:49.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d51ffb700 1 -- 172.21.15.33:0/1314844321 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6400e040 con 0x55e2cf7ed860 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d51ffb700 1 -- 172.21.15.33:0/1314844321 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9d640093a0 con 0x55e2cf7ed860 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/1314844321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2cf695270 con 0x55e2cf7ed860 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d51ffb700 1 -- 172.21.15.33:0/1314844321 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d64004db0 con 0x55e2cf7ed860 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d7cb0cb80 10 client.26530 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/1314844321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e2cf569b60 con 0x55e2cf7ed860 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d51ffb700 1 -- 172.21.15.33:0/1314844321 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e2cf695270 con 0x55e2cf7ed860 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d51ffb700 1 -- 172.21.15.33:0/1314844321 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f9d6401c030 con 0x55e2cf7ed860 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d7cb0cb80 10 client.26530 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d7cb0cb80 10 client.26530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d7cb0cb80 20 client.26530 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/1314844321 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cf801d20 0x55e2cf804140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d7cb0cb80 4 client.26530 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.428+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/1314844321 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2cf7bb810 con 0x55e2cf801d20 2024-05-08T21:22:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.429+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/1314844321 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cf801d20 0x55e2cf804140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.429+0000 7f9d51ffb700 10 client.26530 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.429+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/1314844321 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cf801d20 0x55e2cf804140 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a13252c358e867e7 server_cookie=63311d89ce51943c in_seq=0 out_seq=0 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.429+0000 7f9d51ffb700 1 -- 172.21.15.33:0/1314844321 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e2cf7bb810 con 0x55e2cf801d20 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.429+0000 7f9d51ffb700 10 client.26530 handle_command_reply: tid=0 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.429+0000 7f9d7cb0cb80 1 client.26530 shutdown 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.430+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/1314844321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 msgr2=0x55e2cf7fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.430+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/1314844321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7fd3b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9d64011e60 tx=0x7f9d64014600 comp rx=0 tx=0).stop 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.430+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/1314844321 shutdown_connections 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.430+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/1314844321 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cf801d20 0x55e2cf804140 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.430+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/1314844321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7fd3b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.430+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/1314844321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.430+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/1314844321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.430+0000 7f9d51ffb700 0 client.26530 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.430+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/1314844321 >> 172.21.15.33:0/1314844321 conn(0x55e2cf74a210 msgr2=0x55e2cf7e65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.430+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/1314844321 shutdown_connections 2024-05-08T21:22:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.430+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/1314844321 wait complete. 2024-05-08T21:22:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.430+0000 7f9d7cb0cb80 20 client.26530 trim_cache size 0 max 16384 2024-05-08T21:22:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.445+0000 7f9d7cb0cb80 1 Processor -- start 2024-05-08T21:22:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.445+0000 7f9d7cb0cb80 1 -- start start 2024-05-08T21:22:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d7cb0cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d7cb0cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7fd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d7cb0cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7f5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d7cb0cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cf7f59a0 con 0x55e2cf7ecea0 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d7cb0cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cf809350 con 0x55e2cf7ec540 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d7cb0cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cf7e8570 con 0x55e2cf7ed860 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d69d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d6a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d69d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7fd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42692/0 (socket says 172.21.15.33:42692) 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d6a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51268/0 (socket says 172.21.15.33:51268) 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d69d9b700 1 -- 172.21.15.33:0/487565093 learned_addr learned my addr 172.21.15.33:0/487565093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d6ad9d700 1 --2- 172.21.15.33:0/487565093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d6a59c700 1 -- 172.21.15.33:0/487565093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 msgr2=0x55e2cf7f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/487565093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d6a59c700 1 -- 172.21.15.33:0/487565093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ecea0 msgr2=0x55e2cf7fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/487565093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d6a59c700 1 -- 172.21.15.33:0/487565093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2cf702560 con 0x55e2cf7ec540 2024-05-08T21:22:49.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.446+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/487565093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f4ae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009230 tx=0x7f9d5c003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1c3c57070a2157e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.447+0000 7f9d597fa700 1 -- 172.21.15.33:0/487565093 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c09a640 con 0x55e2cf7ec540 2024-05-08T21:22:49.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.447+0000 7f9d597fa700 1 -- 172.21.15.33:0/487565093 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d5c0982e0 con 0x55e2cf7ec540 2024-05-08T21:22:49.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.447+0000 7f9d597fa700 1 -- 172.21.15.33:0/487565093 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c09a400 con 0x55e2cf7ec540 2024-05-08T21:22:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.447+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/487565093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 msgr2=0x55e2cf7f4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.447+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/487565093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f4ae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009230 tx=0x7f9d5c003f50 comp rx=0 tx=0).stop 2024-05-08T21:22:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/487565093 shutdown_connections 2024-05-08T21:22:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/487565093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/487565093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f4ae0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/487565093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/487565093 >> 172.21.15.33:0/487565093 conn(0x55e2cf74a210 msgr2=0x55e2cf7f07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/487565093 shutdown_connections 2024-05-08T21:22:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/487565093 wait complete. 2024-05-08T21:22:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 -- start start 2024-05-08T21:22:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 Processor -- start 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7fc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cf7fcce0 con 0x55e2cf7ecea0 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cf7fce50 con 0x55e2cf7ec540 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cf7fd010 con 0x55e2cf7ed860 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d7cb0cb80 10 client.0 _ll_register_callbacks cb 0x55e2cf6e1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d6a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.448+0000 7f9d6a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:51282/0 (socket says 172.21.15.33:51282) 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d6a59c700 1 -- 172.21.15.33:0/424076576 learned_addr learned my addr 172.21.15.33:0/424076576 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d6ad9d700 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7fc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d69d9b700 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d6a59c700 1 -- 172.21.15.33:0/424076576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 msgr2=0x55e2cf7fc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7fc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d6a59c700 1 -- 172.21.15.33:0/424076576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ecea0 msgr2=0x55e2cf7f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d6a59c700 1 -- 172.21.15.33:0/424076576 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2cf5bd3b0 con 0x55e2cf7ec540 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d6a59c700 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f9240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0963f0 tx=0x7f9d5c09de90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3a877cb2402ca8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d51ffb700 1 -- 172.21.15.33:0/424076576 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c09a640 con 0x55e2cf7ec540 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d51ffb700 1 -- 172.21.15.33:0/424076576 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9d5c05fe40 con 0x55e2cf7ec540 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.449+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/424076576 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d5c099f40 con 0x55e2cf7ec540 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.450+0000 7f9d51ffb700 1 -- 172.21.15.33:0/424076576 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d5c0996a0 con 0x55e2cf7ec540 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.450+0000 7f9d7cb0cb80 10 client.26516 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.450+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/424076576 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55e2cf5c1cd0 con 0x55e2cf7ec540 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.450+0000 7f9d51ffb700 1 -- 172.21.15.33:0/424076576 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d5c064020 con 0x55e2cf7ec540 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.450+0000 7f9d51ffb700 1 -- 172.21.15.33:0/424076576 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f9d5c0a1fc0 con 0x55e2cf7ec540 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.450+0000 7f9d7cb0cb80 10 client.26516 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.450+0000 7f9d7cb0cb80 10 client.26516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.450+0000 7f9d7cb0cb80 20 client.26516 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.450+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cf858ed0 0x55e2cf85b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.450+0000 7f9d7cb0cb80 4 client.26516 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:22:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.450+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/424076576 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e2cf734ec0 con 0x55e2cf858ed0 2024-05-08T21:22:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.450+0000 7f9d69d9b700 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cf858ed0 0x55e2cf85b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d69d9b700 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cf858ed0 0x55e2cf85b2f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64bb2bc2e580dd4a server_cookie=3e8347fe3fcace7d in_seq=0 out_seq=0 2024-05-08T21:22:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d51ffb700 10 client.26516 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d51ffb700 1 -- 172.21.15.33:0/424076576 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e2cf734ec0 con 0x55e2cf858ed0 2024-05-08T21:22:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d51ffb700 10 client.26516 handle_command_reply: tid=0 2024-05-08T21:22:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d7cb0cb80 1 client.26516 shutdown 2024-05-08T21:22:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/424076576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 msgr2=0x55e2cf7f9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f9240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0963f0 tx=0x7f9d5c09de90 comp rx=0 tx=0).stop 2024-05-08T21:22:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/424076576 shutdown_connections 2024-05-08T21:22:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cf858ed0 0x55e2cf85b2f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cf7ed860 0x55e2cf7fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cf7ec540 0x55e2cf7f9240 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d7cb0cb80 1 --2- 172.21.15.33:0/424076576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cf7ecea0 0x55e2cf7f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d51ffb700 0 client.26516 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/424076576 >> 172.21.15.33:0/424076576 conn(0x55e2cf74a210 msgr2=0x55e2cf7f4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/424076576 shutdown_connections 2024-05-08T21:22:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d7cb0cb80 1 -- 172.21.15.33:0/424076576 wait complete. 2024-05-08T21:22:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.451+0000 7f9d7cb0cb80 20 client.26516 trim_cache size 0 max 16384 2024-05-08T21:22:49.459 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:22:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.452+0000 7f9d70952700 1 -- 172.21.15.33:0/1002596100 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9d540a1120 msgr2=0x7f9d540a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.452+0000 7f9d70952700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9d540a1120 0x7f9d540a35e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c1981b0 tx=0x7f9d6001a040 comp rx=0 tx=0).stop 2024-05-08T21:22:49.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.452+0000 7f9d70952700 1 -- 172.21.15.33:0/1002596100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c0ffd60 msgr2=0x7f9d6c196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.452+0000 7f9d70952700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c0ffd60 0x7f9d6c196d00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00e9c0 tx=0x7f9d5c00ecd0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.452+0000 7f9d70952700 1 -- 172.21.15.33:0/1002596100 shutdown_connections 2024-05-08T21:22:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.452+0000 7f9d70952700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9d540a1120 0x7f9d540a35e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.452+0000 7f9d70952700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6c1018f0 0x7f9d6c19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.452+0000 7f9d70952700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c0ffd60 0x7f9d6c196d00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.452+0000 7f9d70952700 1 --2- 172.21.15.33:0/1002596100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c100f50 0x7f9d6c197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.452+0000 7f9d70952700 1 -- 172.21.15.33:0/1002596100 >> 172.21.15.33:0/1002596100 conn(0x7f9d6c0fb290 msgr2=0x7f9d6c068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.452+0000 7f9d70952700 1 -- 172.21.15.33:0/1002596100 shutdown_connections 2024-05-08T21:22:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:49.452+0000 7f9d70952700 1 -- 172.21.15.33:0/1002596100 wait complete. 2024-05-08T21:22:49.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:49 smithi033 ceph-mon[42528]: pgmap v405: 129 pgs: 129 active+clean; 188 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 353 KiB/s wr, 108 op/s 2024-05-08T21:22:49.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:49 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2531246819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:22:49.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:49 smithi071 ceph-mon[47074]: pgmap v405: 129 pgs: 129 active+clean; 188 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 353 KiB/s wr, 108 op/s 2024-05-08T21:22:49.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:49 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2531246819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:22:49.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:49 smithi082 ceph-mon[47512]: pgmap v405: 129 pgs: 129 active+clean; 188 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 353 KiB/s wr, 108 op/s 2024-05-08T21:22:49.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:49 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2531246819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:22:50.470 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:22:50.470 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:22:50.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.634+0000 7f3ce9893700 1 -- 172.21.15.33:0/1947466818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ce4106b50 msgr2=0x7f3ce4108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.634+0000 7f3ce9893700 1 --2- 172.21.15.33:0/1947466818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ce4106b50 0x7f3ce4108fc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00b0a0 tx=0x7f3cdc00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.634+0000 7f3ce9893700 1 -- 172.21.15.33:0/1947466818 shutdown_connections 2024-05-08T21:22:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.634+0000 7f3ce9893700 1 --2- 172.21.15.33:0/1947466818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ce4106b50 0x7f3ce4108fc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.634+0000 7f3ce9893700 1 --2- 172.21.15.33:0/1947466818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce41018f0 0x7f3ce4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.634+0000 7f3ce9893700 1 --2- 172.21.15.33:0/1947466818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ce4104220 0x7f3ce4106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.634+0000 7f3ce9893700 1 -- 172.21.15.33:0/1947466818 >> 172.21.15.33:0/1947466818 conn(0x7f3ce40fb300 msgr2=0x7f3ce40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.634+0000 7f3ce9893700 1 -- 172.21.15.33:0/1947466818 shutdown_connections 2024-05-08T21:22:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.634+0000 7f3ce9893700 1 -- 172.21.15.33:0/1947466818 wait complete. 2024-05-08T21:22:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.635+0000 7f3ce9893700 1 Processor -- start 2024-05-08T21:22:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.635+0000 7f3ce9893700 1 -- start start 2024-05-08T21:22:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce9893700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ce41018f0 0x7f3ce4196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce9893700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ce4104220 0x7f3ce4197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce9893700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce4106b50 0x7f3ce419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce9893700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce419e8f0 con 0x7f3ce4106b50 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce9893700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce419ea30 con 0x7f3ce4104220 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce9893700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce419ebc0 con 0x7f3ce41018f0 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce27fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ce4104220 0x7f3ce4197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ce41018f0 0x7f3ce4196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce27fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ce4104220 0x7f3ce4197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57546/0 (socket says 172.21.15.33:57546) 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ce41018f0 0x7f3ce4196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:36868/0 (socket says 172.21.15.33:36868) 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce27fc700 1 -- 172.21.15.33:0/1533183473 learned_addr learned my addr 172.21.15.33:0/1533183473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce37fe700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce4106b50 0x7f3ce419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce27fc700 1 -- 172.21.15.33:0/1533183473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ce41018f0 msgr2=0x7f3ce4196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce27fc700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ce41018f0 0x7f3ce4196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce27fc700 1 -- 172.21.15.33:0/1533183473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce4106b50 msgr2=0x7f3ce419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce27fc700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce4106b50 0x7f3ce419e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce27fc700 1 -- 172.21.15.33:0/1533183473 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd8009520 con 0x7f3ce4104220 2024-05-08T21:22:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce37fe700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce4106b50 0x7f3ce419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.636+0000 7f3ce27fc700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ce4104220 0x7f3ce4197300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800eb30 tx=0x7f3cd800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.637+0000 7f3ce8891700 1 -- 172.21.15.33:0/1533183473 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8009880 con 0x7f3ce4104220 2024-05-08T21:22:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.637+0000 7f3ce8891700 1 -- 172.21.15.33:0/1533183473 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3cd80044b0 con 0x7f3ce4104220 2024-05-08T21:22:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.637+0000 7f3ce8891700 1 -- 172.21.15.33:0/1533183473 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8003ce0 con 0x7f3ce4104220 2024-05-08T21:22:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.637+0000 7f3ce9893700 1 -- 172.21.15.33:0/1533183473 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cdc009d00 con 0x7f3ce4104220 2024-05-08T21:22:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.637+0000 7f3ce9893700 1 -- 172.21.15.33:0/1533183473 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce419f1d0 con 0x7f3ce4104220 2024-05-08T21:22:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.639+0000 7f3ce8891700 1 -- 172.21.15.33:0/1533183473 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3cd8018430 con 0x7f3ce4104220 2024-05-08T21:22:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.640+0000 7f3ce8891700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ccc0a11a0 0x7f3ccc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.640+0000 7f3ce8891700 1 -- 172.21.15.33:0/1533183473 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3cd8014070 con 0x7f3ce4104220 2024-05-08T21:22:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.640+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ccc0a11a0 0x7f3ccc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.640+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ccc0a11a0 0x7f3ccc0a3660 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009c60 tx=0x7f3cd4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:50.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.641+0000 7f3cf5a4db80 1 Processor -- start 2024-05-08T21:22:50.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.641+0000 7f3cf5a4db80 1 -- start start 2024-05-08T21:22:50.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.641+0000 7f3cf5a4db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 0x5564ecaffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.641+0000 7f3cf5a4db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00330 0x5564ecb007b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.641+0000 7f3cf5a4db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00cf0 0x5564ecb01170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.641+0000 7f3cf5a4db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564eca521d0 con 0x5564ecb00cf0 2024-05-08T21:22:50.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.641+0000 7f3cf5a4db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564eca4fb90 con 0x5564ecaff9d0 2024-05-08T21:22:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.641+0000 7f3cf5a4db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564eca4b480 con 0x5564ecb00330 2024-05-08T21:22:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.641+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 0x5564ecaffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.641+0000 7f3ce37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00cf0 0x5564ecb01170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.641+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 0x5564ecaffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57556/0 (socket says 172.21.15.33:57556) 2024-05-08T21:22:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.641+0000 7f3ce37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00cf0 0x5564ecb01170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51552/0 (socket says 172.21.15.33:51552) 2024-05-08T21:22:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.642+0000 7f3ce27fc700 1 --2- 172.21.15.33:0/3857380406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00330 0x5564ecb007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.642+0000 7f3ce2ffd700 1 -- 172.21.15.33:0/3857380406 learned_addr learned my addr 172.21.15.33:0/3857380406 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.642+0000 7f3ce2ffd700 1 -- 172.21.15.33:0/3857380406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00330 msgr2=0x5564ecb007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.642+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/3857380406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00330 0x5564ecb007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.642+0000 7f3ce2ffd700 1 -- 172.21.15.33:0/3857380406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00cf0 msgr2=0x5564ecb01170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.642+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/3857380406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00cf0 0x5564ecb01170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.642+0000 7f3ce2ffd700 1 -- 172.21.15.33:0/3857380406 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564ec8cc430 con 0x5564ecaff9d0 2024-05-08T21:22:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.642+0000 7f3ce37fe700 1 --2- 172.21.15.33:0/3857380406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00cf0 0x5564ecb01170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.642+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/3857380406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 0x5564ecaffdf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e510 tx=0x7f3cd4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f46f889bbb291df6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.642+0000 7f3cc9ffb700 1 -- 172.21.15.33:0/3857380406 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd400ee30 con 0x5564ecaff9d0 2024-05-08T21:22:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.642+0000 7f3cc9ffb700 1 -- 172.21.15.33:0/3857380406 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3cd4021070 con 0x5564ecaff9d0 2024-05-08T21:22:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.642+0000 7f3cc9ffb700 1 -- 172.21.15.33:0/3857380406 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4025700 con 0x5564ecaff9d0 2024-05-08T21:22:50.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.643+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/3857380406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 msgr2=0x5564ecaffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.643+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/3857380406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 0x5564ecaffdf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e510 tx=0x7f3cd4014000 comp rx=0 tx=0).stop 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.643+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/3857380406 shutdown_connections 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.643+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/3857380406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00cf0 0x5564ecb01170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.643+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/3857380406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 0x5564ecaffdf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.643+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/3857380406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00330 0x5564ecb007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.643+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/3857380406 >> 172.21.15.33:0/3857380406 conn(0x5564eca59290 msgr2=0x5564ecaf5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.643+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/3857380406 shutdown_connections 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.643+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/3857380406 wait complete. 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.643+0000 7f3cf5a4db80 1 -- start start 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3cf5a4db80 1 Processor -- start 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3cf5a4db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecaff9d0 0x5564ecb092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3cf5a4db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00330 0x5564ecb09800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3cf5a4db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecb00cf0 0x5564ecb10840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3cf5a4db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ec9e75e0 con 0x5564ecb00330 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3cf5a4db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ec9e7280 con 0x5564ecb00cf0 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3cf5a4db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ec8ce720 con 0x5564ecaff9d0 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3cf5a4db80 10 client.0 _ll_register_callbacks cb 0x5564ec9f0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3ce27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00330 0x5564ecb09800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecaff9d0 0x5564ecb092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3ce27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00330 0x5564ecb09800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51556/0 (socket says 172.21.15.33:51556) 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3ce27fc700 1 -- 172.21.15.33:0/413990905 learned_addr learned my addr 172.21.15.33:0/413990905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.644+0000 7f3ce37fe700 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecb00cf0 0x5564ecb10840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3ce2ffd700 1 -- 172.21.15.33:0/413990905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecb00cf0 msgr2=0x5564ecb10840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecb00cf0 0x5564ecb10840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3ce2ffd700 1 -- 172.21.15.33:0/413990905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00330 msgr2=0x5564ecb09800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00330 0x5564ecb09800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3ce2ffd700 1 -- 172.21.15.33:0/413990905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce4109bb0 con 0x5564ecaff9d0 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3ce37fe700 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecb00cf0 0x5564ecb10840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:50.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecaff9d0 0x5564ecb092c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e510 tx=0x7f3cd401dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4cac20fa3ed6154 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3cc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3cc2ffd700 1 -- 172.21.15.33:0/413990905 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4005250 con 0x5564ecaff9d0 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3cc2ffd700 1 -- 172.21.15.33:0/413990905 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3cd40053b0 con 0x5564ecaff9d0 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/413990905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564ec9a42f0 con 0x5564ecaff9d0 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3cc2ffd700 1 -- 172.21.15.33:0/413990905 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cd4005570 con 0x5564ecaff9d0 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3cf5a4db80 10 client.26548 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.645+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/413990905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f3cd4017070 con 0x5564ecaff9d0 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.646+0000 7f3cc2ffd700 1 -- 172.21.15.33:0/413990905 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5564ec9a42f0 con 0x5564ecaff9d0 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.646+0000 7f3cc2ffd700 1 -- 172.21.15.33:0/413990905 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f3cd4004aa0 con 0x5564ecaff9d0 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.646+0000 7f3cf5a4db80 10 client.26548 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.646+0000 7f3cf5a4db80 10 client.26548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.646+0000 7f3cf5a4db80 20 client.26548 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.646+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5564ecb14fc0 0x5564ecb173e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.646+0000 7f3cf5a4db80 4 client.26548 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.646+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/413990905 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564ecaca890 con 0x5564ecb14fc0 2024-05-08T21:22:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.646+0000 7f3ce27fc700 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5564ecb14fc0 0x5564ecb173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.646+0000 7f3cc2ffd700 10 client.26548 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.647+0000 7f3ce27fc700 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5564ecb14fc0 0x5564ecb173e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb48996789b3ce8 server_cookie=28885bad7e54f587 in_seq=0 out_seq=0 2024-05-08T21:22:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cc2ffd700 1 -- 172.21.15.33:0/413990905 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5564ecaca890 con 0x5564ecb14fc0 2024-05-08T21:22:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cc2ffd700 10 client.26548 handle_command_reply: tid=0 2024-05-08T21:22:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cf5a4db80 1 client.26548 shutdown 2024-05-08T21:22:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/413990905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecaff9d0 msgr2=0x5564ecb092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecaff9d0 0x5564ecb092c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400e510 tx=0x7f3cd401dde0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/413990905 shutdown_connections 2024-05-08T21:22:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5564ecb14fc0 0x5564ecb173e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecb00cf0 0x5564ecb10840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecaff9d0 0x5564ecb092c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/413990905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00330 0x5564ecb09800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cc2ffd700 0 client.26548 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:50.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/413990905 >> 172.21.15.33:0/413990905 conn(0x5564eca59290 msgr2=0x5564ecaf5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:50.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/413990905 shutdown_connections 2024-05-08T21:22:50.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.648+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/413990905 wait complete. 2024-05-08T21:22:50.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.649+0000 7f3cf5a4db80 20 client.26548 trim_cache size 0 max 16384 2024-05-08T21:22:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.669+0000 7f3cf5a4db80 1 Processor -- start 2024-05-08T21:22:50.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.669+0000 7f3cf5a4db80 1 -- start start 2024-05-08T21:22:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3cf5a4db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecaff9d0 0x5564ecb09780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3cf5a4db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecb00330 0x5564ecb10800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3cf5a4db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00cf0 0x5564ecb01a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3cf5a4db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ecb08e90 con 0x5564ecaff9d0 2024-05-08T21:22:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3cf5a4db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ecb119c0 con 0x5564ecb00330 2024-05-08T21:22:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3cf5a4db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ecaee150 con 0x5564ecb00cf0 2024-05-08T21:22:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecaff9d0 0x5564ecb09780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecaff9d0 0x5564ecb09780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51558/0 (socket says 172.21.15.33:51558) 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3ce2ffd700 1 -- 172.21.15.33:0/3537179907 learned_addr learned my addr 172.21.15.33:0/3537179907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3ce37fe700 1 --2- 172.21.15.33:0/3537179907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00cf0 0x5564ecb01a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3ce27fc700 1 --2- 172.21.15.33:0/3537179907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecb00330 0x5564ecb10800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3ce2ffd700 1 -- 172.21.15.33:0/3537179907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00cf0 msgr2=0x5564ecb01a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/3537179907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00cf0 0x5564ecb01a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3ce2ffd700 1 -- 172.21.15.33:0/3537179907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecb00330 msgr2=0x5564ecb10800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/3537179907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecb00330 0x5564ecb10800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3ce2ffd700 1 -- 172.21.15.33:0/3537179907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564ec878be0 con 0x5564ecaff9d0 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3ce37fe700 1 --2- 172.21.15.33:0/3537179907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00cf0 0x5564ecb01a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.670+0000 7f3ce27fc700 1 --2- 172.21.15.33:0/3537179907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecb00330 0x5564ecb10800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.671+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/3537179907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecaff9d0 0x5564ecb09780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005d90 tx=0x7f3cd400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3ef951adcb748b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.671+0000 7f3cca7fc700 1 -- 172.21.15.33:0/3537179907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd401d4b0 con 0x5564ecaff9d0 2024-05-08T21:22:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.671+0000 7f3cca7fc700 1 -- 172.21.15.33:0/3537179907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3cd40056e0 con 0x5564ecaff9d0 2024-05-08T21:22:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.671+0000 7f3cca7fc700 1 -- 172.21.15.33:0/3537179907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd401d990 con 0x5564ecaff9d0 2024-05-08T21:22:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.672+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/3537179907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecaff9d0 msgr2=0x5564ecb09780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.672+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/3537179907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecaff9d0 0x5564ecb09780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005d90 tx=0x7f3cd400ef90 comp rx=0 tx=0).stop 2024-05-08T21:22:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.672+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/3537179907 shutdown_connections 2024-05-08T21:22:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.672+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/3537179907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00cf0 0x5564ecb01a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.672+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/3537179907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecaff9d0 0x5564ecb09780 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.672+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/3537179907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecb00330 0x5564ecb10800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.672+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/3537179907 >> 172.21.15.33:0/3537179907 conn(0x5564eca59290 msgr2=0x5564ecb09070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.672+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/3537179907 shutdown_connections 2024-05-08T21:22:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.672+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/3537179907 wait complete. 2024-05-08T21:22:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.672+0000 7f3cf5a4db80 1 -- start start 2024-05-08T21:22:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3cf5a4db80 1 Processor -- start 2024-05-08T21:22:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3cf5a4db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 0x5564ecafdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3cf5a4db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00330 0x5564ecafe110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3cf5a4db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00cf0 0x5564ecb10800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3cf5a4db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ecafe730 con 0x5564ecb00330 2024-05-08T21:22:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3cf5a4db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ecafe8a0 con 0x5564ecaff9d0 2024-05-08T21:22:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3cf5a4db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ecafea60 con 0x5564ecb00cf0 2024-05-08T21:22:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3cf5a4db80 10 client.0 _ll_register_callbacks cb 0x5564ec9f0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 0x5564ecafdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3ce2ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 0x5564ecafdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57580/0 (socket says 172.21.15.33:57580) 2024-05-08T21:22:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3ce37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00cf0 0x5564ecb10800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3ce2ffd700 1 -- 172.21.15.33:0/2723248448 learned_addr learned my addr 172.21.15.33:0/2723248448 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.673+0000 7f3ce27fc700 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00330 0x5564ecafe110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3ce37fe700 1 -- 172.21.15.33:0/2723248448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 msgr2=0x5564ecafdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3ce37fe700 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 0x5564ecafdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3ce37fe700 1 -- 172.21.15.33:0/2723248448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00330 msgr2=0x5564ecafe110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3ce37fe700 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00330 0x5564ecafe110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3ce37fe700 1 -- 172.21.15.33:0/2723248448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce419f1d0 con 0x5564ecb00cf0 2024-05-08T21:22:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 0x5564ecafdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3ce37fe700 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00cf0 0x5564ecb10800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009fd0 tx=0x7f3cdc003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d66ff573470fc33 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3cc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:22:50.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3cc2ffd700 1 -- 172.21.15.33:0/2723248448 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc022070 con 0x5564ecb00cf0 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3cc2ffd700 1 -- 172.21.15.33:0/2723248448 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3cdc005040 con 0x5564ecb00cf0 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/2723248448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564eca43f40 con 0x5564ecb00cf0 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3cc2ffd700 1 -- 172.21.15.33:0/2723248448 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cdc00e040 con 0x5564ecb00cf0 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3cf5a4db80 10 client.26554 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.674+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/2723248448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f3cdc0074e0 con 0x5564ecb00cf0 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.675+0000 7f3cc2ffd700 1 -- 172.21.15.33:0/2723248448 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5564eca43f40 con 0x5564ecb00cf0 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.675+0000 7f3cc2ffd700 1 -- 172.21.15.33:0/2723248448 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f3cdc00b0d0 con 0x5564ecb00cf0 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.675+0000 7f3cf5a4db80 10 client.26554 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.675+0000 7f3cf5a4db80 10 client.26554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.675+0000 7f3cf5a4db80 20 client.26554 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.675+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5564ecb68ec0 0x5564ecb6b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.675+0000 7f3cf5a4db80 4 client.26554 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.675+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/2723248448 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5564ec899d70 con 0x5564ecb68ec0 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.675+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5564ecb68ec0 0x5564ecb6b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.675+0000 7f3cc2ffd700 10 client.26554 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.675+0000 7f3ce2ffd700 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5564ecb68ec0 0x5564ecb6b2e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df4cc0acd2ef956 server_cookie=918ce3582b122047 in_seq=0 out_seq=0 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.676+0000 7f3cc2ffd700 1 -- 172.21.15.33:0/2723248448 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5564ec899d70 con 0x5564ecb68ec0 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.676+0000 7f3cc2ffd700 10 client.26554 handle_command_reply: tid=0 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.676+0000 7f3cf5a4db80 1 client.26554 shutdown 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/2723248448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00cf0 msgr2=0x5564ecb10800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00cf0 0x5564ecb10800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009fd0 tx=0x7f3cdc003c30 comp rx=0 tx=0).stop 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/2723248448 shutdown_connections 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5564ecb68ec0 0x5564ecb6b2e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564ecb00cf0 0x5564ecb10800 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564ecaff9d0 0x5564ecafdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3cc2ffd700 0 client.26554 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3cf5a4db80 1 --2- 172.21.15.33:0/2723248448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5564ecb00330 0x5564ecafe110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/2723248448 >> 172.21.15.33:0/2723248448 conn(0x5564eca59290 msgr2=0x5564ecb01a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:50.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/2723248448 shutdown_connections 2024-05-08T21:22:50.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3cf5a4db80 1 -- 172.21.15.33:0/2723248448 wait complete. 2024-05-08T21:22:50.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3cf5a4db80 20 client.26554 trim_cache size 0 max 16384 2024-05-08T21:22:50.683 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:22:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:22:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "ede2c340-a52b-43ae-aecb-49db72683ca5", 2024-05-08T21:22:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:22:50.683 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:22:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3ce9893700 1 -- 172.21.15.33:0/1533183473 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ccc0a11a0 msgr2=0x7f3ccc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3ce9893700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ccc0a11a0 0x7f3ccc0a3660 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009c60 tx=0x7f3cd4009260 comp rx=0 tx=0).stop 2024-05-08T21:22:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3ce9893700 1 -- 172.21.15.33:0/1533183473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ce4104220 msgr2=0x7f3ce4197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.677+0000 7f3ce9893700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ce4104220 0x7f3ce4197300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800eb30 tx=0x7f3cd800ee40 comp rx=0 tx=0).stop 2024-05-08T21:22:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.678+0000 7f3ce9893700 1 -- 172.21.15.33:0/1533183473 shutdown_connections 2024-05-08T21:22:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.678+0000 7f3ce9893700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce4106b50 0x7f3ce419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.678+0000 7f3ce9893700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ce41018f0 0x7f3ce4196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.678+0000 7f3ce9893700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3ccc0a11a0 0x7f3ccc0a3660 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.678+0000 7f3ce9893700 1 --2- 172.21.15.33:0/1533183473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ce4104220 0x7f3ce4197300 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3cd800eb30 tx=0x7f3cd800ee40 comp rx=0 tx=0).stop 2024-05-08T21:22:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.678+0000 7f3ce9893700 1 -- 172.21.15.33:0/1533183473 >> 172.21.15.33:0/1533183473 conn(0x7f3ce40fb300 msgr2=0x7f3ce41023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.678+0000 7f3ce9893700 1 -- 172.21.15.33:0/1533183473 shutdown_connections 2024-05-08T21:22:50.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.678+0000 7f3ce9893700 1 -- 172.21.15.33:0/1533183473 wait complete. 2024-05-08T21:22:50.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:22:50.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.886+0000 7f125559f700 1 -- 172.21.15.33:0/4187132947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250101830 msgr2=0x7f1250103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.886+0000 7f125559f700 1 --2- 172.21.15.33:0/4187132947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250101830 0x7f1250103c20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1240009990 tx=0x7f1240009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.886+0000 7f125559f700 1 -- 172.21.15.33:0/4187132947 shutdown_connections 2024-05-08T21:22:50.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.886+0000 7f125559f700 1 --2- 172.21.15.33:0/4187132947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1250106a90 0x7f1250108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.886+0000 7f125559f700 1 --2- 172.21.15.33:0/4187132947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250101830 0x7f1250103c20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.886+0000 7f125559f700 1 --2- 172.21.15.33:0/4187132947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1250104160 0x7f1250106550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.886+0000 7f125559f700 1 -- 172.21.15.33:0/4187132947 >> 172.21.15.33:0/4187132947 conn(0x7f12500fb160 msgr2=0x7f12500fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:50.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.886+0000 7f125559f700 1 -- 172.21.15.33:0/4187132947 shutdown_connections 2024-05-08T21:22:50.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.886+0000 7f125559f700 1 -- 172.21.15.33:0/4187132947 wait complete. 2024-05-08T21:22:50.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.887+0000 7f125559f700 1 Processor -- start 2024-05-08T21:22:50.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.888+0000 7f125559f700 1 -- start start 2024-05-08T21:22:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.888+0000 7f125559f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1250101830 0x7f1250194700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.888+0000 7f125559f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250104160 0x7f1250194c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.888+0000 7f125559f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1250106a90 0x7f12501a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.888+0000 7f125559f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1250195480 con 0x7f1250104160 2024-05-08T21:22:50.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.888+0000 7f125559f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12501a4e40 con 0x7f1250106a90 2024-05-08T21:22:50.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.888+0000 7f125559f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12501a4f80 con 0x7f1250101830 2024-05-08T21:22:50.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.889+0000 7f124f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250104160 0x7f1250194c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.889+0000 7f1254d9e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1250106a90 0x7f12501a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.889+0000 7f1254d9e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1250106a90 0x7f12501a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57600/0 (socket says 172.21.15.33:57600) 2024-05-08T21:22:50.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.889+0000 7f1254d9e700 1 -- 172.21.15.33:0/910769548 learned_addr learned my addr 172.21.15.33:0/910769548 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:50.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.889+0000 7f124ffff700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1250101830 0x7f1250194700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.889+0000 7f124ffff700 1 -- 172.21.15.33:0/910769548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1250106a90 msgr2=0x7f12501a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.889+0000 7f124ffff700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1250106a90 0x7f12501a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.889+0000 7f124ffff700 1 -- 172.21.15.33:0/910769548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250104160 msgr2=0x7f1250194c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.889+0000 7f124ffff700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250104160 0x7f1250194c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.889+0000 7f124ffff700 1 -- 172.21.15.33:0/910769548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12400095f0 con 0x7f1250101830 2024-05-08T21:22:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.890+0000 7f124ffff700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1250101830 0x7f1250194700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1240003f60 tx=0x7f1240005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.890+0000 7f124d7fa700 1 -- 172.21.15.33:0/910769548 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124001d070 con 0x7f1250101830 2024-05-08T21:22:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.890+0000 7f124d7fa700 1 -- 172.21.15.33:0/910769548 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f124000bdb0 con 0x7f1250101830 2024-05-08T21:22:50.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.890+0000 7f124d7fa700 1 -- 172.21.15.33:0/910769548 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240022d30 con 0x7f1250101830 2024-05-08T21:22:50.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.890+0000 7f125559f700 1 -- 172.21.15.33:0/910769548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12501a5180 con 0x7f1250101830 2024-05-08T21:22:50.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.890+0000 7f125559f700 1 -- 172.21.15.33:0/910769548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12501a55d0 con 0x7f1250101830 2024-05-08T21:22:50.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.893+0000 7f124d7fa700 1 -- 172.21.15.33:0/910769548 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1240022470 con 0x7f1250101830 2024-05-08T21:22:50.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.895+0000 7f12629bbb80 1 Processor -- start 2024-05-08T21:22:50.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.895+0000 7f12629bbb80 1 -- start start 2024-05-08T21:22:50.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.896+0000 7f12629bbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 0x55a3046c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.896+0000 7f12629bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.896+0000 7f12629bbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046ca9b0 0x55a3046cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.896+0000 7f12629bbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a30460f150 con 0x55a3046c9630 2024-05-08T21:22:50.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.896+0000 7f12629bbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a30460cb10 con 0x55a3046ca9b0 2024-05-08T21:22:50.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.896+0000 7f12629bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304608400 con 0x55a3046c9f90 2024-05-08T21:22:50.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.896+0000 7f124d7fa700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f12380a12c0 0x7f12380a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.897+0000 7f124f7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046ca9b0 0x55a3046cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.897+0000 7f1254d9e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 0x55a3046c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.897+0000 7f124d7fa700 1 -- 172.21.15.33:0/910769548 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f12400ce1d0 con 0x7f1250101830 2024-05-08T21:22:50.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.897+0000 7f1254d9e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 0x55a3046c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51594/0 (socket says 172.21.15.33:51594) 2024-05-08T21:22:50.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.897+0000 7f1254d9e700 1 -- 172.21.15.33:0/1562788425 learned_addr learned my addr 172.21.15.33:0/1562788425 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:50.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.897+0000 7f124ffff700 1 --2- 172.21.15.33:0/1562788425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.897+0000 7f1254d9e700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f12380a12c0 0x7f12380a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.897+0000 7f124f7fe700 1 -- 172.21.15.33:0/1562788425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 msgr2=0x55a3046ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.897+0000 7f124f7fe700 1 --2- 172.21.15.33:0/1562788425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.897+0000 7f124f7fe700 1 -- 172.21.15.33:0/1562788425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 msgr2=0x55a3046c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.897+0000 7f124f7fe700 1 --2- 172.21.15.33:0/1562788425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 0x55a3046c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.897+0000 7f124f7fe700 1 -- 172.21.15.33:0/1562788425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3044893b0 con 0x55a3046ca9b0 2024-05-08T21:22:50.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.898+0000 7f124f7fe700 1 --2- 172.21.15.33:0/1562788425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046ca9b0 0x55a3046cae60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1244009cb0 tx=0x7f1244011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccd4307f4a2a644d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:50.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.898+0000 7f12367fc700 1 -- 172.21.15.33:0/1562788425 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244011d20 con 0x55a3046ca9b0 2024-05-08T21:22:50.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.898+0000 7f12367fc700 1 -- 172.21.15.33:0/1562788425 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1244011e80 con 0x55a3046ca9b0 2024-05-08T21:22:50.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.898+0000 7f12367fc700 1 -- 172.21.15.33:0/1562788425 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244005280 con 0x55a3046ca9b0 2024-05-08T21:22:50.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.899+0000 7f12629bbb80 1 -- 172.21.15.33:0/1562788425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046ca9b0 msgr2=0x55a3046cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.899+0000 7f12629bbb80 1 --2- 172.21.15.33:0/1562788425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046ca9b0 0x55a3046cae60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1244009cb0 tx=0x7f1244011440 comp rx=0 tx=0).stop 2024-05-08T21:22:50.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.900+0000 7f1254d9e700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f12380a12c0 0x7f12380a3780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55a3046c93d0 tx=0x7f1248004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:50.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.900+0000 7f12629bbb80 1 -- 172.21.15.33:0/1562788425 shutdown_connections 2024-05-08T21:22:50.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.900+0000 7f12629bbb80 1 --2- 172.21.15.33:0/1562788425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046ca9b0 0x55a3046cae60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.900+0000 7f12629bbb80 1 --2- 172.21.15.33:0/1562788425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 0x55a3046c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.900+0000 7f12629bbb80 1 --2- 172.21.15.33:0/1562788425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.900+0000 7f12629bbb80 1 -- 172.21.15.33:0/1562788425 >> 172.21.15.33:0/1562788425 conn(0x55a304616210 msgr2=0x55a3046b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:50.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.900+0000 7f12629bbb80 1 -- 172.21.15.33:0/1562788425 shutdown_connections 2024-05-08T21:22:50.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.900+0000 7f12629bbb80 1 -- 172.21.15.33:0/1562788425 wait complete. 2024-05-08T21:22:50.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.901+0000 7f12629bbb80 1 -- start start 2024-05-08T21:22:50.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.901+0000 7f12629bbb80 1 Processor -- start 2024-05-08T21:22:50.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.902+0000 7f12629bbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 0x55a3046d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.902+0000 7f12629bbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9f90 0x55a3046d3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.902+0000 7f12629bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046ca9b0 0x55a3046da570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.902+0000 7f12629bbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3045a4560 con 0x55a3046c9f90 2024-05-08T21:22:50.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.902+0000 7f12629bbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3045a4200 con 0x55a3046c9630 2024-05-08T21:22:50.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.902+0000 7f12629bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a30448b6a0 con 0x55a3046ca9b0 2024-05-08T21:22:50.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.902+0000 7f12629bbb80 10 client.0 _ll_register_callbacks cb 0x55a3045adb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:50.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.902+0000 7f124ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 0x55a3046d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.902+0000 7f124ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 0x55a3046d2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57608/0 (socket says 172.21.15.33:57608) 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.902+0000 7f124ffff700 1 -- 172.21.15.33:0/2259281388 learned_addr learned my addr 172.21.15.33:0/2259281388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.902+0000 7f124f7fe700 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9f90 0x55a3046d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f1254d9e700 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046ca9b0 0x55a3046da570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f124ffff700 1 -- 172.21.15.33:0/2259281388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046ca9b0 msgr2=0x55a3046da570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f124ffff700 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046ca9b0 0x55a3046da570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f124ffff700 1 -- 172.21.15.33:0/2259281388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9f90 msgr2=0x55a3046d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f124ffff700 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9f90 0x55a3046d3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f124ffff700 1 -- 172.21.15.33:0/2259281388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12440098e0 con 0x55a3046c9630 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f124f7fe700 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9f90 0x55a3046d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f124ffff700 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 0x55a3046d2ff0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12400a31b0 tx=0x7f1240005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9e721c7631b95a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f122f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f122f7fe700 1 -- 172.21.15.33:0/2259281388 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12400a3450 con 0x55a3046c9630 2024-05-08T21:22:50.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f122f7fe700 1 -- 172.21.15.33:0/2259281388 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f124009b9f0 con 0x55a3046c9630 2024-05-08T21:22:50.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f12629bbb80 1 -- 172.21.15.33:0/2259281388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a304687810 con 0x55a3046c9630 2024-05-08T21:22:50.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f122f7fe700 1 -- 172.21.15.33:0/2259281388 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f124009b2f0 con 0x55a3046c9630 2024-05-08T21:22:50.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f12629bbb80 10 client.26546 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:50.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.903+0000 7f12629bbb80 1 -- 172.21.15.33:0/2259281388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a304435b60 con 0x55a3046c9630 2024-05-08T21:22:50.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.904+0000 7f122f7fe700 1 -- 172.21.15.33:0/2259281388 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a304687810 con 0x55a3046c9630 2024-05-08T21:22:50.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.904+0000 7f122f7fe700 1 -- 172.21.15.33:0/2259281388 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f1240084d00 con 0x55a3046c9630 2024-05-08T21:22:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.904+0000 7f12629bbb80 10 client.26546 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.904+0000 7f12629bbb80 10 client.26546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.904+0000 7f12629bbb80 20 client.26546 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.905+0000 7f12629bbb80 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a3046dea70 0x55a3046e0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.905+0000 7f12629bbb80 4 client.26546 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:22:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.905+0000 7f12629bbb80 1 -- 172.21.15.33:0/2259281388 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a304600ec0 con 0x55a3046dea70 2024-05-08T21:22:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.905+0000 7f124f7fe700 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a3046dea70 0x55a3046e0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.905+0000 7f124f7fe700 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a3046dea70 0x55a3046e0e90 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b95ddb281c49de8a server_cookie=d91390dbc2cb5f8 in_seq=0 out_seq=0 2024-05-08T21:22:50.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.905+0000 7f122f7fe700 10 client.26546 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:50.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f122f7fe700 1 -- 172.21.15.33:0/2259281388 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a304600ec0 con 0x55a3046dea70 2024-05-08T21:22:50.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f122f7fe700 10 client.26546 handle_command_reply: tid=0 2024-05-08T21:22:50.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f12629bbb80 1 client.26546 shutdown 2024-05-08T21:22:50.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f12629bbb80 1 -- 172.21.15.33:0/2259281388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 msgr2=0x55a3046d2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f12629bbb80 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 0x55a3046d2ff0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12400a31b0 tx=0x7f1240005fb0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f12629bbb80 1 -- 172.21.15.33:0/2259281388 shutdown_connections 2024-05-08T21:22:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f12629bbb80 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a3046dea70 0x55a3046e0e90 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f12629bbb80 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046ca9b0 0x55a3046da570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f12629bbb80 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 0x55a3046d2ff0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f12629bbb80 1 --2- 172.21.15.33:0/2259281388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9f90 0x55a3046d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f122f7fe700 0 client.26546 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f12629bbb80 1 -- 172.21.15.33:0/2259281388 >> 172.21.15.33:0/2259281388 conn(0x55a304616210 msgr2=0x55a3046b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f12629bbb80 1 -- 172.21.15.33:0/2259281388 shutdown_connections 2024-05-08T21:22:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f12629bbb80 1 -- 172.21.15.33:0/2259281388 wait complete. 2024-05-08T21:22:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.906+0000 7f12629bbb80 20 client.26546 trim_cache size 0 max 16384 2024-05-08T21:22:50.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.927+0000 7f12629bbb80 1 Processor -- start 2024-05-08T21:22:50.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.927+0000 7f12629bbb80 1 -- start start 2024-05-08T21:22:50.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.927+0000 7f12629bbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 0x55a3046cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.927+0000 7f12629bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046da530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.927+0000 7f12629bbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046ca9b0 0x55a3046d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.927+0000 7f12629bbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a304687810 con 0x55a3046c9630 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.927+0000 7f12629bbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3046db6f0 con 0x55a3046ca9b0 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.927+0000 7f12629bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3046d4000 con 0x55a3046c9f90 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f124f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046da530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f124ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 0x55a3046cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f124f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046da530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:36944/0 (socket says 172.21.15.33:36944) 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f124f7fe700 1 -- 172.21.15.33:0/3334888893 learned_addr learned my addr 172.21.15.33:0/3334888893 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f124ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 0x55a3046cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51622/0 (socket says 172.21.15.33:51622) 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f1254d9e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046ca9b0 0x55a3046d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f124f7fe700 1 -- 172.21.15.33:0/3334888893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046ca9b0 msgr2=0x55a3046d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f124f7fe700 1 --2- 172.21.15.33:0/3334888893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046ca9b0 0x55a3046d2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f124f7fe700 1 -- 172.21.15.33:0/3334888893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 msgr2=0x55a3046cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f124f7fe700 1 --2- 172.21.15.33:0/3334888893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 0x55a3046cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f124f7fe700 1 -- 172.21.15.33:0/3334888893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3045ce560 con 0x55a3046c9f90 2024-05-08T21:22:50.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f124ffff700 1 --2- 172.21.15.33:0/3334888893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 0x55a3046cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:22:50.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f124f7fe700 1 --2- 172.21.15.33:0/3334888893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046da530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1244000c00 tx=0x7f1244009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a79894ec8652caf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:50.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.928+0000 7f1236ffd700 1 -- 172.21.15.33:0/3334888893 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244005ce0 con 0x55a3046c9f90 2024-05-08T21:22:50.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.929+0000 7f1236ffd700 1 -- 172.21.15.33:0/3334888893 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12440044e0 con 0x55a3046c9f90 2024-05-08T21:22:50.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.929+0000 7f1236ffd700 1 -- 172.21.15.33:0/3334888893 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244003e90 con 0x55a3046c9f90 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.929+0000 7f12629bbb80 1 -- 172.21.15.33:0/3334888893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 msgr2=0x55a3046da530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.929+0000 7f12629bbb80 1 --2- 172.21.15.33:0/3334888893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046da530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1244000c00 tx=0x7f1244009260 comp rx=0 tx=0).stop 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.929+0000 7f12629bbb80 1 -- 172.21.15.33:0/3334888893 shutdown_connections 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.929+0000 7f12629bbb80 1 --2- 172.21.15.33:0/3334888893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046ca9b0 0x55a3046d2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.929+0000 7f12629bbb80 1 --2- 172.21.15.33:0/3334888893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046c9630 0x55a3046cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.929+0000 7f12629bbb80 1 --2- 172.21.15.33:0/3334888893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046da530 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.929+0000 7f12629bbb80 1 -- 172.21.15.33:0/3334888893 >> 172.21.15.33:0/3334888893 conn(0x55a304616210 msgr2=0x55a3046b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.929+0000 7f12629bbb80 1 -- 172.21.15.33:0/3334888893 shutdown_connections 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.929+0000 7f12629bbb80 1 -- 172.21.15.33:0/3334888893 wait complete. 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.930+0000 7f12629bbb80 1 -- start start 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.930+0000 7f12629bbb80 1 Processor -- start 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.930+0000 7f12629bbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 0x55a3046d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.930+0000 7f12629bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046d6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.930+0000 7f12629bbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046ca9b0 0x55a3046d8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.930+0000 7f12629bbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3046d94d0 con 0x55a3046ca9b0 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.930+0000 7f12629bbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3046d9640 con 0x55a3046c9630 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.930+0000 7f12629bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3046d97b0 con 0x55a3046c9f90 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.930+0000 7f12629bbb80 10 client.0 _ll_register_callbacks cb 0x55a3045adb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f124ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 0x55a3046d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f124f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046d6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f1254d9e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046ca9b0 0x55a3046d8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f124ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 0x55a3046d5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57638/0 (socket says 172.21.15.33:57638) 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f124ffff700 1 -- 172.21.15.33:0/457600261 learned_addr learned my addr 172.21.15.33:0/457600261 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f124f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046d6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:36950/0 (socket says 172.21.15.33:36950) 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f124ffff700 1 -- 172.21.15.33:0/457600261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 msgr2=0x55a3046d6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f124ffff700 1 --2- 172.21.15.33:0/457600261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046d6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f124ffff700 1 -- 172.21.15.33:0/457600261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046ca9b0 msgr2=0x55a3046d8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f124ffff700 1 --2- 172.21.15.33:0/457600261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046ca9b0 0x55a3046d8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f124ffff700 1 -- 172.21.15.33:0/457600261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a304435b60 con 0x55a3046c9630 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f124ffff700 1 --2- 172.21.15.33:0/457600261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 0x55a3046d5bd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1240097d60 tx=0x7f1240009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79725e06a12e87d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f122f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f122f7fe700 1 -- 172.21.15.33:0/457600261 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12400b9050 con 0x55a3046c9630 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.931+0000 7f122f7fe700 1 -- 172.21.15.33:0/457600261 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f124009b9f0 con 0x55a3046c9630 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.932+0000 7f12629bbb80 1 -- 172.21.15.33:0/457600261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a304561270 con 0x55a3046c9630 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.932+0000 7f122f7fe700 1 -- 172.21.15.33:0/457600261 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12400b0020 con 0x55a3046c9630 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.932+0000 7f12629bbb80 10 client.26552 fetch_fsmap learned FSMap version 34 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.932+0000 7f12629bbb80 1 -- 172.21.15.33:0/457600261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f124401d040 con 0x55a3046c9630 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.932+0000 7f122f7fe700 1 -- 172.21.15.33:0/457600261 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a304561270 con 0x55a3046c9630 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.932+0000 7f122f7fe700 1 -- 172.21.15.33:0/457600261 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f1240084b40 con 0x55a3046c9630 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.932+0000 7f12629bbb80 10 client.26552 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.932+0000 7f12629bbb80 10 client.26552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.932+0000 7f12629bbb80 20 client.26552 populate_metadata read hostname 'smithi033' 2024-05-08T21:22:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.932+0000 7f12629bbb80 1 --2- 172.21.15.33:0/457600261 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a3047353f0 0x55a304737810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:22:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.932+0000 7f12629bbb80 4 client.26552 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:22:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.932+0000 7f12629bbb80 1 -- 172.21.15.33:0/457600261 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a304456cf0 con 0x55a3047353f0 2024-05-08T21:22:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.933+0000 7f124f7fe700 1 --2- 172.21.15.33:0/457600261 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a3047353f0 0x55a304737810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:22:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.934+0000 7f122f7fe700 10 client.26552 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.934+0000 7f124f7fe700 1 --2- 172.21.15.33:0/457600261 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a3047353f0 0x55a304737810 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e51d296f26a5843c server_cookie=25b127990c796730 in_seq=0 out_seq=0 2024-05-08T21:22:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.934+0000 7f122f7fe700 1 -- 172.21.15.33:0/457600261 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a304456cf0 con 0x55a3047353f0 2024-05-08T21:22:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.934+0000 7f122f7fe700 10 client.26552 handle_command_reply: tid=0 2024-05-08T21:22:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.934+0000 7f12629bbb80 1 client.26552 shutdown 2024-05-08T21:22:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.934+0000 7f12629bbb80 1 -- 172.21.15.33:0/457600261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 msgr2=0x55a3046d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.934+0000 7f12629bbb80 1 --2- 172.21.15.33:0/457600261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 0x55a3046d5bd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1240097d60 tx=0x7f1240009ca0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f12629bbb80 1 -- 172.21.15.33:0/457600261 shutdown_connections 2024-05-08T21:22:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f12629bbb80 1 --2- 172.21.15.33:0/457600261 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a3047353f0 0x55a304737810 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f12629bbb80 1 --2- 172.21.15.33:0/457600261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3046ca9b0 0x55a3046d8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f12629bbb80 1 --2- 172.21.15.33:0/457600261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3046c9630 0x55a3046d5bd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f12629bbb80 1 --2- 172.21.15.33:0/457600261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a3046c9f90 0x55a3046d6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f122f7fe700 0 client.26552 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:22:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f12629bbb80 1 -- 172.21.15.33:0/457600261 >> 172.21.15.33:0/457600261 conn(0x55a304616210 msgr2=0x55a3046d24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f12629bbb80 1 -- 172.21.15.33:0/457600261 shutdown_connections 2024-05-08T21:22:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f12629bbb80 1 -- 172.21.15.33:0/457600261 wait complete. 2024-05-08T21:22:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f12629bbb80 20 client.26552 trim_cache size 0 max 16384 2024-05-08T21:22:50.940 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:22:50.941 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:22:50.941 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:22:50.941 INFO:teuthology.orchestra.run.smithi033.stdout: "ede2c340-a52b-43ae-aecb-49db72683ca5": { 2024-05-08T21:22:50.941 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:22:50.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "ede2c340-a52b-43ae-aecb-49db72683ca5", 2024-05-08T21:22:50.941 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:22:50.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:22:50.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:22:50.941 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:22:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f125559f700 1 -- 172.21.15.33:0/910769548 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f12380a12c0 msgr2=0x7f12380a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f125559f700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f12380a12c0 0x7f12380a3780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x55a3046c93d0 tx=0x7f1248004ad0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f125559f700 1 -- 172.21.15.33:0/910769548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1250101830 msgr2=0x7f1250194700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:22:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.935+0000 7f125559f700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1250101830 0x7f1250194700 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1240003f60 tx=0x7f1240005740 comp rx=0 tx=0).stop 2024-05-08T21:22:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.936+0000 7f125559f700 1 -- 172.21.15.33:0/910769548 shutdown_connections 2024-05-08T21:22:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.936+0000 7f125559f700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f12380a12c0 0x7f12380a3780 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.936+0000 7f125559f700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1250106a90 0x7f12501a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.936+0000 7f125559f700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1250101830 0x7f1250194700 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.936+0000 7f125559f700 1 --2- 172.21.15.33:0/910769548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1250104160 0x7f1250194c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:22:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.936+0000 7f125559f700 1 -- 172.21.15.33:0/910769548 >> 172.21.15.33:0/910769548 conn(0x7f12500fb160 msgr2=0x7f1250100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:22:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.936+0000 7f125559f700 1 -- 172.21.15.33:0/910769548 shutdown_connections 2024-05-08T21:22:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:22:50.936+0000 7f125559f700 1 -- 172.21.15.33:0/910769548 wait complete. 2024-05-08T21:22:50.952 INFO:tasks.cephfs.filesystem:scrub status for tag:ede2c340-a52b-43ae-aecb-49db72683ca5 - {'path': '/', 'tag': 'ede2c340-a52b-43ae-aecb-49db72683ca5', 'options': 'recursive,force'} 2024-05-08T21:22:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:51 smithi033 ceph-mon[42528]: pgmap v406: 129 pgs: 129 active+clean; 190 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-05-08T21:22:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:51 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:51 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:51 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:51 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:51 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:51.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:51 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:51.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:51 smithi071 ceph-mon[47074]: pgmap v406: 129 pgs: 129 active+clean; 190 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-05-08T21:22:51.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:51 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:51.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:51 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:51.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:51 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:51.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:51 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:51.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:51 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:51.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:51 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:51 smithi082 ceph-mon[47512]: pgmap v406: 129 pgs: 129 active+clean; 190 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-05-08T21:22:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:51 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:51 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:51 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:51 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:51 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:51.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:51 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:52.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:52 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:22:52.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:52 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:52.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:52 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:52.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:52 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:52.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:52 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:22:52.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:52 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:52.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:52 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:52.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:52 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:52.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:52 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:22:52.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:52 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:52.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:52 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:52.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:52 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:52.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:52 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:22:52.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:52 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:52.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:52 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:52.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:52 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:52.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:52 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:22:52.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:52 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:52.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:52 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:52.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:52 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:52.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:52 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:22:52.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:52 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:52.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:52 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:52.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:52 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:22:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:53 smithi033 ceph-mon[42528]: pgmap v407: 129 pgs: 129 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 238 op/s 2024-05-08T21:22:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:53 smithi071 ceph-mon[47074]: pgmap v407: 129 pgs: 129 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 238 op/s 2024-05-08T21:22:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:53 smithi082 ceph-mon[47512]: pgmap v407: 129 pgs: 129 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 238 op/s 2024-05-08T21:22:55.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:55 smithi033 ceph-mon[42528]: pgmap v408: 129 pgs: 129 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 220 op/s 2024-05-08T21:22:55.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:55 smithi071 ceph-mon[47074]: pgmap v408: 129 pgs: 129 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 220 op/s 2024-05-08T21:22:55.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:55 smithi082 ceph-mon[47512]: pgmap v408: 129 pgs: 129 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 220 op/s 2024-05-08T21:22:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:57 smithi033 ceph-mon[42528]: pgmap v409: 129 pgs: 129 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 370 op/s 2024-05-08T21:22:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:57 smithi071 ceph-mon[47074]: pgmap v409: 129 pgs: 129 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 370 op/s 2024-05-08T21:22:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:57 smithi082 ceph-mon[47512]: pgmap v409: 129 pgs: 129 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 370 op/s 2024-05-08T21:22:59.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:22:59 smithi033 ceph-mon[42528]: pgmap v410: 129 pgs: 129 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 400 op/s 2024-05-08T21:22:59.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:22:59 smithi071 ceph-mon[47074]: pgmap v410: 129 pgs: 129 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 400 op/s 2024-05-08T21:22:59.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:22:59 smithi082 ceph-mon[47512]: pgmap v410: 129 pgs: 129 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 400 op/s 2024-05-08T21:23:01.642 INFO:tasks.workunit.client.0.smithi033.stdout:10000 trunc from 0xbf6ab to 0x628a6 2024-05-08T21:23:01.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:01 smithi033 ceph-mon[42528]: pgmap v411: 129 pgs: 129 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.3 MiB/s wr, 552 op/s 2024-05-08T21:23:01.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:01 smithi071 ceph-mon[47074]: pgmap v411: 129 pgs: 129 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.3 MiB/s wr, 552 op/s 2024-05-08T21:23:01.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:01 smithi082 ceph-mon[47512]: pgmap v411: 129 pgs: 129 active+clean; 193 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.3 MiB/s wr, 552 op/s 2024-05-08T21:23:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:03 smithi033 ceph-mon[42528]: pgmap v412: 129 pgs: 129 active+clean; 194 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 486 op/s 2024-05-08T21:23:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:03 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:23:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:03 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:23:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:03 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:23:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:03 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:23:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:03 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:23:03.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:03 smithi071 ceph-mon[47074]: pgmap v412: 129 pgs: 129 active+clean; 194 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 486 op/s 2024-05-08T21:23:03.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:03 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:23:03.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:03 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:23:03.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:03 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:23:03.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:03 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:23:03.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:03 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:23:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:03 smithi082 ceph-mon[47512]: pgmap v412: 129 pgs: 129 active+clean; 194 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 486 op/s 2024-05-08T21:23:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:03 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:23:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:03 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:23:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:03 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:23:03.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:03 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:23:03.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:03 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:23:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:05 smithi033 ceph-mon[42528]: pgmap v413: 129 pgs: 129 active+clean; 194 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 430 op/s 2024-05-08T21:23:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:05 smithi071 ceph-mon[47074]: pgmap v413: 129 pgs: 129 active+clean; 194 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 430 op/s 2024-05-08T21:23:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:05 smithi082 ceph-mon[47512]: pgmap v413: 129 pgs: 129 active+clean; 194 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 430 op/s 2024-05-08T21:23:07.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:07 smithi033 ceph-mon[42528]: pgmap v414: 129 pgs: 129 active+clean; 194 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.8 MiB/s wr, 591 op/s 2024-05-08T21:23:07.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:07 smithi071 ceph-mon[47074]: pgmap v414: 129 pgs: 129 active+clean; 194 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.8 MiB/s wr, 591 op/s 2024-05-08T21:23:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:07 smithi082 ceph-mon[47512]: pgmap v414: 129 pgs: 129 active+clean; 194 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.8 MiB/s wr, 591 op/s 2024-05-08T21:23:09.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:09 smithi033 ceph-mon[42528]: pgmap v415: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 474 op/s 2024-05-08T21:23:09.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:09 smithi071 ceph-mon[47074]: pgmap v415: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 474 op/s 2024-05-08T21:23:09.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:09 smithi082 ceph-mon[47512]: pgmap v415: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 474 op/s 2024-05-08T21:23:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:11 smithi033 ceph-mon[42528]: pgmap v416: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.6 MiB/s wr, 581 op/s 2024-05-08T21:23:11.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:11 smithi071 ceph-mon[47074]: pgmap v416: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.6 MiB/s wr, 581 op/s 2024-05-08T21:23:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:11 smithi082 ceph-mon[47512]: pgmap v416: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.6 MiB/s wr, 581 op/s 2024-05-08T21:23:13.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:13 smithi033 ceph-mon[42528]: pgmap v417: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-05-08T21:23:13.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:13 smithi071 ceph-mon[47074]: pgmap v417: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-05-08T21:23:13.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:13 smithi082 ceph-mon[47512]: pgmap v417: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-05-08T21:23:15.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:15 smithi033 ceph-mon[42528]: pgmap v418: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 425 op/s 2024-05-08T21:23:15.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:15 smithi071 ceph-mon[47074]: pgmap v418: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 425 op/s 2024-05-08T21:23:15.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:15 smithi082 ceph-mon[47512]: pgmap v418: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 425 op/s 2024-05-08T21:23:16.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:23:16.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:23:16.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:16 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:23:16.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:23:16.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:23:16.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:16 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:23:16.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:23:16.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:23:16.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:16 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:23:17.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:17 smithi033 ceph-mon[42528]: pgmap v419: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.6 MiB/s wr, 571 op/s 2024-05-08T21:23:17.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:17 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:23:17.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:17 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:23:17.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:17 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:23:17.774 INFO:tasks.workunit.client.0.smithi033.stdout:20000 mapwrite 0xe10fb thru 0xe4c85 (0x3b8b bytes) 2024-05-08T21:23:17.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:17 smithi071 ceph-mon[47074]: pgmap v419: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.6 MiB/s wr, 571 op/s 2024-05-08T21:23:17.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:17 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:23:17.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:17 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:23:17.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:17 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:23:17.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:17 smithi082 ceph-mon[47512]: pgmap v419: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.6 MiB/s wr, 571 op/s 2024-05-08T21:23:17.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:17 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:23:17.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:17 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:23:17.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:17 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:23:19.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:19 smithi033 ceph-mon[42528]: pgmap v420: 129 pgs: 129 active+clean; 197 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 454 op/s 2024-05-08T21:23:19.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:19 smithi071 ceph-mon[47074]: pgmap v420: 129 pgs: 129 active+clean; 197 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 454 op/s 2024-05-08T21:23:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:19 smithi082 ceph-mon[47512]: pgmap v420: 129 pgs: 129 active+clean; 197 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 454 op/s 2024-05-08T21:23:20.954 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:23:21.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.113+0000 7f3aab206700 1 -- 172.21.15.33:0/1152165717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa41013f0 msgr2=0x7f3aa4101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.113+0000 7f3aab206700 1 --2- 172.21.15.33:0/1152165717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa41013f0 0x7f3aa4101810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3a980098f0 tx=0x7f3a98009c00 comp rx=0 tx=0).stop 2024-05-08T21:23:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.113+0000 7f3aab206700 1 -- 172.21.15.33:0/1152165717 shutdown_connections 2024-05-08T21:23:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.113+0000 7f3aab206700 1 --2- 172.21.15.33:0/1152165717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa4100a00 0x7f3aa41082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.113+0000 7f3aab206700 1 --2- 172.21.15.33:0/1152165717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa41013f0 0x7f3aa4101810 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.113+0000 7f3aab206700 1 --2- 172.21.15.33:0/1152165717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4100040 0x7f3aa41004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.113+0000 7f3aab206700 1 -- 172.21.15.33:0/1152165717 >> 172.21.15.33:0/1152165717 conn(0x7f3aa40faad0 msgr2=0x7f3aa40fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.113+0000 7f3aab206700 1 -- 172.21.15.33:0/1152165717 shutdown_connections 2024-05-08T21:23:21.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.113+0000 7f3aab206700 1 -- 172.21.15.33:0/1152165717 wait complete. 2024-05-08T21:23:21.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.113+0000 7f3aab206700 1 Processor -- start 2024-05-08T21:23:21.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.113+0000 7f3aab206700 1 -- start start 2024-05-08T21:23:21.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aab206700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4100040 0x7f3aa4198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aab206700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa4100a00 0x7f3aa4199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aab206700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa41013f0 0x7f3aa41a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aab206700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa41a0af0 con 0x7f3aa41013f0 2024-05-08T21:23:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aab206700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa41a0c30 con 0x7f3aa4100a00 2024-05-08T21:23:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aab206700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa41a0dc0 con 0x7f3aa4100040 2024-05-08T21:23:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aa8fa2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4100040 0x7f3aa4198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aa97a3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa41013f0 0x7f3aa41a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aa8fa2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4100040 0x7f3aa4198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42540/0 (socket says 172.21.15.33:42540) 2024-05-08T21:23:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/81239022 learned_addr learned my addr 172.21.15.33:0/81239022 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aa3fff700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa4100a00 0x7f3aa4199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/81239022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa4100a00 msgr2=0x7f3aa4199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aa8fa2700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa4100a00 0x7f3aa4199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/81239022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa41013f0 msgr2=0x7f3aa41a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aa8fa2700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa41013f0 0x7f3aa41a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/81239022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a98009520 con 0x7f3aa4100040 2024-05-08T21:23:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.114+0000 7f3aa8fa2700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4100040 0x7f3aa4198fc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009810 tx=0x7f3a9800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.115+0000 7f3aa1ffb700 1 -- 172.21.15.33:0/81239022 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9801c070 con 0x7f3aa4100040 2024-05-08T21:23:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.115+0000 7f3aab206700 1 -- 172.21.15.33:0/81239022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa41a1040 con 0x7f3aa4100040 2024-05-08T21:23:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.115+0000 7f3aa1ffb700 1 -- 172.21.15.33:0/81239022 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a9800bec0 con 0x7f3aa4100040 2024-05-08T21:23:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.115+0000 7f3aa1ffb700 1 -- 172.21.15.33:0/81239022 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98003900 con 0x7f3aa4100040 2024-05-08T21:23:21.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.115+0000 7f3aab206700 1 -- 172.21.15.33:0/81239022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa41a14c0 con 0x7f3aa4100040 2024-05-08T21:23:21.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.117+0000 7f3aa1ffb700 1 -- 172.21.15.33:0/81239022 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3a98020030 con 0x7f3aa4100040 2024-05-08T21:23:21.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.118+0000 7f3aa1ffb700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3a940a11a0 0x7f3a940a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.118+0000 7f3aa3fff700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3a940a11a0 0x7f3a940a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.118+0000 7f3aa1ffb700 1 -- 172.21.15.33:0/81239022 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3a980cdc60 con 0x7f3aa4100040 2024-05-08T21:23:21.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.118+0000 7f3aa3fff700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3a940a11a0 0x7f3a940a3660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3a90005fd0 tx=0x7f3a90009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.119+0000 7f3ab73c0b80 1 Processor -- start 2024-05-08T21:23:21.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.119+0000 7f3ab73c0b80 1 -- start start 2024-05-08T21:23:21.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.119+0000 7f3ab73c0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 0x564a8f56d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.119+0000 7f3ab73c0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 0x564a8f56e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.119+0000 7f3ab73c0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f56ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.119+0000 7f3ab73c0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8f4c4150 con 0x564a8f56e860 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.119+0000 7f3ab73c0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8f4c1b10 con 0x564a8f56d540 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.119+0000 7f3ab73c0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8f4bd400 con 0x564a8f56dea0 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.119+0000 7f3aa8fa2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 0x564a8f56d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.119+0000 7f3aa8fa2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 0x564a8f56d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:42796/0 (socket says 172.21.15.33:42796) 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.119+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/772948846 learned_addr learned my addr 172.21.15.33:0/772948846 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.119+0000 7f3aa3fff700 1 --2- 172.21.15.33:0/772948846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 0x564a8f56e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.120+0000 7f3aa97a3700 1 --2- 172.21.15.33:0/772948846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f56ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.120+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/772948846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 msgr2=0x564a8f56e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.120+0000 7f3aa8fa2700 1 --2- 172.21.15.33:0/772948846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 0x564a8f56e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.120+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/772948846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 msgr2=0x564a8f56ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.120+0000 7f3aa8fa2700 1 --2- 172.21.15.33:0/772948846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f56ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.120+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/772948846 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a8f33e3b0 con 0x564a8f56d540 2024-05-08T21:23:21.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.120+0000 7f3aa97a3700 1 --2- 172.21.15.33:0/772948846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f56ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:23:21.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.120+0000 7f3aa8fa2700 1 --2- 172.21.15.33:0/772948846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 0x564a8f56d960 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3a980a2c80 tx=0x7f3a98009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b82a0688bd8980c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.120+0000 7f3a8effd700 1 -- 172.21.15.33:0/772948846 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a980b2070 con 0x564a8f56d540 2024-05-08T21:23:21.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.120+0000 7f3a8effd700 1 -- 172.21.15.33:0/772948846 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a9809ebf0 con 0x564a8f56d540 2024-05-08T21:23:21.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.120+0000 7f3a8effd700 1 -- 172.21.15.33:0/772948846 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a980974d0 con 0x564a8f56d540 2024-05-08T21:23:21.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.121+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/772948846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 msgr2=0x564a8f56d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.121+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/772948846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 0x564a8f56d960 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3a980a2c80 tx=0x7f3a98009c00 comp rx=0 tx=0).stop 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.121+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/772948846 shutdown_connections 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.121+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/772948846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f56ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.121+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/772948846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 0x564a8f56d960 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.121+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/772948846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 0x564a8f56e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.121+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/772948846 >> 172.21.15.33:0/772948846 conn(0x564a8f4cb210 msgr2=0x564a8f567de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.121+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/772948846 shutdown_connections 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.121+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/772948846 wait complete. 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.122+0000 7f3ab73c0b80 1 -- start start 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.122+0000 7f3ab73c0b80 1 Processor -- start 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.122+0000 7f3ab73c0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56d540 0x564a8f576ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.122+0000 7f3ab73c0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56dea0 0x564a8f577010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.122+0000 7f3ab73c0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f57e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.122+0000 7f3ab73c0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8f459560 con 0x564a8f56e860 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.122+0000 7f3ab73c0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8f459200 con 0x564a8f56dea0 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.122+0000 7f3ab73c0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8f3406a0 con 0x564a8f56d540 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.122+0000 7f3ab73c0b80 10 client.0 _ll_register_callbacks cb 0x564a8f462b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3aa8fa2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56d540 0x564a8f576ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3aa97a3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f57e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3aa8fa2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56d540 0x564a8f576ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42572/0 (socket says 172.21.15.33:42572) 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/667649406 learned_addr learned my addr 172.21.15.33:0/667649406 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3aa3fff700 1 --2- 172.21.15.33:0/667649406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56dea0 0x564a8f577010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/667649406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56dea0 msgr2=0x564a8f577010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3aa8fa2700 1 --2- 172.21.15.33:0/667649406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56dea0 0x564a8f577010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/667649406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 msgr2=0x564a8f57e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3aa8fa2700 1 --2- 172.21.15.33:0/667649406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f57e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/667649406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa41a14c0 con 0x564a8f56d540 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3aa8fa2700 1 --2- 172.21.15.33:0/667649406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56d540 0x564a8f576ad0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a980092a0 tx=0x7f3a9800bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d427ec53417c74bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3a83fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3a83fff700 1 -- 172.21.15.33:0/667649406 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a980b2070 con 0x564a8f56d540 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3a83fff700 1 -- 172.21.15.33:0/667649406 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3a980a31b0 con 0x564a8f56d540 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/667649406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a8f416270 con 0x564a8f56d540 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3a83fff700 1 -- 172.21.15.33:0/667649406 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a98021380 con 0x564a8f56d540 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3ab73c0b80 10 client.26581 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.123+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/667649406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564a8f2eab60 con 0x564a8f56d540 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.124+0000 7f3a83fff700 1 -- 172.21.15.33:0/667649406 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564a8f416270 con 0x564a8f56d540 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.124+0000 7f3a83fff700 1 -- 172.21.15.33:0/667649406 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f3a9809a2b0 con 0x564a8f56d540 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.124+0000 7f3ab73c0b80 10 client.26581 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.124+0000 7f3ab73c0b80 10 client.26581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.124+0000 7f3ab73c0b80 20 client.26581 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:21.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.124+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/667649406 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564a8f582470 0x564a8f584890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.124+0000 7f3ab73c0b80 4 client.26581 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.124+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/667649406 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a8f53c810 con 0x564a8f582470 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.124+0000 7f3aa3fff700 1 --2- 172.21.15.33:0/667649406 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564a8f582470 0x564a8f584890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.124+0000 7f3a83fff700 10 client.26581 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.124+0000 7f3aa3fff700 1 --2- 172.21.15.33:0/667649406 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564a8f582470 0x564a8f584890 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c74cadb7591238d1 server_cookie=a556fd654ed2a4fb in_seq=0 out_seq=0 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3a83fff700 1 -- 172.21.15.33:0/667649406 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a8f53c810 con 0x564a8f582470 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3a83fff700 10 client.26581 handle_command_reply: tid=0 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3ab73c0b80 1 client.26581 shutdown 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/667649406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56d540 msgr2=0x564a8f576ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/667649406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56d540 0x564a8f576ad0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a980092a0 tx=0x7f3a9800bbe0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/667649406 shutdown_connections 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/667649406 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564a8f582470 0x564a8f584890 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/667649406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f57e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3a83fff700 0 client.26581 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:21.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/667649406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56d540 0x564a8f576ad0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/667649406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56dea0 0x564a8f577010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/667649406 >> 172.21.15.33:0/667649406 conn(0x564a8f4cb210 msgr2=0x564a8f5675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/667649406 shutdown_connections 2024-05-08T21:23:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.125+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/667649406 wait complete. 2024-05-08T21:23:21.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.126+0000 7f3ab73c0b80 20 client.26581 trim_cache size 0 max 16384 2024-05-08T21:23:21.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3ab73c0b80 1 Processor -- start 2024-05-08T21:23:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3ab73c0b80 1 -- start start 2024-05-08T21:23:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3ab73c0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56d540 0x564a8f576050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3ab73c0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 0x564a8f57dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3ab73c0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56e860 0x564a8f575800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3ab73c0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8f589aa0 con 0x564a8f56d540 2024-05-08T21:23:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3ab73c0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8f5690d0 con 0x564a8f56e860 2024-05-08T21:23:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3ab73c0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8f560040 con 0x564a8f56dea0 2024-05-08T21:23:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3aa8fa2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56d540 0x564a8f576050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3aa8fa2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56d540 0x564a8f576050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46962/0 (socket says 172.21.15.33:46962) 2024-05-08T21:23:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3aa97a3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56e860 0x564a8f575800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/3033266978 learned_addr learned my addr 172.21.15.33:0/3033266978 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.146+0000 7f3aa3fff700 1 --2- 172.21.15.33:0/3033266978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 0x564a8f57dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.147+0000 7f3aa97a3700 1 -- 172.21.15.33:0/3033266978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 msgr2=0x564a8f57dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.147+0000 7f3aa97a3700 1 --2- 172.21.15.33:0/3033266978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 0x564a8f57dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.147+0000 7f3aa97a3700 1 -- 172.21.15.33:0/3033266978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56d540 msgr2=0x564a8f576050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.147+0000 7f3aa97a3700 1 --2- 172.21.15.33:0/3033266978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56d540 0x564a8f576050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.147+0000 7f3aa97a3700 1 -- 172.21.15.33:0/3033266978 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a8f483560 con 0x564a8f56e860 2024-05-08T21:23:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.147+0000 7f3aa97a3700 1 --2- 172.21.15.33:0/3033266978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56e860 0x564a8f575800 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x564a8f568d40 tx=0x7f3a9c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=462935b35e66f74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.147+0000 7f3a8f7fe700 1 -- 172.21.15.33:0/3033266978 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c00e040 con 0x564a8f56e860 2024-05-08T21:23:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.147+0000 7f3a8f7fe700 1 -- 172.21.15.33:0/3033266978 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a9c00f040 con 0x564a8f56e860 2024-05-08T21:23:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.147+0000 7f3a8f7fe700 1 -- 172.21.15.33:0/3033266978 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c004870 con 0x564a8f56e860 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.148+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3033266978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56e860 msgr2=0x564a8f575800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.148+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/3033266978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56e860 0x564a8f575800 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x564a8f568d40 tx=0x7f3a9c00b040 comp rx=0 tx=0).stop 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.148+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3033266978 shutdown_connections 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.148+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/3033266978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56e860 0x564a8f575800 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.148+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/3033266978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56d540 0x564a8f576050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.148+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/3033266978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 0x564a8f57dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.148+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3033266978 >> 172.21.15.33:0/3033266978 conn(0x564a8f4cb210 msgr2=0x564a8f56ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.148+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3033266978 shutdown_connections 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.148+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3033266978 wait complete. 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.148+0000 7f3ab73c0b80 1 -- start start 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3ab73c0b80 1 Processor -- start 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3ab73c0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 0x564a8f57ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3ab73c0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 0x564a8f57bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3ab73c0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f57ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3ab73c0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8f57d3b0 con 0x564a8f56e860 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3ab73c0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8f57d520 con 0x564a8f56d540 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3ab73c0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8f57d6e0 con 0x564a8f56dea0 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3ab73c0b80 10 client.0 _ll_register_callbacks cb 0x564a8f462b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3aa97a3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f57ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3aa8fa2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 0x564a8f57ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3aa8fa2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 0x564a8f57ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:42820/0 (socket says 172.21.15.33:42820) 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3aa3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 0x564a8f57bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3aa97a3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f57ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46968/0 (socket says 172.21.15.33:46968) 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.149+0000 7f3aa8fa2700 1 -- 172.21.15.33:0/3611236065 learned_addr learned my addr 172.21.15.33:0/3611236065 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3aa97a3700 1 -- 172.21.15.33:0/3611236065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 msgr2=0x564a8f57bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3aa97a3700 1 --2- 172.21.15.33:0/3611236065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 0x564a8f57bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3aa97a3700 1 -- 172.21.15.33:0/3611236065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 msgr2=0x564a8f57ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3aa97a3700 1 --2- 172.21.15.33:0/3611236065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 0x564a8f57ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3aa97a3700 1 -- 172.21.15.33:0/3611236065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c0090f0 con 0x564a8f56e860 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3aa97a3700 1 --2- 172.21.15.33:0/3611236065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f57ce70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c000f80 tx=0x7f3a9c0043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cffff6b61eda84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3a83fff700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3a83fff700 1 -- 172.21.15.33:0/3611236065 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c00e030 con 0x564a8f56e860 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3a83fff700 1 -- 172.21.15.33:0/3611236065 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3a9c004cd0 con 0x564a8f56e860 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3611236065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a9809ebf0 con 0x564a8f56e860 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3a83fff700 1 -- 172.21.15.33:0/3611236065 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a9c004e90 con 0x564a8f56e860 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3ab73c0b80 10 client.16701 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.150+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3611236065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x564a8f342cd0 con 0x564a8f56e860 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.151+0000 7f3a83fff700 1 -- 172.21.15.33:0/3611236065 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a9809ebf0 con 0x564a8f56e860 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.151+0000 7f3a83fff700 1 -- 172.21.15.33:0/3611236065 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f3a9c005600 con 0x564a8f56e860 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.151+0000 7f3ab73c0b80 10 client.16701 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:21.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.151+0000 7f3ab73c0b80 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.151+0000 7f3ab73c0b80 20 client.16701 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.151+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/3611236065 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564a8f5d9510 0x564a8f5db930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.151+0000 7f3ab73c0b80 4 client.16701 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.151+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3611236065 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564a8f4b5ec0 con 0x564a8f5d9510 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.151+0000 7f3aa8fa2700 1 --2- 172.21.15.33:0/3611236065 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564a8f5d9510 0x564a8f5db930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.151+0000 7f3aa8fa2700 1 --2- 172.21.15.33:0/3611236065 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564a8f5d9510 0x564a8f5db930 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=144d5a62f39d2018 server_cookie=5bfcd9b17a7e26c3 in_seq=0 out_seq=0 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.151+0000 7f3a83fff700 10 client.16701 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3a83fff700 1 -- 172.21.15.33:0/3611236065 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564a8f4b5ec0 con 0x564a8f5d9510 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3a83fff700 10 client.16701 handle_command_reply: tid=0 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3ab73c0b80 1 client.16701 shutdown 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3611236065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 msgr2=0x564a8f57ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/3611236065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f57ce70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c000f80 tx=0x7f3a9c0043a0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3611236065 shutdown_connections 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/3611236065 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564a8f5d9510 0x564a8f5db930 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/3611236065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564a8f56e860 0x564a8f57ce70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/3611236065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a8f56d540 0x564a8f57ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3a83fff700 0 client.16701 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:21.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3ab73c0b80 1 --2- 172.21.15.33:0/3611236065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564a8f56dea0 0x564a8f57bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3611236065 >> 172.21.15.33:0/3611236065 conn(0x564a8f4cb210 msgr2=0x564a8f575800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3611236065 shutdown_connections 2024-05-08T21:23:21.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3ab73c0b80 1 -- 172.21.15.33:0/3611236065 wait complete. 2024-05-08T21:23:21.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.152+0000 7f3ab73c0b80 20 client.16701 trim_cache size 0 max 16384 2024-05-08T21:23:21.159 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:23:21.159 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:23:21.159 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:23:21.160 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:23:21.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.153+0000 7f3aab206700 1 -- 172.21.15.33:0/81239022 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3a940a11a0 msgr2=0x7f3a940a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.153+0000 7f3aab206700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3a940a11a0 0x7f3a940a3660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3a90005fd0 tx=0x7f3a90009310 comp rx=0 tx=0).stop 2024-05-08T21:23:21.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.153+0000 7f3aab206700 1 -- 172.21.15.33:0/81239022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4100040 msgr2=0x7f3aa4198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.153+0000 7f3aab206700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4100040 0x7f3aa4198fc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009810 tx=0x7f3a9800b790 comp rx=0 tx=0).stop 2024-05-08T21:23:21.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.153+0000 7f3aab206700 1 -- 172.21.15.33:0/81239022 shutdown_connections 2024-05-08T21:23:21.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.153+0000 7f3aab206700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3aa41013f0 0x7f3aa41a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.153+0000 7f3aab206700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4100040 0x7f3aa4198fc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.153+0000 7f3aab206700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3a940a11a0 0x7f3a940a3660 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.153+0000 7f3aab206700 1 --2- 172.21.15.33:0/81239022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3aa4100a00 0x7f3aa4199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.153+0000 7f3aab206700 1 -- 172.21.15.33:0/81239022 >> 172.21.15.33:0/81239022 conn(0x7f3aa40faad0 msgr2=0x7f3aa40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.153+0000 7f3aab206700 1 -- 172.21.15.33:0/81239022 shutdown_connections 2024-05-08T21:23:21.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.153+0000 7f3aab206700 1 -- 172.21.15.33:0/81239022 wait complete. 2024-05-08T21:23:21.169 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:23:21.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:23:21.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.320+0000 7f66d9ec2700 1 -- 172.21.15.33:0/2787062027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d40ffda0 msgr2=0x7f66d41001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.320+0000 7f66d9ec2700 1 --2- 172.21.15.33:0/2787062027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d40ffda0 0x7f66d41001c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66c40098f0 tx=0x7f66c4009c00 comp rx=0 tx=0).stop 2024-05-08T21:23:21.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.320+0000 7f66d9ec2700 1 -- 172.21.15.33:0/2787062027 shutdown_connections 2024-05-08T21:23:21.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.320+0000 7f66d9ec2700 1 --2- 172.21.15.33:0/2787062027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d4101930 0x7f66d41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.320+0000 7f66d9ec2700 1 --2- 172.21.15.33:0/2787062027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d40ffda0 0x7f66d41001c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.320+0000 7f66d9ec2700 1 --2- 172.21.15.33:0/2787062027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66d4100f90 0x7f66d41013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.320+0000 7f66d9ec2700 1 -- 172.21.15.33:0/2787062027 >> 172.21.15.33:0/2787062027 conn(0x7f66d40fb310 msgr2=0x7f66d40fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.320+0000 7f66d9ec2700 1 -- 172.21.15.33:0/2787062027 shutdown_connections 2024-05-08T21:23:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.320+0000 7f66d9ec2700 1 -- 172.21.15.33:0/2787062027 wait complete. 2024-05-08T21:23:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.320+0000 7f66d9ec2700 1 Processor -- start 2024-05-08T21:23:21.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66d9ec2700 1 -- start start 2024-05-08T21:23:21.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66d9ec2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66d40ffda0 0x7f66d4103ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66d9ec2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d4100f90 0x7f66d4102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66d9ec2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d4101930 0x7f66d41028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66d9ec2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d4103050 con 0x7f66d4101930 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66d9ec2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d41031c0 con 0x7f66d40ffda0 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66d9ec2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d41a0e70 con 0x7f66d4100f90 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66cbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d4100f90 0x7f66d4102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66d3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d4101930 0x7f66d41028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66d37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66d40ffda0 0x7f66d4103ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66cbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d4100f90 0x7f66d4102370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42594/0 (socket says 172.21.15.33:42594) 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66d37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66d40ffda0 0x7f66d4103ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:42836/0 (socket says 172.21.15.33:42836) 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66cbfff700 1 -- 172.21.15.33:0/3911885280 learned_addr learned my addr 172.21.15.33:0/3911885280 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66cbfff700 1 -- 172.21.15.33:0/3911885280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66d40ffda0 msgr2=0x7f66d4103ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66cbfff700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66d40ffda0 0x7f66d4103ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66cbfff700 1 -- 172.21.15.33:0/3911885280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d4101930 msgr2=0x7f66d41028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66cbfff700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d4101930 0x7f66d41028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66cbfff700 1 -- 172.21.15.33:0/3911885280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc0095f0 con 0x7f66d4100f90 2024-05-08T21:23:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66d37fe700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66d40ffda0 0x7f66d4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.321+0000 7f66d3fff700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d4101930 0x7f66d41028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.322+0000 7f66cbfff700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d4100f90 0x7f66d4102370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00ec10 tx=0x7f66bc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.322+0000 7f66d17fa700 1 -- 172.21.15.33:0/3911885280 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc009a10 con 0x7f66d4100f90 2024-05-08T21:23:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.322+0000 7f66d17fa700 1 -- 172.21.15.33:0/3911885280 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66bc0044b0 con 0x7f66d4100f90 2024-05-08T21:23:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.322+0000 7f66d17fa700 1 -- 172.21.15.33:0/3911885280 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc003e60 con 0x7f66d4100f90 2024-05-08T21:23:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.322+0000 7f66d9ec2700 1 -- 172.21.15.33:0/3911885280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c4009520 con 0x7f66d4100f90 2024-05-08T21:23:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.322+0000 7f66d9ec2700 1 -- 172.21.15.33:0/3911885280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d41a1480 con 0x7f66d4100f90 2024-05-08T21:23:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.323+0000 7f66d9ec2700 1 -- 172.21.15.33:0/3911885280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d4067480 con 0x7f66d4100f90 2024-05-08T21:23:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.324+0000 7f66d17fa700 1 -- 172.21.15.33:0/3911885280 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f66bc003fe0 con 0x7f66d4100f90 2024-05-08T21:23:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.325+0000 7f66d17fa700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f66c00a11a0 0x7f66c00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.325+0000 7f66d17fa700 1 -- 172.21.15.33:0/3911885280 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f66bc014070 con 0x7f66d4100f90 2024-05-08T21:23:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.325+0000 7f66d37fe700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f66c00a11a0 0x7f66c00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.326+0000 7f66d37fe700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f66c00a11a0 0x7f66c00a3660 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f66c40098c0 tx=0x7f66c401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.329+0000 7f66d17fa700 1 -- 172.21.15.33:0/3911885280 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f66bc0964e0 con 0x7f66d4100f90 2024-05-08T21:23:21.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.632+0000 7f66d9ec2700 1 -- 172.21.15.33:0/3911885280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66d410b6b0 con 0x7f66d4100f90 2024-05-08T21:23:21.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.633+0000 7f66d17fa700 1 -- 172.21.15.33:0/3911885280 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10914 (secure 0 0 0) 0x7f66bc099b00 con 0x7f66d4100f90 2024-05-08T21:23:21.634 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:23:21.635 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:22:25.305999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:23:21.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.635+0000 7f66d9ec2700 1 -- 172.21.15.33:0/3911885280 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f66c00a11a0 msgr2=0x7f66c00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.635+0000 7f66d9ec2700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f66c00a11a0 0x7f66c00a3660 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f66c40098c0 tx=0x7f66c401a040 comp rx=0 tx=0).stop 2024-05-08T21:23:21.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.635+0000 7f66d9ec2700 1 -- 172.21.15.33:0/3911885280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d4100f90 msgr2=0x7f66d4102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.635+0000 7f66d9ec2700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d4100f90 0x7f66d4102370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00ec10 tx=0x7f66bc00ef20 comp rx=0 tx=0).stop 2024-05-08T21:23:21.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.635+0000 7f66d9ec2700 1 -- 172.21.15.33:0/3911885280 shutdown_connections 2024-05-08T21:23:21.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.636+0000 7f66d9ec2700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66d4101930 0x7f66d41028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.636+0000 7f66d9ec2700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f66c00a11a0 0x7f66c00a3660 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.636+0000 7f66d9ec2700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66d40ffda0 0x7f66d4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.636+0000 7f66d9ec2700 1 --2- 172.21.15.33:0/3911885280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66d4100f90 0x7f66d4102370 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.636+0000 7f66d9ec2700 1 -- 172.21.15.33:0/3911885280 >> 172.21.15.33:0/3911885280 conn(0x7f66d40fb310 msgr2=0x7f66d4069020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.636+0000 7f66d9ec2700 1 -- 172.21.15.33:0/3911885280 shutdown_connections 2024-05-08T21:23:21.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.636+0000 7f66d9ec2700 1 -- 172.21.15.33:0/3911885280 wait complete. 2024-05-08T21:23:21.638 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 34 2024-05-08T21:23:21.652 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:23:21.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:21 smithi033 ceph-mon[42528]: pgmap v421: 129 pgs: 129 active+clean; 197 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 548 op/s 2024-05-08T21:23:21.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.806+0000 7f72142ee700 1 -- 172.21.15.33:0/1899704409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f720c101910 msgr2=0x7f720c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.806+0000 7f72142ee700 1 --2- 172.21.15.33:0/1899704409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f720c101910 0x7f720c103d00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7208009990 tx=0x7f7208009ca0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.806+0000 7f72142ee700 1 -- 172.21.15.33:0/1899704409 shutdown_connections 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.806+0000 7f72142ee700 1 --2- 172.21.15.33:0/1899704409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f720c106b70 0x7f720c108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.806+0000 7f72142ee700 1 --2- 172.21.15.33:0/1899704409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f720c101910 0x7f720c103d00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.806+0000 7f72142ee700 1 --2- 172.21.15.33:0/1899704409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f720c104240 0x7f720c106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.806+0000 7f72142ee700 1 -- 172.21.15.33:0/1899704409 >> 172.21.15.33:0/1899704409 conn(0x7f720c0fb300 msgr2=0x7f720c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.806+0000 7f72142ee700 1 -- 172.21.15.33:0/1899704409 shutdown_connections 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.806+0000 7f72142ee700 1 -- 172.21.15.33:0/1899704409 wait complete. 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.807+0000 7f72142ee700 1 Processor -- start 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.807+0000 7f72142ee700 1 -- start start 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.807+0000 7f72142ee700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f720c101910 0x7f720c101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.807+0000 7f72142ee700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f720c104240 0x7f720c0ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.807+0000 7f72142ee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f720c106b70 0x7f720c0ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.807+0000 7f72142ee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c1005a0 con 0x7f720c106b70 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.807+0000 7f72142ee700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c100710 con 0x7f720c104240 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.807+0000 7f72142ee700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c1a0d00 con 0x7f720c101910 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.807+0000 7f721208a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f720c101910 0x7f720c101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.807+0000 7f721288b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f720c106b70 0x7f720c0ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.807+0000 7f721208a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f720c101910 0x7f720c101230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42602/0 (socket says 172.21.15.33:42602) 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f721208a700 1 -- 172.21.15.33:0/1034479299 learned_addr learned my addr 172.21.15.33:0/1034479299 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f7211889700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f720c104240 0x7f720c0ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f721288b700 1 -- 172.21.15.33:0/1034479299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f720c101910 msgr2=0x7f720c101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f721288b700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f720c101910 0x7f720c101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f721288b700 1 -- 172.21.15.33:0/1034479299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f720c104240 msgr2=0x7f720c0ff8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f721288b700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f720c104240 0x7f720c0ff8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f721288b700 1 -- 172.21.15.33:0/1034479299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72080095f0 con 0x7f720c106b70 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f7211889700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f720c104240 0x7f720c0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:23:21.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f721208a700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f720c101910 0x7f720c101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:21.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f721288b700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f720c106b70 0x7f720c0ffe00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7204007f10 tx=0x7f720400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f72037fe700 1 -- 172.21.15.33:0/1034479299 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204004db0 con 0x7f720c106b70 2024-05-08T21:23:21.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f72037fe700 1 -- 172.21.15.33:0/1034479299 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f720400f040 con 0x7f720c106b70 2024-05-08T21:23:21.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f72142ee700 1 -- 172.21.15.33:0/1034479299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c1a0f90 con 0x7f720c106b70 2024-05-08T21:23:21.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f72037fe700 1 -- 172.21.15.33:0/1034479299 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204005960 con 0x7f720c106b70 2024-05-08T21:23:21.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.808+0000 7f72142ee700 1 -- 172.21.15.33:0/1034479299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c1a13c0 con 0x7f720c106b70 2024-05-08T21:23:21.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.811+0000 7f72037fe700 1 -- 172.21.15.33:0/1034479299 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f720400db70 con 0x7f720c106b70 2024-05-08T21:23:21.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.812+0000 7f72037fe700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71f80a1120 0x7f71f80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.812+0000 7f721208a700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71f80a1120 0x7f71f80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.813+0000 7f72037fe700 1 -- 172.21.15.33:0/1034479299 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f72040cc5a0 con 0x7f720c106b70 2024-05-08T21:23:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.813+0000 7f72204a8b80 1 Processor -- start 2024-05-08T21:23:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.813+0000 7f721208a700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71f80a1120 0x7f71f80a35e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f720800b2e0 tx=0x7f720800b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.813+0000 7f72204a8b80 1 -- start start 2024-05-08T21:23:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.813+0000 7f72204a8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385540 0x55bfed385960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.813+0000 7f72204a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385ea0 0x55bfed386320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.813+0000 7f72204a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed386860 0x55bfed386ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.813+0000 7f72204a8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfed2dc150 con 0x55bfed385540 2024-05-08T21:23:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.813+0000 7f72204a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfed2d9b10 con 0x55bfed386860 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.813+0000 7f72204a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfed2d5400 con 0x55bfed385ea0 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f721208a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385540 0x55bfed385960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f7211889700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385ea0 0x55bfed386320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f721288b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed386860 0x55bfed386ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f7211889700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385ea0 0x55bfed386320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42612/0 (socket says 172.21.15.33:42612) 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f721208a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385540 0x55bfed385960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47018/0 (socket says 172.21.15.33:47018) 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f7211889700 1 -- 172.21.15.33:0/4136534491 learned_addr learned my addr 172.21.15.33:0/4136534491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f721288b700 1 -- 172.21.15.33:0/4136534491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385ea0 msgr2=0x55bfed386320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f721288b700 1 --2- 172.21.15.33:0/4136534491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385ea0 0x55bfed386320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f721288b700 1 -- 172.21.15.33:0/4136534491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385540 msgr2=0x55bfed385960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f721288b700 1 --2- 172.21.15.33:0/4136534491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385540 0x55bfed385960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f721288b700 1 -- 172.21.15.33:0/4136534491 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfed1563b0 con 0x55bfed386860 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f7211889700 1 --2- 172.21.15.33:0/4136534491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385ea0 0x55bfed386320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:21.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f721288b700 1 --2- 172.21.15.33:0/4136534491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed386860 0x55bfed386ce0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f72040c9cf0 tx=0x7f7204099440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cca08156c89808e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f7200da4700 1 -- 172.21.15.33:0/4136534491 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204005ac0 con 0x55bfed386860 2024-05-08T21:23:21.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f7200da4700 1 -- 172.21.15.33:0/4136534491 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f720400de20 con 0x55bfed386860 2024-05-08T21:23:21.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.814+0000 7f7200da4700 1 -- 172.21.15.33:0/4136534491 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720409d510 con 0x55bfed386860 2024-05-08T21:23:21.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.815+0000 7f72204a8b80 1 -- 172.21.15.33:0/4136534491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed386860 msgr2=0x55bfed386ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.815+0000 7f72204a8b80 1 --2- 172.21.15.33:0/4136534491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed386860 0x55bfed386ce0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f72040c9cf0 tx=0x7f7204099440 comp rx=0 tx=0).stop 2024-05-08T21:23:21.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.815+0000 7f72204a8b80 1 -- 172.21.15.33:0/4136534491 shutdown_connections 2024-05-08T21:23:21.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.815+0000 7f72204a8b80 1 --2- 172.21.15.33:0/4136534491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed386860 0x55bfed386ce0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.815+0000 7f72204a8b80 1 --2- 172.21.15.33:0/4136534491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385540 0x55bfed385960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.815+0000 7f72204a8b80 1 --2- 172.21.15.33:0/4136534491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385ea0 0x55bfed386320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.815+0000 7f72204a8b80 1 -- 172.21.15.33:0/4136534491 >> 172.21.15.33:0/4136534491 conn(0x55bfed2e3210 msgr2=0x55bfed37fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.815+0000 7f72204a8b80 1 -- 172.21.15.33:0/4136534491 shutdown_connections 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.815+0000 7f72204a8b80 1 -- 172.21.15.33:0/4136534491 wait complete. 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f72204a8b80 1 -- start start 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f72204a8b80 1 Processor -- start 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f72204a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385540 0x55bfed38ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f72204a8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385ea0 0x55bfed38f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f72204a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed386860 0x55bfed3963f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f72204a8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfed271560 con 0x55bfed385ea0 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f72204a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfed271200 con 0x55bfed385540 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f72204a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfed1586a0 con 0x55bfed386860 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f72204a8b80 10 client.0 _ll_register_callbacks cb 0x55bfed27ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:21.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f7211889700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385ea0 0x55bfed38f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f7211889700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385ea0 0x55bfed38f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47034/0 (socket says 172.21.15.33:47034) 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f7211889700 1 -- 172.21.15.33:0/3813198468 learned_addr learned my addr 172.21.15.33:0/3813198468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.816+0000 7f721288b700 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed386860 0x55bfed3963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f721208a700 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385540 0x55bfed38ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f7211889700 1 -- 172.21.15.33:0/3813198468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed386860 msgr2=0x55bfed3963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f7211889700 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed386860 0x55bfed3963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f7211889700 1 -- 172.21.15.33:0/3813198468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385540 msgr2=0x55bfed38ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f7211889700 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385540 0x55bfed38ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f7211889700 1 -- 172.21.15.33:0/3813198468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc009a10 con 0x55bfed385ea0 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f721208a700 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385540 0x55bfed38ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f7211889700 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385ea0 0x55bfed38f3b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f71fc012980 tx=0x7f71fc012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205a1bff7660394 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f71f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f71f5ffb700 1 -- 172.21.15.33:0/3813198468 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc017070 con 0x55bfed385ea0 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f71f5ffb700 1 -- 172.21.15.33:0/3813198468 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f71fc011ac0 con 0x55bfed385ea0 2024-05-08T21:23:21.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f72204a8b80 1 -- 172.21.15.33:0/3813198468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfed22e270 con 0x55bfed385ea0 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f71f5ffb700 1 -- 172.21.15.33:0/3813198468 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71fc011c80 con 0x55bfed385ea0 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f72204a8b80 10 client.16719 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.817+0000 7f72204a8b80 1 -- 172.21.15.33:0/3813198468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f720c1a13c0 con 0x55bfed385ea0 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.818+0000 7f71f5ffb700 1 -- 172.21.15.33:0/3813198468 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bfed22e270 con 0x55bfed385ea0 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.818+0000 7f71f5ffb700 1 -- 172.21.15.33:0/3813198468 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f71fc00f460 con 0x55bfed385ea0 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.818+0000 7f72204a8b80 10 client.16719 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.818+0000 7f72204a8b80 10 client.16719 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.818+0000 7f72204a8b80 20 client.16719 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.818+0000 7f72204a8b80 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55bfed39ad80 0x55bfed39d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.818+0000 7f72204a8b80 4 client.16719 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.818+0000 7f72204a8b80 1 -- 172.21.15.33:0/3813198468 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfed354810 con 0x55bfed39ad80 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.818+0000 7f721208a700 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55bfed39ad80 0x55bfed39d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f71f5ffb700 10 client.16719 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f721208a700 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55bfed39ad80 0x55bfed39d1a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a3c0c364f9dd501 server_cookie=87fe669dfe0edab6 in_seq=0 out_seq=0 2024-05-08T21:23:21.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f71f5ffb700 1 -- 172.21.15.33:0/3813198468 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfed354810 con 0x55bfed39ad80 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f71f5ffb700 10 client.16719 handle_command_reply: tid=0 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f72204a8b80 1 client.16719 shutdown 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f72204a8b80 1 -- 172.21.15.33:0/3813198468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385ea0 msgr2=0x55bfed38f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f72204a8b80 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385ea0 0x55bfed38f3b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f71fc012980 tx=0x7f71fc012c90 comp rx=0 tx=0).stop 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f72204a8b80 1 -- 172.21.15.33:0/3813198468 shutdown_connections 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f72204a8b80 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55bfed39ad80 0x55bfed39d1a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f72204a8b80 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed386860 0x55bfed3963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f71f5ffb700 0 client.16719 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f72204a8b80 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385540 0x55bfed38ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f72204a8b80 1 --2- 172.21.15.33:0/3813198468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed385ea0 0x55bfed38f3b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.819+0000 7f72204a8b80 1 -- 172.21.15.33:0/3813198468 >> 172.21.15.33:0/3813198468 conn(0x55bfed2e3210 msgr2=0x55bfed37f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.820+0000 7f72204a8b80 1 -- 172.21.15.33:0/3813198468 shutdown_connections 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.820+0000 7f72204a8b80 1 -- 172.21.15.33:0/3813198468 wait complete. 2024-05-08T21:23:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.820+0000 7f72204a8b80 20 client.16719 trim_cache size 0 max 16384 2024-05-08T21:23:21.827 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:21 smithi071 ceph-mon[47074]: pgmap v421: 129 pgs: 129 active+clean; 197 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 548 op/s 2024-05-08T21:23:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f72204a8b80 1 Processor -- start 2024-05-08T21:23:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f72204a8b80 1 -- start start 2024-05-08T21:23:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f72204a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385540 0x55bfed38da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f72204a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385ea0 0x55bfed38e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f72204a8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 0x55bfed3963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f72204a8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfed37cdc0 con 0x55bfed386860 2024-05-08T21:23:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f72204a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfed381040 con 0x55bfed385ea0 2024-05-08T21:23:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f72204a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfed3a23b0 con 0x55bfed385540 2024-05-08T21:23:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f721208a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385540 0x55bfed38da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f721208a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385540 0x55bfed38da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42628/0 (socket says 172.21.15.33:42628) 2024-05-08T21:23:21.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f721208a700 1 -- 172.21.15.33:0/1588990021 learned_addr learned my addr 172.21.15.33:0/1588990021 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f721288b700 1 --2- 172.21.15.33:0/1588990021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 0x55bfed3963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.836+0000 7f7211889700 1 --2- 172.21.15.33:0/1588990021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385ea0 0x55bfed38e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.837+0000 7f721288b700 1 -- 172.21.15.33:0/1588990021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385540 msgr2=0x55bfed38da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.837+0000 7f721288b700 1 --2- 172.21.15.33:0/1588990021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385540 0x55bfed38da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.837+0000 7f721288b700 1 -- 172.21.15.33:0/1588990021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385ea0 msgr2=0x55bfed38e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.837+0000 7f721288b700 1 --2- 172.21.15.33:0/1588990021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385ea0 0x55bfed38e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.837+0000 7f721288b700 1 -- 172.21.15.33:0/1588990021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfed102b60 con 0x55bfed386860 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.837+0000 7f7211889700 1 --2- 172.21.15.33:0/1588990021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385ea0 0x55bfed38e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.837+0000 7f721208a700 1 --2- 172.21.15.33:0/1588990021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385540 0x55bfed38da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.837+0000 7f721288b700 1 --2- 172.21.15.33:0/1588990021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 0x55bfed3963b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7204099370 tx=0x7f7204084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fad2c8ca60e1ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.837+0000 7f72015a5700 1 -- 172.21.15.33:0/1588990021 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204005ac0 con 0x55bfed386860 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.837+0000 7f72015a5700 1 -- 172.21.15.33:0/1588990021 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7204099d70 con 0x55bfed386860 2024-05-08T21:23:21.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.837+0000 7f72015a5700 1 -- 172.21.15.33:0/1588990021 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720409d4a0 con 0x55bfed386860 2024-05-08T21:23:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.838+0000 7f72204a8b80 1 -- 172.21.15.33:0/1588990021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 msgr2=0x55bfed3963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.838+0000 7f72204a8b80 1 --2- 172.21.15.33:0/1588990021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 0x55bfed3963b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7204099370 tx=0x7f7204084040 comp rx=0 tx=0).stop 2024-05-08T21:23:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.838+0000 7f72204a8b80 1 -- 172.21.15.33:0/1588990021 shutdown_connections 2024-05-08T21:23:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.838+0000 7f72204a8b80 1 --2- 172.21.15.33:0/1588990021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 0x55bfed3963b0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.838+0000 7f72204a8b80 1 --2- 172.21.15.33:0/1588990021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385540 0x55bfed38da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.838+0000 7f72204a8b80 1 --2- 172.21.15.33:0/1588990021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385ea0 0x55bfed38e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.838+0000 7f72204a8b80 1 -- 172.21.15.33:0/1588990021 >> 172.21.15.33:0/1588990021 conn(0x55bfed2e3210 msgr2=0x55bfed37c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.838+0000 7f72204a8b80 1 -- 172.21.15.33:0/1588990021 shutdown_connections 2024-05-08T21:23:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.838+0000 7f72204a8b80 1 -- 172.21.15.33:0/1588990021 wait complete. 2024-05-08T21:23:21.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.838+0000 7f72204a8b80 1 -- start start 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f72204a8b80 1 Processor -- start 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f72204a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385540 0x55bfed391710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f72204a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385ea0 0x55bfed391c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f72204a8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 0x55bfed394b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f72204a8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfed3950c0 con 0x55bfed386860 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f72204a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfed395230 con 0x55bfed385540 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f72204a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfed3953f0 con 0x55bfed385ea0 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f72204a8b80 10 client.0 _ll_register_callbacks cb 0x55bfed27ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f721288b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 0x55bfed394b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f721288b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 0x55bfed394b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47054/0 (socket says 172.21.15.33:47054) 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f721288b700 1 -- 172.21.15.33:0/2585405178 learned_addr learned my addr 172.21.15.33:0/2585405178 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f721208a700 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385540 0x55bfed391710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f7211889700 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385ea0 0x55bfed391c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f721288b700 1 -- 172.21.15.33:0/2585405178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385ea0 msgr2=0x55bfed391c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f721288b700 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385ea0 0x55bfed391c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f721288b700 1 -- 172.21.15.33:0/2585405178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385540 msgr2=0x55bfed391710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f721288b700 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385540 0x55bfed391710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f721288b700 1 -- 172.21.15.33:0/2585405178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c1a13c0 con 0x55bfed386860 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f721288b700 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 0x55bfed394b80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7204095b70 tx=0x7f7204005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16a1a135406eb6f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.839+0000 7f71f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f71f5ffb700 1 -- 172.21.15.33:0/2585405178 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204099670 con 0x55bfed386860 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f71f5ffb700 1 -- 172.21.15.33:0/2585405178 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f72040cc320 con 0x55bfed386860 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f72204a8b80 1 -- 172.21.15.33:0/2585405178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f720400de20 con 0x55bfed386860 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f71f5ffb700 1 -- 172.21.15.33:0/2585405178 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72040c9da0 con 0x55bfed386860 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f72204a8b80 10 client.16725 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f72204a8b80 1 -- 172.21.15.33:0/2585405178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55bfed1563b0 con 0x55bfed386860 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f71f5ffb700 1 -- 172.21.15.33:0/2585405178 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72040a5020 con 0x55bfed386860 2024-05-08T21:23:21.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f71f5ffb700 1 -- 172.21.15.33:0/2585405178 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f7204098b60 con 0x55bfed386860 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f72204a8b80 10 client.16725 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f72204a8b80 10 client.16725 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f72204a8b80 20 client.16725 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f72204a8b80 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55bfed3f1350 0x55bfed3f3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f72204a8b80 4 client.16725 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f72204a8b80 1 -- 172.21.15.33:0/2585405178 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bfed2cdec0 con 0x55bfed3f1350 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.840+0000 7f721208a700 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55bfed3f1350 0x55bfed3f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f71f5ffb700 10 client.16725 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f721208a700 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55bfed3f1350 0x55bfed3f3740 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78987f38653073c7 server_cookie=87cdf33624ff4674 in_seq=0 out_seq=0 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f71f5ffb700 1 -- 172.21.15.33:0/2585405178 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bfed2cdec0 con 0x55bfed3f1350 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f71f5ffb700 10 client.16725 handle_command_reply: tid=0 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f72204a8b80 1 client.16725 shutdown 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f72204a8b80 1 -- 172.21.15.33:0/2585405178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 msgr2=0x55bfed394b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f72204a8b80 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 0x55bfed394b80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7204095b70 tx=0x7f7204005b50 comp rx=0 tx=0).stop 2024-05-08T21:23:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f72204a8b80 1 -- 172.21.15.33:0/2585405178 shutdown_connections 2024-05-08T21:23:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f72204a8b80 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bfed386860 0x55bfed394b80 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f72204a8b80 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bfed385540 0x55bfed391710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f72204a8b80 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55bfed3f1350 0x55bfed3f3740 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f72204a8b80 1 --2- 172.21.15.33:0/2585405178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfed385ea0 0x55bfed391c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f71f5ffb700 0 client.16725 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:23:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f72204a8b80 1 -- 172.21.15.33:0/2585405178 >> 172.21.15.33:0/2585405178 conn(0x55bfed2e3210 msgr2=0x55bfed38ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f72204a8b80 1 -- 172.21.15.33:0/2585405178 shutdown_connections 2024-05-08T21:23:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.841+0000 7f72204a8b80 1 -- 172.21.15.33:0/2585405178 wait complete. 2024-05-08T21:23:21.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72204a8b80 20 client.16725 trim_cache size 0 max 16384 2024-05-08T21:23:21.849 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:23:21.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72142ee700 1 -- 172.21.15.33:0/1034479299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71f80a1120 msgr2=0x7f71f80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72142ee700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71f80a1120 0x7f71f80a35e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f720800b2e0 tx=0x7f720800b610 comp rx=0 tx=0).stop 2024-05-08T21:23:21.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72142ee700 1 -- 172.21.15.33:0/1034479299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f720c106b70 msgr2=0x7f720c0ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:21.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72142ee700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f720c106b70 0x7f720c0ffe00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7204007f10 tx=0x7f720400d3b0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72142ee700 1 -- 172.21.15.33:0/1034479299 shutdown_connections 2024-05-08T21:23:21.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72142ee700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f720c106b70 0x7f720c0ffe00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72142ee700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f720c101910 0x7f720c101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72142ee700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71f80a1120 0x7f71f80a35e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72142ee700 1 --2- 172.21.15.33:0/1034479299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f720c104240 0x7f720c0ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:21.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72142ee700 1 -- 172.21.15.33:0/1034479299 >> 172.21.15.33:0/1034479299 conn(0x7f720c0fb300 msgr2=0x7f720c103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:21.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72142ee700 1 -- 172.21.15.33:0/1034479299 shutdown_connections 2024-05-08T21:23:21.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:21.842+0000 7f72142ee700 1 -- 172.21.15.33:0/1034479299 wait complete. 2024-05-08T21:23:21.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:21 smithi082 ceph-mon[47512]: pgmap v421: 129 pgs: 129 active+clean; 197 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.4 MiB/s wr, 548 op/s 2024-05-08T21:23:21.858 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:23:22.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.050+0000 7fa09fb49700 1 -- 172.21.15.33:0/2749573850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa098100f50 msgr2=0x7fa0981013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.050+0000 7fa09fb49700 1 --2- 172.21.15.33:0/2749573850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa098100f50 0x7fa0981013b0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa0940099c0 tx=0x7fa094009cd0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.050+0000 7fa09fb49700 1 -- 172.21.15.33:0/2749573850 shutdown_connections 2024-05-08T21:23:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.050+0000 7fa09fb49700 1 --2- 172.21.15.33:0/2749573850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0981018f0 0x7fa0981081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.050+0000 7fa09fb49700 1 --2- 172.21.15.33:0/2749573850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0980ffd60 0x7fa098100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.050+0000 7fa09fb49700 1 --2- 172.21.15.33:0/2749573850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa098100f50 0x7fa0981013b0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.050+0000 7fa09fb49700 1 -- 172.21.15.33:0/2749573850 >> 172.21.15.33:0/2749573850 conn(0x7fa0980fb290 msgr2=0x7fa0980fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.050+0000 7fa09fb49700 1 -- 172.21.15.33:0/2749573850 shutdown_connections 2024-05-08T21:23:22.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.050+0000 7fa09fb49700 1 -- 172.21.15.33:0/2749573850 wait complete. 2024-05-08T21:23:22.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.051+0000 7fa09fb49700 1 Processor -- start 2024-05-08T21:23:22.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.051+0000 7fa09fb49700 1 -- start start 2024-05-08T21:23:22.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.051+0000 7fa09fb49700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0980ffd60 0x7fa0981972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.051+0000 7fa09fb49700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa098100f50 0x7fa098199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.051+0000 7fa09fb49700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0981018f0 0x7fa09819be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.051+0000 7fa09fb49700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09819c5e0 con 0x7fa0980ffd60 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.051+0000 7fa09fb49700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09819c750 con 0x7fa098100f50 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.051+0000 7fa09fb49700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09819c910 con 0x7fa0981018f0 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09d8e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0980ffd60 0x7fa0981972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09e0e6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0981018f0 0x7fa09819be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09d8e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0980ffd60 0x7fa0981972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47078/0 (socket says 172.21.15.33:47078) 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09d0e4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa098100f50 0x7fa098199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09d8e5700 1 -- 172.21.15.33:0/3256936887 learned_addr learned my addr 172.21.15.33:0/3256936887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09d8e5700 1 -- 172.21.15.33:0/3256936887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0981018f0 msgr2=0x7fa09819be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09d8e5700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0981018f0 0x7fa09819be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09d8e5700 1 -- 172.21.15.33:0/3256936887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa098100f50 msgr2=0x7fa098199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09d8e5700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa098100f50 0x7fa098199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09d8e5700 1 -- 172.21.15.33:0/3256936887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0940095f0 con 0x7fa0980ffd60 2024-05-08T21:23:22.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09e0e6700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0981018f0 0x7fa09819be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09d0e4700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa098100f50 0x7fa098199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09d8e5700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0980ffd60 0x7fa0981972c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa08800e970 tx=0x7fa08800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa08effd700 1 -- 172.21.15.33:0/3256936887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088009790 con 0x7fa0980ffd60 2024-05-08T21:23:22.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa08effd700 1 -- 172.21.15.33:0/3256936887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa08800c7e0 con 0x7fa0980ffd60 2024-05-08T21:23:22.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa08effd700 1 -- 172.21.15.33:0/3256936887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088003bd0 con 0x7fa0980ffd60 2024-05-08T21:23:22.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09fb49700 1 -- 172.21.15.33:0/3256936887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09819cbf0 con 0x7fa0980ffd60 2024-05-08T21:23:22.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.052+0000 7fa09fb49700 1 -- 172.21.15.33:0/3256936887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09819d080 con 0x7fa0980ffd60 2024-05-08T21:23:22.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.056+0000 7fa08effd700 1 -- 172.21.15.33:0/3256936887 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa08800c950 con 0x7fa0980ffd60 2024-05-08T21:23:22.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.057+0000 7fa08effd700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa0840a1120 0x7fa0840a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.057+0000 7fa0abd03b80 1 Processor -- start 2024-05-08T21:23:22.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.057+0000 7fa08effd700 1 -- 172.21.15.33:0/3256936887 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa088014070 con 0x7fa0980ffd60 2024-05-08T21:23:22.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.057+0000 7fa09d0e4700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa0840a1120 0x7fa0840a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.057+0000 7fa0abd03b80 1 -- start start 2024-05-08T21:23:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.057+0000 7fa09d0e4700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa0840a1120 0x7fa0840a35e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa094006010 tx=0x7fa09401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa0abd03b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441540 0x55c9bc441960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa0abd03b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc442320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa0abd03b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 0x55c9bc442ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa0abd03b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9bc398150 con 0x55c9bc442860 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa0abd03b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9bc395b10 con 0x55c9bc441ea0 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa0abd03b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9bc391400 con 0x55c9bc441540 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09e0e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 0x55c9bc442ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09d8e5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441540 0x55c9bc441960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09d0e4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc442320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09d8e5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441540 0x55c9bc441960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42644/0 (socket says 172.21.15.33:42644) 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09e0e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 0x55c9bc442ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47086/0 (socket says 172.21.15.33:47086) 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09d8e5700 1 -- 172.21.15.33:0/1471429058 learned_addr learned my addr 172.21.15.33:0/1471429058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09d0e4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc442320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:42918/0 (socket says 172.21.15.33:42918) 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09d8e5700 1 -- 172.21.15.33:0/1471429058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 msgr2=0x55c9bc442320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09d8e5700 1 --2- 172.21.15.33:0/1471429058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc442320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09d8e5700 1 -- 172.21.15.33:0/1471429058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 msgr2=0x55c9bc442ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09d8e5700 1 --2- 172.21.15.33:0/1471429058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 0x55c9bc442ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09d8e5700 1 -- 172.21.15.33:0/1471429058 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9bc2123b0 con 0x55c9bc441540 2024-05-08T21:23:22.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa09d8e5700 1 --2- 172.21.15.33:0/1471429058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441540 0x55c9bc441960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa0880a1e80 tx=0x7fa088099890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77411ab6c97f2078 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.058+0000 7fa083fff700 1 -- 172.21.15.33:0/1471429058 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0880b1070 con 0x55c9bc441540 2024-05-08T21:23:22.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.059+0000 7fa083fff700 1 -- 172.21.15.33:0/1471429058 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa088099f30 con 0x55c9bc441540 2024-05-08T21:23:22.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.059+0000 7fa083fff700 1 -- 172.21.15.33:0/1471429058 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0880982d0 con 0x55c9bc441540 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.059+0000 7fa0abd03b80 1 -- 172.21.15.33:0/1471429058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441540 msgr2=0x55c9bc441960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.059+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/1471429058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441540 0x55c9bc441960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa0880a1e80 tx=0x7fa088099890 comp rx=0 tx=0).stop 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.059+0000 7fa0abd03b80 1 -- 172.21.15.33:0/1471429058 shutdown_connections 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.059+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/1471429058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 0x55c9bc442ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.059+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/1471429058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441540 0x55c9bc441960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.059+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/1471429058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc442320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.059+0000 7fa0abd03b80 1 -- 172.21.15.33:0/1471429058 >> 172.21.15.33:0/1471429058 conn(0x55c9bc39f210 msgr2=0x55c9bc43bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.059+0000 7fa0abd03b80 1 -- 172.21.15.33:0/1471429058 shutdown_connections 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.059+0000 7fa0abd03b80 1 -- 172.21.15.33:0/1471429058 wait complete. 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.060+0000 7fa0abd03b80 1 -- start start 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.060+0000 7fa0abd03b80 1 Processor -- start 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa0abd03b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441540 0x55c9bc44ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa0abd03b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc44b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa0abd03b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 0x55c9bc452400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa0abd03b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9bc32d560 con 0x55c9bc442860 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa0abd03b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9bc32d200 con 0x55c9bc441ea0 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa0abd03b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9bc2146a0 con 0x55c9bc441540 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa0abd03b80 10 client.0 _ll_register_callbacks cb 0x55c9bc336b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa09e0e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 0x55c9bc452400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa09d0e4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc44b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa09e0e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 0x55c9bc452400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47088/0 (socket says 172.21.15.33:47088) 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa09e0e6700 1 -- 172.21.15.33:0/3509750743 learned_addr learned my addr 172.21.15.33:0/3509750743 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa09d8e5700 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441540 0x55c9bc44ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa09e0e6700 1 -- 172.21.15.33:0/3509750743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441540 msgr2=0x55c9bc44ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa09e0e6700 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441540 0x55c9bc44ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa09e0e6700 1 -- 172.21.15.33:0/3509750743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 msgr2=0x55c9bc44b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa09e0e6700 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc44b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa09e0e6700 1 -- 172.21.15.33:0/3509750743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09819d080 con 0x55c9bc442860 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa09d0e4700 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc44b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa09e0e6700 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 0x55c9bc452400 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa090004700 tx=0x7fa090004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a363b93b431b0c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.061+0000 7fa0817fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0817fa700 1 -- 172.21.15.33:0/3509750743 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090030b90 con 0x55c9bc442860 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0817fa700 1 -- 172.21.15.33:0/3509750743 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa090030cf0 con 0x55c9bc442860 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3509750743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9bc2ea270 con 0x55c9bc442860 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0817fa700 1 -- 172.21.15.33:0/3509750743 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa090030eb0 con 0x55c9bc442860 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0abd03b80 10 client.16737 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3509750743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c9bc1beb60 con 0x55c9bc442860 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0817fa700 1 -- 172.21.15.33:0/3509750743 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c9bc2ea270 con 0x55c9bc442860 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0817fa700 1 -- 172.21.15.33:0/3509750743 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fa09003e070 con 0x55c9bc442860 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0abd03b80 10 client.16737 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0abd03b80 10 client.16737 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:23:22.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0abd03b80 20 client.16737 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55c9bc456d90 0x55c9bc4591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0abd03b80 4 client.16737 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.062+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3509750743 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9bc410810 con 0x55c9bc456d90 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.063+0000 7fa09d8e5700 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55c9bc456d90 0x55c9bc4591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.063+0000 7fa09d8e5700 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55c9bc456d90 0x55c9bc4591b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a66dab463a5b0c23 server_cookie=6b4e6dcb26dc53fe in_seq=0 out_seq=0 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.063+0000 7fa0817fa700 10 client.16737 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.063+0000 7fa0817fa700 1 -- 172.21.15.33:0/3509750743 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c9bc410810 con 0x55c9bc456d90 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.063+0000 7fa0817fa700 10 client.16737 handle_command_reply: tid=0 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0abd03b80 1 client.16737 shutdown 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3509750743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 msgr2=0x55c9bc452400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 0x55c9bc452400 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa090004700 tx=0x7fa090004730 comp rx=0 tx=0).stop 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3509750743 shutdown_connections 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc442860 0x55c9bc452400 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55c9bc456d90 0x55c9bc4591b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441540 0x55c9bc44ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/3509750743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc44b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0817fa700 0 client.16737 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:23:22.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3509750743 >> 172.21.15.33:0/3509750743 conn(0x55c9bc39f210 msgr2=0x55c9bc43b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3509750743 shutdown_connections 2024-05-08T21:23:22.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3509750743 wait complete. 2024-05-08T21:23:22.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.064+0000 7fa0abd03b80 20 client.16737 trim_cache size 0 max 16384 2024-05-08T21:23:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.081+0000 7fa0abd03b80 1 Processor -- start 2024-05-08T21:23:22.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.081+0000 7fa0abd03b80 1 -- start start 2024-05-08T21:23:22.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa0abd03b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc441540 0x55c9bc449a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa0abd03b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc44a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa0abd03b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc442860 0x55c9bc452320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa0abd03b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9bc448b70 con 0x55c9bc441540 2024-05-08T21:23:22.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa0abd03b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9bc434040 con 0x55c9bc441ea0 2024-05-08T21:23:22.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa0abd03b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9bc43d0d0 con 0x55c9bc442860 2024-05-08T21:23:22.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa09d0e4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc44a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa09d8e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc441540 0x55c9bc449a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa09e0e6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc442860 0x55c9bc452320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa09d0e4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc44a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:42930/0 (socket says 172.21.15.33:42930) 2024-05-08T21:23:22.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa09d0e4700 1 -- 172.21.15.33:0/2202772657 learned_addr learned my addr 172.21.15.33:0/2202772657 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa09e0e6700 1 -- 172.21.15.33:0/2202772657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 msgr2=0x55c9bc44a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa09e0e6700 1 --2- 172.21.15.33:0/2202772657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc44a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa09e0e6700 1 -- 172.21.15.33:0/2202772657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc441540 msgr2=0x55c9bc449a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa09e0e6700 1 --2- 172.21.15.33:0/2202772657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc441540 0x55c9bc449a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa09e0e6700 1 -- 172.21.15.33:0/2202772657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9bc357560 con 0x55c9bc442860 2024-05-08T21:23:22.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.082+0000 7fa09d0e4700 1 --2- 172.21.15.33:0/2202772657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc44a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.083+0000 7fa09e0e6700 1 --2- 172.21.15.33:0/2202772657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc442860 0x55c9bc452320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa09000b010 tx=0x7fa090005aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6e35fcd97c398df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.083+0000 7fa08cda4700 1 -- 172.21.15.33:0/2202772657 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa090040ad0 con 0x55c9bc442860 2024-05-08T21:23:22.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.083+0000 7fa08cda4700 1 -- 172.21.15.33:0/2202772657 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa090030e20 con 0x55c9bc442860 2024-05-08T21:23:22.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.083+0000 7fa08cda4700 1 -- 172.21.15.33:0/2202772657 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09004ea30 con 0x55c9bc442860 2024-05-08T21:23:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.084+0000 7fa0abd03b80 1 -- 172.21.15.33:0/2202772657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc442860 msgr2=0x55c9bc452320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.084+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/2202772657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc442860 0x55c9bc452320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa09000b010 tx=0x7fa090005aa0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.084+0000 7fa0abd03b80 1 -- 172.21.15.33:0/2202772657 shutdown_connections 2024-05-08T21:23:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.084+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/2202772657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc442860 0x55c9bc452320 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.084+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/2202772657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc441540 0x55c9bc449a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.084+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/2202772657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc441ea0 0x55c9bc44a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.084+0000 7fa0abd03b80 1 -- 172.21.15.33:0/2202772657 >> 172.21.15.33:0/2202772657 conn(0x55c9bc39f210 msgr2=0x55c9bc438210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.084+0000 7fa0abd03b80 1 -- 172.21.15.33:0/2202772657 shutdown_connections 2024-05-08T21:23:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.084+0000 7fa0abd03b80 1 -- 172.21.15.33:0/2202772657 wait complete. 2024-05-08T21:23:22.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.084+0000 7fa0abd03b80 1 -- start start 2024-05-08T21:23:22.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.084+0000 7fa0abd03b80 1 Processor -- start 2024-05-08T21:23:22.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa0abd03b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc441540 0x55c9bc44fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa0abd03b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441ea0 0x55c9bc450040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa0abd03b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc442860 0x55c9bc451000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa0abd03b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9bc451540 con 0x55c9bc441540 2024-05-08T21:23:22.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa0abd03b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9bc4516b0 con 0x55c9bc442860 2024-05-08T21:23:22.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa0abd03b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9bc451870 con 0x55c9bc441ea0 2024-05-08T21:23:22.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa0abd03b80 10 client.0 _ll_register_callbacks cb 0x55c9bc336b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:22.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa09d8e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc441540 0x55c9bc44fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa09d0e4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441ea0 0x55c9bc450040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa09d8e5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc441540 0x55c9bc44fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47110/0 (socket says 172.21.15.33:47110) 2024-05-08T21:23:22.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa09d8e5700 1 -- 172.21.15.33:0/3560984188 learned_addr learned my addr 172.21.15.33:0/3560984188 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa09e0e6700 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc442860 0x55c9bc451000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa09d8e5700 1 -- 172.21.15.33:0/3560984188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441ea0 msgr2=0x55c9bc450040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa09d8e5700 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441ea0 0x55c9bc450040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa09d8e5700 1 -- 172.21.15.33:0/3560984188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc442860 msgr2=0x55c9bc451000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa09d8e5700 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc442860 0x55c9bc451000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa09d8e5700 1 -- 172.21.15.33:0/3560984188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa098108af0 con 0x55c9bc441540 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.085+0000 7fa09e0e6700 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc442860 0x55c9bc451000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.086+0000 7fa09d8e5700 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc441540 0x55c9bc44fb00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa088099c30 tx=0x7fa08809dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15724b1a2f789cd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.086+0000 7fa0817fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.086+0000 7fa0817fa700 1 -- 172.21.15.33:0/3560984188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0880b1070 con 0x55c9bc441540 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.086+0000 7fa0817fa700 1 -- 172.21.15.33:0/3560984188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa088098070 con 0x55c9bc441540 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.086+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3560984188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa088099f30 con 0x55c9bc441540 2024-05-08T21:23:22.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.086+0000 7fa0817fa700 1 -- 172.21.15.33:0/3560984188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa088096270 con 0x55c9bc441540 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.086+0000 7fa0abd03b80 10 client.16743 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.086+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3560984188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c9bc1beb60 con 0x55c9bc441540 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.086+0000 7fa0817fa700 1 -- 172.21.15.33:0/3560984188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa088099f30 con 0x55c9bc441540 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa0817fa700 1 -- 172.21.15.33:0/3560984188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fa08809a270 con 0x55c9bc441540 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa0abd03b80 10 client.16743 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa0abd03b80 10 client.16743 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa0abd03b80 20 client.16743 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55c9bc4ad8b0 0x55c9bc4afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa0abd03b80 4 client.16743 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3560984188 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c9bc389ec0 con 0x55c9bc4ad8b0 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa09d0e4700 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55c9bc4ad8b0 0x55c9bc4afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa09d0e4700 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55c9bc4ad8b0 0x55c9bc4afca0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c838873ecd8f716e server_cookie=e1d281067ee35927 in_seq=0 out_seq=0 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa0817fa700 10 client.16743 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa0817fa700 1 -- 172.21.15.33:0/3560984188 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c9bc389ec0 con 0x55c9bc4ad8b0 2024-05-08T21:23:22.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa0817fa700 10 client.16743 handle_command_reply: tid=0 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.087+0000 7fa0abd03b80 1 client.16743 shutdown 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.088+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3560984188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc441540 msgr2=0x55c9bc44fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.088+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc441540 0x55c9bc44fb00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa088099c30 tx=0x7fa08809dea0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.088+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3560984188 shutdown_connections 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.088+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9bc442860 0x55c9bc451000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.088+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9bc441540 0x55c9bc44fb00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.088+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55c9bc4ad8b0 0x55c9bc4afca0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.088+0000 7fa0abd03b80 1 --2- 172.21.15.33:0/3560984188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9bc441ea0 0x55c9bc450040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.088+0000 7fa0817fa700 0 client.16743 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.088+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3560984188 >> 172.21.15.33:0/3560984188 conn(0x55c9bc39f210 msgr2=0x55c9bc452ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.088+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3560984188 shutdown_connections 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.088+0000 7fa0abd03b80 1 -- 172.21.15.33:0/3560984188 wait complete. 2024-05-08T21:23:22.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.088+0000 7fa0abd03b80 20 client.16743 trim_cache size 0 max 16384 2024-05-08T21:23:22.098 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:23:22.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.089+0000 7fa09fb49700 1 -- 172.21.15.33:0/3256936887 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa0840a1120 msgr2=0x7fa0840a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.089+0000 7fa09fb49700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa0840a1120 0x7fa0840a35e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa094006010 tx=0x7fa09401a040 comp rx=0 tx=0).stop 2024-05-08T21:23:22.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.089+0000 7fa09fb49700 1 -- 172.21.15.33:0/3256936887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0980ffd60 msgr2=0x7fa0981972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.089+0000 7fa09fb49700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0980ffd60 0x7fa0981972c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa08800e970 tx=0x7fa08800ec80 comp rx=0 tx=0).stop 2024-05-08T21:23:22.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.089+0000 7fa09fb49700 1 -- 172.21.15.33:0/3256936887 shutdown_connections 2024-05-08T21:23:22.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.089+0000 7fa09fb49700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0981018f0 0x7fa09819be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.089+0000 7fa09fb49700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0980ffd60 0x7fa0981972c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.089+0000 7fa09fb49700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa0840a1120 0x7fa0840a35e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.089+0000 7fa09fb49700 1 --2- 172.21.15.33:0/3256936887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa098100f50 0x7fa098199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.089+0000 7fa09fb49700 1 -- 172.21.15.33:0/3256936887 >> 172.21.15.33:0/3256936887 conn(0x7fa0980fb290 msgr2=0x7fa0980fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.089+0000 7fa09fb49700 1 -- 172.21.15.33:0/3256936887 shutdown_connections 2024-05-08T21:23:22.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.089+0000 7fa09fb49700 1 -- 172.21.15.33:0/3256936887 wait complete. 2024-05-08T21:23:22.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:23:22.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.254+0000 7f3e634aa700 1 -- 172.21.15.33:0/3648489966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c1018f0 msgr2=0x7f3e5c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.254+0000 7f3e634aa700 1 --2- 172.21.15.33:0/3648489966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c1018f0 0x7f3e5c1081a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400b0d0 tx=0x7f3e5400b3e0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.254+0000 7f3e634aa700 1 -- 172.21.15.33:0/3648489966 shutdown_connections 2024-05-08T21:23:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.254+0000 7f3e634aa700 1 --2- 172.21.15.33:0/3648489966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c1018f0 0x7f3e5c1081a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.254+0000 7f3e634aa700 1 --2- 172.21.15.33:0/3648489966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.254+0000 7f3e634aa700 1 --2- 172.21.15.33:0/3648489966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c100f50 0x7f3e5c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.254+0000 7f3e634aa700 1 -- 172.21.15.33:0/3648489966 >> 172.21.15.33:0/3648489966 conn(0x7f3e5c0fb290 msgr2=0x7f3e5c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.254+0000 7f3e634aa700 1 -- 172.21.15.33:0/3648489966 shutdown_connections 2024-05-08T21:23:22.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.254+0000 7f3e634aa700 1 -- 172.21.15.33:0/3648489966 wait complete. 2024-05-08T21:23:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.254+0000 7f3e634aa700 1 Processor -- start 2024-05-08T21:23:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.254+0000 7f3e634aa700 1 -- start start 2024-05-08T21:23:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e634aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e634aa700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e5c100f50 0x7f3e5c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e634aa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c1018f0 0x7f3e5c1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e634aa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1a0b50 con 0x7f3e5c0ffd60 2024-05-08T21:23:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e634aa700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1a0c90 con 0x7f3e5c100f50 2024-05-08T21:23:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e634aa700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1a0e20 con 0x7f3e5c1018f0 2024-05-08T21:23:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e61246700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e61246700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47124/0 (socket says 172.21.15.33:47124) 2024-05-08T21:23:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e61246700 1 -- 172.21.15.33:0/673316645 learned_addr learned my addr 172.21.15.33:0/673316645 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e60a45700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e5c100f50 0x7f3e5c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e61a47700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c1018f0 0x7f3e5c1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e61246700 1 -- 172.21.15.33:0/673316645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c1018f0 msgr2=0x7f3e5c1a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e61246700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c1018f0 0x7f3e5c1a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e61246700 1 -- 172.21.15.33:0/673316645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e5c100f50 msgr2=0x7f3e5c199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e61246700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e5c100f50 0x7f3e5c199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e61246700 1 -- 172.21.15.33:0/673316645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e54009d00 con 0x7f3e5c0ffd60 2024-05-08T21:23:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.255+0000 7f3e60a45700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e5c100f50 0x7f3e5c199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.256+0000 7f3e61246700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c199020 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800e970 tx=0x7f3e5800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.256+0000 7f3e527fc700 1 -- 172.21.15.33:0/673316645 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e58009790 con 0x7f3e5c0ffd60 2024-05-08T21:23:22.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.256+0000 7f3e527fc700 1 -- 172.21.15.33:0/673316645 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e5800c7e0 con 0x7f3e5c0ffd60 2024-05-08T21:23:22.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.256+0000 7f3e634aa700 1 -- 172.21.15.33:0/673316645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5c1a1100 con 0x7f3e5c0ffd60 2024-05-08T21:23:22.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.256+0000 7f3e527fc700 1 -- 172.21.15.33:0/673316645 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e58003bd0 con 0x7f3e5c0ffd60 2024-05-08T21:23:22.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.256+0000 7f3e634aa700 1 -- 172.21.15.33:0/673316645 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c1a1610 con 0x7f3e5c0ffd60 2024-05-08T21:23:22.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.258+0000 7f3e527fc700 1 -- 172.21.15.33:0/673316645 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f3e5800c950 con 0x7f3e5c0ffd60 2024-05-08T21:23:22.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.259+0000 7f3e527fc700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3e480a1120 0x7f3e480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.259+0000 7f3e527fc700 1 -- 172.21.15.33:0/673316645 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3e58014070 con 0x7f3e5c0ffd60 2024-05-08T21:23:22.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.260+0000 7f3e60a45700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3e480a1120 0x7f3e480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.260+0000 7f3e60a45700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3e480a1120 0x7f3e480a35e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c011490 tx=0x7f3e4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e6f664b80 1 Processor -- start 2024-05-08T21:23:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e6f664b80 1 -- start start 2024-05-08T21:23:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e6f664b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91a540 0x56266a91a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e6f664b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91aea0 0x56266a91b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e6f664b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91b860 0x56266a91bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e6f664b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266a871150 con 0x56266a91a540 2024-05-08T21:23:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e6f664b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266a86eb10 con 0x56266a91aea0 2024-05-08T21:23:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e6f664b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266a86a400 con 0x56266a91b860 2024-05-08T21:23:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e61a47700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91b860 0x56266a91bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e61246700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91a540 0x56266a91a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e61a47700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91b860 0x56266a91bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42686/0 (socket says 172.21.15.33:42686) 2024-05-08T21:23:22.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e61a47700 1 -- 172.21.15.33:0/3152187605 learned_addr learned my addr 172.21.15.33:0/3152187605 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.261+0000 7f3e60a45700 1 --2- 172.21.15.33:0/3152187605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91aea0 0x56266a91b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.262+0000 7f3e61a47700 1 -- 172.21.15.33:0/3152187605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91aea0 msgr2=0x56266a91b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.262+0000 7f3e61a47700 1 --2- 172.21.15.33:0/3152187605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91aea0 0x56266a91b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.262+0000 7f3e61a47700 1 -- 172.21.15.33:0/3152187605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91a540 msgr2=0x56266a91a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.262+0000 7f3e61a47700 1 --2- 172.21.15.33:0/3152187605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91a540 0x56266a91a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.262+0000 7f3e61a47700 1 -- 172.21.15.33:0/3152187605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56266a6eb3b0 con 0x56266a91b860 2024-05-08T21:23:22.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.262+0000 7f3e61a47700 1 --2- 172.21.15.33:0/3152187605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91b860 0x56266a91bce0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009c80 tx=0x7f3e540092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b02489a99e99b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.262+0000 7f3e477fe700 1 -- 172.21.15.33:0/3152187605 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5400e040 con 0x56266a91b860 2024-05-08T21:23:22.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.262+0000 7f3e477fe700 1 -- 172.21.15.33:0/3152187605 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e54009480 con 0x56266a91b860 2024-05-08T21:23:22.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.262+0000 7f3e477fe700 1 -- 172.21.15.33:0/3152187605 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54003dc0 con 0x56266a91b860 2024-05-08T21:23:22.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.263+0000 7f3e6f664b80 1 -- 172.21.15.33:0/3152187605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91b860 msgr2=0x56266a91bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.263+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/3152187605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91b860 0x56266a91bce0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009c80 tx=0x7f3e540092e0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.263+0000 7f3e6f664b80 1 -- 172.21.15.33:0/3152187605 shutdown_connections 2024-05-08T21:23:22.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.263+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/3152187605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91b860 0x56266a91bce0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.263+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/3152187605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91a540 0x56266a91a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.263+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/3152187605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91aea0 0x56266a91b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.263+0000 7f3e6f664b80 1 -- 172.21.15.33:0/3152187605 >> 172.21.15.33:0/3152187605 conn(0x56266a878210 msgr2=0x56266a914de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.263+0000 7f3e6f664b80 1 -- 172.21.15.33:0/3152187605 shutdown_connections 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.263+0000 7f3e6f664b80 1 -- 172.21.15.33:0/3152187605 wait complete. 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.263+0000 7f3e6f664b80 1 -- start start 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e6f664b80 1 Processor -- start 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e6f664b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91a540 0x56266a923da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e6f664b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 0x56266a9242e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e6f664b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91b860 0x56266a92b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e6f664b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266a806560 con 0x56266a91aea0 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e6f664b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266a806200 con 0x56266a91b860 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e6f664b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266a6ed6a0 con 0x56266a91a540 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e6f664b80 10 client.0 _ll_register_callbacks cb 0x56266a80fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e60a45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 0x56266a9242e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e60a45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 0x56266a9242e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47152/0 (socket says 172.21.15.33:47152) 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e60a45700 1 -- 172.21.15.33:0/680195308 learned_addr learned my addr 172.21.15.33:0/680195308 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e61246700 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91a540 0x56266a923da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.264+0000 7f3e61a47700 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91b860 0x56266a92b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e61a47700 1 -- 172.21.15.33:0/680195308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91a540 msgr2=0x56266a923da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e61a47700 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91a540 0x56266a923da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e61a47700 1 -- 172.21.15.33:0/680195308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 msgr2=0x56266a9242e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e61a47700 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 0x56266a9242e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e61a47700 1 -- 172.21.15.33:0/680195308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c108af0 con 0x56266a91b860 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e60a45700 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 0x56266a9242e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e61a47700 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91b860 0x56266a92b320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3e540079c0 tx=0x7f3e54004d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d99f9648120bb887 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e44ff9700 1 -- 172.21.15.33:0/680195308 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5400e070 con 0x56266a91b860 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e44ff9700 1 -- 172.21.15.33:0/680195308 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3e54025450 con 0x56266a91b860 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e6f664b80 1 -- 172.21.15.33:0/680195308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56266a7c3270 con 0x56266a91b860 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e44ff9700 1 -- 172.21.15.33:0/680195308 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e54025610 con 0x56266a91b860 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e6f664b80 10 client.26624 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.265+0000 7f3e6f664b80 1 -- 172.21.15.33:0/680195308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56266a697b60 con 0x56266a91b860 2024-05-08T21:23:22.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.266+0000 7f3e44ff9700 1 -- 172.21.15.33:0/680195308 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e54037080 con 0x56266a91b860 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.266+0000 7f3e44ff9700 1 -- 172.21.15.33:0/680195308 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f3e54019030 con 0x56266a91b860 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.266+0000 7f3e6f664b80 10 client.26624 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.266+0000 7f3e6f664b80 10 client.26624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.266+0000 7f3e6f664b80 20 client.26624 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.266+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x56266a92f740 0x56266a931b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.266+0000 7f3e6f664b80 4 client.26624 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.266+0000 7f3e6f664b80 1 -- 172.21.15.33:0/680195308 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56266a8e9810 con 0x56266a92f740 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.266+0000 7f3e61246700 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x56266a92f740 0x56266a931b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e44ff9700 10 client.26624 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e61246700 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x56266a92f740 0x56266a931b30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ddf2a79d5401c27 server_cookie=9433d2fbdbf45468 in_seq=0 out_seq=0 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e44ff9700 1 -- 172.21.15.33:0/680195308 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56266a8e9810 con 0x56266a92f740 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e44ff9700 10 client.26624 handle_command_reply: tid=0 2024-05-08T21:23:22.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e6f664b80 1 client.26624 shutdown 2024-05-08T21:23:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e6f664b80 1 -- 172.21.15.33:0/680195308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91b860 msgr2=0x56266a92b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91b860 0x56266a92b320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3e540079c0 tx=0x7f3e54004d40 comp rx=0 tx=0).stop 2024-05-08T21:23:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e6f664b80 1 -- 172.21.15.33:0/680195308 shutdown_connections 2024-05-08T21:23:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x56266a92f740 0x56266a931b30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91b860 0x56266a92b320 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91a540 0x56266a923da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/680195308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 0x56266a9242e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e44ff9700 0 client.26624 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:23:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.267+0000 7f3e6f664b80 1 -- 172.21.15.33:0/680195308 >> 172.21.15.33:0/680195308 conn(0x56266a878210 msgr2=0x56266a9145a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.268+0000 7f3e6f664b80 1 -- 172.21.15.33:0/680195308 shutdown_connections 2024-05-08T21:23:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.268+0000 7f3e6f664b80 1 -- 172.21.15.33:0/680195308 wait complete. 2024-05-08T21:23:22.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.268+0000 7f3e6f664b80 20 client.26624 trim_cache size 0 max 16384 2024-05-08T21:23:22.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.283+0000 7f3e6f664b80 1 Processor -- start 2024-05-08T21:23:22.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.283+0000 7f3e6f664b80 1 -- start start 2024-05-08T21:23:22.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e6f664b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91a540 0x56266a91a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e6f664b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 0x56266a92b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e6f664b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91b860 0x56266a9232b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e6f664b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266a90f800 con 0x56266a91aea0 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e6f664b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266a922d50 con 0x56266a91b860 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e6f664b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266a9219f0 con 0x56266a91a540 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e60a45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 0x56266a92b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e60a45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 0x56266a92b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47160/0 (socket says 172.21.15.33:47160) 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e61a47700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91b860 0x56266a9232b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e60a45700 1 -- 172.21.15.33:0/169357557 learned_addr learned my addr 172.21.15.33:0/169357557 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e61246700 1 --2- 172.21.15.33:0/169357557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91a540 0x56266a91a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e61a47700 1 -- 172.21.15.33:0/169357557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91a540 msgr2=0x56266a91a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e61a47700 1 --2- 172.21.15.33:0/169357557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91a540 0x56266a91a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e61a47700 1 -- 172.21.15.33:0/169357557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 msgr2=0x56266a92b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e61a47700 1 --2- 172.21.15.33:0/169357557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 0x56266a92b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e61a47700 1 -- 172.21.15.33:0/169357557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56266a830560 con 0x56266a91b860 2024-05-08T21:23:22.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.284+0000 7f3e61a47700 1 --2- 172.21.15.33:0/169357557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91b860 0x56266a9232b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e54002230 tx=0x7f3e5400b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1f8e7ba708b049 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.285+0000 7f3e47fff700 1 -- 172.21.15.33:0/169357557 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5400e040 con 0x56266a91b860 2024-05-08T21:23:22.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.285+0000 7f3e47fff700 1 -- 172.21.15.33:0/169357557 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56266a7c3270 con 0x56266a91b860 2024-05-08T21:23:22.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.285+0000 7f3e47fff700 1 -- 172.21.15.33:0/169357557 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54003c60 con 0x56266a91b860 2024-05-08T21:23:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.286+0000 7f3e6f664b80 1 -- 172.21.15.33:0/169357557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91b860 msgr2=0x56266a9232b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.286+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/169357557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91b860 0x56266a9232b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e54002230 tx=0x7f3e5400b730 comp rx=0 tx=0).stop 2024-05-08T21:23:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.286+0000 7f3e6f664b80 1 -- 172.21.15.33:0/169357557 shutdown_connections 2024-05-08T21:23:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.286+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/169357557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91b860 0x56266a9232b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.286+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/169357557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91a540 0x56266a91a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.286+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/169357557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91aea0 0x56266a92b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.286+0000 7f3e6f664b80 1 -- 172.21.15.33:0/169357557 >> 172.21.15.33:0/169357557 conn(0x56266a878210 msgr2=0x56266a91bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.286+0000 7f3e6f664b80 1 -- 172.21.15.33:0/169357557 shutdown_connections 2024-05-08T21:23:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.286+0000 7f3e6f664b80 1 -- 172.21.15.33:0/169357557 wait complete. 2024-05-08T21:23:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.286+0000 7f3e6f664b80 1 -- start start 2024-05-08T21:23:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.286+0000 7f3e6f664b80 1 Processor -- start 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e6f664b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91a540 0x56266a926a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e6f664b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91aea0 0x56266a926f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e6f664b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91b860 0x56266a929f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e6f664b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266a92a470 con 0x56266a91b860 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e6f664b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266a92a5e0 con 0x56266a91a540 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e6f664b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266a92a7a0 con 0x56266a91aea0 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e6f664b80 10 client.0 _ll_register_callbacks cb 0x56266a80fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e61a47700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91b860 0x56266a929f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e61246700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91a540 0x56266a926a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e61246700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91a540 0x56266a926a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:43016/0 (socket says 172.21.15.33:43016) 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e61a47700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91b860 0x56266a929f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47172/0 (socket says 172.21.15.33:47172) 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e61246700 1 -- 172.21.15.33:0/2338757604 learned_addr learned my addr 172.21.15.33:0/2338757604 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e60a45700 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91aea0 0x56266a926f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e61246700 1 -- 172.21.15.33:0/2338757604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91aea0 msgr2=0x56266a926f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e61246700 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91aea0 0x56266a926f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e61246700 1 -- 172.21.15.33:0/2338757604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91b860 msgr2=0x56266a929f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e61246700 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91b860 0x56266a929f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e61246700 1 -- 172.21.15.33:0/2338757604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56266a697b60 con 0x56266a91a540 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.287+0000 7f3e61a47700 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91b860 0x56266a929f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e61246700 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91a540 0x56266a926a20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800efd0 tx=0x7f3e58099b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd622381968ceb18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e44ff9700 1 -- 172.21.15.33:0/2338757604 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5809dd60 con 0x56266a91a540 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e6f664b80 1 -- 172.21.15.33:0/2338757604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e54009480 con 0x56266a91a540 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e44ff9700 1 -- 172.21.15.33:0/2338757604 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f3e580a23b0 con 0x56266a91a540 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e44ff9700 1 -- 172.21.15.33:0/2338757604 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e580982d0 con 0x56266a91a540 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e6f664b80 10 client.26630 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e6f664b80 1 -- 172.21.15.33:0/2338757604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56266a6efcd0 con 0x56266a91a540 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e44ff9700 1 -- 172.21.15.33:0/2338757604 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e580b6030 con 0x56266a91a540 2024-05-08T21:23:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e44ff9700 1 -- 172.21.15.33:0/2338757604 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f3e5809a220 con 0x56266a91a540 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e6f664b80 10 client.26630 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e6f664b80 10 client.26630 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e6f664b80 20 client.26630 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x56266a986580 0x56266a9889a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e6f664b80 4 client.26630 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.288+0000 7f3e6f664b80 1 -- 172.21.15.33:0/2338757604 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56266a862ec0 con 0x56266a986580 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.289+0000 7f3e60a45700 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x56266a986580 0x56266a9889a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.289+0000 7f3e44ff9700 10 client.26630 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.289+0000 7f3e60a45700 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x56266a986580 0x56266a9889a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=326457f6ea058963 server_cookie=c6518d060ec031a6 in_seq=0 out_seq=0 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.289+0000 7f3e44ff9700 1 -- 172.21.15.33:0/2338757604 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56266a862ec0 con 0x56266a986580 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.289+0000 7f3e44ff9700 10 client.26630 handle_command_reply: tid=0 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.289+0000 7f3e6f664b80 1 client.26630 shutdown 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e6f664b80 1 -- 172.21.15.33:0/2338757604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91a540 msgr2=0x56266a926a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91a540 0x56266a926a20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800efd0 tx=0x7f3e58099b90 comp rx=0 tx=0).stop 2024-05-08T21:23:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e6f664b80 1 -- 172.21.15.33:0/2338757604 shutdown_connections 2024-05-08T21:23:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x56266a986580 0x56266a9889a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56266a91b860 0x56266a929f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56266a91a540 0x56266a926a20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e44ff9700 0 client.26630 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:23:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e6f664b80 1 --2- 172.21.15.33:0/2338757604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56266a91aea0 0x56266a926f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e6f664b80 1 -- 172.21.15.33:0/2338757604 >> 172.21.15.33:0/2338757604 conn(0x56266a878210 msgr2=0x56266a936490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e6f664b80 1 -- 172.21.15.33:0/2338757604 shutdown_connections 2024-05-08T21:23:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e6f664b80 1 -- 172.21.15.33:0/2338757604 wait complete. 2024-05-08T21:23:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e6f664b80 20 client.26630 trim_cache size 0 max 16384 2024-05-08T21:23:22.298 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:23:22.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e634aa700 1 -- 172.21.15.33:0/673316645 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3e480a1120 msgr2=0x7f3e480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e634aa700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3e480a1120 0x7f3e480a35e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c011490 tx=0x7f3e4c009260 comp rx=0 tx=0).stop 2024-05-08T21:23:22.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e634aa700 1 -- 172.21.15.33:0/673316645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c0ffd60 msgr2=0x7f3e5c199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e634aa700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c199020 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800e970 tx=0x7f3e5800ec80 comp rx=0 tx=0).stop 2024-05-08T21:23:22.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.290+0000 7f3e634aa700 1 -- 172.21.15.33:0/673316645 shutdown_connections 2024-05-08T21:23:22.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.291+0000 7f3e634aa700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e5c1018f0 0x7f3e5c1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.291+0000 7f3e634aa700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c199020 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.291+0000 7f3e634aa700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f3e480a1120 0x7f3e480a35e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.291+0000 7f3e634aa700 1 --2- 172.21.15.33:0/673316645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e5c100f50 0x7f3e5c199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.291+0000 7f3e634aa700 1 -- 172.21.15.33:0/673316645 >> 172.21.15.33:0/673316645 conn(0x7f3e5c0fb290 msgr2=0x7f3e5c0681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.291+0000 7f3e634aa700 1 -- 172.21.15.33:0/673316645 shutdown_connections 2024-05-08T21:23:22.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.291+0000 7f3e634aa700 1 -- 172.21.15.33:0/673316645 wait complete. 2024-05-08T21:23:22.307 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:23:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.457+0000 7fcebd69f700 1 -- 172.21.15.33:0/1168924565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceb81006d0 msgr2=0x7fceb8100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.457+0000 7fcebd69f700 1 --2- 172.21.15.33:0/1168924565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceb81006d0 0x7fceb8100b30 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fceac00aad0 tx=0x7fceac00ade0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.457+0000 7fcebd69f700 1 -- 172.21.15.33:0/1168924565 shutdown_connections 2024-05-08T21:23:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.457+0000 7fcebd69f700 1 --2- 172.21.15.33:0/1168924565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb8101070 0x7fceb8107920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.457+0000 7fcebd69f700 1 --2- 172.21.15.33:0/1168924565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb80ff4e0 0x7fceb80ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.457+0000 7fcebd69f700 1 --2- 172.21.15.33:0/1168924565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceb81006d0 0x7fceb8100b30 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.457+0000 7fcebd69f700 1 -- 172.21.15.33:0/1168924565 >> 172.21.15.33:0/1168924565 conn(0x7fceb80faa50 msgr2=0x7fceb80fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.457+0000 7fcebd69f700 1 -- 172.21.15.33:0/1168924565 shutdown_connections 2024-05-08T21:23:22.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.457+0000 7fcebd69f700 1 -- 172.21.15.33:0/1168924565 wait complete. 2024-05-08T21:23:22.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.457+0000 7fcebd69f700 1 Processor -- start 2024-05-08T21:23:22.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.458+0000 7fcebd69f700 1 -- start start 2024-05-08T21:23:22.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.458+0000 7fcebd69f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb80ff4e0 0x7fceb8196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.458+0000 7fcebd69f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb81006d0 0x7fceb81948e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.458+0000 7fcebd69f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceb8101070 0x7fceb8194e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.458+0000 7fcebd69f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb81955c0 con 0x7fceb8101070 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.458+0000 7fcebd69f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb8195730 con 0x7fceb80ff4e0 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.458+0000 7fcebd69f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb81a0d70 con 0x7fceb81006d0 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.458+0000 7fceb77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceb8101070 0x7fceb8194e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.458+0000 7fceb6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb80ff4e0 0x7fceb8196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.458+0000 7fceb77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceb8101070 0x7fceb8194e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47184/0 (socket says 172.21.15.33:47184) 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.458+0000 7fceb77fe700 1 -- 172.21.15.33:0/2152768264 learned_addr learned my addr 172.21.15.33:0/2152768264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.458+0000 7fceb67fc700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb81006d0 0x7fceb81948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.459+0000 7fceb77fe700 1 -- 172.21.15.33:0/2152768264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb81006d0 msgr2=0x7fceb81948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.459+0000 7fceb77fe700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb81006d0 0x7fceb81948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.459+0000 7fceb77fe700 1 -- 172.21.15.33:0/2152768264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb80ff4e0 msgr2=0x7fceb8196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.459+0000 7fceb77fe700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb80ff4e0 0x7fceb8196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.459+0000 7fceb77fe700 1 -- 172.21.15.33:0/2152768264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac00a700 con 0x7fceb8101070 2024-05-08T21:23:22.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.459+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb80ff4e0 0x7fceb8196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.459+0000 7fceb77fe700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceb8101070 0x7fceb8194e20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fceb0007ee0 tx=0x7fceb000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.459+0000 7fce9ffff700 1 -- 172.21.15.33:0/2152768264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb000dcc0 con 0x7fceb8101070 2024-05-08T21:23:22.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.459+0000 7fcebd69f700 1 -- 172.21.15.33:0/2152768264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb81a0fa0 con 0x7fceb8101070 2024-05-08T21:23:22.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.459+0000 7fce9ffff700 1 -- 172.21.15.33:0/2152768264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fceb000f040 con 0x7fceb8101070 2024-05-08T21:23:22.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.459+0000 7fce9ffff700 1 -- 172.21.15.33:0/2152768264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb00046e0 con 0x7fceb8101070 2024-05-08T21:23:22.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.459+0000 7fcebd69f700 1 -- 172.21.15.33:0/2152768264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb81a1430 con 0x7fceb8101070 2024-05-08T21:23:22.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.462+0000 7fce9ffff700 1 -- 172.21.15.33:0/2152768264 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fceb0003a00 con 0x7fceb8101070 2024-05-08T21:23:22.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.463+0000 7fce9ffff700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcea00a1120 0x7fcea00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.463+0000 7fce9ffff700 1 -- 172.21.15.33:0/2152768264 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fceb00cc4a0 con 0x7fceb8101070 2024-05-08T21:23:22.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.463+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcea00a1120 0x7fcea00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.463+0000 7fcec9859b80 1 Processor -- start 2024-05-08T21:23:22.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fcec9859b80 1 -- start start 2024-05-08T21:23:22.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcea00a1120 0x7fcea00a35e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fceb8100530 tx=0x7fcea8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fcec9859b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a1a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fcec9859b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1aea0 0x563810a1b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fcec9859b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 0x563810a1bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fcec9859b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563810971150 con 0x563810a1b860 2024-05-08T21:23:22.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fcec9859b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56381096eb10 con 0x563810a1aea0 2024-05-08T21:23:22.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fcec9859b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56381096a400 con 0x563810a1a540 2024-05-08T21:23:22.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 0x563810a1bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a1a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a1a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42726/0 (socket says 172.21.15.33:42726) 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb6ffd700 1 -- 172.21.15.33:0/2751330037 learned_addr learned my addr 172.21.15.33:0/2751330037 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb77fe700 1 --2- 172.21.15.33:0/2751330037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 0x563810a1bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47188/0 (socket says 172.21.15.33:47188) 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb67fc700 1 --2- 172.21.15.33:0/2751330037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1aea0 0x563810a1b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb6ffd700 1 -- 172.21.15.33:0/2751330037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1aea0 msgr2=0x563810a1b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/2751330037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1aea0 0x563810a1b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb6ffd700 1 -- 172.21.15.33:0/2751330037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 msgr2=0x563810a1bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/2751330037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 0x563810a1bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb6ffd700 1 -- 172.21.15.33:0/2751330037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638107eb3b0 con 0x563810a1a540 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb67fc700 1 --2- 172.21.15.33:0/2751330037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1aea0 0x563810a1b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.464+0000 7fceb77fe700 1 --2- 172.21.15.33:0/2751330037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 0x563810a1bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.465+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/2751330037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a1a960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcea8004400 tx=0x7fcea8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed798729d5706a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.465+0000 7fce9d7fa700 1 -- 172.21.15.33:0/2751330037 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8003d10 con 0x563810a1a540 2024-05-08T21:23:22.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.465+0000 7fce9d7fa700 1 -- 172.21.15.33:0/2751330037 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcea8003e70 con 0x563810a1a540 2024-05-08T21:23:22.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.465+0000 7fce9d7fa700 1 -- 172.21.15.33:0/2751330037 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea80267c0 con 0x563810a1a540 2024-05-08T21:23:22.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.466+0000 7fcec9859b80 1 -- 172.21.15.33:0/2751330037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 msgr2=0x563810a1a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.466+0000 7fcec9859b80 1 --2- 172.21.15.33:0/2751330037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a1a960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcea8004400 tx=0x7fcea8014000 comp rx=0 tx=0).stop 2024-05-08T21:23:22.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.466+0000 7fcec9859b80 1 -- 172.21.15.33:0/2751330037 shutdown_connections 2024-05-08T21:23:22.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.466+0000 7fcec9859b80 1 --2- 172.21.15.33:0/2751330037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 0x563810a1bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.466+0000 7fcec9859b80 1 --2- 172.21.15.33:0/2751330037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a1a960 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.466+0000 7fcec9859b80 1 --2- 172.21.15.33:0/2751330037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1aea0 0x563810a1b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.466+0000 7fcec9859b80 1 -- 172.21.15.33:0/2751330037 >> 172.21.15.33:0/2751330037 conn(0x563810978210 msgr2=0x563810a14de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.466+0000 7fcec9859b80 1 -- 172.21.15.33:0/2751330037 shutdown_connections 2024-05-08T21:23:22.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.466+0000 7fcec9859b80 1 -- 172.21.15.33:0/2751330037 wait complete. 2024-05-08T21:23:22.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.466+0000 7fcec9859b80 1 -- start start 2024-05-08T21:23:22.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.466+0000 7fcec9859b80 1 Processor -- start 2024-05-08T21:23:22.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fcec9859b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a23e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fcec9859b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1aea0 0x563810a243a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fcec9859b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 0x563810a2b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fcec9859b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563810906560 con 0x563810a1b860 2024-05-08T21:23:22.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fcec9859b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563810906200 con 0x563810a1aea0 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fcec9859b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638107ed6a0 con 0x563810a1a540 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fceb77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 0x563810a2b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fcec9859b80 10 client.0 _ll_register_callbacks cb 0x56381090fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fceb77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 0x563810a2b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47200/0 (socket says 172.21.15.33:47200) 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fceb77fe700 1 -- 172.21.15.33:0/3829007230 learned_addr learned my addr 172.21.15.33:0/3829007230 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fceb67fc700 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1aea0 0x563810a243a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a23e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fceb77fe700 1 -- 172.21.15.33:0/3829007230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 msgr2=0x563810a23e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fceb77fe700 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a23e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fceb77fe700 1 -- 172.21.15.33:0/3829007230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1aea0 msgr2=0x563810a243a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fceb77fe700 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1aea0 0x563810a243a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fceb77fe700 1 -- 172.21.15.33:0/3829007230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb81a1430 con 0x563810a1b860 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a23e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:23:22.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fceb77fe700 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 0x563810a2b3e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fceb00952f0 tx=0x7fceb0009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df45e1934cba6d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fce9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fce9aff5700 1 -- 172.21.15.33:0/3829007230 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb009d170 con 0x563810a1b860 2024-05-08T21:23:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fce9aff5700 1 -- 172.21.15.33:0/3829007230 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fceb0004000 con 0x563810a1b860 2024-05-08T21:23:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.467+0000 7fcec9859b80 1 -- 172.21.15.33:0/3829007230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638108c3270 con 0x563810a1b860 2024-05-08T21:23:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.468+0000 7fce9aff5700 1 -- 172.21.15.33:0/3829007230 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fceb0099570 con 0x563810a1b860 2024-05-08T21:23:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.468+0000 7fcec9859b80 10 client.16773 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.468+0000 7fcec9859b80 1 -- 172.21.15.33:0/3829007230 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fcea8017070 con 0x563810a1b860 2024-05-08T21:23:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.468+0000 7fce9aff5700 1 -- 172.21.15.33:0/3829007230 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5638108c3270 con 0x563810a1b860 2024-05-08T21:23:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.468+0000 7fce9aff5700 1 -- 172.21.15.33:0/3829007230 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fceb00997d0 con 0x563810a1b860 2024-05-08T21:23:22.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.468+0000 7fcec9859b80 10 client.16773 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.468+0000 7fcec9859b80 10 client.16773 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:23:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.468+0000 7fcec9859b80 20 client.16773 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.468+0000 7fcec9859b80 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563810a2fc50 0x563810a32070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.468+0000 7fcec9859b80 4 client.16773 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.468+0000 7fcec9859b80 1 -- 172.21.15.33:0/3829007230 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638109e9810 con 0x563810a2fc50 2024-05-08T21:23:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.468+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563810a2fc50 0x563810a32070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.469+0000 7fce9aff5700 10 client.16773 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:23:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.469+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563810a2fc50 0x563810a32070 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a27ca392bfda6ab server_cookie=e80f625173c5e03b in_seq=0 out_seq=0 2024-05-08T21:23:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.469+0000 7fce9aff5700 1 -- 172.21.15.33:0/3829007230 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5638109e9810 con 0x563810a2fc50 2024-05-08T21:23:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.469+0000 7fce9aff5700 10 client.16773 handle_command_reply: tid=0 2024-05-08T21:23:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.469+0000 7fcec9859b80 1 client.16773 shutdown 2024-05-08T21:23:22.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.470+0000 7fcec9859b80 1 -- 172.21.15.33:0/3829007230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 msgr2=0x563810a2b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.470+0000 7fcec9859b80 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 0x563810a2b3e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fceb00952f0 tx=0x7fceb0009410 comp rx=0 tx=0).stop 2024-05-08T21:23:22.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.470+0000 7fcec9859b80 1 -- 172.21.15.33:0/3829007230 shutdown_connections 2024-05-08T21:23:22.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.470+0000 7fcec9859b80 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563810a2fc50 0x563810a32070 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.470+0000 7fcec9859b80 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1b860 0x563810a2b3e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.470+0000 7fcec9859b80 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a23e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.470+0000 7fcec9859b80 1 --2- 172.21.15.33:0/3829007230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1aea0 0x563810a243a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.470+0000 7fcec9859b80 1 -- 172.21.15.33:0/3829007230 >> 172.21.15.33:0/3829007230 conn(0x563810978210 msgr2=0x563810a145a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.470+0000 7fce9aff5700 0 client.16773 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:23:22.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.470+0000 7fcec9859b80 1 -- 172.21.15.33:0/3829007230 shutdown_connections 2024-05-08T21:23:22.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.470+0000 7fcec9859b80 1 -- 172.21.15.33:0/3829007230 wait complete. 2024-05-08T21:23:22.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.470+0000 7fcec9859b80 20 client.16773 trim_cache size 0 max 16384 2024-05-08T21:23:22.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.485+0000 7fcec9859b80 1 Processor -- start 2024-05-08T21:23:22.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.485+0000 7fcec9859b80 1 -- start start 2024-05-08T21:23:22.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.485+0000 7fcec9859b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1a540 0x563810a2ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.485+0000 7fcec9859b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 0x563810a2b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.485+0000 7fcec9859b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1b860 0x563810a21df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.485+0000 7fcec9859b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563810a16830 con 0x563810a1aea0 2024-05-08T21:23:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.485+0000 7fcec9859b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563810a2d140 con 0x563810a1a540 2024-05-08T21:23:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.485+0000 7fcec9859b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563810a37260 con 0x563810a1b860 2024-05-08T21:23:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.485+0000 7fceb67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 0x563810a2b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.485+0000 7fceb6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1a540 0x563810a2ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.485+0000 7fceb67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 0x563810a2b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47212/0 (socket says 172.21.15.33:47212) 2024-05-08T21:23:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.486+0000 7fceb67fc700 1 -- 172.21.15.33:0/1479273526 learned_addr learned my addr 172.21.15.33:0/1479273526 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.486+0000 7fceb77fe700 1 --2- 172.21.15.33:0/1479273526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1b860 0x563810a21df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.486+0000 7fceb6ffd700 1 -- 172.21.15.33:0/1479273526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1b860 msgr2=0x563810a21df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.486+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/1479273526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1b860 0x563810a21df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.486+0000 7fceb6ffd700 1 -- 172.21.15.33:0/1479273526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 msgr2=0x563810a2b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.486+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/1479273526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 0x563810a2b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.486+0000 7fceb6ffd700 1 -- 172.21.15.33:0/1479273526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563810797b60 con 0x563810a1a540 2024-05-08T21:23:22.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.486+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/1479273526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1a540 0x563810a2ba90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x563810a32920 tx=0x7fcea8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b97455bfa384d4ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.486+0000 7fce9dffb700 1 -- 172.21.15.33:0/1479273526 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea802a040 con 0x563810a1a540 2024-05-08T21:23:22.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.486+0000 7fce9dffb700 1 -- 172.21.15.33:0/1479273526 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcea80056e0 con 0x563810a1a540 2024-05-08T21:23:22.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.486+0000 7fce9dffb700 1 -- 172.21.15.33:0/1479273526 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8004e70 con 0x563810a1a540 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.487+0000 7fcec9859b80 1 -- 172.21.15.33:0/1479273526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1a540 msgr2=0x563810a2ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.487+0000 7fcec9859b80 1 --2- 172.21.15.33:0/1479273526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1a540 0x563810a2ba90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x563810a32920 tx=0x7fcea8017000 comp rx=0 tx=0).stop 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.487+0000 7fcec9859b80 1 -- 172.21.15.33:0/1479273526 shutdown_connections 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.487+0000 7fcec9859b80 1 --2- 172.21.15.33:0/1479273526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1b860 0x563810a21df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.487+0000 7fcec9859b80 1 --2- 172.21.15.33:0/1479273526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1a540 0x563810a2ba90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.487+0000 7fcec9859b80 1 --2- 172.21.15.33:0/1479273526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 0x563810a2b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.487+0000 7fcec9859b80 1 -- 172.21.15.33:0/1479273526 >> 172.21.15.33:0/1479273526 conn(0x563810978210 msgr2=0x563810a1bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.487+0000 7fcec9859b80 1 -- 172.21.15.33:0/1479273526 shutdown_connections 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.487+0000 7fcec9859b80 1 -- 172.21.15.33:0/1479273526 wait complete. 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fcec9859b80 1 -- start start 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fcec9859b80 1 Processor -- start 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fcec9859b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a26f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fcec9859b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 0x563810a274b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fcec9859b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1b860 0x563810a2a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fcec9859b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563810a2aa10 con 0x563810a1aea0 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fcec9859b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563810a2ab80 con 0x563810a1b860 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fcec9859b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563810a2ad40 con 0x563810a1a540 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fcec9859b80 10 client.0 _ll_register_callbacks cb 0x56381090fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fceb67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 0x563810a274b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fceb6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a26f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fceb67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 0x563810a274b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47224/0 (socket says 172.21.15.33:47224) 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fceb77fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1b860 0x563810a2a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fceb6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a26f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42760/0 (socket says 172.21.15.33:42760) 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fceb67fc700 1 -- 172.21.15.33:0/1394443496 learned_addr learned my addr 172.21.15.33:0/1394443496 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.488+0000 7fceb6ffd700 1 -- 172.21.15.33:0/1394443496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1b860 msgr2=0x563810a2a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1b860 0x563810a2a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fceb6ffd700 1 -- 172.21.15.33:0/1394443496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 msgr2=0x563810a274b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 0x563810a274b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fceb6ffd700 1 -- 172.21.15.33:0/1394443496 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638107eb3b0 con 0x563810a1a540 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fceb67fc700 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 0x563810a274b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fceb77fe700 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1b860 0x563810a2a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fceb6ffd700 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a26f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcea8005850 tx=0x7fcea8005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66aafeb12dc3e3b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fce9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fce9aff5700 1 -- 172.21.15.33:0/1394443496 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea802a030 con 0x563810a1a540 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fcec9859b80 1 -- 172.21.15.33:0/1394443496 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcea8003e70 con 0x563810a1a540 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fce9aff5700 1 -- 172.21.15.33:0/1394443496 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcea8004bf0 con 0x563810a1a540 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fce9aff5700 1 -- 172.21.15.33:0/1394443496 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcea8004db0 con 0x563810a1a540 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fcec9859b80 10 client.26665 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fcec9859b80 1 -- 172.21.15.33:0/1394443496 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x563810930560 con 0x563810a1a540 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fce9aff5700 1 -- 172.21.15.33:0/1394443496 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcea8003e70 con 0x563810a1a540 2024-05-08T21:23:22.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.489+0000 7fce9aff5700 1 -- 172.21.15.33:0/1394443496 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fcea8024020 con 0x563810a1a540 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.490+0000 7fcec9859b80 10 client.26665 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.490+0000 7fcec9859b80 10 client.26665 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.490+0000 7fcec9859b80 20 client.26665 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.490+0000 7fcec9859b80 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563810a86ae0 0x563810a88f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.490+0000 7fcec9859b80 4 client.26665 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.490+0000 7fcec9859b80 1 -- 172.21.15.33:0/1394443496 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563810962ec0 con 0x563810a86ae0 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.490+0000 7fceb67fc700 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563810a86ae0 0x563810a88f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.490+0000 7fce9aff5700 10 client.26665 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.490+0000 7fceb67fc700 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563810a86ae0 0x563810a88f00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc9e9a832eedc9c6 server_cookie=c1656ffc4fa3a9b5 in_seq=0 out_seq=0 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.490+0000 7fce9aff5700 1 -- 172.21.15.33:0/1394443496 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563810962ec0 con 0x563810a86ae0 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.490+0000 7fce9aff5700 10 client.26665 handle_command_reply: tid=0 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.490+0000 7fcec9859b80 1 client.26665 shutdown 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcec9859b80 1 -- 172.21.15.33:0/1394443496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 msgr2=0x563810a26f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcec9859b80 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a26f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcea8005850 tx=0x7fcea8005d50 comp rx=0 tx=0).stop 2024-05-08T21:23:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcec9859b80 1 -- 172.21.15.33:0/1394443496 shutdown_connections 2024-05-08T21:23:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcec9859b80 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x563810a86ae0 0x563810a88f00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcec9859b80 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563810a1b860 0x563810a2a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fce9aff5700 0 client.26665 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:23:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcec9859b80 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563810a1a540 0x563810a26f70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcec9859b80 1 --2- 172.21.15.33:0/1394443496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563810a1aea0 0x563810a274b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcec9859b80 1 -- 172.21.15.33:0/1394443496 >> 172.21.15.33:0/1394443496 conn(0x563810978210 msgr2=0x563810a1e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcec9859b80 1 -- 172.21.15.33:0/1394443496 shutdown_connections 2024-05-08T21:23:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcec9859b80 1 -- 172.21.15.33:0/1394443496 wait complete. 2024-05-08T21:23:22.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcec9859b80 20 client.26665 trim_cache size 0 max 16384 2024-05-08T21:23:22.499 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:23:22.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcebd69f700 1 -- 172.21.15.33:0/2152768264 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcea00a1120 msgr2=0x7fcea00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcebd69f700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcea00a1120 0x7fcea00a35e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fceb8100530 tx=0x7fcea8009260 comp rx=0 tx=0).stop 2024-05-08T21:23:22.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcebd69f700 1 -- 172.21.15.33:0/2152768264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceb8101070 msgr2=0x7fceb8194e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcebd69f700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceb8101070 0x7fceb8194e20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fceb0007ee0 tx=0x7fceb000d3b0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcebd69f700 1 -- 172.21.15.33:0/2152768264 shutdown_connections 2024-05-08T21:23:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcebd69f700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fceb8101070 0x7fceb8194e20 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcebd69f700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcea00a1120 0x7fcea00a35e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcebd69f700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fceb80ff4e0 0x7fceb8196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcebd69f700 1 --2- 172.21.15.33:0/2152768264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceb81006d0 0x7fceb81948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.491+0000 7fcebd69f700 1 -- 172.21.15.33:0/2152768264 >> 172.21.15.33:0/2152768264 conn(0x7fceb80faa50 msgr2=0x7fceb8103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.492+0000 7fcebd69f700 1 -- 172.21.15.33:0/2152768264 shutdown_connections 2024-05-08T21:23:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.492+0000 7fcebd69f700 1 -- 172.21.15.33:0/2152768264 wait complete. 2024-05-08T21:23:22.509 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:23:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.658+0000 7f55fa81b700 1 -- 172.21.15.33:0/1268532351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4101940 msgr2=0x7f55f4103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.658+0000 7f55fa81b700 1 --2- 172.21.15.33:0/1268532351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4101940 0x7f55f4103d30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009990 tx=0x7f55e4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.658+0000 7f55fa81b700 1 -- 172.21.15.33:0/1268532351 shutdown_connections 2024-05-08T21:23:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.658+0000 7f55fa81b700 1 --2- 172.21.15.33:0/1268532351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f4106ba0 0x7f55f4109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.658+0000 7f55fa81b700 1 --2- 172.21.15.33:0/1268532351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4101940 0x7f55f4103d30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.658+0000 7f55fa81b700 1 --2- 172.21.15.33:0/1268532351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4104270 0x7f55f4106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.658+0000 7f55fa81b700 1 -- 172.21.15.33:0/1268532351 >> 172.21.15.33:0/1268532351 conn(0x7f55f40fb330 msgr2=0x7f55f40fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.658+0000 7f55fa81b700 1 -- 172.21.15.33:0/1268532351 shutdown_connections 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.658+0000 7f55fa81b700 1 -- 172.21.15.33:0/1268532351 wait complete. 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.659+0000 7f55fa81b700 1 Processor -- start 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.659+0000 7f55fa81b700 1 -- start start 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.659+0000 7f55fa81b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4101940 0x7f55f4196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.659+0000 7f55fa81b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4104270 0x7f55f4197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.659+0000 7f55fa81b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f4106ba0 0x7f55f419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.659+0000 7f55fa81b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f419e830 con 0x7f55f4104270 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.659+0000 7f55fa81b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f419e9a0 con 0x7f55f4101940 2024-05-08T21:23:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.659+0000 7f55fa81b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f419eb60 con 0x7f55f4106ba0 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4101940 0x7f55f4196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4101940 0x7f55f4196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:43110/0 (socket says 172.21.15.33:43110) 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4104270 0x7f55f4197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f3fff700 1 -- 172.21.15.33:0/983758290 learned_addr learned my addr 172.21.15.33:0/983758290 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f8db8700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f4106ba0 0x7f55f419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f8db8700 1 -- 172.21.15.33:0/983758290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4101940 msgr2=0x7f55f4196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f8db8700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4101940 0x7f55f4196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f8db8700 1 -- 172.21.15.33:0/983758290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4104270 msgr2=0x7f55f4197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f8db8700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4104270 0x7f55f4197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f8db8700 1 -- 172.21.15.33:0/983758290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e40095f0 con 0x7f55f4106ba0 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f37fe700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4104270 0x7f55f4197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f8db8700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f4106ba0 0x7f55f419e260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00efc0 tx=0x7f55ec00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f17fa700 1 -- 172.21.15.33:0/983758290 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec018070 con 0x7f55f4106ba0 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f17fa700 1 -- 172.21.15.33:0/983758290 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55ec00f040 con 0x7f55f4106ba0 2024-05-08T21:23:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55fa81b700 1 -- 172.21.15.33:0/983758290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f40ff730 con 0x7f55f4106ba0 2024-05-08T21:23:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.660+0000 7f55f17fa700 1 -- 172.21.15.33:0/983758290 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00cba0 con 0x7f55f4106ba0 2024-05-08T21:23:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.661+0000 7f55fa81b700 1 -- 172.21.15.33:0/983758290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f40ffc40 con 0x7f55f4106ba0 2024-05-08T21:23:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.663+0000 7f55f17fa700 1 -- 172.21.15.33:0/983758290 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f55ec004000 con 0x7f55f4106ba0 2024-05-08T21:23:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.664+0000 7f55f17fa700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f55dc0a1120 0x7f55dc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.664+0000 7f55f17fa700 1 -- 172.21.15.33:0/983758290 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f55ec0cc760 con 0x7f55f4106ba0 2024-05-08T21:23:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.664+0000 7f55f3fff700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f55dc0a1120 0x7f55dc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.665+0000 7f55f3fff700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f55dc0a1120 0x7f55dc0a35e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f55f41981c0 tx=0x7f55e400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.665+0000 7f56069d5b80 1 Processor -- start 2024-05-08T21:23:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.665+0000 7f56069d5b80 1 -- start start 2024-05-08T21:23:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.665+0000 7f56069d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4f540 0x55dc12d4f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.665+0000 7f56069d5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4fea0 0x55dc12d50320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.665+0000 7f56069d5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d50ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.665+0000 7f56069d5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc12ca6150 con 0x55dc12d50860 2024-05-08T21:23:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.665+0000 7f56069d5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc12ca3b10 con 0x55dc12d4fea0 2024-05-08T21:23:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.665+0000 7f56069d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc12c9f400 con 0x55dc12d4f540 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.665+0000 7f55f8db8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d50ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55f8db8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d50ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47246/0 (socket says 172.21.15.33:47246) 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55f8db8700 1 -- 172.21.15.33:0/629697747 learned_addr learned my addr 172.21.15.33:0/629697747 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55f3fff700 1 --2- 172.21.15.33:0/629697747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4f540 0x55dc12d4f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55f37fe700 1 --2- 172.21.15.33:0/629697747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4fea0 0x55dc12d50320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55f8db8700 1 -- 172.21.15.33:0/629697747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4f540 msgr2=0x55dc12d4f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55f8db8700 1 --2- 172.21.15.33:0/629697747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4f540 0x55dc12d4f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55f8db8700 1 -- 172.21.15.33:0/629697747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4fea0 msgr2=0x55dc12d50320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55f8db8700 1 --2- 172.21.15.33:0/629697747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4fea0 0x55dc12d50320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55f8db8700 1 -- 172.21.15.33:0/629697747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc12b203b0 con 0x55dc12d50860 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55f8db8700 1 --2- 172.21.15.33:0/629697747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d50ce0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0a1780 tx=0x7f55ec095d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bb328608faf9e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55da7fc700 1 -- 172.21.15.33:0/629697747 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0092e0 con 0x55dc12d50860 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55da7fc700 1 -- 172.21.15.33:0/629697747 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55ec099fc0 con 0x55dc12d50860 2024-05-08T21:23:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.666+0000 7f55da7fc700 1 -- 172.21.15.33:0/629697747 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec097c60 con 0x55dc12d50860 2024-05-08T21:23:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.667+0000 7f56069d5b80 1 -- 172.21.15.33:0/629697747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 msgr2=0x55dc12d50ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.667+0000 7f56069d5b80 1 --2- 172.21.15.33:0/629697747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d50ce0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0a1780 tx=0x7f55ec095d30 comp rx=0 tx=0).stop 2024-05-08T21:23:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.667+0000 7f56069d5b80 1 -- 172.21.15.33:0/629697747 shutdown_connections 2024-05-08T21:23:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.667+0000 7f56069d5b80 1 --2- 172.21.15.33:0/629697747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d50ce0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.667+0000 7f56069d5b80 1 --2- 172.21.15.33:0/629697747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4f540 0x55dc12d4f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.667+0000 7f56069d5b80 1 --2- 172.21.15.33:0/629697747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4fea0 0x55dc12d50320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.667+0000 7f56069d5b80 1 -- 172.21.15.33:0/629697747 >> 172.21.15.33:0/629697747 conn(0x55dc12cad210 msgr2=0x55dc12d49de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.667+0000 7f56069d5b80 1 -- 172.21.15.33:0/629697747 shutdown_connections 2024-05-08T21:23:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.667+0000 7f56069d5b80 1 -- 172.21.15.33:0/629697747 wait complete. 2024-05-08T21:23:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.667+0000 7f56069d5b80 1 -- start start 2024-05-08T21:23:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f56069d5b80 1 Processor -- start 2024-05-08T21:23:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f56069d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4f540 0x55dc12d58e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f56069d5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4fea0 0x55dc12d59340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f56069d5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d60380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f56069d5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc12c3b560 con 0x55dc12d50860 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f56069d5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc12c3b200 con 0x55dc12d4fea0 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f56069d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc12b226a0 con 0x55dc12d4f540 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f56069d5b80 10 client.0 _ll_register_callbacks cb 0x55dc12c44b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4f540 0x55dc12d58e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f55f37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4fea0 0x55dc12d59340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f55f8db8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d60380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4f540 0x55dc12d58e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42808/0 (socket says 172.21.15.33:42808) 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f55f37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4fea0 0x55dc12d59340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:43114/0 (socket says 172.21.15.33:43114) 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.668+0000 7f55f3fff700 1 -- 172.21.15.33:0/3116789685 learned_addr learned my addr 172.21.15.33:0/3116789685 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f55f37fe700 1 -- 172.21.15.33:0/3116789685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4f540 msgr2=0x55dc12d58e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f55f37fe700 1 --2- 172.21.15.33:0/3116789685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4f540 0x55dc12d58e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f55f37fe700 1 -- 172.21.15.33:0/3116789685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 msgr2=0x55dc12d60380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f55f37fe700 1 --2- 172.21.15.33:0/3116789685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d60380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f55f37fe700 1 -- 172.21.15.33:0/3116789685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f40ffc40 con 0x55dc12d4fea0 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f55f37fe700 1 --2- 172.21.15.33:0/3116789685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4fea0 0x55dc12d59340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009f90 tx=0x7f55e800b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bb5048463a4e2f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f55d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f55d37fe700 1 -- 172.21.15.33:0/3116789685 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8015070 con 0x55dc12d4fea0 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f55d37fe700 1 -- 172.21.15.33:0/3116789685 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f55e8004250 con 0x55dc12d4fea0 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f56069d5b80 1 -- 172.21.15.33:0/3116789685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc12bf8270 con 0x55dc12d4fea0 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f55d37fe700 1 -- 172.21.15.33:0/3116789685 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55e800be70 con 0x55dc12d4fea0 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f56069d5b80 10 client.26660 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f56069d5b80 1 -- 172.21.15.33:0/3116789685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55dc12accb60 con 0x55dc12d4fea0 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.669+0000 7f55d37fe700 1 -- 172.21.15.33:0/3116789685 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dc12bf8270 con 0x55dc12d4fea0 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.670+0000 7f55d37fe700 1 -- 172.21.15.33:0/3116789685 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f55e801b4b0 con 0x55dc12d4fea0 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.670+0000 7f56069d5b80 10 client.26660 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.670+0000 7f56069d5b80 10 client.26660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.670+0000 7f56069d5b80 20 client.26660 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.670+0000 7f56069d5b80 1 --2- 172.21.15.33:0/3116789685 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc12d64d10 0x55dc12d67130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.670+0000 7f56069d5b80 4 client.26660 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.670+0000 7f56069d5b80 1 -- 172.21.15.33:0/3116789685 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc12d1e810 con 0x55dc12d64d10 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.670+0000 7f55f3fff700 1 --2- 172.21.15.33:0/3116789685 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc12d64d10 0x55dc12d67130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.670+0000 7f55d37fe700 10 client.26660 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.670+0000 7f55f3fff700 1 --2- 172.21.15.33:0/3116789685 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc12d64d10 0x55dc12d67130 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d3dbeb8cc728e98 server_cookie=906b462123e78838 in_seq=0 out_seq=0 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f55d37fe700 1 -- 172.21.15.33:0/3116789685 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc12d1e810 con 0x55dc12d64d10 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f55d37fe700 10 client.26660 handle_command_reply: tid=0 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f56069d5b80 1 client.26660 shutdown 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f56069d5b80 1 -- 172.21.15.33:0/3116789685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4fea0 msgr2=0x55dc12d59340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f56069d5b80 1 --2- 172.21.15.33:0/3116789685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4fea0 0x55dc12d59340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009f90 tx=0x7f55e800b5a0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f56069d5b80 1 -- 172.21.15.33:0/3116789685 shutdown_connections 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f56069d5b80 1 --2- 172.21.15.33:0/3116789685 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc12d64d10 0x55dc12d67130 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f56069d5b80 1 --2- 172.21.15.33:0/3116789685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d60380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f56069d5b80 1 --2- 172.21.15.33:0/3116789685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4f540 0x55dc12d58e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f55d37fe700 0 client.26660 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f56069d5b80 1 --2- 172.21.15.33:0/3116789685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4fea0 0x55dc12d59340 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f56069d5b80 1 -- 172.21.15.33:0/3116789685 >> 172.21.15.33:0/3116789685 conn(0x55dc12cad210 msgr2=0x55dc12d495a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f56069d5b80 1 -- 172.21.15.33:0/3116789685 shutdown_connections 2024-05-08T21:23:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f56069d5b80 1 -- 172.21.15.33:0/3116789685 wait complete. 2024-05-08T21:23:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.671+0000 7f56069d5b80 20 client.26660 trim_cache size 0 max 16384 2024-05-08T21:23:22.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.687+0000 7f56069d5b80 1 Processor -- start 2024-05-08T21:23:22.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.687+0000 7f56069d5b80 1 -- start start 2024-05-08T21:23:22.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.687+0000 7f56069d5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d4f540 0x55dc12d58380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.687+0000 7f56069d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4fea0 0x55dc12d602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.687+0000 7f56069d5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d50860 0x55dc12d56e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.687+0000 7f56069d5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc12d6c340 con 0x55dc12d4f540 2024-05-08T21:23:22.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.687+0000 7f56069d5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc12d4b830 con 0x55dc12d50860 2024-05-08T21:23:22.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.687+0000 7f56069d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc12d59a80 con 0x55dc12d4fea0 2024-05-08T21:23:22.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.687+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d4f540 0x55dc12d58380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.687+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d4f540 0x55dc12d58380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47272/0 (socket says 172.21.15.33:47272) 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.687+0000 7f55f3fff700 1 -- 172.21.15.33:0/270926189 learned_addr learned my addr 172.21.15.33:0/270926189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.687+0000 7f55f37fe700 1 --2- 172.21.15.33:0/270926189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4fea0 0x55dc12d602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.688+0000 7f55f8db8700 1 --2- 172.21.15.33:0/270926189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d50860 0x55dc12d56e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.688+0000 7f55f37fe700 1 -- 172.21.15.33:0/270926189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d50860 msgr2=0x55dc12d56e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.688+0000 7f55f37fe700 1 --2- 172.21.15.33:0/270926189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d50860 0x55dc12d56e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.688+0000 7f55f37fe700 1 -- 172.21.15.33:0/270926189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d4f540 msgr2=0x55dc12d58380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.688+0000 7f55f37fe700 1 --2- 172.21.15.33:0/270926189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d4f540 0x55dc12d58380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.688+0000 7f55f37fe700 1 -- 172.21.15.33:0/270926189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc12c65560 con 0x55dc12d4fea0 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.688+0000 7f55f8db8700 1 --2- 172.21.15.33:0/270926189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d50860 0x55dc12d56e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.688+0000 7f55f3fff700 1 --2- 172.21.15.33:0/270926189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d4f540 0x55dc12d58380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.688+0000 7f55f37fe700 1 --2- 172.21.15.33:0/270926189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4fea0 0x55dc12d602a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55e8012010 tx=0x7f55e8005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5581377cd6c05f70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.688+0000 7f55daffd700 1 -- 172.21.15.33:0/270926189 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8015040 con 0x55dc12d4fea0 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.688+0000 7f55daffd700 1 -- 172.21.15.33:0/270926189 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55e8016040 con 0x55dc12d4fea0 2024-05-08T21:23:22.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.688+0000 7f55daffd700 1 -- 172.21.15.33:0/270926189 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8005800 con 0x55dc12d4fea0 2024-05-08T21:23:22.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.689+0000 7f56069d5b80 1 -- 172.21.15.33:0/270926189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4fea0 msgr2=0x55dc12d602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.689+0000 7f56069d5b80 1 --2- 172.21.15.33:0/270926189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4fea0 0x55dc12d602a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55e8012010 tx=0x7f55e8005e40 comp rx=0 tx=0).stop 2024-05-08T21:23:22.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.689+0000 7f56069d5b80 1 -- 172.21.15.33:0/270926189 shutdown_connections 2024-05-08T21:23:22.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.689+0000 7f56069d5b80 1 --2- 172.21.15.33:0/270926189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d50860 0x55dc12d56e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.689+0000 7f56069d5b80 1 --2- 172.21.15.33:0/270926189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d4f540 0x55dc12d58380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.689+0000 7f56069d5b80 1 --2- 172.21.15.33:0/270926189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4fea0 0x55dc12d602a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.689+0000 7f56069d5b80 1 -- 172.21.15.33:0/270926189 >> 172.21.15.33:0/270926189 conn(0x55dc12cad210 msgr2=0x55dc12d46210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.689+0000 7f56069d5b80 1 -- 172.21.15.33:0/270926189 shutdown_connections 2024-05-08T21:23:22.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.689+0000 7f56069d5b80 1 -- 172.21.15.33:0/270926189 wait complete. 2024-05-08T21:23:22.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.689+0000 7f56069d5b80 1 -- start start 2024-05-08T21:23:22.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f56069d5b80 1 Processor -- start 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f56069d5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4f540 0x55dc12d5bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f56069d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4fea0 0x55dc12d5c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f56069d5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d5f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f56069d5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc12d5f630 con 0x55dc12d50860 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f56069d5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc12d5f7a0 con 0x55dc12d4f540 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f56069d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc12d5f960 con 0x55dc12d4fea0 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f56069d5b80 10 client.0 _ll_register_callbacks cb 0x55dc12c44b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f55f8db8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d5f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4f540 0x55dc12d5bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f55f37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4fea0 0x55dc12d5c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f55f8db8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d5f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47286/0 (socket says 172.21.15.33:47286) 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4f540 0x55dc12d5bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:43138/0 (socket says 172.21.15.33:43138) 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.690+0000 7f55f8db8700 1 -- 172.21.15.33:0/709019952 learned_addr learned my addr 172.21.15.33:0/709019952 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:22.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f55f8db8700 1 -- 172.21.15.33:0/709019952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4fea0 msgr2=0x55dc12d5c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f55f8db8700 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4fea0 0x55dc12d5c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f55f8db8700 1 -- 172.21.15.33:0/709019952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4f540 msgr2=0x55dc12d5bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f55f8db8700 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4f540 0x55dc12d5bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f55f8db8700 1 -- 172.21.15.33:0/709019952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f4109c00 con 0x55dc12d50860 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f55f37fe700 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4fea0 0x55dc12d5c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f55f3fff700 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4f540 0x55dc12d5bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f55f8db8700 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d5f0f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f55ec096330 tx=0x7f55ec004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b08911072db4f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f55d37fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f55d37fe700 1 -- 172.21.15.33:0/709019952 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0a1d20 con 0x55dc12d50860 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f55d37fe700 1 -- 172.21.15.33:0/709019952 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f55ec0092e0 con 0x55dc12d50860 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f56069d5b80 1 -- 172.21.15.33:0/709019952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55ec099fc0 con 0x55dc12d50860 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f55d37fe700 1 -- 172.21.15.33:0/709019952 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55ec09d6f0 con 0x55dc12d50860 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f56069d5b80 10 client.16797 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.691+0000 7f56069d5b80 1 -- 172.21.15.33:0/709019952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55dc12b203b0 con 0x55dc12d50860 2024-05-08T21:23:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.692+0000 7f55d37fe700 1 -- 172.21.15.33:0/709019952 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55ec099fc0 con 0x55dc12d50860 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.692+0000 7f55d37fe700 1 -- 172.21.15.33:0/709019952 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f55ec098db0 con 0x55dc12d50860 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.692+0000 7f56069d5b80 10 client.16797 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.692+0000 7f56069d5b80 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.692+0000 7f56069d5b80 20 client.16797 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.692+0000 7f56069d5b80 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc12dbb8c0 0x55dc12dbdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.692+0000 7f56069d5b80 4 client.16797 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.692+0000 7f56069d5b80 1 -- 172.21.15.33:0/709019952 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc12c97ec0 con 0x55dc12dbb8c0 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.692+0000 7f55f3fff700 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc12dbb8c0 0x55dc12dbdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.692+0000 7f55d37fe700 10 client.16797 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.692+0000 7f55f3fff700 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc12dbb8c0 0x55dc12dbdce0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6021111ab5e14b server_cookie=3f462bee32c482a5 in_seq=0 out_seq=0 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f55d37fe700 1 -- 172.21.15.33:0/709019952 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc12c97ec0 con 0x55dc12dbb8c0 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f55d37fe700 10 client.16797 handle_command_reply: tid=0 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f56069d5b80 1 client.16797 shutdown 2024-05-08T21:23:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f56069d5b80 1 -- 172.21.15.33:0/709019952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 msgr2=0x55dc12d5f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f56069d5b80 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d5f0f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f55ec096330 tx=0x7f55ec004b80 comp rx=0 tx=0).stop 2024-05-08T21:23:22.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f56069d5b80 1 -- 172.21.15.33:0/709019952 shutdown_connections 2024-05-08T21:23:22.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f56069d5b80 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55dc12dbb8c0 0x55dc12dbdce0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f56069d5b80 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc12d50860 0x55dc12d5f0f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f55d37fe700 0 client.16797 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:22.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f56069d5b80 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dc12d4f540 0x55dc12d5bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f56069d5b80 1 --2- 172.21.15.33:0/709019952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc12d4fea0 0x55dc12d5c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f56069d5b80 1 -- 172.21.15.33:0/709019952 >> 172.21.15.33:0/709019952 conn(0x55dc12cad210 msgr2=0x55dc12d537d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f56069d5b80 1 -- 172.21.15.33:0/709019952 shutdown_connections 2024-05-08T21:23:22.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f56069d5b80 1 -- 172.21.15.33:0/709019952 wait complete. 2024-05-08T21:23:22.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.693+0000 7f56069d5b80 20 client.16797 trim_cache size 0 max 16384 2024-05-08T21:23:22.703 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:23:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.694+0000 7f55fa81b700 1 -- 172.21.15.33:0/983758290 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f55dc0a1120 msgr2=0x7f55dc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.694+0000 7f55fa81b700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f55dc0a1120 0x7f55dc0a35e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f55f41981c0 tx=0x7f55e400b610 comp rx=0 tx=0).stop 2024-05-08T21:23:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.694+0000 7f55fa81b700 1 -- 172.21.15.33:0/983758290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f4106ba0 msgr2=0x7f55f419e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.694+0000 7f55fa81b700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f4106ba0 0x7f55f419e260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00efc0 tx=0x7f55ec00c3a0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.694+0000 7f55fa81b700 1 -- 172.21.15.33:0/983758290 shutdown_connections 2024-05-08T21:23:22.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.694+0000 7f55fa81b700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f4106ba0 0x7f55f419e260 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.694+0000 7f55fa81b700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f55dc0a1120 0x7f55dc0a35e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.694+0000 7f55fa81b700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4101940 0x7f55f4196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.694+0000 7f55fa81b700 1 --2- 172.21.15.33:0/983758290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55f4104270 0x7f55f4197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:22.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.694+0000 7f55fa81b700 1 -- 172.21.15.33:0/983758290 >> 172.21.15.33:0/983758290 conn(0x7f55f40fb330 msgr2=0x7f55f4103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:22.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.694+0000 7f55fa81b700 1 -- 172.21.15.33:0/983758290 shutdown_connections 2024-05-08T21:23:22.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:22.694+0000 7f55fa81b700 1 -- 172.21.15.33:0/983758290 wait complete. 2024-05-08T21:23:22.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:22 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3911885280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:23:22.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:22 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3911885280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:23:22.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:22 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3911885280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:23:23.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:23 smithi033 ceph-mon[42528]: pgmap v422: 129 pgs: 129 active+clean; 198 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 438 op/s 2024-05-08T21:23:23.712 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:23:23.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:23:23.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:23 smithi071 ceph-mon[47074]: pgmap v422: 129 pgs: 129 active+clean; 198 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 438 op/s 2024-05-08T21:23:23.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:23 smithi082 ceph-mon[47512]: pgmap v422: 129 pgs: 129 active+clean; 198 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 438 op/s 2024-05-08T21:23:23.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.874+0000 7f63c2125700 1 -- 172.21.15.33:0/366644776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63bc100f50 msgr2=0x7f63bc1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:23.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.874+0000 7f63c2125700 1 --2- 172.21.15.33:0/366644776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63bc100f50 0x7f63bc1013b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009920 tx=0x7f63b0009c30 comp rx=0 tx=0).stop 2024-05-08T21:23:23.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.874+0000 7f63c2125700 1 -- 172.21.15.33:0/366644776 shutdown_connections 2024-05-08T21:23:23.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.874+0000 7f63c2125700 1 --2- 172.21.15.33:0/366644776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63bc1018f0 0x7f63bc1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.874+0000 7f63c2125700 1 --2- 172.21.15.33:0/366644776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc0ffd60 0x7f63bc100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.874+0000 7f63c2125700 1 --2- 172.21.15.33:0/366644776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63bc100f50 0x7f63bc1013b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.874+0000 7f63c2125700 1 -- 172.21.15.33:0/366644776 >> 172.21.15.33:0/366644776 conn(0x7f63bc0fb290 msgr2=0x7f63bc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:23.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.874+0000 7f63c2125700 1 -- 172.21.15.33:0/366644776 shutdown_connections 2024-05-08T21:23:23.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.874+0000 7f63c2125700 1 -- 172.21.15.33:0/366644776 wait complete. 2024-05-08T21:23:23.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.875+0000 7f63c2125700 1 Processor -- start 2024-05-08T21:23:23.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.875+0000 7f63c2125700 1 -- start start 2024-05-08T21:23:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.875+0000 7f63c2125700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63bc0ffd60 0x7f63bc196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.875+0000 7f63c2125700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc100f50 0x7f63bc197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.875+0000 7f63c2125700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63bc1018f0 0x7f63bc19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.875+0000 7f63c2125700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc19e8f0 con 0x7f63bc100f50 2024-05-08T21:23:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.875+0000 7f63c2125700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc19ea30 con 0x7f63bc0ffd60 2024-05-08T21:23:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.875+0000 7f63c2125700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc19ebc0 con 0x7f63bc1018f0 2024-05-08T21:23:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.875+0000 7f63baffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc100f50 0x7f63bc197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.875+0000 7f63baffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc100f50 0x7f63bc197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47306/0 (socket says 172.21.15.33:47306) 2024-05-08T21:23:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63bb7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63bc0ffd60 0x7f63bc196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63baffd700 1 -- 172.21.15.33:0/3413200079 learned_addr learned my addr 172.21.15.33:0/3413200079 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63bbfff700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63bc1018f0 0x7f63bc19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63baffd700 1 -- 172.21.15.33:0/3413200079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63bc1018f0 msgr2=0x7f63bc19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63baffd700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63bc1018f0 0x7f63bc19e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63baffd700 1 -- 172.21.15.33:0/3413200079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63bc0ffd60 msgr2=0x7f63bc196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63baffd700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63bc0ffd60 0x7f63bc196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63baffd700 1 -- 172.21.15.33:0/3413200079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b0009520 con 0x7f63bc100f50 2024-05-08T21:23:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63baffd700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc100f50 0x7f63bc197300 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009840 tx=0x7f63b000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63b8ff9700 1 -- 172.21.15.33:0/3413200079 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b001c070 con 0x7f63bc100f50 2024-05-08T21:23:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63c2125700 1 -- 172.21.15.33:0/3413200079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc19ee40 con 0x7f63bc100f50 2024-05-08T21:23:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63c2125700 1 -- 172.21.15.33:0/3413200079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc19f260 con 0x7f63bc100f50 2024-05-08T21:23:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63b8ff9700 1 -- 172.21.15.33:0/3413200079 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63b000bec0 con 0x7f63bc100f50 2024-05-08T21:23:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.876+0000 7f63b8ff9700 1 -- 172.21.15.33:0/3413200079 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0003ab0 con 0x7f63bc100f50 2024-05-08T21:23:23.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.880+0000 7f63b8ff9700 1 -- 172.21.15.33:0/3413200079 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f63b0003c10 con 0x7f63bc100f50 2024-05-08T21:23:23.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.880+0000 7f63b8ff9700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f63a40a1120 0x7f63a40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.880+0000 7f63bb7fe700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f63a40a1120 0x7f63a40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.880+0000 7f63b8ff9700 1 -- 172.21.15.33:0/3413200079 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f63b00cd810 con 0x7f63bc100f50 2024-05-08T21:23:23.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.881+0000 7f63bb7fe700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f63a40a1120 0x7f63a40a35e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f63bc100db0 tx=0x7f63ac009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:23.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.881+0000 7f63ce2dfb80 1 Processor -- start 2024-05-08T21:23:23.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63ce2dfb80 1 -- start start 2024-05-08T21:23:23.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63ce2dfb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6ba1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63ce2dfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 0x55eaf6ba27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63ce2dfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6ba3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63ce2dfb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf6af41d0 con 0x55eaf6ba19d0 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63ce2dfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf6af1b90 con 0x55eaf6ba2cf0 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63ce2dfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf6aed480 con 0x55eaf6ba2330 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6ba1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63baffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 0x55eaf6ba27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63baffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 0x55eaf6ba27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42856/0 (socket says 172.21.15.33:42856) 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63baffd700 1 -- 172.21.15.33:0/3913032009 learned_addr learned my addr 172.21.15.33:0/3913032009 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63bbfff700 1 --2- 172.21.15.33:0/3913032009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6ba3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63baffd700 1 -- 172.21.15.33:0/3913032009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2cf0 msgr2=0x55eaf6ba3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63baffd700 1 --2- 172.21.15.33:0/3913032009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6ba3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63baffd700 1 -- 172.21.15.33:0/3913032009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 msgr2=0x55eaf6ba1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63baffd700 1 --2- 172.21.15.33:0/3913032009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6ba1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.882+0000 7f63baffd700 1 -- 172.21.15.33:0/3913032009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaf696e430 con 0x55eaf6ba2330 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.883+0000 7f63baffd700 1 --2- 172.21.15.33:0/3913032009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 0x55eaf6ba27b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63b0005e90 tx=0x7f63b000f8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4757ee2e9e4df4b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.883+0000 7f63a1ffb700 1 -- 172.21.15.33:0/3913032009 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b009b070 con 0x55eaf6ba2330 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.883+0000 7f63a1ffb700 1 -- 172.21.15.33:0/3913032009 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63b009a970 con 0x55eaf6ba2330 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.883+0000 7f63a1ffb700 1 -- 172.21.15.33:0/3913032009 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0099e60 con 0x55eaf6ba2330 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.884+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/3913032009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 msgr2=0x55eaf6ba27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.884+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/3913032009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 0x55eaf6ba27b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63b0005e90 tx=0x7f63b000f8b0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.884+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/3913032009 shutdown_connections 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.884+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/3913032009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6ba3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.884+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/3913032009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6ba1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.884+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/3913032009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 0x55eaf6ba27b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.884+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/3913032009 >> 172.21.15.33:0/3913032009 conn(0x55eaf6afb290 msgr2=0x55eaf6b97e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.884+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/3913032009 shutdown_connections 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.884+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/3913032009 wait complete. 2024-05-08T21:23:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.884+0000 7f63ce2dfb80 1 -- start start 2024-05-08T21:23:23.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.884+0000 7f63ce2dfb80 1 Processor -- start 2024-05-08T21:23:23.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63ce2dfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba19d0 0x55eaf6baafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63ce2dfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 0x55eaf6bab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63ce2dfb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6bb2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63ce2dfb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf6a895e0 con 0x55eaf6ba2cf0 2024-05-08T21:23:23.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63ce2dfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf6a89280 con 0x55eaf6ba2330 2024-05-08T21:23:23.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63ce2dfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf6970720 con 0x55eaf6ba19d0 2024-05-08T21:23:23.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63ce2dfb80 10 client.0 _ll_register_callbacks cb 0x55eaf6a92bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63bb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba19d0 0x55eaf6baafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6bb2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63baffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 0x55eaf6bab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63bb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba19d0 0x55eaf6baafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42864/0 (socket says 172.21.15.33:42864) 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63baffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 0x55eaf6bab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:43172/0 (socket says 172.21.15.33:43172) 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6bb2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47322/0 (socket says 172.21.15.33:47322) 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63bb7fe700 1 -- 172.21.15.33:0/4074854248 learned_addr learned my addr 172.21.15.33:0/4074854248 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63bb7fe700 1 -- 172.21.15.33:0/4074854248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 msgr2=0x55eaf6bab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63bb7fe700 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 0x55eaf6bab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63bb7fe700 1 -- 172.21.15.33:0/4074854248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba2cf0 msgr2=0x55eaf6bb2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63bb7fe700 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6bb2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63bb7fe700 1 -- 172.21.15.33:0/4074854248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc19f260 con 0x55eaf6ba19d0 2024-05-08T21:23:23.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63baffd700 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 0x55eaf6bab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63bbfff700 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6bb2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f63bb7fe700 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba19d0 0x55eaf6baafc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f63ac01dc00 tx=0x7f63ac0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a67e50354391064a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.885+0000 7f639affd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f639affd700 1 -- 172.21.15.33:0/4074854248 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac022070 con 0x55eaf6ba19d0 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f639affd700 1 -- 172.21.15.33:0/4074854248 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f63ac01ca20 con 0x55eaf6ba19d0 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/4074854248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eaf6a462f0 con 0x55eaf6ba19d0 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f639affd700 1 -- 172.21.15.33:0/4074854248 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63ac01cc20 con 0x55eaf6ba19d0 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f63ce2dfb80 10 client.26695 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/4074854248 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55eaf691abe0 con 0x55eaf6ba19d0 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f639affd700 1 -- 172.21.15.33:0/4074854248 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55eaf6a462f0 con 0x55eaf6ba19d0 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f639affd700 1 -- 172.21.15.33:0/4074854248 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f63ac026ca0 con 0x55eaf6ba19d0 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f63ce2dfb80 10 client.26695 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f63ce2dfb80 10 client.26695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f63ce2dfb80 20 client.26695 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:23.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55eaf6bb6e50 0x55eaf6bb9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f63ce2dfb80 4 client.26695 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.886+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/4074854248 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eaf6b6c890 con 0x55eaf6bb6e50 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.887+0000 7f63baffd700 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55eaf6bb6e50 0x55eaf6bb9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.887+0000 7f63baffd700 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55eaf6bb6e50 0x55eaf6bb9270 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d6f458892bfa4dc server_cookie=f00414b477ca2202 in_seq=0 out_seq=0 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.887+0000 7f639affd700 10 client.26695 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.887+0000 7f639affd700 1 -- 172.21.15.33:0/4074854248 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eaf6b6c890 con 0x55eaf6bb6e50 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.887+0000 7f639affd700 10 client.26695 handle_command_reply: tid=0 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.887+0000 7f63ce2dfb80 1 client.26695 shutdown 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.888+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/4074854248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba19d0 msgr2=0x55eaf6baafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.888+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba19d0 0x55eaf6baafc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f63ac01dc00 tx=0x7f63ac0047c0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.888+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/4074854248 shutdown_connections 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.888+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55eaf6bb6e50 0x55eaf6bb9270 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.888+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6bb2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.888+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba19d0 0x55eaf6baafc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.888+0000 7f639affd700 0 client.26695 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:23.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.888+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/4074854248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 0x55eaf6bab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.888+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/4074854248 >> 172.21.15.33:0/4074854248 conn(0x55eaf6afb290 msgr2=0x55eaf6b97620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:23.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.888+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/4074854248 shutdown_connections 2024-05-08T21:23:23.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.888+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/4074854248 wait complete. 2024-05-08T21:23:23.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.888+0000 7f63ce2dfb80 20 client.26695 trim_cache size 0 max 16384 2024-05-08T21:23:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63ce2dfb80 1 Processor -- start 2024-05-08T21:23:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63ce2dfb80 1 -- start start 2024-05-08T21:23:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63ce2dfb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6baa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63ce2dfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 0x55eaf6bb24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63ce2dfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6baad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63ce2dfb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf6bb36c0 con 0x55eaf6ba19d0 2024-05-08T21:23:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63ce2dfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf6bb4250 con 0x55eaf6ba2330 2024-05-08T21:23:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63ce2dfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf6b94e40 con 0x55eaf6ba2cf0 2024-05-08T21:23:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6baa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63baffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 0x55eaf6bb24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63baffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 0x55eaf6bb24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:43188/0 (socket says 172.21.15.33:43188) 2024-05-08T21:23:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6baa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47326/0 (socket says 172.21.15.33:47326) 2024-05-08T21:23:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63baffd700 1 -- 172.21.15.33:0/1912497326 learned_addr learned my addr 172.21.15.33:0/1912497326 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.908+0000 7f63bbfff700 1 --2- 172.21.15.33:0/1912497326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6baad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.909+0000 7f63baffd700 1 -- 172.21.15.33:0/1912497326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2cf0 msgr2=0x55eaf6baad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.909+0000 7f63baffd700 1 --2- 172.21.15.33:0/1912497326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6baad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.909+0000 7f63baffd700 1 -- 172.21.15.33:0/1912497326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 msgr2=0x55eaf6baa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.909+0000 7f63baffd700 1 --2- 172.21.15.33:0/1912497326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6baa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.909+0000 7f63baffd700 1 -- 172.21.15.33:0/1912497326 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaf6b94560 con 0x55eaf6ba2330 2024-05-08T21:23:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.909+0000 7f63baffd700 1 --2- 172.21.15.33:0/1912497326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 0x55eaf6bb24d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f63b00098f0 tx=0x7f63b0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81b0633fbfc87f6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.909+0000 7f63a27fc700 1 -- 172.21.15.33:0/1912497326 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b009a740 con 0x55eaf6ba2330 2024-05-08T21:23:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.909+0000 7f63a27fc700 1 -- 172.21.15.33:0/1912497326 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63b009e7a0 con 0x55eaf6ba2330 2024-05-08T21:23:23.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.909+0000 7f63a27fc700 1 -- 172.21.15.33:0/1912497326 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00a28d0 con 0x55eaf6ba2330 2024-05-08T21:23:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.910+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/1912497326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 msgr2=0x55eaf6bb24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.910+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/1912497326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 0x55eaf6bb24d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f63b00098f0 tx=0x7f63b0009260 comp rx=0 tx=0).stop 2024-05-08T21:23:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.910+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/1912497326 shutdown_connections 2024-05-08T21:23:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.910+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/1912497326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6baad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.910+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/1912497326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6baa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.910+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/1912497326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2330 0x55eaf6bb24d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.910+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/1912497326 >> 172.21.15.33:0/1912497326 conn(0x55eaf6afb290 msgr2=0x55eaf6ba92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.910+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/1912497326 shutdown_connections 2024-05-08T21:23:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.910+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/1912497326 wait complete. 2024-05-08T21:23:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.910+0000 7f63ce2dfb80 1 -- start start 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63ce2dfb80 1 Processor -- start 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63ce2dfb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6bae380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63ce2dfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 0x55eaf6bae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63ce2dfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6bb18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63ce2dfb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf6bb1e20 con 0x55eaf6ba19d0 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63ce2dfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf6bb1f90 con 0x55eaf6ba2cf0 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63ce2dfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaf6bb2150 con 0x55eaf6ba2330 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63ce2dfb80 10 client.0 _ll_register_callbacks cb 0x55eaf6a92bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6bae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6bae380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47338/0 (socket says 172.21.15.33:47338) 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63bb7fe700 1 -- 172.21.15.33:0/433383969 learned_addr learned my addr 172.21.15.33:0/433383969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63baffd700 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 0x55eaf6bae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63bbfff700 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6bb18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63bb7fe700 1 -- 172.21.15.33:0/433383969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 msgr2=0x55eaf6bae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63bb7fe700 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 0x55eaf6bae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63bb7fe700 1 -- 172.21.15.33:0/433383969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2cf0 msgr2=0x55eaf6bb18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63bb7fe700 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6bb18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63bb7fe700 1 -- 172.21.15.33:0/433383969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaf696e430 con 0x55eaf6ba19d0 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.911+0000 7f63baffd700 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 0x55eaf6bae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.912+0000 7f63bb7fe700 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6bae380 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f63ac016040 tx=0x7f63ac00e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286138be6c248f49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.912+0000 7f639affd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.912+0000 7f639affd700 1 -- 172.21.15.33:0/433383969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac022070 con 0x55eaf6ba19d0 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.912+0000 7f639affd700 1 -- 172.21.15.33:0/433383969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f63ac009e70 con 0x55eaf6ba19d0 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.912+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/433383969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63b009a970 con 0x55eaf6ba19d0 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.912+0000 7f639affd700 1 -- 172.21.15.33:0/433383969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63ac01cc30 con 0x55eaf6ba19d0 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.912+0000 7f63ce2dfb80 10 client.16815 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.912+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/433383969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55eaf6ab35e0 con 0x55eaf6ba19d0 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.912+0000 7f639affd700 1 -- 172.21.15.33:0/433383969 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63b009a970 con 0x55eaf6ba19d0 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.913+0000 7f639affd700 1 -- 172.21.15.33:0/433383969 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f63ac026670 con 0x55eaf6ba19d0 2024-05-08T21:23:23.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.913+0000 7f63ce2dfb80 10 client.16815 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.913+0000 7f63ce2dfb80 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.913+0000 7f63ce2dfb80 20 client.16815 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.913+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55eaf6c0e4e0 0x55eaf6c10900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.913+0000 7f63ce2dfb80 4 client.16815 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.913+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/433383969 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55eaf6ae5f40 con 0x55eaf6c0e4e0 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.913+0000 7f63baffd700 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55eaf6c0e4e0 0x55eaf6c10900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.913+0000 7f63baffd700 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55eaf6c0e4e0 0x55eaf6c10900 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c21fdef6719a3 server_cookie=2c5ce11fda0d240e in_seq=0 out_seq=0 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.913+0000 7f639affd700 10 client.16815 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.914+0000 7f639affd700 1 -- 172.21.15.33:0/433383969 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55eaf6ae5f40 con 0x55eaf6c0e4e0 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.914+0000 7f639affd700 10 client.16815 handle_command_reply: tid=0 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.914+0000 7f63ce2dfb80 1 client.16815 shutdown 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.914+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/433383969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 msgr2=0x55eaf6bae380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:23.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.914+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6bae380 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f63ac016040 tx=0x7f63ac00e800 comp rx=0 tx=0).stop 2024-05-08T21:23:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.914+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/433383969 shutdown_connections 2024-05-08T21:23:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.914+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55eaf6c0e4e0 0x55eaf6c10900 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.914+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eaf6ba2cf0 0x55eaf6bb18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.914+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eaf6ba19d0 0x55eaf6bae380 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.914+0000 7f63ce2dfb80 1 --2- 172.21.15.33:0/433383969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eaf6ba2330 0x55eaf6bae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.914+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/433383969 >> 172.21.15.33:0/433383969 conn(0x55eaf6afb290 msgr2=0x55eaf6ba5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.914+0000 7f639affd700 0 client.16815 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/433383969 shutdown_connections 2024-05-08T21:23:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63ce2dfb80 1 -- 172.21.15.33:0/433383969 wait complete. 2024-05-08T21:23:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63ce2dfb80 20 client.16815 trim_cache size 0 max 16384 2024-05-08T21:23:23.922 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:23:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:23:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "1e353d11-4e4c-4a5a-91c7-58022856e3ca", 2024-05-08T21:23:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:23:23.923 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:23:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63c2125700 1 -- 172.21.15.33:0/3413200079 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f63a40a1120 msgr2=0x7f63a40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63c2125700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f63a40a1120 0x7f63a40a35e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f63bc100db0 tx=0x7f63ac009310 comp rx=0 tx=0).stop 2024-05-08T21:23:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63c2125700 1 -- 172.21.15.33:0/3413200079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc100f50 msgr2=0x7f63bc197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63c2125700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc100f50 0x7f63bc197300 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009840 tx=0x7f63b000b790 comp rx=0 tx=0).stop 2024-05-08T21:23:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63c2125700 1 -- 172.21.15.33:0/3413200079 shutdown_connections 2024-05-08T21:23:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63c2125700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63bc1018f0 0x7f63bc19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63c2125700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f63a40a1120 0x7f63a40a35e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63c2125700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63bc0ffd60 0x7f63bc196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63c2125700 1 --2- 172.21.15.33:0/3413200079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc100f50 0x7f63bc197300 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63c2125700 1 -- 172.21.15.33:0/3413200079 >> 172.21.15.33:0/3413200079 conn(0x7f63bc0fb290 msgr2=0x7f63bc068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.915+0000 7f63c2125700 1 -- 172.21.15.33:0/3413200079 shutdown_connections 2024-05-08T21:23:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:23.916+0000 7f63c2125700 1 -- 172.21.15.33:0/3413200079 wait complete. 2024-05-08T21:23:23.934 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:23:24.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.121+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/427206767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c4101fd0 msgr2=0x7fe2c4102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:24.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.121+0000 7fe2ca8e1700 1 --2- 172.21.15.33:0/427206767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c4101fd0 0x7fe2c4102450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009990 tx=0x7fe2b8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.123+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/427206767 shutdown_connections 2024-05-08T21:23:24.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.123+0000 7fe2ca8e1700 1 --2- 172.21.15.33:0/427206767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c4102990 0x7fe2c41092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.123+0000 7fe2ca8e1700 1 --2- 172.21.15.33:0/427206767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c4100e80 0x7fe2c41012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.123+0000 7fe2ca8e1700 1 --2- 172.21.15.33:0/427206767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c4101fd0 0x7fe2c4102450 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.123+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/427206767 >> 172.21.15.33:0/427206767 conn(0x7fe2c40fc3d0 msgr2=0x7fe2c40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:24.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.123+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/427206767 shutdown_connections 2024-05-08T21:23:24.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.123+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/427206767 wait complete. 2024-05-08T21:23:24.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.123+0000 7fe2ca8e1700 1 Processor -- start 2024-05-08T21:23:24.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.124+0000 7fe2ca8e1700 1 -- start start 2024-05-08T21:23:24.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.124+0000 7fe2ca8e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c4100e80 0x7fe2c4196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.124+0000 7fe2ca8e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c4101fd0 0x7fe2c4197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.124+0000 7fe2ca8e1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c4102990 0x7fe2c419e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.124+0000 7fe2ca8e1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c419e7f0 con 0x7fe2c4100e80 2024-05-08T21:23:24.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.124+0000 7fe2ca8e1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c419e960 con 0x7fe2c4102990 2024-05-08T21:23:24.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.124+0000 7fe2ca8e1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c40731d0 con 0x7fe2c4101fd0 2024-05-08T21:23:24.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.124+0000 7fe2c8e7e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c4102990 0x7fe2c419e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.124+0000 7fe2c8e7e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c4102990 0x7fe2c419e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:43224/0 (socket says 172.21.15.33:43224) 2024-05-08T21:23:24.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.124+0000 7fe2c8e7e700 1 -- 172.21.15.33:0/3435185572 learned_addr learned my addr 172.21.15.33:0/3435185572 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:24.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.124+0000 7fe2c3fff700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c4100e80 0x7fe2c4196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.124+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c4101fd0 0x7fe2c4197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.125+0000 7fe2c8e7e700 1 -- 172.21.15.33:0/3435185572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c4101fd0 msgr2=0x7fe2c4197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:24.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.125+0000 7fe2c8e7e700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c4101fd0 0x7fe2c4197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.125+0000 7fe2c8e7e700 1 -- 172.21.15.33:0/3435185572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c4100e80 msgr2=0x7fe2c4196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:24.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.125+0000 7fe2c8e7e700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c4100e80 0x7fe2c4196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.125+0000 7fe2c8e7e700 1 -- 172.21.15.33:0/3435185572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b80095f0 con 0x7fe2c4102990 2024-05-08T21:23:24.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.125+0000 7fe2c8e7e700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c4102990 0x7fe2c419e220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00ef30 tx=0x7fe2bc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:24.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.125+0000 7fe2c17fa700 1 -- 172.21.15.33:0/3435185572 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc018070 con 0x7fe2c4102990 2024-05-08T21:23:24.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.125+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/3435185572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c4073450 con 0x7fe2c4102990 2024-05-08T21:23:24.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.125+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/3435185572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c4073960 con 0x7fe2c4102990 2024-05-08T21:23:24.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.125+0000 7fe2c17fa700 1 -- 172.21.15.33:0/3435185572 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2bc00f040 con 0x7fe2c4102990 2024-05-08T21:23:24.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.125+0000 7fe2c17fa700 1 -- 172.21.15.33:0/3435185572 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc00cc70 con 0x7fe2c4102990 2024-05-08T21:23:24.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.131+0000 7fe2d6a9bb80 1 Processor -- start 2024-05-08T21:23:24.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.132+0000 7fe2c17fa700 1 -- 172.21.15.33:0/3435185572 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe2bc007500 con 0x7fe2c4102990 2024-05-08T21:23:24.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.132+0000 7fe2d6a9bb80 1 -- start start 2024-05-08T21:23:24.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.132+0000 7fe2d6a9bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1540 0x557c071f1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.132+0000 7fe2d6a9bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1ea0 0x557c071f2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.132+0000 7fe2d6a9bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c071f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.132+0000 7fe2d6a9bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c07148150 con 0x557c071f1540 2024-05-08T21:23:24.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.132+0000 7fe2d6a9bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c07145b10 con 0x557c071f2860 2024-05-08T21:23:24.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.132+0000 7fe2d6a9bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c07141400 con 0x557c071f1ea0 2024-05-08T21:23:24.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.132+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1ea0 0x557c071f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.133+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1ea0 0x557c071f2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:42916/0 (socket says 172.21.15.33:42916) 2024-05-08T21:23:24.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.133+0000 7fe2c3fff700 1 -- 172.21.15.33:0/1017738580 learned_addr learned my addr 172.21.15.33:0/1017738580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:24.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.133+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/1017738580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c071f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.133+0000 7fe2c17fa700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe2ac0a1120 0x7fe2ac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.133+0000 7fe2c17fa700 1 -- 172.21.15.33:0/3435185572 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe2bc0cc7e0 con 0x7fe2c4102990 2024-05-08T21:23:24.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.133+0000 7fe2c37fe700 1 -- 172.21.15.33:0/1017738580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1ea0 msgr2=0x557c071f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:24.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.133+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/1017738580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1ea0 0x557c071f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.133+0000 7fe2c37fe700 1 -- 172.21.15.33:0/1017738580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1540 msgr2=0x557c071f1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:24.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.133+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/1017738580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1540 0x557c071f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.133+0000 7fe2c8e7e700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe2ac0a1120 0x7fe2ac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.133+0000 7fe2c37fe700 1 -- 172.21.15.33:0/1017738580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c06fc23b0 con 0x557c071f2860 2024-05-08T21:23:24.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.133+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/1017738580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c071f2ce0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80098e0 tx=0x7fe2b8005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ef0ff3bc445dcbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:24.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.134+0000 7fe2c8e7e700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe2ac0a1120 0x7fe2ac0a35e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0090a0 tx=0x7fe2bc004ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:24.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.134+0000 7fe2aa7fc700 1 -- 172.21.15.33:0/1017738580 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8004200 con 0x557c071f2860 2024-05-08T21:23:24.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.134+0000 7fe2aa7fc700 1 -- 172.21.15.33:0/1017738580 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2b801e070 con 0x557c071f2860 2024-05-08T21:23:24.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.134+0000 7fe2aa7fc700 1 -- 172.21.15.33:0/1017738580 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b800bce0 con 0x557c071f2860 2024-05-08T21:23:24.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.135+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/1017738580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 msgr2=0x557c071f2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:24.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.135+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/1017738580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c071f2ce0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80098e0 tx=0x7fe2b8005ed0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.135+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/1017738580 shutdown_connections 2024-05-08T21:23:24.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.135+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/1017738580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c071f2ce0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.135+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/1017738580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1540 0x557c071f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.135+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/1017738580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1ea0 0x557c071f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.135+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/1017738580 >> 172.21.15.33:0/1017738580 conn(0x557c0714f210 msgr2=0x557c071ebde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:24.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.135+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/1017738580 shutdown_connections 2024-05-08T21:23:24.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.136+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/1017738580 wait complete. 2024-05-08T21:23:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.136+0000 7fe2d6a9bb80 1 -- start start 2024-05-08T21:23:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.136+0000 7fe2d6a9bb80 1 Processor -- start 2024-05-08T21:23:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2d6a9bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f1540 0x557c07203670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2d6a9bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c07203bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2d6a9bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f2860 0x557c0720ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2d6a9bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c070dd560 con 0x557c071f1ea0 2024-05-08T21:23:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2d6a9bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c070dd200 con 0x557c071f1540 2024-05-08T21:23:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2d6a9bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c06fc46a0 con 0x557c071f2860 2024-05-08T21:23:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2d6a9bb80 10 client.0 _ll_register_callbacks cb 0x557c070e6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c07203bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c07203bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47394/0 (socket says 172.21.15.33:47394) 2024-05-08T21:23:24.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2c3fff700 1 -- 172.21.15.33:0/2972975588 learned_addr learned my addr 172.21.15.33:0/2972975588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:24.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f1540 0x557c07203670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f2860 0x557c0720ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2c3fff700 1 -- 172.21.15.33:0/2972975588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f2860 msgr2=0x557c0720ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:24.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2c3fff700 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f2860 0x557c0720ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2c3fff700 1 -- 172.21.15.33:0/2972975588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f1540 msgr2=0x557c07203670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:24.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2c3fff700 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f1540 0x557c07203670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.137+0000 7fe2c3fff700 1 -- 172.21.15.33:0/2972975588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c4109be0 con 0x557c071f1ea0 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.138+0000 7fe2c3fff700 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c07203bb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400bb70 tx=0x7fe2b400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88910b44b2b3a6ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.138+0000 7fe2a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.138+0000 7fe2a37fe700 1 -- 172.21.15.33:0/2972975588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4016070 con 0x557c071f1ea0 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.138+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2972975588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c0709a270 con 0x557c071f1ea0 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.138+0000 7fe2a37fe700 1 -- 172.21.15.33:0/2972975588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe2b4003d00 con 0x557c071f1ea0 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.138+0000 7fe2a37fe700 1 -- 172.21.15.33:0/2972975588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2b4004830 con 0x557c071f1ea0 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.138+0000 7fe2d6a9bb80 10 client.16824 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.138+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2972975588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557c06f6eb60 con 0x557c071f1ea0 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.139+0000 7fe2a37fe700 1 -- 172.21.15.33:0/2972975588 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557c0709a270 con 0x557c071f1ea0 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.139+0000 7fe2a37fe700 1 -- 172.21.15.33:0/2972975588 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fe2b4005440 con 0x557c071f1ea0 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.139+0000 7fe2d6a9bb80 10 client.16824 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.139+0000 7fe2d6a9bb80 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:24.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.139+0000 7fe2d6a9bb80 20 client.16824 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.139+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x557c07220480 0x557c072228a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.139+0000 7fe2d6a9bb80 4 client.16824 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.139+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2972975588 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c071c0810 con 0x557c07220480 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.139+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x557c07220480 0x557c072228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.140+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x557c07220480 0x557c072228a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46342fed7f9db1c3 server_cookie=72195280708b57ed in_seq=0 out_seq=0 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.140+0000 7fe2a37fe700 10 client.16824 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2a37fe700 1 -- 172.21.15.33:0/2972975588 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c071c0810 con 0x557c07220480 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2a37fe700 10 client.16824 handle_command_reply: tid=0 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2d6a9bb80 1 client.16824 shutdown 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2972975588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 msgr2=0x557c07203bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c07203bb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400bb70 tx=0x7fe2b400be80 comp rx=0 tx=0).stop 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2972975588 shutdown_connections 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x557c07220480 0x557c072228a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f2860 0x557c0720ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f1540 0x557c07203670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2a37fe700 0 client.16824 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:24.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/2972975588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c07203bb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2972975588 >> 172.21.15.33:0/2972975588 conn(0x557c0714f210 msgr2=0x557c071eb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:24.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2972975588 shutdown_connections 2024-05-08T21:23:24.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2972975588 wait complete. 2024-05-08T21:23:24.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.141+0000 7fe2d6a9bb80 20 client.16824 trim_cache size 0 max 16384 2024-05-08T21:23:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.162+0000 7fe2d6a9bb80 1 Processor -- start 2024-05-08T21:23:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.162+0000 7fe2d6a9bb80 1 -- start start 2024-05-08T21:23:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.162+0000 7fe2d6a9bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1540 0x557c07202b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.162+0000 7fe2d6a9bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c0720ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.162+0000 7fe2d6a9bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c071f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.162+0000 7fe2d6a9bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c071ed8c0 con 0x557c071f1ea0 2024-05-08T21:23:24.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.162+0000 7fe2d6a9bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c07203210 con 0x557c071f2860 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.162+0000 7fe2d6a9bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c071e4120 con 0x557c071f1540 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c0720ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c0720ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47398/0 (socket says 172.21.15.33:47398) 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2c3fff700 1 -- 172.21.15.33:0/3682136732 learned_addr learned my addr 172.21.15.33:0/3682136732 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/3682136732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1540 0x557c07202b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/3682136732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c071f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2c3fff700 1 -- 172.21.15.33:0/3682136732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1540 msgr2=0x557c07202b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2c3fff700 1 --2- 172.21.15.33:0/3682136732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1540 0x557c07202b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2c3fff700 1 -- 172.21.15.33:0/3682136732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 msgr2=0x557c071f8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2c3fff700 1 --2- 172.21.15.33:0/3682136732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c071f8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2c3fff700 1 -- 172.21.15.33:0/3682136732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c07107560 con 0x557c071f1ea0 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2c3fff700 1 --2- 172.21.15.33:0/3682136732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c0720ab10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4001eb0 tx=0x7fe2b400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6832aeb60e3b2fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2aaffd700 1 -- 172.21.15.33:0/3682136732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4016070 con 0x557c071f1ea0 2024-05-08T21:23:24.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2aaffd700 1 -- 172.21.15.33:0/3682136732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2b4017040 con 0x557c071f1ea0 2024-05-08T21:23:24.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.163+0000 7fe2aaffd700 1 -- 172.21.15.33:0/3682136732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b401ba60 con 0x557c071f1ea0 2024-05-08T21:23:24.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.164+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/3682136732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 msgr2=0x557c0720ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:24.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.164+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/3682136732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c0720ab10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4001eb0 tx=0x7fe2b400bef0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.164+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/3682136732 shutdown_connections 2024-05-08T21:23:24.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.164+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/3682136732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c071f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.164+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/3682136732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1540 0x557c07202b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.164+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/3682136732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c0720ab10 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.164+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/3682136732 >> 172.21.15.33:0/3682136732 conn(0x557c0714f210 msgr2=0x557c071e8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.164+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/3682136732 shutdown_connections 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.164+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/3682136732 wait complete. 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.165+0000 7fe2d6a9bb80 1 -- start start 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.165+0000 7fe2d6a9bb80 1 Processor -- start 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.165+0000 7fe2d6a9bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1540 0x557c07208a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.165+0000 7fe2d6a9bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c07208f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.165+0000 7fe2d6a9bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c07209f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.165+0000 7fe2d6a9bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0720a490 con 0x557c071f1ea0 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.165+0000 7fe2d6a9bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0720a600 con 0x557c071f2860 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.165+0000 7fe2d6a9bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0720a7c0 con 0x557c071f1540 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c07208f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2c37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1540 0x557c07208a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2d6a9bb80 10 client.0 _ll_register_callbacks cb 0x557c070e6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c07208f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47406/0 (socket says 172.21.15.33:47406) 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2c3fff700 1 -- 172.21.15.33:0/2373396835 learned_addr learned my addr 172.21.15.33:0/2373396835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c07209f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2c3fff700 1 -- 172.21.15.33:0/2373396835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1540 msgr2=0x557c07208a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2c3fff700 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1540 0x557c07208a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2c3fff700 1 -- 172.21.15.33:0/2373396835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 msgr2=0x557c07209f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2c3fff700 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c07209f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2c3fff700 1 -- 172.21.15.33:0/2373396835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c06f6eb60 con 0x557c071f1ea0 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1540 0x557c07208a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2c3fff700 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c07208f90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005720 tx=0x7fe2b401bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a57824d05060f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2a37fe700 1 -- 172.21.15.33:0/2373396835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4016030 con 0x557c071f1ea0 2024-05-08T21:23:24.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2a37fe700 1 -- 172.21.15.33:0/2373396835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe2b4000e40 con 0x557c071f1ea0 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.166+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2373396835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2b801e070 con 0x557c071f1ea0 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.167+0000 7fe2a37fe700 1 -- 172.21.15.33:0/2373396835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2b401b5e0 con 0x557c071f1ea0 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.167+0000 7fe2d6a9bb80 10 client.16830 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.167+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2373396835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557c06fc6cd0 con 0x557c071f1ea0 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.167+0000 7fe2a37fe700 1 -- 172.21.15.33:0/2373396835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2b801e070 con 0x557c071f1ea0 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.167+0000 7fe2a37fe700 1 -- 172.21.15.33:0/2373396835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fe2b40114d0 con 0x557c071f1ea0 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.167+0000 7fe2d6a9bb80 10 client.16830 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.167+0000 7fe2d6a9bb80 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.167+0000 7fe2d6a9bb80 20 client.16830 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.167+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x557c072778d0 0x557c07279cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.167+0000 7fe2d6a9bb80 4 client.16830 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.167+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2373396835 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557c07139ec0 con 0x557c072778d0 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.167+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x557c072778d0 0x557c07279cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2c37fe700 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x557c072778d0 0x557c07279cf0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e53f1ee31110d3e7 server_cookie=acc2ad300ee85e46 in_seq=0 out_seq=0 2024-05-08T21:23:24.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2a37fe700 10 client.16830 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2a37fe700 1 -- 172.21.15.33:0/2373396835 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557c07139ec0 con 0x557c072778d0 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2a37fe700 10 client.16830 handle_command_reply: tid=0 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2d6a9bb80 1 client.16830 shutdown 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2373396835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 msgr2=0x557c07208f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c07208f90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005720 tx=0x7fe2b401bad0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2373396835 shutdown_connections 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x557c072778d0 0x557c07279cf0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557c071f2860 0x557c07209f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557c071f1540 0x557c07208a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2d6a9bb80 1 --2- 172.21.15.33:0/2373396835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c071f1ea0 0x557c07208f90 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2373396835 >> 172.21.15.33:0/2373396835 conn(0x557c0714f210 msgr2=0x557c071f58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2a37fe700 0 client.16830 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.168+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2373396835 shutdown_connections 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2d6a9bb80 1 -- 172.21.15.33:0/2373396835 wait complete. 2024-05-08T21:23:24.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2d6a9bb80 20 client.16830 trim_cache size 0 max 16384 2024-05-08T21:23:24.177 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:23:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:23:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:23:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "1e353d11-4e4c-4a5a-91c7-58022856e3ca": { 2024-05-08T21:23:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:23:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "1e353d11-4e4c-4a5a-91c7-58022856e3ca", 2024-05-08T21:23:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:23:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:23:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:23:24.177 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:23:24.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/3435185572 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe2ac0a1120 msgr2=0x7fe2ac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:24.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2ca8e1700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe2ac0a1120 0x7fe2ac0a35e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0090a0 tx=0x7fe2bc004ad0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/3435185572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c4102990 msgr2=0x7fe2c419e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:24.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2ca8e1700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c4102990 0x7fe2c419e220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00ef30 tx=0x7fe2bc00c3a0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/3435185572 shutdown_connections 2024-05-08T21:23:24.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2ca8e1700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe2ac0a1120 0x7fe2ac0a35e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2ca8e1700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2c4102990 0x7fe2c419e220 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2ca8e1700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2c4100e80 0x7fe2c4196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2ca8e1700 1 --2- 172.21.15.33:0/3435185572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2c4101fd0 0x7fe2c4197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:24.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/3435185572 >> 172.21.15.33:0/3435185572 conn(0x7fe2c40fc3d0 msgr2=0x7fe2c41054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:24.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/3435185572 shutdown_connections 2024-05-08T21:23:24.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:24.169+0000 7fe2ca8e1700 1 -- 172.21.15.33:0/3435185572 wait complete. 2024-05-08T21:23:24.186 INFO:tasks.cephfs.filesystem:scrub status for tag:1e353d11-4e4c-4a5a-91c7-58022856e3ca - {'path': '/', 'tag': '1e353d11-4e4c-4a5a-91c7-58022856e3ca', 'options': 'recursive,force'} 2024-05-08T21:23:24.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:24 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:24.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:24 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:24.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:24 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:24.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:24 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:24.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:24 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:24.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:24 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:24.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:24 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:24.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:24 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:24.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:24 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:24.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:24 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:24.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:24 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:24.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:24 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:24.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:24 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:24.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:24 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:24.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:24 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:25 smithi033 ceph-mon[42528]: pgmap v423: 129 pgs: 129 active+clean; 198 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 385 op/s 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:25 smithi071 ceph-mon[47074]: pgmap v423: 129 pgs: 129 active+clean; 198 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 385 op/s 2024-05-08T21:23:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:23:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:25.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:25.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:25 smithi082 ceph-mon[47512]: pgmap v423: 129 pgs: 129 active+clean; 198 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 385 op/s 2024-05-08T21:23:27.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:27 smithi033 ceph-mon[42528]: pgmap v424: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.2 MiB/s wr, 526 op/s 2024-05-08T21:23:27.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:27 smithi071 ceph-mon[47074]: pgmap v424: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.2 MiB/s wr, 526 op/s 2024-05-08T21:23:27.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:27 smithi082 ceph-mon[47512]: pgmap v424: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.2 MiB/s wr, 526 op/s 2024-05-08T21:23:29.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:29 smithi033 ceph-mon[42528]: pgmap v425: 129 pgs: 129 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 424 op/s 2024-05-08T21:23:29.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:29 smithi071 ceph-mon[47074]: pgmap v425: 129 pgs: 129 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 424 op/s 2024-05-08T21:23:29.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:29 smithi082 ceph-mon[47512]: pgmap v425: 129 pgs: 129 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 424 op/s 2024-05-08T21:23:31.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:31 smithi033 ceph-mon[42528]: pgmap v426: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.2 MiB/s wr, 521 op/s 2024-05-08T21:23:31.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:31 smithi071 ceph-mon[47074]: pgmap v426: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.2 MiB/s wr, 521 op/s 2024-05-08T21:23:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:31 smithi082 ceph-mon[47512]: pgmap v426: 129 pgs: 129 active+clean; 198 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.2 MiB/s wr, 521 op/s 2024-05-08T21:23:33.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:33 smithi033 ceph-mon[42528]: pgmap v427: 129 pgs: 129 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 418 op/s 2024-05-08T21:23:33.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:33 smithi071 ceph-mon[47074]: pgmap v427: 129 pgs: 129 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 418 op/s 2024-05-08T21:23:33.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:33 smithi082 ceph-mon[47512]: pgmap v427: 129 pgs: 129 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 418 op/s 2024-05-08T21:23:35.172 INFO:tasks.workunit.client.0.smithi033.stdout:30000 mapwrite 0xb7634 thru 0xbbad6 (0x44a3 bytes) 2024-05-08T21:23:35.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:35 smithi033 ceph-mon[42528]: pgmap v428: 129 pgs: 129 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 373 op/s 2024-05-08T21:23:35.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:35 smithi071 ceph-mon[47074]: pgmap v428: 129 pgs: 129 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 373 op/s 2024-05-08T21:23:35.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:35 smithi082 ceph-mon[47512]: pgmap v428: 129 pgs: 129 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.8 MiB/s wr, 373 op/s 2024-05-08T21:23:37.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:37 smithi033 ceph-mon[42528]: pgmap v429: 129 pgs: 129 active+clean; 200 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.9 MiB/s wr, 499 op/s 2024-05-08T21:23:37.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:37 smithi071 ceph-mon[47074]: pgmap v429: 129 pgs: 129 active+clean; 200 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.9 MiB/s wr, 499 op/s 2024-05-08T21:23:37.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:37 smithi082 ceph-mon[47512]: pgmap v429: 129 pgs: 129 active+clean; 200 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.9 MiB/s wr, 499 op/s 2024-05-08T21:23:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:38 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:23:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:38 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:23:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:38 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:23:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:38 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:23:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:38 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:23:38.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:38 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:23:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:38 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:23:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:38 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:23:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:38 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:23:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:38 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:23:38.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:38 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:23:38.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:38 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:23:38.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:38 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:23:38.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:38 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:23:38.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:38 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:23:39.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:39 smithi033 ceph-mon[42528]: pgmap v430: 129 pgs: 129 active+clean; 201 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 403 op/s 2024-05-08T21:23:39.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:39 smithi071 ceph-mon[47074]: pgmap v430: 129 pgs: 129 active+clean; 201 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 403 op/s 2024-05-08T21:23:39.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:39 smithi082 ceph-mon[47512]: pgmap v430: 129 pgs: 129 active+clean; 201 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 403 op/s 2024-05-08T21:23:41.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:41 smithi033 ceph-mon[42528]: pgmap v431: 129 pgs: 129 active+clean; 201 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 475 op/s 2024-05-08T21:23:41.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:41 smithi071 ceph-mon[47074]: pgmap v431: 129 pgs: 129 active+clean; 201 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 475 op/s 2024-05-08T21:23:41.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:41 smithi082 ceph-mon[47512]: pgmap v431: 129 pgs: 129 active+clean; 201 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.8 MiB/s wr, 475 op/s 2024-05-08T21:23:43.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:43 smithi033 ceph-mon[42528]: pgmap v432: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 375 op/s 2024-05-08T21:23:43.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:43 smithi071 ceph-mon[47074]: pgmap v432: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 375 op/s 2024-05-08T21:23:43.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:43 smithi082 ceph-mon[47512]: pgmap v432: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 375 op/s 2024-05-08T21:23:45.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:45 smithi033 ceph-mon[42528]: pgmap v433: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 334 op/s 2024-05-08T21:23:45.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:45 smithi071 ceph-mon[47074]: pgmap v433: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 334 op/s 2024-05-08T21:23:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:45 smithi082 ceph-mon[47512]: pgmap v433: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 334 op/s 2024-05-08T21:23:47.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:47 smithi033 ceph-mon[42528]: pgmap v434: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 452 op/s 2024-05-08T21:23:47.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:47 smithi071 ceph-mon[47074]: pgmap v434: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 452 op/s 2024-05-08T21:23:47.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:47 smithi082 ceph-mon[47512]: pgmap v434: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 452 op/s 2024-05-08T21:23:49.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:49 smithi033 ceph-mon[42528]: pgmap v435: 129 pgs: 129 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2024-05-08T21:23:49.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:49 smithi071 ceph-mon[47074]: pgmap v435: 129 pgs: 129 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2024-05-08T21:23:49.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:49 smithi082 ceph-mon[47512]: pgmap v435: 129 pgs: 129 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2024-05-08T21:23:51.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:51 smithi033 ceph-mon[42528]: pgmap v436: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 428 op/s 2024-05-08T21:23:51.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:51 smithi071 ceph-mon[47074]: pgmap v436: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 428 op/s 2024-05-08T21:23:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:51 smithi082 ceph-mon[47512]: pgmap v436: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.5 MiB/s wr, 428 op/s 2024-05-08T21:23:53.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:53 smithi033 ceph-mon[42528]: pgmap v437: 129 pgs: 129 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 347 op/s 2024-05-08T21:23:53.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:53 smithi071 ceph-mon[47074]: pgmap v437: 129 pgs: 129 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 347 op/s 2024-05-08T21:23:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:53 smithi082 ceph-mon[47512]: pgmap v437: 129 pgs: 129 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 347 op/s 2024-05-08T21:23:54.188 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:23:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.345+0000 7fa5b02a1700 1 -- 172.21.15.33:0/360742962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a81000b0 msgr2=0x7fa5a8100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.345+0000 7fa5b02a1700 1 --2- 172.21.15.33:0/360742962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a81000b0 0x7fa5a8100530 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5980099c0 tx=0x7fa598009cd0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.345+0000 7fa5b02a1700 1 -- 172.21.15.33:0/360742962 shutdown_connections 2024-05-08T21:23:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.345+0000 7fa5b02a1700 1 --2- 172.21.15.33:0/360742962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5a8100a70 0x7fa5a8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.345+0000 7fa5b02a1700 1 --2- 172.21.15.33:0/360742962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5a80ff750 0x7fa5a80ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.345+0000 7fa5b02a1700 1 --2- 172.21.15.33:0/360742962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a81000b0 0x7fa5a8100530 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.345+0000 7fa5b02a1700 1 -- 172.21.15.33:0/360742962 >> 172.21.15.33:0/360742962 conn(0x7fa5a80fb310 msgr2=0x7fa5a80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.345+0000 7fa5b02a1700 1 -- 172.21.15.33:0/360742962 shutdown_connections 2024-05-08T21:23:54.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.345+0000 7fa5b02a1700 1 -- 172.21.15.33:0/360742962 wait complete. 2024-05-08T21:23:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5b02a1700 1 Processor -- start 2024-05-08T21:23:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5b02a1700 1 -- start start 2024-05-08T21:23:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5b02a1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5a80ff750 0x7fa5a8198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5b02a1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a81000b0 0x7fa5a8199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5b02a1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5a8100a70 0x7fa5a81a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5b02a1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a81a0a30 con 0x7fa5a80ff750 2024-05-08T21:23:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5b02a1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a81a0b70 con 0x7fa5a81000b0 2024-05-08T21:23:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5b02a1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a81a0d00 con 0x7fa5a8100a70 2024-05-08T21:23:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5ae03d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5a80ff750 0x7fa5a8198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5ae03d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5a80ff750 0x7fa5a8198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36890/0 (socket says 172.21.15.33:36890) 2024-05-08T21:23:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5ae03d700 1 -- 172.21.15.33:0/973080934 learned_addr learned my addr 172.21.15.33:0/973080934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:54.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a81000b0 0x7fa5a8199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5ae83e700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5a8100a70 0x7fa5a81a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5ae03d700 1 -- 172.21.15.33:0/973080934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5a8100a70 msgr2=0x7fa5a81a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5a8100a70 0x7fa5a81a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5ae03d700 1 -- 172.21.15.33:0/973080934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a81000b0 msgr2=0x7fa5a8199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a81000b0 0x7fa5a8199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.346+0000 7fa5ae03d700 1 -- 172.21.15.33:0/973080934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5980095f0 con 0x7fa5a80ff750 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.347+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a81000b0 0x7fa5a8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.347+0000 7fa5ae83e700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5a8100a70 0x7fa5a81a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.347+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5a80ff750 0x7fa5a8198f00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400e9a0 tx=0x7fa5a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.347+0000 7fa59f7fe700 1 -- 172.21.15.33:0/973080934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4009790 con 0x7fa5a80ff750 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.347+0000 7fa59f7fe700 1 -- 172.21.15.33:0/973080934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa5a4010430 con 0x7fa5a80ff750 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.347+0000 7fa5b02a1700 1 -- 172.21.15.33:0/973080934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a81a0fe0 con 0x7fa5a80ff750 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.347+0000 7fa59f7fe700 1 -- 172.21.15.33:0/973080934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a400cea0 con 0x7fa5a80ff750 2024-05-08T21:23:54.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.347+0000 7fa5b02a1700 1 -- 172.21.15.33:0/973080934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a81a1400 con 0x7fa5a80ff750 2024-05-08T21:23:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.350+0000 7fa59f7fe700 1 -- 172.21.15.33:0/973080934 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa5a40105a0 con 0x7fa5a80ff750 2024-05-08T21:23:54.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.351+0000 7fa59f7fe700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa5940a1120 0x7fa5940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.351+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa5940a1120 0x7fa5940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.351+0000 7fa59f7fe700 1 -- 172.21.15.33:0/973080934 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa5a4014070 con 0x7fa5a80ff750 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.351+0000 7fa5bc45bb80 1 Processor -- start 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5bc45bb80 1 -- start start 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa5940a1120 0x7fa5940a35e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa598000c00 tx=0x7fa59801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5bc45bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48c540 0x562bff48c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5bc45bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48cea0 0x562bff48d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5bc45bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48d860 0x562bff48dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5bc45bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bff3e3150 con 0x562bff48c540 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5bc45bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bff3e0b10 con 0x562bff48d860 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5bc45bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bff3dc400 con 0x562bff48cea0 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5ad83c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48cea0 0x562bff48d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5ad83c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48cea0 0x562bff48d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43310/0 (socket says 172.21.15.33:43310) 2024-05-08T21:23:54.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5ad83c700 1 -- 172.21.15.33:0/796208154 learned_addr learned my addr 172.21.15.33:0/796208154 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5ae83e700 1 --2- 172.21.15.33:0/796208154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48d860 0x562bff48dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/796208154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48c540 0x562bff48c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5ad83c700 1 -- 172.21.15.33:0/796208154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48d860 msgr2=0x562bff48dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/796208154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48d860 0x562bff48dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5ad83c700 1 -- 172.21.15.33:0/796208154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48c540 msgr2=0x562bff48c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/796208154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48c540 0x562bff48c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.352+0000 7fa5ad83c700 1 -- 172.21.15.33:0/796208154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bff25d3b0 con 0x562bff48cea0 2024-05-08T21:23:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.353+0000 7fa5ae83e700 1 --2- 172.21.15.33:0/796208154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48d860 0x562bff48dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.353+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/796208154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48cea0 0x562bff48d320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5980099c0 tx=0x7fa59801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddff3bffc77ecc34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.353+0000 7fa59cda4700 1 -- 172.21.15.33:0/796208154 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5980056a0 con 0x562bff48cea0 2024-05-08T21:23:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.353+0000 7fa59cda4700 1 -- 172.21.15.33:0/796208154 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa59801e040 con 0x562bff48cea0 2024-05-08T21:23:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.353+0000 7fa59cda4700 1 -- 172.21.15.33:0/796208154 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59800fa60 con 0x562bff48cea0 2024-05-08T21:23:54.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.354+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/796208154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48cea0 msgr2=0x562bff48d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.354+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/796208154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48cea0 0x562bff48d320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5980099c0 tx=0x7fa59801f000 comp rx=0 tx=0).stop 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.354+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/796208154 shutdown_connections 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.354+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/796208154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48d860 0x562bff48dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.354+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/796208154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48c540 0x562bff48c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.354+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/796208154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48cea0 0x562bff48d320 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.354+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/796208154 >> 172.21.15.33:0/796208154 conn(0x562bff3ea210 msgr2=0x562bff486de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.354+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/796208154 shutdown_connections 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.354+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/796208154 wait complete. 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.354+0000 7fa5bc45bb80 1 -- start start 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.354+0000 7fa5bc45bb80 1 Processor -- start 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5bc45bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48c540 0x562bff495da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5bc45bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48cea0 0x562bff4962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5bc45bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48d860 0x562bff49d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5bc45bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bff378560 con 0x562bff48c540 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5bc45bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bff378200 con 0x562bff48cea0 2024-05-08T21:23:54.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5bc45bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bff25f6a0 con 0x562bff48d860 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5bc45bb80 10 client.0 _ll_register_callbacks cb 0x562bff381b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ae03d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48c540 0x562bff495da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ad83c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48cea0 0x562bff4962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ae03d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48c540 0x562bff495da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36908/0 (socket says 172.21.15.33:36908) 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ae03d700 1 -- 172.21.15.33:0/2585249704 learned_addr learned my addr 172.21.15.33:0/2585249704 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ae83e700 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48d860 0x562bff49d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ad83c700 1 -- 172.21.15.33:0/2585249704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48d860 msgr2=0x562bff49d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48d860 0x562bff49d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ad83c700 1 -- 172.21.15.33:0/2585249704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48c540 msgr2=0x562bff495da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48c540 0x562bff495da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ad83c700 1 -- 172.21.15.33:0/2585249704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a0003a60 con 0x562bff48cea0 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ae83e700 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48d860 0x562bff49d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48c540 0x562bff495da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.355+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48cea0 0x562bff4962e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa598009990 tx=0x7fa598009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=578b2c7f06013a1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:54.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.356+0000 7fa591ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.356+0000 7fa591ffb700 1 -- 172.21.15.33:0/2585249704 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598035070 con 0x562bff48cea0 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.356+0000 7fa591ffb700 1 -- 172.21.15.33:0/2585249704 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa59800fd80 con 0x562bff48cea0 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.356+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2585249704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bff335270 con 0x562bff48cea0 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.356+0000 7fa591ffb700 1 -- 172.21.15.33:0/2585249704 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa598003760 con 0x562bff48cea0 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.356+0000 7fa5bc45bb80 10 client.26714 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.356+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2585249704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fa5a810bdf0 con 0x562bff48cea0 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.356+0000 7fa591ffb700 1 -- 172.21.15.33:0/2585249704 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562bff335270 con 0x562bff48cea0 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.356+0000 7fa591ffb700 1 -- 172.21.15.33:0/2585249704 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fa598024970 con 0x562bff48cea0 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.357+0000 7fa5bc45bb80 10 client.26714 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.357+0000 7fa5bc45bb80 10 client.26714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.357+0000 7fa5bc45bb80 20 client.26714 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.357+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562bff4a1740 0x562bff4a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.357+0000 7fa5bc45bb80 4 client.26714 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.357+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2585249704 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bff45b810 con 0x562bff4a1740 2024-05-08T21:23:54.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.357+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562bff4a1740 0x562bff4a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562bff4a1740 0x562bff4a3b30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17a017c5c0a38ca server_cookie=9019aa9549b82f12 in_seq=0 out_seq=0 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa591ffb700 10 client.26714 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa591ffb700 1 -- 172.21.15.33:0/2585249704 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562bff45b810 con 0x562bff4a1740 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa591ffb700 10 client.26714 handle_command_reply: tid=0 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa5bc45bb80 1 client.26714 shutdown 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2585249704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48cea0 msgr2=0x562bff4962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48cea0 0x562bff4962e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa598009990 tx=0x7fa598009310 comp rx=0 tx=0).stop 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2585249704 shutdown_connections 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562bff4a1740 0x562bff4a3b30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48d860 0x562bff49d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48c540 0x562bff495da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/2585249704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48cea0 0x562bff4962e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa591ffb700 0 client.26714 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.358+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2585249704 >> 172.21.15.33:0/2585249704 conn(0x562bff3ea210 msgr2=0x562bff4865a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.359+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2585249704 shutdown_connections 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.359+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2585249704 wait complete. 2024-05-08T21:23:54.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.359+0000 7fa5bc45bb80 20 client.26714 trim_cache size 0 max 16384 2024-05-08T21:23:54.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.378+0000 7fa5bc45bb80 1 Processor -- start 2024-05-08T21:23:54.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.378+0000 7fa5bc45bb80 1 -- start start 2024-05-08T21:23:54.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5bc45bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 0x562bff495d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5bc45bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48cea0 0x562bff49d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5bc45bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48d860 0x562bff4962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5bc45bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bff496ca0 con 0x562bff48d860 2024-05-08T21:23:54.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5bc45bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bff495910 con 0x562bff48c540 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5bc45bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bff488570 con 0x562bff48cea0 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5ae83e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48d860 0x562bff4962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5ae03d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 0x562bff495d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5ae03d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 0x562bff495d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57540/0 (socket says 172.21.15.33:57540) 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5ae03d700 1 -- 172.21.15.33:0/2649727973 learned_addr learned my addr 172.21.15.33:0/2649727973 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5ae83e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48d860 0x562bff4962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36920/0 (socket says 172.21.15.33:36920) 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/2649727973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48cea0 0x562bff49d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5ae03d700 1 -- 172.21.15.33:0/2649727973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48cea0 msgr2=0x562bff49d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/2649727973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48cea0 0x562bff49d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5ae03d700 1 -- 172.21.15.33:0/2649727973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48d860 msgr2=0x562bff4962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/2649727973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48d860 0x562bff4962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5ae03d700 1 -- 172.21.15.33:0/2649727973 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bff209b60 con 0x562bff48c540 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.379+0000 7fa5ae83e700 1 --2- 172.21.15.33:0/2649727973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48d860 0x562bff4962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.380+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/2649727973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 0x562bff495d60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4096970 tx=0x7fa5a401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d8336154ca7623 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:54.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.380+0000 7fa59d5a5700 1 -- 172.21.15.33:0/2649727973 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a409a600 con 0x562bff48c540 2024-05-08T21:23:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.380+0000 7fa59d5a5700 1 -- 172.21.15.33:0/2649727973 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa5a4099f00 con 0x562bff48c540 2024-05-08T21:23:54.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.380+0000 7fa59d5a5700 1 -- 172.21.15.33:0/2649727973 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40982a0 con 0x562bff48c540 2024-05-08T21:23:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.381+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2649727973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 msgr2=0x562bff495d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.381+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/2649727973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 0x562bff495d60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4096970 tx=0x7fa5a401a040 comp rx=0 tx=0).stop 2024-05-08T21:23:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.381+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2649727973 shutdown_connections 2024-05-08T21:23:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.381+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/2649727973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48d860 0x562bff4962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.381+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/2649727973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 0x562bff495d60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.381+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/2649727973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48cea0 0x562bff49d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.381+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2649727973 >> 172.21.15.33:0/2649727973 conn(0x562bff3ea210 msgr2=0x562bff4907d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.381+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2649727973 shutdown_connections 2024-05-08T21:23:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.381+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/2649727973 wait complete. 2024-05-08T21:23:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.381+0000 7fa5bc45bb80 1 -- start start 2024-05-08T21:23:54.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.381+0000 7fa5bc45bb80 1 Processor -- start 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5bc45bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 0x562bff4986f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5bc45bb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48cea0 0x562bff498c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5bc45bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48d860 0x562bff49bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5bc45bb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bff49c190 con 0x562bff48cea0 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5bc45bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bff49c300 con 0x562bff48c540 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5bc45bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bff49c4c0 con 0x562bff48d860 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5bc45bb80 10 client.0 _ll_register_callbacks cb 0x562bff381b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5ae03d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 0x562bff4986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5ae03d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 0x562bff4986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57546/0 (socket says 172.21.15.33:57546) 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5ae03d700 1 -- 172.21.15.33:0/502995167 learned_addr learned my addr 172.21.15.33:0/502995167 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48cea0 0x562bff498c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5ae83e700 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48d860 0x562bff49bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5ae03d700 1 -- 172.21.15.33:0/502995167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48d860 msgr2=0x562bff49bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48d860 0x562bff49bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5ae03d700 1 -- 172.21.15.33:0/502995167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48cea0 msgr2=0x562bff498c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:54.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48cea0 0x562bff498c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5ae03d700 1 -- 172.21.15.33:0/502995167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a0004350 con 0x562bff48c540 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa5ae03d700 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 0x562bff4986f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4096400 tx=0x7fa5a40a23f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff5251d0edae367 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa591ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.382+0000 7fa591ffb700 1 -- 172.21.15.33:0/502995167 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40b82c0 con 0x562bff48c540 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa591ffb700 1 -- 172.21.15.33:0/502995167 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa5a40a1ef0 con 0x562bff48c540 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/502995167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa59801e040 con 0x562bff48c540 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa591ffb700 1 -- 172.21.15.33:0/502995167 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5a40a20b0 con 0x562bff48c540 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa5bc45bb80 10 client.26720 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/502995167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fa5a81a1400 con 0x562bff48c540 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa591ffb700 1 -- 172.21.15.33:0/502995167 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa59801e040 con 0x562bff48c540 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa591ffb700 1 -- 172.21.15.33:0/502995167 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fa5a4095cf0 con 0x562bff48c540 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa5bc45bb80 10 client.26720 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa5bc45bb80 10 client.26720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa5bc45bb80 20 client.26720 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562bff4f7fa0 0x562bff4fa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa5bc45bb80 4 client.26720 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:23:54.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/502995167 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562bff3d4ec0 con 0x562bff4f7fa0 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.383+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562bff4f7fa0 0x562bff4fa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa591ffb700 10 client.26720 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5ad83c700 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562bff4f7fa0 0x562bff4fa3c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8683292effff4a1b server_cookie=9e0a9bb91cc35dc7 in_seq=0 out_seq=0 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa591ffb700 1 -- 172.21.15.33:0/502995167 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562bff3d4ec0 con 0x562bff4f7fa0 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa591ffb700 10 client.26720 handle_command_reply: tid=0 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5bc45bb80 1 client.26720 shutdown 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/502995167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 msgr2=0x562bff4986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 0x562bff4986f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4096400 tx=0x7fa5a40a23f0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/502995167 shutdown_connections 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562bff4f7fa0 0x562bff4fa3c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bff48d860 0x562bff49bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562bff48c540 0x562bff4986f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa591ffb700 0 client.26720 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5bc45bb80 1 --2- 172.21.15.33:0/502995167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562bff48cea0 0x562bff498c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/502995167 >> 172.21.15.33:0/502995167 conn(0x562bff3ea210 msgr2=0x562bff495d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:54.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/502995167 shutdown_connections 2024-05-08T21:23:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5bc45bb80 1 -- 172.21.15.33:0/502995167 wait complete. 2024-05-08T21:23:54.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.384+0000 7fa5bc45bb80 20 client.26720 trim_cache size 0 max 16384 2024-05-08T21:23:54.391 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:23:54.391 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:23:54.391 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:23:54.391 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:23:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.385+0000 7fa5b02a1700 1 -- 172.21.15.33:0/973080934 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa5940a1120 msgr2=0x7fa5940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.385+0000 7fa5b02a1700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa5940a1120 0x7fa5940a35e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa598000c00 tx=0x7fa59801a040 comp rx=0 tx=0).stop 2024-05-08T21:23:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.385+0000 7fa5b02a1700 1 -- 172.21.15.33:0/973080934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5a80ff750 msgr2=0x7fa5a8198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.385+0000 7fa5b02a1700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5a80ff750 0x7fa5a8198f00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400e9a0 tx=0x7fa5a400ecb0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.385+0000 7fa5b02a1700 1 -- 172.21.15.33:0/973080934 shutdown_connections 2024-05-08T21:23:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.385+0000 7fa5b02a1700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5a8100a70 0x7fa5a81a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.385+0000 7fa5b02a1700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa5a80ff750 0x7fa5a8198f00 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.385+0000 7fa5b02a1700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa5940a1120 0x7fa5940a35e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.385+0000 7fa5b02a1700 1 --2- 172.21.15.33:0/973080934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a81000b0 0x7fa5a8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.385+0000 7fa5b02a1700 1 -- 172.21.15.33:0/973080934 >> 172.21.15.33:0/973080934 conn(0x7fa5a80fb310 msgr2=0x7fa5a80fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.385+0000 7fa5b02a1700 1 -- 172.21.15.33:0/973080934 shutdown_connections 2024-05-08T21:23:54.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.385+0000 7fa5b02a1700 1 -- 172.21.15.33:0/973080934 wait complete. 2024-05-08T21:23:54.401 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:23:54.401 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:23:54.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.548+0000 7ffa6377d700 1 -- 172.21.15.33:0/3771630797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c0ffd80 msgr2=0x7ffa5c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.548+0000 7ffa6377d700 1 --2- 172.21.15.33:0/3771630797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c0ffd80 0x7ffa5c1001a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffa580098f0 tx=0x7ffa58009c00 comp rx=0 tx=0).stop 2024-05-08T21:23:54.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.548+0000 7ffa6377d700 1 -- 172.21.15.33:0/3771630797 shutdown_connections 2024-05-08T21:23:54.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.548+0000 7ffa6377d700 1 --2- 172.21.15.33:0/3771630797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c101910 0x7ffa5c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.548+0000 7ffa6377d700 1 --2- 172.21.15.33:0/3771630797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c0ffd80 0x7ffa5c1001a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.548+0000 7ffa6377d700 1 --2- 172.21.15.33:0/3771630797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa5c100f70 0x7ffa5c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.548+0000 7ffa6377d700 1 -- 172.21.15.33:0/3771630797 >> 172.21.15.33:0/3771630797 conn(0x7ffa5c0fb2f0 msgr2=0x7ffa5c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:54.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.548+0000 7ffa6377d700 1 -- 172.21.15.33:0/3771630797 shutdown_connections 2024-05-08T21:23:54.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.548+0000 7ffa6377d700 1 -- 172.21.15.33:0/3771630797 wait complete. 2024-05-08T21:23:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.549+0000 7ffa6377d700 1 Processor -- start 2024-05-08T21:23:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.549+0000 7ffa6377d700 1 -- start start 2024-05-08T21:23:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa6377d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c0ffd80 0x7ffa5c195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa6377d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa5c100f70 0x7ffa5c196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa6377d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c101910 0x7ffa5c19d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa6377d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c19d7b0 con 0x7ffa5c0ffd80 2024-05-08T21:23:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa6377d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c19d920 con 0x7ffa5c100f70 2024-05-08T21:23:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa6377d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c19dae0 con 0x7ffa5c101910 2024-05-08T21:23:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa61519700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c0ffd80 0x7ffa5c195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa61d1a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c101910 0x7ffa5c19d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa60d18700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa5c100f70 0x7ffa5c196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa61d1a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c101910 0x7ffa5c19d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43356/0 (socket says 172.21.15.33:43356) 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa61d1a700 1 -- 172.21.15.33:0/1026947746 learned_addr learned my addr 172.21.15.33:0/1026947746 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa61519700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c0ffd80 0x7ffa5c195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36952/0 (socket says 172.21.15.33:36952) 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa61d1a700 1 -- 172.21.15.33:0/1026947746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa5c100f70 msgr2=0x7ffa5c196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa61d1a700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa5c100f70 0x7ffa5c196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa61d1a700 1 -- 172.21.15.33:0/1026947746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c0ffd80 msgr2=0x7ffa5c195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa61d1a700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c0ffd80 0x7ffa5c195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa61d1a700 1 -- 172.21.15.33:0/1026947746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa58009520 con 0x7ffa5c101910 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa60d18700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa5c100f70 0x7ffa5c196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.550+0000 7ffa61d1a700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c101910 0x7ffa5c19d1e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400ef30 tx=0x7ffa5400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.551+0000 7ffa527fc700 1 -- 172.21.15.33:0/1026947746 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54018070 con 0x7ffa5c101910 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.551+0000 7ffa527fc700 1 -- 172.21.15.33:0/1026947746 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffa5400f040 con 0x7ffa5c101910 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.551+0000 7ffa527fc700 1 -- 172.21.15.33:0/1026947746 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400cbb0 con 0x7ffa5c101910 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.551+0000 7ffa6377d700 1 -- 172.21.15.33:0/1026947746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c19dd60 con 0x7ffa5c101910 2024-05-08T21:23:54.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.551+0000 7ffa6377d700 1 -- 172.21.15.33:0/1026947746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c19e190 con 0x7ffa5c101910 2024-05-08T21:23:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.552+0000 7ffa6377d700 1 -- 172.21.15.33:0/1026947746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5c107390 con 0x7ffa5c101910 2024-05-08T21:23:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.553+0000 7ffa527fc700 1 -- 172.21.15.33:0/1026947746 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ffa5400cd90 con 0x7ffa5c101910 2024-05-08T21:23:54.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.554+0000 7ffa527fc700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ffa480a11a0 0x7ffa480a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.554+0000 7ffa527fc700 1 -- 172.21.15.33:0/1026947746 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ffa540cc790 con 0x7ffa5c101910 2024-05-08T21:23:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.554+0000 7ffa61519700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ffa480a11a0 0x7ffa480a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.554+0000 7ffa61519700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ffa480a11a0 0x7ffa480a3660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009fd0 tx=0x7ffa58019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:54.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.558+0000 7ffa527fc700 1 -- 172.21.15.33:0/1026947746 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ffa54095bf0 con 0x7ffa5c101910 2024-05-08T21:23:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.858+0000 7ffa6377d700 1 -- 172.21.15.33:0/1026947746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa5c067480 con 0x7ffa5c101910 2024-05-08T21:23:54.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.859+0000 7ffa527fc700 1 -- 172.21.15.33:0/1026947746 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10914 (secure 0 0 0) 0x7ffa54099210 con 0x7ffa5c101910 2024-05-08T21:23:54.861 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:23:54.861 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:22:25.305999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:23:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.862+0000 7ffa6377d700 1 -- 172.21.15.33:0/1026947746 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ffa480a11a0 msgr2=0x7ffa480a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.862+0000 7ffa6377d700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ffa480a11a0 0x7ffa480a3660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009fd0 tx=0x7ffa58019040 comp rx=0 tx=0).stop 2024-05-08T21:23:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.862+0000 7ffa6377d700 1 -- 172.21.15.33:0/1026947746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c101910 msgr2=0x7ffa5c19d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.862+0000 7ffa6377d700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c101910 0x7ffa5c19d1e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400ef30 tx=0x7ffa5400c3a0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.862+0000 7ffa6377d700 1 -- 172.21.15.33:0/1026947746 shutdown_connections 2024-05-08T21:23:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.862+0000 7ffa6377d700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c101910 0x7ffa5c19d1e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.862+0000 7ffa6377d700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa5c0ffd80 0x7ffa5c195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.862+0000 7ffa6377d700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ffa480a11a0 0x7ffa480a3660 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.862+0000 7ffa6377d700 1 --2- 172.21.15.33:0/1026947746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa5c100f70 0x7ffa5c196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.862+0000 7ffa6377d700 1 -- 172.21.15.33:0/1026947746 >> 172.21.15.33:0/1026947746 conn(0x7ffa5c0fb2f0 msgr2=0x7ffa5c1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.862+0000 7ffa6377d700 1 -- 172.21.15.33:0/1026947746 shutdown_connections 2024-05-08T21:23:54.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:54.862+0000 7ffa6377d700 1 -- 172.21.15.33:0/1026947746 wait complete. 2024-05-08T21:23:54.865 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 34 2024-05-08T21:23:54.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:23:55.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.026+0000 7fdf730b3700 1 -- 172.21.15.33:0/2010014011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6c106b40 msgr2=0x7fdf6c108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.026+0000 7fdf730b3700 1 --2- 172.21.15.33:0/2010014011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6c106b40 0x7fdf6c108fb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400b0a0 tx=0x7fdf6400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.026+0000 7fdf730b3700 1 -- 172.21.15.33:0/2010014011 shutdown_connections 2024-05-08T21:23:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.026+0000 7fdf730b3700 1 --2- 172.21.15.33:0/2010014011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6c106b40 0x7fdf6c108fb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.026+0000 7fdf730b3700 1 --2- 172.21.15.33:0/2010014011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c1018e0 0x7fdf6c103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.026+0000 7fdf730b3700 1 --2- 172.21.15.33:0/2010014011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf6c104210 0x7fdf6c106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.026+0000 7fdf730b3700 1 -- 172.21.15.33:0/2010014011 >> 172.21.15.33:0/2010014011 conn(0x7fdf6c0fb290 msgr2=0x7fdf6c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.026+0000 7fdf730b3700 1 -- 172.21.15.33:0/2010014011 shutdown_connections 2024-05-08T21:23:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.026+0000 7fdf730b3700 1 -- 172.21.15.33:0/2010014011 wait complete. 2024-05-08T21:23:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.026+0000 7fdf730b3700 1 Processor -- start 2024-05-08T21:23:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.026+0000 7fdf730b3700 1 -- start start 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf730b3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c1018e0 0x7fdf6c101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf730b3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6c104210 0x7fdf6c0ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf730b3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf6c106b40 0x7fdf6c0ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf730b3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c100570 con 0x7fdf6c106b40 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf730b3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c1006e0 con 0x7fdf6c104210 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf730b3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c1a0e30 con 0x7fdf6c1018e0 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf70e4f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c1018e0 0x7fdf6c101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf6bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6c104210 0x7fdf6c0ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf71650700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf6c106b40 0x7fdf6c0ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf70e4f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c1018e0 0x7fdf6c101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43380/0 (socket says 172.21.15.33:43380) 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf70e4f700 1 -- 172.21.15.33:0/2429598784 learned_addr learned my addr 172.21.15.33:0/2429598784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf70e4f700 1 -- 172.21.15.33:0/2429598784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6c104210 msgr2=0x7fdf6c0ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6c104210 0x7fdf6c0ff890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf70e4f700 1 -- 172.21.15.33:0/2429598784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf6c106b40 msgr2=0x7fdf6c0ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf6c106b40 0x7fdf6c0ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf70e4f700 1 -- 172.21.15.33:0/2429598784 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf58009520 con 0x7fdf6c1018e0 2024-05-08T21:23:55.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf6bfff700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6c104210 0x7fdf6c0ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf71650700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf6c106b40 0x7fdf6c0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.027+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c1018e0 0x7fdf6c101200 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000e9a0 tx=0x7fdf6000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.028+0000 7fdf69ffb700 1 -- 172.21.15.33:0/2429598784 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60009790 con 0x7fdf6c1018e0 2024-05-08T21:23:55.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.028+0000 7fdf69ffb700 1 -- 172.21.15.33:0/2429598784 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf60010430 con 0x7fdf6c1018e0 2024-05-08T21:23:55.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.028+0000 7fdf730b3700 1 -- 172.21.15.33:0/2429598784 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf64009d00 con 0x7fdf6c1018e0 2024-05-08T21:23:55.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.028+0000 7fdf69ffb700 1 -- 172.21.15.33:0/2429598784 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000cea0 con 0x7fdf6c1018e0 2024-05-08T21:23:55.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.028+0000 7fdf730b3700 1 -- 172.21.15.33:0/2429598784 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6c1a13d0 con 0x7fdf6c1018e0 2024-05-08T21:23:55.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.030+0000 7fdf69ffb700 1 -- 172.21.15.33:0/2429598784 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdf600105a0 con 0x7fdf6c1018e0 2024-05-08T21:23:55.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.031+0000 7fdf69ffb700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdf5c0a11a0 0x7fdf5c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.031+0000 7fdf69ffb700 1 -- 172.21.15.33:0/2429598784 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fdf60014070 con 0x7fdf6c1018e0 2024-05-08T21:23:55.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.031+0000 7fdf6bfff700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdf5c0a11a0 0x7fdf5c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.032+0000 7fdf6bfff700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdf5c0a11a0 0x7fdf5c0a3660 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009fd0 tx=0x7fdf58009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.032+0000 7fdf7f26db80 1 Processor -- start 2024-05-08T21:23:55.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.032+0000 7fdf7f26db80 1 -- start start 2024-05-08T21:23:55.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf7f26db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6540 0x55e61c8e6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf7f26db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e6ea0 0x55e61c8e7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf7f26db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e7860 0x55e61c8e7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf7f26db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61c83d150 con 0x55e61c8e6540 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf7f26db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61c83ab10 con 0x55e61c8e6ea0 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf7f26db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61c836400 con 0x55e61c8e7860 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf70e4f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6540 0x55e61c8e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf6bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e6ea0 0x55e61c8e7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf70e4f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6540 0x55e61c8e6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36978/0 (socket says 172.21.15.33:36978) 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1009064170 learned_addr learned my addr 172.21.15.33:0/1009064170 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf71650700 1 --2- 172.21.15.33:0/1009064170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e7860 0x55e61c8e7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1009064170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e7860 msgr2=0x55e61c8e7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/1009064170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e7860 0x55e61c8e7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1009064170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e6ea0 msgr2=0x55e61c8e7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/1009064170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e6ea0 0x55e61c8e7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1009064170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e61c6b73b0 con 0x55e61c8e6540 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf6bfff700 1 --2- 172.21.15.33:0/1009064170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e6ea0 0x55e61c8e7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/1009064170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6540 0x55e61c8e6960 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdf600963f0 tx=0x7fdf60099920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246a0c2ccb0d44 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf56ffd700 1 -- 172.21.15.33:0/1009064170 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60099ef0 con 0x55e61c8e6540 2024-05-08T21:23:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.033+0000 7fdf56ffd700 1 -- 172.21.15.33:0/1009064170 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf6009dd20 con 0x55e61c8e6540 2024-05-08T21:23:55.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.034+0000 7fdf56ffd700 1 -- 172.21.15.33:0/1009064170 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf600a1ee0 con 0x55e61c8e6540 2024-05-08T21:23:55.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.034+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1009064170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6540 msgr2=0x55e61c8e6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.034+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1009064170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6540 0x55e61c8e6960 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdf600963f0 tx=0x7fdf60099920 comp rx=0 tx=0).stop 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.034+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1009064170 shutdown_connections 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.034+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1009064170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e7860 0x55e61c8e7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.034+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1009064170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6540 0x55e61c8e6960 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.034+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1009064170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e6ea0 0x55e61c8e7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.034+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1009064170 >> 172.21.15.33:0/1009064170 conn(0x55e61c844210 msgr2=0x55e61c8e0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.034+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1009064170 shutdown_connections 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.034+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1009064170 wait complete. 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.035+0000 7fdf7f26db80 1 -- start start 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.035+0000 7fdf7f26db80 1 Processor -- start 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.035+0000 7fdf7f26db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8efd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.035+0000 7fdf7f26db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.035+0000 7fdf7f26db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e7860 0x55e61c8f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.035+0000 7fdf7f26db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61c7d2560 con 0x55e61c8e7860 2024-05-08T21:23:55.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.035+0000 7fdf7f26db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61c7d2200 con 0x55e61c8e6ea0 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.035+0000 7fdf7f26db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61c6b96a0 con 0x55e61c8e6540 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf7f26db80 10 client.0 _ll_register_callbacks cb 0x55e61c7dbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf71650700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e7860 0x55e61c8f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf71650700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e7860 0x55e61c8f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36984/0 (socket says 172.21.15.33:36984) 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf70e4f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf71650700 1 -- 172.21.15.33:0/1095833364 learned_addr learned my addr 172.21.15.33:0/1095833364 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf6bfff700 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1095833364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e6ea0 msgr2=0x55e61c8f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1095833364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e7860 msgr2=0x55e61c8f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e7860 0x55e61c8f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1095833364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf58017050 con 0x55e61c8e6540 2024-05-08T21:23:55.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf6bfff700 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8efd40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c2d0 tx=0x7fdf6009a720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc22e1483b610d39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf4bfff700 1 -- 172.21.15.33:0/1095833364 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf600b6070 con 0x55e61c8e6540 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf4bfff700 1 -- 172.21.15.33:0/1095833364 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fdf60098290 con 0x55e61c8e6540 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.036+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1095833364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e61c78f270 con 0x55e61c8e6540 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.037+0000 7fdf4bfff700 1 -- 172.21.15.33:0/1095833364 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf600b82b0 con 0x55e61c8e6540 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.037+0000 7fdf7f26db80 10 client.26755 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.037+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1095833364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fdf6c1a13d0 con 0x55e61c8e6540 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.037+0000 7fdf4bfff700 1 -- 172.21.15.33:0/1095833364 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf600a8080 con 0x55e61c8e6540 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.037+0000 7fdf4bfff700 1 -- 172.21.15.33:0/1095833364 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fdf60095ce0 con 0x55e61c8e6540 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.037+0000 7fdf7f26db80 10 client.26755 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.037+0000 7fdf7f26db80 10 client.26755 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.037+0000 7fdf7f26db80 20 client.26755 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:55.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.037+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e61c8fb7c0 0x55e61c8fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.037+0000 7fdf7f26db80 4 client.26755 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.037+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1095833364 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e61c8b5810 con 0x55e61c8fb7c0 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.037+0000 7fdf6bfff700 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e61c8fb7c0 0x55e61c8fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.038+0000 7fdf6bfff700 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e61c8fb7c0 0x55e61c8fdbe0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3505351b7daf2622 server_cookie=cbcf466cfaf7840e in_seq=0 out_seq=0 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.038+0000 7fdf4bfff700 10 client.26755 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.038+0000 7fdf4bfff700 1 -- 172.21.15.33:0/1095833364 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e61c8b5810 con 0x55e61c8fb7c0 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.038+0000 7fdf4bfff700 10 client.26755 handle_command_reply: tid=0 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.038+0000 7fdf7f26db80 1 client.26755 shutdown 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.039+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1095833364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 msgr2=0x55e61c8efd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.039+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8efd40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c2d0 tx=0x7fdf6009a720 comp rx=0 tx=0).stop 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.039+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1095833364 shutdown_connections 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.039+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e7860 0x55e61c8f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.039+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e61c8fb7c0 0x55e61c8fdbe0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.039+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8efd40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.039+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1095833364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.039+0000 7fdf4bfff700 0 client.26755 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:23:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.039+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1095833364 >> 172.21.15.33:0/1095833364 conn(0x55e61c844210 msgr2=0x55e61c8e05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.039+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1095833364 shutdown_connections 2024-05-08T21:23:55.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.039+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1095833364 wait complete. 2024-05-08T21:23:55.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.039+0000 7fdf7f26db80 20 client.26755 trim_cache size 0 max 16384 2024-05-08T21:23:55.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf7f26db80 1 Processor -- start 2024-05-08T21:23:55.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf7f26db80 1 -- start start 2024-05-08T21:23:55.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf7f26db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf7f26db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf7f26db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e7860 0x55e61c8edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf7f26db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61c8fb4f0 con 0x55e61c8e6ea0 2024-05-08T21:23:55.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf7f26db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61c8d9040 con 0x55e61c8e7860 2024-05-08T21:23:55.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf7f26db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61c8eb020 con 0x55e61c8e6540 2024-05-08T21:23:55.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf70e4f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36988/0 (socket says 172.21.15.33:36988) 2024-05-08T21:23:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf6bfff700 1 -- 172.21.15.33:0/1119341294 learned_addr learned my addr 172.21.15.33:0/1119341294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.055+0000 7fdf71650700 1 --2- 172.21.15.33:0/1119341294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e7860 0x55e61c8edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.056+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1119341294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e7860 msgr2=0x55e61c8edd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.056+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/1119341294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e7860 0x55e61c8edd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.056+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1119341294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6ea0 msgr2=0x55e61c8f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.056+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/1119341294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.056+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1119341294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e61c663b60 con 0x55e61c8e6540 2024-05-08T21:23:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.056+0000 7fdf6bfff700 1 --2- 172.21.15.33:0/1119341294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.056+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/1119341294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8fdbe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf60099bf0 tx=0x7fdf60099920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=762592cde0fd9afb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.056+0000 7fdf577fe700 1 -- 172.21.15.33:0/1119341294 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf600993e0 con 0x55e61c8e6540 2024-05-08T21:23:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.056+0000 7fdf577fe700 1 -- 172.21.15.33:0/1119341294 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e61c78f270 con 0x55e61c8e6540 2024-05-08T21:23:55.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.056+0000 7fdf577fe700 1 -- 172.21.15.33:0/1119341294 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60099ef0 con 0x55e61c8e6540 2024-05-08T21:23:55.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.057+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1119341294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 msgr2=0x55e61c8fdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.057+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1119341294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8fdbe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdf60099bf0 tx=0x7fdf60099920 comp rx=0 tx=0).stop 2024-05-08T21:23:55.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.057+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1119341294 shutdown_connections 2024-05-08T21:23:55.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.057+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1119341294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e7860 0x55e61c8edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.057+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1119341294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8fdbe0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.057+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1119341294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.057+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1119341294 >> 172.21.15.33:0/1119341294 conn(0x55e61c844210 msgr2=0x55e61c8e7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.057+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1119341294 shutdown_connections 2024-05-08T21:23:55.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.057+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1119341294 wait complete. 2024-05-08T21:23:55.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.058+0000 7fdf7f26db80 1 -- start start 2024-05-08T21:23:55.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.058+0000 7fdf7f26db80 1 Processor -- start 2024-05-08T21:23:55.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.058+0000 7fdf7f26db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8f2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.058+0000 7fdf7f26db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.058+0000 7fdf7f26db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e7860 0x55e61c8f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.058+0000 7fdf7f26db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61c8f6090 con 0x55e61c8e6ea0 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.058+0000 7fdf7f26db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61c8f6200 con 0x55e61c8e7860 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.058+0000 7fdf7f26db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61c8f63c0 con 0x55e61c8e6540 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.058+0000 7fdf7f26db80 10 client.0 _ll_register_callbacks cb 0x55e61c7dbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.058+0000 7fdf70e4f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf70e4f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8f2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43400/0 (socket says 172.21.15.33:43400) 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1166131280 learned_addr learned my addr 172.21.15.33:0/1166131280 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf71650700 1 --2- 172.21.15.33:0/1166131280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e7860 0x55e61c8f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1166131280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e7860 msgr2=0x55e61c8f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/1166131280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e7860 0x55e61c8f5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1166131280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6ea0 msgr2=0x55e61c8f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/1166131280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf70e4f700 1 -- 172.21.15.33:0/1166131280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5800e8e0 con 0x55e61c8e6540 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf70e4f700 1 --2- 172.21.15.33:0/1166131280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8f2640 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf60099870 tx=0x7fdf60095f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6521cbb20702a76f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf4bfff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf4bfff700 1 -- 172.21.15.33:0/1166131280 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf600b6060 con 0x55e61c8e6540 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf4bfff700 1 -- 172.21.15.33:0/1166131280 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fdf60099ef0 con 0x55e61c8e6540 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1166131280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf6009dd20 con 0x55e61c8e6540 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf4bfff700 1 -- 172.21.15.33:0/1166131280 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf600993e0 con 0x55e61c8e6540 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf7f26db80 10 client.26761 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.059+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1166131280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fdf6c1a13d0 con 0x55e61c8e6540 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.060+0000 7fdf4bfff700 1 -- 172.21.15.33:0/1166131280 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf600a8030 con 0x55e61c8e6540 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.060+0000 7fdf4bfff700 1 -- 172.21.15.33:0/1166131280 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fdf600b2c60 con 0x55e61c8e6540 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.060+0000 7fdf7f26db80 10 client.26761 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:55.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.060+0000 7fdf7f26db80 10 client.26761 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.060+0000 7fdf7f26db80 20 client.26761 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.060+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1166131280 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e61c951ec0 0x55e61c9542e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.060+0000 7fdf7f26db80 4 client.26761 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.060+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1166131280 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e61c82eec0 con 0x55e61c951ec0 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.060+0000 7fdf6bfff700 1 --2- 172.21.15.33:0/1166131280 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e61c951ec0 0x55e61c9542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf4bfff700 10 client.26761 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf6bfff700 1 --2- 172.21.15.33:0/1166131280 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e61c951ec0 0x55e61c9542e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20fe6a0839cd6374 server_cookie=fb8538ffe0b14e22 in_seq=0 out_seq=0 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf4bfff700 1 -- 172.21.15.33:0/1166131280 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e61c82eec0 con 0x55e61c951ec0 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf4bfff700 10 client.26761 handle_command_reply: tid=0 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf7f26db80 1 client.26761 shutdown 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1166131280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 msgr2=0x55e61c8f2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1166131280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8f2640 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdf60099870 tx=0x7fdf60095f10 comp rx=0 tx=0).stop 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1166131280 shutdown_connections 2024-05-08T21:23:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1166131280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61c8e7860 0x55e61c8f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1166131280 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e61c951ec0 0x55e61c9542e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1166131280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61c8e6540 0x55e61c8f2640 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf7f26db80 1 --2- 172.21.15.33:0/1166131280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e61c8e6ea0 0x55e61c8f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1166131280 >> 172.21.15.33:0/1166131280 conn(0x55e61c844210 msgr2=0x55e61c8ef250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf4bfff700 0 client.26761 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:23:55.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1166131280 shutdown_connections 2024-05-08T21:23:55.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.061+0000 7fdf7f26db80 1 -- 172.21.15.33:0/1166131280 wait complete. 2024-05-08T21:23:55.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.062+0000 7fdf7f26db80 20 client.26761 trim_cache size 0 max 16384 2024-05-08T21:23:55.068 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:23:55.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.062+0000 7fdf730b3700 1 -- 172.21.15.33:0/2429598784 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdf5c0a11a0 msgr2=0x7fdf5c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.062+0000 7fdf730b3700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdf5c0a11a0 0x7fdf5c0a3660 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009fd0 tx=0x7fdf58009260 comp rx=0 tx=0).stop 2024-05-08T21:23:55.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.062+0000 7fdf730b3700 1 -- 172.21.15.33:0/2429598784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c1018e0 msgr2=0x7fdf6c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.062+0000 7fdf730b3700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c1018e0 0x7fdf6c101200 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000e9a0 tx=0x7fdf6000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.062+0000 7fdf730b3700 1 -- 172.21.15.33:0/2429598784 shutdown_connections 2024-05-08T21:23:55.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.062+0000 7fdf730b3700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf6c106b40 0x7fdf6c0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.062+0000 7fdf730b3700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf6c1018e0 0x7fdf6c101200 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.062+0000 7fdf730b3700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdf5c0a11a0 0x7fdf5c0a3660 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.062+0000 7fdf730b3700 1 --2- 172.21.15.33:0/2429598784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf6c104210 0x7fdf6c0ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.062+0000 7fdf730b3700 1 -- 172.21.15.33:0/2429598784 >> 172.21.15.33:0/2429598784 conn(0x7fdf6c0fb290 msgr2=0x7fdf6c103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.063+0000 7fdf730b3700 1 -- 172.21.15.33:0/2429598784 shutdown_connections 2024-05-08T21:23:55.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.063+0000 7fdf730b3700 1 -- 172.21.15.33:0/2429598784 wait complete. 2024-05-08T21:23:55.078 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:23:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.228+0000 7f7d40a46700 1 -- 172.21.15.33:0/2030421026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d3c0739a0 msgr2=0x7f7d3c073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.228+0000 7f7d40a46700 1 --2- 172.21.15.33:0/2030421026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d3c0739a0 0x7f7d3c073e20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009990 tx=0x7f7d30009ca0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.228+0000 7f7d40a46700 1 -- 172.21.15.33:0/2030421026 shutdown_connections 2024-05-08T21:23:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.228+0000 7f7d40a46700 1 --2- 172.21.15.33:0/2030421026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c108d70 0x7f7d3c10b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.228+0000 7f7d40a46700 1 --2- 172.21.15.33:0/2030421026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d3c074f80 0x7f7d3c0733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.228+0000 7f7d40a46700 1 --2- 172.21.15.33:0/2030421026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d3c0739a0 0x7f7d3c073e20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.228+0000 7f7d40a46700 1 -- 172.21.15.33:0/2030421026 >> 172.21.15.33:0/2030421026 conn(0x7f7d3c0fc430 msgr2=0x7f7d3c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.228+0000 7f7d40a46700 1 -- 172.21.15.33:0/2030421026 shutdown_connections 2024-05-08T21:23:55.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.228+0000 7f7d40a46700 1 -- 172.21.15.33:0/2030421026 wait complete. 2024-05-08T21:23:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.229+0000 7f7d40a46700 1 Processor -- start 2024-05-08T21:23:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.229+0000 7f7d40a46700 1 -- start start 2024-05-08T21:23:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.229+0000 7f7d40a46700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d3c0739a0 0x7f7d3c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.229+0000 7f7d40a46700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d3c074f80 0x7f7d3c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.229+0000 7f7d40a46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c108d70 0x7f7d3c1a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.229+0000 7f7d40a46700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c1a0a50 con 0x7f7d3c108d70 2024-05-08T21:23:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.229+0000 7f7d40a46700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c1a0b90 con 0x7f7d3c0739a0 2024-05-08T21:23:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.229+0000 7f7d40a46700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c1a0d20 con 0x7f7d3c074f80 2024-05-08T21:23:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.229+0000 7f7d3a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d3c0739a0 0x7f7d3c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.229+0000 7f7d39d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d3c074f80 0x7f7d3c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.229+0000 7f7d3a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d3c0739a0 0x7f7d3c198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57654/0 (socket says 172.21.15.33:57654) 2024-05-08T21:23:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.229+0000 7f7d3a59c700 1 -- 172.21.15.33:0/2117948699 learned_addr learned my addr 172.21.15.33:0/2117948699 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.230+0000 7f7d3ad9d700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c108d70 0x7f7d3c1a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.230+0000 7f7d3a59c700 1 -- 172.21.15.33:0/2117948699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d3c074f80 msgr2=0x7f7d3c199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.230+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d3c074f80 0x7f7d3c199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.230+0000 7f7d3a59c700 1 -- 172.21.15.33:0/2117948699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c108d70 msgr2=0x7f7d3c1a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.230+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c108d70 0x7f7d3c1a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.230+0000 7f7d3a59c700 1 -- 172.21.15.33:0/2117948699 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d300095f0 con 0x7f7d3c0739a0 2024-05-08T21:23:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.230+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d3c0739a0 0x7f7d3c198f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e9a0 tx=0x7f7d2c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.230+0000 7f7d2b7fe700 1 -- 172.21.15.33:0/2117948699 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c009790 con 0x7f7d3c0739a0 2024-05-08T21:23:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.230+0000 7f7d2b7fe700 1 -- 172.21.15.33:0/2117948699 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d2c010430 con 0x7f7d3c0739a0 2024-05-08T21:23:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.230+0000 7f7d40a46700 1 -- 172.21.15.33:0/2117948699 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3c1a0fa0 con 0x7f7d3c0739a0 2024-05-08T21:23:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.230+0000 7f7d2b7fe700 1 -- 172.21.15.33:0/2117948699 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00cea0 con 0x7f7d3c0739a0 2024-05-08T21:23:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.230+0000 7f7d40a46700 1 -- 172.21.15.33:0/2117948699 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c1a1420 con 0x7f7d3c0739a0 2024-05-08T21:23:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.233+0000 7f7d2b7fe700 1 -- 172.21.15.33:0/2117948699 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7d2c018680 con 0x7f7d3c0739a0 2024-05-08T21:23:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.233+0000 7f7d2b7fe700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7d240a1120 0x7f7d240a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.234+0000 7f7d2b7fe700 1 -- 172.21.15.33:0/2117948699 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7d2c014070 con 0x7f7d3c0739a0 2024-05-08T21:23:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.234+0000 7f7d39d9b700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7d240a1120 0x7f7d240a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.234+0000 7f7d39d9b700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7d240a1120 0x7f7d240a35e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7d30006010 tx=0x7f7d3000b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.234+0000 7f7d4cc00b80 1 Processor -- start 2024-05-08T21:23:55.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.235+0000 7f7d4cc00b80 1 -- start start 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.235+0000 7f7d4cc00b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 0x561885796960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.235+0000 7f7d4cc00b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796ea0 0x561885797320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.235+0000 7f7d4cc00b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885797860 0x561885797ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.235+0000 7f7d4cc00b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618856ed150 con 0x561885796540 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.235+0000 7f7d4cc00b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618856eab10 con 0x561885796ea0 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.235+0000 7f7d4cc00b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618856e6400 con 0x561885797860 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.235+0000 7f7d3a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 0x561885796960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.235+0000 7f7d3a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 0x561885796960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37024/0 (socket says 172.21.15.33:37024) 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.235+0000 7f7d3a59c700 1 -- 172.21.15.33:0/3546607864 learned_addr learned my addr 172.21.15.33:0/3546607864 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.235+0000 7f7d3ad9d700 1 --2- 172.21.15.33:0/3546607864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885797860 0x561885797ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.235+0000 7f7d39d9b700 1 --2- 172.21.15.33:0/3546607864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796ea0 0x561885797320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.236+0000 7f7d3a59c700 1 -- 172.21.15.33:0/3546607864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885797860 msgr2=0x561885797ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.236+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/3546607864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885797860 0x561885797ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.236+0000 7f7d3a59c700 1 -- 172.21.15.33:0/3546607864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796ea0 msgr2=0x561885797320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.236+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/3546607864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796ea0 0x561885797320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.236+0000 7f7d3a59c700 1 -- 172.21.15.33:0/3546607864 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618855673b0 con 0x561885796540 2024-05-08T21:23:55.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.236+0000 7f7d39d9b700 1 --2- 172.21.15.33:0/3546607864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796ea0 0x561885797320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.236+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/3546607864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 0x561885796960 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c09ad40 tx=0x7f7d2c09acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3009dd4cef1913d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.236+0000 7f7d28ff9700 1 -- 172.21.15.33:0/3546607864 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c09adb0 con 0x561885796540 2024-05-08T21:23:55.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.236+0000 7f7d28ff9700 1 -- 172.21.15.33:0/3546607864 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d2c09ebe0 con 0x561885796540 2024-05-08T21:23:55.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.236+0000 7f7d28ff9700 1 -- 172.21.15.33:0/3546607864 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c09b010 con 0x561885796540 2024-05-08T21:23:55.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.237+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3546607864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 msgr2=0x561885796960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.237+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3546607864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 0x561885796960 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c09ad40 tx=0x7f7d2c09acb0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.237+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3546607864 shutdown_connections 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.237+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3546607864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885797860 0x561885797ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.237+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3546607864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 0x561885796960 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.237+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3546607864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796ea0 0x561885797320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.237+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3546607864 >> 172.21.15.33:0/3546607864 conn(0x5618856f4210 msgr2=0x561885790de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.237+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3546607864 shutdown_connections 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.237+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3546607864 wait complete. 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.237+0000 7f7d4cc00b80 1 -- start start 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.237+0000 7f7d4cc00b80 1 Processor -- start 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d4cc00b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 0x56188579fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d4cc00b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796ea0 0x5618857a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d4cc00b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885797860 0x5618857a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d4cc00b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885682560 con 0x561885796540 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d4cc00b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561885682200 con 0x561885796ea0 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d4cc00b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618855696a0 con 0x561885797860 2024-05-08T21:23:55.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d4cc00b80 10 client.0 _ll_register_callbacks cb 0x56188568bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d3a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 0x56188579fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d3a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 0x56188579fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37028/0 (socket says 172.21.15.33:37028) 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d39d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796ea0 0x5618857a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d3a59c700 1 -- 172.21.15.33:0/3147137376 learned_addr learned my addr 172.21.15.33:0/3147137376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d3ad9d700 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885797860 0x5618857a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d3a59c700 1 -- 172.21.15.33:0/3147137376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885797860 msgr2=0x5618857a72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885797860 0x5618857a72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d3a59c700 1 -- 172.21.15.33:0/3147137376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796ea0 msgr2=0x5618857a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796ea0 0x5618857a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d3a59c700 1 -- 172.21.15.33:0/3147137376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c10be00 con 0x561885796540 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.238+0000 7f7d3ad9d700 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885797860 0x5618857a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 0x56188579fd40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c2d0 tx=0x7f7d2c010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0820be611b0e4ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d21ffb700 1 -- 172.21.15.33:0/3147137376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c09b210 con 0x561885796540 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d21ffb700 1 -- 172.21.15.33:0/3147137376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7d2c09adb0 con 0x561885796540 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3147137376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56188563f270 con 0x561885796540 2024-05-08T21:23:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d21ffb700 1 -- 172.21.15.33:0/3147137376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d2c0a2da0 con 0x561885796540 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d4cc00b80 10 client.16884 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3147137376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f7d3c1a1420 con 0x561885796540 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d21ffb700 1 -- 172.21.15.33:0/3147137376 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56188563f270 con 0x561885796540 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d21ffb700 1 -- 172.21.15.33:0/3147137376 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f7d2c099d10 con 0x561885796540 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d4cc00b80 10 client.16884 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d4cc00b80 10 client.16884 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d4cc00b80 20 client.16884 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5618857ab840 0x5618857adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d4cc00b80 4 client.16884 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.239+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3147137376 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561885765810 con 0x5618857ab840 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.240+0000 7f7d39d9b700 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5618857ab840 0x5618857adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.240+0000 7f7d39d9b700 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5618857ab840 0x5618857adc60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=453abf7712f68b02 server_cookie=4595fdbfde38fb5 in_seq=0 out_seq=0 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.240+0000 7f7d21ffb700 10 client.16884 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:23:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d21ffb700 1 -- 172.21.15.33:0/3147137376 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561885765810 con 0x5618857ab840 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d21ffb700 10 client.16884 handle_command_reply: tid=0 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d4cc00b80 1 client.16884 shutdown 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3147137376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 msgr2=0x56188579fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 0x56188579fd40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00c2d0 tx=0x7f7d2c010b80 comp rx=0 tx=0).stop 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3147137376 shutdown_connections 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5618857ab840 0x5618857adc60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885797860 0x5618857a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796540 0x56188579fd40 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d21ffb700 0 client.16884 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3147137376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796ea0 0x5618857a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3147137376 >> 172.21.15.33:0/3147137376 conn(0x5618856f4210 msgr2=0x5618857905a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3147137376 shutdown_connections 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3147137376 wait complete. 2024-05-08T21:23:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.241+0000 7f7d4cc00b80 20 client.16884 trim_cache size 0 max 16384 2024-05-08T21:23:55.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d4cc00b80 1 Processor -- start 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d4cc00b80 1 -- start start 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d4cc00b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796540 0x5618857adc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d4cc00b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796ea0 0x561885796920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d4cc00b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885797860 0x5618857a7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d4cc00b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56188579ed80 con 0x561885796ea0 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d4cc00b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618857b2e10 con 0x561885797860 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d4cc00b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618857ab510 con 0x561885796540 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d39d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796ea0 0x561885796920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d39d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796ea0 0x561885796920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37036/0 (socket says 172.21.15.33:37036) 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d39d9b700 1 -- 172.21.15.33:0/2082632992 learned_addr learned my addr 172.21.15.33:0/2082632992 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/2082632992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796540 0x5618857adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.257+0000 7f7d3ad9d700 1 --2- 172.21.15.33:0/2082632992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885797860 0x5618857a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.258+0000 7f7d39d9b700 1 -- 172.21.15.33:0/2082632992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796540 msgr2=0x5618857adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.258+0000 7f7d39d9b700 1 --2- 172.21.15.33:0/2082632992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796540 0x5618857adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.258+0000 7f7d39d9b700 1 -- 172.21.15.33:0/2082632992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885797860 msgr2=0x5618857a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.258+0000 7f7d39d9b700 1 --2- 172.21.15.33:0/2082632992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885797860 0x5618857a7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.258+0000 7f7d39d9b700 1 -- 172.21.15.33:0/2082632992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561885513b60 con 0x561885796ea0 2024-05-08T21:23:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.258+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/2082632992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796540 0x5618857adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.258+0000 7f7d3ad9d700 1 --2- 172.21.15.33:0/2082632992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885797860 0x5618857a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.258+0000 7f7d39d9b700 1 --2- 172.21.15.33:0/2082632992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796ea0 0x561885796920 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7d30004360 tx=0x7f7d30023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f12f7c99061f46 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.258+0000 7f7d297fa700 1 -- 172.21.15.33:0/2082632992 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d30003c20 con 0x561885796ea0 2024-05-08T21:23:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.258+0000 7f7d297fa700 1 -- 172.21.15.33:0/2082632992 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d3001e040 con 0x561885796ea0 2024-05-08T21:23:55.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.258+0000 7f7d297fa700 1 -- 172.21.15.33:0/2082632992 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d300367e0 con 0x561885796ea0 2024-05-08T21:23:55.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.259+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/2082632992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796ea0 msgr2=0x561885796920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.259+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/2082632992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796ea0 0x561885796920 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7d30004360 tx=0x7f7d30023000 comp rx=0 tx=0).stop 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.259+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/2082632992 shutdown_connections 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.259+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/2082632992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885797860 0x5618857a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.259+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/2082632992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796540 0x5618857adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.259+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/2082632992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885796ea0 0x561885796920 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.259+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/2082632992 >> 172.21.15.33:0/2082632992 conn(0x5618856f4210 msgr2=0x561885797c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.259+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/2082632992 shutdown_connections 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.259+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/2082632992 wait complete. 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.259+0000 7f7d4cc00b80 1 -- start start 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.259+0000 7f7d4cc00b80 1 Processor -- start 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d4cc00b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796540 0x5618857a2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d4cc00b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796ea0 0x5618857a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d4cc00b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885797860 0x5618857a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d4cc00b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618857a60e0 con 0x561885797860 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d4cc00b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618857a6250 con 0x561885796540 2024-05-08T21:23:55.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d4cc00b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618857a6410 con 0x561885796ea0 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d4cc00b80 10 client.0 _ll_register_callbacks cb 0x56188568bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d39d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796ea0 0x5618857a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d3ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885797860 0x5618857a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d3a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796540 0x5618857a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d39d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796ea0 0x5618857a2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43450/0 (socket says 172.21.15.33:43450) 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d39d9b700 1 -- 172.21.15.33:0/3580607069 learned_addr learned my addr 172.21.15.33:0/3580607069 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d39d9b700 1 -- 172.21.15.33:0/3580607069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796540 msgr2=0x5618857a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d39d9b700 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796540 0x5618857a2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d39d9b700 1 -- 172.21.15.33:0/3580607069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885797860 msgr2=0x5618857a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d39d9b700 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885797860 0x5618857a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d39d9b700 1 -- 172.21.15.33:0/3580607069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c1a1420 con 0x561885796ea0 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796540 0x5618857a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.260+0000 7f7d3ad9d700 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885797860 0x5618857a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d39d9b700 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796ea0 0x5618857a2b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d30000c00 tx=0x7f7d30005450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d1a4247b9841b72 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d21ffb700 1 -- 172.21.15.33:0/3580607069 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d300367a0 con 0x561885796ea0 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d21ffb700 1 -- 172.21.15.33:0/3580607069 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7d3000ba70 con 0x561885796ea0 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3580607069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d2c09ebe0 con 0x561885796ea0 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d21ffb700 1 -- 172.21.15.33:0/3580607069 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d30036900 con 0x561885796ea0 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d4cc00b80 10 client.26779 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3580607069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f7d340127f0 con 0x561885796ea0 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d21ffb700 1 -- 172.21.15.33:0/3580607069 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d2c09ebe0 con 0x561885796ea0 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d21ffb700 1 -- 172.21.15.33:0/3580607069 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f7d3002f070 con 0x561885796ea0 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d4cc00b80 10 client.26779 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d4cc00b80 10 client.26779 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.261+0000 7f7d4cc00b80 20 client.26779 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.262+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x561885801db0 0x5618858041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.262+0000 7f7d4cc00b80 4 client.26779 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.262+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3580607069 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5618856deec0 con 0x561885801db0 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.262+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x561885801db0 0x5618858041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.262+0000 7f7d21ffb700 10 client.26779 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.262+0000 7f7d3a59c700 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x561885801db0 0x5618858041d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f53dcc47836970dc server_cookie=36132d72f1cdb702 in_seq=0 out_seq=0 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.262+0000 7f7d21ffb700 1 -- 172.21.15.33:0/3580607069 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5618856deec0 con 0x561885801db0 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.262+0000 7f7d21ffb700 10 client.26779 handle_command_reply: tid=0 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.262+0000 7f7d4cc00b80 1 client.26779 shutdown 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3580607069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796ea0 msgr2=0x5618857a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796ea0 0x5618857a2b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d30000c00 tx=0x7f7d30005450 comp rx=0 tx=0).stop 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3580607069 shutdown_connections 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561885797860 0x5618857a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561885796540 0x5618857a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x561885801db0 0x5618858041d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d4cc00b80 1 --2- 172.21.15.33:0/3580607069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561885796ea0 0x5618857a2b80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d21ffb700 0 client.26779 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3580607069 >> 172.21.15.33:0/3580607069 conn(0x5618856f4210 msgr2=0x56188579f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3580607069 shutdown_connections 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d4cc00b80 1 -- 172.21.15.33:0/3580607069 wait complete. 2024-05-08T21:23:55.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d4cc00b80 20 client.26779 trim_cache size 0 max 16384 2024-05-08T21:23:55.270 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:23:55.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d40a46700 1 -- 172.21.15.33:0/2117948699 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7d240a1120 msgr2=0x7f7d240a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d40a46700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7d240a1120 0x7f7d240a35e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7d30006010 tx=0x7f7d3000b610 comp rx=0 tx=0).stop 2024-05-08T21:23:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d40a46700 1 -- 172.21.15.33:0/2117948699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d3c0739a0 msgr2=0x7f7d3c198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d40a46700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d3c0739a0 0x7f7d3c198f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00e9a0 tx=0x7f7d2c00ecb0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d40a46700 1 -- 172.21.15.33:0/2117948699 shutdown_connections 2024-05-08T21:23:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d40a46700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d3c108d70 0x7f7d3c1a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d40a46700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7d240a1120 0x7f7d240a35e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d40a46700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d3c0739a0 0x7f7d3c198f20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d40a46700 1 --2- 172.21.15.33:0/2117948699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d3c074f80 0x7f7d3c199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.263+0000 7f7d40a46700 1 -- 172.21.15.33:0/2117948699 >> 172.21.15.33:0/2117948699 conn(0x7f7d3c0fc430 msgr2=0x7f7d3c104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.264+0000 7f7d40a46700 1 -- 172.21.15.33:0/2117948699 shutdown_connections 2024-05-08T21:23:55.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.264+0000 7f7d40a46700 1 -- 172.21.15.33:0/2117948699 wait complete. 2024-05-08T21:23:55.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:23:55.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.431+0000 7fd6727cc700 1 -- 172.21.15.33:0/1611817499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd66c0739a0 msgr2=0x7fd66c073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.431+0000 7fd6727cc700 1 --2- 172.21.15.33:0/1611817499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd66c0739a0 0x7fd66c073e20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd660009990 tx=0x7fd660009ca0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.431+0000 7fd6727cc700 1 -- 172.21.15.33:0/1611817499 shutdown_connections 2024-05-08T21:23:55.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.431+0000 7fd6727cc700 1 --2- 172.21.15.33:0/1611817499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66c108d40 0x7fd66c10b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.431+0000 7fd6727cc700 1 --2- 172.21.15.33:0/1611817499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd66c074f80 0x7fd66c0733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.431+0000 7fd6727cc700 1 --2- 172.21.15.33:0/1611817499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd66c0739a0 0x7fd66c073e20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.431+0000 7fd6727cc700 1 -- 172.21.15.33:0/1611817499 >> 172.21.15.33:0/1611817499 conn(0x7fd66c0fc400 msgr2=0x7fd66c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.431+0000 7fd6727cc700 1 -- 172.21.15.33:0/1611817499 shutdown_connections 2024-05-08T21:23:55.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.431+0000 7fd6727cc700 1 -- 172.21.15.33:0/1611817499 wait complete. 2024-05-08T21:23:55.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.431+0000 7fd6727cc700 1 Processor -- start 2024-05-08T21:23:55.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd6727cc700 1 -- start start 2024-05-08T21:23:55.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd6727cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd66c0739a0 0x7fd66c198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd6727cc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66c074f80 0x7fd66c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd6727cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd66c108d40 0x7fd66c1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd6727cc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c1a09e0 con 0x7fd66c0739a0 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd6727cc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c1a0b20 con 0x7fd66c074f80 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd6727cc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c1a0cb0 con 0x7fd66c108d40 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66c074f80 0x7fd66c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd66bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd66c0739a0 0x7fd66c198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66c074f80 0x7fd66c1993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57708/0 (socket says 172.21.15.33:57708) 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd66bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd66c0739a0 0x7fd66c198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37068/0 (socket says 172.21.15.33:37068) 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd66b7fe700 1 -- 172.21.15.33:0/4071376295 learned_addr learned my addr 172.21.15.33:0/4071376295 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.432+0000 7fd670d69700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd66c108d40 0x7fd66c1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.433+0000 7fd66b7fe700 1 -- 172.21.15.33:0/4071376295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd66c108d40 msgr2=0x7fd66c1a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.433+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd66c108d40 0x7fd66c1a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.433+0000 7fd66b7fe700 1 -- 172.21.15.33:0/4071376295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd66c0739a0 msgr2=0x7fd66c198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.433+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd66c0739a0 0x7fd66c198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.433+0000 7fd66b7fe700 1 -- 172.21.15.33:0/4071376295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6600095f0 con 0x7fd66c074f80 2024-05-08T21:23:55.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.433+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66c074f80 0x7fd66c1993f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd660003ec0 tx=0x7fd660004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.433+0000 7fd6697fa700 1 -- 172.21.15.33:0/4071376295 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66001d070 con 0x7fd66c074f80 2024-05-08T21:23:55.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.433+0000 7fd6697fa700 1 -- 172.21.15.33:0/4071376295 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd66000bdb0 con 0x7fd66c074f80 2024-05-08T21:23:55.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.433+0000 7fd6727cc700 1 -- 172.21.15.33:0/4071376295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66c1a0f30 con 0x7fd66c074f80 2024-05-08T21:23:55.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.433+0000 7fd6697fa700 1 -- 172.21.15.33:0/4071376295 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66000fdc0 con 0x7fd66c074f80 2024-05-08T21:23:55.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.433+0000 7fd6727cc700 1 -- 172.21.15.33:0/4071376295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66c1a1350 con 0x7fd66c074f80 2024-05-08T21:23:55.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.436+0000 7fd6697fa700 1 -- 172.21.15.33:0/4071376295 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd66000f560 con 0x7fd66c074f80 2024-05-08T21:23:55.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.437+0000 7fd6697fa700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd6540a11a0 0x7fd6540a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.437+0000 7fd66bfff700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd6540a11a0 0x7fd6540a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.437+0000 7fd6697fa700 1 -- 172.21.15.33:0/4071376295 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd6600cdd80 con 0x7fd66c074f80 2024-05-08T21:23:55.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.437+0000 7fd67e986b80 1 Processor -- start 2024-05-08T21:23:55.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.437+0000 7fd67e986b80 1 -- start start 2024-05-08T21:23:55.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.437+0000 7fd66bfff700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd6540a11a0 0x7fd6540a3660 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009c60 tx=0x7fd65c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.437+0000 7fd67e986b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f7f540 0x55cc21f7f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.437+0000 7fd67e986b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f80320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd67e986b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f80860 0x55cc21f80ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd67e986b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21ed6150 con 0x55cc21f80860 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd67e986b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21ed3b10 con 0x55cc21f7f540 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd67e986b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21ecf400 con 0x55cc21f7fea0 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f80320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f80320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43478/0 (socket says 172.21.15.33:43478) 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd66b7fe700 1 -- 172.21.15.33:0/3328212519 learned_addr learned my addr 172.21.15.33:0/3328212519 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd670d69700 1 --2- 172.21.15.33:0/3328212519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f80860 0x55cc21f80ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd66bfff700 1 --2- 172.21.15.33:0/3328212519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f7f540 0x55cc21f7f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd66b7fe700 1 -- 172.21.15.33:0/3328212519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f7f540 msgr2=0x55cc21f7f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/3328212519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f7f540 0x55cc21f7f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd66b7fe700 1 -- 172.21.15.33:0/3328212519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f80860 msgr2=0x55cc21f80ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/3328212519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f80860 0x55cc21f80ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd66b7fe700 1 -- 172.21.15.33:0/3328212519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc21d503b0 con 0x55cc21f7fea0 2024-05-08T21:23:55.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd66bfff700 1 --2- 172.21.15.33:0/3328212519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f7f540 0x55cc21f7f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:23:55.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/3328212519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f80320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd6600a2d60 tx=0x7fd66009ac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c08325bdc23c373c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd6527fc700 1 -- 172.21.15.33:0/3328212519 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66009a800 con 0x55cc21f7fea0 2024-05-08T21:23:55.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd6527fc700 1 -- 172.21.15.33:0/3328212519 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd66009b5a0 con 0x55cc21f7fea0 2024-05-08T21:23:55.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.438+0000 7fd6527fc700 1 -- 172.21.15.33:0/3328212519 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66009a5f0 con 0x55cc21f7fea0 2024-05-08T21:23:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.439+0000 7fd67e986b80 1 -- 172.21.15.33:0/3328212519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 msgr2=0x55cc21f80320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.439+0000 7fd67e986b80 1 --2- 172.21.15.33:0/3328212519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f80320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd6600a2d60 tx=0x7fd66009ac70 comp rx=0 tx=0).stop 2024-05-08T21:23:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.439+0000 7fd67e986b80 1 -- 172.21.15.33:0/3328212519 shutdown_connections 2024-05-08T21:23:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.439+0000 7fd67e986b80 1 --2- 172.21.15.33:0/3328212519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f80860 0x55cc21f80ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.439+0000 7fd67e986b80 1 --2- 172.21.15.33:0/3328212519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f7f540 0x55cc21f7f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.439+0000 7fd67e986b80 1 --2- 172.21.15.33:0/3328212519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f80320 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.439+0000 7fd67e986b80 1 -- 172.21.15.33:0/3328212519 >> 172.21.15.33:0/3328212519 conn(0x55cc21edd210 msgr2=0x55cc21f79de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.439+0000 7fd67e986b80 1 -- 172.21.15.33:0/3328212519 shutdown_connections 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.439+0000 7fd67e986b80 1 -- 172.21.15.33:0/3328212519 wait complete. 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.440+0000 7fd67e986b80 1 -- start start 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.440+0000 7fd67e986b80 1 Processor -- start 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.440+0000 7fd67e986b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7f540 0x55cc21f88ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.440+0000 7fd67e986b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f88ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.440+0000 7fd67e986b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 0x55cc21f90030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.440+0000 7fd67e986b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21e6b560 con 0x55cc21f7f540 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.440+0000 7fd67e986b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21e6b200 con 0x55cc21f80860 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.440+0000 7fd67e986b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21d526a0 con 0x55cc21f7fea0 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.440+0000 7fd67e986b80 10 client.0 _ll_register_callbacks cb 0x55cc21e74b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.440+0000 7fd66bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7f540 0x55cc21f88ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.440+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f88ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd66bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7f540 0x55cc21f88ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37084/0 (socket says 172.21.15.33:37084) 2024-05-08T21:23:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd66bfff700 1 -- 172.21.15.33:0/1471615311 learned_addr learned my addr 172.21.15.33:0/1471615311 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f88ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43490/0 (socket says 172.21.15.33:43490) 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd670d69700 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 0x55cc21f90030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd66b7fe700 1 -- 172.21.15.33:0/1471615311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 msgr2=0x55cc21f90030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 0x55cc21f90030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd66b7fe700 1 -- 172.21.15.33:0/1471615311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7f540 msgr2=0x55cc21f88ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7f540 0x55cc21f88ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd66b7fe700 1 -- 172.21.15.33:0/1471615311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66c1a1350 con 0x55cc21f7fea0 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f88ff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd6600a3450 tx=0x7fd66000f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67532065679d0012 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd64b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd64b7fe700 1 -- 172.21.15.33:0/1471615311 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66009a800 con 0x55cc21f7fea0 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd64b7fe700 1 -- 172.21.15.33:0/1471615311 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd66009aea0 con 0x55cc21f7fea0 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd67e986b80 1 -- 172.21.15.33:0/1471615311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc21e28270 con 0x55cc21f7fea0 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd64b7fe700 1 -- 172.21.15.33:0/1471615311 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd66000b920 con 0x55cc21f7fea0 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd67e986b80 10 client.26791 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.441+0000 7fd67e986b80 1 -- 172.21.15.33:0/1471615311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55cc21cfcb60 con 0x55cc21f7fea0 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.442+0000 7fd64b7fe700 1 -- 172.21.15.33:0/1471615311 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd66007d070 con 0x55cc21f7fea0 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.442+0000 7fd64b7fe700 1 -- 172.21.15.33:0/1471615311 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fd660099e00 con 0x55cc21f7fea0 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.442+0000 7fd67e986b80 10 client.26791 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.442+0000 7fd67e986b80 10 client.26791 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.442+0000 7fd67e986b80 20 client.26791 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.442+0000 7fd67e986b80 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55cc21f94320 0x55cc21f96740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.442+0000 7fd67e986b80 4 client.26791 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.442+0000 7fd67e986b80 1 -- 172.21.15.33:0/1471615311 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc21f4e810 con 0x55cc21f94320 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.442+0000 7fd66bfff700 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55cc21f94320 0x55cc21f96740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.442+0000 7fd64b7fe700 10 client.26791 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.442+0000 7fd66bfff700 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55cc21f94320 0x55cc21f96740 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7bc958125046b411 server_cookie=ae9ab07ca10979c7 in_seq=0 out_seq=0 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd64b7fe700 1 -- 172.21.15.33:0/1471615311 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc21f4e810 con 0x55cc21f94320 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd64b7fe700 10 client.26791 handle_command_reply: tid=0 2024-05-08T21:23:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd67e986b80 1 client.26791 shutdown 2024-05-08T21:23:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd67e986b80 1 -- 172.21.15.33:0/1471615311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 msgr2=0x55cc21f88ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd67e986b80 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f88ff0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd6600a3450 tx=0x7fd66000f460 comp rx=0 tx=0).stop 2024-05-08T21:23:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd67e986b80 1 -- 172.21.15.33:0/1471615311 shutdown_connections 2024-05-08T21:23:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd67e986b80 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55cc21f94320 0x55cc21f96740 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd67e986b80 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 0x55cc21f90030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd67e986b80 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7f540 0x55cc21f88ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd67e986b80 1 --2- 172.21.15.33:0/1471615311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f88ff0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd64b7fe700 0 client.26791 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:23:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd67e986b80 1 -- 172.21.15.33:0/1471615311 >> 172.21.15.33:0/1471615311 conn(0x55cc21edd210 msgr2=0x55cc21f795a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd67e986b80 1 -- 172.21.15.33:0/1471615311 shutdown_connections 2024-05-08T21:23:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.443+0000 7fd67e986b80 1 -- 172.21.15.33:0/1471615311 wait complete. 2024-05-08T21:23:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.444+0000 7fd67e986b80 20 client.26791 trim_cache size 0 max 16384 2024-05-08T21:23:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.460+0000 7fd67e986b80 1 Processor -- start 2024-05-08T21:23:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.460+0000 7fd67e986b80 1 -- start start 2024-05-08T21:23:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.460+0000 7fd67e986b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7f540 0x55cc21f906e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.460+0000 7fd67e986b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7fea0 0x55cc21f8fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.460+0000 7fd67e986b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 0x55cc21f87fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.460+0000 7fd67e986b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21f7b570 con 0x55cc21f7fea0 2024-05-08T21:23:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.460+0000 7fd67e986b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21f9b8f0 con 0x55cc21f80860 2024-05-08T21:23:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.460+0000 7fd67e986b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21f87a50 con 0x55cc21f7f540 2024-05-08T21:23:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.460+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7fea0 0x55cc21f8fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7fea0 0x55cc21f8fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37096/0 (socket says 172.21.15.33:37096) 2024-05-08T21:23:55.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd66b7fe700 1 -- 172.21.15.33:0/2667503278 learned_addr learned my addr 172.21.15.33:0/2667503278 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd670d69700 1 --2- 172.21.15.33:0/2667503278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 0x55cc21f87fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd66bfff700 1 --2- 172.21.15.33:0/2667503278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7f540 0x55cc21f906e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd66b7fe700 1 -- 172.21.15.33:0/2667503278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7f540 msgr2=0x55cc21f906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/2667503278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7f540 0x55cc21f906e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd66b7fe700 1 -- 172.21.15.33:0/2667503278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 msgr2=0x55cc21f87fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/2667503278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 0x55cc21f87fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd66b7fe700 1 -- 172.21.15.33:0/2667503278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc21e95560 con 0x55cc21f7fea0 2024-05-08T21:23:55.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/2667503278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7fea0 0x55cc21f8fff0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd66009ae30 tx=0x7fd6600cb540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ca8bb9fe0f46f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd652ffd700 1 -- 172.21.15.33:0/2667503278 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600ba8e0 con 0x55cc21f7fea0 2024-05-08T21:23:55.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd652ffd700 1 -- 172.21.15.33:0/2667503278 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd66007d070 con 0x55cc21f7fea0 2024-05-08T21:23:55.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.461+0000 7fd652ffd700 1 -- 172.21.15.33:0/2667503278 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66009aea0 con 0x55cc21f7fea0 2024-05-08T21:23:55.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.462+0000 7fd67e986b80 1 -- 172.21.15.33:0/2667503278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7fea0 msgr2=0x55cc21f8fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.462+0000 7fd67e986b80 1 --2- 172.21.15.33:0/2667503278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7fea0 0x55cc21f8fff0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd66009ae30 tx=0x7fd6600cb540 comp rx=0 tx=0).stop 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.462+0000 7fd67e986b80 1 -- 172.21.15.33:0/2667503278 shutdown_connections 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.462+0000 7fd67e986b80 1 --2- 172.21.15.33:0/2667503278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 0x55cc21f87fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.462+0000 7fd67e986b80 1 --2- 172.21.15.33:0/2667503278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7f540 0x55cc21f906e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.462+0000 7fd67e986b80 1 --2- 172.21.15.33:0/2667503278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7fea0 0x55cc21f8fff0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.462+0000 7fd67e986b80 1 -- 172.21.15.33:0/2667503278 >> 172.21.15.33:0/2667503278 conn(0x55cc21edd210 msgr2=0x55cc21f80c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.462+0000 7fd67e986b80 1 -- 172.21.15.33:0/2667503278 shutdown_connections 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.462+0000 7fd67e986b80 1 -- 172.21.15.33:0/2667503278 wait complete. 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.463+0000 7fd67e986b80 1 -- start start 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.463+0000 7fd67e986b80 1 Processor -- start 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.463+0000 7fd67e986b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7f540 0x55cc21f8bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.463+0000 7fd67e986b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f8c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.463+0000 7fd67e986b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 0x55cc21f8f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.463+0000 7fd67e986b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21f8f560 con 0x55cc21f7f540 2024-05-08T21:23:55.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.463+0000 7fd67e986b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21f8f6d0 con 0x55cc21f80860 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.463+0000 7fd67e986b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc21f8f890 con 0x55cc21f7fea0 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.463+0000 7fd67e986b80 10 client.0 _ll_register_callbacks cb 0x55cc21e74b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f8c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f8c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43508/0 (socket says 172.21.15.33:43508) 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd66b7fe700 1 -- 172.21.15.33:0/1636883813 learned_addr learned my addr 172.21.15.33:0/1636883813 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd670d69700 1 --2- 172.21.15.33:0/1636883813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 0x55cc21f8f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd66b7fe700 1 -- 172.21.15.33:0/1636883813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 msgr2=0x55cc21f8f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/1636883813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 0x55cc21f8f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd66b7fe700 1 -- 172.21.15.33:0/1636883813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7f540 msgr2=0x55cc21f8bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/1636883813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7f540 0x55cc21f8bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd66b7fe700 1 -- 172.21.15.33:0/1636883813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc21cfcb60 con 0x55cc21f7fea0 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd66b7fe700 1 --2- 172.21.15.33:0/1636883813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f8c000 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd6600a2d60 tx=0x7fd66000f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=238e0c73f76f9d4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd64b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd64b7fe700 1 -- 172.21.15.33:0/1636883813 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66009a000 con 0x55cc21f7fea0 2024-05-08T21:23:55.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd67e986b80 1 -- 172.21.15.33:0/1636883813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd66009b5a0 con 0x55cc21f7fea0 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd64b7fe700 1 -- 172.21.15.33:0/1636883813 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd66009ecd0 con 0x55cc21f7fea0 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.464+0000 7fd64b7fe700 1 -- 172.21.15.33:0/1636883813 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd66009aea0 con 0x55cc21f7fea0 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.465+0000 7fd67e986b80 10 client.26797 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.465+0000 7fd67e986b80 1 -- 172.21.15.33:0/1636883813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55cc21d54cd0 con 0x55cc21f7fea0 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.465+0000 7fd64b7fe700 1 -- 172.21.15.33:0/1636883813 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd66009b5a0 con 0x55cc21f7fea0 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.465+0000 7fd64b7fe700 1 -- 172.21.15.33:0/1636883813 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fd6600840d0 con 0x55cc21f7fea0 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.465+0000 7fd67e986b80 10 client.26797 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.465+0000 7fd67e986b80 10 client.26797 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.465+0000 7fd67e986b80 20 client.26797 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.465+0000 7fd67e986b80 1 --2- 172.21.15.33:0/1636883813 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55cc21feadc0 0x55cc21fed1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.465+0000 7fd67e986b80 4 client.26797 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.465+0000 7fd67e986b80 1 -- 172.21.15.33:0/1636883813 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cc21ec7ec0 con 0x55cc21feadc0 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.465+0000 7fd66bfff700 1 --2- 172.21.15.33:0/1636883813 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55cc21feadc0 0x55cc21fed1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd66bfff700 1 --2- 172.21.15.33:0/1636883813 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55cc21feadc0 0x55cc21fed1b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92afe8e16b0717b1 server_cookie=4f80043b7d2d03d8 in_seq=0 out_seq=0 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd64b7fe700 10 client.26797 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd64b7fe700 1 -- 172.21.15.33:0/1636883813 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cc21ec7ec0 con 0x55cc21feadc0 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd64b7fe700 10 client.26797 handle_command_reply: tid=0 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd67e986b80 1 client.26797 shutdown 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd67e986b80 1 -- 172.21.15.33:0/1636883813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 msgr2=0x55cc21f8c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd67e986b80 1 --2- 172.21.15.33:0/1636883813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f8c000 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd6600a2d60 tx=0x7fd66000f460 comp rx=0 tx=0).stop 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd67e986b80 1 -- 172.21.15.33:0/1636883813 shutdown_connections 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd67e986b80 1 --2- 172.21.15.33:0/1636883813 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55cc21feadc0 0x55cc21fed1b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd67e986b80 1 --2- 172.21.15.33:0/1636883813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc21f80860 0x55cc21f8f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd67e986b80 1 --2- 172.21.15.33:0/1636883813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc21f7f540 0x55cc21f8bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd64b7fe700 0 client.26797 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd67e986b80 1 --2- 172.21.15.33:0/1636883813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc21f7fea0 0x55cc21f8c000 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd67e986b80 1 -- 172.21.15.33:0/1636883813 >> 172.21.15.33:0/1636883813 conn(0x55cc21edd210 msgr2=0x55cc21f906e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd67e986b80 1 -- 172.21.15.33:0/1636883813 shutdown_connections 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd67e986b80 1 -- 172.21.15.33:0/1636883813 wait complete. 2024-05-08T21:23:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.466+0000 7fd67e986b80 20 client.26797 trim_cache size 0 max 16384 2024-05-08T21:23:55.473 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:23:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.467+0000 7fd6727cc700 1 -- 172.21.15.33:0/4071376295 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd6540a11a0 msgr2=0x7fd6540a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.467+0000 7fd6727cc700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd6540a11a0 0x7fd6540a3660 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009c60 tx=0x7fd65c009260 comp rx=0 tx=0).stop 2024-05-08T21:23:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.467+0000 7fd6727cc700 1 -- 172.21.15.33:0/4071376295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66c074f80 msgr2=0x7fd66c1993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.467+0000 7fd6727cc700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66c074f80 0x7fd66c1993f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd660003ec0 tx=0x7fd660004730 comp rx=0 tx=0).stop 2024-05-08T21:23:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.467+0000 7fd6727cc700 1 -- 172.21.15.33:0/4071376295 shutdown_connections 2024-05-08T21:23:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.467+0000 7fd6727cc700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd66c108d40 0x7fd66c1a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.467+0000 7fd6727cc700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd66c0739a0 0x7fd66c198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.467+0000 7fd6727cc700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd6540a11a0 0x7fd6540a3660 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.467+0000 7fd6727cc700 1 --2- 172.21.15.33:0/4071376295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd66c074f80 0x7fd66c1993f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.467+0000 7fd6727cc700 1 -- 172.21.15.33:0/4071376295 >> 172.21.15.33:0/4071376295 conn(0x7fd66c0fc400 msgr2=0x7fd66c104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.467+0000 7fd6727cc700 1 -- 172.21.15.33:0/4071376295 shutdown_connections 2024-05-08T21:23:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.467+0000 7fd6727cc700 1 -- 172.21.15.33:0/4071376295 wait complete. 2024-05-08T21:23:55.483 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:23:55.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.636+0000 7fdbeb910700 1 -- 172.21.15.33:0/4004339905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe40ffda0 msgr2=0x7fdbe41001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.636+0000 7fdbeb910700 1 --2- 172.21.15.33:0/4004339905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe40ffda0 0x7fdbe41001c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00099c0 tx=0x7fdbe0009cd0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.636+0000 7fdbeb910700 1 -- 172.21.15.33:0/4004339905 shutdown_connections 2024-05-08T21:23:55.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.636+0000 7fdbeb910700 1 --2- 172.21.15.33:0/4004339905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe4101930 0x7fdbe41081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.636+0000 7fdbeb910700 1 --2- 172.21.15.33:0/4004339905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe40ffda0 0x7fdbe41001c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.636+0000 7fdbeb910700 1 --2- 172.21.15.33:0/4004339905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbe4100f90 0x7fdbe41013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.636+0000 7fdbeb910700 1 -- 172.21.15.33:0/4004339905 >> 172.21.15.33:0/4004339905 conn(0x7fdbe40fb330 msgr2=0x7fdbe40fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.636+0000 7fdbeb910700 1 -- 172.21.15.33:0/4004339905 shutdown_connections 2024-05-08T21:23:55.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.637+0000 7fdbeb910700 1 -- 172.21.15.33:0/4004339905 wait complete. 2024-05-08T21:23:55.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.637+0000 7fdbeb910700 1 Processor -- start 2024-05-08T21:23:55.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.637+0000 7fdbeb910700 1 -- start start 2024-05-08T21:23:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbeb910700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe40ffda0 0x7fdbe4196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbeb910700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbe4100f90 0x7fdbe4197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbeb910700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe4101930 0x7fdbe419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbeb910700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe419e870 con 0x7fdbe40ffda0 2024-05-08T21:23:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbeb910700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe419e9b0 con 0x7fdbe4100f90 2024-05-08T21:23:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbeb910700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe419eb70 con 0x7fdbe4101930 2024-05-08T21:23:55.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbe96ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe40ffda0 0x7fdbe4196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbe96ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe40ffda0 0x7fdbe4196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37132/0 (socket says 172.21.15.33:37132) 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbe96ac700 1 -- 172.21.15.33:0/3783595421 learned_addr learned my addr 172.21.15.33:0/3783595421 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbe8eab700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbe4100f90 0x7fdbe4197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbe9ead700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe4101930 0x7fdbe419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbe8eab700 1 -- 172.21.15.33:0/3783595421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe4101930 msgr2=0x7fdbe419e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbe8eab700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe4101930 0x7fdbe419e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbe8eab700 1 -- 172.21.15.33:0/3783595421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe40ffda0 msgr2=0x7fdbe4196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbe8eab700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe40ffda0 0x7fdbe4196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbe8eab700 1 -- 172.21.15.33:0/3783595421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe00095f0 con 0x7fdbe4100f90 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.638+0000 7fdbe8eab700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbe4100f90 0x7fdbe4197320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400ea20 tx=0x7fdbd400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.639+0000 7fdbda7fc700 1 -- 172.21.15.33:0/3783595421 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd400cbd0 con 0x7fdbe4100f90 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.639+0000 7fdbeb910700 1 -- 172.21.15.33:0/3783595421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe419ee50 con 0x7fdbe4100f90 2024-05-08T21:23:55.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.639+0000 7fdbda7fc700 1 -- 172.21.15.33:0/3783595421 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbd400ee60 con 0x7fdbe4100f90 2024-05-08T21:23:55.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.639+0000 7fdbda7fc700 1 -- 172.21.15.33:0/3783595421 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4018420 con 0x7fdbe4100f90 2024-05-08T21:23:55.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.639+0000 7fdbeb910700 1 -- 172.21.15.33:0/3783595421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe419f2d0 con 0x7fdbe4100f90 2024-05-08T21:23:55.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.641+0000 7fdbda7fc700 1 -- 172.21.15.33:0/3783595421 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fdbd4018580 con 0x7fdbe4100f90 2024-05-08T21:23:55.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.642+0000 7fdbda7fc700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdbd00a1120 0x7fdbd00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.642+0000 7fdbda7fc700 1 -- 172.21.15.33:0/3783595421 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fdbd4014070 con 0x7fdbe4100f90 2024-05-08T21:23:55.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.642+0000 7fdbe96ac700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdbd00a1120 0x7fdbd00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.643+0000 7fdbe96ac700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdbd00a1120 0x7fdbd00a35e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000b580 tx=0x7fdbe001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.643+0000 7fdbf7acab80 1 Processor -- start 2024-05-08T21:23:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.643+0000 7fdbf7acab80 1 -- start start 2024-05-08T21:23:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.643+0000 7fdbf7acab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 0x55f2480a5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbf7acab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a5ea0 0x55f2480a6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbf7acab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a6860 0x55f2480a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbf7acab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f247ffc150 con 0x55f2480a5ea0 2024-05-08T21:23:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbf7acab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f247ff9b10 con 0x55f2480a6860 2024-05-08T21:23:55.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbf7acab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f247ff5400 con 0x55f2480a5540 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe96ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 0x55f2480a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe8eab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a5ea0 0x55f2480a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe96ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 0x55f2480a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43534/0 (socket says 172.21.15.33:43534) 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe8eab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a5ea0 0x55f2480a6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37146/0 (socket says 172.21.15.33:37146) 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe96ac700 1 -- 172.21.15.33:0/3234877071 learned_addr learned my addr 172.21.15.33:0/3234877071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe9ead700 1 --2- 172.21.15.33:0/3234877071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a6860 0x55f2480a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe96ac700 1 -- 172.21.15.33:0/3234877071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a6860 msgr2=0x55f2480a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe96ac700 1 --2- 172.21.15.33:0/3234877071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a6860 0x55f2480a6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe96ac700 1 -- 172.21.15.33:0/3234877071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a5ea0 msgr2=0x55f2480a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe96ac700 1 --2- 172.21.15.33:0/3234877071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a5ea0 0x55f2480a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe96ac700 1 -- 172.21.15.33:0/3234877071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f247e763b0 con 0x55f2480a5540 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe8eab700 1 --2- 172.21.15.33:0/3234877071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a5ea0 0x55f2480a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.644+0000 7fdbe96ac700 1 --2- 172.21.15.33:0/3234877071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 0x55f2480a5960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00098e0 tx=0x7fdbe001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef320973bf509d60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.645+0000 7fdbcf7fe700 1 -- 172.21.15.33:0/3234877071 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0004400 con 0x55f2480a5540 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.645+0000 7fdbcf7fe700 1 -- 172.21.15.33:0/3234877071 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbe001e040 con 0x55f2480a5540 2024-05-08T21:23:55.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.645+0000 7fdbcf7fe700 1 -- 172.21.15.33:0/3234877071 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000f840 con 0x55f2480a5540 2024-05-08T21:23:55.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.645+0000 7fdbf7acab80 1 -- 172.21.15.33:0/3234877071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 msgr2=0x55f2480a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.645+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/3234877071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 0x55f2480a5960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdbe00098e0 tx=0x7fdbe001f000 comp rx=0 tx=0).stop 2024-05-08T21:23:55.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.646+0000 7fdbf7acab80 1 -- 172.21.15.33:0/3234877071 shutdown_connections 2024-05-08T21:23:55.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.646+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/3234877071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a6860 0x55f2480a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.646+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/3234877071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 0x55f2480a5960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.646+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/3234877071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a5ea0 0x55f2480a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.646+0000 7fdbf7acab80 1 -- 172.21.15.33:0/3234877071 >> 172.21.15.33:0/3234877071 conn(0x55f248003210 msgr2=0x55f24809fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.646+0000 7fdbf7acab80 1 -- 172.21.15.33:0/3234877071 shutdown_connections 2024-05-08T21:23:55.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.646+0000 7fdbf7acab80 1 -- 172.21.15.33:0/3234877071 wait complete. 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.646+0000 7fdbf7acab80 1 -- start start 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.646+0000 7fdbf7acab80 1 Processor -- start 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbf7acab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5540 0x55f2480aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbf7acab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5ea0 0x55f2480af310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbf7acab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbf7acab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f247f91560 con 0x55f2480a6860 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbf7acab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f247f91200 con 0x55f2480a5540 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbf7acab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f247e786a0 con 0x55f2480a5ea0 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbf7acab80 10 client.0 _ll_register_callbacks cb 0x55f247f9ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbe9ead700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbe9ead700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480b6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37160/0 (socket says 172.21.15.33:37160) 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbe96ac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5540 0x55f2480aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbe9ead700 1 -- 172.21.15.33:0/2635959756 learned_addr learned my addr 172.21.15.33:0/2635959756 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbe8eab700 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5ea0 0x55f2480af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbe9ead700 1 -- 172.21.15.33:0/2635959756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5ea0 msgr2=0x55f2480af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbe9ead700 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5ea0 0x55f2480af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbe9ead700 1 -- 172.21.15.33:0/2635959756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5540 msgr2=0x55f2480aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbe9ead700 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5540 0x55f2480aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbe9ead700 1 -- 172.21.15.33:0/2635959756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe419f2d0 con 0x55f2480a6860 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbe8eab700 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5ea0 0x55f2480af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbe9ead700 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480b6350 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc011e10 tx=0x7fdbdc014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e86176950d7f81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.647+0000 7fdbccff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbccff9700 1 -- 172.21.15.33:0/2635959756 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00e040 con 0x55f2480a6860 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbccff9700 1 -- 172.21.15.33:0/2635959756 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fdbdc003a00 con 0x55f2480a6860 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbf7acab80 1 -- 172.21.15.33:0/2635959756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f247f4e270 con 0x55f2480a6860 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbccff9700 1 -- 172.21.15.33:0/2635959756 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbdc003bc0 con 0x55f2480a6860 2024-05-08T21:23:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbf7acab80 10 client.16917 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbf7acab80 1 -- 172.21.15.33:0/2635959756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fdbe4069510 con 0x55f2480a6860 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbccff9700 1 -- 172.21.15.33:0/2635959756 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f247f4e270 con 0x55f2480a6860 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbccff9700 1 -- 172.21.15.33:0/2635959756 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fdbdc01c070 con 0x55f2480a6860 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbf7acab80 10 client.16917 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbf7acab80 10 client.16917 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbf7acab80 20 client.16917 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f2480bad00 0x55f2480bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbf7acab80 4 client.16917 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.648+0000 7fdbf7acab80 1 -- 172.21.15.33:0/2635959756 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f248074810 con 0x55f2480bad00 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.649+0000 7fdbe96ac700 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f2480bad00 0x55f2480bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.649+0000 7fdbe96ac700 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f2480bad00 0x55f2480bd120 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e45e31c336b4e7c0 server_cookie=591535069a45095a in_seq=0 out_seq=0 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.649+0000 7fdbccff9700 10 client.16917 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.650+0000 7fdbccff9700 1 -- 172.21.15.33:0/2635959756 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f248074810 con 0x55f2480bad00 2024-05-08T21:23:55.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.650+0000 7fdbccff9700 10 client.16917 handle_command_reply: tid=0 2024-05-08T21:23:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.650+0000 7fdbf7acab80 1 client.16917 shutdown 2024-05-08T21:23:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.650+0000 7fdbf7acab80 1 -- 172.21.15.33:0/2635959756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 msgr2=0x55f2480b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.650+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480b6350 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc011e10 tx=0x7fdbdc014600 comp rx=0 tx=0).stop 2024-05-08T21:23:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.650+0000 7fdbf7acab80 1 -- 172.21.15.33:0/2635959756 shutdown_connections 2024-05-08T21:23:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.650+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f2480bad00 0x55f2480bd120 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.650+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480b6350 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.650+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5540 0x55f2480aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.650+0000 7fdbccff9700 0 client.16917 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:23:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.650+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/2635959756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5ea0 0x55f2480af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.650+0000 7fdbf7acab80 1 -- 172.21.15.33:0/2635959756 >> 172.21.15.33:0/2635959756 conn(0x55f248003210 msgr2=0x55f24809f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.653+0000 7fdbf7acab80 1 -- 172.21.15.33:0/2635959756 shutdown_connections 2024-05-08T21:23:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.653+0000 7fdbf7acab80 1 -- 172.21.15.33:0/2635959756 wait complete. 2024-05-08T21:23:55.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.653+0000 7fdbf7acab80 20 client.16917 trim_cache size 0 max 16384 2024-05-08T21:23:55.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.669+0000 7fdbf7acab80 1 Processor -- start 2024-05-08T21:23:55.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.669+0000 7fdbf7acab80 1 -- start start 2024-05-08T21:23:55.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.669+0000 7fdbf7acab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5540 0x55f2480b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.669+0000 7fdbf7acab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5ea0 0x55f2480b6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbf7acab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbf7acab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2480a15f0 con 0x55f2480a6860 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbf7acab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2480c2360 con 0x55f2480a5540 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbf7acab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2480aca40 con 0x55f2480a5ea0 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbe96ac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5540 0x55f2480b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbe9ead700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbe8eab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5ea0 0x55f2480b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbe96ac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5540 0x55f2480b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57780/0 (socket says 172.21.15.33:57780) 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbe96ac700 1 -- 172.21.15.33:0/3609229800 learned_addr learned my addr 172.21.15.33:0/3609229800 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbe96ac700 1 -- 172.21.15.33:0/3609229800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5ea0 msgr2=0x55f2480b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbe96ac700 1 --2- 172.21.15.33:0/3609229800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5ea0 0x55f2480b6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbe96ac700 1 -- 172.21.15.33:0/3609229800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 msgr2=0x55f2480a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbe96ac700 1 --2- 172.21.15.33:0/3609229800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbe96ac700 1 -- 172.21.15.33:0/3609229800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f247e22b60 con 0x55f2480a5540 2024-05-08T21:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbe8eab700 1 --2- 172.21.15.33:0/3609229800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5ea0 0x55f2480b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.670+0000 7fdbe96ac700 1 --2- 172.21.15.33:0/3609229800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5540 0x55f2480b6a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0005f50 tx=0x7fdbe0009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aaf7744a0c23cfa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.671+0000 7fdbcffff700 1 -- 172.21.15.33:0/3609229800 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0035040 con 0x55f2480a5540 2024-05-08T21:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.671+0000 7fdbcffff700 1 -- 172.21.15.33:0/3609229800 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbe000b850 con 0x55f2480a5540 2024-05-08T21:23:55.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.671+0000 7fdbcffff700 1 -- 172.21.15.33:0/3609229800 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe003b640 con 0x55f2480a5540 2024-05-08T21:23:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.672+0000 7fdbf7acab80 1 -- 172.21.15.33:0/3609229800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5540 msgr2=0x55f2480b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.672+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/3609229800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5540 0x55f2480b6a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0005f50 tx=0x7fdbe0009cd0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.672+0000 7fdbf7acab80 1 -- 172.21.15.33:0/3609229800 shutdown_connections 2024-05-08T21:23:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.672+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/3609229800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.672+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/3609229800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5540 0x55f2480b6a00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.672+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/3609229800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5ea0 0x55f2480b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.672+0000 7fdbf7acab80 1 -- 172.21.15.33:0/3609229800 >> 172.21.15.33:0/3609229800 conn(0x55f248003210 msgr2=0x55f2480a6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.672+0000 7fdbf7acab80 1 -- 172.21.15.33:0/3609229800 shutdown_connections 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.672+0000 7fdbf7acab80 1 -- 172.21.15.33:0/3609229800 wait complete. 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.672+0000 7fdbf7acab80 1 -- start start 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.672+0000 7fdbf7acab80 1 Processor -- start 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbf7acab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 0x55f2480b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbf7acab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5ea0 0x55f2480b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbf7acab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbf7acab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2480b5180 con 0x55f2480a6860 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbf7acab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2480b52f0 con 0x55f2480a5ea0 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbf7acab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2480b54b0 con 0x55f2480a5540 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbf7acab80 10 client.0 _ll_register_callbacks cb 0x55f247f9ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbe96ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 0x55f2480b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbe8eab700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5ea0 0x55f2480b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbe8eab700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5ea0 0x55f2480b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57790/0 (socket says 172.21.15.33:57790) 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbe9ead700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbe96ac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 0x55f2480b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43556/0 (socket says 172.21.15.33:43556) 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbe8eab700 1 -- 172.21.15.33:0/1322158623 learned_addr learned my addr 172.21.15.33:0/1322158623 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbe8eab700 1 -- 172.21.15.33:0/1322158623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 msgr2=0x55f2480b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbe8eab700 1 --2- 172.21.15.33:0/1322158623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 0x55f2480b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbe8eab700 1 -- 172.21.15.33:0/1322158623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 msgr2=0x55f2480b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbe8eab700 1 --2- 172.21.15.33:0/1322158623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbe8eab700 1 -- 172.21.15.33:0/1322158623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe419ee50 con 0x55f2480a5ea0 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbe8eab700 1 --2- 172.21.15.33:0/1322158623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5ea0 0x55f2480b1c20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4099b00 tx=0x7fdbd40995d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85cce9fc88fa2d47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbccff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.673+0000 7fdbccff9700 1 -- 172.21.15.33:0/1322158623 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4095df0 con 0x55f2480a5ea0 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbccff9700 1 -- 172.21.15.33:0/1322158623 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fdbd4095f50 con 0x55f2480a5ea0 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbf7acab80 1 -- 172.21.15.33:0/1322158623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbe001e040 con 0x55f2480a5ea0 2024-05-08T21:23:55.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbccff9700 1 -- 172.21.15.33:0/1322158623 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbd4099e00 con 0x55f2480a5ea0 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbf7acab80 10 client.26798 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbf7acab80 1 -- 172.21.15.33:0/1322158623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f247e763b0 con 0x55f2480a5ea0 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbccff9700 1 -- 172.21.15.33:0/1322158623 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbe001e040 con 0x55f2480a5ea0 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbccff9700 1 -- 172.21.15.33:0/1322158623 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fdbd40b7fc0 con 0x55f2480a5ea0 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbf7acab80 10 client.26798 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbf7acab80 10 client.26798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbf7acab80 20 client.26798 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/1322158623 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f248111740 0x55f248113b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbf7acab80 4 client.26798 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbf7acab80 1 -- 172.21.15.33:0/1322158623 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f247fedec0 con 0x55f248111740 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.674+0000 7fdbe96ac700 1 --2- 172.21.15.33:0/1322158623 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f248111740 0x55f248113b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbccff9700 10 client.26798 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:23:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbe96ac700 1 --2- 172.21.15.33:0/1322158623 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f248111740 0x55f248113b30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2badbe218884ca49 server_cookie=5efee695e5aed166 in_seq=0 out_seq=0 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbccff9700 1 -- 172.21.15.33:0/1322158623 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f247fedec0 con 0x55f248111740 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbccff9700 10 client.26798 handle_command_reply: tid=0 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbf7acab80 1 client.26798 shutdown 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbf7acab80 1 -- 172.21.15.33:0/1322158623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5ea0 msgr2=0x55f2480b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/1322158623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5ea0 0x55f2480b1c20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4099b00 tx=0x7fdbd40995d0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbf7acab80 1 -- 172.21.15.33:0/1322158623 shutdown_connections 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/1322158623 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55f248111740 0x55f248113b30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/1322158623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2480a6860 0x55f2480b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/1322158623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2480a5540 0x55f2480b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbccff9700 0 client.26798 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbf7acab80 1 --2- 172.21.15.33:0/1322158623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2480a5ea0 0x55f2480b1c20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbf7acab80 1 -- 172.21.15.33:0/1322158623 >> 172.21.15.33:0/1322158623 conn(0x55f248003210 msgr2=0x55f2480adab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbf7acab80 1 -- 172.21.15.33:0/1322158623 shutdown_connections 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbf7acab80 1 -- 172.21.15.33:0/1322158623 wait complete. 2024-05-08T21:23:55.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.675+0000 7fdbf7acab80 20 client.26798 trim_cache size 0 max 16384 2024-05-08T21:23:55.684 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:23:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.676+0000 7fdbeb910700 1 -- 172.21.15.33:0/3783595421 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdbd00a1120 msgr2=0x7fdbd00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.676+0000 7fdbeb910700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdbd00a1120 0x7fdbd00a35e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000b580 tx=0x7fdbe001a040 comp rx=0 tx=0).stop 2024-05-08T21:23:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.676+0000 7fdbeb910700 1 -- 172.21.15.33:0/3783595421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbe4100f90 msgr2=0x7fdbe4197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.676+0000 7fdbeb910700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbe4100f90 0x7fdbe4197320 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400ea20 tx=0x7fdbd400ed30 comp rx=0 tx=0).stop 2024-05-08T21:23:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.676+0000 7fdbeb910700 1 -- 172.21.15.33:0/3783595421 shutdown_connections 2024-05-08T21:23:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.676+0000 7fdbeb910700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe4101930 0x7fdbe419e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.676+0000 7fdbeb910700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe40ffda0 0x7fdbe4196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.676+0000 7fdbeb910700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fdbd00a1120 0x7fdbd00a35e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.676+0000 7fdbeb910700 1 --2- 172.21.15.33:0/3783595421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbe4100f90 0x7fdbe4197320 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.676+0000 7fdbeb910700 1 -- 172.21.15.33:0/3783595421 >> 172.21.15.33:0/3783595421 conn(0x7fdbe40fb330 msgr2=0x7fdbe41043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.676+0000 7fdbeb910700 1 -- 172.21.15.33:0/3783595421 shutdown_connections 2024-05-08T21:23:55.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.676+0000 7fdbeb910700 1 -- 172.21.15.33:0/3783595421 wait complete. 2024-05-08T21:23:55.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:55 smithi033 ceph-mon[42528]: pgmap v438: 129 pgs: 129 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 304 op/s 2024-05-08T21:23:55.691 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:55 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1026947746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:23:55.692 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:23:55.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:55 smithi071 ceph-mon[47074]: pgmap v438: 129 pgs: 129 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 304 op/s 2024-05-08T21:23:55.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:55 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1026947746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:23:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.847+0000 7f5842739700 1 -- 172.21.15.33:0/45584273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f583c101940 msgr2=0x7f583c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.847+0000 7f5842739700 1 --2- 172.21.15.33:0/45584273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f583c101940 0x7f583c103d30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f582c009990 tx=0x7f582c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.847+0000 7f5842739700 1 -- 172.21.15.33:0/45584273 shutdown_connections 2024-05-08T21:23:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.847+0000 7f5842739700 1 --2- 172.21.15.33:0/45584273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f583c106ba0 0x7f583c109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.847+0000 7f5842739700 1 --2- 172.21.15.33:0/45584273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f583c101940 0x7f583c103d30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.847+0000 7f5842739700 1 --2- 172.21.15.33:0/45584273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f583c104270 0x7f583c106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.847+0000 7f5842739700 1 -- 172.21.15.33:0/45584273 >> 172.21.15.33:0/45584273 conn(0x7f583c0fb330 msgr2=0x7f583c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.847+0000 7f5842739700 1 -- 172.21.15.33:0/45584273 shutdown_connections 2024-05-08T21:23:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.847+0000 7f5842739700 1 -- 172.21.15.33:0/45584273 wait complete. 2024-05-08T21:23:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.848+0000 7f5842739700 1 Processor -- start 2024-05-08T21:23:55.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.848+0000 7f5842739700 1 -- start start 2024-05-08T21:23:55.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.848+0000 7f5842739700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f583c101940 0x7f583c101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.848+0000 7f5842739700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f583c104270 0x7f583c0ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.848+0000 7f5842739700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f583c106ba0 0x7f583c0ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.848+0000 7f5842739700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c1005d0 con 0x7f583c106ba0 2024-05-08T21:23:55.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.848+0000 7f5842739700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c100740 con 0x7f583c104270 2024-05-08T21:23:55.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.848+0000 7f5842739700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c1a0db0 con 0x7f583c101940 2024-05-08T21:23:55.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f583bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f583c101940 0x7f583c101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f5840cd6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f583c106ba0 0x7f583c0ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f583b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f583c104270 0x7f583c0ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f583bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f583c101940 0x7f583c101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43574/0 (socket says 172.21.15.33:43574) 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f5840cd6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f583c106ba0 0x7f583c0ffe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37186/0 (socket says 172.21.15.33:37186) 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f583bfff700 1 -- 172.21.15.33:0/1666405894 learned_addr learned my addr 172.21.15.33:0/1666405894 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f583b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f583c104270 0x7f583c0ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57808/0 (socket says 172.21.15.33:57808) 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f583bfff700 1 -- 172.21.15.33:0/1666405894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f583c104270 msgr2=0x7f583c0ff8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f583bfff700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f583c104270 0x7f583c0ff8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f583bfff700 1 -- 172.21.15.33:0/1666405894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f583c106ba0 msgr2=0x7f583c0ffe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f583bfff700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f583c106ba0 0x7f583c0ffe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f583bfff700 1 -- 172.21.15.33:0/1666405894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5834009c40 con 0x7f583c101940 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f583b7fe700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f583c104270 0x7f583c0ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f583bfff700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f583c101940 0x7f583c101260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f582c003ec0 tx=0x7f582c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f58397fa700 1 -- 172.21.15.33:0/1666405894 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c01d070 con 0x7f583c101940 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f58397fa700 1 -- 172.21.15.33:0/1666405894 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f582c00bdb0 con 0x7f583c101940 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f5842739700 1 -- 172.21.15.33:0/1666405894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c0095f0 con 0x7f583c101940 2024-05-08T21:23:55.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f58397fa700 1 -- 172.21.15.33:0/1666405894 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c00fe50 con 0x7f583c101940 2024-05-08T21:23:55.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.849+0000 7f5842739700 1 -- 172.21.15.33:0/1666405894 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583c1a11c0 con 0x7f583c101940 2024-05-08T21:23:55.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:55 smithi082 ceph-mon[47512]: pgmap v438: 129 pgs: 129 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.4 MiB/s wr, 304 op/s 2024-05-08T21:23:55.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:55 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1026947746' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:23:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.852+0000 7f58397fa700 1 -- 172.21.15.33:0/1666405894 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f582c00f5f0 con 0x7f583c101940 2024-05-08T21:23:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.852+0000 7f58397fa700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f58240a1120 0x7f58240a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.853+0000 7f58397fa700 1 -- 172.21.15.33:0/1666405894 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f582c0cdf00 con 0x7f583c101940 2024-05-08T21:23:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.853+0000 7f583b7fe700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f58240a1120 0x7f58240a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.853+0000 7f583b7fe700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f58240a1120 0x7f58240a35e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5830011490 tx=0x7f5830009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f584e8f3b80 1 Processor -- start 2024-05-08T21:23:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f584e8f3b80 1 -- start start 2024-05-08T21:23:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f584e8f3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06b540 0x5604cb06b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f584e8f3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb06c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f584e8f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06c860 0x5604cb06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f584e8f3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604cafc2150 con 0x5604cb06bea0 2024-05-08T21:23:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f584e8f3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604cafbfb10 con 0x5604cb06b540 2024-05-08T21:23:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f584e8f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604cafbb400 con 0x5604cb06c860 2024-05-08T21:23:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f583b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb06c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f583bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06b540 0x5604cb06b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f583bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06b540 0x5604cb06b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57812/0 (socket says 172.21.15.33:57812) 2024-05-08T21:23:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f583b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb06c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37192/0 (socket says 172.21.15.33:37192) 2024-05-08T21:23:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f583bfff700 1 -- 172.21.15.33:0/3858524511 learned_addr learned my addr 172.21.15.33:0/3858524511 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.854+0000 7f5840cd6700 1 --2- 172.21.15.33:0/3858524511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06c860 0x5604cb06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.855+0000 7f583bfff700 1 -- 172.21.15.33:0/3858524511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06c860 msgr2=0x5604cb06cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.855+0000 7f583bfff700 1 --2- 172.21.15.33:0/3858524511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06c860 0x5604cb06cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.855+0000 7f583bfff700 1 -- 172.21.15.33:0/3858524511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 msgr2=0x5604cb06c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.855+0000 7f583bfff700 1 --2- 172.21.15.33:0/3858524511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb06c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.855+0000 7f583bfff700 1 -- 172.21.15.33:0/3858524511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604cae3c3b0 con 0x5604cb06b540 2024-05-08T21:23:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.855+0000 7f583bfff700 1 --2- 172.21.15.33:0/3858524511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06b540 0x5604cb06b960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f582c097ad0 tx=0x7f582c09af00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe631e74416b573 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.855+0000 7f58227fc700 1 -- 172.21.15.33:0/3858524511 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0aeea0 con 0x5604cb06b540 2024-05-08T21:23:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.855+0000 7f58227fc700 1 -- 172.21.15.33:0/3858524511 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f582c09b760 con 0x5604cb06b540 2024-05-08T21:23:55.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.855+0000 7f58227fc700 1 -- 172.21.15.33:0/3858524511 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0a1070 con 0x5604cb06b540 2024-05-08T21:23:55.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.856+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3858524511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06b540 msgr2=0x5604cb06b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.856+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3858524511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06b540 0x5604cb06b960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f582c097ad0 tx=0x7f582c09af00 comp rx=0 tx=0).stop 2024-05-08T21:23:55.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.856+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3858524511 shutdown_connections 2024-05-08T21:23:55.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.856+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3858524511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06c860 0x5604cb06cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.856+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3858524511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06b540 0x5604cb06b960 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.856+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3858524511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb06c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.856+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3858524511 >> 172.21.15.33:0/3858524511 conn(0x5604cafc9210 msgr2=0x5604cb065de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.856+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3858524511 shutdown_connections 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.856+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3858524511 wait complete. 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.856+0000 7f584e8f3b80 1 -- start start 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f584e8f3b80 1 Processor -- start 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f584e8f3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06b540 0x5604cb074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f584e8f3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb075010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f584e8f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06c860 0x5604cb07c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f584e8f3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604caf57560 con 0x5604cb06bea0 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f584e8f3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604caf57200 con 0x5604cb06b540 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f584e8f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604cae3e6a0 con 0x5604cb06c860 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f584e8f3b80 10 client.0 _ll_register_callbacks cb 0x5604caf60b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f583b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb075010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f583b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb075010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37198/0 (socket says 172.21.15.33:37198) 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f583b7fe700 1 -- 172.21.15.33:0/3909230670 learned_addr learned my addr 172.21.15.33:0/3909230670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f5840cd6700 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06c860 0x5604cb07c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.857+0000 7f583bfff700 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06b540 0x5604cb074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f5840cd6700 1 -- 172.21.15.33:0/3909230670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06b540 msgr2=0x5604cb074ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f5840cd6700 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06b540 0x5604cb074ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f5840cd6700 1 -- 172.21.15.33:0/3909230670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 msgr2=0x5604cb075010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f5840cd6700 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb075010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f5840cd6700 1 -- 172.21.15.33:0/3909230670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583c1a11c0 con 0x5604cb06c860 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f5840cd6700 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06c860 0x5604cb07c050 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5834009fd0 tx=0x7f583400e610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b66c542e2a66e941 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f581b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f581b7fe700 1 -- 172.21.15.33:0/3909230670 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583400e040 con 0x5604cb06c860 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f581b7fe700 1 -- 172.21.15.33:0/3909230670 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f583400eaf0 con 0x5604cb06c860 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3909230670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604caf14270 con 0x5604cb06c860 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f581b7fe700 1 -- 172.21.15.33:0/3909230670 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f583400ecb0 con 0x5604cb06c860 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f584e8f3b80 10 client.26827 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3909230670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5604cade8b60 con 0x5604cb06c860 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.858+0000 7f581b7fe700 1 -- 172.21.15.33:0/3909230670 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5604caf14270 con 0x5604cb06c860 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.859+0000 7f581b7fe700 1 -- 172.21.15.33:0/3909230670 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f583401c020 con 0x5604cb06c860 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.859+0000 7f584e8f3b80 10 client.26827 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:55.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.859+0000 7f584e8f3b80 10 client.26827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.859+0000 7f584e8f3b80 20 client.26827 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.859+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5604cb080990 0x5604cb082db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.859+0000 7f584e8f3b80 4 client.26827 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.859+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3909230670 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604cb03a810 con 0x5604cb080990 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.859+0000 7f583bfff700 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5604cb080990 0x5604cb082db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.859+0000 7f583bfff700 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5604cb080990 0x5604cb082db0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e32cb19b8a0b9a server_cookie=326bc983bb490d2 in_seq=0 out_seq=0 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.859+0000 7f581b7fe700 10 client.26827 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f581b7fe700 1 -- 172.21.15.33:0/3909230670 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5604cb03a810 con 0x5604cb080990 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f581b7fe700 10 client.26827 handle_command_reply: tid=0 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f584e8f3b80 1 client.26827 shutdown 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3909230670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06c860 msgr2=0x5604cb07c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06c860 0x5604cb07c050 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5834009fd0 tx=0x7f583400e610 comp rx=0 tx=0).stop 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3909230670 shutdown_connections 2024-05-08T21:23:55.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5604cb080990 0x5604cb082db0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06c860 0x5604cb07c050 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06b540 0x5604cb074ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3909230670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb075010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f581b7fe700 0 client.26827 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3909230670 >> 172.21.15.33:0/3909230670 conn(0x5604cafc9210 msgr2=0x5604cb0655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3909230670 shutdown_connections 2024-05-08T21:23:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3909230670 wait complete. 2024-05-08T21:23:55.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.860+0000 7f584e8f3b80 20 client.26827 trim_cache size 0 max 16384 2024-05-08T21:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.876+0000 7f584e8f3b80 1 Processor -- start 2024-05-08T21:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.876+0000 7f584e8f3b80 1 -- start start 2024-05-08T21:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.876+0000 7f584e8f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06b540 0x5604cb073770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.876+0000 7f584e8f3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb07c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.876+0000 7f584e8f3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06c860 0x5604cb065cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.876+0000 7f584e8f3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604cb074640 con 0x5604cb06bea0 2024-05-08T21:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.876+0000 7f584e8f3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604cb07dd60 con 0x5604cb06c860 2024-05-08T21:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.876+0000 7f584e8f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604cb062dc0 con 0x5604cb06b540 2024-05-08T21:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.876+0000 7f583b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb07c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.876+0000 7f583b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb07c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37204/0 (socket says 172.21.15.33:37204) 2024-05-08T21:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.877+0000 7f583bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06b540 0x5604cb073770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.877+0000 7f583b7fe700 1 -- 172.21.15.33:0/1970604717 learned_addr learned my addr 172.21.15.33:0/1970604717 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.877+0000 7f5840cd6700 1 --2- 172.21.15.33:0/1970604717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06c860 0x5604cb065cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.877+0000 7f583bfff700 1 -- 172.21.15.33:0/1970604717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06c860 msgr2=0x5604cb065cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.877+0000 7f583bfff700 1 --2- 172.21.15.33:0/1970604717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06c860 0x5604cb065cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.877+0000 7f583bfff700 1 -- 172.21.15.33:0/1970604717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 msgr2=0x5604cb07c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.877+0000 7f583bfff700 1 --2- 172.21.15.33:0/1970604717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb07c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.877+0000 7f583bfff700 1 -- 172.21.15.33:0/1970604717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604caf81560 con 0x5604cb06b540 2024-05-08T21:23:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.877+0000 7f583bfff700 1 --2- 172.21.15.33:0/1970604717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06b540 0x5604cb073770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f582c005b40 tx=0x7f582c09b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e0b861fd502ad7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.877+0000 7f5822ffd700 1 -- 172.21.15.33:0/1970604717 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0aeea0 con 0x5604cb06b540 2024-05-08T21:23:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.877+0000 7f5822ffd700 1 -- 172.21.15.33:0/1970604717 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f582c09b060 con 0x5604cb06b540 2024-05-08T21:23:55.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.877+0000 7f5822ffd700 1 -- 172.21.15.33:0/1970604717 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0a1140 con 0x5604cb06b540 2024-05-08T21:23:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.878+0000 7f584e8f3b80 1 -- 172.21.15.33:0/1970604717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06b540 msgr2=0x5604cb073770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.878+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/1970604717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06b540 0x5604cb073770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f582c005b40 tx=0x7f582c09b8d0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.878+0000 7f584e8f3b80 1 -- 172.21.15.33:0/1970604717 shutdown_connections 2024-05-08T21:23:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.878+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/1970604717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06c860 0x5604cb065cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.878+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/1970604717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06b540 0x5604cb073770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.878+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/1970604717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06bea0 0x5604cb07c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.878+0000 7f584e8f3b80 1 -- 172.21.15.33:0/1970604717 >> 172.21.15.33:0/1970604717 conn(0x5604cafc9210 msgr2=0x5604cb06cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.878+0000 7f584e8f3b80 1 -- 172.21.15.33:0/1970604717 shutdown_connections 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.878+0000 7f584e8f3b80 1 -- 172.21.15.33:0/1970604717 wait complete. 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.879+0000 7f584e8f3b80 1 -- start start 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.879+0000 7f584e8f3b80 1 Processor -- start 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.879+0000 7f584e8f3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06b540 0x5604cb077620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.879+0000 7f584e8f3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06bea0 0x5604cb077b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.879+0000 7f584e8f3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06c860 0x5604cb07ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.879+0000 7f584e8f3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604cb07b070 con 0x5604cb06b540 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.879+0000 7f584e8f3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604cb07b1e0 con 0x5604cb06c860 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.879+0000 7f584e8f3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604cb07b3a0 con 0x5604cb06bea0 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.879+0000 7f584e8f3b80 10 client.0 _ll_register_callbacks cb 0x5604caf60b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.879+0000 7f583bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06b540 0x5604cb077620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f583bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06b540 0x5604cb077620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37214/0 (socket says 172.21.15.33:37214) 2024-05-08T21:23:55.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f583bfff700 1 -- 172.21.15.33:0/3857264452 learned_addr learned my addr 172.21.15.33:0/3857264452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f583b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06bea0 0x5604cb077b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f5840cd6700 1 --2- 172.21.15.33:0/3857264452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06c860 0x5604cb07ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f583bfff700 1 -- 172.21.15.33:0/3857264452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06bea0 msgr2=0x5604cb077b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f583bfff700 1 --2- 172.21.15.33:0/3857264452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06bea0 0x5604cb077b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f583bfff700 1 -- 172.21.15.33:0/3857264452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06c860 msgr2=0x5604cb07ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f583bfff700 1 --2- 172.21.15.33:0/3857264452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06c860 0x5604cb07ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f583bfff700 1 -- 172.21.15.33:0/3857264452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c0b2040 con 0x5604cb06b540 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f583bfff700 1 --2- 172.21.15.33:0/3857264452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06b540 0x5604cb077620 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f582c09aff0 tx=0x7f582c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6424adb5b7b6237 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f581b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f581b7fe700 1 -- 172.21.15.33:0/3857264452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c0aeea0 con 0x5604cb06b540 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f581b7fe700 1 -- 172.21.15.33:0/3857264452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f582c0a1120 con 0x5604cb06b540 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3857264452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f582c09b760 con 0x5604cb06b540 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f581b7fe700 1 -- 172.21.15.33:0/3857264452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f582c09a550 con 0x5604cb06b540 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f584e8f3b80 10 client.16941 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.880+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3857264452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5604cae40cd0 con 0x5604cb06b540 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.881+0000 7f581b7fe700 1 -- 172.21.15.33:0/3857264452 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f582c09b760 con 0x5604cb06b540 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.881+0000 7f581b7fe700 1 -- 172.21.15.33:0/3857264452 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f582c0b3b80 con 0x5604cb06b540 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.881+0000 7f584e8f3b80 10 client.16941 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.881+0000 7f584e8f3b80 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.881+0000 7f584e8f3b80 20 client.16941 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.881+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3857264452 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5604cb0d7090 0x5604cb0d94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.881+0000 7f584e8f3b80 4 client.16941 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.881+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3857264452 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5604cafb3ec0 con 0x5604cb0d7090 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.881+0000 7f583b7fe700 1 --2- 172.21.15.33:0/3857264452 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5604cb0d7090 0x5604cb0d94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.881+0000 7f581b7fe700 10 client.16941 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.881+0000 7f583b7fe700 1 --2- 172.21.15.33:0/3857264452 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5604cb0d7090 0x5604cb0d94b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f65871fd77e97a9a server_cookie=f5f166529f2799c2 in_seq=0 out_seq=0 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f581b7fe700 1 -- 172.21.15.33:0/3857264452 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5604cafb3ec0 con 0x5604cb0d7090 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f581b7fe700 10 client.16941 handle_command_reply: tid=0 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f584e8f3b80 1 client.16941 shutdown 2024-05-08T21:23:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3857264452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06b540 msgr2=0x5604cb077620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3857264452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06b540 0x5604cb077620 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f582c09aff0 tx=0x7f582c005c00 comp rx=0 tx=0).stop 2024-05-08T21:23:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3857264452 shutdown_connections 2024-05-08T21:23:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3857264452 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5604cb0d7090 0x5604cb0d94b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3857264452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604cb06c860 0x5604cb07ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3857264452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604cb06b540 0x5604cb077620 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f581b7fe700 0 client.16941 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f584e8f3b80 1 --2- 172.21.15.33:0/3857264452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604cb06bea0 0x5604cb077b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3857264452 >> 172.21.15.33:0/3857264452 conn(0x5604cafc9210 msgr2=0x5604cb06f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3857264452 shutdown_connections 2024-05-08T21:23:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f584e8f3b80 1 -- 172.21.15.33:0/3857264452 wait complete. 2024-05-08T21:23:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.882+0000 7f584e8f3b80 20 client.16941 trim_cache size 0 max 16384 2024-05-08T21:23:55.890 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:23:55.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.883+0000 7f5842739700 1 -- 172.21.15.33:0/1666405894 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f58240a1120 msgr2=0x7f58240a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.883+0000 7f5842739700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f58240a1120 0x7f58240a35e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5830011490 tx=0x7f5830009260 comp rx=0 tx=0).stop 2024-05-08T21:23:55.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.883+0000 7f5842739700 1 -- 172.21.15.33:0/1666405894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f583c101940 msgr2=0x7f583c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:55.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.883+0000 7f5842739700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f583c101940 0x7f583c101260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f582c003ec0 tx=0x7f582c005740 comp rx=0 tx=0).stop 2024-05-08T21:23:55.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.883+0000 7f5842739700 1 -- 172.21.15.33:0/1666405894 shutdown_connections 2024-05-08T21:23:55.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.883+0000 7f5842739700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f583c106ba0 0x7f583c0ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.883+0000 7f5842739700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f583c101940 0x7f583c101260 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.883+0000 7f5842739700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f58240a1120 0x7f58240a35e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.883+0000 7f5842739700 1 --2- 172.21.15.33:0/1666405894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f583c104270 0x7f583c0ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:55.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.883+0000 7f5842739700 1 -- 172.21.15.33:0/1666405894 >> 172.21.15.33:0/1666405894 conn(0x7f583c0fb330 msgr2=0x7f583c102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:55.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.883+0000 7f5842739700 1 -- 172.21.15.33:0/1666405894 shutdown_connections 2024-05-08T21:23:55.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:55.883+0000 7f5842739700 1 -- 172.21.15.33:0/1666405894 wait complete. 2024-05-08T21:23:56.900 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:23:56.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:23:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.057+0000 7fd726d4c700 1 -- 172.21.15.33:0/3119830915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7200ff750 msgr2=0x7fd7200ffb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.057+0000 7fd726d4c700 1 --2- 172.21.15.33:0/3119830915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7200ff750 0x7fd7200ffb70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd714009990 tx=0x7fd714009ca0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.057+0000 7fd726d4c700 1 -- 172.21.15.33:0/3119830915 shutdown_connections 2024-05-08T21:23:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.057+0000 7fd726d4c700 1 --2- 172.21.15.33:0/3119830915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd720100a70 0x7fd720100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.057+0000 7fd726d4c700 1 --2- 172.21.15.33:0/3119830915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7200ff750 0x7fd7200ffb70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.057+0000 7fd726d4c700 1 --2- 172.21.15.33:0/3119830915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7201000b0 0x7fd720100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.057+0000 7fd726d4c700 1 -- 172.21.15.33:0/3119830915 >> 172.21.15.33:0/3119830915 conn(0x7fd7200fb290 msgr2=0x7fd7200fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.057+0000 7fd726d4c700 1 -- 172.21.15.33:0/3119830915 shutdown_connections 2024-05-08T21:23:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.057+0000 7fd726d4c700 1 -- 172.21.15.33:0/3119830915 wait complete. 2024-05-08T21:23:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.058+0000 7fd726d4c700 1 Processor -- start 2024-05-08T21:23:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.058+0000 7fd726d4c700 1 -- start start 2024-05-08T21:23:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.058+0000 7fd726d4c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7200ff750 0x7fd720198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.058+0000 7fd726d4c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7201000b0 0x7fd720199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.058+0000 7fd726d4c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd720100a70 0x7fd7201a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.058+0000 7fd726d4c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7201a0a20 con 0x7fd7201000b0 2024-05-08T21:23:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.058+0000 7fd726d4c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7201a0b60 con 0x7fd720100a70 2024-05-08T21:23:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.058+0000 7fd726d4c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7201a0cf0 con 0x7fd7200ff750 2024-05-08T21:23:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.058+0000 7fd71ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7201000b0 0x7fd720199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd71ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7201000b0 0x7fd720199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37226/0 (socket says 172.21.15.33:37226) 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd71ffff700 1 -- 172.21.15.33:0/357657176 learned_addr learned my addr 172.21.15.33:0/357657176 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd724ae8700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7200ff750 0x7fd720198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd7252e9700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd720100a70 0x7fd7201a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd71ffff700 1 -- 172.21.15.33:0/357657176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7200ff750 msgr2=0x7fd720198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd71ffff700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7200ff750 0x7fd720198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd71ffff700 1 -- 172.21.15.33:0/357657176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd720100a70 msgr2=0x7fd7201a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd71ffff700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd720100a70 0x7fd7201a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd71ffff700 1 -- 172.21.15.33:0/357657176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7140095f0 con 0x7fd7201000b0 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd724ae8700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7200ff750 0x7fd720198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd71ffff700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7201000b0 0x7fd720199480 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00e9a0 tx=0x7fd70c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd71dffb700 1 -- 172.21.15.33:0/357657176 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c009790 con 0x7fd7201000b0 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd71dffb700 1 -- 172.21.15.33:0/357657176 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd70c010430 con 0x7fd7201000b0 2024-05-08T21:23:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd726d4c700 1 -- 172.21.15.33:0/357657176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7201a0fd0 con 0x7fd7201000b0 2024-05-08T21:23:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.059+0000 7fd71dffb700 1 -- 172.21.15.33:0/357657176 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00cea0 con 0x7fd7201000b0 2024-05-08T21:23:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.060+0000 7fd726d4c700 1 -- 172.21.15.33:0/357657176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7201a1450 con 0x7fd7201000b0 2024-05-08T21:23:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.062+0000 7fd71dffb700 1 -- 172.21.15.33:0/357657176 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd70c0105a0 con 0x7fd7201000b0 2024-05-08T21:23:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.063+0000 7fd71dffb700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7100a1120 0x7fd7100a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.063+0000 7fd71dffb700 1 -- 172.21.15.33:0/357657176 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd70c014070 con 0x7fd7201000b0 2024-05-08T21:23:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.063+0000 7fd724ae8700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7100a1120 0x7fd7100a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.064+0000 7fd724ae8700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7100a1120 0x7fd7100a35e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd714005910 tx=0x7fd71400b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:57.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd732f06b80 1 Processor -- start 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd732f06b80 1 -- start start 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd732f06b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4d5c0 0x55cea2d4d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd732f06b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4df20 0x55cea2d4e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd732f06b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4e8e0 0x55cea2d4ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd732f06b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea2ca41d0 con 0x55cea2d4d5c0 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd732f06b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea2ca1b90 con 0x55cea2d4e8e0 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd732f06b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea2c9d480 con 0x55cea2d4df20 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd71ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4df20 0x55cea2d4e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd71ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4df20 0x55cea2d4e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43642/0 (socket says 172.21.15.33:43642) 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd71ffff700 1 -- 172.21.15.33:0/1902783234 learned_addr learned my addr 172.21.15.33:0/1902783234 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd724ae8700 1 --2- 172.21.15.33:0/1902783234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4d5c0 0x55cea2d4d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd7252e9700 1 --2- 172.21.15.33:0/1902783234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4e8e0 0x55cea2d4ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd71ffff700 1 -- 172.21.15.33:0/1902783234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4e8e0 msgr2=0x55cea2d4ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd71ffff700 1 --2- 172.21.15.33:0/1902783234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4e8e0 0x55cea2d4ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd71ffff700 1 -- 172.21.15.33:0/1902783234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4d5c0 msgr2=0x55cea2d4d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd71ffff700 1 --2- 172.21.15.33:0/1902783234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4d5c0 0x55cea2d4d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.065+0000 7fd71ffff700 1 -- 172.21.15.33:0/1902783234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cea2b1e430 con 0x55cea2d4df20 2024-05-08T21:23:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.066+0000 7fd71ffff700 1 --2- 172.21.15.33:0/1902783234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4df20 0x55cea2d4e3a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0a1e50 tx=0x7fd70c099930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d51fdb18e86e230 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.066+0000 7fd70affd700 1 -- 172.21.15.33:0/1902783234 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c09a600 con 0x55cea2d4df20 2024-05-08T21:23:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.066+0000 7fd70affd700 1 -- 172.21.15.33:0/1902783234 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd70c099f00 con 0x55cea2d4df20 2024-05-08T21:23:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.066+0000 7fd70affd700 1 -- 172.21.15.33:0/1902783234 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c099630 con 0x55cea2d4df20 2024-05-08T21:23:57.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.067+0000 7fd732f06b80 1 -- 172.21.15.33:0/1902783234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4df20 msgr2=0x55cea2d4e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.067+0000 7fd732f06b80 1 --2- 172.21.15.33:0/1902783234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4df20 0x55cea2d4e3a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0a1e50 tx=0x7fd70c099930 comp rx=0 tx=0).stop 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.067+0000 7fd732f06b80 1 -- 172.21.15.33:0/1902783234 shutdown_connections 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.067+0000 7fd732f06b80 1 --2- 172.21.15.33:0/1902783234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4e8e0 0x55cea2d4ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.067+0000 7fd732f06b80 1 --2- 172.21.15.33:0/1902783234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4d5c0 0x55cea2d4d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.067+0000 7fd732f06b80 1 --2- 172.21.15.33:0/1902783234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4df20 0x55cea2d4e3a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.067+0000 7fd732f06b80 1 -- 172.21.15.33:0/1902783234 >> 172.21.15.33:0/1902783234 conn(0x55cea2cab290 msgr2=0x55cea2d47e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.067+0000 7fd732f06b80 1 -- 172.21.15.33:0/1902783234 shutdown_connections 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.067+0000 7fd732f06b80 1 -- 172.21.15.33:0/1902783234 wait complete. 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.067+0000 7fd732f06b80 1 -- start start 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.067+0000 7fd732f06b80 1 Processor -- start 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd732f06b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4d5c0 0x55cea2d56e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd732f06b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4df20 0x55cea2d573b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd732f06b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4e8e0 0x55cea2d5e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd732f06b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea2c395e0 con 0x55cea2d4df20 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd732f06b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea2c39280 con 0x55cea2d4d5c0 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd732f06b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea2b20720 con 0x55cea2d4e8e0 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd732f06b80 10 client.0 _ll_register_callbacks cb 0x55cea2c42bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd71ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4df20 0x55cea2d573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd71ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4df20 0x55cea2d573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37246/0 (socket says 172.21.15.33:37246) 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd71ffff700 1 -- 172.21.15.33:0/654209526 learned_addr learned my addr 172.21.15.33:0/654209526 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd7252e9700 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4e8e0 0x55cea2d5e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd724ae8700 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4d5c0 0x55cea2d56e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd7252e9700 1 -- 172.21.15.33:0/654209526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4d5c0 msgr2=0x55cea2d56e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd7252e9700 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4d5c0 0x55cea2d56e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd7252e9700 1 -- 172.21.15.33:0/654209526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4df20 msgr2=0x55cea2d573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd7252e9700 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4df20 0x55cea2d573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd7252e9700 1 -- 172.21.15.33:0/654209526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7201a1450 con 0x55cea2d4e8e0 2024-05-08T21:23:57.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.068+0000 7fd71ffff700 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4df20 0x55cea2d573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd7252e9700 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4e8e0 0x55cea2d5e3f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd718012d90 tx=0x7fd718014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5296a842f7db0e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd6fffff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd6fffff700 1 -- 172.21.15.33:0/654209526 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71800e040 con 0x55cea2d4e8e0 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd6fffff700 1 -- 172.21.15.33:0/654209526 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd718014a80 con 0x55cea2d4e8e0 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd732f06b80 1 -- 172.21.15.33:0/654209526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cea2bf62f0 con 0x55cea2d4e8e0 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd6fffff700 1 -- 172.21.15.33:0/654209526 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd718014c40 con 0x55cea2d4e8e0 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd732f06b80 10 client.26845 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd732f06b80 1 -- 172.21.15.33:0/654209526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55cea2acabe0 con 0x55cea2d4e8e0 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd6fffff700 1 -- 172.21.15.33:0/654209526 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cea2bf62f0 con 0x55cea2d4e8e0 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd6fffff700 1 -- 172.21.15.33:0/654209526 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fd71801c020 con 0x55cea2d4e8e0 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd732f06b80 10 client.26845 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd732f06b80 10 client.26845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd732f06b80 20 client.26845 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd732f06b80 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cea2d62d80 0x55cea2d651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd732f06b80 4 client.26845 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.069+0000 7fd732f06b80 1 -- 172.21.15.33:0/654209526 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cea2d1c890 con 0x55cea2d62d80 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.070+0000 7fd724ae8700 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cea2d62d80 0x55cea2d651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.070+0000 7fd724ae8700 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cea2d62d80 0x55cea2d651a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b608b4d51314aa3 server_cookie=cd20d8929a54f4bf in_seq=0 out_seq=0 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.070+0000 7fd6fffff700 10 client.26845 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd6fffff700 1 -- 172.21.15.33:0/654209526 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cea2d1c890 con 0x55cea2d62d80 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd6fffff700 10 client.26845 handle_command_reply: tid=0 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd732f06b80 1 client.26845 shutdown 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd732f06b80 1 -- 172.21.15.33:0/654209526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4e8e0 msgr2=0x55cea2d5e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd732f06b80 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4e8e0 0x55cea2d5e3f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd718012d90 tx=0x7fd718014600 comp rx=0 tx=0).stop 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd732f06b80 1 -- 172.21.15.33:0/654209526 shutdown_connections 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd732f06b80 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cea2d62d80 0x55cea2d651a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd732f06b80 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4e8e0 0x55cea2d5e3f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd732f06b80 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4d5c0 0x55cea2d56e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd6fffff700 0 client.26845 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd732f06b80 1 --2- 172.21.15.33:0/654209526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4df20 0x55cea2d573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd732f06b80 1 -- 172.21.15.33:0/654209526 >> 172.21.15.33:0/654209526 conn(0x55cea2cab290 msgr2=0x55cea2d47620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd732f06b80 1 -- 172.21.15.33:0/654209526 shutdown_connections 2024-05-08T21:23:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd732f06b80 1 -- 172.21.15.33:0/654209526 wait complete. 2024-05-08T21:23:57.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.071+0000 7fd732f06b80 20 client.26845 trim_cache size 0 max 16384 2024-05-08T21:23:57.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.091+0000 7fd732f06b80 1 Processor -- start 2024-05-08T21:23:57.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.091+0000 7fd732f06b80 1 -- start start 2024-05-08T21:23:57.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd732f06b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4d5c0 0x55cea2d54f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd732f06b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 0x55cea2d5e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd732f06b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4e8e0 0x55cea2d518f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd732f06b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea2d42880 con 0x55cea2d4d5c0 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd732f06b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea2d498b0 con 0x55cea2d4df20 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd732f06b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea2d54bb0 con 0x55cea2d4e8e0 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd724ae8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4d5c0 0x55cea2d54f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd71ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 0x55cea2d5e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd724ae8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4d5c0 0x55cea2d54f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37248/0 (socket says 172.21.15.33:37248) 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd7252e9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4e8e0 0x55cea2d518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd724ae8700 1 -- 172.21.15.33:0/2453839449 learned_addr learned my addr 172.21.15.33:0/2453839449 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd71ffff700 1 -- 172.21.15.33:0/2453839449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4e8e0 msgr2=0x55cea2d518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd71ffff700 1 --2- 172.21.15.33:0/2453839449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4e8e0 0x55cea2d518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd71ffff700 1 -- 172.21.15.33:0/2453839449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4d5c0 msgr2=0x55cea2d54f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd71ffff700 1 --2- 172.21.15.33:0/2453839449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4d5c0 0x55cea2d54f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd71ffff700 1 -- 172.21.15.33:0/2453839449 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cea2c635e0 con 0x55cea2d4df20 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd7252e9700 1 --2- 172.21.15.33:0/2453839449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4e8e0 0x55cea2d518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.092+0000 7fd71ffff700 1 --2- 172.21.15.33:0/2453839449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 0x55cea2d5e3b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd70c099e40 tx=0x7fd70c099700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b632698476d36e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:57.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.093+0000 7fd70b7fe700 1 -- 172.21.15.33:0/2453839449 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c09a600 con 0x55cea2d4df20 2024-05-08T21:23:57.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.093+0000 7fd70b7fe700 1 -- 172.21.15.33:0/2453839449 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd70c098380 con 0x55cea2d4df20 2024-05-08T21:23:57.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.093+0000 7fd70b7fe700 1 -- 172.21.15.33:0/2453839449 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0a1ef0 con 0x55cea2d4df20 2024-05-08T21:23:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.093+0000 7fd732f06b80 1 -- 172.21.15.33:0/2453839449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 msgr2=0x55cea2d5e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.093+0000 7fd732f06b80 1 --2- 172.21.15.33:0/2453839449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 0x55cea2d5e3b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd70c099e40 tx=0x7fd70c099700 comp rx=0 tx=0).stop 2024-05-08T21:23:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.094+0000 7fd732f06b80 1 -- 172.21.15.33:0/2453839449 shutdown_connections 2024-05-08T21:23:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.094+0000 7fd732f06b80 1 --2- 172.21.15.33:0/2453839449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4e8e0 0x55cea2d518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.094+0000 7fd732f06b80 1 --2- 172.21.15.33:0/2453839449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4d5c0 0x55cea2d54f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.094+0000 7fd732f06b80 1 --2- 172.21.15.33:0/2453839449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 0x55cea2d5e3b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.094+0000 7fd732f06b80 1 -- 172.21.15.33:0/2453839449 >> 172.21.15.33:0/2453839449 conn(0x55cea2cab290 msgr2=0x55cea2d4ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.094+0000 7fd732f06b80 1 -- 172.21.15.33:0/2453839449 shutdown_connections 2024-05-08T21:23:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.094+0000 7fd732f06b80 1 -- 172.21.15.33:0/2453839449 wait complete. 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.094+0000 7fd732f06b80 1 -- start start 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.094+0000 7fd732f06b80 1 Processor -- start 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd732f06b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4d5c0 0x55cea2d5a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd732f06b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 0x55cea2d5aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd732f06b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4e8e0 0x55cea2d5d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd732f06b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea2d5df20 con 0x55cea2d4e8e0 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd732f06b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea2d5e090 con 0x55cea2d4df20 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd732f06b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cea2d5e250 con 0x55cea2d4d5c0 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd732f06b80 10 client.0 _ll_register_callbacks cb 0x55cea2c42bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd71ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 0x55cea2d5aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd7252e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4e8e0 0x55cea2d5d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd71ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 0x55cea2d5aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57894/0 (socket says 172.21.15.33:57894) 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd724ae8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4d5c0 0x55cea2d5a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd71ffff700 1 -- 172.21.15.33:0/402203397 learned_addr learned my addr 172.21.15.33:0/402203397 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd7252e9700 1 -- 172.21.15.33:0/402203397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4d5c0 msgr2=0x55cea2d5a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd7252e9700 1 --2- 172.21.15.33:0/402203397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4d5c0 0x55cea2d5a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd7252e9700 1 -- 172.21.15.33:0/402203397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 msgr2=0x55cea2d5aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd7252e9700 1 --2- 172.21.15.33:0/402203397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 0x55cea2d5aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd7252e9700 1 -- 172.21.15.33:0/402203397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cea2acabe0 con 0x55cea2d4e8e0 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd71ffff700 1 --2- 172.21.15.33:0/402203397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 0x55cea2d5aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd7252e9700 1 --2- 172.21.15.33:0/402203397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4e8e0 0x55cea2d5d9e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd718013010 tx=0x7fd718009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943b42462c0d5dd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.095+0000 7fd6fffff700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd6fffff700 1 -- 172.21.15.33:0/402203397 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71800e040 con 0x55cea2d4e8e0 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd6fffff700 1 -- 172.21.15.33:0/402203397 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd718004630 con 0x55cea2d4e8e0 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd732f06b80 1 -- 172.21.15.33:0/402203397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd70c099f00 con 0x55cea2d4e8e0 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd6fffff700 1 -- 172.21.15.33:0/402203397 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd718014d60 con 0x55cea2d4e8e0 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd732f06b80 10 client.16959 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:57.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd732f06b80 1 -- 172.21.15.33:0/402203397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55cea2b1e430 con 0x55cea2d4e8e0 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd6fffff700 1 -- 172.21.15.33:0/402203397 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd70c099f00 con 0x55cea2d4e8e0 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd6fffff700 1 -- 172.21.15.33:0/402203397 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fd718014320 con 0x55cea2d4e8e0 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd732f06b80 10 client.16959 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd732f06b80 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd732f06b80 20 client.16959 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd732f06b80 1 --2- 172.21.15.33:0/402203397 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cea2dba4c0 0x55cea2dbc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd732f06b80 4 client.16959 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.096+0000 7fd732f06b80 1 -- 172.21.15.33:0/402203397 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cea2c95f40 con 0x55cea2dba4c0 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.097+0000 7fd724ae8700 1 --2- 172.21.15.33:0/402203397 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cea2dba4c0 0x55cea2dbc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.097+0000 7fd6fffff700 10 client.16959 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.097+0000 7fd724ae8700 1 --2- 172.21.15.33:0/402203397 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cea2dba4c0 0x55cea2dbc8e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80ad861cfdcbeaeb server_cookie=f1a8decc44d6d6a2 in_seq=0 out_seq=0 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd6fffff700 1 -- 172.21.15.33:0/402203397 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cea2c95f40 con 0x55cea2dba4c0 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd6fffff700 10 client.16959 handle_command_reply: tid=0 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd732f06b80 1 client.16959 shutdown 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd732f06b80 1 -- 172.21.15.33:0/402203397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4e8e0 msgr2=0x55cea2d5d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd732f06b80 1 --2- 172.21.15.33:0/402203397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4e8e0 0x55cea2d5d9e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd718013010 tx=0x7fd718009510 comp rx=0 tx=0).stop 2024-05-08T21:23:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd732f06b80 1 -- 172.21.15.33:0/402203397 shutdown_connections 2024-05-08T21:23:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd732f06b80 1 --2- 172.21.15.33:0/402203397 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cea2dba4c0 0x55cea2dbc8e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd732f06b80 1 --2- 172.21.15.33:0/402203397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cea2d4e8e0 0x55cea2d5d9e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd732f06b80 1 --2- 172.21.15.33:0/402203397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cea2d4d5c0 0x55cea2d5a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd732f06b80 1 --2- 172.21.15.33:0/402203397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cea2d4df20 0x55cea2d5aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd732f06b80 1 -- 172.21.15.33:0/402203397 >> 172.21.15.33:0/402203397 conn(0x55cea2cab290 msgr2=0x55cea2d54ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd6fffff700 0 client.16959 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd732f06b80 1 -- 172.21.15.33:0/402203397 shutdown_connections 2024-05-08T21:23:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.098+0000 7fd732f06b80 1 -- 172.21.15.33:0/402203397 wait complete. 2024-05-08T21:23:57.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd732f06b80 20 client.16959 trim_cache size 0 max 16384 2024-05-08T21:23:57.104 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:23:57.105 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:23:57.105 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "d947cfbb-2855-4688-be0d-83321f2de4e6", 2024-05-08T21:23:57.105 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:23:57.105 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:23:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd726d4c700 1 -- 172.21.15.33:0/357657176 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7100a1120 msgr2=0x7fd7100a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd726d4c700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7100a1120 0x7fd7100a35e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd714005910 tx=0x7fd71400b5b0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd726d4c700 1 -- 172.21.15.33:0/357657176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7201000b0 msgr2=0x7fd720199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd726d4c700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7201000b0 0x7fd720199480 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00e9a0 tx=0x7fd70c00ecb0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd726d4c700 1 -- 172.21.15.33:0/357657176 shutdown_connections 2024-05-08T21:23:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd726d4c700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd7100a1120 0x7fd7100a35e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd726d4c700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd720100a70 0x7fd7201a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd726d4c700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7200ff750 0x7fd720198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd726d4c700 1 --2- 172.21.15.33:0/357657176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7201000b0 0x7fd720199480 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd726d4c700 1 -- 172.21.15.33:0/357657176 >> 172.21.15.33:0/357657176 conn(0x7fd7200fb290 msgr2=0x7fd7200fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd726d4c700 1 -- 172.21.15.33:0/357657176 shutdown_connections 2024-05-08T21:23:57.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.099+0000 7fd726d4c700 1 -- 172.21.15.33:0/357657176 wait complete. 2024-05-08T21:23:57.117 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:23:57.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.304+0000 7fe6a7fff700 1 -- 172.21.15.33:0/3275453383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a8100de0 msgr2=0x7fe6a8101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.304+0000 7fe6a7fff700 1 --2- 172.21.15.33:0/3275453383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a8100de0 0x7fe6a8101240 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009990 tx=0x7fe69c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.304+0000 7fe6a7fff700 1 -- 172.21.15.33:0/3275453383 shutdown_connections 2024-05-08T21:23:57.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.304+0000 7fe6a7fff700 1 --2- 172.21.15.33:0/3275453383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a8101780 0x7fe6a8108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.304+0000 7fe6a7fff700 1 --2- 172.21.15.33:0/3275453383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a80ffbf0 0x7fe6a8100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.304+0000 7fe6a7fff700 1 --2- 172.21.15.33:0/3275453383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a8100de0 0x7fe6a8101240 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.304+0000 7fe6a7fff700 1 -- 172.21.15.33:0/3275453383 >> 172.21.15.33:0/3275453383 conn(0x7fe6a80fb180 msgr2=0x7fe6a80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:57.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.304+0000 7fe6a7fff700 1 -- 172.21.15.33:0/3275453383 shutdown_connections 2024-05-08T21:23:57.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.304+0000 7fe6a7fff700 1 -- 172.21.15.33:0/3275453383 wait complete. 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.305+0000 7fe6a7fff700 1 Processor -- start 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.306+0000 7fe6a7fff700 1 -- start start 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.306+0000 7fe6a7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a80ffbf0 0x7fe6a81947e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.306+0000 7fe6a7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a8100de0 0x7fe6a8194d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.306+0000 7fe6a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a8101780 0x7fe6a819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.306+0000 7fe6a7fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a819c310 con 0x7fe6a8101780 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.306+0000 7fe6a7fff700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a819c480 con 0x7fe6a80ffbf0 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.306+0000 7fe6a7fff700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a819c640 con 0x7fe6a8100de0 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.306+0000 7fe6a77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a8101780 0x7fe6a819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.306+0000 7fe6a77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a8101780 0x7fe6a819bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37268/0 (socket says 172.21.15.33:37268) 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.306+0000 7fe6a77fe700 1 -- 172.21.15.33:0/516412583 learned_addr learned my addr 172.21.15.33:0/516412583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.306+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a8100de0 0x7fe6a8194d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.306+0000 7fe6a6ffd700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a80ffbf0 0x7fe6a81947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.307+0000 7fe6a77fe700 1 -- 172.21.15.33:0/516412583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a8100de0 msgr2=0x7fe6a8194d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.307+0000 7fe6a77fe700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a8100de0 0x7fe6a8194d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.307+0000 7fe6a77fe700 1 -- 172.21.15.33:0/516412583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a80ffbf0 msgr2=0x7fe6a81947e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.307+0000 7fe6a77fe700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a80ffbf0 0x7fe6a81947e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.307+0000 7fe6a77fe700 1 -- 172.21.15.33:0/516412583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c0095f0 con 0x7fe6a8101780 2024-05-08T21:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.307+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a8100de0 0x7fe6a8194d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.307+0000 7fe6a77fe700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a8101780 0x7fe6a819bd40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000ef30 tx=0x7fe6a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.307+0000 7fe6aca01700 1 -- 172.21.15.33:0/516412583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0018070 con 0x7fe6a8101780 2024-05-08T21:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.307+0000 7fe6aca01700 1 -- 172.21.15.33:0/516412583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6a000f040 con 0x7fe6a8101780 2024-05-08T21:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.307+0000 7fe6aca01700 1 -- 172.21.15.33:0/516412583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a000cc70 con 0x7fe6a8101780 2024-05-08T21:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.307+0000 7fe6a7fff700 1 -- 172.21.15.33:0/516412583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a819c860 con 0x7fe6a8101780 2024-05-08T21:23:57.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.308+0000 7fe6a7fff700 1 -- 172.21.15.33:0/516412583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a819cd20 con 0x7fe6a8101780 2024-05-08T21:23:57.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.311+0000 7fe6aca01700 1 -- 172.21.15.33:0/516412583 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe6a0007500 con 0x7fe6a8101780 2024-05-08T21:23:57.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.311+0000 7fe6aca01700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6900a12c0 0x7fe6900a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.312+0000 7fe6aca01700 1 -- 172.21.15.33:0/516412583 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe6a00cc7d0 con 0x7fe6a8101780 2024-05-08T21:23:57.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.312+0000 7fe6b9bbdb80 1 Processor -- start 2024-05-08T21:23:57.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.312+0000 7fe6b9bbdb80 1 -- start start 2024-05-08T21:23:57.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.312+0000 7fe6b9bbdb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7e620 0x556c96f7ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.312+0000 7fe6b9bbdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7ef80 0x556c96f7f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.312+0000 7fe6b9bbdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7f9a0 0x556c96f7fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.312+0000 7fe6b9bbdb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c96ed5150 con 0x556c96f7e620 2024-05-08T21:23:57.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.312+0000 7fe6b9bbdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c96ed2b10 con 0x556c96f7ef80 2024-05-08T21:23:57.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.312+0000 7fe6b9bbdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c96ece400 con 0x556c96f7f9a0 2024-05-08T21:23:57.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.312+0000 7fe6a6ffd700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6900a12c0 0x7fe6900a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.313+0000 7fe6a77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7f9a0 0x556c96f7fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.313+0000 7fe6a77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7f9a0 0x556c96f7fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43690/0 (socket says 172.21.15.33:43690) 2024-05-08T21:23:57.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.313+0000 7fe6a77fe700 1 -- 172.21.15.33:0/1850702498 learned_addr learned my addr 172.21.15.33:0/1850702498 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:57.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.313+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/1850702498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7ef80 0x556c96f7f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.313+0000 7fe6a6ffd700 1 --2- 172.21.15.33:0/1850702498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7e620 0x556c96f7ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.313+0000 7fe6a67fc700 1 -- 172.21.15.33:0/1850702498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7f9a0 msgr2=0x556c96f7fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.313+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/1850702498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7f9a0 0x556c96f7fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.313+0000 7fe6a67fc700 1 -- 172.21.15.33:0/1850702498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7e620 msgr2=0x556c96f7ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.313+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/1850702498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7e620 0x556c96f7ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.313+0000 7fe6a67fc700 1 -- 172.21.15.33:0/1850702498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c96d4f3b0 con 0x556c96f7ef80 2024-05-08T21:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.313+0000 7fe6a77fe700 1 --2- 172.21.15.33:0/1850702498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7f9a0 0x556c96f7fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.314+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/1850702498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7ef80 0x556c96f7f410 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe69c006010 tx=0x7fe69c005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c41550819664c98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.314+0000 7fe68dffb700 1 -- 172.21.15.33:0/1850702498 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c01d070 con 0x556c96f7ef80 2024-05-08T21:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.314+0000 7fe6a6ffd700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6900a12c0 0x7fe6900a3780 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe698005fd0 tx=0x7fe698012520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.314+0000 7fe68dffb700 1 -- 172.21.15.33:0/1850702498 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe69c01e040 con 0x556c96f7ef80 2024-05-08T21:23:57.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.314+0000 7fe68dffb700 1 -- 172.21.15.33:0/1850702498 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c005230 con 0x556c96f7ef80 2024-05-08T21:23:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.315+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/1850702498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7ef80 msgr2=0x556c96f7f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.315+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/1850702498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7ef80 0x556c96f7f410 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe69c006010 tx=0x7fe69c005c70 comp rx=0 tx=0).stop 2024-05-08T21:23:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.315+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/1850702498 shutdown_connections 2024-05-08T21:23:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.315+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/1850702498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7f9a0 0x556c96f7fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.315+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/1850702498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7e620 0x556c96f7ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.315+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/1850702498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7ef80 0x556c96f7f410 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.315+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/1850702498 >> 172.21.15.33:0/1850702498 conn(0x556c96edc210 msgr2=0x556c96f79140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.316+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/1850702498 shutdown_connections 2024-05-08T21:23:57.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.316+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/1850702498 wait complete. 2024-05-08T21:23:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.316+0000 7fe6b9bbdb80 1 -- start start 2024-05-08T21:23:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.316+0000 7fe6b9bbdb80 1 Processor -- start 2024-05-08T21:23:57.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.317+0000 7fe6b9bbdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7e620 0x556c96f99020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.317+0000 7fe6b9bbdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7ef80 0x556c96f99560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.317+0000 7fe6b9bbdb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7f9a0 0x556c96fa0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.317+0000 7fe6b9bbdb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c96e6a560 con 0x556c96f7f9a0 2024-05-08T21:23:57.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.317+0000 7fe6b9bbdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c96e6a200 con 0x556c96f7e620 2024-05-08T21:23:57.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.317+0000 7fe6b9bbdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c96d516a0 con 0x556c96f7ef80 2024-05-08T21:23:57.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.317+0000 7fe6b9bbdb80 10 client.0 _ll_register_callbacks cb 0x556c96e73b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:57.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.317+0000 7fe6a67fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7ef80 0x556c96f99560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.317+0000 7fe6a6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7e620 0x556c96f99020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.317+0000 7fe6a67fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7ef80 0x556c96f99560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43698/0 (socket says 172.21.15.33:43698) 2024-05-08T21:23:57.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.317+0000 7fe6a67fc700 1 -- 172.21.15.33:0/581583726 learned_addr learned my addr 172.21.15.33:0/581583726 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:57.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.317+0000 7fe6a77fe700 1 --2- 172.21.15.33:0/581583726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7f9a0 0x556c96fa0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.318+0000 7fe6a67fc700 1 -- 172.21.15.33:0/581583726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7e620 msgr2=0x556c96f99020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.318+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/581583726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7e620 0x556c96f99020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.318+0000 7fe6a67fc700 1 -- 172.21.15.33:0/581583726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7f9a0 msgr2=0x556c96fa0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.318+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/581583726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7f9a0 0x556c96fa0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.318+0000 7fe6a67fc700 1 -- 172.21.15.33:0/581583726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a819cd20 con 0x556c96f7ef80 2024-05-08T21:23:57.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.320+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/581583726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7ef80 0x556c96f99560 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe69c000c00 tx=0x7fe69c00b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35d9e6fc12307361 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:57.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.320+0000 7fe686ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:23:57.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.320+0000 7fe686ffd700 1 -- 172.21.15.33:0/581583726 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c01d030 con 0x556c96f7ef80 2024-05-08T21:23:57.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.320+0000 7fe686ffd700 1 -- 172.21.15.33:0/581583726 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe69c0047e0 con 0x556c96f7ef80 2024-05-08T21:23:57.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.321+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/581583726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c96e27270 con 0x556c96f7ef80 2024-05-08T21:23:57.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.321+0000 7fe686ffd700 1 -- 172.21.15.33:0/581583726 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe69c003a90 con 0x556c96f7ef80 2024-05-08T21:23:57.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.321+0000 7fe6b9bbdb80 10 client.26863 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:57.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.321+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/581583726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fe6a8108980 con 0x556c96f7ef80 2024-05-08T21:23:57.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.321+0000 7fe686ffd700 1 -- 172.21.15.33:0/581583726 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556c96e27270 con 0x556c96f7ef80 2024-05-08T21:23:57.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.321+0000 7fe686ffd700 1 -- 172.21.15.33:0/581583726 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fe69c00f460 con 0x556c96f7ef80 2024-05-08T21:23:57.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.321+0000 7fe6b9bbdb80 10 client.26863 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:57.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.321+0000 7fe6b9bbdb80 10 client.26863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:57.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.321+0000 7fe6b9bbdb80 20 client.26863 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:57.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.321+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/581583726 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556c96fa48b0 0x556c96fa6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.321+0000 7fe6b9bbdb80 4 client.26863 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:23:57.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.322+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/581583726 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c96f4d810 con 0x556c96fa48b0 2024-05-08T21:23:57.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.322+0000 7fe6a6ffd700 1 --2- 172.21.15.33:0/581583726 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556c96fa48b0 0x556c96fa6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.322+0000 7fe6a6ffd700 1 --2- 172.21.15.33:0/581583726 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556c96fa48b0 0x556c96fa6cd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232b2e8a434cb943 server_cookie=c9164e388a7e2ce in_seq=0 out_seq=0 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.323+0000 7fe686ffd700 10 client.26863 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.323+0000 7fe686ffd700 1 -- 172.21.15.33:0/581583726 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c96f4d810 con 0x556c96fa48b0 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.323+0000 7fe686ffd700 10 client.26863 handle_command_reply: tid=0 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.323+0000 7fe6b9bbdb80 1 client.26863 shutdown 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.323+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/581583726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7ef80 msgr2=0x556c96f99560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.323+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/581583726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7ef80 0x556c96f99560 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe69c000c00 tx=0x7fe69c00b410 comp rx=0 tx=0).stop 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.323+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/581583726 shutdown_connections 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.323+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/581583726 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556c96fa48b0 0x556c96fa6cd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.323+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/581583726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7f9a0 0x556c96fa0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.323+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/581583726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7e620 0x556c96f99020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.323+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/581583726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7ef80 0x556c96f99560 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.323+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/581583726 >> 172.21.15.33:0/581583726 conn(0x556c96edc210 msgr2=0x556c96f787b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:57.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.324+0000 7fe686ffd700 0 client.26863 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:57.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.324+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/581583726 shutdown_connections 2024-05-08T21:23:57.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.324+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/581583726 wait complete. 2024-05-08T21:23:57.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.324+0000 7fe6b9bbdb80 20 client.26863 trim_cache size 0 max 16384 2024-05-08T21:23:57.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6b9bbdb80 1 Processor -- start 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6b9bbdb80 1 -- start start 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6b9bbdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7e620 0x556c96f7fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6b9bbdb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7ef80 0x556c96fa0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6b9bbdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7f9a0 0x556c96fab630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6b9bbdb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c96f73a90 con 0x556c96f7ef80 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6b9bbdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c96fa16d0 con 0x556c96f7f9a0 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6b9bbdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c96f86830 con 0x556c96f7e620 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6a67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7ef80 0x556c96fa0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6a6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7e620 0x556c96f7fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6a6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7e620 0x556c96f7fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:43704/0 (socket says 172.21.15.33:43704) 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6a6ffd700 1 -- 172.21.15.33:0/3961567748 learned_addr learned my addr 172.21.15.33:0/3961567748 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6a67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7ef80 0x556c96fa0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37298/0 (socket says 172.21.15.33:37298) 2024-05-08T21:23:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6a77fe700 1 --2- 172.21.15.33:0/3961567748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7f9a0 0x556c96fab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6a6ffd700 1 -- 172.21.15.33:0/3961567748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7f9a0 msgr2=0x556c96fab630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6a6ffd700 1 --2- 172.21.15.33:0/3961567748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7f9a0 0x556c96fab630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6a6ffd700 1 -- 172.21.15.33:0/3961567748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7ef80 msgr2=0x556c96fa0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6a6ffd700 1 --2- 172.21.15.33:0/3961567748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7ef80 0x556c96fa0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6a6ffd700 1 -- 172.21.15.33:0/3961567748 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c96cfbb60 con 0x556c96f7e620 2024-05-08T21:23:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.350+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/3961567748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7ef80 0x556c96fa0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:23:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.351+0000 7fe6a6ffd700 1 --2- 172.21.15.33:0/3961567748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7e620 0x556c96f7fd80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8100c40 tx=0x7fe698005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92540ee44fed9d31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.351+0000 7fe68e7fc700 1 -- 172.21.15.33:0/3961567748 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698028e00 con 0x556c96f7e620 2024-05-08T21:23:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.351+0000 7fe68e7fc700 1 -- 172.21.15.33:0/3961567748 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe6980286e0 con 0x556c96f7e620 2024-05-08T21:23:57.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.351+0000 7fe68e7fc700 1 -- 172.21.15.33:0/3961567748 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69801f770 con 0x556c96f7e620 2024-05-08T21:23:57.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.352+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3961567748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7e620 msgr2=0x556c96f7fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.352+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/3961567748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7e620 0x556c96f7fd80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8100c40 tx=0x7fe698005e50 comp rx=0 tx=0).stop 2024-05-08T21:23:57.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.352+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3961567748 shutdown_connections 2024-05-08T21:23:57.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.352+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/3961567748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7f9a0 0x556c96fab630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.352+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/3961567748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7e620 0x556c96f7fd80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.352+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/3961567748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7ef80 0x556c96fa0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.352+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3961567748 >> 172.21.15.33:0/3961567748 conn(0x556c96edc210 msgr2=0x556c96fa68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.352+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3961567748 shutdown_connections 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.352+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3961567748 wait complete. 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.352+0000 7fe6b9bbdb80 1 -- start start 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6b9bbdb80 1 Processor -- start 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6b9bbdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7e620 0x556c96f9e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6b9bbdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7ef80 0x556c96f9e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6b9bbdb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7f9a0 0x556c96f9f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6b9bbdb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c96f9fa50 con 0x556c96f7f9a0 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6b9bbdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c96f9fbc0 con 0x556c96f7e620 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6b9bbdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c96f9fd80 con 0x556c96f7ef80 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6b9bbdb80 10 client.0 _ll_register_callbacks cb 0x556c96e73b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6a6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7e620 0x556c96f9e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6a77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7f9a0 0x556c96f9f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6a6ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7e620 0x556c96f9e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57952/0 (socket says 172.21.15.33:57952) 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6a6ffd700 1 -- 172.21.15.33:0/3393345479 learned_addr learned my addr 172.21.15.33:0/3393345479 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/3393345479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7ef80 0x556c96f9e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6a6ffd700 1 -- 172.21.15.33:0/3393345479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7ef80 msgr2=0x556c96f9e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6a6ffd700 1 --2- 172.21.15.33:0/3393345479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7ef80 0x556c96f9e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6a6ffd700 1 -- 172.21.15.33:0/3393345479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7f9a0 msgr2=0x556c96f9f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6a6ffd700 1 --2- 172.21.15.33:0/3393345479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7f9a0 0x556c96f9f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.353+0000 7fe6a6ffd700 1 -- 172.21.15.33:0/3393345479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a8108980 con 0x556c96f7e620 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.354+0000 7fe6a6ffd700 1 --2- 172.21.15.33:0/3393345479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7e620 0x556c96f9e0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe698005ec0 tx=0x7fe69800aa90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffd76e87b1d963e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.354+0000 7fe686ffd700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.354+0000 7fe686ffd700 1 -- 172.21.15.33:0/3393345479 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698028250 con 0x556c96f7e620 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.354+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3393345479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe69c01e040 con 0x556c96f7e620 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.354+0000 7fe686ffd700 1 -- 172.21.15.33:0/3393345479 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe6980283b0 con 0x556c96f7e620 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.354+0000 7fe686ffd700 1 -- 172.21.15.33:0/3393345479 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe698028570 con 0x556c96f7e620 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.354+0000 7fe6b9bbdb80 10 client.26852 fetch_fsmap learned FSMap version 34 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.354+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3393345479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fe698028b70 con 0x556c96f7e620 2024-05-08T21:23:57.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.354+0000 7fe686ffd700 1 -- 172.21.15.33:0/3393345479 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe69c01e040 con 0x556c96f7e620 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.354+0000 7fe686ffd700 1 -- 172.21.15.33:0/3393345479 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fe698004a20 con 0x556c96f7e620 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.354+0000 7fe6b9bbdb80 10 client.26852 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.354+0000 7fe6b9bbdb80 10 client.26852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.355+0000 7fe6b9bbdb80 20 client.26852 populate_metadata read hostname 'smithi033' 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.355+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/3393345479 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556c96ffbbc0 0x556c96ffdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.355+0000 7fe6b9bbdb80 4 client.26852 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.355+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3393345479 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556c96ec6ec0 con 0x556c96ffbbc0 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.355+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/3393345479 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556c96ffbbc0 0x556c96ffdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.355+0000 7fe686ffd700 10 client.26852 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.355+0000 7fe6a67fc700 1 --2- 172.21.15.33:0/3393345479 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556c96ffbbc0 0x556c96ffdfe0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ec4a4600521208 server_cookie=bf82c9c0490ea4c7 in_seq=0 out_seq=0 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.355+0000 7fe686ffd700 1 -- 172.21.15.33:0/3393345479 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556c96ec6ec0 con 0x556c96ffbbc0 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.355+0000 7fe686ffd700 10 client.26852 handle_command_reply: tid=0 2024-05-08T21:23:57.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.355+0000 7fe6b9bbdb80 1 client.26852 shutdown 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3393345479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7e620 msgr2=0x556c96f9e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/3393345479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7e620 0x556c96f9e0a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe698005ec0 tx=0x7fe69800aa90 comp rx=0 tx=0).stop 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3393345479 shutdown_connections 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/3393345479 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556c96ffbbc0 0x556c96ffdfe0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/3393345479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96f7f9a0 0x556c96f9f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/3393345479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c96f7e620 0x556c96f9e0a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6b9bbdb80 1 --2- 172.21.15.33:0/3393345479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c96f7ef80 0x556c96f9e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe686ffd700 0 client.26852 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3393345479 >> 172.21.15.33:0/3393345479 conn(0x556c96edc210 msgr2=0x556c96fab630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3393345479 shutdown_connections 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6b9bbdb80 1 -- 172.21.15.33:0/3393345479 wait complete. 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6b9bbdb80 20 client.26852 trim_cache size 0 max 16384 2024-05-08T21:23:57.363 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:23:57.364 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:23:57.364 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:23:57.364 INFO:teuthology.orchestra.run.smithi033.stdout: "d947cfbb-2855-4688-be0d-83321f2de4e6": { 2024-05-08T21:23:57.364 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:23:57.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "d947cfbb-2855-4688-be0d-83321f2de4e6", 2024-05-08T21:23:57.364 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:23:57.364 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:23:57.364 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:23:57.364 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:23:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6a7fff700 1 -- 172.21.15.33:0/516412583 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6900a12c0 msgr2=0x7fe6900a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6a7fff700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6900a12c0 0x7fe6900a3780 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe698005fd0 tx=0x7fe698012520 comp rx=0 tx=0).stop 2024-05-08T21:23:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6a7fff700 1 -- 172.21.15.33:0/516412583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a8101780 msgr2=0x7fe6a819bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:23:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6a7fff700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a8101780 0x7fe6a819bd40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000ef30 tx=0x7fe6a000c3a0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6a7fff700 1 -- 172.21.15.33:0/516412583 shutdown_connections 2024-05-08T21:23:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6a7fff700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6a8101780 0x7fe6a819bd40 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6a7fff700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe6900a12c0 0x7fe6900a3780 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6a7fff700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6a80ffbf0 0x7fe6a81947e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.356+0000 7fe6a7fff700 1 --2- 172.21.15.33:0/516412583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a8100de0 0x7fe6a8194d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:23:57.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.357+0000 7fe6a7fff700 1 -- 172.21.15.33:0/516412583 >> 172.21.15.33:0/516412583 conn(0x7fe6a80fb180 msgr2=0x7fe6a80fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:23:57.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.357+0000 7fe6a7fff700 1 -- 172.21.15.33:0/516412583 shutdown_connections 2024-05-08T21:23:57.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:23:57.357+0000 7fe6a7fff700 1 -- 172.21.15.33:0/516412583 wait complete. 2024-05-08T21:23:57.374 INFO:tasks.cephfs.filesystem:scrub status for tag:d947cfbb-2855-4688-be0d-83321f2de4e6 - {'path': '/', 'tag': 'd947cfbb-2855-4688-be0d-83321f2de4e6', 'options': 'recursive,force'} 2024-05-08T21:23:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: pgmap v439: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 415 op/s 2024-05-08T21:23:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:23:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:57.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:57.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:57 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: pgmap v439: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 415 op/s 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:57.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:57 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: pgmap v439: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.4 MiB/s wr, 415 op/s 2024-05-08T21:23:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:23:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:57.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:57.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:57.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:57.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:57.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:57 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:58.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:58 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:58.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:58 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:23:58.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:58 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:58.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:58 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:58.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:58 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:58.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:58 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:58.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:58 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:23:58.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:58 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:58.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:58 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:58.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:58 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:58.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:58 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:58.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:58 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:23:58.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:58 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:58.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:58 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:58.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:58 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:23:59.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:23:59 smithi071 ceph-mon[47074]: pgmap v440: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 331 op/s 2024-05-08T21:23:59.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:23:59 smithi082 ceph-mon[47512]: pgmap v440: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 331 op/s 2024-05-08T21:23:59.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:23:59 smithi033 ceph-mon[42528]: pgmap v440: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 331 op/s 2024-05-08T21:24:01.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:01 smithi071 ceph-mon[47074]: pgmap v441: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 399 op/s 2024-05-08T21:24:01.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:01 smithi082 ceph-mon[47512]: pgmap v441: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 399 op/s 2024-05-08T21:24:01.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:01 smithi033 ceph-mon[42528]: pgmap v441: 129 pgs: 129 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 399 op/s 2024-05-08T21:24:03.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:03 smithi071 ceph-mon[47074]: pgmap v442: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 314 op/s 2024-05-08T21:24:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:03 smithi082 ceph-mon[47512]: pgmap v442: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 314 op/s 2024-05-08T21:24:03.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:03 smithi033 ceph-mon[42528]: pgmap v442: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 314 op/s 2024-05-08T21:24:05.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:05 smithi071 ceph-mon[47074]: pgmap v443: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-05-08T21:24:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:05 smithi082 ceph-mon[47512]: pgmap v443: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-05-08T21:24:05.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:05 smithi033 ceph-mon[42528]: pgmap v443: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-05-08T21:24:07.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:07 smithi071 ceph-mon[47074]: pgmap v444: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 423 op/s 2024-05-08T21:24:07.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:07 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:24:07.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:07 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:24:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:07 smithi082 ceph-mon[47512]: pgmap v444: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 423 op/s 2024-05-08T21:24:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:07 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:24:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:07 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:24:07.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:07 smithi033 ceph-mon[42528]: pgmap v444: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 423 op/s 2024-05-08T21:24:07.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:07 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:24:07.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:07 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:24:08.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:08 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:24:08.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:08 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:24:08.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:08 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:24:08.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:08 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:24:08.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:08 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:24:08.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:08 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:24:08.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:08 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:24:08.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:08 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:24:08.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:08 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:24:09.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:09 smithi071 ceph-mon[47074]: pgmap v445: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 311 op/s 2024-05-08T21:24:09.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:09 smithi082 ceph-mon[47512]: pgmap v445: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 311 op/s 2024-05-08T21:24:09.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:09 smithi033 ceph-mon[42528]: pgmap v445: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 311 op/s 2024-05-08T21:24:11.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:11 smithi071 ceph-mon[47074]: pgmap v446: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 384 op/s 2024-05-08T21:24:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:11 smithi082 ceph-mon[47512]: pgmap v446: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 384 op/s 2024-05-08T21:24:11.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:11 smithi033 ceph-mon[42528]: pgmap v446: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 384 op/s 2024-05-08T21:24:13.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:13 smithi071 ceph-mon[47074]: pgmap v447: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 317 op/s 2024-05-08T21:24:13.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:13 smithi082 ceph-mon[47512]: pgmap v447: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 317 op/s 2024-05-08T21:24:13.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:13 smithi033 ceph-mon[42528]: pgmap v447: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 317 op/s 2024-05-08T21:24:15.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:15 smithi071 ceph-mon[47074]: pgmap v448: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-05-08T21:24:15.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:15 smithi082 ceph-mon[47512]: pgmap v448: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-05-08T21:24:15.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:15 smithi033 ceph-mon[42528]: pgmap v448: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-05-08T21:24:17.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:17 smithi071 ceph-mon[47074]: pgmap v449: 129 pgs: 129 active+clean; 207 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 431 op/s 2024-05-08T21:24:17.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:17 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:24:17.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:17 smithi082 ceph-mon[47512]: pgmap v449: 129 pgs: 129 active+clean; 207 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 431 op/s 2024-05-08T21:24:17.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:17 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:24:17.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:17 smithi033 ceph-mon[42528]: pgmap v449: 129 pgs: 129 active+clean; 207 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 431 op/s 2024-05-08T21:24:17.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:17 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:24:18.084 INFO:tasks.workunit.client.0.smithi033.stdout:50000 trunc from 0xa4fa5 to 0x60d70 2024-05-08T21:24:18.094 INFO:tasks.workunit.client.0.smithi033.stdout:All 50000 operations completed A-OK! 2024-05-08T21:24:18.094 INFO:tasks.workunit.client.0.smithi033.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-05-08T21:24:18.096 INFO:tasks.workunit.client.0.smithi033.stdout:Seed set to 1 2024-05-08T21:24:18.239 INFO:tasks.workunit.client.0.smithi033.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-08T21:24:18.239 INFO:tasks.workunit.client.0.smithi033.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-08T21:24:18.239 INFO:tasks.workunit.client.0.smithi033.stderr:main: filesystem does not support clone range, disabling! 2024-05-08T21:24:18.239 INFO:tasks.workunit.client.0.smithi033.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-08T21:24:18.243 INFO:tasks.workunit.client.0.smithi033.stdout:truncating to largest ever: 0x5a0dfb 2024-05-08T21:24:18.265 INFO:tasks.workunit.client.0.smithi033.stdout:copying to largest ever: 0x5c415c 2024-05-08T21:24:18.265 INFO:tasks.workunit.client.0.smithi033.stdout:truncating to largest ever: 0x9c27b4 2024-05-08T21:24:18.356 INFO:tasks.workunit.client.0.smithi033.stdout:fallocating to largest ever: 0xa00000 2024-05-08T21:24:18.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:24:18.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:24:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:24:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:24:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:24:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:24:18.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:24:18.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:24:18.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:24:19.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:19 smithi071 ceph-mon[47074]: pgmap v450: 129 pgs: 129 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 286 op/s 2024-05-08T21:24:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:19 smithi082 ceph-mon[47512]: pgmap v450: 129 pgs: 129 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 286 op/s 2024-05-08T21:24:19.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:19 smithi033 ceph-mon[42528]: pgmap v450: 129 pgs: 129 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.2 MiB/s wr, 286 op/s 2024-05-08T21:24:21.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:21 smithi071 ceph-mon[47074]: pgmap v451: 129 pgs: 129 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 371 op/s 2024-05-08T21:24:21.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:21 smithi082 ceph-mon[47512]: pgmap v451: 129 pgs: 129 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 371 op/s 2024-05-08T21:24:21.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:21 smithi033 ceph-mon[42528]: pgmap v451: 129 pgs: 129 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 371 op/s 2024-05-08T21:24:23.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:23 smithi071 ceph-mon[47074]: pgmap v452: 129 pgs: 129 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 372 op/s 2024-05-08T21:24:23.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:23 smithi082 ceph-mon[47512]: pgmap v452: 129 pgs: 129 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 372 op/s 2024-05-08T21:24:23.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:23 smithi033 ceph-mon[42528]: pgmap v452: 129 pgs: 129 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.7 MiB/s wr, 372 op/s 2024-05-08T21:24:25.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:25 smithi071 ceph-mon[47074]: pgmap v453: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.6 MiB/s wr, 349 op/s 2024-05-08T21:24:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:25 smithi082 ceph-mon[47512]: pgmap v453: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.6 MiB/s wr, 349 op/s 2024-05-08T21:24:25.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:25 smithi033 ceph-mon[42528]: pgmap v453: 129 pgs: 129 active+clean; 212 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.6 MiB/s wr, 349 op/s 2024-05-08T21:24:27.375 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:24:27.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.541+0000 7fc5f7814700 1 -- 172.21.15.33:0/1333567212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f0100090 msgr2=0x7fc5f0100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:27.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.541+0000 7fc5f7814700 1 --2- 172.21.15.33:0/1333567212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f0100090 0x7fc5f0100510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00099c0 tx=0x7fc5e0009cd0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.542+0000 7fc5f7814700 1 -- 172.21.15.33:0/1333567212 shutdown_connections 2024-05-08T21:24:27.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.542+0000 7fc5f7814700 1 --2- 172.21.15.33:0/1333567212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0100a50 0x7fc5f0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.542+0000 7fc5f7814700 1 --2- 172.21.15.33:0/1333567212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f00ff730 0x7fc5f00ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.542+0000 7fc5f7814700 1 --2- 172.21.15.33:0/1333567212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f0100090 0x7fc5f0100510 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.542+0000 7fc5f7814700 1 -- 172.21.15.33:0/1333567212 >> 172.21.15.33:0/1333567212 conn(0x7fc5f00fb290 msgr2=0x7fc5f00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:27.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.542+0000 7fc5f7814700 1 -- 172.21.15.33:0/1333567212 shutdown_connections 2024-05-08T21:24:27.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.542+0000 7fc5f7814700 1 -- 172.21.15.33:0/1333567212 wait complete. 2024-05-08T21:24:27.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.543+0000 7fc5f7814700 1 Processor -- start 2024-05-08T21:24:27.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.543+0000 7fc5f7814700 1 -- start start 2024-05-08T21:24:27.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.543+0000 7fc5f7814700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f00ff730 0x7fc5f0198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.543+0000 7fc5f7814700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0100090 0x7fc5f01994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.543+0000 7fc5f7814700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f0100a50 0x7fc5f01a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.543+0000 7fc5f7814700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01a0a90 con 0x7fc5f0100090 2024-05-08T21:24:27.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.543+0000 7fc5f7814700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01a0bd0 con 0x7fc5f00ff730 2024-05-08T21:24:27.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.543+0000 7fc5f7814700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01a0d60 con 0x7fc5f0100a50 2024-05-08T21:24:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.543+0000 7fc5f55b0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f00ff730 0x7fc5f0198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.543+0000 7fc5f4daf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0100090 0x7fc5f01994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.543+0000 7fc5f55b0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f00ff730 0x7fc5f0198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:48572/0 (socket says 172.21.15.33:48572) 2024-05-08T21:24:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.543+0000 7fc5f55b0700 1 -- 172.21.15.33:0/2917056430 learned_addr learned my addr 172.21.15.33:0/2917056430 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.544+0000 7fc5f55b0700 1 -- 172.21.15.33:0/2917056430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f0100a50 msgr2=0x7fc5f01a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.544+0000 7fc5f55b0700 1 --2- 172.21.15.33:0/2917056430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f0100a50 0x7fc5f01a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.544+0000 7fc5f55b0700 1 -- 172.21.15.33:0/2917056430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0100090 msgr2=0x7fc5f01994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.544+0000 7fc5f55b0700 1 --2- 172.21.15.33:0/2917056430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0100090 0x7fc5f01994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.544+0000 7fc5f55b0700 1 -- 172.21.15.33:0/2917056430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e8009c40 con 0x7fc5f00ff730 2024-05-08T21:24:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.544+0000 7fc5f55b0700 1 --2- 172.21.15.33:0/2917056430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f00ff730 0x7fc5f0198f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00d8d0 tx=0x7fc5ec00dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.544+0000 7fc5e67fc700 1 -- 172.21.15.33:0/2917056430 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0098d0 con 0x7fc5f00ff730 2024-05-08T21:24:27.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.544+0000 7fc5e67fc700 1 -- 172.21.15.33:0/2917056430 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5ec010460 con 0x7fc5f00ff730 2024-05-08T21:24:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.544+0000 7fc5f7814700 1 -- 172.21.15.33:0/2917056430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e00095f0 con 0x7fc5f00ff730 2024-05-08T21:24:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.544+0000 7fc5e67fc700 1 -- 172.21.15.33:0/2917056430 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec00f620 con 0x7fc5f00ff730 2024-05-08T21:24:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.544+0000 7fc5f7814700 1 -- 172.21.15.33:0/2917056430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f01a1320 con 0x7fc5f00ff730 2024-05-08T21:24:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.547+0000 7fc5e67fc700 1 -- 172.21.15.33:0/2917056430 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc5ec009a30 con 0x7fc5f00ff730 2024-05-08T21:24:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.548+0000 7fc5e67fc700 1 --2- 172.21.15.33:0/2917056430 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5dc0a11a0 0x7fc5dc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.548+0000 7fc5e67fc700 1 -- 172.21.15.33:0/2917056430 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc5ec0cc9b0 con 0x7fc5f00ff730 2024-05-08T21:24:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.548+0000 7fc5f4daf700 1 --2- 172.21.15.33:0/2917056430 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5dc0a11a0 0x7fc5dc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.549+0000 7fc5f4daf700 1 --2- 172.21.15.33:0/2917056430 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5dc0a11a0 0x7fc5dc0a3660 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0000c00 tx=0x7fc5e0003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.549+0000 7fc6039ceb80 1 Processor -- start 2024-05-08T21:24:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.549+0000 7fc6039ceb80 1 -- start start 2024-05-08T21:24:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.549+0000 7fc6039ceb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73f540 0x5606fb73f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.549+0000 7fc6039ceb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73fea0 0x5606fb740320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.549+0000 7fc6039ceb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb740860 0x5606fb740ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.549+0000 7fc6039ceb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fb696150 con 0x5606fb740860 2024-05-08T21:24:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.549+0000 7fc6039ceb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fb693b10 con 0x5606fb73fea0 2024-05-08T21:24:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.549+0000 7fc6039ceb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fb68f400 con 0x5606fb73f540 2024-05-08T21:24:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f55b0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73f540 0x5606fb73f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f4daf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73fea0 0x5606fb740320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f55b0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73f540 0x5606fb73f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:33778/0 (socket says 172.21.15.33:33778) 2024-05-08T21:24:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f55b0700 1 -- 172.21.15.33:0/1564912829 learned_addr learned my addr 172.21.15.33:0/1564912829 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f5db1700 1 --2- 172.21.15.33:0/1564912829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb740860 0x5606fb740ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f55b0700 1 -- 172.21.15.33:0/1564912829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73fea0 msgr2=0x5606fb740320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f55b0700 1 --2- 172.21.15.33:0/1564912829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73fea0 0x5606fb740320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f55b0700 1 -- 172.21.15.33:0/1564912829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb740860 msgr2=0x5606fb740ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f55b0700 1 --2- 172.21.15.33:0/1564912829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb740860 0x5606fb740ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f55b0700 1 -- 172.21.15.33:0/1564912829 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606fb5103b0 con 0x5606fb73f540 2024-05-08T21:24:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f5db1700 1 --2- 172.21.15.33:0/1564912829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb740860 0x5606fb740ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f4daf700 1 --2- 172.21.15.33:0/1564912829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73fea0 0x5606fb740320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5f55b0700 1 --2- 172.21.15.33:0/1564912829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73f540 0x5606fb73f960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0a19d0 tx=0x7fc5ec099460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a002cc99af846231 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.550+0000 7fc5db7fe700 1 -- 172.21.15.33:0/1564912829 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec099a80 con 0x5606fb73f540 2024-05-08T21:24:27.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.551+0000 7fc5db7fe700 1 -- 172.21.15.33:0/1564912829 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5ec09d8b0 con 0x5606fb73f540 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.551+0000 7fc5db7fe700 1 -- 172.21.15.33:0/1564912829 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec099ce0 con 0x5606fb73f540 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.551+0000 7fc6039ceb80 1 -- 172.21.15.33:0/1564912829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73f540 msgr2=0x5606fb73f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.551+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/1564912829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73f540 0x5606fb73f960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0a19d0 tx=0x7fc5ec099460 comp rx=0 tx=0).stop 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.552+0000 7fc6039ceb80 1 -- 172.21.15.33:0/1564912829 shutdown_connections 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.552+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/1564912829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb740860 0x5606fb740ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.552+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/1564912829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73f540 0x5606fb73f960 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.552+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/1564912829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73fea0 0x5606fb740320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.552+0000 7fc6039ceb80 1 -- 172.21.15.33:0/1564912829 >> 172.21.15.33:0/1564912829 conn(0x5606fb69d210 msgr2=0x5606fb739de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.552+0000 7fc6039ceb80 1 -- 172.21.15.33:0/1564912829 shutdown_connections 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.552+0000 7fc6039ceb80 1 -- 172.21.15.33:0/1564912829 wait complete. 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.552+0000 7fc6039ceb80 1 -- start start 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.552+0000 7fc6039ceb80 1 Processor -- start 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc6039ceb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73f540 0x5606fb748e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc6039ceb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73fea0 0x5606fb749360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc6039ceb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 0x5606fb750350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc6039ceb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fb62b560 con 0x5606fb73fea0 2024-05-08T21:24:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc6039ceb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fb62b200 con 0x5606fb73f540 2024-05-08T21:24:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc6039ceb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fb5126a0 con 0x5606fb740860 2024-05-08T21:24:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc6039ceb80 10 client.0 _ll_register_callbacks cb 0x5606fb634b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc5f4daf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73fea0 0x5606fb749360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc5f5db1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 0x5606fb750350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc5f55b0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73f540 0x5606fb748e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc5f4daf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73fea0 0x5606fb749360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55474/0 (socket says 172.21.15.33:55474) 2024-05-08T21:24:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc5f4daf700 1 -- 172.21.15.33:0/3833712348 learned_addr learned my addr 172.21.15.33:0/3833712348 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc5f5db1700 1 -- 172.21.15.33:0/3833712348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73f540 msgr2=0x5606fb748e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc5f5db1700 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73f540 0x5606fb748e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc5f5db1700 1 -- 172.21.15.33:0/3833712348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73fea0 msgr2=0x5606fb749360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc5f5db1700 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73fea0 0x5606fb749360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc5f5db1700 1 -- 172.21.15.33:0/3833712348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e800e140 con 0x5606fb740860 2024-05-08T21:24:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc5f4daf700 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73fea0 0x5606fb749360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.553+0000 7fc5f55b0700 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73f540 0x5606fb748e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc5f5db1700 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 0x5606fb750350 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5606fb73ad40 tx=0x7fc5e8007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6aa4dff5327c773 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc5d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:24:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc5d8ff9700 1 -- 172.21.15.33:0/3833712348 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8003a00 con 0x5606fb740860 2024-05-08T21:24:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc5d8ff9700 1 -- 172.21.15.33:0/3833712348 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc5e8003b60 con 0x5606fb740860 2024-05-08T21:24:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3833712348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606fb5e8270 con 0x5606fb740860 2024-05-08T21:24:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc5d8ff9700 1 -- 172.21.15.33:0/3833712348 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5e8003d20 con 0x5606fb740860 2024-05-08T21:24:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc6039ceb80 10 client.26878 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3833712348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fc5f01a1320 con 0x5606fb740860 2024-05-08T21:24:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc5d8ff9700 1 -- 172.21.15.33:0/3833712348 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5606fb5e8270 con 0x5606fb740860 2024-05-08T21:24:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc5d8ff9700 1 -- 172.21.15.33:0/3833712348 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fc5e801e060 con 0x5606fb740860 2024-05-08T21:24:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc6039ceb80 10 client.26878 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc6039ceb80 10 client.26878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:24:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc6039ceb80 20 client.26878 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5606fb754d10 0x5606fb757130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc6039ceb80 4 client.26878 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:24:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.554+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3833712348 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606fb70e810 con 0x5606fb754d10 2024-05-08T21:24:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.555+0000 7fc5f55b0700 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5606fb754d10 0x5606fb757130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.555+0000 7fc5d8ff9700 10 client.26878 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:27.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.555+0000 7fc5f55b0700 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5606fb754d10 0x5606fb757130 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7c40bdfd0cb036 server_cookie=754a4b176be87bd9 in_seq=0 out_seq=0 2024-05-08T21:24:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc5d8ff9700 1 -- 172.21.15.33:0/3833712348 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5606fb70e810 con 0x5606fb754d10 2024-05-08T21:24:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc5d8ff9700 10 client.26878 handle_command_reply: tid=0 2024-05-08T21:24:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc6039ceb80 1 client.26878 shutdown 2024-05-08T21:24:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3833712348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 msgr2=0x5606fb750350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 0x5606fb750350 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5606fb73ad40 tx=0x7fc5e8007f90 comp rx=0 tx=0).stop 2024-05-08T21:24:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3833712348 shutdown_connections 2024-05-08T21:24:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5606fb754d10 0x5606fb757130 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 0x5606fb750350 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73f540 0x5606fb748e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc5d8ff9700 0 client.26878 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/3833712348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73fea0 0x5606fb749360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3833712348 >> 172.21.15.33:0/3833712348 conn(0x5606fb69d210 msgr2=0x5606fb7395a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3833712348 shutdown_connections 2024-05-08T21:24:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3833712348 wait complete. 2024-05-08T21:24:27.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.556+0000 7fc6039ceb80 20 client.26878 trim_cache size 0 max 16384 2024-05-08T21:24:27.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.578+0000 7fc6039ceb80 1 Processor -- start 2024-05-08T21:24:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.578+0000 7fc6039ceb80 1 -- start start 2024-05-08T21:24:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.578+0000 7fc6039ceb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73f540 0x5606fb73a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.578+0000 7fc6039ceb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73fea0 0x5606fb750310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.578+0000 7fc6039ceb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 0x5606fb746e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.578+0000 7fc6039ceb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fb748990 con 0x5606fb73f540 2024-05-08T21:24:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.578+0000 7fc6039ceb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fb736dc0 con 0x5606fb73fea0 2024-05-08T21:24:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.578+0000 7fc6039ceb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fb749f50 con 0x5606fb740860 2024-05-08T21:24:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5f55b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73f540 0x5606fb73a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5f5db1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 0x5606fb746e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5f4daf700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73fea0 0x5606fb750310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5f55b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73f540 0x5606fb73a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55484/0 (socket says 172.21.15.33:55484) 2024-05-08T21:24:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5f5db1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 0x5606fb746e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:33806/0 (socket says 172.21.15.33:33806) 2024-05-08T21:24:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5f55b0700 1 -- 172.21.15.33:0/168747107 learned_addr learned my addr 172.21.15.33:0/168747107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:27.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5f5db1700 1 -- 172.21.15.33:0/168747107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73fea0 msgr2=0x5606fb750310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5f5db1700 1 --2- 172.21.15.33:0/168747107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73fea0 0x5606fb750310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5f5db1700 1 -- 172.21.15.33:0/168747107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73f540 msgr2=0x5606fb73a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5f5db1700 1 --2- 172.21.15.33:0/168747107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73f540 0x5606fb73a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5f5db1700 1 -- 172.21.15.33:0/168747107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606fb4bcb60 con 0x5606fb740860 2024-05-08T21:24:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5f5db1700 1 --2- 172.21.15.33:0/168747107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 0x5606fb746e20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8002230 tx=0x7fc5e8004a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3553c141461deda server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.579+0000 7fc5dbfff700 1 -- 172.21.15.33:0/168747107 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80099a0 con 0x5606fb740860 2024-05-08T21:24:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.580+0000 7fc5dbfff700 1 -- 172.21.15.33:0/168747107 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5e8009e90 con 0x5606fb740860 2024-05-08T21:24:27.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.580+0000 7fc5dbfff700 1 -- 172.21.15.33:0/168747107 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8017de0 con 0x5606fb740860 2024-05-08T21:24:27.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.580+0000 7fc6039ceb80 1 -- 172.21.15.33:0/168747107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 msgr2=0x5606fb746e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:27.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.580+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/168747107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 0x5606fb746e20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8002230 tx=0x7fc5e8004a90 comp rx=0 tx=0).stop 2024-05-08T21:24:27.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.580+0000 7fc6039ceb80 1 -- 172.21.15.33:0/168747107 shutdown_connections 2024-05-08T21:24:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.580+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/168747107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb740860 0x5606fb746e20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.580+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/168747107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb73f540 0x5606fb73a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.580+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/168747107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73fea0 0x5606fb750310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.581+0000 7fc6039ceb80 1 -- 172.21.15.33:0/168747107 >> 172.21.15.33:0/168747107 conn(0x5606fb69d210 msgr2=0x5606fb740c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.581+0000 7fc6039ceb80 1 -- 172.21.15.33:0/168747107 shutdown_connections 2024-05-08T21:24:27.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.581+0000 7fc6039ceb80 1 -- 172.21.15.33:0/168747107 wait complete. 2024-05-08T21:24:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.581+0000 7fc6039ceb80 1 -- start start 2024-05-08T21:24:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.581+0000 7fc6039ceb80 1 Processor -- start 2024-05-08T21:24:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc6039ceb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73f540 0x5606fb74c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc6039ceb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73fea0 0x5606fb74e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc6039ceb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb740860 0x5606fb74ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc6039ceb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fb74f580 con 0x5606fb740860 2024-05-08T21:24:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc6039ceb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fb74f6f0 con 0x5606fb73f540 2024-05-08T21:24:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc6039ceb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606fb74f8b0 con 0x5606fb73fea0 2024-05-08T21:24:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc6039ceb80 10 client.0 _ll_register_callbacks cb 0x5606fb634b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5f4daf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73fea0 0x5606fb74e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5f55b0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73f540 0x5606fb74c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5f4daf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73fea0 0x5606fb74e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:33820/0 (socket says 172.21.15.33:33820) 2024-05-08T21:24:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5f4daf700 1 -- 172.21.15.33:0/3528324444 learned_addr learned my addr 172.21.15.33:0/3528324444 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5f5db1700 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb740860 0x5606fb74ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5f4daf700 1 -- 172.21.15.33:0/3528324444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73f540 msgr2=0x5606fb74c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5f4daf700 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73f540 0x5606fb74c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5f4daf700 1 -- 172.21.15.33:0/3528324444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb740860 msgr2=0x5606fb74ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:27.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5f4daf700 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb740860 0x5606fb74ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5f4daf700 1 -- 172.21.15.33:0/3528324444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f01a1320 con 0x5606fb73fea0 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5f55b0700 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73f540 0x5606fb74c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5f4daf700 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73fea0 0x5606fb74e840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00099c0 tx=0x7fc5e0009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f1870badec19ec9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.582+0000 7fc5d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc5d8ff9700 1 -- 172.21.15.33:0/3528324444 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e002b030 con 0x5606fb73fea0 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc5d8ff9700 1 -- 172.21.15.33:0/3528324444 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc5e000f870 con 0x5606fb73fea0 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3528324444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5ec09d8b0 con 0x5606fb73fea0 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc5d8ff9700 1 -- 172.21.15.33:0/3528324444 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5e000fa30 con 0x5606fb73fea0 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc6039ceb80 10 client.26884 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3528324444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5606fb655560 con 0x5606fb73fea0 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc5d8ff9700 1 -- 172.21.15.33:0/3528324444 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5ec09d8b0 con 0x5606fb73fea0 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc5d8ff9700 1 -- 172.21.15.33:0/3528324444 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fc5e0032c00 con 0x5606fb73fea0 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc6039ceb80 10 client.26884 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc6039ceb80 10 client.26884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:24:27.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc6039ceb80 20 client.26884 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5606fb7bcdf0 0x5606fb7bf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc6039ceb80 4 client.26884 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.583+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3528324444 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5606fb687ec0 con 0x5606fb7bcdf0 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc5f55b0700 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5606fb7bcdf0 0x5606fb7bf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc5d8ff9700 10 client.26884 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc5f55b0700 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5606fb7bcdf0 0x5606fb7bf1e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56f9bf36f1365c6 server_cookie=f1e0bfaaed0fb2ac in_seq=0 out_seq=0 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc5d8ff9700 1 -- 172.21.15.33:0/3528324444 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5606fb687ec0 con 0x5606fb7bcdf0 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc5d8ff9700 10 client.26884 handle_command_reply: tid=0 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc6039ceb80 1 client.26884 shutdown 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3528324444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73fea0 msgr2=0x5606fb74e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73fea0 0x5606fb74e840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00099c0 tx=0x7fc5e0009cd0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3528324444 shutdown_connections 2024-05-08T21:24:27.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5606fb7bcdf0 0x5606fb7bf1e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606fb740860 0x5606fb74ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606fb73f540 0x5606fb74c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc6039ceb80 1 --2- 172.21.15.33:0/3528324444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5606fb73fea0 0x5606fb74e840 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc5d8ff9700 0 client.26884 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:27.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.584+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3528324444 >> 172.21.15.33:0/3528324444 conn(0x5606fb69d210 msgr2=0x5606fb746dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:27.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3528324444 shutdown_connections 2024-05-08T21:24:27.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc6039ceb80 1 -- 172.21.15.33:0/3528324444 wait complete. 2024-05-08T21:24:27.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc6039ceb80 20 client.26884 trim_cache size 0 max 16384 2024-05-08T21:24:27.595 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:24:27.595 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:24:27.595 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:24:27.595 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:24:27.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc5f7814700 1 -- 172.21.15.33:0/2917056430 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5dc0a11a0 msgr2=0x7fc5dc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:27.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc5f7814700 1 --2- 172.21.15.33:0/2917056430 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5dc0a11a0 0x7fc5dc0a3660 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0000c00 tx=0x7fc5e0003680 comp rx=0 tx=0).stop 2024-05-08T21:24:27.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc5f7814700 1 -- 172.21.15.33:0/2917056430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f00ff730 msgr2=0x7fc5f0198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:27.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc5f7814700 1 --2- 172.21.15.33:0/2917056430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f00ff730 0x7fc5f0198f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00d8d0 tx=0x7fc5ec00dbe0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc5f7814700 1 -- 172.21.15.33:0/2917056430 shutdown_connections 2024-05-08T21:24:27.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc5f7814700 1 --2- 172.21.15.33:0/2917056430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f0100a50 0x7fc5f01a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc5f7814700 1 --2- 172.21.15.33:0/2917056430 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc5dc0a11a0 0x7fc5dc0a3660 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc5f7814700 1 --2- 172.21.15.33:0/2917056430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5f00ff730 0x7fc5f0198f60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc5f7814700 1 --2- 172.21.15.33:0/2917056430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f0100090 0x7fc5f01994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc5f7814700 1 -- 172.21.15.33:0/2917056430 >> 172.21.15.33:0/2917056430 conn(0x7fc5f00fb290 msgr2=0x7fc5f00fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:27.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc5f7814700 1 -- 172.21.15.33:0/2917056430 shutdown_connections 2024-05-08T21:24:27.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.585+0000 7fc5f7814700 1 -- 172.21.15.33:0/2917056430 wait complete. 2024-05-08T21:24:27.603 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:24:27.604 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:24:27.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:27 smithi033 ceph-mon[42528]: pgmap v454: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.9 MiB/s wr, 523 op/s 2024-05-08T21:24:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.762+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2898811344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc1000b0 msgr2=0x7ff4cc100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.762+0000 7ff4d22dc700 1 --2- 172.21.15.33:0/2898811344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc1000b0 0x7ff4cc100530 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009990 tx=0x7ff4c0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.762+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2898811344 shutdown_connections 2024-05-08T21:24:27.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.762+0000 7ff4d22dc700 1 --2- 172.21.15.33:0/2898811344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc100a70 0x7ff4cc100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.762+0000 7ff4d22dc700 1 --2- 172.21.15.33:0/2898811344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4cc0ff750 0x7ff4cc0ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.762+0000 7ff4d22dc700 1 --2- 172.21.15.33:0/2898811344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc1000b0 0x7ff4cc100530 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.762+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2898811344 >> 172.21.15.33:0/2898811344 conn(0x7ff4cc0fb310 msgr2=0x7ff4cc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:27.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.762+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2898811344 shutdown_connections 2024-05-08T21:24:27.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.762+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2898811344 wait complete. 2024-05-08T21:24:27.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.763+0000 7ff4d22dc700 1 Processor -- start 2024-05-08T21:24:27.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.763+0000 7ff4d22dc700 1 -- start start 2024-05-08T21:24:27.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4d22dc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc0ff750 0x7ff4cc198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4d22dc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc1000b0 0x7ff4cc199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4d22dc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4cc100a70 0x7ff4cc1a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4d22dc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1a0a80 con 0x7ff4cc0ff750 2024-05-08T21:24:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4d22dc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1a0bc0 con 0x7ff4cc100a70 2024-05-08T21:24:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4d22dc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1a0d50 con 0x7ff4cc1000b0 2024-05-08T21:24:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4cb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc1000b0 0x7ff4cc199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc0ff750 0x7ff4cc198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4cb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc1000b0 0x7ff4cc199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:33842/0 (socket says 172.21.15.33:33842) 2024-05-08T21:24:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4cb7fe700 1 -- 172.21.15.33:0/2741262981 learned_addr learned my addr 172.21.15.33:0/2741262981 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4d0879700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4cc100a70 0x7ff4cc1a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4cb7fe700 1 -- 172.21.15.33:0/2741262981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4cc100a70 msgr2=0x7ff4cc1a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4cb7fe700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4cc100a70 0x7ff4cc1a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4cb7fe700 1 -- 172.21.15.33:0/2741262981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc0ff750 msgr2=0x7ff4cc198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4cb7fe700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc0ff750 0x7ff4cc198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4cb7fe700 1 -- 172.21.15.33:0/2741262981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c00095f0 con 0x7ff4cc1000b0 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4d0879700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4cc100a70 0x7ff4cc1a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4cbfff700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc0ff750 0x7ff4cc198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.764+0000 7ff4cb7fe700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc1000b0 0x7ff4cc199490 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005910 tx=0x7ff4c0022af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.765+0000 7ff4c97fa700 1 -- 172.21.15.33:0/2741262981 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c000bbc0 con 0x7ff4cc1000b0 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.765+0000 7ff4c97fa700 1 -- 172.21.15.33:0/2741262981 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4c000bd20 con 0x7ff4cc1000b0 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.765+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2741262981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc101960 con 0x7ff4cc1000b0 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.765+0000 7ff4c97fa700 1 -- 172.21.15.33:0/2741262981 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c000fad0 con 0x7ff4cc1000b0 2024-05-08T21:24:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.765+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2741262981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc101e60 con 0x7ff4cc1000b0 2024-05-08T21:24:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.766+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2741262981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc10a530 con 0x7ff4cc1000b0 2024-05-08T21:24:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.767+0000 7ff4c97fa700 1 -- 172.21.15.33:0/2741262981 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff4c000fc30 con 0x7ff4cc1000b0 2024-05-08T21:24:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.768+0000 7ff4c97fa700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4b40a11a0 0x7ff4b40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.768+0000 7ff4cbfff700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4b40a11a0 0x7ff4b40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.768+0000 7ff4c97fa700 1 -- 172.21.15.33:0/2741262981 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff4c00d69c0 con 0x7ff4cc1000b0 2024-05-08T21:24:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.768+0000 7ff4cbfff700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4b40a11a0 0x7ff4b40a3660 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc011490 tx=0x7ff4bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:27.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:27.771+0000 7ff4c97fa700 1 -- 172.21.15.33:0/2741262981 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff4c009fe60 con 0x7ff4cc1000b0 2024-05-08T21:24:27.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:27 smithi071 ceph-mon[47074]: pgmap v454: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.9 MiB/s wr, 523 op/s 2024-05-08T21:24:27.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:27 smithi082 ceph-mon[47512]: pgmap v454: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.9 MiB/s wr, 523 op/s 2024-05-08T21:24:28.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.106+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2741262981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4cc068160 con 0x7ff4cc1000b0 2024-05-08T21:24:28.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.107+0000 7ff4c97fa700 1 -- 172.21.15.33:0/2741262981 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10914 (secure 0 0 0) 0x7ff4c00a3480 con 0x7ff4cc1000b0 2024-05-08T21:24:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:24:28.110 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:22:25.305999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:24:28.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.110+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2741262981 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4b40a11a0 msgr2=0x7ff4b40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.110+0000 7ff4d22dc700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4b40a11a0 0x7ff4b40a3660 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc011490 tx=0x7ff4bc009260 comp rx=0 tx=0).stop 2024-05-08T21:24:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.110+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2741262981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc1000b0 msgr2=0x7ff4cc199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.110+0000 7ff4d22dc700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc1000b0 0x7ff4cc199490 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005910 tx=0x7ff4c0022af0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.110+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2741262981 shutdown_connections 2024-05-08T21:24:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.110+0000 7ff4d22dc700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4b40a11a0 0x7ff4b40a3660 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.110+0000 7ff4d22dc700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4cc100a70 0x7ff4cc1a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.110+0000 7ff4d22dc700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4cc0ff750 0x7ff4cc198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.110+0000 7ff4d22dc700 1 --2- 172.21.15.33:0/2741262981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4cc1000b0 0x7ff4cc199490 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.110+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2741262981 >> 172.21.15.33:0/2741262981 conn(0x7ff4cc0fb310 msgr2=0x7ff4cc0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.110+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2741262981 shutdown_connections 2024-05-08T21:24:28.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.110+0000 7ff4d22dc700 1 -- 172.21.15.33:0/2741262981 wait complete. 2024-05-08T21:24:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 34 2024-05-08T21:24:28.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:24:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.296+0000 7f7b7c787700 1 -- 172.21.15.33:0/4118502299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b74104240 msgr2=0x7f7b74106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.296+0000 7f7b7c787700 1 --2- 172.21.15.33:0/4118502299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b74104240 0x7f7b74106630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7b640098f0 tx=0x7f7b64009c00 comp rx=0 tx=0).stop 2024-05-08T21:24:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.296+0000 7f7b7c787700 1 -- 172.21.15.33:0/4118502299 shutdown_connections 2024-05-08T21:24:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.296+0000 7f7b7c787700 1 --2- 172.21.15.33:0/4118502299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b74106b70 0x7f7b74108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.296+0000 7f7b7c787700 1 --2- 172.21.15.33:0/4118502299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b74101910 0x7f7b74103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.296+0000 7f7b7c787700 1 --2- 172.21.15.33:0/4118502299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b74104240 0x7f7b74106630 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.296+0000 7f7b7c787700 1 -- 172.21.15.33:0/4118502299 >> 172.21.15.33:0/4118502299 conn(0x7f7b740fb300 msgr2=0x7f7b740fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.296+0000 7f7b7c787700 1 -- 172.21.15.33:0/4118502299 shutdown_connections 2024-05-08T21:24:28.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.297+0000 7f7b7c787700 1 -- 172.21.15.33:0/4118502299 wait complete. 2024-05-08T21:24:28.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.298+0000 7f7b7c787700 1 Processor -- start 2024-05-08T21:24:28.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.298+0000 7f7b7c787700 1 -- start start 2024-05-08T21:24:28.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.298+0000 7f7b7c787700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b74101910 0x7f7b74101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.298+0000 7f7b7c787700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b74104240 0x7f7b740ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.298+0000 7f7b7c787700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b74106b70 0x7f7b740ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.298+0000 7f7b7c787700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741005a0 con 0x7f7b74106b70 2024-05-08T21:24:28.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.298+0000 7f7b7c787700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74100710 con 0x7f7b74101910 2024-05-08T21:24:28.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.298+0000 7f7b7c787700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741a0e00 con 0x7f7b74104240 2024-05-08T21:24:28.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.298+0000 7f7b7a523700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b74101910 0x7f7b74101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.298+0000 7f7b7a523700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b74101910 0x7f7b74101230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:48650/0 (socket says 172.21.15.33:48650) 2024-05-08T21:24:28.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.298+0000 7f7b7a523700 1 -- 172.21.15.33:0/708807272 learned_addr learned my addr 172.21.15.33:0/708807272 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.299+0000 7f7b7a523700 1 -- 172.21.15.33:0/708807272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b74104240 msgr2=0x7f7b740ff8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.299+0000 7f7b7a523700 1 --2- 172.21.15.33:0/708807272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b74104240 0x7f7b740ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.299+0000 7f7b7a523700 1 -- 172.21.15.33:0/708807272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b74106b70 msgr2=0x7f7b740ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.299+0000 7f7b7a523700 1 --2- 172.21.15.33:0/708807272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b74106b70 0x7f7b740ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.299+0000 7f7b7a523700 1 -- 172.21.15.33:0/708807272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c009d00 con 0x7f7b74101910 2024-05-08T21:24:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.299+0000 7f7b7a523700 1 --2- 172.21.15.33:0/708807272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b74101910 0x7f7b74101230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000e9a0 tx=0x7f7b7000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.299+0000 7f7b6b7fe700 1 -- 172.21.15.33:0/708807272 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000caf0 con 0x7f7b74101910 2024-05-08T21:24:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.300+0000 7f7b6b7fe700 1 -- 172.21.15.33:0/708807272 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b7000cc50 con 0x7f7b74101910 2024-05-08T21:24:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.300+0000 7f7b6b7fe700 1 -- 172.21.15.33:0/708807272 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70018660 con 0x7f7b74101910 2024-05-08T21:24:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.300+0000 7f7b7c787700 1 -- 172.21.15.33:0/708807272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b64009520 con 0x7f7b74101910 2024-05-08T21:24:28.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.300+0000 7f7b7c787700 1 -- 172.21.15.33:0/708807272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b741a13c0 con 0x7f7b74101910 2024-05-08T21:24:28.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.304+0000 7f7b88941b80 1 Processor -- start 2024-05-08T21:24:28.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.304+0000 7f7b88941b80 1 -- start start 2024-05-08T21:24:28.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.304+0000 7f7b88941b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5f540 0x565484b5f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.304+0000 7f7b88941b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5fea0 0x565484b60320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.304+0000 7f7b88941b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 0x565484b60ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.304+0000 7f7b88941b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565484ab6150 con 0x565484b5f540 2024-05-08T21:24:28.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.304+0000 7f7b88941b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565484ab3b10 con 0x565484b60860 2024-05-08T21:24:28.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.304+0000 7f7b88941b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x565484aaf400 con 0x565484b5fea0 2024-05-08T21:24:28.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b79d22700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 0x565484b60ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b7ad24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5f540 0x565484b5f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b79d22700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 0x565484b60ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:48654/0 (socket says 172.21.15.33:48654) 2024-05-08T21:24:28.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b79d22700 1 -- 172.21.15.33:0/3710476864 learned_addr learned my addr 172.21.15.33:0/3710476864 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b79d22700 1 -- 172.21.15.33:0/3710476864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5fea0 msgr2=0x565484b60320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:24:28.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b79d22700 1 --2- 172.21.15.33:0/3710476864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5fea0 0x565484b60320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b79d22700 1 -- 172.21.15.33:0/3710476864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5f540 msgr2=0x565484b5f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b79d22700 1 --2- 172.21.15.33:0/3710476864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5f540 0x565484b5f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b79d22700 1 -- 172.21.15.33:0/3710476864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654849303b0 con 0x565484b60860 2024-05-08T21:24:28.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b6b7fe700 1 -- 172.21.15.33:0/708807272 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7b70010430 con 0x7f7b74101910 2024-05-08T21:24:28.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b79d22700 1 --2- 172.21.15.33:0/3710476864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 0x565484b60ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009230 tx=0x7f7b64009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4684fe9870baf084 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b68ff9700 1 -- 172.21.15.33:0/3710476864 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6401c070 con 0x565484b60860 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b68ff9700 1 -- 172.21.15.33:0/3710476864 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b6400fc60 con 0x565484b60860 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.305+0000 7f7b68ff9700 1 -- 172.21.15.33:0/3710476864 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b64005040 con 0x565484b60860 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.306+0000 7f7b6b7fe700 1 --2- 172.21.15.33:0/708807272 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7b600a11a0 0x7f7b600a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.306+0000 7f7b7ad24700 1 --2- 172.21.15.33:0/708807272 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7b600a11a0 0x7f7b600a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.306+0000 7f7b6b7fe700 1 -- 172.21.15.33:0/708807272 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7b70014070 con 0x7f7b74101910 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.306+0000 7f7b88941b80 1 -- 172.21.15.33:0/3710476864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 msgr2=0x565484b60ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.306+0000 7f7b88941b80 1 --2- 172.21.15.33:0/3710476864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 0x565484b60ce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009230 tx=0x7f7b64009480 comp rx=0 tx=0).stop 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.306+0000 7f7b88941b80 1 -- 172.21.15.33:0/3710476864 shutdown_connections 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.306+0000 7f7b88941b80 1 --2- 172.21.15.33:0/3710476864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 0x565484b60ce0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.306+0000 7f7b88941b80 1 --2- 172.21.15.33:0/3710476864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5f540 0x565484b5f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.306+0000 7f7b88941b80 1 --2- 172.21.15.33:0/3710476864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5fea0 0x565484b60320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.306+0000 7f7b88941b80 1 -- 172.21.15.33:0/3710476864 >> 172.21.15.33:0/3710476864 conn(0x565484abd210 msgr2=0x565484b59de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.307+0000 7f7b88941b80 1 -- 172.21.15.33:0/3710476864 shutdown_connections 2024-05-08T21:24:28.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.307+0000 7f7b7ad24700 1 --2- 172.21.15.33:0/708807272 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7b600a11a0 0x7f7b600a3660 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009fd0 tx=0x7f7b6c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.307+0000 7f7b88941b80 1 -- 172.21.15.33:0/3710476864 wait complete. 2024-05-08T21:24:28.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.307+0000 7f7b88941b80 1 -- start start 2024-05-08T21:24:28.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.307+0000 7f7b88941b80 1 Processor -- start 2024-05-08T21:24:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.307+0000 7f7b88941b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5f540 0x565484b68e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.307+0000 7f7b88941b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b5fea0 0x565484b69340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.307+0000 7f7b88941b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b60860 0x565484b70380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.307+0000 7f7b88941b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565484a4b560 con 0x565484b5f540 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.307+0000 7f7b88941b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565484a4b200 con 0x565484b5fea0 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.307+0000 7f7b88941b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654849326a0 con 0x565484b60860 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b88941b80 10 client.0 _ll_register_callbacks cb 0x565484a54b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b7ad24700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b60860 0x565484b70380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b7ad24700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b60860 0x565484b70380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:33870/0 (socket says 172.21.15.33:33870) 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b7ad24700 1 -- 172.21.15.33:0/2786392676 learned_addr learned my addr 172.21.15.33:0/2786392676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b79d22700 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b5fea0 0x565484b69340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b7ad24700 1 -- 172.21.15.33:0/2786392676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b5fea0 msgr2=0x565484b69340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b7ad24700 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b5fea0 0x565484b69340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b7ad24700 1 -- 172.21.15.33:0/2786392676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5f540 msgr2=0x565484b68e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b7ad24700 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5f540 0x565484b68e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b7ad24700 1 -- 172.21.15.33:0/2786392676 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b74109bd0 con 0x565484b60860 2024-05-08T21:24:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b79d22700 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b5fea0 0x565484b69340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b7ad24700 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b60860 0x565484b70380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c017e60 tx=0x7f7b6c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2c126afc2a4a78c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.308+0000 7f7b5dffb700 1 -- 172.21.15.33:0/2786392676 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c00e040 con 0x565484b60860 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b5dffb700 1 -- 172.21.15.33:0/2786392676 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7b6c015e30 con 0x565484b60860 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b88941b80 1 -- 172.21.15.33:0/2786392676 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565484a08270 con 0x565484b60860 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b5dffb700 1 -- 172.21.15.33:0/2786392676 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b6c017e90 con 0x565484b60860 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b88941b80 10 client.26896 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b88941b80 1 -- 172.21.15.33:0/2786392676 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5654848dcb60 con 0x565484b60860 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b5dffb700 1 -- 172.21.15.33:0/2786392676 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x565484a08270 con 0x565484b60860 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b5dffb700 1 -- 172.21.15.33:0/2786392676 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f7b6c021560 con 0x565484b60860 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b88941b80 10 client.26896 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b88941b80 10 client.26896 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b88941b80 20 client.26896 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x565484b74d40 0x565484b77160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b88941b80 4 client.26896 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:24:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.309+0000 7f7b88941b80 1 -- 172.21.15.33:0/2786392676 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565484b2e810 con 0x565484b74d40 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.310+0000 7f7b7a523700 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x565484b74d40 0x565484b77160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.310+0000 7f7b7a523700 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x565484b74d40 0x565484b77160 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5898ecd3dd8e53f server_cookie=87385e876eacffea in_seq=0 out_seq=0 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.310+0000 7f7b5dffb700 10 client.26896 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b5dffb700 1 -- 172.21.15.33:0/2786392676 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565484b2e810 con 0x565484b74d40 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b5dffb700 10 client.26896 handle_command_reply: tid=0 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b88941b80 1 client.26896 shutdown 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b88941b80 1 -- 172.21.15.33:0/2786392676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b60860 msgr2=0x565484b70380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b60860 0x565484b70380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c017e60 tx=0x7f7b6c004ad0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b88941b80 1 -- 172.21.15.33:0/2786392676 shutdown_connections 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x565484b74d40 0x565484b77160 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b60860 0x565484b70380 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5f540 0x565484b68e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2786392676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b5fea0 0x565484b69340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b88941b80 1 -- 172.21.15.33:0/2786392676 >> 172.21.15.33:0/2786392676 conn(0x565484abd210 msgr2=0x565484b595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b5dffb700 0 client.26896 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:24:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b88941b80 1 -- 172.21.15.33:0/2786392676 shutdown_connections 2024-05-08T21:24:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b88941b80 1 -- 172.21.15.33:0/2786392676 wait complete. 2024-05-08T21:24:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.311+0000 7f7b88941b80 20 client.26896 trim_cache size 0 max 16384 2024-05-08T21:24:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.328+0000 7f7b88941b80 1 Processor -- start 2024-05-08T21:24:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.328+0000 7f7b88941b80 1 -- start start 2024-05-08T21:24:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b88941b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5f540 0x565484b77130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b88941b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5fea0 0x565484b70340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b88941b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 0x565484b66d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b88941b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565484b54800 con 0x565484b5fea0 2024-05-08T21:24:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b88941b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565484b720b0 con 0x565484b60860 2024-05-08T21:24:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b88941b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x565484b5b0d0 con 0x565484b5f540 2024-05-08T21:24:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b7ad24700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 0x565484b66d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b79d22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5fea0 0x565484b70340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b7a523700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5f540 0x565484b77130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b7ad24700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 0x565484b66d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:48670/0 (socket says 172.21.15.33:48670) 2024-05-08T21:24:28.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b7ad24700 1 -- 172.21.15.33:0/2479930464 learned_addr learned my addr 172.21.15.33:0/2479930464 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b7ad24700 1 -- 172.21.15.33:0/2479930464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5f540 msgr2=0x565484b77130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b7ad24700 1 --2- 172.21.15.33:0/2479930464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5f540 0x565484b77130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b7ad24700 1 -- 172.21.15.33:0/2479930464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5fea0 msgr2=0x565484b70340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b7ad24700 1 --2- 172.21.15.33:0/2479930464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5fea0 0x565484b70340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.329+0000 7f7b7ad24700 1 -- 172.21.15.33:0/2479930464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565484a75560 con 0x565484b60860 2024-05-08T21:24:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.330+0000 7f7b7ad24700 1 --2- 172.21.15.33:0/2479930464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 0x565484b66d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009350 tx=0x7f7b6c0061e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43fe927eec8e1ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.330+0000 7f7b697fa700 1 -- 172.21.15.33:0/2479930464 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c00e070 con 0x565484b60860 2024-05-08T21:24:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.330+0000 7f7b697fa700 1 -- 172.21.15.33:0/2479930464 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b6c00f040 con 0x565484b60860 2024-05-08T21:24:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.330+0000 7f7b697fa700 1 -- 172.21.15.33:0/2479930464 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c022690 con 0x565484b60860 2024-05-08T21:24:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.331+0000 7f7b88941b80 1 -- 172.21.15.33:0/2479930464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 msgr2=0x565484b66d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.331+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2479930464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 0x565484b66d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009350 tx=0x7f7b6c0061e0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.331+0000 7f7b88941b80 1 -- 172.21.15.33:0/2479930464 shutdown_connections 2024-05-08T21:24:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.331+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2479930464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b60860 0x565484b66d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.331+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2479930464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5f540 0x565484b77130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.331+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2479930464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b5fea0 0x565484b70340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.331+0000 7f7b88941b80 1 -- 172.21.15.33:0/2479930464 >> 172.21.15.33:0/2479930464 conn(0x565484abd210 msgr2=0x565484b56210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.331+0000 7f7b88941b80 1 -- 172.21.15.33:0/2479930464 shutdown_connections 2024-05-08T21:24:28.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.331+0000 7f7b88941b80 1 -- 172.21.15.33:0/2479930464 wait complete. 2024-05-08T21:24:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.331+0000 7f7b88941b80 1 -- start start 2024-05-08T21:24:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.331+0000 7f7b88941b80 1 Processor -- start 2024-05-08T21:24:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b88941b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b5f540 0x565484b6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b88941b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5fea0 0x565484b6bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b88941b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b60860 0x565484b6ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b88941b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565484b6f010 con 0x565484b60860 2024-05-08T21:24:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b88941b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565484b6f180 con 0x565484b5f540 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b88941b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x565484b6f340 con 0x565484b5fea0 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b88941b80 10 client.0 _ll_register_callbacks cb 0x565484a54b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b79d22700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5fea0 0x565484b6bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b79d22700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5fea0 0x565484b6bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:33890/0 (socket says 172.21.15.33:33890) 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b79d22700 1 -- 172.21.15.33:0/2766067215 learned_addr learned my addr 172.21.15.33:0/2766067215 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b7ad24700 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b60860 0x565484b6ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b7a523700 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b5f540 0x565484b6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b79d22700 1 -- 172.21.15.33:0/2766067215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b5f540 msgr2=0x565484b6b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b79d22700 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b5f540 0x565484b6b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b79d22700 1 -- 172.21.15.33:0/2766067215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b60860 msgr2=0x565484b6ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b79d22700 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b60860 0x565484b6ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.332+0000 7f7b79d22700 1 -- 172.21.15.33:0/2766067215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654848dcb60 con 0x565484b5fea0 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b79d22700 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5fea0 0x565484b6bab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7b640094f0 tx=0x7f7b640056a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d828fb0bf45323af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b5dffb700 1 -- 172.21.15.33:0/2766067215 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6401c040 con 0x565484b5fea0 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b5dffb700 1 -- 172.21.15.33:0/2766067215 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7b6400b970 con 0x565484b5fea0 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b88941b80 1 -- 172.21.15.33:0/2766067215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b6400fc60 con 0x565484b5fea0 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b5dffb700 1 -- 172.21.15.33:0/2766067215 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b6400f7f0 con 0x565484b5fea0 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b88941b80 10 client.26902 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b88941b80 1 -- 172.21.15.33:0/2766067215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x565484934cd0 con 0x565484b5fea0 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b5dffb700 1 -- 172.21.15.33:0/2766067215 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b6400fc60 con 0x565484b5fea0 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b5dffb700 1 -- 172.21.15.33:0/2766067215 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f7b64015020 con 0x565484b5fea0 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b88941b80 10 client.26902 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b88941b80 10 client.26902 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b88941b80 20 client.26902 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x565484bcb340 0x565484bcd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b88941b80 4 client.26902 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:24:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.333+0000 7f7b88941b80 1 -- 172.21.15.33:0/2766067215 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565484aa7ec0 con 0x565484bcb340 2024-05-08T21:24:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b7a523700 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x565484bcb340 0x565484bcd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b7a523700 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x565484bcb340 0x565484bcd760 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fceac5b93f0ee9b0 server_cookie=99d6d7e7689848c3 in_seq=0 out_seq=0 2024-05-08T21:24:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b5dffb700 10 client.26902 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:24:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b5dffb700 1 -- 172.21.15.33:0/2766067215 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565484aa7ec0 con 0x565484bcb340 2024-05-08T21:24:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b5dffb700 10 client.26902 handle_command_reply: tid=0 2024-05-08T21:24:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b88941b80 1 client.26902 shutdown 2024-05-08T21:24:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b88941b80 1 -- 172.21.15.33:0/2766067215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5fea0 msgr2=0x565484b6bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5fea0 0x565484b6bab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7b640094f0 tx=0x7f7b640056a0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b88941b80 1 -- 172.21.15.33:0/2766067215 shutdown_connections 2024-05-08T21:24:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565484b60860 0x565484b6ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565484b5f540 0x565484b6b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x565484bcb340 0x565484bcd760 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b88941b80 1 --2- 172.21.15.33:0/2766067215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x565484b5fea0 0x565484b6bab0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b5dffb700 0 client.26902 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:24:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b88941b80 1 -- 172.21.15.33:0/2766067215 >> 172.21.15.33:0/2766067215 conn(0x565484abd210 msgr2=0x565484b66d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.334+0000 7f7b88941b80 1 -- 172.21.15.33:0/2766067215 shutdown_connections 2024-05-08T21:24:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b88941b80 1 -- 172.21.15.33:0/2766067215 wait complete. 2024-05-08T21:24:28.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b88941b80 20 client.26902 trim_cache size 0 max 16384 2024-05-08T21:24:28.342 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:24:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b7c787700 1 -- 172.21.15.33:0/708807272 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7b600a11a0 msgr2=0x7f7b600a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b7c787700 1 --2- 172.21.15.33:0/708807272 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7b600a11a0 0x7f7b600a3660 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009fd0 tx=0x7f7b6c00b040 comp rx=0 tx=0).stop 2024-05-08T21:24:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b7c787700 1 -- 172.21.15.33:0/708807272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b74101910 msgr2=0x7f7b74101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b7c787700 1 --2- 172.21.15.33:0/708807272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b74101910 0x7f7b74101230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000e9a0 tx=0x7f7b7000ed60 comp rx=0 tx=0).stop 2024-05-08T21:24:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b7c787700 1 -- 172.21.15.33:0/708807272 shutdown_connections 2024-05-08T21:24:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b7c787700 1 --2- 172.21.15.33:0/708807272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b74106b70 0x7f7b740ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b7c787700 1 --2- 172.21.15.33:0/708807272 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7b600a11a0 0x7f7b600a3660 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b7c787700 1 --2- 172.21.15.33:0/708807272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b74101910 0x7f7b74101230 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b7c787700 1 --2- 172.21.15.33:0/708807272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b74104240 0x7f7b740ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b7c787700 1 -- 172.21.15.33:0/708807272 >> 172.21.15.33:0/708807272 conn(0x7f7b740fb300 msgr2=0x7f7b74102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b7c787700 1 -- 172.21.15.33:0/708807272 shutdown_connections 2024-05-08T21:24:28.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.335+0000 7f7b7c787700 1 -- 172.21.15.33:0/708807272 wait complete. 2024-05-08T21:24:28.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:24:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.520+0000 7f000846d700 1 -- 172.21.15.33:0/1899640498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00001013e0 msgr2=0x7f0000101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.520+0000 7f000846d700 1 --2- 172.21.15.33:0/1899640498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00001013e0 0x7f0000101800 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efffc009990 tx=0x7efffc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.520+0000 7f000846d700 1 -- 172.21.15.33:0/1899640498 shutdown_connections 2024-05-08T21:24:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.520+0000 7f000846d700 1 --2- 172.21.15.33:0/1899640498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00000ff890 0x7f00000ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.520+0000 7f000846d700 1 --2- 172.21.15.33:0/1899640498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00001013e0 0x7f0000101800 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.520+0000 7f000846d700 1 --2- 172.21.15.33:0/1899640498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00000feed0 0x7f00000ff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.520+0000 7f000846d700 1 -- 172.21.15.33:0/1899640498 >> 172.21.15.33:0/1899640498 conn(0x7f00000faa50 msgr2=0x7f00000fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.520+0000 7f000846d700 1 -- 172.21.15.33:0/1899640498 shutdown_connections 2024-05-08T21:24:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.520+0000 7f000846d700 1 -- 172.21.15.33:0/1899640498 wait complete. 2024-05-08T21:24:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f000846d700 1 Processor -- start 2024-05-08T21:24:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f000846d700 1 -- start start 2024-05-08T21:24:28.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f000846d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00000feed0 0x7f0000198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f000846d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00000ff890 0x7f0000199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f000846d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00001013e0 0x7f00001a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f000846d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001a0a80 con 0x7f00000feed0 2024-05-08T21:24:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f000846d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001a0bc0 con 0x7f00001013e0 2024-05-08T21:24:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f000846d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00001a0d50 con 0x7f00000ff890 2024-05-08T21:24:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f0006209700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00000feed0 0x7f0000198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f0006209700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00000feed0 0x7f0000198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55590/0 (socket says 172.21.15.33:55590) 2024-05-08T21:24:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f0006209700 1 -- 172.21.15.33:0/2650778298 learned_addr learned my addr 172.21.15.33:0/2650778298 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f0006a0a700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00001013e0 0x7f00001a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.521+0000 7f0005a08700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00000ff890 0x7f0000199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.522+0000 7f0006a0a700 1 -- 172.21.15.33:0/2650778298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00000ff890 msgr2=0x7f0000199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.522+0000 7f0006a0a700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00000ff890 0x7f0000199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.522+0000 7f0006a0a700 1 -- 172.21.15.33:0/2650778298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00000feed0 msgr2=0x7f0000198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.522+0000 7f0006a0a700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00000feed0 0x7f0000198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.522+0000 7f0006a0a700 1 -- 172.21.15.33:0/2650778298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff0009520 con 0x7f00001013e0 2024-05-08T21:24:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.522+0000 7f0006209700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00000feed0 0x7f0000198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.522+0000 7f0006a0a700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00001013e0 0x7f00001a04b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efff800efb0 tx=0x7efff800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.522+0000 7efff77fe700 1 -- 172.21.15.33:0/2650778298 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8003a00 con 0x7f00001013e0 2024-05-08T21:24:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.522+0000 7efff77fe700 1 -- 172.21.15.33:0/2650778298 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efff800f040 con 0x7f00001013e0 2024-05-08T21:24:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.522+0000 7efff77fe700 1 -- 172.21.15.33:0/2650778298 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff80044d0 con 0x7f00001013e0 2024-05-08T21:24:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.522+0000 7f000846d700 1 -- 172.21.15.33:0/2650778298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff8009c40 con 0x7f00001013e0 2024-05-08T21:24:28.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.523+0000 7f000846d700 1 -- 172.21.15.33:0/2650778298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efffc0095f0 con 0x7f00001013e0 2024-05-08T21:24:28.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.526+0000 7efff77fe700 1 -- 172.21.15.33:0/2650778298 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7efff800cbc0 con 0x7f00001013e0 2024-05-08T21:24:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.526+0000 7efff77fe700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7effec0a11a0 0x7effec0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.526+0000 7efff77fe700 1 -- 172.21.15.33:0/2650778298 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7efff80cc780 con 0x7f00001013e0 2024-05-08T21:24:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.527+0000 7f0006209700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7effec0a11a0 0x7effec0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.527+0000 7f0006209700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7effec0a11a0 0x7effec0a3660 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efffc00b2e0 tx=0x7efffc00b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.527+0000 7f0014627b80 1 Processor -- start 2024-05-08T21:24:28.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.528+0000 7f0014627b80 1 -- start start 2024-05-08T21:24:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.528+0000 7f0014627b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a7540 0x55981c1a7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.528+0000 7f0014627b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7ea0 0x55981c1a8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.528+0000 7f0014627b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a8860 0x55981c1a8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.528+0000 7f0014627b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55981c0fe150 con 0x55981c1a7ea0 2024-05-08T21:24:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.528+0000 7f0014627b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55981c0fbb10 con 0x55981c1a7540 2024-05-08T21:24:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.528+0000 7f0014627b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55981c0f7400 con 0x55981c1a8860 2024-05-08T21:24:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.528+0000 7f0006a0a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a8860 0x55981c1a8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.528+0000 7f0005a08700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7ea0 0x55981c1a8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.528+0000 7f0006a0a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a8860 0x55981c1a8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:33924/0 (socket says 172.21.15.33:33924) 2024-05-08T21:24:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.528+0000 7f0006a0a700 1 -- 172.21.15.33:0/1828983207 learned_addr learned my addr 172.21.15.33:0/1828983207 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.528+0000 7f0006209700 1 --2- 172.21.15.33:0/1828983207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a7540 0x55981c1a7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.529+0000 7f0006a0a700 1 -- 172.21.15.33:0/1828983207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a7540 msgr2=0x55981c1a7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.529+0000 7f0006a0a700 1 --2- 172.21.15.33:0/1828983207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a7540 0x55981c1a7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.529+0000 7f0006a0a700 1 -- 172.21.15.33:0/1828983207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7ea0 msgr2=0x55981c1a8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.529+0000 7f0006a0a700 1 --2- 172.21.15.33:0/1828983207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7ea0 0x55981c1a8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.529+0000 7f0006a0a700 1 -- 172.21.15.33:0/1828983207 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55981bf783b0 con 0x55981c1a8860 2024-05-08T21:24:28.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.529+0000 7f0006a0a700 1 --2- 172.21.15.33:0/1828983207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a8860 0x55981c1a8ce0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efff80090a0 tx=0x7efff80994f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72d7f33eb3e7099c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.529+0000 7efff4da4700 1 -- 172.21.15.33:0/1828983207 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8084040 con 0x55981c1a8860 2024-05-08T21:24:28.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.529+0000 7efff4da4700 1 -- 172.21.15.33:0/1828983207 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efff80998e0 con 0x55981c1a8860 2024-05-08T21:24:28.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.529+0000 7efff4da4700 1 -- 172.21.15.33:0/1828983207 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff8099da0 con 0x55981c1a8860 2024-05-08T21:24:28.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.530+0000 7f0014627b80 1 -- 172.21.15.33:0/1828983207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a8860 msgr2=0x55981c1a8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.530+0000 7f0014627b80 1 --2- 172.21.15.33:0/1828983207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a8860 0x55981c1a8ce0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efff80090a0 tx=0x7efff80994f0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.530+0000 7f0014627b80 1 -- 172.21.15.33:0/1828983207 shutdown_connections 2024-05-08T21:24:28.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.530+0000 7f0014627b80 1 --2- 172.21.15.33:0/1828983207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a8860 0x55981c1a8ce0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.530+0000 7f0014627b80 1 --2- 172.21.15.33:0/1828983207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a7540 0x55981c1a7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.530+0000 7f0014627b80 1 --2- 172.21.15.33:0/1828983207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7ea0 0x55981c1a8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.530+0000 7f0014627b80 1 -- 172.21.15.33:0/1828983207 >> 172.21.15.33:0/1828983207 conn(0x55981c105210 msgr2=0x55981c1a1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.530+0000 7f0014627b80 1 -- 172.21.15.33:0/1828983207 shutdown_connections 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.530+0000 7f0014627b80 1 -- 172.21.15.33:0/1828983207 wait complete. 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0014627b80 1 -- start start 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0014627b80 1 Processor -- start 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0014627b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7540 0x55981c1b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0014627b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7ea0 0x55981c1b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0014627b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a8860 0x55981c1c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0014627b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55981c093560 con 0x55981c1a7540 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0014627b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55981c093200 con 0x55981c1a8860 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0014627b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55981bf7a6a0 con 0x55981c1a7ea0 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0014627b80 10 client.0 _ll_register_callbacks cb 0x55981c09cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0005a08700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7ea0 0x55981c1b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0006a0a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a8860 0x55981c1c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0005a08700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7ea0 0x55981c1b9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:33938/0 (socket says 172.21.15.33:33938) 2024-05-08T21:24:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.531+0000 7f0005a08700 1 -- 172.21.15.33:0/4039219655 learned_addr learned my addr 172.21.15.33:0/4039219655 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.532+0000 7f0005a08700 1 -- 172.21.15.33:0/4039219655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a8860 msgr2=0x55981c1c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.532+0000 7f0005a08700 1 --2- 172.21.15.33:0/4039219655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a8860 0x55981c1c0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.532+0000 7f0005a08700 1 -- 172.21.15.33:0/4039219655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7540 msgr2=0x55981c1b9640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.532+0000 7f0005a08700 1 --2- 172.21.15.33:0/4039219655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7540 0x55981c1b9640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.532+0000 7f0005a08700 1 -- 172.21.15.33:0/4039219655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efffc0095f0 con 0x55981c1a7ea0 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.532+0000 7f0005a08700 1 --2- 172.21.15.33:0/4039219655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7ea0 0x55981c1b9b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efff00129e0 tx=0x7efff0012cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=380d96f5a82e41e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.532+0000 7effe9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.532+0000 7effe9ffb700 1 -- 172.21.15.33:0/4039219655 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0017070 con 0x55981c1a7ea0 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.532+0000 7effe9ffb700 1 -- 172.21.15.33:0/4039219655 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7efff000f830 con 0x55981c1a7ea0 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.532+0000 7f0014627b80 1 -- 172.21.15.33:0/4039219655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55981c050270 con 0x55981c1a7ea0 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.532+0000 7effe9ffb700 1 -- 172.21.15.33:0/4039219655 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7efff00122e0 con 0x55981c1a7ea0 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.533+0000 7f0014627b80 10 client.26914 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.533+0000 7f0014627b80 1 -- 172.21.15.33:0/4039219655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55981bf24b60 con 0x55981c1a7ea0 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.533+0000 7effe9ffb700 1 -- 172.21.15.33:0/4039219655 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55981c050270 con 0x55981c1a7ea0 2024-05-08T21:24:28.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.533+0000 7effe9ffb700 1 -- 172.21.15.33:0/4039219655 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7efff0010ae0 con 0x55981c1a7ea0 2024-05-08T21:24:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.533+0000 7f0014627b80 10 client.26914 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.533+0000 7f0014627b80 10 client.26914 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:24:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.533+0000 7f0014627b80 20 client.26914 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.533+0000 7f0014627b80 1 --2- 172.21.15.33:0/4039219655 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55981c1c5500 0x55981c1c7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.533+0000 7f0014627b80 4 client.26914 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:24:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.533+0000 7f0014627b80 1 -- 172.21.15.33:0/4039219655 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55981c176810 con 0x55981c1c5500 2024-05-08T21:24:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.533+0000 7f0006209700 1 --2- 172.21.15.33:0/4039219655 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55981c1c5500 0x55981c1c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.534+0000 7effe9ffb700 10 client.26914 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:24:28.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.534+0000 7f0006209700 1 --2- 172.21.15.33:0/4039219655 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55981c1c5500 0x55981c1c7920 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=16c9d7cc5fc65fa4 server_cookie=1f9371da93e25c59 in_seq=0 out_seq=0 2024-05-08T21:24:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.534+0000 7effe9ffb700 1 -- 172.21.15.33:0/4039219655 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55981c176810 con 0x55981c1c5500 2024-05-08T21:24:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.534+0000 7effe9ffb700 10 client.26914 handle_command_reply: tid=0 2024-05-08T21:24:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.534+0000 7f0014627b80 1 client.26914 shutdown 2024-05-08T21:24:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.534+0000 7f0014627b80 1 -- 172.21.15.33:0/4039219655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7ea0 msgr2=0x55981c1b9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.534+0000 7f0014627b80 1 --2- 172.21.15.33:0/4039219655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7ea0 0x55981c1b9b80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efff00129e0 tx=0x7efff0012cf0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.535+0000 7f0014627b80 1 -- 172.21.15.33:0/4039219655 shutdown_connections 2024-05-08T21:24:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.535+0000 7f0014627b80 1 --2- 172.21.15.33:0/4039219655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a8860 0x55981c1c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.535+0000 7f0014627b80 1 --2- 172.21.15.33:0/4039219655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7540 0x55981c1b9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.535+0000 7f0014627b80 1 --2- 172.21.15.33:0/4039219655 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55981c1c5500 0x55981c1c7920 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.535+0000 7f0014627b80 1 --2- 172.21.15.33:0/4039219655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7ea0 0x55981c1b9b80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.535+0000 7effe9ffb700 0 client.26914 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:24:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.535+0000 7f0014627b80 1 -- 172.21.15.33:0/4039219655 >> 172.21.15.33:0/4039219655 conn(0x55981c105210 msgr2=0x55981c1a15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.535+0000 7f0014627b80 1 -- 172.21.15.33:0/4039219655 shutdown_connections 2024-05-08T21:24:28.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.535+0000 7f0014627b80 1 -- 172.21.15.33:0/4039219655 wait complete. 2024-05-08T21:24:28.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.535+0000 7f0014627b80 20 client.26914 trim_cache size 0 max 16384 2024-05-08T21:24:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.553+0000 7f0014627b80 1 Processor -- start 2024-05-08T21:24:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.553+0000 7f0014627b80 1 -- start start 2024-05-08T21:24:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0014627b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 0x55981c1c1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0014627b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7ea0 0x55981c1c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0014627b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a8860 0x55981c1ab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0014627b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55981c1ba830 con 0x55981c1a7ea0 2024-05-08T21:24:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0014627b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55981c1a2800 con 0x55981c1a8860 2024-05-08T21:24:28.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0014627b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55981c1b9120 con 0x55981c1a7540 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0006a0a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a8860 0x55981c1ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0005a08700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7ea0 0x55981c1c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0006a0a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a8860 0x55981c1ab830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:48730/0 (socket says 172.21.15.33:48730) 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0006a0a700 1 -- 172.21.15.33:0/3218446898 learned_addr learned my addr 172.21.15.33:0/3218446898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0006209700 1 --2- 172.21.15.33:0/3218446898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 0x55981c1c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0005a08700 1 -- 172.21.15.33:0/3218446898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 msgr2=0x55981c1c1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0005a08700 1 --2- 172.21.15.33:0/3218446898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 0x55981c1c1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0005a08700 1 -- 172.21.15.33:0/3218446898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a8860 msgr2=0x55981c1ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0005a08700 1 --2- 172.21.15.33:0/3218446898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a8860 0x55981c1ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0005a08700 1 -- 172.21.15.33:0/3218446898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55981c0bd560 con 0x55981c1a7ea0 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0006209700 1 --2- 172.21.15.33:0/3218446898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 0x55981c1c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.554+0000 7f0005a08700 1 --2- 172.21.15.33:0/3218446898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7ea0 0x55981c1c0b80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efff0000c00 tx=0x7efff0009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773c406c4c63e76a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.555+0000 7efff55a5700 1 -- 172.21.15.33:0/3218446898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff0017070 con 0x55981c1a7ea0 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.555+0000 7efff55a5700 1 -- 172.21.15.33:0/3218446898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efff0018040 con 0x55981c1a7ea0 2024-05-08T21:24:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.555+0000 7efff55a5700 1 -- 172.21.15.33:0/3218446898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff00056d0 con 0x55981c1a7ea0 2024-05-08T21:24:28.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.556+0000 7f0014627b80 1 -- 172.21.15.33:0/3218446898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7ea0 msgr2=0x55981c1c0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.556+0000 7f0014627b80 1 --2- 172.21.15.33:0/3218446898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7ea0 0x55981c1c0b80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efff0000c00 tx=0x7efff0009400 comp rx=0 tx=0).stop 2024-05-08T21:24:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.556+0000 7f0014627b80 1 -- 172.21.15.33:0/3218446898 shutdown_connections 2024-05-08T21:24:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.556+0000 7f0014627b80 1 --2- 172.21.15.33:0/3218446898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a8860 0x55981c1ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.556+0000 7f0014627b80 1 --2- 172.21.15.33:0/3218446898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 0x55981c1c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.556+0000 7f0014627b80 1 --2- 172.21.15.33:0/3218446898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a7ea0 0x55981c1c0b80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.556+0000 7f0014627b80 1 -- 172.21.15.33:0/3218446898 >> 172.21.15.33:0/3218446898 conn(0x55981c105210 msgr2=0x55981c1aee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.556+0000 7f0014627b80 1 -- 172.21.15.33:0/3218446898 shutdown_connections 2024-05-08T21:24:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.556+0000 7f0014627b80 1 -- 172.21.15.33:0/3218446898 wait complete. 2024-05-08T21:24:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.556+0000 7f0014627b80 1 -- start start 2024-05-08T21:24:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.556+0000 7f0014627b80 1 Processor -- start 2024-05-08T21:24:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.557+0000 7f0014627b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 0x55981c1c03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.557+0000 7f0014627b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a7ea0 0x55981c1c08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.557+0000 7f0014627b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a8860 0x55981c1bce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.557+0000 7f0014627b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55981c1bd3e0 con 0x55981c1a8860 2024-05-08T21:24:28.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.557+0000 7f0014627b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55981c1bd550 con 0x55981c1a7ea0 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.557+0000 7f0014627b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55981c1bd710 con 0x55981c1a7540 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.557+0000 7f0014627b80 10 client.0 _ll_register_callbacks cb 0x55981c09cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7f0006209700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 0x55981c1c03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7f0006209700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 0x55981c1c03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:33964/0 (socket says 172.21.15.33:33964) 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7f0006209700 1 -- 172.21.15.33:0/1663960579 learned_addr learned my addr 172.21.15.33:0/1663960579 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7f0005a08700 1 --2- 172.21.15.33:0/1663960579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a7ea0 0x55981c1c08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7f0006209700 1 -- 172.21.15.33:0/1663960579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a7ea0 msgr2=0x55981c1c08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7f0006209700 1 --2- 172.21.15.33:0/1663960579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a7ea0 0x55981c1c08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7f0006209700 1 -- 172.21.15.33:0/1663960579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a8860 msgr2=0x55981c1bce50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7f0006209700 1 --2- 172.21.15.33:0/1663960579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a8860 0x55981c1bce50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7f0006209700 1 -- 172.21.15.33:0/1663960579 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0000100770 con 0x55981c1a7540 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7f0006209700 1 --2- 172.21.15.33:0/1663960579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 0x55981c1c03b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efffc005910 tx=0x7efffc009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca840638195bd468 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7effe9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:24:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7effe9ffb700 1 -- 172.21.15.33:0/1663960579 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc00bb20 con 0x55981c1a7540 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7effe9ffb700 1 -- 172.21.15.33:0/1663960579 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7efffc00bc80 con 0x55981c1a7540 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.558+0000 7f0014627b80 1 -- 172.21.15.33:0/1663960579 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efff80998e0 con 0x55981c1a7540 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.559+0000 7effe9ffb700 1 -- 172.21.15.33:0/1663960579 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7efffc00be40 con 0x55981c1a7540 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.559+0000 7f0014627b80 10 client.26920 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.559+0000 7f0014627b80 1 -- 172.21.15.33:0/1663960579 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55981bf24b60 con 0x55981c1a7540 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.559+0000 7effe9ffb700 1 -- 172.21.15.33:0/1663960579 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efff80998e0 con 0x55981c1a7540 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.559+0000 7effe9ffb700 1 -- 172.21.15.33:0/1663960579 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7efffc00f460 con 0x55981c1a7540 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.559+0000 7f0014627b80 10 client.26920 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.559+0000 7f0014627b80 10 client.26920 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.559+0000 7f0014627b80 20 client.26920 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.559+0000 7f0014627b80 1 --2- 172.21.15.33:0/1663960579 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55981c21cf70 0x55981c21f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.559+0000 7f0014627b80 4 client.26920 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.559+0000 7f0014627b80 1 -- 172.21.15.33:0/1663960579 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55981c0efec0 con 0x55981c21cf70 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.559+0000 7f0005a08700 1 --2- 172.21.15.33:0/1663960579 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55981c21cf70 0x55981c21f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.560+0000 7effe9ffb700 10 client.26920 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.560+0000 7f0005a08700 1 --2- 172.21.15.33:0/1663960579 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55981c21cf70 0x55981c21f390 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f6a66d68f61c2a7d server_cookie=203f61e3c16a5aa3 in_seq=0 out_seq=0 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.560+0000 7effe9ffb700 1 -- 172.21.15.33:0/1663960579 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55981c0efec0 con 0x55981c21cf70 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.560+0000 7effe9ffb700 10 client.26920 handle_command_reply: tid=0 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.560+0000 7f0014627b80 1 client.26920 shutdown 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.560+0000 7f0014627b80 1 -- 172.21.15.33:0/1663960579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 msgr2=0x55981c1c03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.560+0000 7f0014627b80 1 --2- 172.21.15.33:0/1663960579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 0x55981c1c03b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efffc005910 tx=0x7efffc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.560+0000 7f0014627b80 1 -- 172.21.15.33:0/1663960579 shutdown_connections 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.560+0000 7f0014627b80 1 --2- 172.21.15.33:0/1663960579 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55981c1a8860 0x55981c1bce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f0014627b80 1 --2- 172.21.15.33:0/1663960579 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55981c21cf70 0x55981c21f390 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f0014627b80 1 --2- 172.21.15.33:0/1663960579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55981c1a7540 0x55981c1c03b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f0014627b80 1 --2- 172.21.15.33:0/1663960579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55981c1a7ea0 0x55981c1c08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f0014627b80 1 -- 172.21.15.33:0/1663960579 >> 172.21.15.33:0/1663960579 conn(0x55981c105210 msgr2=0x55981c1ab830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7effe9ffb700 0 client.26920 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:24:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f0014627b80 1 -- 172.21.15.33:0/1663960579 shutdown_connections 2024-05-08T21:24:28.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f0014627b80 1 -- 172.21.15.33:0/1663960579 wait complete. 2024-05-08T21:24:28.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f0014627b80 20 client.26920 trim_cache size 0 max 16384 2024-05-08T21:24:28.567 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:24:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f000846d700 1 -- 172.21.15.33:0/2650778298 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7effec0a11a0 msgr2=0x7effec0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f000846d700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7effec0a11a0 0x7effec0a3660 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efffc00b2e0 tx=0x7efffc00b610 comp rx=0 tx=0).stop 2024-05-08T21:24:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f000846d700 1 -- 172.21.15.33:0/2650778298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00001013e0 msgr2=0x7f00001a04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f000846d700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00001013e0 0x7f00001a04b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efff800efb0 tx=0x7efff800c3a0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f000846d700 1 -- 172.21.15.33:0/2650778298 shutdown_connections 2024-05-08T21:24:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f000846d700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7effec0a11a0 0x7effec0a3660 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f000846d700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00001013e0 0x7f00001a04b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f000846d700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00000feed0 0x7f0000198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f000846d700 1 --2- 172.21.15.33:0/2650778298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00000ff890 0x7f0000199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f000846d700 1 -- 172.21.15.33:0/2650778298 >> 172.21.15.33:0/2650778298 conn(0x7f00000faa50 msgr2=0x7f00000fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.561+0000 7f000846d700 1 -- 172.21.15.33:0/2650778298 shutdown_connections 2024-05-08T21:24:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.562+0000 7f000846d700 1 -- 172.21.15.33:0/2650778298 wait complete. 2024-05-08T21:24:28.580 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:24:28.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:28 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2741262981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:24:28.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.735+0000 7f48b6c05700 1 -- 172.21.15.33:0/3841513045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b01030a0 msgr2=0x7f48b01034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.735+0000 7f48b6c05700 1 --2- 172.21.15.33:0/3841513045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b01030a0 0x7f48b01034c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009990 tx=0x7f48a4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.735+0000 7f48b6c05700 1 -- 172.21.15.33:0/3841513045 shutdown_connections 2024-05-08T21:24:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.735+0000 7f48b6c05700 1 --2- 172.21.15.33:0/3841513045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0104c30 0x7f48b010b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.735+0000 7f48b6c05700 1 --2- 172.21.15.33:0/3841513045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b01030a0 0x7f48b01034c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.735+0000 7f48b6c05700 1 --2- 172.21.15.33:0/3841513045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48b0104290 0x7f48b01046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.735+0000 7f48b6c05700 1 -- 172.21.15.33:0/3841513045 >> 172.21.15.33:0/3841513045 conn(0x7f48b00fe610 msgr2=0x7f48b0100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.735+0000 7f48b6c05700 1 -- 172.21.15.33:0/3841513045 shutdown_connections 2024-05-08T21:24:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.735+0000 7f48b6c05700 1 -- 172.21.15.33:0/3841513045 wait complete. 2024-05-08T21:24:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.736+0000 7f48b6c05700 1 Processor -- start 2024-05-08T21:24:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.736+0000 7f48b6c05700 1 -- start start 2024-05-08T21:24:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48b6c05700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48b01030a0 0x7f48b0198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48b6c05700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0104290 0x7f48b01994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48b6c05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0104c30 0x7f48b01a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48b6c05700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b01a0ac0 con 0x7f48b0104c30 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48b6c05700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b01a0c00 con 0x7f48b01030a0 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48b6c05700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b01a0d90 con 0x7f48b0104290 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48affff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0104290 0x7f48b01994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48affff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0104290 0x7f48b01994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:33982/0 (socket says 172.21.15.33:33982) 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48affff700 1 -- 172.21.15.33:0/131483750 learned_addr learned my addr 172.21.15.33:0/131483750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48b49a1700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48b01030a0 0x7f48b0198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48affff700 1 -- 172.21.15.33:0/131483750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48b01030a0 msgr2=0x7f48b0198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48affff700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48b01030a0 0x7f48b0198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48affff700 1 -- 172.21.15.33:0/131483750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0104c30 msgr2=0x7f48b01a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48affff700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0104c30 0x7f48b01a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48affff700 1 -- 172.21.15.33:0/131483750 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a40095f0 con 0x7f48b0104290 2024-05-08T21:24:28.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.737+0000 7f48affff700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0104290 0x7f48b01994d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f489c00e9a0 tx=0x7f489c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.738+0000 7f48adffb700 1 -- 172.21.15.33:0/131483750 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c009790 con 0x7f48b0104290 2024-05-08T21:24:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.738+0000 7f48adffb700 1 -- 172.21.15.33:0/131483750 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f489c010430 con 0x7f48b0104290 2024-05-08T21:24:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.738+0000 7f48b6c05700 1 -- 172.21.15.33:0/131483750 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b01a1070 con 0x7f48b0104290 2024-05-08T21:24:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.738+0000 7f48adffb700 1 -- 172.21.15.33:0/131483750 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c018680 con 0x7f48b0104290 2024-05-08T21:24:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.738+0000 7f48b6c05700 1 -- 172.21.15.33:0/131483750 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b01a1580 con 0x7f48b0104290 2024-05-08T21:24:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.740+0000 7f48adffb700 1 -- 172.21.15.33:0/131483750 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f489c00cb50 con 0x7f48b0104290 2024-05-08T21:24:28.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.741+0000 7f48adffb700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48a00a1120 0x7f48a00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.741+0000 7f48adffb700 1 -- 172.21.15.33:0/131483750 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f489c014070 con 0x7f48b0104290 2024-05-08T21:24:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.742+0000 7f48b49a1700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48a00a1120 0x7f48a00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.742+0000 7f48c2dbfb80 1 Processor -- start 2024-05-08T21:24:28.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.742+0000 7f48c2dbfb80 1 -- start start 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.742+0000 7f48b49a1700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48a00a1120 0x7f48a00a35e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f48a400b2e0 tx=0x7f48a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48c2dbfb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 0x562671d5b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48c2dbfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5bea0 0x562671d5c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48c2dbfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5c860 0x562671d5cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48c2dbfb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671cb2150 con 0x562671d5b540 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48c2dbfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671cafb10 con 0x562671d5bea0 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48c2dbfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671cab400 con 0x562671d5c860 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48b51a2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5c860 0x562671d5cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48b51a2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5c860 0x562671d5cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:33992/0 (socket says 172.21.15.33:33992) 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48b51a2700 1 -- 172.21.15.33:0/2042130536 learned_addr learned my addr 172.21.15.33:0/2042130536 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48affff700 1 --2- 172.21.15.33:0/2042130536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5bea0 0x562671d5c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48b49a1700 1 --2- 172.21.15.33:0/2042130536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 0x562671d5b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48b51a2700 1 -- 172.21.15.33:0/2042130536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5bea0 msgr2=0x562671d5c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48b51a2700 1 --2- 172.21.15.33:0/2042130536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5bea0 0x562671d5c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48b51a2700 1 -- 172.21.15.33:0/2042130536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 msgr2=0x562671d5b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48b51a2700 1 --2- 172.21.15.33:0/2042130536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 0x562671d5b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48b51a2700 1 -- 172.21.15.33:0/2042130536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562671b2c3b0 con 0x562671d5c860 2024-05-08T21:24:28.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48affff700 1 --2- 172.21.15.33:0/2042130536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5bea0 0x562671d5c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:24:28.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48b49a1700 1 --2- 172.21.15.33:0/2042130536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 0x562671d5b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:24:28.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.743+0000 7f48b51a2700 1 --2- 172.21.15.33:0/2042130536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5c860 0x562671d5cce0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f48a800e430 tx=0x7f48a800e740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=389d285c4915bd9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.744+0000 7f489affd700 1 -- 172.21.15.33:0/2042130536 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a8018070 con 0x562671d5c860 2024-05-08T21:24:28.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.744+0000 7f489affd700 1 -- 172.21.15.33:0/2042130536 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48a800f040 con 0x562671d5c860 2024-05-08T21:24:28.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.744+0000 7f489affd700 1 -- 172.21.15.33:0/2042130536 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a8004280 con 0x562671d5c860 2024-05-08T21:24:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.744+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/2042130536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5c860 msgr2=0x562671d5cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.744+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/2042130536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5c860 0x562671d5cce0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f48a800e430 tx=0x7f48a800e740 comp rx=0 tx=0).stop 2024-05-08T21:24:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.745+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/2042130536 shutdown_connections 2024-05-08T21:24:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.745+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/2042130536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5c860 0x562671d5cce0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.745+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/2042130536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 0x562671d5b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.745+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/2042130536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5bea0 0x562671d5c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.745+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/2042130536 >> 172.21.15.33:0/2042130536 conn(0x562671cb9210 msgr2=0x562671d55de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.745+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/2042130536 shutdown_connections 2024-05-08T21:24:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.745+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/2042130536 wait complete. 2024-05-08T21:24:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.745+0000 7f48c2dbfb80 1 -- start start 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.745+0000 7f48c2dbfb80 1 Processor -- start 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48c2dbfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5b540 0x562671d64dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48c2dbfb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 0x562671d65300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48c2dbfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 0x562671d6c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48c2dbfb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671c47560 con 0x562671d5bea0 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48c2dbfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671c47200 con 0x562671d5c860 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48c2dbfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671b2e6a0 con 0x562671d5b540 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48c2dbfb80 10 client.0 _ll_register_callbacks cb 0x562671c50b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 0x562671d65300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 0x562671d65300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55650/0 (socket says 172.21.15.33:55650) 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48affff700 1 -- 172.21.15.33:0/4000741809 learned_addr learned my addr 172.21.15.33:0/4000741809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48b49a1700 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5b540 0x562671d64dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48b51a2700 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 0x562671d6c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48b49a1700 1 -- 172.21.15.33:0/4000741809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 msgr2=0x562671d6c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48b49a1700 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 0x562671d6c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48b49a1700 1 -- 172.21.15.33:0/4000741809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 msgr2=0x562671d65300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48b49a1700 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 0x562671d65300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.746+0000 7f48b49a1700 1 -- 172.21.15.33:0/4000741809 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a800e140 con 0x562671d5b540 2024-05-08T21:24:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f48b49a1700 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5b540 0x562671d64dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f48a4000c00 tx=0x7f48a401a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=126c5ae2c5d7121 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f488ffff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:24:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f488ffff700 1 -- 172.21.15.33:0/4000741809 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4003ae0 con 0x562671d5b540 2024-05-08T21:24:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f488ffff700 1 -- 172.21.15.33:0/4000741809 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f48a4003c40 con 0x562671d5b540 2024-05-08T21:24:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/4000741809 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562671c04270 con 0x562671d5b540 2024-05-08T21:24:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f488ffff700 1 -- 172.21.15.33:0/4000741809 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48a4004500 con 0x562671d5b540 2024-05-08T21:24:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f48c2dbfb80 10 client.26932 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/4000741809 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562671ad8b60 con 0x562671d5b540 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f488ffff700 1 -- 172.21.15.33:0/4000741809 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562671c04270 con 0x562671d5b540 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f488ffff700 1 -- 172.21.15.33:0/4000741809 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f48a400f460 con 0x562671d5b540 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f48c2dbfb80 10 client.26932 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f48c2dbfb80 10 client.26932 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f48c2dbfb80 20 client.26932 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562671d70bc0 0x562671d72fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f48c2dbfb80 4 client.26932 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.747+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/4000741809 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562671d2a810 con 0x562671d70bc0 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.748+0000 7f48affff700 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562671d70bc0 0x562671d72fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.748+0000 7f48affff700 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562671d70bc0 0x562671d72fe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1df315a58c58945c server_cookie=3a6581af52b50a6e in_seq=0 out_seq=0 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.748+0000 7f488ffff700 10 client.26932 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:24:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f488ffff700 1 -- 172.21.15.33:0/4000741809 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562671d2a810 con 0x562671d70bc0 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f488ffff700 10 client.26932 handle_command_reply: tid=0 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f48c2dbfb80 1 client.26932 shutdown 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/4000741809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5b540 msgr2=0x562671d64dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5b540 0x562671d64dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f48a4000c00 tx=0x7f48a401a050 comp rx=0 tx=0).stop 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/4000741809 shutdown_connections 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562671d70bc0 0x562671d72fe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 0x562671d6c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5b540 0x562671d64dc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/4000741809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 0x562671d65300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/4000741809 >> 172.21.15.33:0/4000741809 conn(0x562671cb9210 msgr2=0x562671d555a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f488ffff700 0 client.26932 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/4000741809 shutdown_connections 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/4000741809 wait complete. 2024-05-08T21:24:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.749+0000 7f48c2dbfb80 20 client.26932 trim_cache size 0 max 16384 2024-05-08T21:24:28.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.766+0000 7f48c2dbfb80 1 Processor -- start 2024-05-08T21:24:28.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.766+0000 7f48c2dbfb80 1 -- start start 2024-05-08T21:24:28.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48c2dbfb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 0x562671d64340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48c2dbfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5bea0 0x562671d6c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48c2dbfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 0x562671d55db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48c2dbfb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671d56a70 con 0x562671d5b540 2024-05-08T21:24:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48c2dbfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671d50800 con 0x562671d5c860 2024-05-08T21:24:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48c2dbfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671d57830 con 0x562671d5bea0 2024-05-08T21:24:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48b49a1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 0x562671d64340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48affff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5bea0 0x562671d6c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48b49a1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 0x562671d64340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55664/0 (socket says 172.21.15.33:55664) 2024-05-08T21:24:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48affff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5bea0 0x562671d6c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:34022/0 (socket says 172.21.15.33:34022) 2024-05-08T21:24:28.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48b49a1700 1 -- 172.21.15.33:0/3115834944 learned_addr learned my addr 172.21.15.33:0/3115834944 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48b49a1700 1 -- 172.21.15.33:0/3115834944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5bea0 msgr2=0x562671d6c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48b49a1700 1 --2- 172.21.15.33:0/3115834944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5bea0 0x562671d6c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48b49a1700 1 -- 172.21.15.33:0/3115834944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 msgr2=0x562671d55db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48b49a1700 1 --2- 172.21.15.33:0/3115834944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 0x562671d55db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48b49a1700 1 -- 172.21.15.33:0/3115834944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562671c71560 con 0x562671d5b540 2024-05-08T21:24:28.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.767+0000 7f48b49a1700 1 --2- 172.21.15.33:0/3115834944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 0x562671d64340 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009ce0 tx=0x7f48a4009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64be1e186897e75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.768+0000 7f489b7fe700 1 -- 172.21.15.33:0/3115834944 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a400f4e0 con 0x562671d5b540 2024-05-08T21:24:28.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.768+0000 7f489b7fe700 1 -- 172.21.15.33:0/3115834944 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48a401e040 con 0x562671d5b540 2024-05-08T21:24:28.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.768+0000 7f489b7fe700 1 -- 172.21.15.33:0/3115834944 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4030860 con 0x562671d5b540 2024-05-08T21:24:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.768+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/3115834944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 msgr2=0x562671d64340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.768+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/3115834944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 0x562671d64340 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009ce0 tx=0x7f48a4009290 comp rx=0 tx=0).stop 2024-05-08T21:24:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.769+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/3115834944 shutdown_connections 2024-05-08T21:24:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.769+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/3115834944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 0x562671d55db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.769+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/3115834944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5b540 0x562671d64340 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.769+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/3115834944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5bea0 0x562671d6c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.769+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/3115834944 >> 172.21.15.33:0/3115834944 conn(0x562671cb9210 msgr2=0x562671d62e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.769+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/3115834944 shutdown_connections 2024-05-08T21:24:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.769+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/3115834944 wait complete. 2024-05-08T21:24:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.769+0000 7f48c2dbfb80 1 -- start start 2024-05-08T21:24:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.769+0000 7f48c2dbfb80 1 Processor -- start 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48c2dbfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5b540 0x562671d67b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48c2dbfb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 0x562671d680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48c2dbfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 0x562671d6b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48c2dbfb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671d6b610 con 0x562671d5bea0 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48c2dbfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671d6b780 con 0x562671d5c860 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48c2dbfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562671d6b940 con 0x562671d5b540 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48c2dbfb80 10 client.0 _ll_register_callbacks cb 0x562671c50b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 0x562671d680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 0x562671d680b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55666/0 (socket says 172.21.15.33:55666) 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48affff700 1 -- 172.21.15.33:0/1767175452 learned_addr learned my addr 172.21.15.33:0/1767175452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48b51a2700 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 0x562671d6b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48b49a1700 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5b540 0x562671d67b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48affff700 1 -- 172.21.15.33:0/1767175452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5b540 msgr2=0x562671d67b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48affff700 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5b540 0x562671d67b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48affff700 1 -- 172.21.15.33:0/1767175452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 msgr2=0x562671d6b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48affff700 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 0x562671d6b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.770+0000 7f48affff700 1 -- 172.21.15.33:0/1767175452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562671ad8b60 con 0x562671d5bea0 2024-05-08T21:24:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.771+0000 7f48affff700 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 0x562671d680b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f489c099f10 tx=0x7f489c099f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fbf5253d4922b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.771+0000 7f488ffff700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:24:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.772+0000 7f488ffff700 1 -- 172.21.15.33:0/1767175452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0a2400 con 0x562671d5bea0 2024-05-08T21:24:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.772+0000 7f488ffff700 1 -- 172.21.15.33:0/1767175452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f489c098320 con 0x562671d5bea0 2024-05-08T21:24:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.772+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/1767175452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48a800f040 con 0x562671d5bea0 2024-05-08T21:24:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.772+0000 7f488ffff700 1 -- 172.21.15.33:0/1767175452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f489c0105a0 con 0x562671d5bea0 2024-05-08T21:24:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.772+0000 7f48c2dbfb80 10 client.17037 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.772+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/1767175452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562671b30cd0 con 0x562671d5bea0 2024-05-08T21:24:28.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.772+0000 7f488ffff700 1 -- 172.21.15.33:0/1767175452 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48a800f040 con 0x562671d5bea0 2024-05-08T21:24:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.773+0000 7f488ffff700 1 -- 172.21.15.33:0/1767175452 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f489c0a1fe0 con 0x562671d5bea0 2024-05-08T21:24:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.773+0000 7f48c2dbfb80 10 client.17037 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.773+0000 7f48c2dbfb80 10 client.17037 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:24:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.773+0000 7f48c2dbfb80 20 client.17037 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.773+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562671dc7b90 0x562671dc9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.773+0000 7f48c2dbfb80 4 client.17037 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:24:28.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.773+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/1767175452 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562671ca3ec0 con 0x562671dc7b90 2024-05-08T21:24:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.774+0000 7f48b49a1700 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562671dc7b90 0x562671dc9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.775+0000 7f48b49a1700 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562671dc7b90 0x562671dc9fb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=438cb3ff06c8d099 server_cookie=c478f1d5aeb9823c in_seq=0 out_seq=0 2024-05-08T21:24:28.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.775+0000 7f488ffff700 10 client.17037 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.775+0000 7f488ffff700 1 -- 172.21.15.33:0/1767175452 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562671ca3ec0 con 0x562671dc7b90 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.775+0000 7f488ffff700 10 client.17037 handle_command_reply: tid=0 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.775+0000 7f48c2dbfb80 1 client.17037 shutdown 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.776+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/1767175452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 msgr2=0x562671d680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.776+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 0x562671d680b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f489c099f10 tx=0x7f489c099f80 comp rx=0 tx=0).stop 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.776+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/1767175452 shutdown_connections 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.776+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562671dc7b90 0x562671dc9fb0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.776+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562671d5c860 0x562671d6b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.776+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562671d5b540 0x562671d67b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.776+0000 7f48c2dbfb80 1 --2- 172.21.15.33:0/1767175452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562671d5bea0 0x562671d680b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.776+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/1767175452 >> 172.21.15.33:0/1767175452 conn(0x562671cb9210 msgr2=0x562671d77940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.776+0000 7f488ffff700 0 client.17037 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.776+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/1767175452 shutdown_connections 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.776+0000 7f48c2dbfb80 1 -- 172.21.15.33:0/1767175452 wait complete. 2024-05-08T21:24:28.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.776+0000 7f48c2dbfb80 20 client.17037 trim_cache size 0 max 16384 2024-05-08T21:24:28.780 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:24:28.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.777+0000 7f48b6c05700 1 -- 172.21.15.33:0/131483750 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48a00a1120 msgr2=0x7f48a00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.777+0000 7f48b6c05700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48a00a1120 0x7f48a00a35e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f48a400b2e0 tx=0x7f48a4005fd0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.777+0000 7f48b6c05700 1 -- 172.21.15.33:0/131483750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0104290 msgr2=0x7f48b01994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.777+0000 7f48b6c05700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0104290 0x7f48b01994d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f489c00e9a0 tx=0x7f489c00ecb0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.777+0000 7f48b6c05700 1 -- 172.21.15.33:0/131483750 shutdown_connections 2024-05-08T21:24:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.777+0000 7f48b6c05700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48b0104c30 0x7f48b01a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.777+0000 7f48b6c05700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f48a00a1120 0x7f48a00a35e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.777+0000 7f48b6c05700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48b01030a0 0x7f48b0198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.777+0000 7f48b6c05700 1 --2- 172.21.15.33:0/131483750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b0104290 0x7f48b01994d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.777+0000 7f48b6c05700 1 -- 172.21.15.33:0/131483750 >> 172.21.15.33:0/131483750 conn(0x7f48b00fe610 msgr2=0x7f48b0107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.777+0000 7f48b6c05700 1 -- 172.21.15.33:0/131483750 shutdown_connections 2024-05-08T21:24:28.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.777+0000 7f48b6c05700 1 -- 172.21.15.33:0/131483750 wait complete. 2024-05-08T21:24:28.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:24:28.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:28 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2741262981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:24:28.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:28 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2741262981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:24:28.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.960+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/862457422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4cc100f50 msgr2=0x7fd4cc1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.960+0000 7fd4d3ee8700 1 --2- 172.21.15.33:0/862457422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4cc100f50 0x7fd4cc1013b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0099c0 tx=0x7fd4bc009cd0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.960+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/862457422 shutdown_connections 2024-05-08T21:24:28.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.960+0000 7fd4d3ee8700 1 --2- 172.21.15.33:0/862457422 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4cc1018f0 0x7fd4cc1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.960+0000 7fd4d3ee8700 1 --2- 172.21.15.33:0/862457422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4cc0ffd60 0x7fd4cc100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.960+0000 7fd4d3ee8700 1 --2- 172.21.15.33:0/862457422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4cc100f50 0x7fd4cc1013b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.960+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/862457422 >> 172.21.15.33:0/862457422 conn(0x7fd4cc0fb290 msgr2=0x7fd4cc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.960+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/862457422 shutdown_connections 2024-05-08T21:24:28.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.960+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/862457422 wait complete. 2024-05-08T21:24:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.961+0000 7fd4d3ee8700 1 Processor -- start 2024-05-08T21:24:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.961+0000 7fd4d3ee8700 1 -- start start 2024-05-08T21:24:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.961+0000 7fd4d3ee8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4cc0ffd60 0x7fd4cc198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.961+0000 7fd4d3ee8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4cc100f50 0x7fd4cc199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.961+0000 7fd4d3ee8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4cc1018f0 0x7fd4cc1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.961+0000 7fd4d3ee8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc1a0a20 con 0x7fd4cc0ffd60 2024-05-08T21:24:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.961+0000 7fd4d3ee8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc1a0b60 con 0x7fd4cc1018f0 2024-05-08T21:24:28.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.961+0000 7fd4d3ee8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc1a0cf0 con 0x7fd4cc100f50 2024-05-08T21:24:28.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4d1c84700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4cc0ffd60 0x7fd4cc198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4d2485700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4cc1018f0 0x7fd4cc1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4d1c84700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4cc0ffd60 0x7fd4cc198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55680/0 (socket says 172.21.15.33:55680) 2024-05-08T21:24:28.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4d1c84700 1 -- 172.21.15.33:0/1398727461 learned_addr learned my addr 172.21.15.33:0/1398727461 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4d1483700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4cc100f50 0x7fd4cc199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4d2485700 1 -- 172.21.15.33:0/1398727461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4cc100f50 msgr2=0x7fd4cc199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4d2485700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4cc100f50 0x7fd4cc199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4d2485700 1 -- 172.21.15.33:0/1398727461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4cc0ffd60 msgr2=0x7fd4cc198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4d2485700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4cc0ffd60 0x7fd4cc198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4d2485700 1 -- 172.21.15.33:0/1398727461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc0095f0 con 0x7fd4cc1018f0 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4d1c84700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4cc0ffd60 0x7fd4cc198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4d2485700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4cc1018f0 0x7fd4cc1a0450 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4007f10 tx=0x7fd4c400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.962+0000 7fd4c2ffd700 1 -- 172.21.15.33:0/1398727461 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400ddd0 con 0x7fd4cc1018f0 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.963+0000 7fd4c2ffd700 1 -- 172.21.15.33:0/1398727461 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4c400f040 con 0x7fd4cc1018f0 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.963+0000 7fd4c2ffd700 1 -- 172.21.15.33:0/1398727461 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40059c0 con 0x7fd4cc1018f0 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.963+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/1398727461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc06aa20 con 0x7fd4cc1018f0 2024-05-08T21:24:28.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.963+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/1398727461 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc06aee0 con 0x7fd4cc1018f0 2024-05-08T21:24:28.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.965+0000 7fd4c2ffd700 1 -- 172.21.15.33:0/1398727461 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd4c4005020 con 0x7fd4cc1018f0 2024-05-08T21:24:28.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.966+0000 7fd4c2ffd700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4b80a11a0 0x7fd4b80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.966+0000 7fd4c2ffd700 1 -- 172.21.15.33:0/1398727461 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd4c40cc480 con 0x7fd4cc1018f0 2024-05-08T21:24:28.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.966+0000 7fd4d1c84700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4b80a11a0 0x7fd4b80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.967+0000 7fd4d1c84700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4b80a11a0 0x7fd4b80a3660 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8005950 tx=0x7fd4c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.967+0000 7fd4e00a2b80 1 Processor -- start 2024-05-08T21:24:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.967+0000 7fd4e00a2b80 1 -- start start 2024-05-08T21:24:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4e00a2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012a2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4e00a2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a2ea0 0x5604012a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4e00a2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a3860 0x5604012a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4e00a2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604011f9150 con 0x5604012a3860 2024-05-08T21:24:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4e00a2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604011f6b10 con 0x5604012a2ea0 2024-05-08T21:24:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4e00a2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604011f2400 con 0x5604012a2540 2024-05-08T21:24:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4d2485700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a3860 0x5604012a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4d2485700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a3860 0x5604012a3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55682/0 (socket says 172.21.15.33:55682) 2024-05-08T21:24:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4d2485700 1 -- 172.21.15.33:0/3125410510 learned_addr learned my addr 172.21.15.33:0/3125410510 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4d1c84700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4d2485700 1 -- 172.21.15.33:0/3125410510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 msgr2=0x5604012a2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4d2485700 1 --2- 172.21.15.33:0/3125410510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012a2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4d2485700 1 -- 172.21.15.33:0/3125410510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a2ea0 msgr2=0x5604012a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4d2485700 1 --2- 172.21.15.33:0/3125410510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a2ea0 0x5604012a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.968+0000 7fd4d2485700 1 -- 172.21.15.33:0/3125410510 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604010733b0 con 0x5604012a3860 2024-05-08T21:24:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.969+0000 7fd4d2485700 1 --2- 172.21.15.33:0/3125410510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a3860 0x5604012a3ce0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40994e0 tx=0x7fd4c40993f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e9d700349ec558 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.969+0000 7fd4b7fff700 1 -- 172.21.15.33:0/3125410510 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4099c50 con 0x5604012a3860 2024-05-08T21:24:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.969+0000 7fd4b7fff700 1 -- 172.21.15.33:0/3125410510 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4c4009e20 con 0x5604012a3860 2024-05-08T21:24:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.969+0000 7fd4b7fff700 1 -- 172.21.15.33:0/3125410510 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40978f0 con 0x5604012a3860 2024-05-08T21:24:28.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.970+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/3125410510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a3860 msgr2=0x5604012a3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.970+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/3125410510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a3860 0x5604012a3ce0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40994e0 tx=0x7fd4c40993f0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.971+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/3125410510 shutdown_connections 2024-05-08T21:24:28.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.971+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/3125410510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a3860 0x5604012a3ce0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.971+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/3125410510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.971+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/3125410510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a2ea0 0x5604012a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.971+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/3125410510 >> 172.21.15.33:0/3125410510 conn(0x560401200210 msgr2=0x56040129cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.971+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/3125410510 shutdown_connections 2024-05-08T21:24:28.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.971+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/3125410510 wait complete. 2024-05-08T21:24:28.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.971+0000 7fd4e00a2b80 1 -- start start 2024-05-08T21:24:28.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.971+0000 7fd4e00a2b80 1 Processor -- start 2024-05-08T21:24:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.972+0000 7fd4e00a2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.972+0000 7fd4e00a2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a2ea0 0x5604012ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.972+0000 7fd4e00a2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a3860 0x5604012b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.972+0000 7fd4e00a2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040118e560 con 0x5604012a3860 2024-05-08T21:24:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.972+0000 7fd4e00a2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56040118e200 con 0x5604012a2ea0 2024-05-08T21:24:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.972+0000 7fd4e00a2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604010756a0 con 0x5604012a2540 2024-05-08T21:24:28.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.972+0000 7fd4e00a2b80 10 client.0 _ll_register_callbacks cb 0x560401197b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:28.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.972+0000 7fd4d1483700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a2ea0 0x5604012ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.972+0000 7fd4d1483700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a2ea0 0x5604012ac3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:48830/0 (socket says 172.21.15.33:48830) 2024-05-08T21:24:28.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.972+0000 7fd4d1483700 1 -- 172.21.15.33:0/2862658521 learned_addr learned my addr 172.21.15.33:0/2862658521 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.973+0000 7fd4d2485700 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a3860 0x5604012b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.973+0000 7fd4d1c84700 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.973+0000 7fd4d1483700 1 -- 172.21.15.33:0/2862658521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 msgr2=0x5604012abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.973+0000 7fd4d1483700 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.973+0000 7fd4d1483700 1 -- 172.21.15.33:0/2862658521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a3860 msgr2=0x5604012b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.973+0000 7fd4d1483700 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a3860 0x5604012b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.973+0000 7fd4d1483700 1 -- 172.21.15.33:0/2862658521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c4095040 con 0x5604012a2ea0 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.973+0000 7fd4d1483700 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a2ea0 0x5604012ac3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00b580 tx=0x7fd4bc005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57edb67d8f1a5991 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.973+0000 7fd4b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4b57fa700 1 -- 172.21.15.33:0/2862658521 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc00bd20 con 0x5604012a2ea0 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4b57fa700 1 -- 172.21.15.33:0/2862658521 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd4bc003760 con 0x5604012a2ea0 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/2862658521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56040114b270 con 0x5604012a2ea0 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4b57fa700 1 -- 172.21.15.33:0/2862658521 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4bc0058e0 con 0x5604012a2ea0 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4e00a2b80 10 client.26936 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/2862658521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56040101fb60 con 0x5604012a2ea0 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4b57fa700 1 -- 172.21.15.33:0/2862658521 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56040114b270 con 0x5604012a2ea0 2024-05-08T21:24:28.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4b57fa700 1 -- 172.21.15.33:0/2862658521 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fd4bc00f930 con 0x5604012a2ea0 2024-05-08T21:24:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4e00a2b80 10 client.26936 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4e00a2b80 10 client.26936 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:24:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4e00a2b80 20 client.26936 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5604012b7c40 0x5604012ba060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4e00a2b80 4 client.26936 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:24:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.974+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/2862658521 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560401271810 con 0x5604012b7c40 2024-05-08T21:24:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.975+0000 7fd4d1c84700 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5604012b7c40 0x5604012ba060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.975+0000 7fd4d1c84700 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5604012b7c40 0x5604012ba060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c9acb0b7aa6f969 server_cookie=e9fc116f1fcc993b in_seq=0 out_seq=0 2024-05-08T21:24:28.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.975+0000 7fd4b57fa700 10 client.26936 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:24:28.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.975+0000 7fd4b57fa700 1 -- 172.21.15.33:0/2862658521 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560401271810 con 0x5604012b7c40 2024-05-08T21:24:28.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.975+0000 7fd4b57fa700 10 client.26936 handle_command_reply: tid=0 2024-05-08T21:24:28.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4e00a2b80 1 client.26936 shutdown 2024-05-08T21:24:28.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/2862658521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a2ea0 msgr2=0x5604012ac3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:28.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a2ea0 0x5604012ac3c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00b580 tx=0x7fd4bc005270 comp rx=0 tx=0).stop 2024-05-08T21:24:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/2862658521 shutdown_connections 2024-05-08T21:24:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5604012b7c40 0x5604012ba060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a3860 0x5604012b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/2862658521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a2ea0 0x5604012ac3c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/2862658521 >> 172.21.15.33:0/2862658521 conn(0x560401200210 msgr2=0x56040129c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4b57fa700 0 client.26936 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:24:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/2862658521 shutdown_connections 2024-05-08T21:24:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/2862658521 wait complete. 2024-05-08T21:24:28.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.976+0000 7fd4e00a2b80 20 client.26936 trim_cache size 0 max 16384 2024-05-08T21:24:28.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.994+0000 7fd4e00a2b80 1 Processor -- start 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.994+0000 7fd4e00a2b80 1 -- start start 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.994+0000 7fd4e00a2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a2540 0x5604012a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.994+0000 7fd4e00a2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2ea0 0x5604012b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.994+0000 7fd4e00a2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a3860 0x5604012a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.994+0000 7fd4e00a2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604012a41e0 con 0x5604012a2540 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.994+0000 7fd4e00a2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604012ab9f0 con 0x5604012a3860 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.994+0000 7fd4e00a2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604012ad070 con 0x5604012a2ea0 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.994+0000 7fd4d1c84700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a2540 0x5604012a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4d1c84700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a2540 0x5604012a3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55704/0 (socket says 172.21.15.33:55704) 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4d1483700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2ea0 0x5604012b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4d1c84700 1 -- 172.21.15.33:0/3400221774 learned_addr learned my addr 172.21.15.33:0/3400221774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4d2485700 1 --2- 172.21.15.33:0/3400221774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a3860 0x5604012a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:28.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4d1483700 1 -- 172.21.15.33:0/3400221774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a3860 msgr2=0x5604012a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4d1483700 1 --2- 172.21.15.33:0/3400221774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a3860 0x5604012a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4d1483700 1 -- 172.21.15.33:0/3400221774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a2540 msgr2=0x5604012a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4d1483700 1 --2- 172.21.15.33:0/3400221774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a2540 0x5604012a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4d1483700 1 -- 172.21.15.33:0/3400221774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604011b8560 con 0x5604012a2ea0 2024-05-08T21:24:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4d1483700 1 --2- 172.21.15.33:0/3400221774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2ea0 0x5604012b33c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00bf10 tx=0x7fd4bc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa041abfbee27570 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4c0ff9700 1 -- 172.21.15.33:0/3400221774 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc00bd20 con 0x5604012a2ea0 2024-05-08T21:24:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4c0ff9700 1 -- 172.21.15.33:0/3400221774 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4bc01e040 con 0x5604012a2ea0 2024-05-08T21:24:28.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.995+0000 7fd4c0ff9700 1 -- 172.21.15.33:0/3400221774 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc003760 con 0x5604012a2ea0 2024-05-08T21:24:29.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.996+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/3400221774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2ea0 msgr2=0x5604012b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.996+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/3400221774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2ea0 0x5604012b33c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00bf10 tx=0x7fd4bc009580 comp rx=0 tx=0).stop 2024-05-08T21:24:29.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.996+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/3400221774 shutdown_connections 2024-05-08T21:24:29.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.996+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/3400221774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a3860 0x5604012a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.996+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/3400221774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a2540 0x5604012a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.996+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/3400221774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2ea0 0x5604012b33c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.996+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/3400221774 >> 172.21.15.33:0/3400221774 conn(0x560401200210 msgr2=0x560401299210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:29.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.996+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/3400221774 shutdown_connections 2024-05-08T21:24:29.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.996+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/3400221774 wait complete. 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.997+0000 7fd4e00a2b80 1 -- start start 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.997+0000 7fd4e00a2b80 1 Processor -- start 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.997+0000 7fd4e00a2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.997+0000 7fd4e00a2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a2ea0 0x5604012b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.997+0000 7fd4e00a2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a3860 0x5604012b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.997+0000 7fd4e00a2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604012b2320 con 0x5604012a2ea0 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.997+0000 7fd4e00a2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604012b2490 con 0x5604012a3860 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.997+0000 7fd4e00a2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604012b2650 con 0x5604012a2540 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4e00a2b80 10 client.0 _ll_register_callbacks cb 0x560401197b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4d1c84700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4d1483700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a2ea0 0x5604012b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4d1c84700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:34108/0 (socket says 172.21.15.33:34108) 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4d1c84700 1 -- 172.21.15.33:0/119005313 learned_addr learned my addr 172.21.15.33:0/119005313 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:29.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4d2485700 1 --2- 172.21.15.33:0/119005313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a3860 0x5604012b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4d1c84700 1 -- 172.21.15.33:0/119005313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a3860 msgr2=0x5604012b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4d1c84700 1 --2- 172.21.15.33:0/119005313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a3860 0x5604012b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4d1c84700 1 -- 172.21.15.33:0/119005313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a2ea0 msgr2=0x5604012b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4d1c84700 1 --2- 172.21.15.33:0/119005313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a2ea0 0x5604012b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4d1c84700 1 -- 172.21.15.33:0/119005313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56040101fb60 con 0x5604012a2540 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4d1c84700 1 --2- 172.21.15.33:0/119005313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012b08e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009c30 tx=0x7fd4c80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec9046ae35ebb38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4b57fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4b57fa700 1 -- 172.21.15.33:0/119005313 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8032050 con 0x5604012a2540 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/119005313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4c4009e20 con 0x5604012a2540 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.998+0000 7fd4b57fa700 1 -- 172.21.15.33:0/119005313 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd4c802daa0 con 0x5604012a2540 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.999+0000 7fd4b57fa700 1 -- 172.21.15.33:0/119005313 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4c802dc60 con 0x5604012a2540 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.999+0000 7fd4e00a2b80 10 client.26956 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.999+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/119005313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x560401077cd0 con 0x5604012a2540 2024-05-08T21:24:29.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.999+0000 7fd4b57fa700 1 -- 172.21.15.33:0/119005313 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4c4009e20 con 0x5604012a2540 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.999+0000 7fd4b57fa700 1 -- 172.21.15.33:0/119005313 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fd4c802bd10 con 0x5604012a2540 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.999+0000 7fd4e00a2b80 10 client.26956 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.999+0000 7fd4e00a2b80 10 client.26956 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.999+0000 7fd4e00a2b80 20 client.26956 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.999+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/119005313 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x56040130e9a0 0x560401310dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.999+0000 7fd4e00a2b80 4 client.26956 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.999+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/119005313 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5604011eaec0 con 0x56040130e9a0 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:28.999+0000 7fd4d1483700 1 --2- 172.21.15.33:0/119005313 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x56040130e9a0 0x560401310dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4b57fa700 10 client.26956 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4d1483700 1 --2- 172.21.15.33:0/119005313 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x56040130e9a0 0x560401310dc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9ce64595306a0bf server_cookie=8d79612d01d27aab in_seq=0 out_seq=0 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4b57fa700 1 -- 172.21.15.33:0/119005313 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5604011eaec0 con 0x56040130e9a0 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4b57fa700 10 client.26956 handle_command_reply: tid=0 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4e00a2b80 1 client.26956 shutdown 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/119005313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 msgr2=0x5604012b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/119005313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012b08e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009c30 tx=0x7fd4c80047c0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/119005313 shutdown_connections 2024-05-08T21:24:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/119005313 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x56040130e9a0 0x560401310dc0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/119005313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604012a3860 0x5604012b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/119005313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5604012a2540 0x5604012b08e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4e00a2b80 1 --2- 172.21.15.33:0/119005313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5604012a2ea0 0x5604012b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4b57fa700 0 client.26956 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:24:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/119005313 >> 172.21.15.33:0/119005313 conn(0x560401200210 msgr2=0x5604012a3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/119005313 shutdown_connections 2024-05-08T21:24:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.000+0000 7fd4e00a2b80 1 -- 172.21.15.33:0/119005313 wait complete. 2024-05-08T21:24:29.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4e00a2b80 20 client.26956 trim_cache size 0 max 16384 2024-05-08T21:24:29.007 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:24:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/1398727461 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4b80a11a0 msgr2=0x7fd4b80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4d3ee8700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4b80a11a0 0x7fd4b80a3660 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8005950 tx=0x7fd4c8009260 comp rx=0 tx=0).stop 2024-05-08T21:24:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/1398727461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4cc1018f0 msgr2=0x7fd4cc1a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4d3ee8700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4cc1018f0 0x7fd4cc1a0450 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4007f10 tx=0x7fd4c400d3b0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/1398727461 shutdown_connections 2024-05-08T21:24:29.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4d3ee8700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd4b80a11a0 0x7fd4b80a3660 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4d3ee8700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4cc1018f0 0x7fd4cc1a0450 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4d3ee8700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4cc0ffd60 0x7fd4cc198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4d3ee8700 1 --2- 172.21.15.33:0/1398727461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4cc100f50 0x7fd4cc199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/1398727461 >> 172.21.15.33:0/1398727461 conn(0x7fd4cc0fb290 msgr2=0x7fd4cc0fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/1398727461 shutdown_connections 2024-05-08T21:24:29.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.001+0000 7fd4d3ee8700 1 -- 172.21.15.33:0/1398727461 wait complete. 2024-05-08T21:24:29.019 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:24:29.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.188+0000 7fb5a1d57700 1 -- 172.21.15.33:0/1928540198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb59c100f60 msgr2=0x7fb59c1013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.188+0000 7fb5a1d57700 1 --2- 172.21.15.33:0/1928540198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb59c100f60 0x7fb59c1013c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb5900098f0 tx=0x7fb590009c00 comp rx=0 tx=0).stop 2024-05-08T21:24:29.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.188+0000 7fb5a1d57700 1 -- 172.21.15.33:0/1928540198 shutdown_connections 2024-05-08T21:24:29.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.188+0000 7fb5a1d57700 1 --2- 172.21.15.33:0/1928540198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb59c101900 0x7fb59c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.188+0000 7fb5a1d57700 1 --2- 172.21.15.33:0/1928540198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb59c0ffd70 0x7fb59c100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.188+0000 7fb5a1d57700 1 --2- 172.21.15.33:0/1928540198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb59c100f60 0x7fb59c1013c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.188+0000 7fb5a1d57700 1 -- 172.21.15.33:0/1928540198 >> 172.21.15.33:0/1928540198 conn(0x7fb59c0fb300 msgr2=0x7fb59c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:29.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.188+0000 7fb5a1d57700 1 -- 172.21.15.33:0/1928540198 shutdown_connections 2024-05-08T21:24:29.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.188+0000 7fb5a1d57700 1 -- 172.21.15.33:0/1928540198 wait complete. 2024-05-08T21:24:29.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.189+0000 7fb5a1d57700 1 Processor -- start 2024-05-08T21:24:29.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.189+0000 7fb5a1d57700 1 -- start start 2024-05-08T21:24:29.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb5a1d57700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb59c0ffd70 0x7fb59c069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb5a1d57700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb59c100f60 0x7fb59c19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb5a1d57700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb59c101900 0x7fb59c19e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb5a1d57700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c19e840 con 0x7fb59c100f60 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb5a1d57700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c19e9b0 con 0x7fb59c0ffd70 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb5a1d57700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c19eb70 con 0x7fb59c101900 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb59b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb59c0ffd70 0x7fb59c069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb59b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb59c0ffd70 0x7fb59c069ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:48862/0 (socket says 172.21.15.33:48862) 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb59b7fe700 1 -- 172.21.15.33:0/353588619 learned_addr learned my addr 172.21.15.33:0/353588619 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb59affd700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb59c100f60 0x7fb59c19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb59bfff700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb59c101900 0x7fb59c19e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb59b7fe700 1 -- 172.21.15.33:0/353588619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb59c101900 msgr2=0x7fb59c19e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb59b7fe700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb59c101900 0x7fb59c19e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb59b7fe700 1 -- 172.21.15.33:0/353588619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb59c100f60 msgr2=0x7fb59c19bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb59b7fe700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb59c100f60 0x7fb59c19bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb59b7fe700 1 -- 172.21.15.33:0/353588619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c009520 con 0x7fb59c0ffd70 2024-05-08T21:24:29.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.190+0000 7fb59b7fe700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb59c0ffd70 0x7fb59c069ff0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00ea50 tx=0x7fb58c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.191+0000 7fb598ff9700 1 -- 172.21.15.33:0/353588619 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c00cbd0 con 0x7fb59c0ffd70 2024-05-08T21:24:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.191+0000 7fb598ff9700 1 -- 172.21.15.33:0/353588619 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb58c00cd30 con 0x7fb59c0ffd70 2024-05-08T21:24:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.191+0000 7fb5a1d57700 1 -- 172.21.15.33:0/353588619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb594009d00 con 0x7fb59c0ffd70 2024-05-08T21:24:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.191+0000 7fb598ff9700 1 -- 172.21.15.33:0/353588619 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c018680 con 0x7fb59c0ffd70 2024-05-08T21:24:29.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.191+0000 7fb5a1d57700 1 -- 172.21.15.33:0/353588619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb590009520 con 0x7fb59c0ffd70 2024-05-08T21:24:29.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.193+0000 7fb598ff9700 1 -- 172.21.15.33:0/353588619 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fb58c010430 con 0x7fb59c0ffd70 2024-05-08T21:24:29.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.194+0000 7fb598ff9700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5840a11a0 0x7fb5840a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.194+0000 7fb598ff9700 1 -- 172.21.15.33:0/353588619 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fb58c014070 con 0x7fb59c0ffd70 2024-05-08T21:24:29.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.194+0000 7fb59affd700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5840a11a0 0x7fb5840a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.195+0000 7fb59affd700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5840a11a0 0x7fb5840a3660 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb59000fa00 tx=0x7fb59001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:29.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb5adf11b80 1 Processor -- start 2024-05-08T21:24:29.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb5adf11b80 1 -- start start 2024-05-08T21:24:29.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb5adf11b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17cf2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb5adf11b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf2ea0 0x555a17cf3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb5adf11b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 0x555a17cf3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb5adf11b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a17c49150 con 0x555a17cf2540 2024-05-08T21:24:29.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb5adf11b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a17c46b10 con 0x555a17cf2ea0 2024-05-08T21:24:29.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb5adf11b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a17c42400 con 0x555a17cf3860 2024-05-08T21:24:29.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb59b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17cf2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb59b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17cf2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55744/0 (socket says 172.21.15.33:55744) 2024-05-08T21:24:29.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb59b7fe700 1 -- 172.21.15.33:0/402599898 learned_addr learned my addr 172.21.15.33:0/402599898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb59affd700 1 --2- 172.21.15.33:0/402599898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf2ea0 0x555a17cf3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.196+0000 7fb59bfff700 1 --2- 172.21.15.33:0/402599898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 0x555a17cf3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.197+0000 7fb59bfff700 1 -- 172.21.15.33:0/402599898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf2ea0 msgr2=0x555a17cf3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.197+0000 7fb59bfff700 1 --2- 172.21.15.33:0/402599898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf2ea0 0x555a17cf3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.197+0000 7fb59bfff700 1 -- 172.21.15.33:0/402599898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 msgr2=0x555a17cf2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.197+0000 7fb59bfff700 1 --2- 172.21.15.33:0/402599898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17cf2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.197+0000 7fb59bfff700 1 -- 172.21.15.33:0/402599898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a17ac33b0 con 0x555a17cf3860 2024-05-08T21:24:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.197+0000 7fb59affd700 1 --2- 172.21.15.33:0/402599898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf2ea0 0x555a17cf3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:24:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.197+0000 7fb59bfff700 1 --2- 172.21.15.33:0/402599898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 0x555a17cf3ce0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb594009480 tx=0x7fb5940099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd83880656ac8f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.197+0000 7fb581ffb700 1 -- 172.21.15.33:0/402599898 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594019070 con 0x555a17cf3860 2024-05-08T21:24:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.197+0000 7fb581ffb700 1 -- 172.21.15.33:0/402599898 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb59400f040 con 0x555a17cf3860 2024-05-08T21:24:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.197+0000 7fb581ffb700 1 -- 172.21.15.33:0/402599898 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59400e030 con 0x555a17cf3860 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.198+0000 7fb5adf11b80 1 -- 172.21.15.33:0/402599898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 msgr2=0x555a17cf3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.198+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/402599898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 0x555a17cf3ce0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb594009480 tx=0x7fb5940099e0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.198+0000 7fb5adf11b80 1 -- 172.21.15.33:0/402599898 shutdown_connections 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.198+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/402599898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 0x555a17cf3ce0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.198+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/402599898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17cf2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.198+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/402599898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf2ea0 0x555a17cf3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.198+0000 7fb5adf11b80 1 -- 172.21.15.33:0/402599898 >> 172.21.15.33:0/402599898 conn(0x555a17c50210 msgr2=0x555a17cecde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.198+0000 7fb5adf11b80 1 -- 172.21.15.33:0/402599898 shutdown_connections 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.198+0000 7fb5adf11b80 1 -- 172.21.15.33:0/402599898 wait complete. 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.198+0000 7fb5adf11b80 1 -- start start 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.199+0000 7fb5adf11b80 1 Processor -- start 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.199+0000 7fb5adf11b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf2540 0x555a17cff280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.199+0000 7fb5adf11b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2ea0 0x555a17cff7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.199+0000 7fb5adf11b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 0x555a17cfbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.199+0000 7fb5adf11b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a17bde560 con 0x555a17cf2ea0 2024-05-08T21:24:29.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.199+0000 7fb5adf11b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a17bde200 con 0x555a17cf3860 2024-05-08T21:24:29.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.199+0000 7fb5adf11b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a17ac56a0 con 0x555a17cf2540 2024-05-08T21:24:29.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.199+0000 7fb5adf11b80 10 client.0 _ll_register_callbacks cb 0x555a17be7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:29.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.199+0000 7fb59bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 0x555a17cfbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.200+0000 7fb59bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 0x555a17cfbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:48892/0 (socket says 172.21.15.33:48892) 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.200+0000 7fb59bfff700 1 -- 172.21.15.33:0/2288150289 learned_addr learned my addr 172.21.15.33:0/2288150289 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.200+0000 7fb59bfff700 1 -- 172.21.15.33:0/2288150289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf2540 msgr2=0x555a17cff280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.200+0000 7fb59bfff700 1 --2- 172.21.15.33:0/2288150289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf2540 0x555a17cff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.200+0000 7fb59bfff700 1 -- 172.21.15.33:0/2288150289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2ea0 msgr2=0x555a17cff7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.200+0000 7fb59bfff700 1 --2- 172.21.15.33:0/2288150289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2ea0 0x555a17cff7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.200+0000 7fb59bfff700 1 -- 172.21.15.33:0/2288150289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb594009160 con 0x555a17cf3860 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.200+0000 7fb59bfff700 1 --2- 172.21.15.33:0/2288150289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 0x555a17cfbe10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb594009610 tx=0x7fb594014990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27b6c5d60212e14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.200+0000 7fb57affd700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.200+0000 7fb57affd700 1 -- 172.21.15.33:0/2288150289 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59400e070 con 0x555a17cf3860 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.201+0000 7fb57affd700 1 -- 172.21.15.33:0/2288150289 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fb594015890 con 0x555a17cf3860 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.201+0000 7fb5adf11b80 1 -- 172.21.15.33:0/2288150289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a17b9b270 con 0x555a17cf3860 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.201+0000 7fb57affd700 1 -- 172.21.15.33:0/2288150289 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb594019040 con 0x555a17cf3860 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.201+0000 7fb5adf11b80 10 client.26954 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.201+0000 7fb5adf11b80 1 -- 172.21.15.33:0/2288150289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555a17a6fb60 con 0x555a17cf3860 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.201+0000 7fb57affd700 1 -- 172.21.15.33:0/2288150289 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555a17b9b270 con 0x555a17cf3860 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.202+0000 7fb57affd700 1 -- 172.21.15.33:0/2288150289 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fb594024aa0 con 0x555a17cf3860 2024-05-08T21:24:29.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.202+0000 7fb5adf11b80 10 client.26954 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:29.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.202+0000 7fb5adf11b80 10 client.26954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:24:29.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.202+0000 7fb5adf11b80 20 client.26954 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:29.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.202+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/2288150289 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555a17d10430 0x555a17d12870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.202+0000 7fb5adf11b80 4 client.26954 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:24:29.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.202+0000 7fb5adf11b80 1 -- 172.21.15.33:0/2288150289 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a17cc1810 con 0x555a17d10430 2024-05-08T21:24:29.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.202+0000 7fb59b7fe700 1 --2- 172.21.15.33:0/2288150289 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555a17d10430 0x555a17d12870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.203+0000 7fb59b7fe700 1 --2- 172.21.15.33:0/2288150289 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555a17d10430 0x555a17d12870 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac724f52d7a52512 server_cookie=363c3852fe3b7bec in_seq=0 out_seq=0 2024-05-08T21:24:29.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.203+0000 7fb57affd700 10 client.26954 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:29.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb57affd700 1 -- 172.21.15.33:0/2288150289 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a17cc1810 con 0x555a17d10430 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb57affd700 10 client.26954 handle_command_reply: tid=0 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb5adf11b80 1 client.26954 shutdown 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb5adf11b80 1 -- 172.21.15.33:0/2288150289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 msgr2=0x555a17cfbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/2288150289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 0x555a17cfbe10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb594009610 tx=0x7fb594014990 comp rx=0 tx=0).stop 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb5adf11b80 1 -- 172.21.15.33:0/2288150289 shutdown_connections 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/2288150289 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555a17d10430 0x555a17d12870 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/2288150289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 0x555a17cfbe10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/2288150289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf2540 0x555a17cff280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/2288150289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2ea0 0x555a17cff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb57affd700 0 client.26954 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb5adf11b80 1 -- 172.21.15.33:0/2288150289 >> 172.21.15.33:0/2288150289 conn(0x555a17c50210 msgr2=0x555a17cec5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb5adf11b80 1 -- 172.21.15.33:0/2288150289 shutdown_connections 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb5adf11b80 1 -- 172.21.15.33:0/2288150289 wait complete. 2024-05-08T21:24:29.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.204+0000 7fb5adf11b80 20 client.26954 trim_cache size 0 max 16384 2024-05-08T21:24:29.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.221+0000 7fb5adf11b80 1 Processor -- start 2024-05-08T21:24:29.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.221+0000 7fb5adf11b80 1 -- start start 2024-05-08T21:24:29.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.221+0000 7fb5adf11b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17cfbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.221+0000 7fb5adf11b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf2ea0 0x555a17cfb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.221+0000 7fb5adf11b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 0x555a17cfaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.221+0000 7fb5adf11b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a17cee570 con 0x555a17cf2540 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.221+0000 7fb5adf11b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a17d0c610 con 0x555a17cf2ea0 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.221+0000 7fb5adf11b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a17ce7800 con 0x555a17cf3860 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb59b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17cfbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb59bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 0x555a17cfaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb59b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17cfbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55770/0 (socket says 172.21.15.33:55770) 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb59affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf2ea0 0x555a17cfb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb59bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 0x555a17cfaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:34152/0 (socket says 172.21.15.33:34152) 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb59b7fe700 1 -- 172.21.15.33:0/35269333 learned_addr learned my addr 172.21.15.33:0/35269333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb59bfff700 1 -- 172.21.15.33:0/35269333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf2ea0 msgr2=0x555a17cfb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb59bfff700 1 --2- 172.21.15.33:0/35269333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf2ea0 0x555a17cfb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb59bfff700 1 -- 172.21.15.33:0/35269333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 msgr2=0x555a17cfbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb59bfff700 1 --2- 172.21.15.33:0/35269333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17cfbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb59bfff700 1 -- 172.21.15.33:0/35269333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a17c08560 con 0x555a17cf3860 2024-05-08T21:24:29.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb59bfff700 1 --2- 172.21.15.33:0/35269333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 0x555a17cfaae0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb594000f80 tx=0x7fb594005060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5188fb142a152a03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:29.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb5827fc700 1 -- 172.21.15.33:0/35269333 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59400e070 con 0x555a17cf3860 2024-05-08T21:24:29.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb5827fc700 1 -- 172.21.15.33:0/35269333 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb594003e90 con 0x555a17cf3860 2024-05-08T21:24:29.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.222+0000 7fb5827fc700 1 -- 172.21.15.33:0/35269333 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594005820 con 0x555a17cf3860 2024-05-08T21:24:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.223+0000 7fb5adf11b80 1 -- 172.21.15.33:0/35269333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 msgr2=0x555a17cfaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.223+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/35269333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 0x555a17cfaae0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb594000f80 tx=0x7fb594005060 comp rx=0 tx=0).stop 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.223+0000 7fb5adf11b80 1 -- 172.21.15.33:0/35269333 shutdown_connections 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.223+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/35269333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf3860 0x555a17cfaae0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.223+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/35269333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17cfbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.223+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/35269333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf2ea0 0x555a17cfb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.224+0000 7fb5adf11b80 1 -- 172.21.15.33:0/35269333 >> 172.21.15.33:0/35269333 conn(0x555a17c50210 msgr2=0x555a17cf3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.224+0000 7fb5adf11b80 1 -- 172.21.15.33:0/35269333 shutdown_connections 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.224+0000 7fb5adf11b80 1 -- 172.21.15.33:0/35269333 wait complete. 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.224+0000 7fb5adf11b80 1 -- start start 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.224+0000 7fb5adf11b80 1 Processor -- start 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb5adf11b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17d1d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb5adf11b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf2ea0 0x555a17d1ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb5adf11b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 0x555a17d653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb5adf11b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a17d20d60 con 0x555a17cf2540 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb5adf11b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a17d20ed0 con 0x555a17cf3860 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb5adf11b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a17d65910 con 0x555a17cf2ea0 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb5adf11b80 10 client.0 _ll_register_callbacks cb 0x555a17be7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb59bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 0x555a17d653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb59bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 0x555a17d653d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:48906/0 (socket says 172.21.15.33:48906) 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb59bfff700 1 -- 172.21.15.33:0/1956736489 learned_addr learned my addr 172.21.15.33:0/1956736489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb59bfff700 1 -- 172.21.15.33:0/1956736489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf2ea0 msgr2=0x555a17d1ded0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb59bfff700 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf2ea0 0x555a17d1ded0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb59bfff700 1 -- 172.21.15.33:0/1956736489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 msgr2=0x555a17d1d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.225+0000 7fb59b7fe700 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17d1d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.226+0000 7fb59bfff700 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17d1d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.226+0000 7fb59bfff700 1 -- 172.21.15.33:0/1956736489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a17a6fb60 con 0x555a17cf3860 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.226+0000 7fb59b7fe700 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17d1d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.226+0000 7fb59bfff700 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 0x555a17d653d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb5940149d0 tx=0x7fb594003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b53339e0176f21ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.226+0000 7fb57affd700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:24:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.226+0000 7fb57affd700 1 -- 172.21.15.33:0/1956736489 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594019030 con 0x555a17cf3860 2024-05-08T21:24:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.226+0000 7fb57affd700 1 -- 172.21.15.33:0/1956736489 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fb594005560 con 0x555a17cf3860 2024-05-08T21:24:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.226+0000 7fb5adf11b80 1 -- 172.21.15.33:0/1956736489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb59400f040 con 0x555a17cf3860 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.226+0000 7fb57affd700 1 -- 172.21.15.33:0/1956736489 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb59400e040 con 0x555a17cf3860 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.226+0000 7fb5adf11b80 10 client.26960 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.226+0000 7fb5adf11b80 1 -- 172.21.15.33:0/1956736489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555a17ac7cd0 con 0x555a17cf3860 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.226+0000 7fb57affd700 1 -- 172.21.15.33:0/1956736489 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb59400f040 con 0x555a17cf3860 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.227+0000 7fb57affd700 1 -- 172.21.15.33:0/1956736489 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fb59402c070 con 0x555a17cf3860 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.227+0000 7fb5adf11b80 10 client.26960 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.227+0000 7fb5adf11b80 10 client.26960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.227+0000 7fb5adf11b80 20 client.26960 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.227+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555a17d66d60 0x555a17d69180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.227+0000 7fb5adf11b80 4 client.26960 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.227+0000 7fb5adf11b80 1 -- 172.21.15.33:0/1956736489 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555a17c3aec0 con 0x555a17d66d60 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.227+0000 7fb59b7fe700 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555a17d66d60 0x555a17d69180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.227+0000 7fb57affd700 10 client.26960 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.228+0000 7fb59b7fe700 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555a17d66d60 0x555a17d69180 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6aa4f6833e0da64 server_cookie=5beb2cc5e9e5a81e in_seq=0 out_seq=0 2024-05-08T21:24:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.228+0000 7fb57affd700 1 -- 172.21.15.33:0/1956736489 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555a17c3aec0 con 0x555a17d66d60 2024-05-08T21:24:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.228+0000 7fb57affd700 10 client.26960 handle_command_reply: tid=0 2024-05-08T21:24:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.228+0000 7fb5adf11b80 1 client.26960 shutdown 2024-05-08T21:24:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.228+0000 7fb5adf11b80 1 -- 172.21.15.33:0/1956736489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 msgr2=0x555a17d653d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.228+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 0x555a17d653d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb5940149d0 tx=0x7fb594003c30 comp rx=0 tx=0).stop 2024-05-08T21:24:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.229+0000 7fb5adf11b80 1 -- 172.21.15.33:0/1956736489 shutdown_connections 2024-05-08T21:24:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.229+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x555a17d66d60 0x555a17d69180 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.229+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555a17cf3860 0x555a17d653d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.229+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a17cf2540 0x555a17d1d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.229+0000 7fb5adf11b80 1 --2- 172.21.15.33:0/1956736489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a17cf2ea0 0x555a17d1ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.229+0000 7fb5adf11b80 1 -- 172.21.15.33:0/1956736489 >> 172.21.15.33:0/1956736489 conn(0x555a17c50210 msgr2=0x555a17d171d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.229+0000 7fb57affd700 0 client.26960 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.229+0000 7fb5adf11b80 1 -- 172.21.15.33:0/1956736489 shutdown_connections 2024-05-08T21:24:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.229+0000 7fb5adf11b80 1 -- 172.21.15.33:0/1956736489 wait complete. 2024-05-08T21:24:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.230+0000 7fb5adf11b80 20 client.26960 trim_cache size 0 max 16384 2024-05-08T21:24:29.235 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:24:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.230+0000 7fb5a1d57700 1 -- 172.21.15.33:0/353588619 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5840a11a0 msgr2=0x7fb5840a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.230+0000 7fb5a1d57700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5840a11a0 0x7fb5840a3660 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb59000fa00 tx=0x7fb59001a040 comp rx=0 tx=0).stop 2024-05-08T21:24:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.230+0000 7fb5a1d57700 1 -- 172.21.15.33:0/353588619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb59c0ffd70 msgr2=0x7fb59c069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.230+0000 7fb5a1d57700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb59c0ffd70 0x7fb59c069ff0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00ea50 tx=0x7fb58c00ed60 comp rx=0 tx=0).stop 2024-05-08T21:24:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.230+0000 7fb5a1d57700 1 -- 172.21.15.33:0/353588619 shutdown_connections 2024-05-08T21:24:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.230+0000 7fb5a1d57700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb59c101900 0x7fb59c19e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.230+0000 7fb5a1d57700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fb5840a11a0 0x7fb5840a3660 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.230+0000 7fb5a1d57700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb59c0ffd70 0x7fb59c069ff0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.230+0000 7fb5a1d57700 1 --2- 172.21.15.33:0/353588619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb59c100f60 0x7fb59c19bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.230+0000 7fb5a1d57700 1 -- 172.21.15.33:0/353588619 >> 172.21.15.33:0/353588619 conn(0x7fb59c0fb300 msgr2=0x7fb59c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.231+0000 7fb5a1d57700 1 -- 172.21.15.33:0/353588619 shutdown_connections 2024-05-08T21:24:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:29.231+0000 7fb5a1d57700 1 -- 172.21.15.33:0/353588619 wait complete. 2024-05-08T21:24:29.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:29 smithi071 ceph-mon[47074]: pgmap v455: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.0 MiB/s wr, 412 op/s 2024-05-08T21:24:29.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:29 smithi082 ceph-mon[47512]: pgmap v455: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.0 MiB/s wr, 412 op/s 2024-05-08T21:24:29.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:29 smithi033 ceph-mon[42528]: pgmap v455: 129 pgs: 129 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.0 MiB/s wr, 412 op/s 2024-05-08T21:24:30.249 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:24:30.250 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:24:30.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.425+0000 7f7191423700 1 -- 172.21.15.33:0/2688912952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f718c100e70 msgr2=0x7f718c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.425+0000 7f7191423700 1 --2- 172.21.15.33:0/2688912952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f718c100e70 0x7f718c101290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f717c0098f0 tx=0x7f717c009c00 comp rx=0 tx=0).stop 2024-05-08T21:24:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.425+0000 7f7191423700 1 -- 172.21.15.33:0/2688912952 shutdown_connections 2024-05-08T21:24:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.425+0000 7f7191423700 1 --2- 172.21.15.33:0/2688912952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c102980 0x7f718c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.425+0000 7f7191423700 1 --2- 172.21.15.33:0/2688912952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f718c100e70 0x7f718c101290 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.425+0000 7f7191423700 1 --2- 172.21.15.33:0/2688912952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c101fc0 0x7f718c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.425+0000 7f7191423700 1 -- 172.21.15.33:0/2688912952 >> 172.21.15.33:0/2688912952 conn(0x7f718c0fc400 msgr2=0x7f718c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.425+0000 7f7191423700 1 -- 172.21.15.33:0/2688912952 shutdown_connections 2024-05-08T21:24:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.425+0000 7f7191423700 1 -- 172.21.15.33:0/2688912952 wait complete. 2024-05-08T21:24:30.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.426+0000 7f7191423700 1 Processor -- start 2024-05-08T21:24:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.426+0000 7f7191423700 1 -- start start 2024-05-08T21:24:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f7191423700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c100e70 0x7f718c196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f7191423700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c101fc0 0x7f718c197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f7191423700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f718c102980 0x7f718c19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f7191423700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c19e820 con 0x7f718c100e70 2024-05-08T21:24:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f7191423700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c19e990 con 0x7f718c102980 2024-05-08T21:24:30.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f7191423700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c19eb50 con 0x7f718c101fc0 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f718affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c100e70 0x7f718c196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f718a7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c101fc0 0x7f718c197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f718a7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c101fc0 0x7f718c197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:35010/0 (socket says 172.21.15.33:35010) 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f718affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c100e70 0x7f718c196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53188/0 (socket says 172.21.15.33:53188) 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f718a7fc700 1 -- 172.21.15.33:0/2397188933 learned_addr learned my addr 172.21.15.33:0/2397188933 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f718b7fe700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f718c102980 0x7f718c19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f718a7fc700 1 -- 172.21.15.33:0/2397188933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f718c102980 msgr2=0x7f718c19e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f718a7fc700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f718c102980 0x7f718c19e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f718a7fc700 1 -- 172.21.15.33:0/2397188933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c100e70 msgr2=0x7f718c196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f718a7fc700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c100e70 0x7f718c196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f718a7fc700 1 -- 172.21.15.33:0/2397188933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c009520 con 0x7f718c101fc0 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.427+0000 7f718a7fc700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c101fc0 0x7f718c197240 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f718000e9c0 tx=0x7f718000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.428+0000 7f7173fff700 1 -- 172.21.15.33:0/2397188933 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7180004a30 con 0x7f718c101fc0 2024-05-08T21:24:30.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.428+0000 7f7173fff700 1 -- 172.21.15.33:0/2397188933 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f718000cc50 con 0x7f718c101fc0 2024-05-08T21:24:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.428+0000 7f7191423700 1 -- 172.21.15.33:0/2397188933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718c19ee30 con 0x7f718c101fc0 2024-05-08T21:24:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.428+0000 7f7173fff700 1 -- 172.21.15.33:0/2397188933 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7180005410 con 0x7f718c101fc0 2024-05-08T21:24:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.428+0000 7f7191423700 1 -- 172.21.15.33:0/2397188933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718c19f2b0 con 0x7f718c101fc0 2024-05-08T21:24:30.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.430+0000 7f7173fff700 1 -- 172.21.15.33:0/2397188933 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7180005570 con 0x7f718c101fc0 2024-05-08T21:24:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.431+0000 7f7173fff700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71740a11a0 0x7f71740a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.431+0000 7f7173fff700 1 -- 172.21.15.33:0/2397188933 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7180014070 con 0x7f718c101fc0 2024-05-08T21:24:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.431+0000 7f718affd700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71740a11a0 0x7f71740a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.432+0000 7f718affd700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71740a11a0 0x7f71740a3660 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f717c005e90 tx=0x7f717c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.432+0000 7f719d5ddb80 1 Processor -- start 2024-05-08T21:24:30.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.432+0000 7f719d5ddb80 1 -- start start 2024-05-08T21:24:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.432+0000 7f719d5ddb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a09d0 0x5651ed9a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.432+0000 7f719d5ddb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1330 0x5651ed9a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.432+0000 7f719d5ddb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1cf0 0x5651ed9a2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.432+0000 7f719d5ddb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ed8f31d0 con 0x5651ed9a1330 2024-05-08T21:24:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.432+0000 7f719d5ddb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ed8f0b90 con 0x5651ed9a1cf0 2024-05-08T21:24:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.432+0000 7f719d5ddb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ed8ec480 con 0x5651ed9a09d0 2024-05-08T21:24:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.432+0000 7f718affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a09d0 0x5651ed9a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.432+0000 7f718affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a09d0 0x5651ed9a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:35014/0 (socket says 172.21.15.33:35014) 2024-05-08T21:24:30.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.432+0000 7f718affd700 1 -- 172.21.15.33:0/3970012768 learned_addr learned my addr 172.21.15.33:0/3970012768 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.433+0000 7f718b7fe700 1 --2- 172.21.15.33:0/3970012768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1cf0 0x5651ed9a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.433+0000 7f718b7fe700 1 -- 172.21.15.33:0/3970012768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a09d0 msgr2=0x5651ed9a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.433+0000 7f718b7fe700 1 --2- 172.21.15.33:0/3970012768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a09d0 0x5651ed9a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.433+0000 7f718b7fe700 1 -- 172.21.15.33:0/3970012768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1330 msgr2=0x5651ed9a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.433+0000 7f718b7fe700 1 --2- 172.21.15.33:0/3970012768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1330 0x5651ed9a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.433+0000 7f718b7fe700 1 -- 172.21.15.33:0/3970012768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651ed76d430 con 0x5651ed9a1cf0 2024-05-08T21:24:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.433+0000 7f718affd700 1 --2- 172.21.15.33:0/3970012768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a09d0 0x5651ed9a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.433+0000 7f718b7fe700 1 --2- 172.21.15.33:0/3970012768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1cf0 0x5651ed9a2170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f71840115b0 tx=0x7f7184009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dc57a5528b08400 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.433+0000 7f71717fa700 1 -- 172.21.15.33:0/3970012768 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718400e040 con 0x5651ed9a1cf0 2024-05-08T21:24:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.433+0000 7f71717fa700 1 -- 172.21.15.33:0/3970012768 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f718400f040 con 0x5651ed9a1cf0 2024-05-08T21:24:30.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.433+0000 7f71717fa700 1 -- 172.21.15.33:0/3970012768 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184004370 con 0x5651ed9a1cf0 2024-05-08T21:24:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.434+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3970012768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1cf0 msgr2=0x5651ed9a2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.434+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3970012768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1cf0 0x5651ed9a2170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f71840115b0 tx=0x7f7184009920 comp rx=0 tx=0).stop 2024-05-08T21:24:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.434+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3970012768 shutdown_connections 2024-05-08T21:24:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.434+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3970012768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1cf0 0x5651ed9a2170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.434+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3970012768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a09d0 0x5651ed9a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.434+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3970012768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1330 0x5651ed9a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.434+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3970012768 >> 172.21.15.33:0/3970012768 conn(0x5651ed8fa290 msgr2=0x5651ed996e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.434+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3970012768 shutdown_connections 2024-05-08T21:24:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.434+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3970012768 wait complete. 2024-05-08T21:24:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f719d5ddb80 1 -- start start 2024-05-08T21:24:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f719d5ddb80 1 Processor -- start 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f719d5ddb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a09d0 0x5651ed9aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f719d5ddb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1330 0x5651ed9aa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f719d5ddb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a1cf0 0x5651ed9b1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f719d5ddb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ed8885e0 con 0x5651ed9a09d0 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f719d5ddb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ed888280 con 0x5651ed9a1330 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f719d5ddb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ed76f720 con 0x5651ed9a1cf0 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f719d5ddb80 10 client.0 _ll_register_callbacks cb 0x5651ed891bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f718affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a09d0 0x5651ed9aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f718affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a09d0 0x5651ed9aa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53198/0 (socket says 172.21.15.33:53198) 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f718affd700 1 -- 172.21.15.33:0/2945010024 learned_addr learned my addr 172.21.15.33:0/2945010024 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f718b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a1cf0 0x5651ed9b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.435+0000 7f718a7fc700 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1330 0x5651ed9aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f718affd700 1 -- 172.21.15.33:0/2945010024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a1cf0 msgr2=0x5651ed9b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f718affd700 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a1cf0 0x5651ed9b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f718affd700 1 -- 172.21.15.33:0/2945010024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1330 msgr2=0x5651ed9aa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f718affd700 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1330 0x5651ed9aa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f718affd700 1 -- 172.21.15.33:0/2945010024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718c109bd0 con 0x5651ed9a09d0 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f718b7fe700 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a1cf0 0x5651ed9b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f718affd700 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a09d0 0x5651ed9aa2f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f717c0098c0 tx=0x7f717c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf99e65ea3620630 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f716eff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f716eff5700 1 -- 172.21.15.33:0/2945010024 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c02b040 con 0x5651ed9a09d0 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f719d5ddb80 1 -- 172.21.15.33:0/2945010024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651ed8452f0 con 0x5651ed9a09d0 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f716eff5700 1 -- 172.21.15.33:0/2945010024 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f717c005240 con 0x5651ed9a09d0 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f716eff5700 1 -- 172.21.15.33:0/2945010024 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f717c005400 con 0x5651ed9a09d0 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f719d5ddb80 10 client.17076 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.436+0000 7f719d5ddb80 1 -- 172.21.15.33:0/2945010024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f7184011230 con 0x5651ed9a09d0 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.437+0000 7f716eff5700 1 -- 172.21.15.33:0/2945010024 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5651ed8452f0 con 0x5651ed9a09d0 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.437+0000 7f716eff5700 1 -- 172.21.15.33:0/2945010024 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f717c037070 con 0x5651ed9a09d0 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.437+0000 7f719d5ddb80 10 client.17076 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.437+0000 7f719d5ddb80 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.437+0000 7f719d5ddb80 20 client.17076 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.437+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5651ed9b6080 0x5651ed9b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.437+0000 7f719d5ddb80 4 client.17076 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.437+0000 7f719d5ddb80 1 -- 172.21.15.33:0/2945010024 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651ed96b890 con 0x5651ed9b6080 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.437+0000 7f718a7fc700 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5651ed9b6080 0x5651ed9b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f718a7fc700 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5651ed9b6080 0x5651ed9b84a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de24ae98fbf63794 server_cookie=34db9d77291022ea in_seq=0 out_seq=0 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f716eff5700 10 client.17076 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f716eff5700 1 -- 172.21.15.33:0/2945010024 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5651ed96b890 con 0x5651ed9b6080 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f716eff5700 10 client.17076 handle_command_reply: tid=0 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f719d5ddb80 1 client.17076 shutdown 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f719d5ddb80 1 -- 172.21.15.33:0/2945010024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a09d0 msgr2=0x5651ed9aa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a09d0 0x5651ed9aa2f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f717c0098c0 tx=0x7f717c009c00 comp rx=0 tx=0).stop 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f719d5ddb80 1 -- 172.21.15.33:0/2945010024 shutdown_connections 2024-05-08T21:24:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5651ed9b6080 0x5651ed9b84a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a1cf0 0x5651ed9b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a09d0 0x5651ed9aa2f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f716eff5700 0 client.17076 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:30.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/2945010024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1330 0x5651ed9aa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.438+0000 7f719d5ddb80 1 -- 172.21.15.33:0/2945010024 >> 172.21.15.33:0/2945010024 conn(0x5651ed8fa290 msgr2=0x5651ed996620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:30.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.439+0000 7f719d5ddb80 1 -- 172.21.15.33:0/2945010024 shutdown_connections 2024-05-08T21:24:30.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.439+0000 7f719d5ddb80 1 -- 172.21.15.33:0/2945010024 wait complete. 2024-05-08T21:24:30.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.439+0000 7f719d5ddb80 20 client.17076 trim_cache size 0 max 16384 2024-05-08T21:24:30.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f719d5ddb80 1 Processor -- start 2024-05-08T21:24:30.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f719d5ddb80 1 -- start start 2024-05-08T21:24:30.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f719d5ddb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a09d0 0x5651ed9a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f719d5ddb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1330 0x5651ed9b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f719d5ddb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1cf0 0x5651ed9aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f719d5ddb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ed993e40 con 0x5651ed9a1cf0 2024-05-08T21:24:30.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f719d5ddb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ed9bd6b0 con 0x5651ed9a1330 2024-05-08T21:24:30.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f719d5ddb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ed9988b0 con 0x5651ed9a09d0 2024-05-08T21:24:30.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f718b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1cf0 0x5651ed9aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f718a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1330 0x5651ed9b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f718b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1cf0 0x5651ed9aa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53200/0 (socket says 172.21.15.33:53200) 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f718a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1330 0x5651ed9b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49646/0 (socket says 172.21.15.33:49646) 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f718b7fe700 1 -- 172.21.15.33:0/3784637535 learned_addr learned my addr 172.21.15.33:0/3784637535 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.463+0000 7f718affd700 1 --2- 172.21.15.33:0/3784637535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a09d0 0x5651ed9a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.464+0000 7f718a7fc700 1 -- 172.21.15.33:0/3784637535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a09d0 msgr2=0x5651ed9a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.464+0000 7f718a7fc700 1 --2- 172.21.15.33:0/3784637535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a09d0 0x5651ed9a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.464+0000 7f718a7fc700 1 -- 172.21.15.33:0/3784637535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1cf0 msgr2=0x5651ed9aa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.464+0000 7f718a7fc700 1 --2- 172.21.15.33:0/3784637535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1cf0 0x5651ed9aa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.464+0000 7f718a7fc700 1 -- 172.21.15.33:0/3784637535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651ed719be0 con 0x5651ed9a1330 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.464+0000 7f718b7fe700 1 --2- 172.21.15.33:0/3784637535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1cf0 0x5651ed9aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.464+0000 7f718a7fc700 1 --2- 172.21.15.33:0/3784637535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1330 0x5651ed9b1830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f71800ca420 tx=0x7f7180009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99b2e307f7e52183 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.464+0000 7f7171ffb700 1 -- 172.21.15.33:0/3784637535 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71800b6070 con 0x5651ed9a1330 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.464+0000 7f7171ffb700 1 -- 172.21.15.33:0/3784637535 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f718009dba0 con 0x5651ed9a1330 2024-05-08T21:24:30.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.464+0000 7f7171ffb700 1 -- 172.21.15.33:0/3784637535 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7180078ea0 con 0x5651ed9a1330 2024-05-08T21:24:30.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.465+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3784637535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1330 msgr2=0x5651ed9b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.465+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3784637535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1330 0x5651ed9b1830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f71800ca420 tx=0x7f7180009550 comp rx=0 tx=0).stop 2024-05-08T21:24:30.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.465+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3784637535 shutdown_connections 2024-05-08T21:24:30.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.465+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3784637535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1cf0 0x5651ed9aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.465+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3784637535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a09d0 0x5651ed9a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.465+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3784637535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a1330 0x5651ed9b1830 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.465+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3784637535 >> 172.21.15.33:0/3784637535 conn(0x5651ed8fa290 msgr2=0x5651ed9a82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:30.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.465+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3784637535 shutdown_connections 2024-05-08T21:24:30.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.465+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3784637535 wait complete. 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f719d5ddb80 1 -- start start 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f719d5ddb80 1 Processor -- start 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f719d5ddb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a09d0 0x5651ed9b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f719d5ddb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a1330 0x5651ed9ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f719d5ddb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1cf0 0x5651ed9ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f719d5ddb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ed9aeb10 con 0x5651ed9a1cf0 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f719d5ddb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ed9aec80 con 0x5651ed9a09d0 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f719d5ddb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651eda0c570 con 0x5651ed9a1330 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f719d5ddb80 10 client.0 _ll_register_callbacks cb 0x5651ed891bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f718a7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a1330 0x5651ed9ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f718b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1cf0 0x5651ed9ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f718affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a09d0 0x5651ed9b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f718a7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a1330 0x5651ed9ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:35042/0 (socket says 172.21.15.33:35042) 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.466+0000 7f718a7fc700 1 -- 172.21.15.33:0/3631419574 learned_addr learned my addr 172.21.15.33:0/3631419574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:30.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f718affd700 1 -- 172.21.15.33:0/3631419574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a1330 msgr2=0x5651ed9ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f718affd700 1 --2- 172.21.15.33:0/3631419574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a1330 0x5651ed9ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f718affd700 1 -- 172.21.15.33:0/3631419574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1cf0 msgr2=0x5651ed9ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f718affd700 1 --2- 172.21.15.33:0/3631419574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1cf0 0x5651ed9ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f718affd700 1 -- 172.21.15.33:0/3631419574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651ed76d430 con 0x5651ed9a09d0 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f718affd700 1 --2- 172.21.15.33:0/3631419574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a09d0 0x5651ed9b14b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f717c009810 tx=0x7f717c020d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab3e499c5fd4cbf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f716eff5700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f716eff5700 1 -- 172.21.15.33:0/3631419574 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c02b070 con 0x5651ed9a09d0 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f716eff5700 1 -- 172.21.15.33:0/3631419574 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f717c025620 con 0x5651ed9a09d0 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3631419574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f718400f040 con 0x5651ed9a09d0 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f716eff5700 1 -- 172.21.15.33:0/3631419574 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f717c00f720 con 0x5651ed9a09d0 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f719d5ddb80 10 client.26978 fetch_fsmap learned FSMap version 34 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.467+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3631419574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f718c19f2b0 con 0x5651ed9a09d0 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.468+0000 7f716eff5700 1 -- 172.21.15.33:0/3631419574 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f718400f040 con 0x5651ed9a09d0 2024-05-08T21:24:30.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.468+0000 7f716eff5700 1 -- 172.21.15.33:0/3631419574 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 34) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f717c020070 con 0x5651ed9a09d0 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.468+0000 7f719d5ddb80 10 client.26978 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.468+0000 7f719d5ddb80 10 client.26978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.468+0000 7f719d5ddb80 20 client.26978 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.468+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3631419574 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5651eda0db80 0x5651eda0ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.468+0000 7f719d5ddb80 4 client.26978 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.468+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3631419574 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5651ed8e4f40 con 0x5651eda0db80 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.468+0000 7f718a7fc700 1 --2- 172.21.15.33:0/3631419574 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5651eda0db80 0x5651eda0ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.468+0000 7f716eff5700 10 client.26978 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.469+0000 7f718a7fc700 1 --2- 172.21.15.33:0/3631419574 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5651eda0db80 0x5651eda0ffa0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a896653db15e356 server_cookie=d4adaa275848d090 in_seq=0 out_seq=0 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.469+0000 7f716eff5700 1 -- 172.21.15.33:0/3631419574 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5651ed8e4f40 con 0x5651eda0db80 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.469+0000 7f716eff5700 10 client.26978 handle_command_reply: tid=0 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.469+0000 7f719d5ddb80 1 client.26978 shutdown 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.470+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3631419574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a09d0 msgr2=0x5651ed9b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.470+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3631419574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a09d0 0x5651ed9b14b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f717c009810 tx=0x7f717c020d80 comp rx=0 tx=0).stop 2024-05-08T21:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.470+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3631419574 shutdown_connections 2024-05-08T21:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.470+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3631419574 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5651eda0db80 0x5651eda0ffa0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.470+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3631419574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651ed9a1cf0 0x5651ed9ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.470+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3631419574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651ed9a09d0 0x5651ed9b14b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.470+0000 7f719d5ddb80 1 --2- 172.21.15.33:0/3631419574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ed9a1330 0x5651ed9ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.470+0000 7f716eff5700 0 client.26978 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.470+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3631419574 >> 172.21.15.33:0/3631419574 conn(0x5651ed8fa290 msgr2=0x5651ed9a9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.470+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3631419574 shutdown_connections 2024-05-08T21:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.470+0000 7f719d5ddb80 1 -- 172.21.15.33:0/3631419574 wait complete. 2024-05-08T21:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.470+0000 7f719d5ddb80 20 client.26978 trim_cache size 0 max 16384 2024-05-08T21:24:30.477 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:24:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:24:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "18bf1d10-bfed-484f-9fee-7d9f42b09677", 2024-05-08T21:24:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:24:30.478 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:24:30.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.471+0000 7f7191423700 1 -- 172.21.15.33:0/2397188933 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71740a11a0 msgr2=0x7f71740a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.471+0000 7f7191423700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71740a11a0 0x7f71740a3660 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f717c005e90 tx=0x7f717c019040 comp rx=0 tx=0).stop 2024-05-08T21:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.471+0000 7f7191423700 1 -- 172.21.15.33:0/2397188933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c101fc0 msgr2=0x7f718c197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.471+0000 7f7191423700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c101fc0 0x7f718c197240 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f718000e9c0 tx=0x7f718000ecd0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.471+0000 7f7191423700 1 -- 172.21.15.33:0/2397188933 shutdown_connections 2024-05-08T21:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.471+0000 7f7191423700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f71740a11a0 0x7f71740a3660 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.471+0000 7f7191423700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f718c102980 0x7f718c19e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.471+0000 7f7191423700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f718c100e70 0x7f718c196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.471+0000 7f7191423700 1 --2- 172.21.15.33:0/2397188933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c101fc0 0x7f718c197240 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.471+0000 7f7191423700 1 -- 172.21.15.33:0/2397188933 >> 172.21.15.33:0/2397188933 conn(0x7f718c0fc400 msgr2=0x7f718c105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.471+0000 7f7191423700 1 -- 172.21.15.33:0/2397188933 shutdown_connections 2024-05-08T21:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.471+0000 7f7191423700 1 -- 172.21.15.33:0/2397188933 wait complete. 2024-05-08T21:24:30.491 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:24:30.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.690+0000 7f19b29a7700 1 -- 172.21.15.33:0/3792651250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ac0ff6d0 msgr2=0x7f19ac101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.690+0000 7f19b29a7700 1 --2- 172.21.15.33:0/3792651250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ac0ff6d0 0x7f19ac101b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f199c009990 tx=0x7f199c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.690+0000 7f19b29a7700 1 -- 172.21.15.33:0/3792651250 shutdown_connections 2024-05-08T21:24:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.690+0000 7f19b29a7700 1 --2- 172.21.15.33:0/3792651250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19ac104a10 0x7f19ac104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.690+0000 7f19b29a7700 1 --2- 172.21.15.33:0/3792651250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ac0ff6d0 0x7f19ac101b00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.690+0000 7f19b29a7700 1 --2- 172.21.15.33:0/3792651250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19ac102040 0x7f19ac1044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.690+0000 7f19b29a7700 1 -- 172.21.15.33:0/3792651250 >> 172.21.15.33:0/3792651250 conn(0x7f19ac0fb290 msgr2=0x7f19ac0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:30.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.691+0000 7f19b29a7700 1 -- 172.21.15.33:0/3792651250 shutdown_connections 2024-05-08T21:24:30.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.692+0000 7f19b29a7700 1 -- 172.21.15.33:0/3792651250 wait complete. 2024-05-08T21:24:30.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.692+0000 7f19b29a7700 1 Processor -- start 2024-05-08T21:24:30.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.693+0000 7f19b29a7700 1 -- start start 2024-05-08T21:24:30.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19b29a7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19ac0ff6d0 0x7f19ac10ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19b29a7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ac102040 0x7f19ac10b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19b29a7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19ac104a10 0x7f19ac1ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19b29a7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac10b790 con 0x7f19ac104a10 2024-05-08T21:24:30.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19b29a7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac10b8d0 con 0x7f19ac0ff6d0 2024-05-08T21:24:30.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19b29a7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac06a9f0 con 0x7f19ac102040 2024-05-08T21:24:30.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19abfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19ac0ff6d0 0x7f19ac10ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19abfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19ac0ff6d0 0x7f19ac10ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:49670/0 (socket says 172.21.15.33:49670) 2024-05-08T21:24:30.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19abfff700 1 -- 172.21.15.33:0/1470603115 learned_addr learned my addr 172.21.15.33:0/1470603115 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:30.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19ab7fe700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ac102040 0x7f19ac10b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19abfff700 1 -- 172.21.15.33:0/1470603115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ac102040 msgr2=0x7f19ac10b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19abfff700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ac102040 0x7f19ac10b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19abfff700 1 -- 172.21.15.33:0/1470603115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19ac104a10 msgr2=0x7f19ac1ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19abfff700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19ac104a10 0x7f19ac1ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.694+0000 7f19abfff700 1 -- 172.21.15.33:0/1470603115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a0009520 con 0x7f19ac0ff6d0 2024-05-08T21:24:30.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.695+0000 7f19abfff700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19ac0ff6d0 0x7f19ac10ac80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f199c005740 tx=0x7f199c005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:30.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.696+0000 7f19a97fa700 1 -- 172.21.15.33:0/1470603115 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c01d070 con 0x7f19ac0ff6d0 2024-05-08T21:24:30.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.696+0000 7f19a97fa700 1 -- 172.21.15.33:0/1470603115 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f199c00beb0 con 0x7f19ac0ff6d0 2024-05-08T21:24:30.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.696+0000 7f19b29a7700 1 -- 172.21.15.33:0/1470603115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199c0095f0 con 0x7f19ac0ff6d0 2024-05-08T21:24:30.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.696+0000 7f19a97fa700 1 -- 172.21.15.33:0/1470603115 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c021410 con 0x7f19ac0ff6d0 2024-05-08T21:24:30.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.696+0000 7f19b29a7700 1 -- 172.21.15.33:0/1470603115 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac06aea0 con 0x7f19ac0ff6d0 2024-05-08T21:24:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.700+0000 7f19beb61b80 1 Processor -- start 2024-05-08T21:24:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.700+0000 7f19beb61b80 1 -- start start 2024-05-08T21:24:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.700+0000 7f19beb61b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1540 0x55fe0a6f1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.700+0000 7f19beb61b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6f2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.700+0000 7f19beb61b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a6f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.700+0000 7f19beb61b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0a648150 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.700+0000 7f19beb61b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0a645b10 con 0x55fe0a6f2860 2024-05-08T21:24:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.700+0000 7f19beb61b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0a641400 con 0x55fe0a6f1540 2024-05-08T21:24:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.700+0000 7f19a97fa700 1 -- 172.21.15.33:0/1470603115 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f199c00f720 con 0x7f19ac0ff6d0 2024-05-08T21:24:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19b0f44700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1540 0x55fe0a6f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19b0f44700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1540 0x55fe0a6f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:35052/0 (socket says 172.21.15.33:35052) 2024-05-08T21:24:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19b0f44700 1 -- 172.21.15.33:0/1889042323 learned_addr learned my addr 172.21.15.33:0/1889042323 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19abfff700 1 --2- 172.21.15.33:0/1889042323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19ab7fe700 1 --2- 172.21.15.33:0/1889042323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a6f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19b0f44700 1 -- 172.21.15.33:0/1889042323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 msgr2=0x55fe0a6f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19b0f44700 1 --2- 172.21.15.33:0/1889042323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a6f2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19b0f44700 1 -- 172.21.15.33:0/1889042323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1ea0 msgr2=0x55fe0a6f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19b0f44700 1 --2- 172.21.15.33:0/1889042323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19b0f44700 1 -- 172.21.15.33:0/1889042323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe0a4c23b0 con 0x55fe0a6f1540 2024-05-08T21:24:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19b0f44700 1 --2- 172.21.15.33:0/1889042323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1540 0x55fe0a6f1960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009c90 tx=0x7f19a4007500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a1f5778d6d3f97d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19a97fa700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f19940a11a0 0x7f19940a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19927fc700 1 -- 172.21.15.33:0/1889042323 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4017070 con 0x55fe0a6f1540 2024-05-08T21:24:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19927fc700 1 -- 172.21.15.33:0/1889042323 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19a400f040 con 0x55fe0a6f1540 2024-05-08T21:24:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19927fc700 1 -- 172.21.15.33:0/1889042323 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4005890 con 0x55fe0a6f1540 2024-05-08T21:24:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.701+0000 7f19a97fa700 1 -- 172.21.15.33:0/1470603115 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f199c0ce0a0 con 0x7f19ac0ff6d0 2024-05-08T21:24:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.702+0000 7f19abfff700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f19940a11a0 0x7f19940a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.703+0000 7f19beb61b80 1 -- 172.21.15.33:0/1889042323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1540 msgr2=0x55fe0a6f1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.703+0000 7f19beb61b80 1 --2- 172.21.15.33:0/1889042323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1540 0x55fe0a6f1960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009c90 tx=0x7f19a4007500 comp rx=0 tx=0).stop 2024-05-08T21:24:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.703+0000 7f19abfff700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f19940a11a0 0x7f19940a3660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f199c0a3080 tx=0x7f199c009ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.703+0000 7f19beb61b80 1 -- 172.21.15.33:0/1889042323 shutdown_connections 2024-05-08T21:24:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.703+0000 7f19beb61b80 1 --2- 172.21.15.33:0/1889042323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a6f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.703+0000 7f19beb61b80 1 --2- 172.21.15.33:0/1889042323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1540 0x55fe0a6f1960 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.703+0000 7f19beb61b80 1 --2- 172.21.15.33:0/1889042323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.703+0000 7f19beb61b80 1 -- 172.21.15.33:0/1889042323 >> 172.21.15.33:0/1889042323 conn(0x55fe0a64f210 msgr2=0x55fe0a6ebde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.703+0000 7f19beb61b80 1 -- 172.21.15.33:0/1889042323 shutdown_connections 2024-05-08T21:24:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.703+0000 7f19beb61b80 1 -- 172.21.15.33:0/1889042323 wait complete. 2024-05-08T21:24:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.703+0000 7f19beb61b80 1 -- start start 2024-05-08T21:24:30.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19beb61b80 1 Processor -- start 2024-05-08T21:24:30.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:30 smithi033 ceph-mon[42528]: pgmap v456: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.7 MiB/s wr, 529 op/s 2024-05-08T21:24:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19beb61b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6fae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19beb61b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6fb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19beb61b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a7023b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19beb61b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0a5dd560 con 0x55fe0a6f1540 2024-05-08T21:24:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19beb61b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0a5dd200 con 0x55fe0a6f2860 2024-05-08T21:24:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19beb61b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0a4c46a0 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19beb61b80 10 client.0 _ll_register_callbacks cb 0x55fe0a5e6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19ab7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6fb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19ab7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6fb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:35064/0 (socket says 172.21.15.33:35064) 2024-05-08T21:24:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19ab7fe700 1 -- 172.21.15.33:0/1238748232 learned_addr learned my addr 172.21.15.33:0/1238748232 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19b0f44700 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a7023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.704+0000 7f19b0f44700 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f19ab7fe700 1 -- 172.21.15.33:0/1238748232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 msgr2=0x55fe0a7023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f19ab7fe700 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a7023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f19ab7fe700 1 -- 172.21.15.33:0/1238748232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 msgr2=0x55fe0a6fae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f19ab7fe700 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6fae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f19ab7fe700 1 -- 172.21.15.33:0/1238748232 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a40098c0 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f19b0f44700 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a7023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f19ab7fe700 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6fb370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f19a0011890 tx=0x7f19a0011ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37d3bef77dee969b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f198b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f198b7fe700 1 -- 172.21.15.33:0/1238748232 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0016070 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f198b7fe700 1 -- 172.21.15.33:0/1238748232 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f19a0004830 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f19beb61b80 1 -- 172.21.15.33:0/1238748232 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe0a59a270 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f198b7fe700 1 -- 172.21.15.33:0/1238748232 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19a00049f0 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f19beb61b80 10 client.26998 fetch_fsmap learned FSMap version 35 2024-05-08T21:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.705+0000 7f19beb61b80 1 -- 172.21.15.33:0/1238748232 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55fe0a46eb60 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.706+0000 7f198b7fe700 1 -- 172.21.15.33:0/1238748232 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fe0a59a270 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.706+0000 7f198b7fe700 1 -- 172.21.15.33:0/1238748232 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 35) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f19a000f800 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.706+0000 7f19beb61b80 10 client.26998 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.706+0000 7f19beb61b80 10 client.26998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.706+0000 7f19beb61b80 20 client.26998 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.706+0000 7f19beb61b80 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fe0a706d20 0x55fe0a709140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.706+0000 7f19beb61b80 4 client.26998 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.706+0000 7f19beb61b80 1 -- 172.21.15.33:0/1238748232 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe0a6c0810 con 0x55fe0a706d20 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.706+0000 7f19b0f44700 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fe0a706d20 0x55fe0a709140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.706+0000 7f198b7fe700 10 client.26998 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.707+0000 7f19b0f44700 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fe0a706d20 0x55fe0a709140 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57fd3c560540ef4 server_cookie=7aa997e13dc011c0 in_seq=0 out_seq=0 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.707+0000 7f198b7fe700 1 -- 172.21.15.33:0/1238748232 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fe0a6c0810 con 0x55fe0a706d20 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.707+0000 7f198b7fe700 10 client.26998 handle_command_reply: tid=0 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.707+0000 7f19beb61b80 1 client.26998 shutdown 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.707+0000 7f19beb61b80 1 -- 172.21.15.33:0/1238748232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 msgr2=0x55fe0a6fb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.707+0000 7f19beb61b80 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6fb370 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f19a0011890 tx=0x7f19a0011ba0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.708+0000 7f19beb61b80 1 -- 172.21.15.33:0/1238748232 shutdown_connections 2024-05-08T21:24:30.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.708+0000 7f19beb61b80 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fe0a706d20 0x55fe0a709140 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.708+0000 7f19beb61b80 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a7023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.708+0000 7f19beb61b80 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6fae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.708+0000 7f19beb61b80 1 --2- 172.21.15.33:0/1238748232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6fb370 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.708+0000 7f19beb61b80 1 -- 172.21.15.33:0/1238748232 >> 172.21.15.33:0/1238748232 conn(0x55fe0a64f210 msgr2=0x55fe0a6eb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:30.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.708+0000 7f198b7fe700 0 client.26998 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:30.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.708+0000 7f19beb61b80 1 -- 172.21.15.33:0/1238748232 shutdown_connections 2024-05-08T21:24:30.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.708+0000 7f19beb61b80 1 -- 172.21.15.33:0/1238748232 wait complete. 2024-05-08T21:24:30.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.708+0000 7f19beb61b80 20 client.26998 trim_cache size 0 max 16384 2024-05-08T21:24:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.729+0000 7f19beb61b80 1 Processor -- start 2024-05-08T21:24:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19beb61b80 1 -- start start 2024-05-08T21:24:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19beb61b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19beb61b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19beb61b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a702370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19beb61b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0a6fa9d0 con 0x55fe0a6f1540 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19beb61b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0a6e8dc0 con 0x55fe0a6f2860 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19beb61b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0a6ed5a0 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19b0f44700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19ab7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19b0f44700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53266/0 (socket says 172.21.15.33:53266) 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19b0f44700 1 -- 172.21.15.33:0/3585547182 learned_addr learned my addr 172.21.15.33:0/3585547182 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19b0f44700 1 --2- 172.21.15.33:0/3585547182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a702370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19ab7fe700 1 -- 172.21.15.33:0/3585547182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 msgr2=0x55fe0a702370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19ab7fe700 1 --2- 172.21.15.33:0/3585547182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a702370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19ab7fe700 1 -- 172.21.15.33:0/3585547182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 msgr2=0x55fe0a6f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19ab7fe700 1 --2- 172.21.15.33:0/3585547182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.730+0000 7f19ab7fe700 1 -- 172.21.15.33:0/3585547182 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe0a607560 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.731+0000 7f19ab7fe700 1 --2- 172.21.15.33:0/3585547182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6f5d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f19a000fc60 tx=0x7f19a000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8afee0891a955cb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.731+0000 7f1992ffd700 1 -- 172.21.15.33:0/3585547182 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0016070 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.731+0000 7f1992ffd700 1 -- 172.21.15.33:0/3585547182 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19a0009390 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.731+0000 7f1992ffd700 1 -- 172.21.15.33:0/3585547182 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a00038e0 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.732+0000 7f19beb61b80 1 -- 172.21.15.33:0/3585547182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 msgr2=0x55fe0a6f5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.732+0000 7f19beb61b80 1 --2- 172.21.15.33:0/3585547182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6f5d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f19a000fc60 tx=0x7f19a000f3c0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.732+0000 7f19beb61b80 1 -- 172.21.15.33:0/3585547182 shutdown_connections 2024-05-08T21:24:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.732+0000 7f19beb61b80 1 --2- 172.21.15.33:0/3585547182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f2860 0x55fe0a702370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.732+0000 7f19beb61b80 1 --2- 172.21.15.33:0/3585547182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.732+0000 7f19beb61b80 1 --2- 172.21.15.33:0/3585547182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6f5d10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.732+0000 7f19beb61b80 1 -- 172.21.15.33:0/3585547182 >> 172.21.15.33:0/3585547182 conn(0x55fe0a64f210 msgr2=0x55fe0a70daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.732+0000 7f19beb61b80 1 -- 172.21.15.33:0/3585547182 shutdown_connections 2024-05-08T21:24:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.732+0000 7f19beb61b80 1 -- 172.21.15.33:0/3585547182 wait complete. 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.732+0000 7f19beb61b80 1 -- start start 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19beb61b80 1 Processor -- start 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19beb61b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6fdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19beb61b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6fe250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19beb61b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f2860 0x55fe0a701270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19beb61b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0a7017b0 con 0x55fe0a6f1540 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19beb61b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0a701920 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19beb61b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe0a701ae0 con 0x55fe0a6f2860 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19beb61b80 10 client.0 _ll_register_callbacks cb 0x55fe0a5e6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19b0f44700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6fdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19b0f44700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6fdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53280/0 (socket says 172.21.15.33:53280) 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19ab7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6fe250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19b0f44700 1 -- 172.21.15.33:0/2224272650 learned_addr learned my addr 172.21.15.33:0/2224272650 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:24:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19b0f44700 1 --2- 172.21.15.33:0/2224272650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f2860 0x55fe0a701270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19ab7fe700 1 -- 172.21.15.33:0/2224272650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f2860 msgr2=0x55fe0a701270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19ab7fe700 1 --2- 172.21.15.33:0/2224272650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f2860 0x55fe0a701270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19ab7fe700 1 -- 172.21.15.33:0/2224272650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 msgr2=0x55fe0a6fdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19ab7fe700 1 --2- 172.21.15.33:0/2224272650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6fdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.733+0000 7f19ab7fe700 1 -- 172.21.15.33:0/2224272650 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe0a46eb60 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f19ab7fe700 1 --2- 172.21.15.33:0/2224272650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6fe250 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19a000fcb0 tx=0x7f19a000fa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2114563af82acb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f198b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f198b7fe700 1 -- 172.21.15.33:0/2224272650 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0036070 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f198b7fe700 1 -- 172.21.15.33:0/2224272650 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f19a00052e0 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f19beb61b80 1 -- 172.21.15.33:0/2224272650 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19a400f040 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f198b7fe700 1 -- 172.21.15.33:0/2224272650 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19a0016040 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f19beb61b80 10 client.26996 fetch_fsmap learned FSMap version 35 2024-05-08T21:24:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f19beb61b80 1 -- 172.21.15.33:0/2224272650 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55fe0a4c6cd0 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f198b7fe700 1 -- 172.21.15.33:0/2224272650 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19a400f040 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f198b7fe700 1 -- 172.21.15.33:0/2224272650 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f19a0039030 con 0x55fe0a6f1ea0 2024-05-08T21:24:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f19beb61b80 10 client.26996 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T21:24:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f19beb61b80 10 client.26996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:24:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f19beb61b80 20 client.26996 populate_metadata read hostname 'smithi033' 2024-05-08T21:24:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f19beb61b80 1 --2- 172.21.15.33:0/2224272650 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fe0a75d950 0x55fe0a75fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:24:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f19beb61b80 4 client.26996 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:24:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.734+0000 7f19beb61b80 1 -- 172.21.15.33:0/2224272650 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fe0a639ec0 con 0x55fe0a75d950 2024-05-08T21:24:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.735+0000 7f19b0f44700 1 --2- 172.21.15.33:0/2224272650 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fe0a75d950 0x55fe0a75fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:24:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.735+0000 7f198b7fe700 10 client.26996 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.735+0000 7f19b0f44700 1 --2- 172.21.15.33:0/2224272650 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fe0a75d950 0x55fe0a75fd70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f36d5521526a69 server_cookie=dc7a2643bce74e4d in_seq=0 out_seq=0 2024-05-08T21:24:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f198b7fe700 1 -- 172.21.15.33:0/2224272650 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fe0a639ec0 con 0x55fe0a75d950 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f198b7fe700 10 client.26996 handle_command_reply: tid=0 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f19beb61b80 1 client.26996 shutdown 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f19beb61b80 1 -- 172.21.15.33:0/2224272650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f1ea0 msgr2=0x55fe0a6fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f19beb61b80 1 --2- 172.21.15.33:0/2224272650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6fe250 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19a000fcb0 tx=0x7f19a000fa20 comp rx=0 tx=0).stop 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f19beb61b80 1 -- 172.21.15.33:0/2224272650 shutdown_connections 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f19beb61b80 1 --2- 172.21.15.33:0/2224272650 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fe0a75d950 0x55fe0a75fd70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f19beb61b80 1 --2- 172.21.15.33:0/2224272650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fe0a6f2860 0x55fe0a701270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f19beb61b80 1 --2- 172.21.15.33:0/2224272650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe0a6f1540 0x55fe0a6fdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f19beb61b80 1 --2- 172.21.15.33:0/2224272650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe0a6f1ea0 0x55fe0a6fe250 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f19beb61b80 1 -- 172.21.15.33:0/2224272650 >> 172.21.15.33:0/2224272650 conn(0x55fe0a64f210 msgr2=0x55fe0a6f57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f198b7fe700 0 client.26996 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f19beb61b80 1 -- 172.21.15.33:0/2224272650 shutdown_connections 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f19beb61b80 1 -- 172.21.15.33:0/2224272650 wait complete. 2024-05-08T21:24:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.736+0000 7f19beb61b80 20 client.26996 trim_cache size 0 max 16384 2024-05-08T21:24:30.743 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:24:30.743 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:24:30.743 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:24:30.743 INFO:teuthology.orchestra.run.smithi033.stdout: "18bf1d10-bfed-484f-9fee-7d9f42b09677": { 2024-05-08T21:24:30.743 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:24:30.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "18bf1d10-bfed-484f-9fee-7d9f42b09677", 2024-05-08T21:24:30.743 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:24:30.743 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:24:30.743 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:24:30.743 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:24:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.737+0000 7f19b29a7700 1 -- 172.21.15.33:0/1470603115 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f19940a11a0 msgr2=0x7f19940a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.737+0000 7f19b29a7700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f19940a11a0 0x7f19940a3660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f199c0a3080 tx=0x7f199c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.737+0000 7f19b29a7700 1 -- 172.21.15.33:0/1470603115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19ac0ff6d0 msgr2=0x7f19ac10ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:24:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.737+0000 7f19b29a7700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19ac0ff6d0 0x7f19ac10ac80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f199c005740 tx=0x7f199c005770 comp rx=0 tx=0).stop 2024-05-08T21:24:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.738+0000 7f19b29a7700 1 -- 172.21.15.33:0/1470603115 shutdown_connections 2024-05-08T21:24:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.738+0000 7f19b29a7700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19ac104a10 0x7f19ac1ad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.738+0000 7f19b29a7700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f19940a11a0 0x7f19940a3660 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.738+0000 7f19b29a7700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19ac0ff6d0 0x7f19ac10ac80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.738+0000 7f19b29a7700 1 --2- 172.21.15.33:0/1470603115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19ac102040 0x7f19ac10b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:24:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.738+0000 7f19b29a7700 1 -- 172.21.15.33:0/1470603115 >> 172.21.15.33:0/1470603115 conn(0x7f19ac0fb290 msgr2=0x7f19ac103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:24:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.738+0000 7f19b29a7700 1 -- 172.21.15.33:0/1470603115 shutdown_connections 2024-05-08T21:24:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:24:30.738+0000 7f19b29a7700 1 -- 172.21.15.33:0/1470603115 wait complete. 2024-05-08T21:24:30.756 INFO:tasks.cephfs.filesystem:scrub status for tag:18bf1d10-bfed-484f-9fee-7d9f42b09677 - {'path': '/', 'tag': '18bf1d10-bfed-484f-9fee-7d9f42b09677', 'options': 'recursive,force'} 2024-05-08T21:24:30.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:30 smithi071 ceph-mon[47074]: pgmap v456: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.7 MiB/s wr, 529 op/s 2024-05-08T21:24:30.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:30 smithi082 ceph-mon[47512]: pgmap v456: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 3.7 MiB/s wr, 529 op/s 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:24:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:24:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:24:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:31 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:24:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:24:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:24:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:24:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:24:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:24:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:24:31.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:24:31.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:24:31.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:24:31.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:31 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:24:31.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:24:31.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:31.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:31 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:32.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:32 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:32.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:32 smithi071 ceph-mon[47074]: pgmap v457: 129 pgs: 129 active+clean; 211 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.8 MiB/s wr, 557 op/s 2024-05-08T21:24:32.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:32 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:32.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:32 smithi082 ceph-mon[47512]: pgmap v457: 129 pgs: 129 active+clean; 211 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.8 MiB/s wr, 557 op/s 2024-05-08T21:24:32.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:32 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:24:32.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:32 smithi033 ceph-mon[42528]: pgmap v457: 129 pgs: 129 active+clean; 211 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.8 MiB/s wr, 557 op/s 2024-05-08T21:24:35.569 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:35 smithi071 ceph-mon[47074]: pgmap v458: 129 pgs: 129 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.3 MiB/s wr, 495 op/s 2024-05-08T21:24:35.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:35 smithi082 ceph-mon[47512]: pgmap v458: 129 pgs: 129 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.3 MiB/s wr, 495 op/s 2024-05-08T21:24:35.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:35 smithi033 ceph-mon[42528]: pgmap v458: 129 pgs: 129 active+clean; 212 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.3 MiB/s wr, 495 op/s 2024-05-08T21:24:37.569 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:37 smithi071 ceph-mon[47074]: pgmap v459: 129 pgs: 129 active+clean; 214 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.4 MiB/s wr, 634 op/s 2024-05-08T21:24:37.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:37 smithi082 ceph-mon[47512]: pgmap v459: 129 pgs: 129 active+clean; 214 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.4 MiB/s wr, 634 op/s 2024-05-08T21:24:37.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:37 smithi033 ceph-mon[42528]: pgmap v459: 129 pgs: 129 active+clean; 214 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.4 MiB/s wr, 634 op/s 2024-05-08T21:24:39.569 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:39 smithi071 ceph-mon[47074]: pgmap v460: 129 pgs: 129 active+clean; 216 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.4 MiB/s wr, 485 op/s 2024-05-08T21:24:39.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:39 smithi082 ceph-mon[47512]: pgmap v460: 129 pgs: 129 active+clean; 216 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.4 MiB/s wr, 485 op/s 2024-05-08T21:24:39.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:39 smithi033 ceph-mon[42528]: pgmap v460: 129 pgs: 129 active+clean; 216 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.4 MiB/s wr, 485 op/s 2024-05-08T21:24:41.569 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:41 smithi071 ceph-mon[47074]: pgmap v461: 129 pgs: 129 active+clean; 217 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 567 op/s 2024-05-08T21:24:41.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:41 smithi082 ceph-mon[47512]: pgmap v461: 129 pgs: 129 active+clean; 217 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 567 op/s 2024-05-08T21:24:41.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:41 smithi033 ceph-mon[42528]: pgmap v461: 129 pgs: 129 active+clean; 217 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 567 op/s 2024-05-08T21:24:43.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:43 smithi071 ceph-mon[47074]: pgmap v462: 129 pgs: 129 active+clean; 215 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 560 op/s 2024-05-08T21:24:43.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:43 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:24:43.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:43 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:24:43.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:43 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:24:43.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:43 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:24:43.570 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:43 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:24:43.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:43 smithi082 ceph-mon[47512]: pgmap v462: 129 pgs: 129 active+clean; 215 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 560 op/s 2024-05-08T21:24:43.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:43 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:24:43.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:43 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:24:43.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:43 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:24:43.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:43 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:24:43.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:43 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:24:43.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:43 smithi033 ceph-mon[42528]: pgmap v462: 129 pgs: 129 active+clean; 215 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 560 op/s 2024-05-08T21:24:43.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:43 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:24:43.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:43 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:24:43.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:43 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:24:43.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:43 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:24:43.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:43 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:24:45.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:45 smithi082 ceph-mon[47512]: pgmap v463: 129 pgs: 129 active+clean; 216 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.9 MiB/s wr, 472 op/s 2024-05-08T21:24:45.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:45 smithi033 ceph-mon[42528]: pgmap v463: 129 pgs: 129 active+clean; 216 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.9 MiB/s wr, 472 op/s 2024-05-08T21:24:45.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:45 smithi071 ceph-mon[47074]: pgmap v463: 129 pgs: 129 active+clean; 216 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.9 MiB/s wr, 472 op/s 2024-05-08T21:24:47.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:47 smithi082 ceph-mon[47512]: pgmap v464: 129 pgs: 129 active+clean; 216 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 601 op/s 2024-05-08T21:24:47.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:47 smithi033 ceph-mon[42528]: pgmap v464: 129 pgs: 129 active+clean; 216 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 601 op/s 2024-05-08T21:24:47.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:47 smithi071 ceph-mon[47074]: pgmap v464: 129 pgs: 129 active+clean; 216 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 601 op/s 2024-05-08T21:24:49.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:49 smithi082 ceph-mon[47512]: pgmap v465: 129 pgs: 129 active+clean; 216 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.0 MiB/s wr, 474 op/s 2024-05-08T21:24:49.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:49 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:24:49.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:49 smithi033 ceph-mon[42528]: pgmap v465: 129 pgs: 129 active+clean; 216 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.0 MiB/s wr, 474 op/s 2024-05-08T21:24:49.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:49 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:24:49.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:49 smithi071 ceph-mon[47074]: pgmap v465: 129 pgs: 129 active+clean; 216 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.0 MiB/s wr, 474 op/s 2024-05-08T21:24:49.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:49 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:24:51.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:51 smithi082 ceph-mon[47512]: pgmap v466: 129 pgs: 129 active+clean; 217 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.3 MiB/s wr, 515 op/s 2024-05-08T21:24:51.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:51 smithi033 ceph-mon[42528]: pgmap v466: 129 pgs: 129 active+clean; 217 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.3 MiB/s wr, 515 op/s 2024-05-08T21:24:51.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:51 smithi071 ceph-mon[47074]: pgmap v466: 129 pgs: 129 active+clean; 217 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.3 MiB/s wr, 515 op/s 2024-05-08T21:24:53.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:53 smithi082 ceph-mon[47512]: pgmap v467: 129 pgs: 129 active+clean; 223 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.7 MiB/s wr, 551 op/s 2024-05-08T21:24:53.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:53 smithi033 ceph-mon[42528]: pgmap v467: 129 pgs: 129 active+clean; 223 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.7 MiB/s wr, 551 op/s 2024-05-08T21:24:53.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:53 smithi071 ceph-mon[47074]: pgmap v467: 129 pgs: 129 active+clean; 223 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.7 MiB/s wr, 551 op/s 2024-05-08T21:24:54.192 INFO:tasks.workunit.client.0.smithi033.stdout:20000 mapwrite 0x840504 thru 0x847a17 (0x7514 bytes) 2024-05-08T21:24:54.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:54 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:24:54.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:54 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:24:54.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:54 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:24:55.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:55 smithi082 ceph-mon[47512]: pgmap v468: 129 pgs: 129 active+clean; 223 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.9 MiB/s wr, 442 op/s 2024-05-08T21:24:55.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:55 smithi033 ceph-mon[42528]: pgmap v468: 129 pgs: 129 active+clean; 223 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.9 MiB/s wr, 442 op/s 2024-05-08T21:24:55.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:55 smithi071 ceph-mon[47074]: pgmap v468: 129 pgs: 129 active+clean; 223 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.9 MiB/s wr, 442 op/s 2024-05-08T21:24:57.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:57 smithi033 ceph-mon[42528]: pgmap v469: 129 pgs: 129 active+clean; 223 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.0 MiB/s wr, 603 op/s 2024-05-08T21:24:57.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:57 smithi071 ceph-mon[47074]: pgmap v469: 129 pgs: 129 active+clean; 223 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.0 MiB/s wr, 603 op/s 2024-05-08T21:24:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:57 smithi082 ceph-mon[47512]: pgmap v469: 129 pgs: 129 active+clean; 223 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.0 MiB/s wr, 603 op/s 2024-05-08T21:24:59.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:24:59 smithi033 ceph-mon[42528]: pgmap v470: 129 pgs: 129 active+clean; 221 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-05-08T21:24:59.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:24:59 smithi071 ceph-mon[47074]: pgmap v470: 129 pgs: 129 active+clean; 221 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-05-08T21:24:59.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:24:59 smithi082 ceph-mon[47512]: pgmap v470: 129 pgs: 129 active+clean; 221 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-05-08T21:25:00.757 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:25:00.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.930+0000 7f190e4f8700 1 -- 172.21.15.33:0/1273445682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1908104230 msgr2=0x7f1908106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:00.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.930+0000 7f190e4f8700 1 --2- 172.21.15.33:0/1273445682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1908104230 0x7f1908106620 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009990 tx=0x7f18fc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.930+0000 7f190e4f8700 1 -- 172.21.15.33:0/1273445682 shutdown_connections 2024-05-08T21:25:00.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.930+0000 7f190e4f8700 1 --2- 172.21.15.33:0/1273445682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1908106b60 0x7f1908108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.930+0000 7f190e4f8700 1 --2- 172.21.15.33:0/1273445682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1908101900 0x7f1908103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.930+0000 7f190e4f8700 1 --2- 172.21.15.33:0/1273445682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1908104230 0x7f1908106620 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.930+0000 7f190e4f8700 1 -- 172.21.15.33:0/1273445682 >> 172.21.15.33:0/1273445682 conn(0x7f19080fb310 msgr2=0x7f19080fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:00.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.930+0000 7f190e4f8700 1 -- 172.21.15.33:0/1273445682 shutdown_connections 2024-05-08T21:25:00.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.930+0000 7f190e4f8700 1 -- 172.21.15.33:0/1273445682 wait complete. 2024-05-08T21:25:00.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.931+0000 7f190e4f8700 1 Processor -- start 2024-05-08T21:25:00.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.931+0000 7f190e4f8700 1 -- start start 2024-05-08T21:25:00.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.931+0000 7f190e4f8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1908101900 0x7f1908101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.931+0000 7f190e4f8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1908104230 0x7f19080ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.931+0000 7f190e4f8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1908106b60 0x7f19080ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.931+0000 7f190e4f8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1908100590 con 0x7f1908101900 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.931+0000 7f190e4f8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1908100700 con 0x7f1908106b60 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.931+0000 7f190e4f8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19081a0db0 con 0x7f1908104230 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f1907fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1908101900 0x7f1908101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f1907fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1908101900 0x7f1908101220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43010/0 (socket says 172.21.15.33:43010) 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f1907fff700 1 -- 172.21.15.33:0/2769031141 learned_addr learned my addr 172.21.15.33:0/2769031141 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f19077fe700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1908104230 0x7f19080ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f190ca95700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1908106b60 0x7f19080ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f1907fff700 1 -- 172.21.15.33:0/2769031141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1908104230 msgr2=0x7f19080ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f1907fff700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1908104230 0x7f19080ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f1907fff700 1 -- 172.21.15.33:0/2769031141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1908106b60 msgr2=0x7f19080ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f1907fff700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1908106b60 0x7f19080ffdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f1907fff700 1 -- 172.21.15.33:0/2769031141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc0095f0 con 0x7f1908101900 2024-05-08T21:25:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f19077fe700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1908104230 0x7f19080ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f190ca95700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1908106b60 0x7f19080ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f1907fff700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1908101900 0x7f1908101220 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f18f800e9a0 tx=0x7f18f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f19057fa700 1 -- 172.21.15.33:0/2769031141 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8009790 con 0x7f1908101900 2024-05-08T21:25:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f190e4f8700 1 -- 172.21.15.33:0/2769031141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19081a0fe0 con 0x7f1908101900 2024-05-08T21:25:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f19057fa700 1 -- 172.21.15.33:0/2769031141 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18f800ce40 con 0x7f1908101900 2024-05-08T21:25:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f19057fa700 1 -- 172.21.15.33:0/2769031141 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8018420 con 0x7f1908101900 2024-05-08T21:25:00.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.932+0000 7f190e4f8700 1 -- 172.21.15.33:0/2769031141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19081a1460 con 0x7f1908101900 2024-05-08T21:25:00.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.935+0000 7f19057fa700 1 -- 172.21.15.33:0/2769031141 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f18f8010430 con 0x7f1908101900 2024-05-08T21:25:00.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.936+0000 7f19057fa700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f18f00a1120 0x7f18f00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.937+0000 7f19077fe700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f18f00a1120 0x7f18f00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.937+0000 7f19057fa700 1 -- 172.21.15.33:0/2769031141 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f18f8014070 con 0x7f1908101900 2024-05-08T21:25:00.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.937+0000 7f19077fe700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f18f00a1120 0x7f18f00a35e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f18fc006010 tx=0x7f18fc00b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:00.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.937+0000 7f191a6b2b80 1 Processor -- start 2024-05-08T21:25:00.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f191a6b2b80 1 -- start start 2024-05-08T21:25:00.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f191a6b2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 0x55f2df783960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f191a6b2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783ea0 0x55f2df784320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f191a6b2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df784860 0x55f2df784ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f191a6b2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2df6da150 con 0x55f2df783ea0 2024-05-08T21:25:00.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f191a6b2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2df6d7b10 con 0x55f2df784860 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f191a6b2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2df6d3400 con 0x55f2df783540 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f1907fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 0x55f2df783960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f190ca95700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df784860 0x55f2df784ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f19077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783ea0 0x55f2df784320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f1907fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 0x55f2df783960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49782/0 (socket says 172.21.15.33:49782) 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f190ca95700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df784860 0x55f2df784ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56248/0 (socket says 172.21.15.33:56248) 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f1907fff700 1 -- 172.21.15.33:0/1269281754 learned_addr learned my addr 172.21.15.33:0/1269281754 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f190ca95700 1 -- 172.21.15.33:0/1269281754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 msgr2=0x55f2df783960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f190ca95700 1 --2- 172.21.15.33:0/1269281754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 0x55f2df783960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f190ca95700 1 -- 172.21.15.33:0/1269281754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783ea0 msgr2=0x55f2df784320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f190ca95700 1 --2- 172.21.15.33:0/1269281754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783ea0 0x55f2df784320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f190ca95700 1 -- 172.21.15.33:0/1269281754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2df5543b0 con 0x55f2df784860 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f1907fff700 1 --2- 172.21.15.33:0/1269281754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 0x55f2df783960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.938+0000 7f190ca95700 1 --2- 172.21.15.33:0/1269281754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df784860 0x55f2df784ce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f19000076e0 tx=0x7f19000079f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c0e499b8bcc89f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:00.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.939+0000 7f18ee7fc700 1 -- 172.21.15.33:0/1269281754 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900004bb0 con 0x55f2df784860 2024-05-08T21:25:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.939+0000 7f18ee7fc700 1 -- 172.21.15.33:0/1269281754 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f190000f040 con 0x55f2df784860 2024-05-08T21:25:00.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.939+0000 7f18ee7fc700 1 -- 172.21.15.33:0/1269281754 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900005220 con 0x55f2df784860 2024-05-08T21:25:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.940+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1269281754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df784860 msgr2=0x55f2df784ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.940+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1269281754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df784860 0x55f2df784ce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f19000076e0 tx=0x7f19000079f0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.940+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1269281754 shutdown_connections 2024-05-08T21:25:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.940+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1269281754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df784860 0x55f2df784ce0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.940+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1269281754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 0x55f2df783960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.940+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1269281754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783ea0 0x55f2df784320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.940+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1269281754 >> 172.21.15.33:0/1269281754 conn(0x55f2df6e1210 msgr2=0x55f2df77dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.941+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1269281754 shutdown_connections 2024-05-08T21:25:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.941+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1269281754 wait complete. 2024-05-08T21:25:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.941+0000 7f191a6b2b80 1 -- start start 2024-05-08T21:25:00.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.941+0000 7f191a6b2b80 1 Processor -- start 2024-05-08T21:25:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f191a6b2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783540 0x55f2df78cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f191a6b2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783ea0 0x55f2df78d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f191a6b2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df784860 0x55f2df794350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f191a6b2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2df66f560 con 0x55f2df783540 2024-05-08T21:25:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f191a6b2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2df66f200 con 0x55f2df783ea0 2024-05-08T21:25:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f191a6b2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2df5566a0 con 0x55f2df784860 2024-05-08T21:25:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f191a6b2b80 10 client.0 _ll_register_callbacks cb 0x55f2df678b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:00.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f1907fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783540 0x55f2df78cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f19077fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783ea0 0x55f2df78d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f1907fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783540 0x55f2df78cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43030/0 (socket says 172.21.15.33:43030) 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f1907fff700 1 -- 172.21.15.33:0/1856666513 learned_addr learned my addr 172.21.15.33:0/1856666513 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f190ca95700 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df784860 0x55f2df794350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f190ca95700 1 -- 172.21.15.33:0/1856666513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783ea0 msgr2=0x55f2df78d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f190ca95700 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783ea0 0x55f2df78d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f190ca95700 1 -- 172.21.15.33:0/1856666513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783540 msgr2=0x55f2df78cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f190ca95700 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783540 0x55f2df78cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f190ca95700 1 -- 172.21.15.33:0/1856666513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19081a1460 con 0x55f2df784860 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f19077fe700 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783ea0 0x55f2df78d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.942+0000 7f1907fff700 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783540 0x55f2df78cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.943+0000 7f190ca95700 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df784860 0x55f2df794350 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55f2df77ed40 tx=0x7f19000159c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772943538f62f31c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.943+0000 7f18e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:00.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.943+0000 7f18e77fe700 1 -- 172.21.15.33:0/1856666513 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900015bb0 con 0x55f2df784860 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.943+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1856666513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2df62c270 con 0x55f2df784860 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.943+0000 7f18e77fe700 1 -- 172.21.15.33:0/1856666513 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1900003a00 con 0x55f2df784860 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.943+0000 7f18e77fe700 1 -- 172.21.15.33:0/1856666513 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1900003bc0 con 0x55f2df784860 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.943+0000 7f191a6b2b80 10 client.27016 fetch_fsmap learned FSMap version 37 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.943+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1856666513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f1900009d80 con 0x55f2df784860 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.943+0000 7f18e77fe700 1 -- 172.21.15.33:0/1856666513 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f2df62c270 con 0x55f2df784860 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.944+0000 7f18e77fe700 1 -- 172.21.15.33:0/1856666513 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 37) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f1900004020 con 0x55f2df784860 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.944+0000 7f191a6b2b80 10 client.27016 fetch_fsmap finished waiting for FSMap version 37 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.944+0000 7f191a6b2b80 10 client.27016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.944+0000 7f191a6b2b80 20 client.27016 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.944+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f2df798aa0 0x55f2df79aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.944+0000 7f191a6b2b80 4 client.27016 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.944+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1856666513 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2df752810 con 0x55f2df798aa0 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.944+0000 7f1907fff700 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f2df798aa0 0x55f2df79aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.944+0000 7f18e77fe700 10 client.27016 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.944+0000 7f1907fff700 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f2df798aa0 0x55f2df79aec0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e32a6f4864bd97 server_cookie=da0a7ceee4f9da3b in_seq=0 out_seq=0 2024-05-08T21:25:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.945+0000 7f18e77fe700 1 -- 172.21.15.33:0/1856666513 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f2df752810 con 0x55f2df798aa0 2024-05-08T21:25:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.945+0000 7f18e77fe700 10 client.27016 handle_command_reply: tid=0 2024-05-08T21:25:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.945+0000 7f191a6b2b80 1 client.27016 shutdown 2024-05-08T21:25:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.945+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1856666513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df784860 msgr2=0x55f2df794350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:00.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.945+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df784860 0x55f2df794350 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55f2df77ed40 tx=0x7f19000159c0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.946+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1856666513 shutdown_connections 2024-05-08T21:25:00.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.946+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f2df798aa0 0x55f2df79aec0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.946+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df784860 0x55f2df794350 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.946+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783540 0x55f2df78cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.946+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1856666513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783ea0 0x55f2df78d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.946+0000 7f18e77fe700 0 client.27016 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:00.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.946+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1856666513 >> 172.21.15.33:0/1856666513 conn(0x55f2df6e1210 msgr2=0x55f2df77d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:00.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.946+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1856666513 shutdown_connections 2024-05-08T21:25:00.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.946+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1856666513 wait complete. 2024-05-08T21:25:00.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.946+0000 7f191a6b2b80 20 client.27016 trim_cache size 0 max 16384 2024-05-08T21:25:00.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f191a6b2b80 1 Processor -- start 2024-05-08T21:25:00.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f191a6b2b80 1 -- start start 2024-05-08T21:25:00.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f191a6b2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 0x55f2df78cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f191a6b2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783ea0 0x55f2df7942e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f191a6b2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df784860 0x55f2df78d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f191a6b2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2df78b710 con 0x55f2df784860 2024-05-08T21:25:00.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f191a6b2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2df7954d0 con 0x55f2df783ea0 2024-05-08T21:25:00.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f191a6b2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2df78c940 con 0x55f2df783540 2024-05-08T21:25:00.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f190ca95700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df784860 0x55f2df78d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f1907fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 0x55f2df78cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f190ca95700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df784860 0x55f2df78d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43034/0 (socket says 172.21.15.33:43034) 2024-05-08T21:25:00.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f190ca95700 1 -- 172.21.15.33:0/2642396851 learned_addr learned my addr 172.21.15.33:0/2642396851 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:00.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f1907fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 0x55f2df78cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49808/0 (socket says 172.21.15.33:49808) 2024-05-08T21:25:00.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.967+0000 7f19077fe700 1 --2- 172.21.15.33:0/2642396851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783ea0 0x55f2df7942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.968+0000 7f1907fff700 1 -- 172.21.15.33:0/2642396851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783ea0 msgr2=0x55f2df7942e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:00.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.968+0000 7f1907fff700 1 --2- 172.21.15.33:0/2642396851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783ea0 0x55f2df7942e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.968+0000 7f1907fff700 1 -- 172.21.15.33:0/2642396851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df784860 msgr2=0x55f2df78d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:00.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.968+0000 7f1907fff700 1 --2- 172.21.15.33:0/2642396851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df784860 0x55f2df78d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.968+0000 7f1907fff700 1 -- 172.21.15.33:0/2642396851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2df500b60 con 0x55f2df783540 2024-05-08T21:25:00.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.968+0000 7f190ca95700 1 --2- 172.21.15.33:0/2642396851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df784860 0x55f2df78d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:00.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.968+0000 7f1907fff700 1 --2- 172.21.15.33:0/2642396851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 0x55f2df78cb80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f18f8099df0 tx=0x7f18f801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ef877d23923cc2e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:00.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.968+0000 7f18eeffd700 1 -- 172.21.15.33:0/2642396851 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f8099f10 con 0x55f2df783540 2024-05-08T21:25:00.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.968+0000 7f18eeffd700 1 -- 172.21.15.33:0/2642396851 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18f8099810 con 0x55f2df783540 2024-05-08T21:25:00.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.968+0000 7f18eeffd700 1 -- 172.21.15.33:0/2642396851 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f80a1ee0 con 0x55f2df783540 2024-05-08T21:25:00.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.969+0000 7f191a6b2b80 1 -- 172.21.15.33:0/2642396851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 msgr2=0x55f2df78cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:00.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.969+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/2642396851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 0x55f2df78cb80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f18f8099df0 tx=0x7f18f801a040 comp rx=0 tx=0).stop 2024-05-08T21:25:00.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.969+0000 7f191a6b2b80 1 -- 172.21.15.33:0/2642396851 shutdown_connections 2024-05-08T21:25:00.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.969+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/2642396851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df784860 0x55f2df78d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.969+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/2642396851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df783540 0x55f2df78cb80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.969+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/2642396851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783ea0 0x55f2df7942e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.969+0000 7f191a6b2b80 1 -- 172.21.15.33:0/2642396851 >> 172.21.15.33:0/2642396851 conn(0x55f2df6e1210 msgr2=0x55f2df78adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.969+0000 7f191a6b2b80 1 -- 172.21.15.33:0/2642396851 shutdown_connections 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.969+0000 7f191a6b2b80 1 -- 172.21.15.33:0/2642396851 wait complete. 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.969+0000 7f191a6b2b80 1 -- start start 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f191a6b2b80 1 Processor -- start 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f191a6b2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783540 0x55f2df790440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f191a6b2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783ea0 0x55f2df790980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f191a6b2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df784860 0x55f2df793950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f191a6b2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2df793e90 con 0x55f2df783ea0 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f191a6b2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2df794000 con 0x55f2df783540 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f191a6b2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2df7941c0 con 0x55f2df784860 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f191a6b2b80 10 client.0 _ll_register_callbacks cb 0x55f2df678b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f1907fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783540 0x55f2df790440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f1907fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783540 0x55f2df790440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56278/0 (socket says 172.21.15.33:56278) 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f19077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783ea0 0x55f2df790980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f190ca95700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df784860 0x55f2df793950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.970+0000 7f1907fff700 1 -- 172.21.15.33:0/1407326918 learned_addr learned my addr 172.21.15.33:0/1407326918 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f190ca95700 1 -- 172.21.15.33:0/1407326918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783540 msgr2=0x55f2df790440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f190ca95700 1 --2- 172.21.15.33:0/1407326918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783540 0x55f2df790440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f190ca95700 1 -- 172.21.15.33:0/1407326918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783ea0 msgr2=0x55f2df790980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f190ca95700 1 --2- 172.21.15.33:0/1407326918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783ea0 0x55f2df790980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f190ca95700 1 -- 172.21.15.33:0/1407326918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1900009d80 con 0x55f2df784860 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f1907fff700 1 --2- 172.21.15.33:0/1407326918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783540 0x55f2df790440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f190ca95700 1 --2- 172.21.15.33:0/1407326918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df784860 0x55f2df793950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1900009160 tx=0x7f1900014f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d91dffbd1c964e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f18e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f18e77fe700 1 -- 172.21.15.33:0/1407326918 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000046a0 con 0x55f2df784860 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f18e77fe700 1 -- 172.21.15.33:0/1407326918 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f19000048e0 con 0x55f2df784860 2024-05-08T21:25:00.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1407326918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f190000f040 con 0x55f2df784860 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f18e77fe700 1 -- 172.21.15.33:0/1407326918 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19000143a0 con 0x55f2df784860 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f191a6b2b80 10 client.27022 fetch_fsmap learned FSMap version 37 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.971+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1407326918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f19081a0fe0 con 0x55f2df784860 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.972+0000 7f18e77fe700 1 -- 172.21.15.33:0/1407326918 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f190000f040 con 0x55f2df784860 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.972+0000 7f18e77fe700 1 -- 172.21.15.33:0/1407326918 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 37) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f190003a070 con 0x55f2df784860 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.972+0000 7f191a6b2b80 10 client.27022 fetch_fsmap finished waiting for FSMap version 37 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.972+0000 7f191a6b2b80 10 client.27022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.972+0000 7f191a6b2b80 20 client.27022 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.972+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1407326918 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f2df7efdf0 0x55f2df7f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.972+0000 7f191a6b2b80 4 client.27022 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.972+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1407326918 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f2df6cbec0 con 0x55f2df7efdf0 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.972+0000 7f1907fff700 1 --2- 172.21.15.33:0/1407326918 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f2df7efdf0 0x55f2df7f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.972+0000 7f18e77fe700 10 client.27022 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.972+0000 7f1907fff700 1 --2- 172.21.15.33:0/1407326918 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f2df7efdf0 0x55f2df7f2210 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=356228fb2959039c server_cookie=278422e58a0e32ac in_seq=0 out_seq=0 2024-05-08T21:25:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f18e77fe700 1 -- 172.21.15.33:0/1407326918 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f2df6cbec0 con 0x55f2df7efdf0 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f18e77fe700 10 client.27022 handle_command_reply: tid=0 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f191a6b2b80 1 client.27022 shutdown 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1407326918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df784860 msgr2=0x55f2df793950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1407326918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df784860 0x55f2df793950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1900009160 tx=0x7f1900014f60 comp rx=0 tx=0).stop 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1407326918 shutdown_connections 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1407326918 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55f2df7efdf0 0x55f2df7f2210 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1407326918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2df784860 0x55f2df793950 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1407326918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f2df783540 0x55f2df790440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f191a6b2b80 1 --2- 172.21.15.33:0/1407326918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f2df783ea0 0x55f2df790980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1407326918 >> 172.21.15.33:0/1407326918 conn(0x55f2df6e1210 msgr2=0x55f2df78cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f18e77fe700 0 client.27022 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1407326918 shutdown_connections 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f191a6b2b80 1 -- 172.21.15.33:0/1407326918 wait complete. 2024-05-08T21:25:00.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.973+0000 7f191a6b2b80 20 client.27022 trim_cache size 0 max 16384 2024-05-08T21:25:00.981 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:25:00.981 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:25:00.981 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:25:00.981 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:25:00.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.974+0000 7f190e4f8700 1 -- 172.21.15.33:0/2769031141 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f18f00a1120 msgr2=0x7f18f00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:00.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.974+0000 7f190e4f8700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f18f00a1120 0x7f18f00a35e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f18fc006010 tx=0x7f18fc00b610 comp rx=0 tx=0).stop 2024-05-08T21:25:00.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.974+0000 7f190e4f8700 1 -- 172.21.15.33:0/2769031141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1908101900 msgr2=0x7f1908101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:00.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.974+0000 7f190e4f8700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1908101900 0x7f1908101220 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f18f800e9a0 tx=0x7f18f800ecb0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.974+0000 7f190e4f8700 1 -- 172.21.15.33:0/2769031141 shutdown_connections 2024-05-08T21:25:00.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.974+0000 7f190e4f8700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f18f00a1120 0x7f18f00a35e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.974+0000 7f190e4f8700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1908106b60 0x7f19080ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.974+0000 7f190e4f8700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1908101900 0x7f1908101220 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.974+0000 7f190e4f8700 1 --2- 172.21.15.33:0/2769031141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1908104230 0x7f19080ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:00.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.974+0000 7f190e4f8700 1 -- 172.21.15.33:0/2769031141 >> 172.21.15.33:0/2769031141 conn(0x7f19080fb310 msgr2=0x7f1908103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:00.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.974+0000 7f190e4f8700 1 -- 172.21.15.33:0/2769031141 shutdown_connections 2024-05-08T21:25:00.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:00.974+0000 7f190e4f8700 1 -- 172.21.15.33:0/2769031141 wait complete. 2024-05-08T21:25:00.992 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:25:00.992 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:25:01.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.154+0000 7f772994f700 1 -- 172.21.15.33:0/1816672724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7724101fd0 msgr2=0x7f7724102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.154+0000 7f772994f700 1 --2- 172.21.15.33:0/1816672724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7724101fd0 0x7f7724102450 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7718009990 tx=0x7f7718009ca0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.155+0000 7f772994f700 1 -- 172.21.15.33:0/1816672724 shutdown_connections 2024-05-08T21:25:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.155+0000 7f772994f700 1 --2- 172.21.15.33:0/1816672724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7724102990 0x7f77241092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.155+0000 7f772994f700 1 --2- 172.21.15.33:0/1816672724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7724100e80 0x7f77241012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.155+0000 7f772994f700 1 --2- 172.21.15.33:0/1816672724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7724101fd0 0x7f7724102450 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.155+0000 7f772994f700 1 -- 172.21.15.33:0/1816672724 >> 172.21.15.33:0/1816672724 conn(0x7f77240fc410 msgr2=0x7f77240fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.155+0000 7f772994f700 1 -- 172.21.15.33:0/1816672724 shutdown_connections 2024-05-08T21:25:01.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.155+0000 7f772994f700 1 -- 172.21.15.33:0/1816672724 wait complete. 2024-05-08T21:25:01.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.156+0000 7f772994f700 1 Processor -- start 2024-05-08T21:25:01.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.156+0000 7f772994f700 1 -- start start 2024-05-08T21:25:01.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.156+0000 7f772994f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7724100e80 0x7f772419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.156+0000 7f772994f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7724101fd0 0x7f772419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.156+0000 7f772994f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7724102990 0x7f77241a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.156+0000 7f772994f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77241a4e60 con 0x7f7724102990 2024-05-08T21:25:01.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.156+0000 7f772994f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772406a9c0 con 0x7f7724101fd0 2024-05-08T21:25:01.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.156+0000 7f772994f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772406ab50 con 0x7f7724100e80 2024-05-08T21:25:01.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.156+0000 7f77227fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7724101fd0 0x7f772419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.156+0000 7f77227fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7724101fd0 0x7f772419d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56292/0 (socket says 172.21.15.33:56292) 2024-05-08T21:25:01.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.156+0000 7f77227fc700 1 -- 172.21.15.33:0/826556690 learned_addr learned my addr 172.21.15.33:0/826556690 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:01.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f7722ffd700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7724100e80 0x7f772419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f77237fe700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7724102990 0x7f77241a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f77227fc700 1 -- 172.21.15.33:0/826556690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7724100e80 msgr2=0x7f772419d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f77227fc700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7724100e80 0x7f772419d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f77227fc700 1 -- 172.21.15.33:0/826556690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7724102990 msgr2=0x7f77241a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f77227fc700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7724102990 0x7f77241a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f77227fc700 1 -- 172.21.15.33:0/826556690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180095f0 con 0x7f7724101fd0 2024-05-08T21:25:01.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f7722ffd700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7724100e80 0x7f772419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:25:01.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f77237fe700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7724102990 0x7f77241a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:25:01.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f77227fc700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7724101fd0 0x7f772419d8c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7718006010 tx=0x7f7718009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f772894d700 1 -- 172.21.15.33:0/826556690 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771801d070 con 0x7f7724101fd0 2024-05-08T21:25:01.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f772894d700 1 -- 172.21.15.33:0/826556690 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f771800bed0 con 0x7f7724101fd0 2024-05-08T21:25:01.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f772894d700 1 -- 172.21.15.33:0/826556690 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718021410 con 0x7f7724101fd0 2024-05-08T21:25:01.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.157+0000 7f772994f700 1 -- 172.21.15.33:0/826556690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772406add0 con 0x7f7724101fd0 2024-05-08T21:25:01.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.158+0000 7f772994f700 1 -- 172.21.15.33:0/826556690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772406b280 con 0x7f7724101fd0 2024-05-08T21:25:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.159+0000 7f772994f700 1 -- 172.21.15.33:0/826556690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77241063d0 con 0x7f7724101fd0 2024-05-08T21:25:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.160+0000 7f772894d700 1 -- 172.21.15.33:0/826556690 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f771800f460 con 0x7f7724101fd0 2024-05-08T21:25:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.161+0000 7f772894d700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f770c0a11a0 0x7f770c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.161+0000 7f772894d700 1 -- 172.21.15.33:0/826556690 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f77180cdfc0 con 0x7f7724101fd0 2024-05-08T21:25:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.161+0000 7f7722ffd700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f770c0a11a0 0x7f770c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.161+0000 7f7722ffd700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f770c0a11a0 0x7f770c0a3660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7714009cb0 tx=0x7f7714009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.165+0000 7f772894d700 1 -- 172.21.15.33:0/826556690 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7718097460 con 0x7f7724101fd0 2024-05-08T21:25:01.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.500+0000 7f772994f700 1 -- 172.21.15.33:0/826556690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7724068160 con 0x7f7724101fd0 2024-05-08T21:25:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.501+0000 7f772894d700 1 -- 172.21.15.33:0/826556690 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+10903 (secure 0 0 0) 0x7f771809aa80 con 0x7f7724101fd0 2024-05-08T21:25:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:25:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:24:53.332067+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:25:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.503+0000 7f772994f700 1 -- 172.21.15.33:0/826556690 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f770c0a11a0 msgr2=0x7f770c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.503+0000 7f772994f700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f770c0a11a0 0x7f770c0a3660 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7714009cb0 tx=0x7f7714009260 comp rx=0 tx=0).stop 2024-05-08T21:25:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.503+0000 7f772994f700 1 -- 172.21.15.33:0/826556690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7724101fd0 msgr2=0x7f772419d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.503+0000 7f772994f700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7724101fd0 0x7f772419d8c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7718006010 tx=0x7f7718009510 comp rx=0 tx=0).stop 2024-05-08T21:25:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.503+0000 7f772994f700 1 -- 172.21.15.33:0/826556690 shutdown_connections 2024-05-08T21:25:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.503+0000 7f772994f700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7724102990 0x7f77241a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.503+0000 7f772994f700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7724100e80 0x7f772419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.503+0000 7f772994f700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f770c0a11a0 0x7f770c0a3660 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.503+0000 7f772994f700 1 --2- 172.21.15.33:0/826556690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7724101fd0 0x7f772419d8c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.503+0000 7f772994f700 1 -- 172.21.15.33:0/826556690 >> 172.21.15.33:0/826556690 conn(0x7f77240fc410 msgr2=0x7f7724103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.503+0000 7f772994f700 1 -- 172.21.15.33:0/826556690 shutdown_connections 2024-05-08T21:25:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.503+0000 7f772994f700 1 -- 172.21.15.33:0/826556690 wait complete. 2024-05-08T21:25:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 37 2024-05-08T21:25:01.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:25:01.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.682+0000 7f5e060a3700 1 -- 172.21.15.33:0/244281695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e00101960 msgr2=0x7f5e00108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.682+0000 7f5e060a3700 1 --2- 172.21.15.33:0/244281695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e00101960 0x7f5e00108210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5df800b0d0 tx=0x7f5df800b3e0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.682+0000 7f5e060a3700 1 -- 172.21.15.33:0/244281695 shutdown_connections 2024-05-08T21:25:01.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.682+0000 7f5e060a3700 1 --2- 172.21.15.33:0/244281695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e00101960 0x7f5e00108210 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.682+0000 7f5e060a3700 1 --2- 172.21.15.33:0/244281695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e000ffdd0 0x7f5e001001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.682+0000 7f5e060a3700 1 --2- 172.21.15.33:0/244281695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e00100fc0 0x7f5e00101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.682+0000 7f5e060a3700 1 -- 172.21.15.33:0/244281695 >> 172.21.15.33:0/244281695 conn(0x7f5e000fb340 msgr2=0x7f5e000fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.682+0000 7f5e060a3700 1 -- 172.21.15.33:0/244281695 shutdown_connections 2024-05-08T21:25:01.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.682+0000 7f5e060a3700 1 -- 172.21.15.33:0/244281695 wait complete. 2024-05-08T21:25:01.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.683+0000 7f5e060a3700 1 Processor -- start 2024-05-08T21:25:01.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.683+0000 7f5e060a3700 1 -- start start 2024-05-08T21:25:01.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.683+0000 7f5e060a3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e000ffdd0 0x7f5e00199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.683+0000 7f5e060a3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e00100fc0 0x7f5e001995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.683+0000 7f5e060a3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e00101960 0x7f5e001a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.683+0000 7f5e060a3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e001a0b90 con 0x7f5e00100fc0 2024-05-08T21:25:01.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.683+0000 7f5e060a3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e001a0cd0 con 0x7f5e00101960 2024-05-08T21:25:01.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.683+0000 7f5e060a3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e001a0e60 con 0x7f5e000ffdd0 2024-05-08T21:25:01.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dff7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e000ffdd0 0x7f5e00199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dfeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e00100fc0 0x7f5e001995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dff7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e000ffdd0 0x7f5e00199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49864/0 (socket says 172.21.15.33:49864) 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dff7fe700 1 -- 172.21.15.33:0/2268550065 learned_addr learned my addr 172.21.15.33:0/2268550065 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dfffff700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e00101960 0x7f5e001a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dff7fe700 1 -- 172.21.15.33:0/2268550065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e00101960 msgr2=0x7f5e001a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dff7fe700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e00101960 0x7f5e001a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dff7fe700 1 -- 172.21.15.33:0/2268550065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e00100fc0 msgr2=0x7f5e001995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dff7fe700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e00100fc0 0x7f5e001995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dff7fe700 1 -- 172.21.15.33:0/2268550065 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df8009d00 con 0x7f5e000ffdd0 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dff7fe700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e000ffdd0 0x7f5e00199060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5df000e970 tx=0x7f5df000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dfcff9700 1 -- 172.21.15.33:0/2268550065 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0009790 con 0x7f5e000ffdd0 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5e060a3700 1 -- 172.21.15.33:0/2268550065 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e001a1140 con 0x7f5e000ffdd0 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dfcff9700 1 -- 172.21.15.33:0/2268550065 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5df000c7e0 con 0x7f5e000ffdd0 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5dfcff9700 1 -- 172.21.15.33:0/2268550065 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0003bd0 con 0x7f5e000ffdd0 2024-05-08T21:25:01.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.684+0000 7f5e060a3700 1 -- 172.21.15.33:0/2268550065 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e001a16a0 con 0x7f5e000ffdd0 2024-05-08T21:25:01.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.687+0000 7f5dfcff9700 1 -- 172.21.15.33:0/2268550065 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5df000c950 con 0x7f5e000ffdd0 2024-05-08T21:25:01.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.688+0000 7f5dfcff9700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5de80a1120 0x7f5de80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.688+0000 7f5dfcff9700 1 -- 172.21.15.33:0/2268550065 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f5df0014070 con 0x7f5e000ffdd0 2024-05-08T21:25:01.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.688+0000 7f5dfeffd700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5de80a1120 0x7f5de80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5e1225db80 1 Processor -- start 2024-05-08T21:25:01.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5dfeffd700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5de80a1120 0x7f5de80a35e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009c60 tx=0x7f5df4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5e1225db80 1 -- start start 2024-05-08T21:25:01.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5e1225db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03800d50 0x562d03801170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5e1225db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d038016b0 0x562d03801b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5e1225db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d03802070 0x562d038024f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5e1225db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0374f150 con 0x562d03800d50 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5e1225db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0374cb10 con 0x562d03802070 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5e1225db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d03748400 con 0x562d038016b0 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5dfeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d038016b0 0x562d03801b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5dfffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d03802070 0x562d038024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5dfeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d038016b0 0x562d03801b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49866/0 (socket says 172.21.15.33:49866) 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5dfeffd700 1 -- 172.21.15.33:0/3518100412 learned_addr learned my addr 172.21.15.33:0/3518100412 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.689+0000 7f5dff7fe700 1 --2- 172.21.15.33:0/3518100412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03800d50 0x562d03801170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.690+0000 7f5dfeffd700 1 -- 172.21.15.33:0/3518100412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d03802070 msgr2=0x562d038024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.690+0000 7f5dfeffd700 1 --2- 172.21.15.33:0/3518100412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d03802070 0x562d038024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.690+0000 7f5dfeffd700 1 -- 172.21.15.33:0/3518100412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03800d50 msgr2=0x562d03801170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.690+0000 7f5dfeffd700 1 --2- 172.21.15.33:0/3518100412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03800d50 0x562d03801170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.690+0000 7f5dfeffd700 1 -- 172.21.15.33:0/3518100412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d035c93b0 con 0x562d038016b0 2024-05-08T21:25:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.690+0000 7f5dfffff700 1 --2- 172.21.15.33:0/3518100412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d03802070 0x562d038024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.690+0000 7f5dfeffd700 1 --2- 172.21.15.33:0/3518100412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d038016b0 0x562d03801b30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5df400e510 tx=0x7f5df4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e460d9839350e6e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.690+0000 7f5de5ffb700 1 -- 172.21.15.33:0/3518100412 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df400ee30 con 0x562d038016b0 2024-05-08T21:25:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.690+0000 7f5de5ffb700 1 -- 172.21.15.33:0/3518100412 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5df4021070 con 0x562d038016b0 2024-05-08T21:25:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.690+0000 7f5de5ffb700 1 -- 172.21.15.33:0/3518100412 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4025700 con 0x562d038016b0 2024-05-08T21:25:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.691+0000 7f5e1225db80 1 -- 172.21.15.33:0/3518100412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d038016b0 msgr2=0x562d03801b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.691+0000 7f5e1225db80 1 --2- 172.21.15.33:0/3518100412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d038016b0 0x562d03801b30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5df400e510 tx=0x7f5df4014000 comp rx=0 tx=0).stop 2024-05-08T21:25:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.691+0000 7f5e1225db80 1 -- 172.21.15.33:0/3518100412 shutdown_connections 2024-05-08T21:25:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.691+0000 7f5e1225db80 1 --2- 172.21.15.33:0/3518100412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d03802070 0x562d038024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.691+0000 7f5e1225db80 1 --2- 172.21.15.33:0/3518100412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03800d50 0x562d03801170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.691+0000 7f5e1225db80 1 --2- 172.21.15.33:0/3518100412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d038016b0 0x562d03801b30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.691+0000 7f5e1225db80 1 -- 172.21.15.33:0/3518100412 >> 172.21.15.33:0/3518100412 conn(0x562d03756210 msgr2=0x562d037f2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.691+0000 7f5e1225db80 1 -- 172.21.15.33:0/3518100412 shutdown_connections 2024-05-08T21:25:01.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.691+0000 7f5e1225db80 1 -- 172.21.15.33:0/3518100412 wait complete. 2024-05-08T21:25:01.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.691+0000 7f5e1225db80 1 -- start start 2024-05-08T21:25:01.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5e1225db80 1 Processor -- start 2024-05-08T21:25:01.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5e1225db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d03800d50 0x562d037f9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5e1225db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d038016b0 0x562d037fa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5e1225db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 0x562d037ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5e1225db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d036e4560 con 0x562d038016b0 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5e1225db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d036e4200 con 0x562d03800d50 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5e1225db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d035cb6a0 con 0x562d03802070 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5e1225db80 10 client.0 _ll_register_callbacks cb 0x562d036edb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5dfffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 0x562d037ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5dfffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 0x562d037ff0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49876/0 (socket says 172.21.15.33:49876) 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5dfffff700 1 -- 172.21.15.33:0/1171305609 learned_addr learned my addr 172.21.15.33:0/1171305609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5dfeffd700 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d038016b0 0x562d037fa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.692+0000 7f5dff7fe700 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d03800d50 0x562d037f9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.693+0000 7f5dfffff700 1 -- 172.21.15.33:0/1171305609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d03800d50 msgr2=0x562d037f9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.693+0000 7f5dfffff700 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d03800d50 0x562d037f9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.693+0000 7f5dfffff700 1 -- 172.21.15.33:0/1171305609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d038016b0 msgr2=0x562d037fa0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.693+0000 7f5dfffff700 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d038016b0 0x562d037fa0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.693+0000 7f5dfffff700 1 -- 172.21.15.33:0/1171305609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e00108b60 con 0x562d03802070 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.693+0000 7f5dfffff700 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 0x562d037ff0e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5df800b0d0 tx=0x7f5df8007850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5586a56c9f93b530 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.693+0000 7f5ddeffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.693+0000 7f5ddeffd700 1 -- 172.21.15.33:0/1171305609 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df800e040 con 0x562d03802070 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.693+0000 7f5ddeffd700 1 -- 172.21.15.33:0/1171305609 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f5df800bca0 con 0x562d03802070 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.693+0000 7f5e1225db80 1 -- 172.21.15.33:0/1171305609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d036a1270 con 0x562d03802070 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.694+0000 7f5ddeffd700 1 -- 172.21.15.33:0/1171305609 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5df800be60 con 0x562d03802070 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.694+0000 7f5e1225db80 10 client.27040 fetch_fsmap learned FSMap version 37 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.694+0000 7f5e1225db80 1 -- 172.21.15.33:0/1171305609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f5df4017070 con 0x562d03802070 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.694+0000 7f5ddeffd700 1 -- 172.21.15.33:0/1171305609 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562d036a1270 con 0x562d03802070 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.694+0000 7f5ddeffd700 1 -- 172.21.15.33:0/1171305609 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 37) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5df800b100 con 0x562d03802070 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.694+0000 7f5e1225db80 10 client.27040 fetch_fsmap finished waiting for FSMap version 37 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.694+0000 7f5e1225db80 10 client.27040 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.694+0000 7f5e1225db80 20 client.27040 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:01.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.694+0000 7f5e1225db80 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x562d0381eb90 0x562d03820fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.694+0000 7f5e1225db80 4 client.27040 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.694+0000 7f5e1225db80 1 -- 172.21.15.33:0/1171305609 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d037c7810 con 0x562d0381eb90 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.694+0000 7f5dff7fe700 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x562d0381eb90 0x562d03820fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.695+0000 7f5ddeffd700 10 client.27040 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.695+0000 7f5dff7fe700 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x562d0381eb90 0x562d03820fb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c8149888e0059053 server_cookie=2d69cd094481917b in_seq=0 out_seq=0 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.695+0000 7f5ddeffd700 1 -- 172.21.15.33:0/1171305609 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d037c7810 con 0x562d0381eb90 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.695+0000 7f5ddeffd700 10 client.27040 handle_command_reply: tid=0 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.695+0000 7f5e1225db80 1 client.27040 shutdown 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.695+0000 7f5e1225db80 1 -- 172.21.15.33:0/1171305609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 msgr2=0x562d037ff0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.695+0000 7f5e1225db80 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 0x562d037ff0e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5df800b0d0 tx=0x7f5df8007850 comp rx=0 tx=0).stop 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.696+0000 7f5e1225db80 1 -- 172.21.15.33:0/1171305609 shutdown_connections 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.696+0000 7f5e1225db80 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x562d0381eb90 0x562d03820fb0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.696+0000 7f5e1225db80 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 0x562d037ff0e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.696+0000 7f5ddeffd700 0 client.27040 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:25:01.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.696+0000 7f5e1225db80 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d03800d50 0x562d037f9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.696+0000 7f5e1225db80 1 --2- 172.21.15.33:0/1171305609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d038016b0 0x562d037fa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.696+0000 7f5e1225db80 1 -- 172.21.15.33:0/1171305609 >> 172.21.15.33:0/1171305609 conn(0x562d03756210 msgr2=0x562d037f25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.696+0000 7f5e1225db80 1 -- 172.21.15.33:0/1171305609 shutdown_connections 2024-05-08T21:25:01.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.696+0000 7f5e1225db80 1 -- 172.21.15.33:0/1171305609 wait complete. 2024-05-08T21:25:01.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.696+0000 7f5e1225db80 20 client.27040 trim_cache size 0 max 16384 2024-05-08T21:25:01.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:01 smithi033 ceph-mon[42528]: pgmap v471: 129 pgs: 129 active+clean; 219 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.3 MiB/s wr, 490 op/s 2024-05-08T21:25:01.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.715+0000 7f5e1225db80 1 Processor -- start 2024-05-08T21:25:01.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.715+0000 7f5e1225db80 1 -- start start 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5e1225db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03800d50 0x562d03820f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5e1225db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d038016b0 0x562d03801130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5e1225db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 0x562d037ff790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5e1225db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d037f3a20 con 0x562d03800d50 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5e1225db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d037f4570 con 0x562d038016b0 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5e1225db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d03826180 con 0x562d03802070 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03800d50 0x562d03820f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dfffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 0x562d037ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03800d50 0x562d03820f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43126/0 (socket says 172.21.15.33:43126) 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dff7fe700 1 -- 172.21.15.33:0/3218691670 learned_addr learned my addr 172.21.15.33:0/3218691670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dfffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 0x562d037ff790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49888/0 (socket says 172.21.15.33:49888) 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dfeffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d038016b0 0x562d03801130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dfffff700 1 -- 172.21.15.33:0/3218691670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d038016b0 msgr2=0x562d03801130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dfffff700 1 --2- 172.21.15.33:0/3218691670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d038016b0 0x562d03801130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dfffff700 1 -- 172.21.15.33:0/3218691670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03800d50 msgr2=0x562d03820f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dfffff700 1 --2- 172.21.15.33:0/3218691670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03800d50 0x562d03820f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dfffff700 1 -- 172.21.15.33:0/3218691670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d03575b60 con 0x562d03802070 2024-05-08T21:25:01.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dfeffd700 1 --2- 172.21.15.33:0/3218691670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d038016b0 0x562d03801130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:01.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.716+0000 7f5dfffff700 1 --2- 172.21.15.33:0/3218691670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 0x562d037ff790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009bc0 tx=0x7f5df8005670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83625ffed1914d60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.717+0000 7f5de67fc700 1 -- 172.21.15.33:0/3218691670 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df800e040 con 0x562d03802070 2024-05-08T21:25:01.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.717+0000 7f5de67fc700 1 -- 172.21.15.33:0/3218691670 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5df8005840 con 0x562d03802070 2024-05-08T21:25:01.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.717+0000 7f5de67fc700 1 -- 172.21.15.33:0/3218691670 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df800baa0 con 0x562d03802070 2024-05-08T21:25:01.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.717+0000 7f5e1225db80 1 -- 172.21.15.33:0/3218691670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 msgr2=0x562d037ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.717+0000 7f5e1225db80 1 --2- 172.21.15.33:0/3218691670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 0x562d037ff790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009bc0 tx=0x7f5df8005670 comp rx=0 tx=0).stop 2024-05-08T21:25:01.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.717+0000 7f5e1225db80 1 -- 172.21.15.33:0/3218691670 shutdown_connections 2024-05-08T21:25:01.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.717+0000 7f5e1225db80 1 --2- 172.21.15.33:0/3218691670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03802070 0x562d037ff790 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.717+0000 7f5e1225db80 1 --2- 172.21.15.33:0/3218691670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03800d50 0x562d03820f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.717+0000 7f5e1225db80 1 --2- 172.21.15.33:0/3218691670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d038016b0 0x562d03801130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.717+0000 7f5e1225db80 1 -- 172.21.15.33:0/3218691670 >> 172.21.15.33:0/3218691670 conn(0x562d03756210 msgr2=0x562d03802450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.718+0000 7f5e1225db80 1 -- 172.21.15.33:0/3218691670 shutdown_connections 2024-05-08T21:25:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.718+0000 7f5e1225db80 1 -- 172.21.15.33:0/3218691670 wait complete. 2024-05-08T21:25:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.718+0000 7f5e1225db80 1 -- start start 2024-05-08T21:25:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.718+0000 7f5e1225db80 1 Processor -- start 2024-05-08T21:25:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.718+0000 7f5e1225db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03800d50 0x562d0382c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.718+0000 7f5e1225db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d038016b0 0x562d0382c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.718+0000 7f5e1225db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03802070 0x562d03873ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.718+0000 7f5e1225db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0382f560 con 0x562d03802070 2024-05-08T21:25:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.718+0000 7f5e1225db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d0382f6d0 con 0x562d038016b0 2024-05-08T21:25:01.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.718+0000 7f5e1225db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d03874010 con 0x562d03800d50 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5e1225db80 10 client.0 _ll_register_callbacks cb 0x562d036edb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5dff7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03800d50 0x562d0382c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5dff7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03800d50 0x562d0382c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49892/0 (socket says 172.21.15.33:49892) 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5dfeffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d038016b0 0x562d0382c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5dff7fe700 1 -- 172.21.15.33:0/48833209 learned_addr learned my addr 172.21.15.33:0/48833209 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5dfffff700 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03802070 0x562d03873ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5dff7fe700 1 -- 172.21.15.33:0/48833209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d038016b0 msgr2=0x562d0382c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5dff7fe700 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d038016b0 0x562d0382c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5dff7fe700 1 -- 172.21.15.33:0/48833209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03802070 msgr2=0x562d03873ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5dff7fe700 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03802070 0x562d03873ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5dff7fe700 1 -- 172.21.15.33:0/48833209 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df402a040 con 0x562d03800d50 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5dfffff700 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03802070 0x562d03873ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5dff7fe700 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03800d50 0x562d0382c050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5df009c010 tx=0x7f5df0005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f019f69d30b768cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5ddeffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:01.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5ddeffd700 1 -- 172.21.15.33:0/48833209 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df009dd80 con 0x562d03800d50 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5ddeffd700 1 -- 172.21.15.33:0/48833209 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f5df00a23d0 con 0x562d03800d50 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5e1225db80 1 -- 172.21.15.33:0/48833209 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5df4021070 con 0x562d03800d50 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.719+0000 7f5ddeffd700 1 -- 172.21.15.33:0/48833209 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5df0096660 con 0x562d03800d50 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.720+0000 7f5e1225db80 10 client.27046 fetch_fsmap learned FSMap version 37 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.720+0000 7f5e1225db80 1 -- 172.21.15.33:0/48833209 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f5df4017070 con 0x562d03800d50 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.720+0000 7f5ddeffd700 1 -- 172.21.15.33:0/48833209 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5df4021070 con 0x562d03800d50 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.720+0000 7f5ddeffd700 1 -- 172.21.15.33:0/48833209 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 37) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5df0099560 con 0x562d03800d50 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.720+0000 7f5e1225db80 10 client.27046 fetch_fsmap finished waiting for FSMap version 37 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.720+0000 7f5e1225db80 10 client.27046 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.720+0000 7f5e1225db80 20 client.27046 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.720+0000 7f5e1225db80 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x562d03875aa0 0x562d03877ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.720+0000 7f5e1225db80 4 client.27046 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.720+0000 7f5e1225db80 1 -- 172.21.15.33:0/48833209 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562d03740ec0 con 0x562d03875aa0 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.720+0000 7f5dfeffd700 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x562d03875aa0 0x562d03877ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5dfeffd700 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x562d03875aa0 0x562d03877ec0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9497ebf8b1e6349 server_cookie=8e809d7d01c6994 in_seq=0 out_seq=0 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5ddeffd700 10 client.27046 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5ddeffd700 1 -- 172.21.15.33:0/48833209 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562d03740ec0 con 0x562d03875aa0 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5ddeffd700 10 client.27046 handle_command_reply: tid=0 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5e1225db80 1 client.27046 shutdown 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5e1225db80 1 -- 172.21.15.33:0/48833209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03800d50 msgr2=0x562d0382c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5e1225db80 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03800d50 0x562d0382c050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5df009c010 tx=0x7f5df0005910 comp rx=0 tx=0).stop 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5e1225db80 1 -- 172.21.15.33:0/48833209 shutdown_connections 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5e1225db80 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562d03802070 0x562d03873ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5e1225db80 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x562d03875aa0 0x562d03877ec0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5e1225db80 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d03800d50 0x562d0382c050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5e1225db80 1 --2- 172.21.15.33:0/48833209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d038016b0 0x562d0382c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5ddeffd700 0 client.27046 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5e1225db80 1 -- 172.21.15.33:0/48833209 >> 172.21.15.33:0/48833209 conn(0x562d03756210 msgr2=0x562d037ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5e1225db80 1 -- 172.21.15.33:0/48833209 shutdown_connections 2024-05-08T21:25:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5e1225db80 1 -- 172.21.15.33:0/48833209 wait complete. 2024-05-08T21:25:01.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.721+0000 7f5e1225db80 20 client.27046 trim_cache size 0 max 16384 2024-05-08T21:25:01.730 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:25:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.722+0000 7f5e060a3700 1 -- 172.21.15.33:0/2268550065 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5de80a1120 msgr2=0x7f5de80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.722+0000 7f5e060a3700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5de80a1120 0x7f5de80a35e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009c60 tx=0x7f5df4009260 comp rx=0 tx=0).stop 2024-05-08T21:25:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.722+0000 7f5e060a3700 1 -- 172.21.15.33:0/2268550065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e000ffdd0 msgr2=0x7f5e00199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.722+0000 7f5e060a3700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e000ffdd0 0x7f5e00199060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5df000e970 tx=0x7f5df000ec80 comp rx=0 tx=0).stop 2024-05-08T21:25:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.722+0000 7f5e060a3700 1 -- 172.21.15.33:0/2268550065 shutdown_connections 2024-05-08T21:25:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.722+0000 7f5e060a3700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5de80a1120 0x7f5de80a35e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.722+0000 7f5e060a3700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e00101960 0x7f5e001a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.722+0000 7f5e060a3700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e000ffdd0 0x7f5e00199060 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.722+0000 7f5e060a3700 1 --2- 172.21.15.33:0/2268550065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e00100fc0 0x7f5e001995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.722+0000 7f5e060a3700 1 -- 172.21.15.33:0/2268550065 >> 172.21.15.33:0/2268550065 conn(0x7f5e000fb340 msgr2=0x7f5e00069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.722+0000 7f5e060a3700 1 -- 172.21.15.33:0/2268550065 shutdown_connections 2024-05-08T21:25:01.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.722+0000 7f5e060a3700 1 -- 172.21.15.33:0/2268550065 wait complete. 2024-05-08T21:25:01.741 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:25:01.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:01 smithi071 ceph-mon[47074]: pgmap v471: 129 pgs: 129 active+clean; 219 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.3 MiB/s wr, 490 op/s 2024-05-08T21:25:01.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:01 smithi082 ceph-mon[47512]: pgmap v471: 129 pgs: 129 active+clean; 219 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.3 MiB/s wr, 490 op/s 2024-05-08T21:25:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.906+0000 7febbef65700 1 -- 172.21.15.33:0/4102076205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febb81018f0 msgr2=0x7febb8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.906+0000 7febbef65700 1 --2- 172.21.15.33:0/4102076205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febb81018f0 0x7febb8103ce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7febac0098f0 tx=0x7febac009c00 comp rx=0 tx=0).stop 2024-05-08T21:25:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.906+0000 7febbef65700 1 -- 172.21.15.33:0/4102076205 shutdown_connections 2024-05-08T21:25:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.906+0000 7febbef65700 1 --2- 172.21.15.33:0/4102076205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febb8106b50 0x7febb8108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.906+0000 7febbef65700 1 --2- 172.21.15.33:0/4102076205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febb81018f0 0x7febb8103ce0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.906+0000 7febbef65700 1 --2- 172.21.15.33:0/4102076205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febb8104220 0x7febb8106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.906+0000 7febbef65700 1 -- 172.21.15.33:0/4102076205 >> 172.21.15.33:0/4102076205 conn(0x7febb80fb300 msgr2=0x7febb80fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.906+0000 7febbef65700 1 -- 172.21.15.33:0/4102076205 shutdown_connections 2024-05-08T21:25:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.906+0000 7febbef65700 1 -- 172.21.15.33:0/4102076205 wait complete. 2024-05-08T21:25:01.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febbef65700 1 Processor -- start 2024-05-08T21:25:01.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febbef65700 1 -- start start 2024-05-08T21:25:01.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febbef65700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febb81018f0 0x7febb8196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febbef65700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febb8104220 0x7febb8197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febbef65700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febb8106b50 0x7febb819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febbef65700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb819e860 con 0x7febb81018f0 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febbef65700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb819e9a0 con 0x7febb8106b50 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febbef65700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb819eb60 con 0x7febb8104220 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febbcd01700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febb81018f0 0x7febb8196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febb7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febb8104220 0x7febb8197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febb7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febb8104220 0x7febb8197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49900/0 (socket says 172.21.15.33:49900) 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febb7fff700 1 -- 172.21.15.33:0/30005156 learned_addr learned my addr 172.21.15.33:0/30005156 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.907+0000 7febbcd01700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febb81018f0 0x7febb8196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43156/0 (socket says 172.21.15.33:43156) 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.908+0000 7febbd502700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febb8106b50 0x7febb819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.908+0000 7febb7fff700 1 -- 172.21.15.33:0/30005156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febb8106b50 msgr2=0x7febb819e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.908+0000 7febb7fff700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febb8106b50 0x7febb819e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.908+0000 7febb7fff700 1 -- 172.21.15.33:0/30005156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febb81018f0 msgr2=0x7febb8196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.908+0000 7febb7fff700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febb81018f0 0x7febb8196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.908+0000 7febb7fff700 1 -- 172.21.15.33:0/30005156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba40095f0 con 0x7febb8104220 2024-05-08T21:25:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.908+0000 7febb7fff700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febb8104220 0x7febb8197310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feba400ec40 tx=0x7feba400ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.908+0000 7febb5ffb700 1 -- 172.21.15.33:0/30005156 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4009a10 con 0x7febb8104220 2024-05-08T21:25:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.908+0000 7febb5ffb700 1 -- 172.21.15.33:0/30005156 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feba40044b0 con 0x7febb8104220 2024-05-08T21:25:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.908+0000 7febb5ffb700 1 -- 172.21.15.33:0/30005156 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4003e60 con 0x7febb8104220 2024-05-08T21:25:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.908+0000 7febbef65700 1 -- 172.21.15.33:0/30005156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac009520 con 0x7febb8104220 2024-05-08T21:25:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.909+0000 7febbef65700 1 -- 172.21.15.33:0/30005156 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb819f170 con 0x7febb8104220 2024-05-08T21:25:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.911+0000 7febb5ffb700 1 -- 172.21.15.33:0/30005156 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7feba4004000 con 0x7febb8104220 2024-05-08T21:25:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.913+0000 7febb5ffb700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feba80a11a0 0x7feba80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.913+0000 7febbcd01700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feba80a11a0 0x7feba80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.913+0000 7febb5ffb700 1 -- 172.21.15.33:0/30005156 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7feba4014070 con 0x7febb8104220 2024-05-08T21:25:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.914+0000 7febcb11fb80 1 Processor -- start 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.914+0000 7febcb11fb80 1 -- start start 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.914+0000 7febbcd01700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feba80a11a0 0x7feba80a3660 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7febac0098c0 tx=0x7febac019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.914+0000 7febcb11fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 0x558739ff7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.914+0000 7febcb11fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 0x558739ff8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.914+0000 7febcb11fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff8860 0x558739ff8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.914+0000 7febcb11fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558739f4e150 con 0x558739ff7ea0 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.914+0000 7febcb11fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558739f4bb10 con 0x558739ff8860 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.914+0000 7febcb11fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558739f47400 con 0x558739ff7540 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.914+0000 7febbd502700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff8860 0x558739ff8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.915+0000 7febb7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 0x558739ff8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.915+0000 7febbd502700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff8860 0x558739ff8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56382/0 (socket says 172.21.15.33:56382) 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.915+0000 7febbd502700 1 -- 172.21.15.33:0/626243154 learned_addr learned my addr 172.21.15.33:0/626243154 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.915+0000 7febbcd01700 1 --2- 172.21.15.33:0/626243154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 0x558739ff7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.915+0000 7febb7fff700 1 -- 172.21.15.33:0/626243154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 msgr2=0x558739ff7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.915+0000 7febb7fff700 1 --2- 172.21.15.33:0/626243154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 0x558739ff7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.915+0000 7febb7fff700 1 -- 172.21.15.33:0/626243154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff8860 msgr2=0x558739ff8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.915+0000 7febb7fff700 1 --2- 172.21.15.33:0/626243154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff8860 0x558739ff8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.915+0000 7febb7fff700 1 -- 172.21.15.33:0/626243154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558739dc83b0 con 0x558739ff7ea0 2024-05-08T21:25:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.915+0000 7febbd502700 1 --2- 172.21.15.33:0/626243154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff8860 0x558739ff8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:01.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.915+0000 7febb7fff700 1 --2- 172.21.15.33:0/626243154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 0x558739ff8320 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7feba400c5b0 tx=0x7feba4099d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7739b13cfc6176e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.916+0000 7feba2ffd700 1 -- 172.21.15.33:0/626243154 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba409a1a0 con 0x558739ff7ea0 2024-05-08T21:25:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.916+0000 7feba2ffd700 1 -- 172.21.15.33:0/626243154 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feba409dfd0 con 0x558739ff7ea0 2024-05-08T21:25:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.916+0000 7feba2ffd700 1 -- 172.21.15.33:0/626243154 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4098540 con 0x558739ff7ea0 2024-05-08T21:25:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.917+0000 7febcb11fb80 1 -- 172.21.15.33:0/626243154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 msgr2=0x558739ff8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.917+0000 7febcb11fb80 1 --2- 172.21.15.33:0/626243154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 0x558739ff8320 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7feba400c5b0 tx=0x7feba4099d60 comp rx=0 tx=0).stop 2024-05-08T21:25:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.917+0000 7febcb11fb80 1 -- 172.21.15.33:0/626243154 shutdown_connections 2024-05-08T21:25:01.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.917+0000 7febcb11fb80 1 --2- 172.21.15.33:0/626243154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff8860 0x558739ff8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.917+0000 7febcb11fb80 1 --2- 172.21.15.33:0/626243154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 0x558739ff7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.917+0000 7febcb11fb80 1 --2- 172.21.15.33:0/626243154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 0x558739ff8320 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.917+0000 7febcb11fb80 1 -- 172.21.15.33:0/626243154 >> 172.21.15.33:0/626243154 conn(0x558739f55210 msgr2=0x558739ff1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.917+0000 7febcb11fb80 1 -- 172.21.15.33:0/626243154 shutdown_connections 2024-05-08T21:25:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.917+0000 7febcb11fb80 1 -- 172.21.15.33:0/626243154 wait complete. 2024-05-08T21:25:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.918+0000 7febcb11fb80 1 -- start start 2024-05-08T21:25:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.918+0000 7febcb11fb80 1 Processor -- start 2024-05-08T21:25:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.918+0000 7febcb11fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 0x55873a000e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.918+0000 7febcb11fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 0x55873a001380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.918+0000 7febcb11fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff8860 0x55873a0083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.918+0000 7febcb11fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558739ee3560 con 0x558739ff7ea0 2024-05-08T21:25:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.918+0000 7febcb11fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558739ee3200 con 0x558739ff8860 2024-05-08T21:25:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.918+0000 7febcb11fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558739dca6a0 con 0x558739ff7540 2024-05-08T21:25:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.918+0000 7febcb11fb80 10 client.0 _ll_register_callbacks cb 0x558739eecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7febb7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 0x55873a001380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7febb7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 0x55873a001380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43170/0 (socket says 172.21.15.33:43170) 2024-05-08T21:25:01.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7febb7fff700 1 -- 172.21.15.33:0/3997254256 learned_addr learned my addr 172.21.15.33:0/3997254256 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7febb7fff700 1 -- 172.21.15.33:0/3997254256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 msgr2=0x55873a000e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:25:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7febb7fff700 1 --2- 172.21.15.33:0/3997254256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 0x55873a000e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7febb7fff700 1 -- 172.21.15.33:0/3997254256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff8860 msgr2=0x55873a0083c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:25:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7febb7fff700 1 --2- 172.21.15.33:0/3997254256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff8860 0x55873a0083c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7febb7fff700 1 -- 172.21.15.33:0/3997254256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb0011330 con 0x558739ff7ea0 2024-05-08T21:25:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7febb7fff700 1 --2- 172.21.15.33:0/3997254256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 0x55873a001380 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7feba4096c10 tx=0x7feba40982e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d879dfad70b3df31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7feb97fff700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:25:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7feb97fff700 1 -- 172.21.15.33:0/3997254256 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba40a2620 con 0x558739ff7ea0 2024-05-08T21:25:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7febcb11fb80 1 -- 172.21.15.33:0/3997254256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558739ea0270 con 0x558739ff7ea0 2024-05-08T21:25:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7feb97fff700 1 -- 172.21.15.33:0/3997254256 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7feba409a1a0 con 0x558739ff7ea0 2024-05-08T21:25:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.919+0000 7feb97fff700 1 -- 172.21.15.33:0/3997254256 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7feba4099690 con 0x558739ff7ea0 2024-05-08T21:25:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.920+0000 7febcb11fb80 10 client.17151 fetch_fsmap learned FSMap version 37 2024-05-08T21:25:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.920+0000 7febcb11fb80 1 -- 172.21.15.33:0/3997254256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7febb819f170 con 0x558739ff7ea0 2024-05-08T21:25:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.920+0000 7feb97fff700 1 -- 172.21.15.33:0/3997254256 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558739ea0270 con 0x558739ff7ea0 2024-05-08T21:25:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.920+0000 7feb97fff700 1 -- 172.21.15.33:0/3997254256 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) v1 ==== 7482+0+0 (secure 0 0 0) 0x7feba4099100 con 0x558739ff7ea0 2024-05-08T21:25:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.920+0000 7febcb11fb80 10 client.17151 fetch_fsmap finished waiting for FSMap version 37 2024-05-08T21:25:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.920+0000 7febcb11fb80 10 client.17151 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:25:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.920+0000 7febcb11fb80 20 client.17151 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.920+0000 7febcb11fb80 1 --2- 172.21.15.33:0/3997254256 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55873a010ac0 0x55873a012ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.920+0000 7febcb11fb80 4 client.17151 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.920+0000 7febcb11fb80 1 -- 172.21.15.33:0/3997254256 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558739fc6810 con 0x55873a010ac0 2024-05-08T21:25:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.920+0000 7febbcd01700 1 --2- 172.21.15.33:0/3997254256 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55873a010ac0 0x55873a012ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.921+0000 7feb97fff700 10 client.17151 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:25:01.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.921+0000 7febbcd01700 1 --2- 172.21.15.33:0/3997254256 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55873a010ac0 0x55873a012ee0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f7bca2f8a3f0573b server_cookie=6e34d04261d7f403 in_seq=0 out_seq=0 2024-05-08T21:25:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.921+0000 7feb97fff700 1 -- 172.21.15.33:0/3997254256 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558739fc6810 con 0x55873a010ac0 2024-05-08T21:25:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.921+0000 7feb97fff700 10 client.17151 handle_command_reply: tid=0 2024-05-08T21:25:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.921+0000 7febcb11fb80 1 client.17151 shutdown 2024-05-08T21:25:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.922+0000 7febcb11fb80 1 -- 172.21.15.33:0/3997254256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 msgr2=0x55873a001380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.922+0000 7febcb11fb80 1 --2- 172.21.15.33:0/3997254256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 0x55873a001380 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7feba4096c10 tx=0x7feba40982e0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.922+0000 7febcb11fb80 1 -- 172.21.15.33:0/3997254256 shutdown_connections 2024-05-08T21:25:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.922+0000 7febcb11fb80 1 --2- 172.21.15.33:0/3997254256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff8860 0x55873a0083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.922+0000 7febcb11fb80 1 --2- 172.21.15.33:0/3997254256 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55873a010ac0 0x55873a012ee0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.922+0000 7febcb11fb80 1 --2- 172.21.15.33:0/3997254256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 0x55873a000e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.922+0000 7febcb11fb80 1 --2- 172.21.15.33:0/3997254256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff7ea0 0x55873a001380 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.922+0000 7feb97fff700 0 client.17151 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:25:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.922+0000 7febcb11fb80 1 -- 172.21.15.33:0/3997254256 >> 172.21.15.33:0/3997254256 conn(0x558739f55210 msgr2=0x558739ff15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.922+0000 7febcb11fb80 1 -- 172.21.15.33:0/3997254256 shutdown_connections 2024-05-08T21:25:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.922+0000 7febcb11fb80 1 -- 172.21.15.33:0/3997254256 wait complete. 2024-05-08T21:25:01.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.923+0000 7febcb11fb80 20 client.17151 trim_cache size 0 max 16384 2024-05-08T21:25:01.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.939+0000 7febcb11fb80 1 Processor -- start 2024-05-08T21:25:01.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.939+0000 7febcb11fb80 1 -- start start 2024-05-08T21:25:01.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.939+0000 7febcb11fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7540 0x55873a00ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.939+0000 7febcb11fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7ea0 0x55873a008380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.939+0000 7febcb11fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff8860 0x55873a000350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.939+0000 7febcb11fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55873a00d950 con 0x558739ff8860 2024-05-08T21:25:01.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.939+0000 7febcb11fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558739fec800 con 0x558739ff7540 2024-05-08T21:25:01.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.939+0000 7febcb11fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558739ff3830 con 0x558739ff7ea0 2024-05-08T21:25:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.939+0000 7febbd502700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff8860 0x55873a000350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.939+0000 7febbcd01700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7540 0x55873a00ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.939+0000 7febbd502700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff8860 0x55873a000350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43172/0 (socket says 172.21.15.33:43172) 2024-05-08T21:25:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.939+0000 7febbd502700 1 -- 172.21.15.33:0/4103116864 learned_addr learned my addr 172.21.15.33:0/4103116864 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:01.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.940+0000 7febb7fff700 1 --2- 172.21.15.33:0/4103116864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7ea0 0x55873a008380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.940+0000 7febbcd01700 1 -- 172.21.15.33:0/4103116864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7ea0 msgr2=0x55873a008380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.940+0000 7febbcd01700 1 --2- 172.21.15.33:0/4103116864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7ea0 0x55873a008380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.940+0000 7febbcd01700 1 -- 172.21.15.33:0/4103116864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff8860 msgr2=0x55873a000350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.940+0000 7febbcd01700 1 --2- 172.21.15.33:0/4103116864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff8860 0x55873a000350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.940+0000 7febbcd01700 1 -- 172.21.15.33:0/4103116864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558739d74b60 con 0x558739ff7540 2024-05-08T21:25:01.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.940+0000 7febbcd01700 1 --2- 172.21.15.33:0/4103116864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7540 0x55873a00ce80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7febac009810 tx=0x7febac01e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3bb33549f63c910 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.940+0000 7feba37fe700 1 -- 172.21.15.33:0/4103116864 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac02b040 con 0x558739ff7540 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.940+0000 7feba37fe700 1 -- 172.21.15.33:0/4103116864 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febac01d040 con 0x558739ff7540 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.940+0000 7feba37fe700 1 -- 172.21.15.33:0/4103116864 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac005040 con 0x558739ff7540 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.941+0000 7febcb11fb80 1 -- 172.21.15.33:0/4103116864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7540 msgr2=0x55873a00ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.941+0000 7febcb11fb80 1 --2- 172.21.15.33:0/4103116864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7540 0x55873a00ce80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7febac009810 tx=0x7febac01e000 comp rx=0 tx=0).stop 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.941+0000 7febcb11fb80 1 -- 172.21.15.33:0/4103116864 shutdown_connections 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.941+0000 7febcb11fb80 1 --2- 172.21.15.33:0/4103116864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff8860 0x55873a000350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.941+0000 7febcb11fb80 1 --2- 172.21.15.33:0/4103116864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7540 0x55873a00ce80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.941+0000 7febcb11fb80 1 --2- 172.21.15.33:0/4103116864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7ea0 0x55873a008380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.941+0000 7febcb11fb80 1 -- 172.21.15.33:0/4103116864 >> 172.21.15.33:0/4103116864 conn(0x558739f55210 msgr2=0x558739ff8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.941+0000 7febcb11fb80 1 -- 172.21.15.33:0/4103116864 shutdown_connections 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.941+0000 7febcb11fb80 1 -- 172.21.15.33:0/4103116864 wait complete. 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febcb11fb80 1 -- start start 2024-05-08T21:25:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febcb11fb80 1 Processor -- start 2024-05-08T21:25:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febcb11fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 0x55873a00a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febcb11fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7ea0 0x55873a00abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febcb11fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff8860 0x55873a008380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febcb11fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55873a00bb60 con 0x558739ff8860 2024-05-08T21:25:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febcb11fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55873a00bcd0 con 0x558739ff7ea0 2024-05-08T21:25:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febcb11fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55873a00be90 con 0x558739ff7540 2024-05-08T21:25:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febcb11fb80 10 client.0 _ll_register_callbacks cb 0x558739eecb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febbd502700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff8860 0x55873a008380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febb7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7ea0 0x55873a00abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febb7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7ea0 0x55873a00abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56406/0 (socket says 172.21.15.33:56406) 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febb7fff700 1 -- 172.21.15.33:0/3000790591 learned_addr learned my addr 172.21.15.33:0/3000790591 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.942+0000 7febbcd01700 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 0x55873a00a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7febb7fff700 1 -- 172.21.15.33:0/3000790591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 msgr2=0x55873a00a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7febb7fff700 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 0x55873a00a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7febb7fff700 1 -- 172.21.15.33:0/3000790591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff8860 msgr2=0x55873a008380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7febb7fff700 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff8860 0x55873a008380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7febb7fff700 1 -- 172.21.15.33:0/3000790591 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb8109bb0 con 0x558739ff7ea0 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7febbd502700 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff8860 0x55873a008380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7febb7fff700 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7ea0 0x55873a00abf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feba40ca930 tx=0x7feba40036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c0aaca0116d6ce8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7feb97fff700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7feb97fff700 1 -- 172.21.15.33:0/3000790591 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba401fe20 con 0x558739ff7ea0 2024-05-08T21:25:01.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7feb97fff700 1 -- 172.21.15.33:0/3000790591 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7feba409a1a0 con 0x558739ff7ea0 2024-05-08T21:25:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7febcb11fb80 1 -- 172.21.15.33:0/3000790591 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feba409dfd0 con 0x558739ff7ea0 2024-05-08T21:25:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7feb97fff700 1 -- 172.21.15.33:0/3000790591 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7feba40a2620 con 0x558739ff7ea0 2024-05-08T21:25:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7febcb11fb80 10 client.27053 fetch_fsmap learned FSMap version 37 2024-05-08T21:25:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.943+0000 7febcb11fb80 1 -- 172.21.15.33:0/3000790591 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558739f0d560 con 0x558739ff7ea0 2024-05-08T21:25:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.944+0000 7feb97fff700 1 -- 172.21.15.33:0/3000790591 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feba409dfd0 con 0x558739ff7ea0 2024-05-08T21:25:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.944+0000 7feb97fff700 1 -- 172.21.15.33:0/3000790591 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 37) v1 ==== 7482+0+0 (secure 0 0 0) 0x7feba40ab070 con 0x558739ff7ea0 2024-05-08T21:25:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.944+0000 7febcb11fb80 10 client.27053 fetch_fsmap finished waiting for FSMap version 37 2024-05-08T21:25:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.944+0000 7febcb11fb80 10 client.27053 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:25:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.944+0000 7febcb11fb80 20 client.27053 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.944+0000 7febcb11fb80 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55873a0633c0 0x55873a0657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.944+0000 7febcb11fb80 4 client.27053 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:25:01.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.944+0000 7febcb11fb80 1 -- 172.21.15.33:0/3000790591 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558739f3fec0 con 0x55873a0633c0 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.944+0000 7febbcd01700 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55873a0633c0 0x55873a0657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.945+0000 7feb97fff700 10 client.27053 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.945+0000 7febbcd01700 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55873a0633c0 0x55873a0657b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=99e631fbc820d040 server_cookie=fb4ba6fd5a55d1ae in_seq=0 out_seq=0 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.946+0000 7feb97fff700 1 -- 172.21.15.33:0/3000790591 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558739f3fec0 con 0x55873a0633c0 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.946+0000 7feb97fff700 10 client.27053 handle_command_reply: tid=0 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.946+0000 7febcb11fb80 1 client.27053 shutdown 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.946+0000 7febcb11fb80 1 -- 172.21.15.33:0/3000790591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7ea0 msgr2=0x55873a00abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.946+0000 7febcb11fb80 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7ea0 0x55873a00abf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feba40ca930 tx=0x7feba40036f0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.946+0000 7febcb11fb80 1 -- 172.21.15.33:0/3000790591 shutdown_connections 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.946+0000 7febcb11fb80 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558739ff8860 0x55873a008380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.946+0000 7febcb11fb80 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55873a0633c0 0x55873a0657b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.946+0000 7febcb11fb80 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558739ff7540 0x55873a00a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.946+0000 7febcb11fb80 1 --2- 172.21.15.33:0/3000790591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558739ff7ea0 0x55873a00abf0 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feba40ca930 tx=0x7feba40036f0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.946+0000 7feb97fff700 0 client.27053 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:25:01.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.946+0000 7febcb11fb80 1 -- 172.21.15.33:0/3000790591 >> 172.21.15.33:0/3000790591 conn(0x558739f55210 msgr2=0x55873a000350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.947+0000 7febcb11fb80 1 -- 172.21.15.33:0/3000790591 shutdown_connections 2024-05-08T21:25:01.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.947+0000 7febcb11fb80 1 -- 172.21.15.33:0/3000790591 wait complete. 2024-05-08T21:25:01.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.948+0000 7febcb11fb80 20 client.27053 trim_cache size 0 max 16384 2024-05-08T21:25:01.952 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:25:01.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.948+0000 7febbef65700 1 -- 172.21.15.33:0/30005156 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feba80a11a0 msgr2=0x7feba80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.948+0000 7febbef65700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feba80a11a0 0x7feba80a3660 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7febac0098c0 tx=0x7febac019040 comp rx=0 tx=0).stop 2024-05-08T21:25:01.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.948+0000 7febbef65700 1 -- 172.21.15.33:0/30005156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febb8104220 msgr2=0x7febb8197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:01.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.948+0000 7febbef65700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febb8104220 0x7febb8197310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feba400ec40 tx=0x7feba400ef50 comp rx=0 tx=0).stop 2024-05-08T21:25:01.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.948+0000 7febbef65700 1 -- 172.21.15.33:0/30005156 shutdown_connections 2024-05-08T21:25:01.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.948+0000 7febbef65700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7feba80a11a0 0x7feba80a3660 secure :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7febac0098c0 tx=0x7febac019040 comp rx=0 tx=0).stop 2024-05-08T21:25:01.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.948+0000 7febbef65700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febb8106b50 0x7febb819e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.948+0000 7febbef65700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febb81018f0 0x7febb8196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:01.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.948+0000 7febbef65700 1 --2- 172.21.15.33:0/30005156 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febb8104220 0x7febb8197310 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feba400ec40 tx=0x7feba400ef50 comp rx=0 tx=0).stop 2024-05-08T21:25:01.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.948+0000 7febbef65700 1 -- 172.21.15.33:0/30005156 >> 172.21.15.33:0/30005156 conn(0x7febb80fb300 msgr2=0x7febb8102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:01.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.949+0000 7febbef65700 1 -- 172.21.15.33:0/30005156 shutdown_connections 2024-05-08T21:25:01.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:01.949+0000 7febbef65700 1 -- 172.21.15.33:0/30005156 wait complete. 2024-05-08T21:25:01.967 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:25:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.134+0000 7f998d582700 1 -- 172.21.15.33:0/1803530291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99880ff740 msgr2=0x7f9988101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.134+0000 7f998d582700 1 --2- 172.21.15.33:0/1803530291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99880ff740 0x7f9988101b70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f99780098f0 tx=0x7f9978009c00 comp rx=0 tx=0).stop 2024-05-08T21:25:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.134+0000 7f998d582700 1 -- 172.21.15.33:0/1803530291 shutdown_connections 2024-05-08T21:25:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.134+0000 7f998d582700 1 --2- 172.21.15.33:0/1803530291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9988104a80 0x7f9988104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.134+0000 7f998d582700 1 --2- 172.21.15.33:0/1803530291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99880ff740 0x7f9988101b70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.134+0000 7f998d582700 1 --2- 172.21.15.33:0/1803530291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99881020b0 0x7f9988104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.134+0000 7f998d582700 1 -- 172.21.15.33:0/1803530291 >> 172.21.15.33:0/1803530291 conn(0x7f99880fb300 msgr2=0x7f99880fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.135+0000 7f998d582700 1 -- 172.21.15.33:0/1803530291 shutdown_connections 2024-05-08T21:25:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.135+0000 7f998d582700 1 -- 172.21.15.33:0/1803530291 wait complete. 2024-05-08T21:25:02.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.135+0000 7f998d582700 1 Processor -- start 2024-05-08T21:25:02.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f998d582700 1 -- start start 2024-05-08T21:25:02.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f998d582700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99880ff740 0x7f998810e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f998d582700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99881020b0 0x7f998810ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f998d582700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9988104a80 0x7f998810f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f998d582700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99881ad600 con 0x7f9988104a80 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f998d582700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99881ad740 con 0x7f99881020b0 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f998d582700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99881ad900 con 0x7f99880ff740 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f9986ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99880ff740 0x7f998810e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f9986ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99880ff740 0x7f998810e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49936/0 (socket says 172.21.15.33:49936) 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f9986ffd700 1 -- 172.21.15.33:0/88224058 learned_addr learned my addr 172.21.15.33:0/88224058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f99867fc700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99881020b0 0x7f998810ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f99877fe700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9988104a80 0x7f998810f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f9986ffd700 1 -- 172.21.15.33:0/88224058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99881020b0 msgr2=0x7f998810ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f9986ffd700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99881020b0 0x7f998810ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f9986ffd700 1 -- 172.21.15.33:0/88224058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9988104a80 msgr2=0x7f998810f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f9986ffd700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9988104a80 0x7f998810f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f9986ffd700 1 -- 172.21.15.33:0/88224058 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9978009520 con 0x7f99880ff740 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.136+0000 7f99867fc700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99881020b0 0x7f998810ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.137+0000 7f9986ffd700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99880ff740 0x7f998810e8d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f997800fb00 tx=0x7f997800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.137+0000 7f996ffff700 1 -- 172.21.15.33:0/88224058 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997801c070 con 0x7f99880ff740 2024-05-08T21:25:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.137+0000 7f996ffff700 1 -- 172.21.15.33:0/88224058 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f997800f7d0 con 0x7f99880ff740 2024-05-08T21:25:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.137+0000 7f998d582700 1 -- 172.21.15.33:0/88224058 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99881adb80 con 0x7f99880ff740 2024-05-08T21:25:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.137+0000 7f996ffff700 1 -- 172.21.15.33:0/88224058 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9978003c10 con 0x7f99880ff740 2024-05-08T21:25:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.137+0000 7f998d582700 1 -- 172.21.15.33:0/88224058 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99881adfa0 con 0x7f99880ff740 2024-05-08T21:25:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.140+0000 7f996ffff700 1 -- 172.21.15.33:0/88224058 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9978005040 con 0x7f99880ff740 2024-05-08T21:25:02.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.141+0000 7f996ffff700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f99700a1120 0x7f99700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.141+0000 7f996ffff700 1 -- 172.21.15.33:0/88224058 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f99780cdad0 con 0x7f99880ff740 2024-05-08T21:25:02.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.141+0000 7f99867fc700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f99700a1120 0x7f99700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.141+0000 7f999973cb80 1 Processor -- start 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.141+0000 7f999973cb80 1 -- start start 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f999973cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa6550 0x5582a1aa6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f999973cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6eb0 0x5582a1aa7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f999973cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa7870 0x5582a1aa7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f999973cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a19ec150 con 0x5582a1aa7870 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f999973cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a19e9b10 con 0x5582a1aa6eb0 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f999973cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a19e5400 con 0x5582a1aa6550 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f99867fc700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f99700a1120 0x7f99700a35e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f997c005950 tx=0x7f997c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f99867fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6eb0 0x5582a1aa7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f99867fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6eb0 0x5582a1aa7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56428/0 (socket says 172.21.15.33:56428) 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f99867fc700 1 -- 172.21.15.33:0/3331391626 learned_addr learned my addr 172.21.15.33:0/3331391626 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f99867fc700 1 -- 172.21.15.33:0/3331391626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa6550 msgr2=0x5582a1aa6970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:25:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f99867fc700 1 --2- 172.21.15.33:0/3331391626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa6550 0x5582a1aa6970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f99867fc700 1 -- 172.21.15.33:0/3331391626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa7870 msgr2=0x5582a1aa7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f99867fc700 1 --2- 172.21.15.33:0/3331391626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa7870 0x5582a1aa7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f99867fc700 1 -- 172.21.15.33:0/3331391626 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582a18663b0 con 0x5582a1aa6eb0 2024-05-08T21:25:02.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.142+0000 7f99867fc700 1 --2- 172.21.15.33:0/3331391626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6eb0 0x5582a1aa7330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f997c0117c0 tx=0x7f997c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27ee0a3cda0c973e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.143+0000 7f996d7fa700 1 -- 172.21.15.33:0/3331391626 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c01e620 con 0x5582a1aa6eb0 2024-05-08T21:25:02.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.143+0000 7f996d7fa700 1 -- 172.21.15.33:0/3331391626 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f997c015070 con 0x5582a1aa6eb0 2024-05-08T21:25:02.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.143+0000 7f996d7fa700 1 -- 172.21.15.33:0/3331391626 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997c01d9b0 con 0x5582a1aa6eb0 2024-05-08T21:25:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 -- 172.21.15.33:0/3331391626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6eb0 msgr2=0x5582a1aa7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 --2- 172.21.15.33:0/3331391626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6eb0 0x5582a1aa7330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f997c0117c0 tx=0x7f997c013000 comp rx=0 tx=0).stop 2024-05-08T21:25:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 -- 172.21.15.33:0/3331391626 shutdown_connections 2024-05-08T21:25:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 --2- 172.21.15.33:0/3331391626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa7870 0x5582a1aa7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 --2- 172.21.15.33:0/3331391626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa6550 0x5582a1aa6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 --2- 172.21.15.33:0/3331391626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6eb0 0x5582a1aa7330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 -- 172.21.15.33:0/3331391626 >> 172.21.15.33:0/3331391626 conn(0x5582a19f3210 msgr2=0x5582a1a8fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 -- 172.21.15.33:0/3331391626 shutdown_connections 2024-05-08T21:25:02.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 -- 172.21.15.33:0/3331391626 wait complete. 2024-05-08T21:25:02.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 -- start start 2024-05-08T21:25:02.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 Processor -- start 2024-05-08T21:25:02.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6550 0x5582a1aafe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa7870 0x5582a1ab7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a1981560 con 0x5582a1aa6550 2024-05-08T21:25:02.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a1981200 con 0x5582a1aa7870 2024-05-08T21:25:02.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a18686a0 con 0x5582a1aa6eb0 2024-05-08T21:25:02.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.144+0000 7f999973cb80 10 client.0 _ll_register_callbacks cb 0x5582a198ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:02.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f99877fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa7870 0x5582a1ab7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f99877fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa7870 0x5582a1ab7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56440/0 (socket says 172.21.15.33:56440) 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f99877fe700 1 -- 172.21.15.33:0/1261616160 learned_addr learned my addr 172.21.15.33:0/1261616160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f99867fc700 1 --2- 172.21.15.33:0/1261616160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f99877fe700 1 -- 172.21.15.33:0/1261616160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa6eb0 msgr2=0x5582a1ab03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f99877fe700 1 --2- 172.21.15.33:0/1261616160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f99877fe700 1 -- 172.21.15.33:0/1261616160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6550 msgr2=0x5582a1aafe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f99877fe700 1 --2- 172.21.15.33:0/1261616160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6550 0x5582a1aafe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f99877fe700 1 -- 172.21.15.33:0/1261616160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997c011440 con 0x5582a1aa7870 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f99877fe700 1 --2- 172.21.15.33:0/1261616160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa7870 0x5582a1ab7410 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9980011ce0 tx=0x7f998000f1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38d03f4cfc9770d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f996aff5700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f996aff5700 1 -- 172.21.15.33:0/1261616160 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000e040 con 0x5582a1aa7870 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f996aff5700 1 -- 172.21.15.33:0/1261616160 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9980003a00 con 0x5582a1aa7870 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.145+0000 7f999973cb80 1 -- 172.21.15.33:0/1261616160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582a193e270 con 0x5582a1aa7870 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.146+0000 7f996aff5700 1 -- 172.21.15.33:0/1261616160 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9980003bc0 con 0x5582a1aa7870 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.146+0000 7f999973cb80 10 client.27065 fetch_fsmap learned FSMap version 37 2024-05-08T21:25:02.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.146+0000 7f999973cb80 1 -- 172.21.15.33:0/1261616160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5582a1812b60 con 0x5582a1aa7870 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.146+0000 7f996aff5700 1 -- 172.21.15.33:0/1261616160 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5582a193e270 con 0x5582a1aa7870 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.146+0000 7f996aff5700 1 -- 172.21.15.33:0/1261616160 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 37) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f99800041b0 con 0x5582a1aa7870 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.146+0000 7f999973cb80 10 client.27065 fetch_fsmap finished waiting for FSMap version 37 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.146+0000 7f999973cb80 10 client.27065 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.146+0000 7f999973cb80 20 client.27065 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.146+0000 7f999973cb80 1 --2- 172.21.15.33:0/1261616160 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5582a1abbbf0 0x5582a1abe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.146+0000 7f999973cb80 4 client.27065 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.146+0000 7f999973cb80 1 -- 172.21.15.33:0/1261616160 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582a1a64810 con 0x5582a1abbbf0 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.146+0000 7f9986ffd700 1 --2- 172.21.15.33:0/1261616160 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5582a1abbbf0 0x5582a1abe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f996aff5700 10 client.27065 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f9986ffd700 1 --2- 172.21.15.33:0/1261616160 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5582a1abbbf0 0x5582a1abe010 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8aef6e732ebd204c server_cookie=34a9f3d330e66ef5 in_seq=0 out_seq=0 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f996aff5700 1 -- 172.21.15.33:0/1261616160 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5582a1a64810 con 0x5582a1abbbf0 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f996aff5700 10 client.27065 handle_command_reply: tid=0 2024-05-08T21:25:02.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f999973cb80 1 client.27065 shutdown 2024-05-08T21:25:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f999973cb80 1 -- 172.21.15.33:0/1261616160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa7870 msgr2=0x5582a1ab7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f999973cb80 1 --2- 172.21.15.33:0/1261616160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa7870 0x5582a1ab7410 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9980011ce0 tx=0x7f998000f1b0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f999973cb80 1 -- 172.21.15.33:0/1261616160 shutdown_connections 2024-05-08T21:25:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f999973cb80 1 --2- 172.21.15.33:0/1261616160 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5582a1abbbf0 0x5582a1abe010 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f999973cb80 1 --2- 172.21.15.33:0/1261616160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa7870 0x5582a1ab7410 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f999973cb80 1 --2- 172.21.15.33:0/1261616160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6550 0x5582a1aafe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f999973cb80 1 --2- 172.21.15.33:0/1261616160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f999973cb80 1 -- 172.21.15.33:0/1261616160 >> 172.21.15.33:0/1261616160 conn(0x5582a19f3210 msgr2=0x5582a1a8f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f996aff5700 0 client.27065 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:25:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f999973cb80 1 -- 172.21.15.33:0/1261616160 shutdown_connections 2024-05-08T21:25:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.147+0000 7f999973cb80 1 -- 172.21.15.33:0/1261616160 wait complete. 2024-05-08T21:25:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.148+0000 7f999973cb80 20 client.27065 trim_cache size 0 max 16384 2024-05-08T21:25:02.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f999973cb80 1 Processor -- start 2024-05-08T21:25:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f999973cb80 1 -- start start 2024-05-08T21:25:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f999973cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6550 0x5582a1ab7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f999973cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f999973cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa7870 0x5582a1aa7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f999973cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a1a90b60 con 0x5582a1aa6eb0 2024-05-08T21:25:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f999973cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a1ac3200 con 0x5582a1aa6550 2024-05-08T21:25:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f999973cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a1aa8190 con 0x5582a1aa7870 2024-05-08T21:25:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f99877fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa7870 0x5582a1aa7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f99867fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f99877fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa7870 0x5582a1aa7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49964/0 (socket says 172.21.15.33:49964) 2024-05-08T21:25:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f99877fe700 1 -- 172.21.15.33:0/3003404266 learned_addr learned my addr 172.21.15.33:0/3003404266 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f9986ffd700 1 --2- 172.21.15.33:0/3003404266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6550 0x5582a1ab7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f99877fe700 1 -- 172.21.15.33:0/3003404266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6550 msgr2=0x5582a1ab7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f99877fe700 1 --2- 172.21.15.33:0/3003404266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6550 0x5582a1ab7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f99877fe700 1 -- 172.21.15.33:0/3003404266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6eb0 msgr2=0x5582a1ab73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f99877fe700 1 --2- 172.21.15.33:0/3003404266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.168+0000 7f99877fe700 1 -- 172.21.15.33:0/3003404266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582a19ab560 con 0x5582a1aa7870 2024-05-08T21:25:02.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.169+0000 7f99877fe700 1 --2- 172.21.15.33:0/3003404266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa7870 0x5582a1aa7c50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9980009ec0 tx=0x7f998000f630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c3020ebe6c2f8d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.169+0000 7f996dffb700 1 -- 172.21.15.33:0/3003404266 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998000e040 con 0x5582a1aa7870 2024-05-08T21:25:02.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.169+0000 7f996dffb700 1 -- 172.21.15.33:0/3003404266 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f998000f040 con 0x5582a1aa7870 2024-05-08T21:25:02.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.169+0000 7f996dffb700 1 -- 172.21.15.33:0/3003404266 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99800075c0 con 0x5582a1aa7870 2024-05-08T21:25:02.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.170+0000 7f999973cb80 1 -- 172.21.15.33:0/3003404266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa7870 msgr2=0x5582a1aa7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.170+0000 7f999973cb80 1 --2- 172.21.15.33:0/3003404266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa7870 0x5582a1aa7c50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9980009ec0 tx=0x7f998000f630 comp rx=0 tx=0).stop 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.170+0000 7f999973cb80 1 -- 172.21.15.33:0/3003404266 shutdown_connections 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.170+0000 7f999973cb80 1 --2- 172.21.15.33:0/3003404266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa7870 0x5582a1aa7c50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.170+0000 7f999973cb80 1 --2- 172.21.15.33:0/3003404266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6550 0x5582a1ab7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.170+0000 7f999973cb80 1 --2- 172.21.15.33:0/3003404266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.170+0000 7f999973cb80 1 -- 172.21.15.33:0/3003404266 >> 172.21.15.33:0/3003404266 conn(0x5582a19f3210 msgr2=0x5582a1a8c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.170+0000 7f999973cb80 1 -- 172.21.15.33:0/3003404266 shutdown_connections 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.170+0000 7f999973cb80 1 -- 172.21.15.33:0/3003404266 wait complete. 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.170+0000 7f999973cb80 1 -- start start 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.170+0000 7f999973cb80 1 Processor -- start 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f999973cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6550 0x5582a1ab4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f999973cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f999973cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa7870 0x5582a1ab5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f999973cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a1ab6390 con 0x5582a1aa6eb0 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f999973cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a1ab6500 con 0x5582a1aa6550 2024-05-08T21:25:02.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f999973cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a1ab66c0 con 0x5582a1aa7870 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f999973cb80 10 client.0 _ll_register_callbacks cb 0x5582a198ab30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f9986ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6550 0x5582a1ab4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f99877fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa7870 0x5582a1ab5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f9986ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6550 0x5582a1ab4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56456/0 (socket says 172.21.15.33:56456) 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f9986ffd700 1 -- 172.21.15.33:0/1809774534 learned_addr learned my addr 172.21.15.33:0/1809774534 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f99867fc700 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f9986ffd700 1 -- 172.21.15.33:0/1809774534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa7870 msgr2=0x5582a1ab5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f9986ffd700 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa7870 0x5582a1ab5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f9986ffd700 1 -- 172.21.15.33:0/1809774534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6eb0 msgr2=0x5582a1ab4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f9986ffd700 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f9986ffd700 1 -- 172.21.15.33:0/1809774534 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582a1812b60 con 0x5582a1aa6550 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.171+0000 7f99867fc700 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.172+0000 7f9986ffd700 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6550 0x5582a1ab4900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f997800f9d0 tx=0x7f997809b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19cdfe935ed16d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.172+0000 7f996aff5700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.172+0000 7f996aff5700 1 -- 172.21.15.33:0/1809774534 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99780a3020 con 0x5582a1aa6550 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.172+0000 7f999973cb80 1 -- 172.21.15.33:0/1809774534 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f997c015070 con 0x5582a1aa6550 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.172+0000 7f996aff5700 1 -- 172.21.15.33:0/1809774534 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f99780ba040 con 0x5582a1aa6550 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.172+0000 7f996aff5700 1 -- 172.21.15.33:0/1809774534 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9978098fd0 con 0x5582a1aa6550 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.172+0000 7f999973cb80 10 client.27071 fetch_fsmap learned FSMap version 37 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.172+0000 7f999973cb80 1 -- 172.21.15.33:0/1809774534 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5582a186acd0 con 0x5582a1aa6550 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.172+0000 7f996aff5700 1 -- 172.21.15.33:0/1809774534 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9978064020 con 0x5582a1aa6550 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.172+0000 7f996aff5700 1 -- 172.21.15.33:0/1809774534 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 37) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f997809afb0 con 0x5582a1aa6550 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.173+0000 7f999973cb80 10 client.27071 fetch_fsmap finished waiting for FSMap version 37 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.173+0000 7f999973cb80 10 client.27071 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.173+0000 7f999973cb80 20 client.27071 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.173+0000 7f999973cb80 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5582a1b12400 0x5582a1b14820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.173+0000 7f999973cb80 4 client.27071 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.173+0000 7f999973cb80 1 -- 172.21.15.33:0/1809774534 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5582a19ddec0 con 0x5582a1b12400 2024-05-08T21:25:02.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.173+0000 7f99867fc700 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5582a1b12400 0x5582a1b14820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.173+0000 7f996aff5700 10 client.27071 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.173+0000 7f99867fc700 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5582a1b12400 0x5582a1b14820 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3ae44187d3bf443 server_cookie=b5465b1d66f93629 in_seq=0 out_seq=0 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.173+0000 7f996aff5700 1 -- 172.21.15.33:0/1809774534 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5582a19ddec0 con 0x5582a1b12400 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.173+0000 7f996aff5700 10 client.27071 handle_command_reply: tid=0 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.173+0000 7f999973cb80 1 client.27071 shutdown 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f999973cb80 1 -- 172.21.15.33:0/1809774534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6550 msgr2=0x5582a1ab4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f999973cb80 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6550 0x5582a1ab4900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f997800f9d0 tx=0x7f997809b330 comp rx=0 tx=0).stop 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f999973cb80 1 -- 172.21.15.33:0/1809774534 shutdown_connections 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f999973cb80 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x5582a1b12400 0x5582a1b14820 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f999973cb80 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5582a1aa7870 0x5582a1ab5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f999973cb80 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582a1aa6550 0x5582a1ab4900 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f996aff5700 0 client.27071 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f999973cb80 1 --2- 172.21.15.33:0/1809774534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5582a1aa6eb0 0x5582a1ab4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f999973cb80 1 -- 172.21.15.33:0/1809774534 >> 172.21.15.33:0/1809774534 conn(0x5582a19f3210 msgr2=0x5582a1aaa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f999973cb80 1 -- 172.21.15.33:0/1809774534 shutdown_connections 2024-05-08T21:25:02.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f999973cb80 1 -- 172.21.15.33:0/1809774534 wait complete. 2024-05-08T21:25:02.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f999973cb80 20 client.27071 trim_cache size 0 max 16384 2024-05-08T21:25:02.181 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:25:02.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.174+0000 7f998d582700 1 -- 172.21.15.33:0/88224058 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f99700a1120 msgr2=0x7f99700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.175+0000 7f998d582700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f99700a1120 0x7f99700a35e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f997c005950 tx=0x7f997c00b410 comp rx=0 tx=0).stop 2024-05-08T21:25:02.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.175+0000 7f998d582700 1 -- 172.21.15.33:0/88224058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99880ff740 msgr2=0x7f998810e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.175+0000 7f998d582700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99880ff740 0x7f998810e8d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f997800fb00 tx=0x7f997800b790 comp rx=0 tx=0).stop 2024-05-08T21:25:02.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.175+0000 7f998d582700 1 -- 172.21.15.33:0/88224058 shutdown_connections 2024-05-08T21:25:02.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.175+0000 7f998d582700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9988104a80 0x7f998810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.175+0000 7f998d582700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99880ff740 0x7f998810e8d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.175+0000 7f998d582700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f99700a1120 0x7f99700a35e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.175+0000 7f998d582700 1 --2- 172.21.15.33:0/88224058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99881020b0 0x7f998810ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.175+0000 7f998d582700 1 -- 172.21.15.33:0/88224058 >> 172.21.15.33:0/88224058 conn(0x7f99880fb300 msgr2=0x7f99880fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.175+0000 7f998d582700 1 -- 172.21.15.33:0/88224058 shutdown_connections 2024-05-08T21:25:02.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.175+0000 7f998d582700 1 -- 172.21.15.33:0/88224058 wait complete. 2024-05-08T21:25:02.192 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:25:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.359+0000 7f987e9ef700 1 -- 172.21.15.33:0/1273865773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98780ffd60 msgr2=0x7f9878100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.359+0000 7f987e9ef700 1 --2- 172.21.15.33:0/1273865773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98780ffd60 0x7f9878100180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9868009990 tx=0x7f9868009ca0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.359+0000 7f987e9ef700 1 -- 172.21.15.33:0/1273865773 shutdown_connections 2024-05-08T21:25:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.359+0000 7f987e9ef700 1 --2- 172.21.15.33:0/1273865773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98781018f0 0x7f98781081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.359+0000 7f987e9ef700 1 --2- 172.21.15.33:0/1273865773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98780ffd60 0x7f9878100180 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.359+0000 7f987e9ef700 1 --2- 172.21.15.33:0/1273865773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9878100f50 0x7f98781013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.359+0000 7f987e9ef700 1 -- 172.21.15.33:0/1273865773 >> 172.21.15.33:0/1273865773 conn(0x7f98780fb290 msgr2=0x7f98780fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.359+0000 7f987e9ef700 1 -- 172.21.15.33:0/1273865773 shutdown_connections 2024-05-08T21:25:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.359+0000 7f987e9ef700 1 -- 172.21.15.33:0/1273865773 wait complete. 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.360+0000 7f987e9ef700 1 Processor -- start 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.360+0000 7f987e9ef700 1 -- start start 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.360+0000 7f987e9ef700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98780ffd60 0x7f987810a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f987e9ef700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878100f50 0x7f9878108cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f987e9ef700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98781018f0 0x7f98781091f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f987e9ef700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9878109990 con 0x7f98781018f0 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f987e9ef700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9878109b00 con 0x7f98780ffd60 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f987e9ef700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98781a0e40 con 0x7f9878100f50 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f9877fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98780ffd60 0x7f987810a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f9877fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98780ffd60 0x7f987810a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56464/0 (socket says 172.21.15.33:56464) 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f9877fff700 1 -- 172.21.15.33:0/611682455 learned_addr learned my addr 172.21.15.33:0/611682455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f987cf8c700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98781018f0 0x7f98781091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f98777fe700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878100f50 0x7f9878108cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f9877fff700 1 -- 172.21.15.33:0/611682455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878100f50 msgr2=0x7f9878108cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f9877fff700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878100f50 0x7f9878108cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f9877fff700 1 -- 172.21.15.33:0/611682455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98781018f0 msgr2=0x7f98781091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f9877fff700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98781018f0 0x7f98781091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f9877fff700 1 -- 172.21.15.33:0/611682455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98680095f0 con 0x7f98780ffd60 2024-05-08T21:25:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f9877fff700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98780ffd60 0x7f987810a620 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f98680095c0 tx=0x7f986800b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.361+0000 7f98757fa700 1 -- 172.21.15.33:0/611682455 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986801d070 con 0x7f98780ffd60 2024-05-08T21:25:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.362+0000 7f98757fa700 1 -- 172.21.15.33:0/611682455 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9868003d20 con 0x7f98780ffd60 2024-05-08T21:25:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.362+0000 7f987e9ef700 1 -- 172.21.15.33:0/611682455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98781a1070 con 0x7f98780ffd60 2024-05-08T21:25:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.362+0000 7f98757fa700 1 -- 172.21.15.33:0/611682455 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986800fe30 con 0x7f98780ffd60 2024-05-08T21:25:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.362+0000 7f987e9ef700 1 -- 172.21.15.33:0/611682455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98781a1510 con 0x7f98780ffd60 2024-05-08T21:25:02.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:02 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/826556690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:25:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.365+0000 7f98757fa700 1 -- 172.21.15.33:0/611682455 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f986802b440 con 0x7f98780ffd60 2024-05-08T21:25:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.366+0000 7f98757fa700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f98600a11a0 0x7f98600a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.366+0000 7f98757fa700 1 -- 172.21.15.33:0/611682455 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f98680cdfe0 con 0x7f98780ffd60 2024-05-08T21:25:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.366+0000 7f98777fe700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f98600a11a0 0x7f98600a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.366+0000 7f988aba9b80 1 Processor -- start 2024-05-08T21:25:02.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.366+0000 7f988aba9b80 1 -- start start 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.366+0000 7f988aba9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 0x5593d605c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.366+0000 7f988aba9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605cea0 0x5593d605d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.366+0000 7f988aba9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605d860 0x5593d605dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.366+0000 7f988aba9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d5fb3150 con 0x5593d605d860 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.366+0000 7f988aba9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d5fb0b10 con 0x5593d605cea0 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.367+0000 7f988aba9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d5fac400 con 0x5593d605c540 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.367+0000 7f987cf8c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605d860 0x5593d605dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.367+0000 7f987cf8c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605d860 0x5593d605dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43274/0 (socket says 172.21.15.33:43274) 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.367+0000 7f987cf8c700 1 -- 172.21.15.33:0/3469878433 learned_addr learned my addr 172.21.15.33:0/3469878433 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.367+0000 7f98777fe700 1 --2- 172.21.15.33:0/3469878433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605cea0 0x5593d605d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.367+0000 7f9877fff700 1 --2- 172.21.15.33:0/3469878433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 0x5593d605c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.368+0000 7f987cf8c700 1 -- 172.21.15.33:0/3469878433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 msgr2=0x5593d605c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.368+0000 7f98777fe700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f98600a11a0 0x7f98600a3660 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f986c009a10 tx=0x7f986c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.368+0000 7f987cf8c700 1 --2- 172.21.15.33:0/3469878433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 0x5593d605c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.368+0000 7f987cf8c700 1 -- 172.21.15.33:0/3469878433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605cea0 msgr2=0x5593d605d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.368+0000 7f987cf8c700 1 --2- 172.21.15.33:0/3469878433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605cea0 0x5593d605d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.368+0000 7f987cf8c700 1 -- 172.21.15.33:0/3469878433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593d5e2d3b0 con 0x5593d605d860 2024-05-08T21:25:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.368+0000 7f9877fff700 1 --2- 172.21.15.33:0/3469878433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 0x5593d605c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:25:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.368+0000 7f98777fe700 1 --2- 172.21.15.33:0/3469878433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605cea0 0x5593d605d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:25:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.368+0000 7f987cf8c700 1 --2- 172.21.15.33:0/3469878433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605d860 0x5593d605dce0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9870011680 tx=0x7f9870009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c32e51cd1fae9a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.370+0000 7f985e7fc700 1 -- 172.21.15.33:0/3469878433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987000e040 con 0x5593d605d860 2024-05-08T21:25:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.370+0000 7f985e7fc700 1 -- 172.21.15.33:0/3469878433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f987000f040 con 0x5593d605d860 2024-05-08T21:25:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.370+0000 7f985e7fc700 1 -- 172.21.15.33:0/3469878433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870004720 con 0x5593d605d860 2024-05-08T21:25:02.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.371+0000 7f988aba9b80 1 -- 172.21.15.33:0/3469878433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605d860 msgr2=0x5593d605dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.371+0000 7f988aba9b80 1 --2- 172.21.15.33:0/3469878433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605d860 0x5593d605dce0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9870011680 tx=0x7f9870009920 comp rx=0 tx=0).stop 2024-05-08T21:25:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.371+0000 7f988aba9b80 1 -- 172.21.15.33:0/3469878433 shutdown_connections 2024-05-08T21:25:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.371+0000 7f988aba9b80 1 --2- 172.21.15.33:0/3469878433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605d860 0x5593d605dce0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.371+0000 7f988aba9b80 1 --2- 172.21.15.33:0/3469878433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 0x5593d605c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.371+0000 7f988aba9b80 1 --2- 172.21.15.33:0/3469878433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605cea0 0x5593d605d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.371+0000 7f988aba9b80 1 -- 172.21.15.33:0/3469878433 >> 172.21.15.33:0/3469878433 conn(0x5593d5fba210 msgr2=0x5593d6056de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.371+0000 7f988aba9b80 1 -- 172.21.15.33:0/3469878433 shutdown_connections 2024-05-08T21:25:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.371+0000 7f988aba9b80 1 -- 172.21.15.33:0/3469878433 wait complete. 2024-05-08T21:25:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.371+0000 7f988aba9b80 1 -- start start 2024-05-08T21:25:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f988aba9b80 1 Processor -- start 2024-05-08T21:25:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f988aba9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605c540 0x5593d6076e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f988aba9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d6077380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f988aba9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605d860 0x5593d607e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f988aba9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d5f48560 con 0x5593d605cea0 2024-05-08T21:25:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f988aba9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d5f48200 con 0x5593d605c540 2024-05-08T21:25:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f988aba9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d5e2f6a0 con 0x5593d605d860 2024-05-08T21:25:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f988aba9b80 10 client.0 _ll_register_callbacks cb 0x5593d5f51b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f9877fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605c540 0x5593d6076e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f98777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d6077380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f9877fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605c540 0x5593d6076e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56492/0 (socket says 172.21.15.33:56492) 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f9877fff700 1 -- 172.21.15.33:0/118809030 learned_addr learned my addr 172.21.15.33:0/118809030 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.372+0000 7f987cf8c700 1 --2- 172.21.15.33:0/118809030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605d860 0x5593d607e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f9877fff700 1 -- 172.21.15.33:0/118809030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605d860 msgr2=0x5593d607e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f9877fff700 1 --2- 172.21.15.33:0/118809030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605d860 0x5593d607e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f9877fff700 1 -- 172.21.15.33:0/118809030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 msgr2=0x5593d6077380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f9877fff700 1 --2- 172.21.15.33:0/118809030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d6077380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f9877fff700 1 -- 172.21.15.33:0/118809030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9870011330 con 0x5593d605c540 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f9877fff700 1 --2- 172.21.15.33:0/118809030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605c540 0x5593d6076e40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f986809b0d0 tx=0x7f986800f8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d4ac689bb37682f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f98577fe700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f98577fe700 1 -- 172.21.15.33:0/118809030 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986809b5a0 con 0x5593d605c540 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f98577fe700 1 -- 172.21.15.33:0/118809030 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f986809b140 con 0x5593d605c540 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f988aba9b80 1 -- 172.21.15.33:0/118809030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593d5f05270 con 0x5593d605c540 2024-05-08T21:25:02.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f98577fe700 1 -- 172.21.15.33:0/118809030 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f986809ef70 con 0x5593d605c540 2024-05-08T21:25:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f988aba9b80 10 client.27083 fetch_fsmap learned FSMap version 37 2024-05-08T21:25:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f988aba9b80 1 -- 172.21.15.33:0/118809030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5593d5dd9b60 con 0x5593d605c540 2024-05-08T21:25:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.373+0000 7f98577fe700 1 -- 172.21.15.33:0/118809030 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98680b9020 con 0x5593d605c540 2024-05-08T21:25:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.374+0000 7f98577fe700 1 -- 172.21.15.33:0/118809030 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 37) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f98680a3130 con 0x5593d605c540 2024-05-08T21:25:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.374+0000 7f988aba9b80 10 client.27083 fetch_fsmap finished waiting for FSMap version 37 2024-05-08T21:25:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.374+0000 7f988aba9b80 10 client.27083 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:25:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.374+0000 7f988aba9b80 20 client.27083 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:02.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.374+0000 7f988aba9b80 1 --2- 172.21.15.33:0/118809030 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5593d60828d0 0x5593d6084cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.374+0000 7f988aba9b80 4 client.27083 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.374+0000 7f988aba9b80 1 -- 172.21.15.33:0/118809030 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593d602b810 con 0x5593d60828d0 2024-05-08T21:25:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.374+0000 7f98777fe700 1 --2- 172.21.15.33:0/118809030 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5593d60828d0 0x5593d6084cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.374+0000 7f98577fe700 10 client.27083 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:25:02.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.374+0000 7f98777fe700 1 --2- 172.21.15.33:0/118809030 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5593d60828d0 0x5593d6084cf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=83996f120e32a3bc server_cookie=dc18395004597d5 in_seq=0 out_seq=0 2024-05-08T21:25:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f98577fe700 1 -- 172.21.15.33:0/118809030 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5593d602b810 con 0x5593d60828d0 2024-05-08T21:25:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f98577fe700 10 client.27083 handle_command_reply: tid=0 2024-05-08T21:25:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f988aba9b80 1 client.27083 shutdown 2024-05-08T21:25:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f988aba9b80 1 -- 172.21.15.33:0/118809030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605c540 msgr2=0x5593d6076e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f988aba9b80 1 --2- 172.21.15.33:0/118809030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605c540 0x5593d6076e40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f986809b0d0 tx=0x7f986800f8b0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f988aba9b80 1 -- 172.21.15.33:0/118809030 shutdown_connections 2024-05-08T21:25:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f988aba9b80 1 --2- 172.21.15.33:0/118809030 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5593d60828d0 0x5593d6084cf0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f988aba9b80 1 --2- 172.21.15.33:0/118809030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605d860 0x5593d607e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f988aba9b80 1 --2- 172.21.15.33:0/118809030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605c540 0x5593d6076e40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f988aba9b80 1 --2- 172.21.15.33:0/118809030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d6077380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f98577fe700 0 client.27083 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:25:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f988aba9b80 1 -- 172.21.15.33:0/118809030 >> 172.21.15.33:0/118809030 conn(0x5593d5fba210 msgr2=0x5593d605b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f988aba9b80 1 -- 172.21.15.33:0/118809030 shutdown_connections 2024-05-08T21:25:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.375+0000 7f988aba9b80 1 -- 172.21.15.33:0/118809030 wait complete. 2024-05-08T21:25:02.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.376+0000 7f988aba9b80 20 client.27083 trim_cache size 0 max 16384 2024-05-08T21:25:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f988aba9b80 1 Processor -- start 2024-05-08T21:25:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f988aba9b80 1 -- start start 2024-05-08T21:25:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f988aba9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605c540 0x5593d607ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f988aba9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d607e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f988aba9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605d860 0x5593d60763c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f988aba9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d6078030 con 0x5593d605cea0 2024-05-08T21:25:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f988aba9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d6089ee0 con 0x5593d605c540 2024-05-08T21:25:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f988aba9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d604f080 con 0x5593d605d860 2024-05-08T21:25:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f98777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d607e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f987cf8c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605d860 0x5593d60763c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f98777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d607e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43292/0 (socket says 172.21.15.33:43292) 2024-05-08T21:25:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f98777fe700 1 -- 172.21.15.33:0/3896297403 learned_addr learned my addr 172.21.15.33:0/3896297403 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f987cf8c700 1 -- 172.21.15.33:0/3896297403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605c540 msgr2=0x5593d607ea70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:25:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f987cf8c700 1 --2- 172.21.15.33:0/3896297403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605c540 0x5593d607ea70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f987cf8c700 1 -- 172.21.15.33:0/3896297403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 msgr2=0x5593d607e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f987cf8c700 1 --2- 172.21.15.33:0/3896297403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d607e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.393+0000 7f987cf8c700 1 -- 172.21.15.33:0/3896297403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593d5f72560 con 0x5593d605d860 2024-05-08T21:25:02.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.394+0000 7f987cf8c700 1 --2- 172.21.15.33:0/3896297403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605d860 0x5593d60763c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f987000b010 tx=0x7f98700060d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faa48da6a8113a82 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.394+0000 7f985effd700 1 -- 172.21.15.33:0/3896297403 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987000e040 con 0x5593d605d860 2024-05-08T21:25:02.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.394+0000 7f985effd700 1 -- 172.21.15.33:0/3896297403 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9870003e90 con 0x5593d605d860 2024-05-08T21:25:02.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.394+0000 7f985effd700 1 -- 172.21.15.33:0/3896297403 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870005990 con 0x5593d605d860 2024-05-08T21:25:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.395+0000 7f988aba9b80 1 -- 172.21.15.33:0/3896297403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605d860 msgr2=0x5593d60763c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.395+0000 7f988aba9b80 1 --2- 172.21.15.33:0/3896297403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605d860 0x5593d60763c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f987000b010 tx=0x7f98700060d0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.395+0000 7f988aba9b80 1 -- 172.21.15.33:0/3896297403 shutdown_connections 2024-05-08T21:25:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.395+0000 7f988aba9b80 1 --2- 172.21.15.33:0/3896297403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605d860 0x5593d60763c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.395+0000 7f988aba9b80 1 --2- 172.21.15.33:0/3896297403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605c540 0x5593d607ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.395+0000 7f988aba9b80 1 --2- 172.21.15.33:0/3896297403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d607e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.395+0000 7f988aba9b80 1 -- 172.21.15.33:0/3896297403 >> 172.21.15.33:0/3896297403 conn(0x5593d5fba210 msgr2=0x5593d605dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.395+0000 7f988aba9b80 1 -- 172.21.15.33:0/3896297403 shutdown_connections 2024-05-08T21:25:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.395+0000 7f988aba9b80 1 -- 172.21.15.33:0/3896297403 wait complete. 2024-05-08T21:25:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.395+0000 7f988aba9b80 1 -- start start 2024-05-08T21:25:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f988aba9b80 1 Processor -- start 2024-05-08T21:25:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f988aba9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 0x5593d607a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f988aba9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d607ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f988aba9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605d860 0x5593d607d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f988aba9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d607dd50 con 0x5593d605cea0 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f988aba9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d607dec0 con 0x5593d605d860 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f988aba9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d607e080 con 0x5593d605c540 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f988aba9b80 10 client.0 _ll_register_callbacks cb 0x5593d5f51b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f9877fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 0x5593d607a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f9877fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 0x5593d607a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:50032/0 (socket says 172.21.15.33:50032) 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f9877fff700 1 -- 172.21.15.33:0/1013185087 learned_addr learned my addr 172.21.15.33:0/1013185087 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f98777fe700 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d607ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f987cf8c700 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605d860 0x5593d607d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f9877fff700 1 -- 172.21.15.33:0/1013185087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605d860 msgr2=0x5593d607d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f9877fff700 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605d860 0x5593d607d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f9877fff700 1 -- 172.21.15.33:0/1013185087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 msgr2=0x5593d607ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f9877fff700 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d607ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.396+0000 7f9877fff700 1 -- 172.21.15.33:0/1013185087 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593d5e2d3b0 con 0x5593d605c540 2024-05-08T21:25:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f9877fff700 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 0x5593d607a580 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f98680a3000 tx=0x7f986800bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93bd1759779def75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f98577fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f98577fe700 1 -- 172.21.15.33:0/1013185087 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868003a80 con 0x5593d605c540 2024-05-08T21:25:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f98577fe700 1 -- 172.21.15.33:0/1013185087 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9868097280 con 0x5593d605c540 2024-05-08T21:25:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f988aba9b80 1 -- 172.21.15.33:0/1013185087 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f987000f040 con 0x5593d605c540 2024-05-08T21:25:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f98577fe700 1 -- 172.21.15.33:0/1013185087 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9868097480 con 0x5593d605c540 2024-05-08T21:25:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f988aba9b80 10 client.27094 fetch_fsmap learned FSMap version 37 2024-05-08T21:25:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f988aba9b80 1 -- 172.21.15.33:0/1013185087 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5593d5e31cd0 con 0x5593d605c540 2024-05-08T21:25:02.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f98577fe700 1 -- 172.21.15.33:0/1013185087 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f987000f040 con 0x5593d605c540 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f98577fe700 1 -- 172.21.15.33:0/1013185087 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 37) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f986809b3a0 con 0x5593d605c540 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f988aba9b80 10 client.27094 fetch_fsmap finished waiting for FSMap version 37 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f988aba9b80 10 client.27094 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f988aba9b80 20 client.27094 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f988aba9b80 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5593d60d98c0 0x5593d60dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f988aba9b80 4 client.27094 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.397+0000 7f988aba9b80 1 -- 172.21.15.33:0/1013185087 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5593d5fa4ec0 con 0x5593d60d98c0 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.398+0000 7f98777fe700 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5593d60d98c0 0x5593d60dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.398+0000 7f98577fe700 10 client.27094 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.398+0000 7f98777fe700 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5593d60d98c0 0x5593d60dbcb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e05aa19cbae98501 server_cookie=97b45ba225090ba1 in_seq=0 out_seq=0 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.398+0000 7f98577fe700 1 -- 172.21.15.33:0/1013185087 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5593d5fa4ec0 con 0x5593d60d98c0 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.398+0000 7f98577fe700 10 client.27094 handle_command_reply: tid=0 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.398+0000 7f988aba9b80 1 client.27094 shutdown 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.399+0000 7f988aba9b80 1 -- 172.21.15.33:0/1013185087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 msgr2=0x5593d607a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.399+0000 7f988aba9b80 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 0x5593d607a580 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f98680a3000 tx=0x7f986800bf90 comp rx=0 tx=0).stop 2024-05-08T21:25:02.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.399+0000 7f988aba9b80 1 -- 172.21.15.33:0/1013185087 shutdown_connections 2024-05-08T21:25:02.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.399+0000 7f988aba9b80 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5593d60d98c0 0x5593d60dbcb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.399+0000 7f988aba9b80 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5593d605d860 0x5593d607d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.399+0000 7f988aba9b80 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5593d605c540 0x5593d607a580 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.399+0000 7f988aba9b80 1 --2- 172.21.15.33:0/1013185087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d605cea0 0x5593d607ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.399+0000 7f988aba9b80 1 -- 172.21.15.33:0/1013185087 >> 172.21.15.33:0/1013185087 conn(0x5593d5fba210 msgr2=0x5593d60763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.399+0000 7f98577fe700 0 client.27094 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:25:02.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.399+0000 7f988aba9b80 1 -- 172.21.15.33:0/1013185087 shutdown_connections 2024-05-08T21:25:02.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.399+0000 7f988aba9b80 1 -- 172.21.15.33:0/1013185087 wait complete. 2024-05-08T21:25:02.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.399+0000 7f988aba9b80 20 client.27094 trim_cache size 0 max 16384 2024-05-08T21:25:02.406 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:25:02.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.400+0000 7f987e9ef700 1 -- 172.21.15.33:0/611682455 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f98600a11a0 msgr2=0x7f98600a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.400+0000 7f987e9ef700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f98600a11a0 0x7f98600a3660 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f986c009a10 tx=0x7f986c009190 comp rx=0 tx=0).stop 2024-05-08T21:25:02.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.400+0000 7f987e9ef700 1 -- 172.21.15.33:0/611682455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98780ffd60 msgr2=0x7f987810a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.400+0000 7f987e9ef700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98780ffd60 0x7f987810a620 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f98680095c0 tx=0x7f986800b9c0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.400+0000 7f987e9ef700 1 -- 172.21.15.33:0/611682455 shutdown_connections 2024-05-08T21:25:02.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.400+0000 7f987e9ef700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98781018f0 0x7f98781091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.400+0000 7f987e9ef700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f98600a11a0 0x7f98600a3660 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.400+0000 7f987e9ef700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98780ffd60 0x7f987810a620 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.400+0000 7f987e9ef700 1 --2- 172.21.15.33:0/611682455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878100f50 0x7f9878108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.400+0000 7f987e9ef700 1 -- 172.21.15.33:0/611682455 >> 172.21.15.33:0/611682455 conn(0x7f98780fb290 msgr2=0x7f98781043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.400+0000 7f987e9ef700 1 -- 172.21.15.33:0/611682455 shutdown_connections 2024-05-08T21:25:02.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.400+0000 7f987e9ef700 1 -- 172.21.15.33:0/611682455 wait complete. 2024-05-08T21:25:02.418 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:25:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.586+0000 7fbfa98f1700 1 -- 172.21.15.33:0/3807056029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa4100f50 msgr2=0x7fbfa41013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.586+0000 7fbfa98f1700 1 --2- 172.21.15.33:0/3807056029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa4100f50 0x7fbfa41013b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbf98009990 tx=0x7fbf98009ca0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.586+0000 7fbfa98f1700 1 -- 172.21.15.33:0/3807056029 shutdown_connections 2024-05-08T21:25:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.586+0000 7fbfa98f1700 1 --2- 172.21.15.33:0/3807056029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa41018f0 0x7fbfa41081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.586+0000 7fbfa98f1700 1 --2- 172.21.15.33:0/3807056029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa40ffd60 0x7fbfa4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.586+0000 7fbfa98f1700 1 --2- 172.21.15.33:0/3807056029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa4100f50 0x7fbfa41013b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.586+0000 7fbfa98f1700 1 -- 172.21.15.33:0/3807056029 >> 172.21.15.33:0/3807056029 conn(0x7fbfa40fb290 msgr2=0x7fbfa40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.586+0000 7fbfa98f1700 1 -- 172.21.15.33:0/3807056029 shutdown_connections 2024-05-08T21:25:02.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.586+0000 7fbfa98f1700 1 -- 172.21.15.33:0/3807056029 wait complete. 2024-05-08T21:25:02.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.587+0000 7fbfa98f1700 1 Processor -- start 2024-05-08T21:25:02.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.587+0000 7fbfa98f1700 1 -- start start 2024-05-08T21:25:02.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.587+0000 7fbfa98f1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa40ffd60 0x7fbfa4103e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa98f1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa4100f50 0x7fbfa4102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa98f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa41018f0 0x7fbfa4102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa98f1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa41031a0 con 0x7fbfa41018f0 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa98f1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4103310 con 0x7fbfa4100f50 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa98f1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa41a0e30 con 0x7fbfa40ffd60 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa27fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa4100f50 0x7fbfa4102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa27fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa4100f50 0x7fbfa4102560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56544/0 (socket says 172.21.15.33:56544) 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa27fc700 1 -- 172.21.15.33:0/1826362573 learned_addr learned my addr 172.21.15.33:0/1826362573 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa2ffd700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa40ffd60 0x7fbfa4103e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa27fc700 1 -- 172.21.15.33:0/1826362573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa40ffd60 msgr2=0x7fbfa4103e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa40ffd60 0x7fbfa4103e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa27fc700 1 -- 172.21.15.33:0/1826362573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa41018f0 msgr2=0x7fbfa4102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa41018f0 0x7fbfa4102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.588+0000 7fbfa27fc700 1 -- 172.21.15.33:0/1826362573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf94009520 con 0x7fbfa4100f50 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.589+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa4100f50 0x7fbfa4102560 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800b2e0 tx=0x7fbf9800ba70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.589+0000 7fbfa88ef700 1 -- 172.21.15.33:0/1826362573 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9801d070 con 0x7fbfa4100f50 2024-05-08T21:25:02.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.589+0000 7fbfa88ef700 1 -- 172.21.15.33:0/1826362573 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf98003d20 con 0x7fbfa4100f50 2024-05-08T21:25:02.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.589+0000 7fbfa88ef700 1 -- 172.21.15.33:0/1826362573 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800fe50 con 0x7fbfa4100f50 2024-05-08T21:25:02.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.589+0000 7fbfa98f1700 1 -- 172.21.15.33:0/1826362573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c009c40 con 0x7fbfa4100f50 2024-05-08T21:25:02.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.589+0000 7fbfa98f1700 1 -- 172.21.15.33:0/1826362573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf980095f0 con 0x7fbfa4100f50 2024-05-08T21:25:02.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.591+0000 7fbfa88ef700 1 -- 172.21.15.33:0/1826362573 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbf98003890 con 0x7fbfa4100f50 2024-05-08T21:25:02.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.592+0000 7fbfa88ef700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbf8c0a11a0 0x7fbf8c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.592+0000 7fbfa88ef700 1 -- 172.21.15.33:0/1826362573 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbf980cded0 con 0x7fbfa4100f50 2024-05-08T21:25:02.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.592+0000 7fbfa2ffd700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbf8c0a11a0 0x7fbf8c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.593+0000 7fbfa2ffd700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbf8c0a11a0 0x7fbf8c0a3660 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4100db0 tx=0x7fbf94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.593+0000 7fbfb5aabb80 1 Processor -- start 2024-05-08T21:25:02.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.593+0000 7fbfb5aabb80 1 -- start start 2024-05-08T21:25:02.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfb5aabb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b7d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfb5aabb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b7e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfb5aabb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7e860 0x561ea7b7ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfb5aabb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ea7ad4150 con 0x561ea7b7dea0 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfb5aabb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ea7ad1b10 con 0x561ea7b7e860 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfb5aabb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ea7acd400 con 0x561ea7b7d540 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfa27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b7e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfa2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b7d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfa27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b7e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43344/0 (socket says 172.21.15.33:43344) 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7e860 0x561ea7b7ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfa2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b7d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:50072/0 (socket says 172.21.15.33:50072) 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfa27fc700 1 -- 172.21.15.33:0/3548314229 learned_addr learned my addr 172.21.15.33:0/3548314229 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfa27fc700 1 -- 172.21.15.33:0/3548314229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 msgr2=0x561ea7b7d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/3548314229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b7d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfa27fc700 1 -- 172.21.15.33:0/3548314229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7e860 msgr2=0x561ea7b7ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/3548314229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7e860 0x561ea7b7ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfa27fc700 1 -- 172.21.15.33:0/3548314229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ea794e3b0 con 0x561ea7b7dea0 2024-05-08T21:25:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.594+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/3548314229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b7e320 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbf980a2ef0 tx=0x7fbf98009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02267b9c5098fc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.595+0000 7fbf89ffb700 1 -- 172.21.15.33:0/3548314229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9809b730 con 0x561ea7b7dea0 2024-05-08T21:25:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.595+0000 7fbf89ffb700 1 -- 172.21.15.33:0/3548314229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf9809b030 con 0x561ea7b7dea0 2024-05-08T21:25:02.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.595+0000 7fbf89ffb700 1 -- 172.21.15.33:0/3548314229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9809a780 con 0x561ea7b7dea0 2024-05-08T21:25:02.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.596+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/3548314229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 msgr2=0x561ea7b7e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.596+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/3548314229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b7e320 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fbf980a2ef0 tx=0x7fbf98009ca0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.596+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/3548314229 shutdown_connections 2024-05-08T21:25:02.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.596+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/3548314229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7e860 0x561ea7b7ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.596+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/3548314229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b7d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.596+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/3548314229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b7e320 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.596+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/3548314229 >> 172.21.15.33:0/3548314229 conn(0x561ea7adb210 msgr2=0x561ea7b77de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.596+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/3548314229 shutdown_connections 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.596+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/3548314229 wait complete. 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.596+0000 7fbfb5aabb80 1 -- start start 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.596+0000 7fbfb5aabb80 1 Processor -- start 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfb5aabb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b86ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfb5aabb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7dea0 0x561ea7b87010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfb5aabb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7e860 0x561ea7b8e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfb5aabb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ea7a69560 con 0x561ea7b7e860 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfb5aabb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ea7a69200 con 0x561ea7b7dea0 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfb5aabb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ea79506a0 con 0x561ea7b7d540 2024-05-08T21:25:02.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfb5aabb80 10 client.0 _ll_register_callbacks cb 0x561ea7a72b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa27fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7dea0 0x561ea7b87010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b86ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7e860 0x561ea7b8e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa27fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7dea0 0x561ea7b87010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56564/0 (socket says 172.21.15.33:56564) 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b86ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:50076/0 (socket says 172.21.15.33:50076) 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa27fc700 1 -- 172.21.15.33:0/2117311401 learned_addr learned my addr 172.21.15.33:0/2117311401 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa2ffd700 1 -- 172.21.15.33:0/2117311401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7dea0 msgr2=0x561ea7b87010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa2ffd700 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7dea0 0x561ea7b87010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa2ffd700 1 -- 172.21.15.33:0/2117311401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7e860 msgr2=0x561ea7b8e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa2ffd700 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7e860 0x561ea7b8e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa2ffd700 1 -- 172.21.15.33:0/2117311401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf980095f0 con 0x561ea7b7d540 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7dea0 0x561ea7b87010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa37fe700 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7e860 0x561ea7b8e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbfa2ffd700 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b86ad0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009880 tx=0x7fbf940047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc5a9d8b50d3c1d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.597+0000 7fbf82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbf82ffd700 1 -- 172.21.15.33:0/2117311401 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94020840 con 0x561ea7b7d540 2024-05-08T21:25:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/2117311401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ea7a26270 con 0x561ea7b7d540 2024-05-08T21:25:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbf82ffd700 1 -- 172.21.15.33:0/2117311401 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbf94018070 con 0x561ea7b7d540 2024-05-08T21:25:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbf82ffd700 1 -- 172.21.15.33:0/2117311401 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf940209a0 con 0x561ea7b7d540 2024-05-08T21:25:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbfb5aabb80 10 client.27106 fetch_fsmap learned FSMap version 37 2024-05-08T21:25:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/2117311401 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x561ea78fab60 con 0x561ea7b7d540 2024-05-08T21:25:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbf82ffd700 1 -- 172.21.15.33:0/2117311401 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561ea7a26270 con 0x561ea7b7d540 2024-05-08T21:25:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbf82ffd700 1 -- 172.21.15.33:0/2117311401 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 37) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fbf94029a20 con 0x561ea7b7d540 2024-05-08T21:25:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbfb5aabb80 10 client.27106 fetch_fsmap finished waiting for FSMap version 37 2024-05-08T21:25:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbfb5aabb80 10 client.27106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbfb5aabb80 20 client.27106 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561ea7b928e0 0x561ea7b94d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbfb5aabb80 4 client.27106 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/2117311401 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ea7b4c810 con 0x561ea7b928e0 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.598+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561ea7b928e0 0x561ea7b94d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.599+0000 7fbf82ffd700 10 client.27106 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.599+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561ea7b928e0 0x561ea7b94d00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67465f43dd18c630 server_cookie=83388a4fdd20fb9f in_seq=0 out_seq=0 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.599+0000 7fbf82ffd700 1 -- 172.21.15.33:0/2117311401 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ea7b4c810 con 0x561ea7b928e0 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.599+0000 7fbf82ffd700 10 client.27106 handle_command_reply: tid=0 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.599+0000 7fbfb5aabb80 1 client.27106 shutdown 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.600+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/2117311401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 msgr2=0x561ea7b86ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.600+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b86ad0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009880 tx=0x7fbf940047c0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.600+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/2117311401 shutdown_connections 2024-05-08T21:25:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.600+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561ea7b928e0 0x561ea7b94d00 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.600+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7e860 0x561ea7b8e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.600+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b86ad0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.600+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/2117311401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7dea0 0x561ea7b87010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.600+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/2117311401 >> 172.21.15.33:0/2117311401 conn(0x561ea7adb210 msgr2=0x561ea7b775a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.600+0000 7fbf82ffd700 0 client.27106 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.600+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/2117311401 shutdown_connections 2024-05-08T21:25:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.600+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/2117311401 wait complete. 2024-05-08T21:25:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.600+0000 7fbfb5aabb80 20 client.27106 trim_cache size 0 max 16384 2024-05-08T21:25:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.619+0000 7fbfb5aabb80 1 Processor -- start 2024-05-08T21:25:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.619+0000 7fbfb5aabb80 1 -- start start 2024-05-08T21:25:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.619+0000 7fbfb5aabb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7d540 0x561ea7b85fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.619+0000 7fbfb5aabb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b8dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.619+0000 7fbfb5aabb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7e860 0x561ea7b817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.619+0000 7fbfb5aabb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ea7b79570 con 0x561ea7b7dea0 2024-05-08T21:25:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.619+0000 7fbfb5aabb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ea7b99ef0 con 0x561ea7b7d540 2024-05-08T21:25:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.619+0000 7fbfb5aabb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ea7b8fdb0 con 0x561ea7b7e860 2024-05-08T21:25:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.619+0000 7fbfa27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b8dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.619+0000 7fbfa27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b8dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43352/0 (socket says 172.21.15.33:43352) 2024-05-08T21:25:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.619+0000 7fbfa27fc700 1 -- 172.21.15.33:0/1634981713 learned_addr learned my addr 172.21.15.33:0/1634981713 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.619+0000 7fbfa37fe700 1 --2- 172.21.15.33:0/1634981713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7e860 0x561ea7b817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.620+0000 7fbfa2ffd700 1 --2- 172.21.15.33:0/1634981713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7d540 0x561ea7b85fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.620+0000 7fbfa27fc700 1 -- 172.21.15.33:0/1634981713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7e860 msgr2=0x561ea7b817d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.620+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/1634981713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7e860 0x561ea7b817d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.620+0000 7fbfa27fc700 1 -- 172.21.15.33:0/1634981713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7d540 msgr2=0x561ea7b85fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.620+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/1634981713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7d540 0x561ea7b85fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.620+0000 7fbfa27fc700 1 -- 172.21.15.33:0/1634981713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ea7a93560 con 0x561ea7b7dea0 2024-05-08T21:25:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.620+0000 7fbfa2ffd700 1 --2- 172.21.15.33:0/1634981713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7d540 0x561ea7b85fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.620+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/1634981713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b8dfe0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbf980058b0 tx=0x7fbf98003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e61166856c4878 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.620+0000 7fbf8a7fc700 1 -- 172.21.15.33:0/1634981713 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf980993d0 con 0x561ea7b7dea0 2024-05-08T21:25:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.620+0000 7fbf8a7fc700 1 -- 172.21.15.33:0/1634981713 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf980a3420 con 0x561ea7b7dea0 2024-05-08T21:25:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.620+0000 7fbf8a7fc700 1 -- 172.21.15.33:0/1634981713 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf980a31f0 con 0x561ea7b7dea0 2024-05-08T21:25:02.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.621+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/1634981713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 msgr2=0x561ea7b8dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.621+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/1634981713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b8dfe0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbf980058b0 tx=0x7fbf98003e90 comp rx=0 tx=0).stop 2024-05-08T21:25:02.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.621+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/1634981713 shutdown_connections 2024-05-08T21:25:02.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.621+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/1634981713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7e860 0x561ea7b817d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.621+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/1634981713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7d540 0x561ea7b85fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.621+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/1634981713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b8dfe0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.621+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/1634981713 >> 172.21.15.33:0/1634981713 conn(0x561ea7adb210 msgr2=0x561ea7b84e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.621+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/1634981713 shutdown_connections 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.621+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/1634981713 wait complete. 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfb5aabb80 1 -- start start 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfb5aabb80 1 Processor -- start 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfb5aabb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b89150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfb5aabb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b89690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfb5aabb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7e860 0x561ea7b8c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfb5aabb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ea7b8cbf0 con 0x561ea7b7dea0 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfb5aabb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ea7b8cd60 con 0x561ea7b7e860 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfb5aabb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ea7b8cf20 con 0x561ea7b7d540 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfb5aabb80 10 client.0 _ll_register_callbacks cb 0x561ea7a72b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfa2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b89150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfa27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b89690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfa2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b89150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:50088/0 (socket says 172.21.15.33:50088) 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.622+0000 7fbfa2ffd700 1 -- 172.21.15.33:0/553670034 learned_addr learned my addr 172.21.15.33:0/553670034 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbfa37fe700 1 --2- 172.21.15.33:0/553670034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7e860 0x561ea7b8c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbfa2ffd700 1 -- 172.21.15.33:0/553670034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7e860 msgr2=0x561ea7b8c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbfa2ffd700 1 --2- 172.21.15.33:0/553670034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7e860 0x561ea7b8c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbfa2ffd700 1 -- 172.21.15.33:0/553670034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 msgr2=0x561ea7b89690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbfa2ffd700 1 --2- 172.21.15.33:0/553670034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b89690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbfa2ffd700 1 -- 172.21.15.33:0/553670034 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9402e040 con 0x561ea7b7d540 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbfa2ffd700 1 --2- 172.21.15.33:0/553670034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b89150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbf94029d80 tx=0x7fbf94004ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0b42f012825cb52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbf82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbf82ffd700 1 -- 172.21.15.33:0/553670034 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94005c90 con 0x561ea7b7d540 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbf82ffd700 1 -- 172.21.15.33:0/553670034 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbf94018040 con 0x561ea7b7d540 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/553670034 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf9809b030 con 0x561ea7b7d540 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbf82ffd700 1 -- 172.21.15.33:0/553670034 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf9401f430 con 0x561ea7b7d540 2024-05-08T21:25:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbfb5aabb80 10 client.27112 fetch_fsmap learned FSMap version 37 2024-05-08T21:25:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.623+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/553670034 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7fbf9807fa90 con 0x561ea7b7d540 2024-05-08T21:25:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.624+0000 7fbf82ffd700 1 -- 172.21.15.33:0/553670034 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf9809b030 con 0x561ea7b7d540 2024-05-08T21:25:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.624+0000 7fbf82ffd700 1 -- 172.21.15.33:0/553670034 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 37) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fbf94005040 con 0x561ea7b7d540 2024-05-08T21:25:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.624+0000 7fbfb5aabb80 10 client.27112 fetch_fsmap finished waiting for FSMap version 37 2024-05-08T21:25:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.624+0000 7fbfb5aabb80 10 client.27112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.624+0000 7fbfb5aabb80 20 client.27112 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.624+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/553670034 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561ea7be9240 0x561ea7beb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.624+0000 7fbfb5aabb80 4 client.27112 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:25:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.624+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/553670034 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561ea7ac5ec0 con 0x561ea7be9240 2024-05-08T21:25:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.624+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/553670034 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561ea7be9240 0x561ea7beb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfa27fc700 1 --2- 172.21.15.33:0/553670034 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561ea7be9240 0x561ea7beb660 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acbb058a21e7f642 server_cookie=19639466cbb9f678 in_seq=0 out_seq=0 2024-05-08T21:25:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbf82ffd700 10 client.27112 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbf82ffd700 1 -- 172.21.15.33:0/553670034 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561ea7ac5ec0 con 0x561ea7be9240 2024-05-08T21:25:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbf82ffd700 10 client.27112 handle_command_reply: tid=0 2024-05-08T21:25:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfb5aabb80 1 client.27112 shutdown 2024-05-08T21:25:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/553670034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 msgr2=0x561ea7b89150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/553670034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b89150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbf94029d80 tx=0x7fbf94004ae0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/553670034 shutdown_connections 2024-05-08T21:25:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/553670034 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561ea7be9240 0x561ea7beb660 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/553670034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561ea7b7e860 0x561ea7b8c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbf82ffd700 0 client.27112 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/553670034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ea7b7d540 0x561ea7b89150 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfb5aabb80 1 --2- 172.21.15.33:0/553670034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561ea7b7dea0 0x561ea7b89690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/553670034 >> 172.21.15.33:0/553670034 conn(0x561ea7adb210 msgr2=0x561ea7b817d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/553670034 shutdown_connections 2024-05-08T21:25:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfb5aabb80 1 -- 172.21.15.33:0/553670034 wait complete. 2024-05-08T21:25:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.625+0000 7fbfb5aabb80 20 client.27112 trim_cache size 0 max 16384 2024-05-08T21:25:02.633 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:25:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.626+0000 7fbfa98f1700 1 -- 172.21.15.33:0/1826362573 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbf8c0a11a0 msgr2=0x7fbf8c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.626+0000 7fbfa98f1700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbf8c0a11a0 0x7fbf8c0a3660 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4100db0 tx=0x7fbf94009260 comp rx=0 tx=0).stop 2024-05-08T21:25:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.626+0000 7fbfa98f1700 1 -- 172.21.15.33:0/1826362573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa4100f50 msgr2=0x7fbfa4102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.626+0000 7fbfa98f1700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa4100f50 0x7fbfa4102560 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbf9800b2e0 tx=0x7fbf9800ba70 comp rx=0 tx=0).stop 2024-05-08T21:25:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.626+0000 7fbfa98f1700 1 -- 172.21.15.33:0/1826362573 shutdown_connections 2024-05-08T21:25:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.626+0000 7fbfa98f1700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa41018f0 0x7fbfa4102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.626+0000 7fbfa98f1700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa40ffd60 0x7fbfa4103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.626+0000 7fbfa98f1700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbf8c0a11a0 0x7fbf8c0a3660 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.626+0000 7fbfa98f1700 1 --2- 172.21.15.33:0/1826362573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa4100f50 0x7fbfa4102560 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.626+0000 7fbfa98f1700 1 -- 172.21.15.33:0/1826362573 >> 172.21.15.33:0/1826362573 conn(0x7fbfa40fb290 msgr2=0x7fbfa4068160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.626+0000 7fbfa98f1700 1 -- 172.21.15.33:0/1826362573 shutdown_connections 2024-05-08T21:25:02.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:02.626+0000 7fbfa98f1700 1 -- 172.21.15.33:0/1826362573 wait complete. 2024-05-08T21:25:02.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:02 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/826556690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:25:02.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:02 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/826556690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:25:03.644 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:25:03.644 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:25:03.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:03 smithi033 ceph-mon[42528]: pgmap v472: 129 pgs: 129 active+clean; 219 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 562 op/s 2024-05-08T21:25:03.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:03 smithi071 ceph-mon[47074]: pgmap v472: 129 pgs: 129 active+clean; 219 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 562 op/s 2024-05-08T21:25:03.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.821+0000 7f8f4d407700 1 -- 172.21.15.33:0/1235036276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48104240 msgr2=0x7f8f48106630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.821+0000 7f8f4d407700 1 --2- 172.21.15.33:0/1235036276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48104240 0x7f8f48106630 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009990 tx=0x7f8f3c009ca0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.821+0000 7f8f4d407700 1 -- 172.21.15.33:0/1235036276 shutdown_connections 2024-05-08T21:25:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.821+0000 7f8f4d407700 1 --2- 172.21.15.33:0/1235036276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f48106b70 0x7f8f48108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.821+0000 7f8f4d407700 1 --2- 172.21.15.33:0/1235036276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f48101910 0x7f8f48103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.821+0000 7f8f4d407700 1 --2- 172.21.15.33:0/1235036276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48104240 0x7f8f48106630 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.821+0000 7f8f4d407700 1 -- 172.21.15.33:0/1235036276 >> 172.21.15.33:0/1235036276 conn(0x7f8f480fb300 msgr2=0x7f8f480fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.821+0000 7f8f4d407700 1 -- 172.21.15.33:0/1235036276 shutdown_connections 2024-05-08T21:25:03.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.821+0000 7f8f4d407700 1 -- 172.21.15.33:0/1235036276 wait complete. 2024-05-08T21:25:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.822+0000 7f8f4d407700 1 Processor -- start 2024-05-08T21:25:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.822+0000 7f8f4d407700 1 -- start start 2024-05-08T21:25:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.822+0000 7f8f4d407700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f48101910 0x7f8f48101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.822+0000 7f8f4d407700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48104240 0x7f8f480ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.822+0000 7f8f4d407700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f48106b70 0x7f8f480ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.822+0000 7f8f4d407700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f481005a0 con 0x7f8f48104240 2024-05-08T21:25:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.822+0000 7f8f4d407700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48100710 con 0x7f8f48101910 2024-05-08T21:25:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.822+0000 7f8f4d407700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f481a0d40 con 0x7f8f48106b70 2024-05-08T21:25:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f477fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f48106b70 0x7f8f480ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48104240 0x7f8f480ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f477fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f48106b70 0x7f8f480ffe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:50108/0 (socket says 172.21.15.33:50108) 2024-05-08T21:25:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48104240 0x7f8f480ff8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43390/0 (socket says 172.21.15.33:43390) 2024-05-08T21:25:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f477fe700 1 -- 172.21.15.33:0/3308544262 learned_addr learned my addr 172.21.15.33:0/3308544262 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f48101910 0x7f8f48101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f477fe700 1 -- 172.21.15.33:0/3308544262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f48101910 msgr2=0x7f8f48101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f477fe700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f48101910 0x7f8f48101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f477fe700 1 -- 172.21.15.33:0/3308544262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48104240 msgr2=0x7f8f480ff8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f477fe700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48104240 0x7f8f480ff8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f477fe700 1 -- 172.21.15.33:0/3308544262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c0095f0 con 0x7f8f48106b70 2024-05-08T21:25:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f477fe700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f48106b70 0x7f8f480ffe00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000ef30 tx=0x7f8f4000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f2ffff700 1 -- 172.21.15.33:0/3308544262 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40018070 con 0x7f8f48106b70 2024-05-08T21:25:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f2ffff700 1 -- 172.21.15.33:0/3308544262 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f4000f040 con 0x7f8f48106b70 2024-05-08T21:25:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f4d407700 1 -- 172.21.15.33:0/3308544262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f481a0f70 con 0x7f8f48106b70 2024-05-08T21:25:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.823+0000 7f8f2ffff700 1 -- 172.21.15.33:0/3308544262 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000cbc0 con 0x7f8f48106b70 2024-05-08T21:25:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.824+0000 7f8f4d407700 1 -- 172.21.15.33:0/3308544262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f481a1340 con 0x7f8f48106b70 2024-05-08T21:25:03.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.826+0000 7f8f2ffff700 1 -- 172.21.15.33:0/3308544262 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8f40003a00 con 0x7f8f48106b70 2024-05-08T21:25:03.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.827+0000 7f8f2ffff700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8f300a1120 0x7f8f300a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.827+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8f300a1120 0x7f8f300a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.827+0000 7f8f2ffff700 1 -- 172.21.15.33:0/3308544262 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8f400cc7e0 con 0x7f8f48106b70 2024-05-08T21:25:03.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.827+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8f300a1120 0x7f8f300a35e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009c60 tx=0x7f8f38009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.828+0000 7f8f595c1b80 1 Processor -- start 2024-05-08T21:25:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.828+0000 7f8f595c1b80 1 -- start start 2024-05-08T21:25:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.828+0000 7f8f595c1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 0x556d7cfc99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.828+0000 7f8f595c1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.828+0000 7f8f595c1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfca8e0 0x556d7cfcad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.828+0000 7f8f595c1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7cf201d0 con 0x556d7cfc9f20 2024-05-08T21:25:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.828+0000 7f8f595c1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7cf1db90 con 0x556d7cfca8e0 2024-05-08T21:25:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.828+0000 7f8f595c1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7cf19480 con 0x556d7cfc95c0 2024-05-08T21:25:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f46ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 0x556d7cfc99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f46ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 0x556d7cfc99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:50122/0 (socket says 172.21.15.33:50122) 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f46ffd700 1 -- 172.21.15.33:0/549220040 learned_addr learned my addr 172.21.15.33:0/549220040 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f467fc700 1 --2- 172.21.15.33:0/549220040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f477fe700 1 --2- 172.21.15.33:0/549220040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfca8e0 0x556d7cfcad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f46ffd700 1 -- 172.21.15.33:0/549220040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfca8e0 msgr2=0x556d7cfcad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/549220040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfca8e0 0x556d7cfcad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f46ffd700 1 -- 172.21.15.33:0/549220040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 msgr2=0x556d7cfca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/549220040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f46ffd700 1 -- 172.21.15.33:0/549220040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7cd9a430 con 0x556d7cfc95c0 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f477fe700 1 --2- 172.21.15.33:0/549220040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfca8e0 0x556d7cfcad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/549220040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 0x556d7cfc99e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800e510 tx=0x7f8f38014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4093993fb21713b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f2d7fa700 1 -- 172.21.15.33:0/549220040 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3800ee30 con 0x556d7cfc95c0 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f2d7fa700 1 -- 172.21.15.33:0/549220040 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f38021070 con 0x556d7cfc95c0 2024-05-08T21:25:03.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.829+0000 7f8f2d7fa700 1 -- 172.21.15.33:0/549220040 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38025700 con 0x556d7cfc95c0 2024-05-08T21:25:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.830+0000 7f8f595c1b80 1 -- 172.21.15.33:0/549220040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 msgr2=0x556d7cfc99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.830+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/549220040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 0x556d7cfc99e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800e510 tx=0x7f8f38014000 comp rx=0 tx=0).stop 2024-05-08T21:25:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.830+0000 7f8f595c1b80 1 -- 172.21.15.33:0/549220040 shutdown_connections 2024-05-08T21:25:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.830+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/549220040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfca8e0 0x556d7cfcad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.830+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/549220040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 0x556d7cfc99e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.830+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/549220040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.830+0000 7f8f595c1b80 1 -- 172.21.15.33:0/549220040 >> 172.21.15.33:0/549220040 conn(0x556d7cf27290 msgr2=0x556d7cfc3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.830+0000 7f8f595c1b80 1 -- 172.21.15.33:0/549220040 shutdown_connections 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.830+0000 7f8f595c1b80 1 -- 172.21.15.33:0/549220040 wait complete. 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f595c1b80 1 -- start start 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f595c1b80 1 Processor -- start 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f595c1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc95c0 0x556d7cfd2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f595c1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfd33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f595c1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 0x556d7cfda400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f595c1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7ceb55e0 con 0x556d7cfc9f20 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f595c1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7ceb5280 con 0x556d7cfc95c0 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f595c1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7cd9c720 con 0x556d7cfca8e0 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f595c1b80 10 client.0 _ll_register_callbacks cb 0x556d7cebebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfd33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f46ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc95c0 0x556d7cfd2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfd33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43414/0 (socket says 172.21.15.33:43414) 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f467fc700 1 -- 172.21.15.33:0/2674900594 learned_addr learned my addr 172.21.15.33:0/2674900594 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.831+0000 7f8f477fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 0x556d7cfda400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f467fc700 1 -- 172.21.15.33:0/2674900594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 msgr2=0x556d7cfda400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f467fc700 1 --2- 172.21.15.33:0/2674900594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 0x556d7cfda400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f467fc700 1 -- 172.21.15.33:0/2674900594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc95c0 msgr2=0x556d7cfd2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f467fc700 1 --2- 172.21.15.33:0/2674900594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc95c0 0x556d7cfd2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f467fc700 1 -- 172.21.15.33:0/2674900594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f38017070 con 0x556d7cfc9f20 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f477fe700 1 --2- 172.21.15.33:0/2674900594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 0x556d7cfda400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f467fc700 1 --2- 172.21.15.33:0/2674900594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfd33c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009990 tx=0x7f8f3c0053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2789004188bc4349 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f2aff5700 1 -- 172.21.15.33:0/2674900594 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c01d030 con 0x556d7cfc9f20 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f2aff5700 1 -- 172.21.15.33:0/2674900594 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8f3c003b50 con 0x556d7cfc9f20 2024-05-08T21:25:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f595c1b80 1 -- 172.21.15.33:0/2674900594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d7ce722f0 con 0x556d7cfc9f20 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f2aff5700 1 -- 172.21.15.33:0/2674900594 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f3c003d10 con 0x556d7cfc9f20 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f595c1b80 10 client.17208 fetch_fsmap learned FSMap version 38 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.832+0000 7f8f595c1b80 1 -- 172.21.15.33:0/2674900594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x556d7cd46be0 con 0x556d7cfc9f20 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.833+0000 7f8f2aff5700 1 -- 172.21.15.33:0/2674900594 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556d7ce722f0 con 0x556d7cfc9f20 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.833+0000 7f8f2aff5700 1 -- 172.21.15.33:0/2674900594 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f8f3c0047a0 con 0x556d7cfc9f20 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.833+0000 7f8f595c1b80 10 client.17208 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.833+0000 7f8f595c1b80 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.833+0000 7f8f595c1b80 20 client.17208 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.833+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/2674900594 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556d7cfdeca0 0x556d7cfe10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.833+0000 7f8f595c1b80 4 client.17208 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.833+0000 7f8f595c1b80 1 -- 172.21.15.33:0/2674900594 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d7cf98890 con 0x556d7cfdeca0 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.833+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/2674900594 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556d7cfdeca0 0x556d7cfe10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.834+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/2674900594 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556d7cfdeca0 0x556d7cfe10c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8743948bae8ea91d server_cookie=2649f3aee913a829 in_seq=0 out_seq=0 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.834+0000 7f8f2aff5700 10 client.17208 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.834+0000 7f8f2aff5700 1 -- 172.21.15.33:0/2674900594 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d7cf98890 con 0x556d7cfdeca0 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.834+0000 7f8f2aff5700 10 client.17208 handle_command_reply: tid=0 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.834+0000 7f8f595c1b80 1 client.17208 shutdown 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.834+0000 7f8f595c1b80 1 -- 172.21.15.33:0/2674900594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 msgr2=0x556d7cfd33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.834+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/2674900594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfd33c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c009990 tx=0x7f8f3c0053b0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.834+0000 7f8f595c1b80 1 -- 172.21.15.33:0/2674900594 shutdown_connections 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.835+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/2674900594 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556d7cfdeca0 0x556d7cfe10c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.835+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/2674900594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 0x556d7cfda400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.835+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/2674900594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc95c0 0x556d7cfd2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.835+0000 7f8f2aff5700 0 client.17208 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.835+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/2674900594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfd33c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.835+0000 7f8f595c1b80 1 -- 172.21.15.33:0/2674900594 >> 172.21.15.33:0/2674900594 conn(0x556d7cf27290 msgr2=0x556d7cfc3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.835+0000 7f8f595c1b80 1 -- 172.21.15.33:0/2674900594 shutdown_connections 2024-05-08T21:25:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.835+0000 7f8f595c1b80 1 -- 172.21.15.33:0/2674900594 wait complete. 2024-05-08T21:25:03.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.835+0000 7f8f595c1b80 20 client.17208 trim_cache size 0 max 16384 2024-05-08T21:25:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:03 smithi082 ceph-mon[47512]: pgmap v472: 129 pgs: 129 active+clean; 219 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.6 MiB/s wr, 562 op/s 2024-05-08T21:25:03.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.858+0000 7f8f595c1b80 1 Processor -- start 2024-05-08T21:25:03.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.858+0000 7f8f595c1b80 1 -- start start 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.858+0000 7f8f595c1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 0x556d7cfd0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.858+0000 7f8f595c1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc9f20 0x556d7cfda3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.858+0000 7f8f595c1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfca8e0 0x556d7cfd2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f595c1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7cfbe880 con 0x556d7cfca8e0 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f595c1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7cfd3b00 con 0x556d7cfc9f20 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f595c1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7cfd0a60 con 0x556d7cfc95c0 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f46ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 0x556d7cfd0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfca8e0 0x556d7cfd2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f467fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc9f20 0x556d7cfda3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f46ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 0x556d7cfd0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:50140/0 (socket says 172.21.15.33:50140) 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f467fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc9f20 0x556d7cfda3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56610/0 (socket says 172.21.15.33:56610) 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f46ffd700 1 -- 172.21.15.33:0/1889395861 learned_addr learned my addr 172.21.15.33:0/1889395861 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfca8e0 0x556d7cfd2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43422/0 (socket says 172.21.15.33:43422) 2024-05-08T21:25:03.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f46ffd700 1 -- 172.21.15.33:0/1889395861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc9f20 msgr2=0x556d7cfda3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:03.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/1889395861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc9f20 0x556d7cfda3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f46ffd700 1 -- 172.21.15.33:0/1889395861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfca8e0 msgr2=0x556d7cfd2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:03.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/1889395861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfca8e0 0x556d7cfd2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f46ffd700 1 -- 172.21.15.33:0/1889395861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7cedf5e0 con 0x556d7cfc95c0 2024-05-08T21:25:03.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f467fc700 1 --2- 172.21.15.33:0/1889395861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc9f20 0x556d7cfda3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:03.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.859+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/1889395861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 0x556d7cfd0de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x556d7cfe19d0 tx=0x7f8f38005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f645dc8d7011f827 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:03.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.860+0000 7f8f2dffb700 1 -- 172.21.15.33:0/1889395861 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38005040 con 0x556d7cfc95c0 2024-05-08T21:25:03.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.860+0000 7f8f2dffb700 1 -- 172.21.15.33:0/1889395861 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f38005630 con 0x556d7cfc95c0 2024-05-08T21:25:03.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.860+0000 7f8f2dffb700 1 -- 172.21.15.33:0/1889395861 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f380338b0 con 0x556d7cfc95c0 2024-05-08T21:25:03.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.860+0000 7f8f595c1b80 1 -- 172.21.15.33:0/1889395861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 msgr2=0x556d7cfd0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:03.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.860+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/1889395861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 0x556d7cfd0de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x556d7cfe19d0 tx=0x7f8f38005e40 comp rx=0 tx=0).stop 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 -- 172.21.15.33:0/1889395861 shutdown_connections 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/1889395861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfca8e0 0x556d7cfd2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/1889395861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfc95c0 0x556d7cfd0de0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/1889395861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc9f20 0x556d7cfda3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 -- 172.21.15.33:0/1889395861 >> 172.21.15.33:0/1889395861 conn(0x556d7cf27290 msgr2=0x556d7cfcacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 -- 172.21.15.33:0/1889395861 shutdown_connections 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 -- 172.21.15.33:0/1889395861 wait complete. 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 -- start start 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 Processor -- start 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc95c0 0x556d7cfd9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfda340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 0x556d7cfd6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7cfd71b0 con 0x556d7cfc9f20 2024-05-08T21:25:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7cfd72f0 con 0x556d7cfc95c0 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7cfd74b0 con 0x556d7cfca8e0 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.861+0000 7f8f595c1b80 10 client.0 _ll_register_callbacks cb 0x556d7cebebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfda340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f477fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 0x556d7cfd6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f477fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 0x556d7cfd6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:50144/0 (socket says 172.21.15.33:50144) 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f477fe700 1 -- 172.21.15.33:0/81714438 learned_addr learned my addr 172.21.15.33:0/81714438 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc95c0 0x556d7cfd9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f477fe700 1 -- 172.21.15.33:0/81714438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc95c0 msgr2=0x556d7cfd9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f477fe700 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc95c0 0x556d7cfd9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f477fe700 1 -- 172.21.15.33:0/81714438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 msgr2=0x556d7cfda340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f477fe700 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfda340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f477fe700 1 -- 172.21.15.33:0/81714438 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d7cd46be0 con 0x556d7cfca8e0 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc95c0 0x556d7cfd9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f477fe700 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 0x556d7cfd6ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f40095df0 tx=0x7f8f40099110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d02b31f0c70e28af server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:03.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f2aff5700 1 -- 172.21.15.33:0/81714438 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400b7ea0 con 0x556d7cfca8e0 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f2aff5700 1 -- 172.21.15.33:0/81714438 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8f40084040 con 0x556d7cfca8e0 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f595c1b80 1 -- 172.21.15.33:0/81714438 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f38021070 con 0x556d7cfca8e0 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f2aff5700 1 -- 172.21.15.33:0/81714438 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f4009d770 con 0x556d7cfca8e0 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f595c1b80 10 client.27130 fetch_fsmap learned FSMap version 38 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.862+0000 7f8f595c1b80 1 -- 172.21.15.33:0/81714438 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f8f3802a040 con 0x556d7cfca8e0 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.863+0000 7f8f2aff5700 1 -- 172.21.15.33:0/81714438 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f38021070 con 0x556d7cfca8e0 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.863+0000 7f8f2aff5700 1 -- 172.21.15.33:0/81714438 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f8f400b1f00 con 0x556d7cfca8e0 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.863+0000 7f8f595c1b80 10 client.27130 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.863+0000 7f8f595c1b80 10 client.27130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.863+0000 7f8f595c1b80 20 client.27130 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.863+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556d7d0369c0 0x556d7d038de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.863+0000 7f8f595c1b80 4 client.27130 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.863+0000 7f8f595c1b80 1 -- 172.21.15.33:0/81714438 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556d7cf11f40 con 0x556d7d0369c0 2024-05-08T21:25:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.863+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556d7d0369c0 0x556d7d038de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:03.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.863+0000 7f8f2aff5700 10 client.27130 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:03.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.863+0000 7f8f46ffd700 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556d7d0369c0 0x556d7d038de0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8958675df0a0aa3 server_cookie=34283861a87ae97b in_seq=0 out_seq=0 2024-05-08T21:25:03.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.864+0000 7f8f2aff5700 1 -- 172.21.15.33:0/81714438 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556d7cf11f40 con 0x556d7d0369c0 2024-05-08T21:25:03.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.864+0000 7f8f2aff5700 10 client.27130 handle_command_reply: tid=0 2024-05-08T21:25:03.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.864+0000 7f8f595c1b80 1 client.27130 shutdown 2024-05-08T21:25:03.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f595c1b80 1 -- 172.21.15.33:0/81714438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 msgr2=0x556d7cfd6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:03.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 0x556d7cfd6ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f40095df0 tx=0x7f8f40099110 comp rx=0 tx=0).stop 2024-05-08T21:25:03.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f595c1b80 1 -- 172.21.15.33:0/81714438 shutdown_connections 2024-05-08T21:25:03.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x556d7d0369c0 0x556d7d038de0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d7cfca8e0 0x556d7cfd6ad0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d7cfc95c0 0x556d7cfd9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f595c1b80 1 --2- 172.21.15.33:0/81714438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d7cfc9f20 0x556d7cfda340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f595c1b80 1 -- 172.21.15.33:0/81714438 >> 172.21.15.33:0/81714438 conn(0x556d7cf27290 msgr2=0x556d7cfcd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:03.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f2aff5700 0 client.27130 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:03.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f595c1b80 1 -- 172.21.15.33:0/81714438 shutdown_connections 2024-05-08T21:25:03.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f595c1b80 1 -- 172.21.15.33:0/81714438 wait complete. 2024-05-08T21:25:03.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f595c1b80 20 client.27130 trim_cache size 0 max 16384 2024-05-08T21:25:03.871 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:25:03.871 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:25:03.871 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "50c74a10-9041-4362-8444-7801ec603027", 2024-05-08T21:25:03.871 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:25:03.871 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:25:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.865+0000 7f8f4d407700 1 -- 172.21.15.33:0/3308544262 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8f300a1120 msgr2=0x7f8f300a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.866+0000 7f8f4d407700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8f300a1120 0x7f8f300a35e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009c60 tx=0x7f8f38009260 comp rx=0 tx=0).stop 2024-05-08T21:25:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.866+0000 7f8f4d407700 1 -- 172.21.15.33:0/3308544262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f48106b70 msgr2=0x7f8f480ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.866+0000 7f8f4d407700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f48106b70 0x7f8f480ffe00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000ef30 tx=0x7f8f4000c3a0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.866+0000 7f8f4d407700 1 -- 172.21.15.33:0/3308544262 shutdown_connections 2024-05-08T21:25:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.866+0000 7f8f4d407700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f48106b70 0x7f8f480ffe00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.866+0000 7f8f4d407700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8f300a1120 0x7f8f300a35e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.866+0000 7f8f4d407700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f48101910 0x7f8f48101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.866+0000 7f8f4d407700 1 --2- 172.21.15.33:0/3308544262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f48104240 0x7f8f480ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.866+0000 7f8f4d407700 1 -- 172.21.15.33:0/3308544262 >> 172.21.15.33:0/3308544262 conn(0x7f8f480fb300 msgr2=0x7f8f48103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.866+0000 7f8f4d407700 1 -- 172.21.15.33:0/3308544262 shutdown_connections 2024-05-08T21:25:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:03.866+0000 7f8f4d407700 1 -- 172.21.15.33:0/3308544262 wait complete. 2024-05-08T21:25:03.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:25:04.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.117+0000 7f1c5f828700 1 -- 172.21.15.33:0/2974495803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c58101920 msgr2=0x7f1c58103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:04.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.117+0000 7f1c5f828700 1 --2- 172.21.15.33:0/2974495803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c58101920 0x7f1c58103d10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1c540098f0 tx=0x7f1c54009c00 comp rx=0 tx=0).stop 2024-05-08T21:25:04.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.117+0000 7f1c5f828700 1 -- 172.21.15.33:0/2974495803 shutdown_connections 2024-05-08T21:25:04.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.117+0000 7f1c5f828700 1 --2- 172.21.15.33:0/2974495803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c58106b80 0x7f1c58108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.117+0000 7f1c5f828700 1 --2- 172.21.15.33:0/2974495803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c58101920 0x7f1c58103d10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.117+0000 7f1c5f828700 1 --2- 172.21.15.33:0/2974495803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c58104250 0x7f1c58106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.117+0000 7f1c5f828700 1 -- 172.21.15.33:0/2974495803 >> 172.21.15.33:0/2974495803 conn(0x7f1c580fb290 msgr2=0x7f1c580fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.118+0000 7f1c5f828700 1 -- 172.21.15.33:0/2974495803 shutdown_connections 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.118+0000 7f1c5f828700 1 -- 172.21.15.33:0/2974495803 wait complete. 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.118+0000 7f1c5f828700 1 Processor -- start 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.118+0000 7f1c5f828700 1 -- start start 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.118+0000 7f1c5f828700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c58101920 0x7f1c58196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.118+0000 7f1c5f828700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c58104250 0x7f1c58197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.118+0000 7f1c5f828700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c58106b80 0x7f1c5819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5f828700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5819e830 con 0x7f1c58101920 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5f828700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5819e9a0 con 0x7f1c58106b80 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5f828700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5819eb60 con 0x7f1c58104250 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5d5c4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c58101920 0x7f1c58196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5d5c4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c58101920 0x7f1c58196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43460/0 (socket says 172.21.15.33:43460) 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5d5c4700 1 -- 172.21.15.33:0/1670882911 learned_addr learned my addr 172.21.15.33:0/1670882911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5ddc5700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c58106b80 0x7f1c5819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c58104250 0x7f1c58197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5d5c4700 1 -- 172.21.15.33:0/1670882911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c58104250 msgr2=0x7f1c58197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5d5c4700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c58104250 0x7f1c58197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5d5c4700 1 -- 172.21.15.33:0/1670882911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c58106b80 msgr2=0x7f1c5819e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5d5c4700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c58106b80 0x7f1c5819e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5d5c4700 1 -- 172.21.15.33:0/1670882911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c54009520 con 0x7f1c58101920 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5d5c4700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c58101920 0x7f1c58196d40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009fd0 tx=0x7f1c5400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c4e7fc700 1 -- 172.21.15.33:0/1670882911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5401c070 con 0x7f1c58101920 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c4e7fc700 1 -- 172.21.15.33:0/1670882911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c5400f840 con 0x7f1c58101920 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c5f828700 1 -- 172.21.15.33:0/1670882911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5819ede0 con 0x7f1c58101920 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.119+0000 7f1c4e7fc700 1 -- 172.21.15.33:0/1670882911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54003a00 con 0x7f1c58101920 2024-05-08T21:25:04.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.120+0000 7f1c5f828700 1 -- 172.21.15.33:0/1670882911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5819f1b0 con 0x7f1c58101920 2024-05-08T21:25:04.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.122+0000 7f1c4e7fc700 1 -- 172.21.15.33:0/1670882911 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1c54020030 con 0x7f1c58101920 2024-05-08T21:25:04.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.123+0000 7f1c4e7fc700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1c440a11a0 0x7f1c440a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.123+0000 7f1c4e7fc700 1 -- 172.21.15.33:0/1670882911 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1c540cdbe0 con 0x7f1c58101920 2024-05-08T21:25:04.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.123+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1c440a11a0 0x7f1c440a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.124+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1c440a11a0 0x7f1c440a3660 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009c60 tx=0x7f1c48009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:04.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.124+0000 7f1c6b9e2b80 1 Processor -- start 2024-05-08T21:25:04.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.124+0000 7f1c6b9e2b80 1 -- start start 2024-05-08T21:25:04.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c6b9e2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beae540 0x55833beae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c6b9e2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 0x55833beaf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c6b9e2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833beafce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c6b9e2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833be05150 con 0x55833beae540 2024-05-08T21:25:04.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c6b9e2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833be02b10 con 0x55833beaf860 2024-05-08T21:25:04.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c6b9e2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833bdfe400 con 0x55833beaeea0 2024-05-08T21:25:04.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c5cdc3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 0x55833beaf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c5ddc5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833beafce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c5cdc3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 0x55833beaf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:50162/0 (socket says 172.21.15.33:50162) 2024-05-08T21:25:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c5cdc3700 1 -- 172.21.15.33:0/2584883495 learned_addr learned my addr 172.21.15.33:0/2584883495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c5cdc3700 1 -- 172.21.15.33:0/2584883495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 msgr2=0x55833beafce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/2584883495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833beafce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c5cdc3700 1 -- 172.21.15.33:0/2584883495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beae540 msgr2=0x55833beae960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:25:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/2584883495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beae540 0x55833beae960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c5cdc3700 1 -- 172.21.15.33:0/2584883495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55833bc7f3b0 con 0x55833beaeea0 2024-05-08T21:25:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.125+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/2584883495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 0x55833beaf320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800e4e0 tx=0x7f1c48014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3c00396f75ab857 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.126+0000 7f1c437fe700 1 -- 172.21.15.33:0/2584883495 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4800ee00 con 0x55833beaeea0 2024-05-08T21:25:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.126+0000 7f1c437fe700 1 -- 172.21.15.33:0/2584883495 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c48021070 con 0x55833beaeea0 2024-05-08T21:25:04.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.126+0000 7f1c437fe700 1 -- 172.21.15.33:0/2584883495 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4801d740 con 0x55833beaeea0 2024-05-08T21:25:04.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.127+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/2584883495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 msgr2=0x55833beaf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:04.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.127+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/2584883495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 0x55833beaf320 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800e4e0 tx=0x7f1c48014000 comp rx=0 tx=0).stop 2024-05-08T21:25:04.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.128+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/2584883495 shutdown_connections 2024-05-08T21:25:04.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.128+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/2584883495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833beafce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.128+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/2584883495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beae540 0x55833beae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.128+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/2584883495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 0x55833beaf320 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.128+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/2584883495 >> 172.21.15.33:0/2584883495 conn(0x55833be0c210 msgr2=0x55833bea8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:04.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.128+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/2584883495 shutdown_connections 2024-05-08T21:25:04.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.129+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/2584883495 wait complete. 2024-05-08T21:25:04.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.130+0000 7f1c6b9e2b80 1 -- start start 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.130+0000 7f1c6b9e2b80 1 Processor -- start 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.130+0000 7f1c6b9e2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beae540 0x55833beb7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.130+0000 7f1c6b9e2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 0x55833beb8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.130+0000 7f1c6b9e2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bebd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.130+0000 7f1c6b9e2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833bd9a560 con 0x55833beaeea0 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.130+0000 7f1c6b9e2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833bd9a200 con 0x55833beaf860 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.130+0000 7f1c6b9e2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833bc816a0 con 0x55833beae540 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c6b9e2b80 10 client.0 _ll_register_callbacks cb 0x55833bda3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c5ddc5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bebd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c5cdc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 0x55833beb8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c5ddc5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bebd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56660/0 (socket says 172.21.15.33:56660) 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c5ddc5700 1 -- 172.21.15.33:0/3264272516 learned_addr learned my addr 172.21.15.33:0/3264272516 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c5d5c4700 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beae540 0x55833beb7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c5ddc5700 1 -- 172.21.15.33:0/3264272516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beae540 msgr2=0x55833beb7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:04.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c5ddc5700 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beae540 0x55833beb7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c5ddc5700 1 -- 172.21.15.33:0/3264272516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 msgr2=0x55833beb8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:04.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c5ddc5700 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 0x55833beb8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c5ddc5700 1 -- 172.21.15.33:0/3264272516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c48017070 con 0x55833beaf860 2024-05-08T21:25:04.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 0x55833beb8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:04.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c5ddc5700 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bebd430 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1c50007640 tx=0x7f1c50007670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=401c6f32152002d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:04.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.131+0000 7f1c40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:04.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.132+0000 7f1c40ff9700 1 -- 172.21.15.33:0/3264272516 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50003b90 con 0x55833beaf860 2024-05-08T21:25:04.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.132+0000 7f1c40ff9700 1 -- 172.21.15.33:0/3264272516 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1c50004200 con 0x55833beaf860 2024-05-08T21:25:04.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.132+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3264272516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55833bd57270 con 0x55833beaf860 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.132+0000 7f1c40ff9700 1 -- 172.21.15.33:0/3264272516 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c50004400 con 0x55833beaf860 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.132+0000 7f1c6b9e2b80 10 client.27131 fetch_fsmap learned FSMap version 38 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.132+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3264272516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55833bc2bb60 con 0x55833beaf860 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.132+0000 7f1c40ff9700 1 -- 172.21.15.33:0/3264272516 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55833bd57270 con 0x55833beaf860 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.132+0000 7f1c40ff9700 1 -- 172.21.15.33:0/3264272516 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1c50016d10 con 0x55833beaf860 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.132+0000 7f1c6b9e2b80 10 client.27131 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.132+0000 7f1c6b9e2b80 10 client.27131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.132+0000 7f1c6b9e2b80 20 client.27131 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.132+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55833bed4cc0 0x55833bed70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.133+0000 7f1c6b9e2b80 4 client.27131 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.133+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3264272516 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55833be7d810 con 0x55833bed4cc0 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.133+0000 7f1c5d5c4700 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55833bed4cc0 0x55833bed70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.133+0000 7f1c40ff9700 10 client.27131 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:04.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.133+0000 7f1c5d5c4700 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55833bed4cc0 0x55833bed70e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e09c2c6cd7ed50c server_cookie=21cc7bb6e86acc38 in_seq=0 out_seq=0 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c40ff9700 1 -- 172.21.15.33:0/3264272516 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55833be7d810 con 0x55833bed4cc0 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c40ff9700 10 client.27131 handle_command_reply: tid=0 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c6b9e2b80 1 client.27131 shutdown 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3264272516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 msgr2=0x55833bebd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bebd430 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1c50007640 tx=0x7f1c50007670 comp rx=0 tx=0).stop 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3264272516 shutdown_connections 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55833bed4cc0 0x55833bed70e0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bebd430 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beae540 0x55833beb7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/3264272516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 0x55833beb8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c40ff9700 0 client.27131 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3264272516 >> 172.21.15.33:0/3264272516 conn(0x55833be0c210 msgr2=0x55833bea85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3264272516 shutdown_connections 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.134+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3264272516 wait complete. 2024-05-08T21:25:04.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.135+0000 7f1c6b9e2b80 20 client.27131 trim_cache size 0 max 16384 2024-05-08T21:25:04.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c6b9e2b80 1 Processor -- start 2024-05-08T21:25:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c6b9e2b80 1 -- start start 2024-05-08T21:25:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c6b9e2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beae540 0x55833beb7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c6b9e2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 0x55833bebd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c6b9e2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bee4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c6b9e2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833bebe5b0 con 0x55833beaeea0 2024-05-08T21:25:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c6b9e2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833beb7a80 con 0x55833beaf860 2024-05-08T21:25:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c6b9e2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833bebeca0 con 0x55833beae540 2024-05-08T21:25:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c5cdc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 0x55833bebd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c5cdc3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 0x55833bebd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43486/0 (socket says 172.21.15.33:43486) 2024-05-08T21:25:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c5cdc3700 1 -- 172.21.15.33:0/3497216922 learned_addr learned my addr 172.21.15.33:0/3497216922 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c5d5c4700 1 --2- 172.21.15.33:0/3497216922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beae540 0x55833beb7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.158+0000 7f1c5ddc5700 1 --2- 172.21.15.33:0/3497216922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bee4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.159+0000 7f1c5cdc3700 1 -- 172.21.15.33:0/3497216922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beae540 msgr2=0x55833beb7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.159+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/3497216922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beae540 0x55833beb7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.159+0000 7f1c5cdc3700 1 -- 172.21.15.33:0/3497216922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 msgr2=0x55833bee4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.159+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/3497216922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bee4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.159+0000 7f1c5cdc3700 1 -- 172.21.15.33:0/3497216922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55833bdc4560 con 0x55833beaeea0 2024-05-08T21:25:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.159+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/3497216922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 0x55833bebd3f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1c48011ab0 tx=0x7f1c48005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c3a771073df730 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.159+0000 7f1c43fff700 1 -- 172.21.15.33:0/3497216922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4800ec80 con 0x55833beaeea0 2024-05-08T21:25:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.159+0000 7f1c43fff700 1 -- 172.21.15.33:0/3497216922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c4800ede0 con 0x55833beaeea0 2024-05-08T21:25:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.159+0000 7f1c43fff700 1 -- 172.21.15.33:0/3497216922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c48003cb0 con 0x55833beaeea0 2024-05-08T21:25:04.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.160+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3497216922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 msgr2=0x55833bebd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:04.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.160+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/3497216922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 0x55833bebd3f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1c48011ab0 tx=0x7f1c48005c20 comp rx=0 tx=0).stop 2024-05-08T21:25:04.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.160+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3497216922 shutdown_connections 2024-05-08T21:25:04.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.160+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/3497216922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bee4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.160+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/3497216922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beae540 0x55833beb7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.160+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/3497216922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beaeea0 0x55833bebd3f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.160+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3497216922 >> 172.21.15.33:0/3497216922 conn(0x55833be0c210 msgr2=0x55833bea5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.160+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3497216922 shutdown_connections 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.160+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/3497216922 wait complete. 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.160+0000 7f1c6b9e2b80 1 -- start start 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c6b9e2b80 1 Processor -- start 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c6b9e2b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beae540 0x55833bee3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c6b9e2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 0x55833bee4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c6b9e2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bee4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c6b9e2b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833bee5800 con 0x55833beae540 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c6b9e2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833bf29c20 con 0x55833beaf860 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c6b9e2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833bf29d60 con 0x55833beaeea0 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c6b9e2b80 10 client.0 _ll_register_callbacks cb 0x55833bda3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c5d5c4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beae540 0x55833bee3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c5cdc3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 0x55833bee4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c5cdc3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 0x55833bee4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:50184/0 (socket says 172.21.15.33:50184) 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c5d5c4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beae540 0x55833bee3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43498/0 (socket says 172.21.15.33:43498) 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c5cdc3700 1 -- 172.21.15.33:0/4009172485 learned_addr learned my addr 172.21.15.33:0/4009172485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.161+0000 7f1c5ddc5700 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bee4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c5cdc3700 1 -- 172.21.15.33:0/4009172485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 msgr2=0x55833bee4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bee4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c5cdc3700 1 -- 172.21.15.33:0/4009172485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beae540 msgr2=0x55833bee3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beae540 0x55833bee3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c5cdc3700 1 -- 172.21.15.33:0/4009172485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4802a040 con 0x55833beaeea0 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c5ddc5700 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bee4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c5cdc3700 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 0x55833bee4200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c480097c0 tx=0x7f1c48004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a68bbbdda5895e8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c40ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c40ff9700 1 -- 172.21.15.33:0/4009172485 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c480050d0 con 0x55833beaeea0 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c40ff9700 1 -- 172.21.15.33:0/4009172485 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1c48005230 con 0x55833beaeea0 2024-05-08T21:25:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/4009172485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c48021070 con 0x55833beaeea0 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c40ff9700 1 -- 172.21.15.33:0/4009172485 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c480053f0 con 0x55833beaeea0 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c6b9e2b80 10 client.27148 fetch_fsmap learned FSMap version 38 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.162+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/4009172485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55833bc83cd0 con 0x55833beaeea0 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.163+0000 7f1c40ff9700 1 -- 172.21.15.33:0/4009172485 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c48021070 con 0x55833beaeea0 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.163+0000 7f1c40ff9700 1 -- 172.21.15.33:0/4009172485 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1c4802d070 con 0x55833beaeea0 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.163+0000 7f1c6b9e2b80 10 client.27148 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.163+0000 7f1c6b9e2b80 10 client.27148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.163+0000 7f1c6b9e2b80 20 client.27148 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.163+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55833bf2b4d0 0x55833bf2d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.163+0000 7f1c6b9e2b80 4 client.27148 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.163+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/4009172485 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55833bdf6ec0 con 0x55833bf2b4d0 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.163+0000 7f1c5d5c4700 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55833bf2b4d0 0x55833bf2d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c40ff9700 10 client.27148 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c5d5c4700 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55833bf2b4d0 0x55833bf2d8f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6eb8bbac8aaba5 server_cookie=632a854f6c83a3cc in_seq=0 out_seq=0 2024-05-08T21:25:04.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c40ff9700 1 -- 172.21.15.33:0/4009172485 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55833bdf6ec0 con 0x55833bf2b4d0 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c40ff9700 10 client.27148 handle_command_reply: tid=0 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c6b9e2b80 1 client.27148 shutdown 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/4009172485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 msgr2=0x55833bee4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 0x55833bee4200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1c480097c0 tx=0x7f1c48004f40 comp rx=0 tx=0).stop 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/4009172485 shutdown_connections 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55833bf2b4d0 0x55833bf2d8f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55833beaf860 0x55833bee4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55833beae540 0x55833bee3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c40ff9700 0 client.27148 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c6b9e2b80 1 --2- 172.21.15.33:0/4009172485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55833beaeea0 0x55833bee4200 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.164+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/4009172485 >> 172.21.15.33:0/4009172485 conn(0x55833be0c210 msgr2=0x55833beb6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/4009172485 shutdown_connections 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c6b9e2b80 1 -- 172.21.15.33:0/4009172485 wait complete. 2024-05-08T21:25:04.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c6b9e2b80 20 client.27148 trim_cache size 0 max 16384 2024-05-08T21:25:04.171 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:25:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:25:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:25:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "50c74a10-9041-4362-8444-7801ec603027": { 2024-05-08T21:25:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:25:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "50c74a10-9041-4362-8444-7801ec603027", 2024-05-08T21:25:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:25:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:25:04.171 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:25:04.171 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:25:04.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c5f828700 1 -- 172.21.15.33:0/1670882911 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1c440a11a0 msgr2=0x7f1c440a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:04.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c5f828700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1c440a11a0 0x7f1c440a3660 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009c60 tx=0x7f1c48009260 comp rx=0 tx=0).stop 2024-05-08T21:25:04.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c5f828700 1 -- 172.21.15.33:0/1670882911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c58101920 msgr2=0x7f1c58196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:04.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c5f828700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c58101920 0x7f1c58196d40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009fd0 tx=0x7f1c5400b790 comp rx=0 tx=0).stop 2024-05-08T21:25:04.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c5f828700 1 -- 172.21.15.33:0/1670882911 shutdown_connections 2024-05-08T21:25:04.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c5f828700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1c440a11a0 0x7f1c440a3660 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c5f828700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c58106b80 0x7f1c5819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c5f828700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c58101920 0x7f1c58196d40 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c5f828700 1 --2- 172.21.15.33:0/1670882911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c58104250 0x7f1c58197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:04.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.165+0000 7f1c5f828700 1 -- 172.21.15.33:0/1670882911 >> 172.21.15.33:0/1670882911 conn(0x7f1c580fb290 msgr2=0x7f1c581023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:04.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.166+0000 7f1c5f828700 1 -- 172.21.15.33:0/1670882911 shutdown_connections 2024-05-08T21:25:04.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:04.166+0000 7f1c5f828700 1 -- 172.21.15.33:0/1670882911 wait complete. 2024-05-08T21:25:04.183 INFO:tasks.cephfs.filesystem:scrub status for tag:50c74a10-9041-4362-8444-7801ec603027 - {'path': '/', 'tag': '50c74a10-9041-4362-8444-7801ec603027', 'options': 'recursive,force'} 2024-05-08T21:25:04.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:04 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:25:04.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:04 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:04.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:04 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:04.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:04 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:04.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:04 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:04.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:04 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:04.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:04 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:04.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:04 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:04.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:04 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:25:04.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:04 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:04.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:04 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:04 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:04 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:04 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:04 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:04 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:04.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:04 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:25:04.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:04 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:04.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:04 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:04.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:04 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:04.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:04 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:04.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:04 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:04.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:04 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:04.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:04 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:05 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:25:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:05 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:05 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:05 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:05 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:05 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:05 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:05 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:05 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:05 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:05 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:05 smithi033 ceph-mon[42528]: pgmap v473: 129 pgs: 129 active+clean; 219 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.6 MiB/s wr, 412 op/s 2024-05-08T21:25:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:05 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:25:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:05 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:05 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:05 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:05 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:05 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:05 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:05 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:05 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:05 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:05 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:05 smithi071 ceph-mon[47074]: pgmap v473: 129 pgs: 129 active+clean; 219 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.6 MiB/s wr, 412 op/s 2024-05-08T21:25:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:05 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:25:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:05 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:05 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:05 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:05 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:05 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:05 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:05 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:05 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:05 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:05 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:05 smithi082 ceph-mon[47512]: pgmap v473: 129 pgs: 129 active+clean; 219 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.6 MiB/s wr, 412 op/s 2024-05-08T21:25:07.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:07 smithi033 ceph-mon[42528]: pgmap v474: 129 pgs: 129 active+clean; 224 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.8 MiB/s wr, 581 op/s 2024-05-08T21:25:07.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:07 smithi071 ceph-mon[47074]: pgmap v474: 129 pgs: 129 active+clean; 224 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.8 MiB/s wr, 581 op/s 2024-05-08T21:25:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:07 smithi082 ceph-mon[47512]: pgmap v474: 129 pgs: 129 active+clean; 224 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.8 MiB/s wr, 581 op/s 2024-05-08T21:25:09.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:09 smithi033 ceph-mon[42528]: pgmap v475: 129 pgs: 129 active+clean; 223 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.9 MiB/s wr, 435 op/s 2024-05-08T21:25:09.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:09 smithi071 ceph-mon[47074]: pgmap v475: 129 pgs: 129 active+clean; 223 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.9 MiB/s wr, 435 op/s 2024-05-08T21:25:09.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:09 smithi082 ceph-mon[47512]: pgmap v475: 129 pgs: 129 active+clean; 223 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.9 MiB/s wr, 435 op/s 2024-05-08T21:25:11.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:11 smithi033 ceph-mon[42528]: pgmap v476: 129 pgs: 129 active+clean; 223 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.3 MiB/s wr, 482 op/s 2024-05-08T21:25:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:11 smithi071 ceph-mon[47074]: pgmap v476: 129 pgs: 129 active+clean; 223 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.3 MiB/s wr, 482 op/s 2024-05-08T21:25:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:11 smithi082 ceph-mon[47512]: pgmap v476: 129 pgs: 129 active+clean; 223 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.3 MiB/s wr, 482 op/s 2024-05-08T21:25:13.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:13 smithi033 ceph-mon[42528]: pgmap v477: 129 pgs: 129 active+clean; 225 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 549 op/s 2024-05-08T21:25:13.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:13 smithi071 ceph-mon[47074]: pgmap v477: 129 pgs: 129 active+clean; 225 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 549 op/s 2024-05-08T21:25:13.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:13 smithi082 ceph-mon[47512]: pgmap v477: 129 pgs: 129 active+clean; 225 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 549 op/s 2024-05-08T21:25:14.168 INFO:tasks.workunit.client.0.smithi033.stdout:30000 punch from 0xa93a7 to 0xacc3d, (0x3896 bytes) 2024-05-08T21:25:14.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:14 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:25:14.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:14 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:25:14.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:14 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:25:15.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:15 smithi033 ceph-mon[42528]: pgmap v478: 129 pgs: 129 active+clean; 225 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.9 MiB/s wr, 411 op/s 2024-05-08T21:25:15.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:15 smithi071 ceph-mon[47074]: pgmap v478: 129 pgs: 129 active+clean; 225 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.9 MiB/s wr, 411 op/s 2024-05-08T21:25:15.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:15 smithi082 ceph-mon[47512]: pgmap v478: 129 pgs: 129 active+clean; 225 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.9 MiB/s wr, 411 op/s 2024-05-08T21:25:17.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:17 smithi033 ceph-mon[42528]: pgmap v479: 129 pgs: 129 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 548 op/s 2024-05-08T21:25:17.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:17 smithi071 ceph-mon[47074]: pgmap v479: 129 pgs: 129 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 548 op/s 2024-05-08T21:25:17.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:17 smithi082 ceph-mon[47512]: pgmap v479: 129 pgs: 129 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 548 op/s 2024-05-08T21:25:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:18 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:25:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:18 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:25:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:18 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:25:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:18 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:25:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:18 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:25:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:25:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:25:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:25:18.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:25:18.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:18 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:25:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:18 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:25:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:18 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:25:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:18 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:25:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:18 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:25:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:25:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:25:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:25:18.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:25:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:18 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:25:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:18 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:25:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:18 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:25:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:18 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:25:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:18 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:25:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:25:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:25:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:25:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:25:19.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:19 smithi033 ceph-mon[42528]: pgmap v480: 129 pgs: 129 active+clean; 222 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.0 MiB/s wr, 435 op/s 2024-05-08T21:25:19.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:19 smithi071 ceph-mon[47074]: pgmap v480: 129 pgs: 129 active+clean; 222 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.0 MiB/s wr, 435 op/s 2024-05-08T21:25:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:19 smithi082 ceph-mon[47512]: pgmap v480: 129 pgs: 129 active+clean; 222 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.0 MiB/s wr, 435 op/s 2024-05-08T21:25:21.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:21 smithi033 ceph-mon[42528]: pgmap v481: 129 pgs: 129 active+clean; 226 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.2 MiB/s wr, 457 op/s 2024-05-08T21:25:21.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:21 smithi071 ceph-mon[47074]: pgmap v481: 129 pgs: 129 active+clean; 226 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.2 MiB/s wr, 457 op/s 2024-05-08T21:25:21.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:21 smithi082 ceph-mon[47512]: pgmap v481: 129 pgs: 129 active+clean; 226 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.2 MiB/s wr, 457 op/s 2024-05-08T21:25:23.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:23 smithi033 ceph-mon[42528]: pgmap v482: 129 pgs: 129 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.4 MiB/s wr, 518 op/s 2024-05-08T21:25:23.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:23 smithi071 ceph-mon[47074]: pgmap v482: 129 pgs: 129 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.4 MiB/s wr, 518 op/s 2024-05-08T21:25:23.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:23 smithi082 ceph-mon[47512]: pgmap v482: 129 pgs: 129 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.4 MiB/s wr, 518 op/s 2024-05-08T21:25:25.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:25 smithi033 ceph-mon[42528]: pgmap v483: 129 pgs: 129 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.6 MiB/s wr, 387 op/s 2024-05-08T21:25:25.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:25 smithi071 ceph-mon[47074]: pgmap v483: 129 pgs: 129 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.6 MiB/s wr, 387 op/s 2024-05-08T21:25:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:25 smithi082 ceph-mon[47512]: pgmap v483: 129 pgs: 129 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.6 MiB/s wr, 387 op/s 2024-05-08T21:25:27.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:27 smithi033 ceph-mon[42528]: pgmap v484: 129 pgs: 129 active+clean; 223 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 512 op/s 2024-05-08T21:25:27.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:27 smithi071 ceph-mon[47074]: pgmap v484: 129 pgs: 129 active+clean; 223 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 512 op/s 2024-05-08T21:25:27.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:27 smithi082 ceph-mon[47512]: pgmap v484: 129 pgs: 129 active+clean; 223 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 512 op/s 2024-05-08T21:25:29.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:29 smithi033 ceph-mon[42528]: pgmap v485: 129 pgs: 129 active+clean; 224 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.7 MiB/s wr, 433 op/s 2024-05-08T21:25:29.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:29 smithi071 ceph-mon[47074]: pgmap v485: 129 pgs: 129 active+clean; 224 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.7 MiB/s wr, 433 op/s 2024-05-08T21:25:29.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:29 smithi082 ceph-mon[47512]: pgmap v485: 129 pgs: 129 active+clean; 224 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.7 MiB/s wr, 433 op/s 2024-05-08T21:25:31.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:31 smithi033 ceph-mon[42528]: pgmap v486: 129 pgs: 129 active+clean; 225 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.0 MiB/s wr, 439 op/s 2024-05-08T21:25:31.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:31 smithi071 ceph-mon[47074]: pgmap v486: 129 pgs: 129 active+clean; 225 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.0 MiB/s wr, 439 op/s 2024-05-08T21:25:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:31 smithi082 ceph-mon[47512]: pgmap v486: 129 pgs: 129 active+clean; 225 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.0 MiB/s wr, 439 op/s 2024-05-08T21:25:33.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:33 smithi033 ceph-mon[42528]: pgmap v487: 129 pgs: 129 active+clean; 227 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.2 MiB/s wr, 501 op/s 2024-05-08T21:25:33.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:33 smithi071 ceph-mon[47074]: pgmap v487: 129 pgs: 129 active+clean; 227 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.2 MiB/s wr, 501 op/s 2024-05-08T21:25:33.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:33 smithi082 ceph-mon[47512]: pgmap v487: 129 pgs: 129 active+clean; 227 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.2 MiB/s wr, 501 op/s 2024-05-08T21:25:34.184 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:25:34.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.350+0000 7fe073a47700 1 -- 172.21.15.33:0/2088391604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe06c106b40 msgr2=0x7fe06c108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.350+0000 7fe073a47700 1 --2- 172.21.15.33:0/2088391604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe06c106b40 0x7fe06c108fb0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe064009f90 tx=0x7fe06400b330 comp rx=0 tx=0).stop 2024-05-08T21:25:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.350+0000 7fe073a47700 1 -- 172.21.15.33:0/2088391604 shutdown_connections 2024-05-08T21:25:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.350+0000 7fe073a47700 1 --2- 172.21.15.33:0/2088391604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe06c106b40 0x7fe06c108fb0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.350+0000 7fe073a47700 1 --2- 172.21.15.33:0/2088391604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe06c1018e0 0x7fe06c103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.350+0000 7fe073a47700 1 --2- 172.21.15.33:0/2088391604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c104210 0x7fe06c106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.350+0000 7fe073a47700 1 -- 172.21.15.33:0/2088391604 >> 172.21.15.33:0/2088391604 conn(0x7fe06c0fb290 msgr2=0x7fe06c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.351+0000 7fe073a47700 1 -- 172.21.15.33:0/2088391604 shutdown_connections 2024-05-08T21:25:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.351+0000 7fe073a47700 1 -- 172.21.15.33:0/2088391604 wait complete. 2024-05-08T21:25:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.351+0000 7fe073a47700 1 Processor -- start 2024-05-08T21:25:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe073a47700 1 -- start start 2024-05-08T21:25:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe073a47700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe06c1018e0 0x7fe06c101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe073a47700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe06c104210 0x7fe06c0ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe073a47700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c106b40 0x7fe06c0ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe073a47700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c100570 con 0x7fe06c104210 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe073a47700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c1006e0 con 0x7fe06c106b40 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe073a47700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c1a0e60 con 0x7fe06c1018e0 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe0717e3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe06c1018e0 0x7fe06c101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe071fe4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c106b40 0x7fe06c0ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe0717e3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe06c1018e0 0x7fe06c101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49360/0 (socket says 172.21.15.33:49360) 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe0717e3700 1 -- 172.21.15.33:0/2788747172 learned_addr learned my addr 172.21.15.33:0/2788747172 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe0717e3700 1 -- 172.21.15.33:0/2788747172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c106b40 msgr2=0x7fe06c0ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe0717e3700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c106b40 0x7fe06c0ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe0717e3700 1 -- 172.21.15.33:0/2788747172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe06c104210 msgr2=0x7fe06c0ff890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe0717e3700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe06c104210 0x7fe06c0ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.352+0000 7fe0717e3700 1 -- 172.21.15.33:0/2788747172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe064009c40 con 0x7fe06c1018e0 2024-05-08T21:25:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.353+0000 7fe071fe4700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c106b40 0x7fe06c0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.353+0000 7fe0717e3700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe06c1018e0 0x7fe06c101200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe06800e9c0 tx=0x7fe06800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.353+0000 7fe0627fc700 1 -- 172.21.15.33:0/2788747172 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06800cbd0 con 0x7fe06c1018e0 2024-05-08T21:25:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.353+0000 7fe0627fc700 1 -- 172.21.15.33:0/2788747172 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe06800eeb0 con 0x7fe06c1018e0 2024-05-08T21:25:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.353+0000 7fe073a47700 1 -- 172.21.15.33:0/2788747172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06c1a1090 con 0x7fe06c1018e0 2024-05-08T21:25:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.353+0000 7fe0627fc700 1 -- 172.21.15.33:0/2788747172 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068005410 con 0x7fe06c1018e0 2024-05-08T21:25:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.353+0000 7fe073a47700 1 -- 172.21.15.33:0/2788747172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06c1a15a0 con 0x7fe06c1018e0 2024-05-08T21:25:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.355+0000 7fe0627fc700 1 -- 172.21.15.33:0/2788747172 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe0680055b0 con 0x7fe06c1018e0 2024-05-08T21:25:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.356+0000 7fe0627fc700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe0580a1120 0x7fe0580a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.356+0000 7fe0627fc700 1 -- 172.21.15.33:0/2788747172 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe068014070 con 0x7fe06c1018e0 2024-05-08T21:25:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.356+0000 7fe070fe2700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe0580a1120 0x7fe0580a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.356+0000 7fe070fe2700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe0580a1120 0x7fe0580a35e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe05c005950 tx=0x7fe05c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.357+0000 7fe07fc01b80 1 Processor -- start 2024-05-08T21:25:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.357+0000 7fe07fc01b80 1 -- start start 2024-05-08T21:25:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.357+0000 7fe07fc01b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6508540 0x55fad6508960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.357+0000 7fe07fc01b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 0x55fad6509320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.357+0000 7fe07fc01b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6509860 0x55fad6509ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.357+0000 7fe07fc01b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad645f150 con 0x55fad6508540 2024-05-08T21:25:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.357+0000 7fe07fc01b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad645cb10 con 0x55fad6508ea0 2024-05-08T21:25:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.357+0000 7fe07fc01b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad6458400 con 0x55fad6509860 2024-05-08T21:25:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe070fe2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 0x55fad6509320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe071fe4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6509860 0x55fad6509ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe071fe4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6509860 0x55fad6509ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49370/0 (socket says 172.21.15.33:49370) 2024-05-08T21:25:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe070fe2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 0x55fad6509320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56000/0 (socket says 172.21.15.33:56000) 2024-05-08T21:25:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe071fe4700 1 -- 172.21.15.33:0/2897631126 learned_addr learned my addr 172.21.15.33:0/2897631126 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe0717e3700 1 --2- 172.21.15.33:0/2897631126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6508540 0x55fad6508960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe071fe4700 1 -- 172.21.15.33:0/2897631126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 msgr2=0x55fad6509320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe071fe4700 1 --2- 172.21.15.33:0/2897631126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 0x55fad6509320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe071fe4700 1 -- 172.21.15.33:0/2897631126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6508540 msgr2=0x55fad6508960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe071fe4700 1 --2- 172.21.15.33:0/2897631126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6508540 0x55fad6508960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe071fe4700 1 -- 172.21.15.33:0/2897631126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fad62d93b0 con 0x55fad6509860 2024-05-08T21:25:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe070fe2700 1 --2- 172.21.15.33:0/2897631126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 0x55fad6509320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:34.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe071fe4700 1 --2- 172.21.15.33:0/2897631126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6509860 0x55fad6509ce0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe064004ea0 tx=0x7fe064004ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=630ce59e3b3be784 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe0577fe700 1 -- 172.21.15.33:0/2897631126 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06400e040 con 0x55fad6509860 2024-05-08T21:25:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe0577fe700 1 -- 172.21.15.33:0/2897631126 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe064007420 con 0x55fad6509860 2024-05-08T21:25:34.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.358+0000 7fe0577fe700 1 -- 172.21.15.33:0/2897631126 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06400bc50 con 0x55fad6509860 2024-05-08T21:25:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.359+0000 7fe07fc01b80 1 -- 172.21.15.33:0/2897631126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6509860 msgr2=0x55fad6509ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.359+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/2897631126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6509860 0x55fad6509ce0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe064004ea0 tx=0x7fe064004ed0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.359+0000 7fe07fc01b80 1 -- 172.21.15.33:0/2897631126 shutdown_connections 2024-05-08T21:25:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.359+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/2897631126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6509860 0x55fad6509ce0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.359+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/2897631126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6508540 0x55fad6508960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.359+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/2897631126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 0x55fad6509320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.359+0000 7fe07fc01b80 1 -- 172.21.15.33:0/2897631126 >> 172.21.15.33:0/2897631126 conn(0x55fad6466210 msgr2=0x55fad6502de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.359+0000 7fe07fc01b80 1 -- 172.21.15.33:0/2897631126 shutdown_connections 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.359+0000 7fe07fc01b80 1 -- 172.21.15.33:0/2897631126 wait complete. 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe07fc01b80 1 -- start start 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe07fc01b80 1 Processor -- start 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe07fc01b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 0x55fad6511e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe07fc01b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6508ea0 0x55fad6512370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe07fc01b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6509860 0x55fad65193b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe07fc01b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad63f4560 con 0x55fad6508ea0 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe07fc01b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad63f4200 con 0x55fad6508540 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe07fc01b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad62db6a0 con 0x55fad6509860 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe07fc01b80 10 client.0 _ll_register_callbacks cb 0x55fad63fdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe0717e3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 0x55fad6511e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe070fe2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6508ea0 0x55fad6512370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe0717e3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 0x55fad6511e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56010/0 (socket says 172.21.15.33:56010) 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe070fe2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6508ea0 0x55fad6512370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44628/0 (socket says 172.21.15.33:44628) 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.360+0000 7fe0717e3700 1 -- 172.21.15.33:0/663921562 learned_addr learned my addr 172.21.15.33:0/663921562 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe0717e3700 1 -- 172.21.15.33:0/663921562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6509860 msgr2=0x55fad65193b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe0717e3700 1 --2- 172.21.15.33:0/663921562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6509860 0x55fad65193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe0717e3700 1 -- 172.21.15.33:0/663921562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6508ea0 msgr2=0x55fad6512370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe0717e3700 1 --2- 172.21.15.33:0/663921562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6508ea0 0x55fad6512370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe0717e3700 1 -- 172.21.15.33:0/663921562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05c00e810 con 0x55fad6508540 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe0717e3700 1 --2- 172.21.15.33:0/663921562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 0x55fad6511e30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe06800c5b0 tx=0x7fe06809a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9a2e4b66423420b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe054ff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe054ff9700 1 -- 172.21.15.33:0/663921562 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0680a2520 con 0x55fad6508540 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe054ff9700 1 -- 172.21.15.33:0/663921562 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe068099590 con 0x55fad6508540 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe07fc01b80 1 -- 172.21.15.33:0/663921562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fad63b1270 con 0x55fad6508540 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe054ff9700 1 -- 172.21.15.33:0/663921562 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe068099750 con 0x55fad6508540 2024-05-08T21:25:34.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe07fc01b80 10 client.27149 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.361+0000 7fe07fc01b80 1 -- 172.21.15.33:0/663921562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fe06c109ba0 con 0x55fad6508540 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.362+0000 7fe054ff9700 1 -- 172.21.15.33:0/663921562 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fad63b1270 con 0x55fad6508540 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.362+0000 7fe054ff9700 1 -- 172.21.15.33:0/663921562 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe0680a2090 con 0x55fad6508540 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.362+0000 7fe07fc01b80 10 client.27149 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.362+0000 7fe07fc01b80 10 client.27149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.362+0000 7fe07fc01b80 20 client.27149 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.362+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/663921562 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fad651dbe0 0x55fad6520000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.362+0000 7fe07fc01b80 4 client.27149 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.362+0000 7fe07fc01b80 1 -- 172.21.15.33:0/663921562 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fad64d7810 con 0x55fad651dbe0 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.362+0000 7fe070fe2700 1 --2- 172.21.15.33:0/663921562 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fad651dbe0 0x55fad6520000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.362+0000 7fe070fe2700 1 --2- 172.21.15.33:0/663921562 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fad651dbe0 0x55fad6520000 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f504e8ea56c514c server_cookie=48ce74c4b16f9f94 in_seq=0 out_seq=0 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.362+0000 7fe054ff9700 10 client.27149 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.363+0000 7fe054ff9700 1 -- 172.21.15.33:0/663921562 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fad64d7810 con 0x55fad651dbe0 2024-05-08T21:25:34.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.363+0000 7fe054ff9700 10 client.27149 handle_command_reply: tid=0 2024-05-08T21:25:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.363+0000 7fe07fc01b80 1 client.27149 shutdown 2024-05-08T21:25:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.363+0000 7fe07fc01b80 1 -- 172.21.15.33:0/663921562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 msgr2=0x55fad6511e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.363+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/663921562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 0x55fad6511e30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe06800c5b0 tx=0x7fe06809a890 comp rx=0 tx=0).stop 2024-05-08T21:25:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.365+0000 7fe07fc01b80 1 -- 172.21.15.33:0/663921562 shutdown_connections 2024-05-08T21:25:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.365+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/663921562 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fad651dbe0 0x55fad6520000 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.365+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/663921562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6509860 0x55fad65193b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.365+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/663921562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 0x55fad6511e30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.365+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/663921562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6508ea0 0x55fad6512370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.365+0000 7fe054ff9700 0 client.27149 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.365+0000 7fe07fc01b80 1 -- 172.21.15.33:0/663921562 >> 172.21.15.33:0/663921562 conn(0x55fad6466210 msgr2=0x55fad65025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.365+0000 7fe07fc01b80 1 -- 172.21.15.33:0/663921562 shutdown_connections 2024-05-08T21:25:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.365+0000 7fe07fc01b80 1 -- 172.21.15.33:0/663921562 wait complete. 2024-05-08T21:25:34.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.366+0000 7fe07fc01b80 20 client.27149 trim_cache size 0 max 16384 2024-05-08T21:25:34.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.387+0000 7fe07fc01b80 1 Processor -- start 2024-05-08T21:25:34.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.387+0000 7fe07fc01b80 1 -- start start 2024-05-08T21:25:34.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.387+0000 7fe07fc01b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 0x55fad6511340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.387+0000 7fe07fc01b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6508ea0 0x55fad6519370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe07fc01b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6509860 0x55fad650c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe07fc01b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad651b140 con 0x55fad6509860 2024-05-08T21:25:34.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe07fc01b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad6525210 con 0x55fad6508540 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe07fc01b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad64fd800 con 0x55fad6508ea0 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe0717e3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 0x55fad6511340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe071fe4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6509860 0x55fad650c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe070fe2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6508ea0 0x55fad6519370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe0717e3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 0x55fad6511340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56018/0 (socket says 172.21.15.33:56018) 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe070fe2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6508ea0 0x55fad6519370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49394/0 (socket says 172.21.15.33:49394) 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe0717e3700 1 -- 172.21.15.33:0/405370228 learned_addr learned my addr 172.21.15.33:0/405370228 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe070fe2700 1 -- 172.21.15.33:0/405370228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 msgr2=0x55fad6511340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe070fe2700 1 --2- 172.21.15.33:0/405370228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 0x55fad6511340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe070fe2700 1 -- 172.21.15.33:0/405370228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6509860 msgr2=0x55fad650c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe070fe2700 1 --2- 172.21.15.33:0/405370228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6509860 0x55fad650c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe070fe2700 1 -- 172.21.15.33:0/405370228 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fad6285b60 con 0x55fad6508ea0 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe071fe4700 1 --2- 172.21.15.33:0/405370228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6509860 0x55fad650c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.388+0000 7fe070fe2700 1 --2- 172.21.15.33:0/405370228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6508ea0 0x55fad6519370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55fad65208b0 tx=0x7fe05c015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59f73cb30588879f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.389+0000 7fe057fff700 1 -- 172.21.15.33:0/405370228 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c022040 con 0x55fad6508ea0 2024-05-08T21:25:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.389+0000 7fe057fff700 1 -- 172.21.15.33:0/405370228 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe05c0278f0 con 0x55fad6508ea0 2024-05-08T21:25:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.389+0000 7fe057fff700 1 -- 172.21.15.33:0/405370228 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c004a70 con 0x55fad6508ea0 2024-05-08T21:25:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.389+0000 7fe07fc01b80 1 -- 172.21.15.33:0/405370228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6508ea0 msgr2=0x55fad6519370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.389+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/405370228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6508ea0 0x55fad6519370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55fad65208b0 tx=0x7fe05c015000 comp rx=0 tx=0).stop 2024-05-08T21:25:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.389+0000 7fe07fc01b80 1 -- 172.21.15.33:0/405370228 shutdown_connections 2024-05-08T21:25:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.389+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/405370228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6509860 0x55fad650c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.389+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/405370228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508540 0x55fad6511340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.389+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/405370228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6508ea0 0x55fad6519370 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.389+0000 7fe07fc01b80 1 -- 172.21.15.33:0/405370228 >> 172.21.15.33:0/405370228 conn(0x55fad6466210 msgr2=0x55fad650fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:34.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.390+0000 7fe07fc01b80 1 -- 172.21.15.33:0/405370228 shutdown_connections 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.390+0000 7fe07fc01b80 1 -- 172.21.15.33:0/405370228 wait complete. 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.390+0000 7fe07fc01b80 1 -- start start 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.390+0000 7fe07fc01b80 1 Processor -- start 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.390+0000 7fe07fc01b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6508540 0x55fad6514f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.390+0000 7fe07fc01b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 0x55fad6515450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.390+0000 7fe07fc01b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6509860 0x55fad6518470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.390+0000 7fe07fc01b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad65189b0 con 0x55fad6509860 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.390+0000 7fe07fc01b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad6518b20 con 0x55fad6508ea0 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.390+0000 7fe07fc01b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad6518ce0 con 0x55fad6508540 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.390+0000 7fe07fc01b80 10 client.0 _ll_register_callbacks cb 0x55fad63fdb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe071fe4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6509860 0x55fad6518470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe070fe2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 0x55fad6515450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe070fe2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 0x55fad6515450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56024/0 (socket says 172.21.15.33:56024) 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe071fe4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6509860 0x55fad6518470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44654/0 (socket says 172.21.15.33:44654) 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe070fe2700 1 -- 172.21.15.33:0/4157026219 learned_addr learned my addr 172.21.15.33:0/4157026219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe070fe2700 1 -- 172.21.15.33:0/4157026219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6508540 msgr2=0x55fad6514f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe070fe2700 1 --2- 172.21.15.33:0/4157026219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6508540 0x55fad6514f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe070fe2700 1 -- 172.21.15.33:0/4157026219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6509860 msgr2=0x55fad6518470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe070fe2700 1 --2- 172.21.15.33:0/4157026219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6509860 0x55fad6518470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe070fe2700 1 -- 172.21.15.33:0/4157026219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fad62d93b0 con 0x55fad6508ea0 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe070fe2700 1 --2- 172.21.15.33:0/4157026219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 0x55fad6515450 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe05c011fd0 tx=0x7fe05c027460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eca760980ee6313 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe054ff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.391+0000 7fe054ff9700 1 -- 172.21.15.33:0/4157026219 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c022040 con 0x55fad6508ea0 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe07fc01b80 1 -- 172.21.15.33:0/4157026219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe064007420 con 0x55fad6508ea0 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe054ff9700 1 -- 172.21.15.33:0/4157026219 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe05c010a50 con 0x55fad6508ea0 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe054ff9700 1 -- 172.21.15.33:0/4157026219 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe05c010c10 con 0x55fad6508ea0 2024-05-08T21:25:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe07fc01b80 10 client.27155 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe07fc01b80 1 -- 172.21.15.33:0/4157026219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fe05c00eb00 con 0x55fad6508ea0 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe054ff9700 1 -- 172.21.15.33:0/4157026219 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe064007420 con 0x55fad6508ea0 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe054ff9700 1 -- 172.21.15.33:0/4157026219 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe05c038030 con 0x55fad6508ea0 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe07fc01b80 10 client.27155 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe07fc01b80 10 client.27155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe07fc01b80 20 client.27155 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/4157026219 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fad6574f30 0x55fad6577350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe07fc01b80 4 client.27155 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.392+0000 7fe07fc01b80 1 -- 172.21.15.33:0/4157026219 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fad6450ec0 con 0x55fad6574f30 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.393+0000 7fe0717e3700 1 --2- 172.21.15.33:0/4157026219 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fad6574f30 0x55fad6577350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.393+0000 7fe054ff9700 10 client.27155 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.393+0000 7fe0717e3700 1 --2- 172.21.15.33:0/4157026219 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fad6574f30 0x55fad6577350 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d9aefe1d78a5d59 server_cookie=8e5f2b4edaae4f6b in_seq=0 out_seq=0 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.393+0000 7fe054ff9700 1 -- 172.21.15.33:0/4157026219 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fad6450ec0 con 0x55fad6574f30 2024-05-08T21:25:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.393+0000 7fe054ff9700 10 client.27155 handle_command_reply: tid=0 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.393+0000 7fe07fc01b80 1 client.27155 shutdown 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe07fc01b80 1 -- 172.21.15.33:0/4157026219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 msgr2=0x55fad6515450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/4157026219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 0x55fad6515450 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe05c011fd0 tx=0x7fe05c027460 comp rx=0 tx=0).stop 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe07fc01b80 1 -- 172.21.15.33:0/4157026219 shutdown_connections 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/4157026219 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55fad6574f30 0x55fad6577350 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/4157026219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fad6509860 0x55fad6518470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/4157026219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fad6508540 0x55fad6514f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe07fc01b80 1 --2- 172.21.15.33:0/4157026219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fad6508ea0 0x55fad6515450 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe054ff9700 0 client.27155 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe07fc01b80 1 -- 172.21.15.33:0/4157026219 >> 172.21.15.33:0/4157026219 conn(0x55fad6466210 msgr2=0x55fad6511be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe07fc01b80 1 -- 172.21.15.33:0/4157026219 shutdown_connections 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe07fc01b80 1 -- 172.21.15.33:0/4157026219 wait complete. 2024-05-08T21:25:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe07fc01b80 20 client.27155 trim_cache size 0 max 16384 2024-05-08T21:25:34.402 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:25:34.402 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:25:34.402 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:25:34.402 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:25:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe073a47700 1 -- 172.21.15.33:0/2788747172 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe0580a1120 msgr2=0x7fe0580a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe073a47700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe0580a1120 0x7fe0580a35e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe05c005950 tx=0x7fe05c0058e0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe073a47700 1 -- 172.21.15.33:0/2788747172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe06c1018e0 msgr2=0x7fe06c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.394+0000 7fe073a47700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe06c1018e0 0x7fe06c101200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe06800e9c0 tx=0x7fe06800ecd0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.395+0000 7fe073a47700 1 -- 172.21.15.33:0/2788747172 shutdown_connections 2024-05-08T21:25:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.395+0000 7fe073a47700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe0580a1120 0x7fe0580a35e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.395+0000 7fe073a47700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06c106b40 0x7fe06c0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.395+0000 7fe073a47700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe06c1018e0 0x7fe06c101200 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.395+0000 7fe073a47700 1 --2- 172.21.15.33:0/2788747172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe06c104210 0x7fe06c0ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.395+0000 7fe073a47700 1 -- 172.21.15.33:0/2788747172 >> 172.21.15.33:0/2788747172 conn(0x7fe06c0fb290 msgr2=0x7fe06c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:34.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.395+0000 7fe073a47700 1 -- 172.21.15.33:0/2788747172 shutdown_connections 2024-05-08T21:25:34.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.395+0000 7fe073a47700 1 -- 172.21.15.33:0/2788747172 wait complete. 2024-05-08T21:25:34.413 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:25:34.413 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:25:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.576+0000 7f29ba9d0700 1 -- 172.21.15.33:0/3455520646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4102ac0 msgr2=0x7f29b4109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.576+0000 7f29ba9d0700 1 --2- 172.21.15.33:0/3455520646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4102ac0 0x7f29b4109370 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00b0a0 tx=0x7f29ac00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.576+0000 7f29ba9d0700 1 -- 172.21.15.33:0/3455520646 shutdown_connections 2024-05-08T21:25:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.576+0000 7f29ba9d0700 1 --2- 172.21.15.33:0/3455520646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4102ac0 0x7f29b4109370 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.576+0000 7f29ba9d0700 1 --2- 172.21.15.33:0/3455520646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b40ff150 0x7f29b40ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.576+0000 7f29ba9d0700 1 --2- 172.21.15.33:0/3455520646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102120 0x7f29b4102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.576+0000 7f29ba9d0700 1 -- 172.21.15.33:0/3455520646 >> 172.21.15.33:0/3455520646 conn(0x7f29b40faa60 msgr2=0x7f29b40fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.577+0000 7f29ba9d0700 1 -- 172.21.15.33:0/3455520646 shutdown_connections 2024-05-08T21:25:34.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.577+0000 7f29ba9d0700 1 -- 172.21.15.33:0/3455520646 wait complete. 2024-05-08T21:25:34.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.577+0000 7f29ba9d0700 1 Processor -- start 2024-05-08T21:25:34.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.577+0000 7f29ba9d0700 1 -- start start 2024-05-08T21:25:34.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29ba9d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b40ff150 0x7f29b4101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29ba9d0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4102120 0x7f29b41000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29ba9d0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102ac0 0x7f29b4100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29ba9d0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b4100db0 con 0x7f29b40ff150 2024-05-08T21:25:34.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29ba9d0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b4100f20 con 0x7f29b4102120 2024-05-08T21:25:34.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29ba9d0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b41a0e60 con 0x7f29b4102ac0 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b8f6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102ac0 0x7f29b4100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b40ff150 0x7f29b4101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b8f6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102ac0 0x7f29b4100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49428/0 (socket says 172.21.15.33:49428) 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b8f6d700 1 -- 172.21.15.33:0/2158554989 learned_addr learned my addr 172.21.15.33:0/2158554989 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b40ff150 0x7f29b4101a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44686/0 (socket says 172.21.15.33:44686) 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b37fe700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4102120 0x7f29b41000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b8f6d700 1 -- 172.21.15.33:0/2158554989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4102120 msgr2=0x7f29b41000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b8f6d700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4102120 0x7f29b41000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b8f6d700 1 -- 172.21.15.33:0/2158554989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b40ff150 msgr2=0x7f29b4101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b8f6d700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b40ff150 0x7f29b4101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b8f6d700 1 -- 172.21.15.33:0/2158554989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac009d00 con 0x7f29b4102ac0 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b37fe700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4102120 0x7f29b41000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.578+0000 7f29b3fff700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b40ff150 0x7f29b4101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.579+0000 7f29b8f6d700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102ac0 0x7f29b4100610 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00b780 tx=0x7f29ac00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.579+0000 7f29b17fa700 1 -- 172.21.15.33:0/2158554989 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00e040 con 0x7f29b4102ac0 2024-05-08T21:25:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.579+0000 7f29ba9d0700 1 -- 172.21.15.33:0/2158554989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b41a1090 con 0x7f29b4102ac0 2024-05-08T21:25:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.579+0000 7f29b17fa700 1 -- 172.21.15.33:0/2158554989 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f29ac003c00 con 0x7f29b4102ac0 2024-05-08T21:25:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.579+0000 7f29b17fa700 1 -- 172.21.15.33:0/2158554989 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0052a0 con 0x7f29b4102ac0 2024-05-08T21:25:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.579+0000 7f29ba9d0700 1 -- 172.21.15.33:0/2158554989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b41a14b0 con 0x7f29b4102ac0 2024-05-08T21:25:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.581+0000 7f29ba9d0700 1 -- 172.21.15.33:0/2158554989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b41083b0 con 0x7f29b4102ac0 2024-05-08T21:25:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.581+0000 7f29b17fa700 1 -- 172.21.15.33:0/2158554989 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f29ac005400 con 0x7f29b4102ac0 2024-05-08T21:25:34.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.582+0000 7f29b17fa700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f299c0a1120 0x7f299c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:34.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.582+0000 7f29b3fff700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f299c0a1120 0x7f299c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:34.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.582+0000 7f29b17fa700 1 -- 172.21.15.33:0/2158554989 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f29ac0cdcf0 con 0x7f29b4102ac0 2024-05-08T21:25:34.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.583+0000 7f29b3fff700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f299c0a1120 0x7f299c0a35e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f29b4069270 tx=0x7f29a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.587+0000 7f29b17fa700 1 -- 172.21.15.33:0/2158554989 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f29ac097190 con 0x7f29b4102ac0 2024-05-08T21:25:34.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.918+0000 7f29ba9d0700 1 -- 172.21.15.33:0/2158554989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29b410c7f0 con 0x7f29b4102ac0 2024-05-08T21:25:34.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.920+0000 7f29b17fa700 1 -- 172.21.15.33:0/2158554989 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10899 (secure 0 0 0) 0x7f29ac09a7b0 con 0x7f29b4102ac0 2024-05-08T21:25:34.921 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:25:34.922 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:25:13.388928+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:25:34.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.922+0000 7f29ba9d0700 1 -- 172.21.15.33:0/2158554989 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f299c0a1120 msgr2=0x7f299c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.922+0000 7f29ba9d0700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f299c0a1120 0x7f299c0a35e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f29b4069270 tx=0x7f29a4009260 comp rx=0 tx=0).stop 2024-05-08T21:25:34.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.922+0000 7f29ba9d0700 1 -- 172.21.15.33:0/2158554989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102ac0 msgr2=0x7f29b4100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:34.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.922+0000 7f29ba9d0700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102ac0 0x7f29b4100610 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00b780 tx=0x7f29ac00bab0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.922+0000 7f29ba9d0700 1 -- 172.21.15.33:0/2158554989 shutdown_connections 2024-05-08T21:25:34.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.922+0000 7f29ba9d0700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29b4102ac0 0x7f29b4100610 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.922+0000 7f29ba9d0700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b40ff150 0x7f29b4101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.922+0000 7f29ba9d0700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f299c0a1120 0x7f299c0a35e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.922+0000 7f29ba9d0700 1 --2- 172.21.15.33:0/2158554989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4102120 0x7f29b41000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:34.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.922+0000 7f29ba9d0700 1 -- 172.21.15.33:0/2158554989 >> 172.21.15.33:0/2158554989 conn(0x7f29b40faa60 msgr2=0x7f29b41054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:34.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.922+0000 7f29ba9d0700 1 -- 172.21.15.33:0/2158554989 shutdown_connections 2024-05-08T21:25:34.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:34.922+0000 7f29ba9d0700 1 -- 172.21.15.33:0/2158554989 wait complete. 2024-05-08T21:25:34.926 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 39 2024-05-08T21:25:34.942 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:25:35.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.102+0000 7f74bc611700 1 -- 172.21.15.33:0/2743075949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b4101940 msgr2=0x7f74b4103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.102+0000 7f74bc611700 1 --2- 172.21.15.33:0/2743075949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b4101940 0x7f74b4103d30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009990 tx=0x7f74b0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.104+0000 7f74bc611700 1 -- 172.21.15.33:0/2743075949 shutdown_connections 2024-05-08T21:25:35.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.104+0000 7f74bc611700 1 --2- 172.21.15.33:0/2743075949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74b4106ba0 0x7f74b4109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.104+0000 7f74bc611700 1 --2- 172.21.15.33:0/2743075949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b4101940 0x7f74b4103d30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.104+0000 7f74bc611700 1 --2- 172.21.15.33:0/2743075949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74b4104270 0x7f74b4106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.104+0000 7f74bc611700 1 -- 172.21.15.33:0/2743075949 >> 172.21.15.33:0/2743075949 conn(0x7f74b40fb330 msgr2=0x7f74b40fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.106+0000 7f74bc611700 1 -- 172.21.15.33:0/2743075949 shutdown_connections 2024-05-08T21:25:35.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.106+0000 7f74bc611700 1 -- 172.21.15.33:0/2743075949 wait complete. 2024-05-08T21:25:35.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.106+0000 7f74bc611700 1 Processor -- start 2024-05-08T21:25:35.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.106+0000 7f74bc611700 1 -- start start 2024-05-08T21:25:35.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74bc611700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b4101940 0x7f74b4196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74bc611700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74b4104270 0x7f74b4197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74bc611700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74b4106ba0 0x7f74b419e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74bc611700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b419e810 con 0x7f74b4101940 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74bc611700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b419e980 con 0x7f74b4106ba0 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74bc611700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b419eb40 con 0x7f74b4104270 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74ba3ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b4101940 0x7f74b4196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74ba3ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b4101940 0x7f74b4196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44702/0 (socket says 172.21.15.33:44702) 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74ba3ad700 1 -- 172.21.15.33:0/351953167 learned_addr learned my addr 172.21.15.33:0/351953167 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74b9bac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74b4104270 0x7f74b4197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74babae700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74b4106ba0 0x7f74b419e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74b9bac700 1 -- 172.21.15.33:0/351953167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74b4106ba0 msgr2=0x7f74b419e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74b9bac700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74b4106ba0 0x7f74b419e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74b9bac700 1 -- 172.21.15.33:0/351953167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b4101940 msgr2=0x7f74b4196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74b9bac700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b4101940 0x7f74b4196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74b9bac700 1 -- 172.21.15.33:0/351953167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac009c40 con 0x7f74b4104270 2024-05-08T21:25:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.107+0000 7f74babae700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74b4106ba0 0x7f74b419e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.108+0000 7f74b9bac700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74b4104270 0x7f74b4197230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f74a400e9f0 tx=0x7f74a400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.108+0000 7f74ab7fe700 1 -- 172.21.15.33:0/351953167 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a400c9c0 con 0x7f74b4104270 2024-05-08T21:25:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.108+0000 7f74bc611700 1 -- 172.21.15.33:0/351953167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b00095f0 con 0x7f74b4104270 2024-05-08T21:25:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.108+0000 7f74ab7fe700 1 -- 172.21.15.33:0/351953167 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74a4010430 con 0x7f74b4104270 2024-05-08T21:25:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.108+0000 7f74ab7fe700 1 -- 172.21.15.33:0/351953167 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4018680 con 0x7f74b4104270 2024-05-08T21:25:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.108+0000 7f74bc611700 1 -- 172.21.15.33:0/351953167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b419f1a0 con 0x7f74b4104270 2024-05-08T21:25:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.110+0000 7f74ab7fe700 1 -- 172.21.15.33:0/351953167 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f74a40105a0 con 0x7f74b4104270 2024-05-08T21:25:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.112+0000 7f74ab7fe700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f74a00a1120 0x7f74a00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.112+0000 7f74ab7fe700 1 -- 172.21.15.33:0/351953167 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f74a4014070 con 0x7f74b4104270 2024-05-08T21:25:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.112+0000 7f74ba3ad700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f74a00a1120 0x7f74a00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.113+0000 7f74ba3ad700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f74a00a1120 0x7f74a00a35e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f74b000b2e0 tx=0x7f74b000b630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.113+0000 7f74c87cbb80 1 Processor -- start 2024-05-08T21:25:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.113+0000 7f74c87cbb80 1 -- start start 2024-05-08T21:25:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.113+0000 7f74c87cbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3540 0x5587ac4d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.113+0000 7f74c87cbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d3ea0 0x5587ac4d4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.113+0000 7f74c87cbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d4860 0x5587ac4d4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.113+0000 7f74c87cbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ac42a150 con 0x5587ac4d3ea0 2024-05-08T21:25:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.113+0000 7f74c87cbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ac427b10 con 0x5587ac4d4860 2024-05-08T21:25:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.113+0000 7f74c87cbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ac423400 con 0x5587ac4d3540 2024-05-08T21:25:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74ba3ad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3540 0x5587ac4d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74b9bac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d3ea0 0x5587ac4d4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74ba3ad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3540 0x5587ac4d3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49444/0 (socket says 172.21.15.33:49444) 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74ba3ad700 1 -- 172.21.15.33:0/3086791679 learned_addr learned my addr 172.21.15.33:0/3086791679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74babae700 1 --2- 172.21.15.33:0/3086791679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d4860 0x5587ac4d4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74ba3ad700 1 -- 172.21.15.33:0/3086791679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d4860 msgr2=0x5587ac4d4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74ba3ad700 1 --2- 172.21.15.33:0/3086791679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d4860 0x5587ac4d4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74ba3ad700 1 -- 172.21.15.33:0/3086791679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d3ea0 msgr2=0x5587ac4d4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74ba3ad700 1 --2- 172.21.15.33:0/3086791679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d3ea0 0x5587ac4d4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74ba3ad700 1 -- 172.21.15.33:0/3086791679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587ac2a43b0 con 0x5587ac4d3540 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74babae700 1 --2- 172.21.15.33:0/3086791679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d4860 0x5587ac4d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74ba3ad700 1 --2- 172.21.15.33:0/3086791679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3540 0x5587ac4d3960 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f74b0005420 tx=0x7f74b0020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28ee616c50c31a80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74a8ff9700 1 -- 172.21.15.33:0/3086791679 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0030070 con 0x5587ac4d3540 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74a8ff9700 1 -- 172.21.15.33:0/3086791679 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74b001e040 con 0x5587ac4d3540 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.114+0000 7f74a8ff9700 1 -- 172.21.15.33:0/3086791679 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0004500 con 0x5587ac4d3540 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.115+0000 7f74c87cbb80 1 -- 172.21.15.33:0/3086791679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3540 msgr2=0x5587ac4d3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.115+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/3086791679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3540 0x5587ac4d3960 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f74b0005420 tx=0x7f74b0020000 comp rx=0 tx=0).stop 2024-05-08T21:25:35.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.115+0000 7f74c87cbb80 1 -- 172.21.15.33:0/3086791679 shutdown_connections 2024-05-08T21:25:35.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.115+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/3086791679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d4860 0x5587ac4d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.115+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/3086791679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3540 0x5587ac4d3960 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.115+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/3086791679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d3ea0 0x5587ac4d4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.115+0000 7f74c87cbb80 1 -- 172.21.15.33:0/3086791679 >> 172.21.15.33:0/3086791679 conn(0x5587ac431210 msgr2=0x5587ac4cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.115+0000 7f74c87cbb80 1 -- 172.21.15.33:0/3086791679 shutdown_connections 2024-05-08T21:25:35.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.115+0000 7f74c87cbb80 1 -- 172.21.15.33:0/3086791679 wait complete. 2024-05-08T21:25:35.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74c87cbb80 1 -- start start 2024-05-08T21:25:35.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74c87cbb80 1 Processor -- start 2024-05-08T21:25:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74c87cbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d3540 0x5587ac4dcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74c87cbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3ea0 0x5587ac4dd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74c87cbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d4860 0x5587ac4e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74c87cbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ac3bf560 con 0x5587ac4d3540 2024-05-08T21:25:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74c87cbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ac3bf200 con 0x5587ac4d3ea0 2024-05-08T21:25:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74c87cbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ac2a66a0 con 0x5587ac4d4860 2024-05-08T21:25:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74c87cbb80 10 client.0 _ll_register_callbacks cb 0x5587ac3c8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74ba3ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d3540 0x5587ac4dcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74b9bac700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3ea0 0x5587ac4dd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74ba3ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d3540 0x5587ac4dcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44724/0 (socket says 172.21.15.33:44724) 2024-05-08T21:25:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74ba3ad700 1 -- 172.21.15.33:0/1296413039 learned_addr learned my addr 172.21.15.33:0/1296413039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.116+0000 7f74babae700 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d4860 0x5587ac4e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f74b9bac700 1 -- 172.21.15.33:0/1296413039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d4860 msgr2=0x5587ac4e42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f74b9bac700 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d4860 0x5587ac4e42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f74b9bac700 1 -- 172.21.15.33:0/1296413039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d3540 msgr2=0x5587ac4dcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f74b9bac700 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d3540 0x5587ac4dcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f74b9bac700 1 -- 172.21.15.33:0/1296413039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac0091b0 con 0x5587ac4d3ea0 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f74babae700 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d4860 0x5587ac4e42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f74b9bac700 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3ea0 0x5587ac4dd2b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f74a40ca700 tx=0x7f74a409a590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677577e404500646 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f749dffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f749dffb700 1 -- 172.21.15.33:0/1296413039 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40b1070 con 0x5587ac4d3ea0 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f749dffb700 1 -- 172.21.15.33:0/1296413039 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f74a409a050 con 0x5587ac4d3ea0 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f74c87cbb80 1 -- 172.21.15.33:0/1296413039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587ac37c270 con 0x5587ac4d3ea0 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f749dffb700 1 -- 172.21.15.33:0/1296413039 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74a409de80 con 0x5587ac4d3ea0 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f74c87cbb80 10 client.27173 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:35.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.117+0000 7f74c87cbb80 1 -- 172.21.15.33:0/1296413039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f74b4109c00 con 0x5587ac4d3ea0 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.118+0000 7f749dffb700 1 -- 172.21.15.33:0/1296413039 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74a4063080 con 0x5587ac4d3ea0 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.118+0000 7f749dffb700 1 -- 172.21.15.33:0/1296413039 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74a40cabc0 con 0x5587ac4d3ea0 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.118+0000 7f74c87cbb80 10 client.27173 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.118+0000 7f74c87cbb80 10 client.27173 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.118+0000 7f74c87cbb80 20 client.27173 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.118+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5587ac4e8b00 0x5587ac4eaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.118+0000 7f74c87cbb80 4 client.27173 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.118+0000 7f74c87cbb80 1 -- 172.21.15.33:0/1296413039 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587ac4a2810 con 0x5587ac4e8b00 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.118+0000 7f74ba3ad700 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5587ac4e8b00 0x5587ac4eaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.118+0000 7f749dffb700 10 client.27173 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.118+0000 7f74ba3ad700 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5587ac4e8b00 0x5587ac4eaf20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b6b74595199824ae server_cookie=c370473872e9d8cb in_seq=0 out_seq=0 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f749dffb700 1 -- 172.21.15.33:0/1296413039 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587ac4a2810 con 0x5587ac4e8b00 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f749dffb700 10 client.27173 handle_command_reply: tid=0 2024-05-08T21:25:35.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f74c87cbb80 1 client.27173 shutdown 2024-05-08T21:25:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f74c87cbb80 1 -- 172.21.15.33:0/1296413039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3ea0 msgr2=0x5587ac4dd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3ea0 0x5587ac4dd2b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f74a40ca700 tx=0x7f74a409a590 comp rx=0 tx=0).stop 2024-05-08T21:25:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f74c87cbb80 1 -- 172.21.15.33:0/1296413039 shutdown_connections 2024-05-08T21:25:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5587ac4e8b00 0x5587ac4eaf20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d4860 0x5587ac4e42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d3540 0x5587ac4dcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/1296413039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3ea0 0x5587ac4dd2b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f749dffb700 0 client.27173 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:25:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f74c87cbb80 1 -- 172.21.15.33:0/1296413039 >> 172.21.15.33:0/1296413039 conn(0x5587ac431210 msgr2=0x5587ac4cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f74c87cbb80 1 -- 172.21.15.33:0/1296413039 shutdown_connections 2024-05-08T21:25:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f74c87cbb80 1 -- 172.21.15.33:0/1296413039 wait complete. 2024-05-08T21:25:35.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.119+0000 7f74c87cbb80 20 client.27173 trim_cache size 0 max 16384 2024-05-08T21:25:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74c87cbb80 1 Processor -- start 2024-05-08T21:25:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74c87cbb80 1 -- start start 2024-05-08T21:25:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74c87cbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3540 0x5587ac4dad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74c87cbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3ea0 0x5587ac4e42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74c87cbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d4860 0x5587ac4dba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74c87cbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ac4f0130 con 0x5587ac4d4860 2024-05-08T21:25:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74c87cbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ac4db600 con 0x5587ac4d3540 2024-05-08T21:25:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74c87cbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ac4daa10 con 0x5587ac4d3ea0 2024-05-08T21:25:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74ba3ad700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3540 0x5587ac4dad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74b9bac700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3ea0 0x5587ac4e42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74ba3ad700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3540 0x5587ac4dad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56070/0 (socket says 172.21.15.33:56070) 2024-05-08T21:25:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74ba3ad700 1 -- 172.21.15.33:0/2595379993 learned_addr learned my addr 172.21.15.33:0/2595379993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74babae700 1 --2- 172.21.15.33:0/2595379993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d4860 0x5587ac4dba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74ba3ad700 1 -- 172.21.15.33:0/2595379993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3ea0 msgr2=0x5587ac4e42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74ba3ad700 1 --2- 172.21.15.33:0/2595379993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3ea0 0x5587ac4e42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74ba3ad700 1 -- 172.21.15.33:0/2595379993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d4860 msgr2=0x5587ac4dba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74ba3ad700 1 --2- 172.21.15.33:0/2595379993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d4860 0x5587ac4dba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.138+0000 7f74ba3ad700 1 -- 172.21.15.33:0/2595379993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587ac250b60 con 0x5587ac4d3540 2024-05-08T21:25:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.140+0000 7f74ba3ad700 1 --2- 172.21.15.33:0/2595379993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3540 0x5587ac4dad90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5587ac4eb830 tx=0x7f74b0023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=960a89a1c37c1447 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.140+0000 7f74a97fa700 1 -- 172.21.15.33:0/2595379993 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0030040 con 0x5587ac4d3540 2024-05-08T21:25:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.140+0000 7f74a97fa700 1 -- 172.21.15.33:0/2595379993 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74b000be10 con 0x5587ac4d3540 2024-05-08T21:25:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.140+0000 7f74a97fa700 1 -- 172.21.15.33:0/2595379993 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0041780 con 0x5587ac4d3540 2024-05-08T21:25:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.141+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2595379993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3540 msgr2=0x5587ac4dad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.141+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/2595379993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3540 0x5587ac4dad90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5587ac4eb830 tx=0x7f74b0023000 comp rx=0 tx=0).stop 2024-05-08T21:25:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.141+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2595379993 shutdown_connections 2024-05-08T21:25:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.141+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/2595379993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d4860 0x5587ac4dba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.141+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/2595379993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3540 0x5587ac4dad90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.141+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/2595379993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3ea0 0x5587ac4e42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.141+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2595379993 >> 172.21.15.33:0/2595379993 conn(0x5587ac431210 msgr2=0x5587ac4ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.141+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2595379993 shutdown_connections 2024-05-08T21:25:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.141+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2595379993 wait complete. 2024-05-08T21:25:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.141+0000 7f74c87cbb80 1 -- start start 2024-05-08T21:25:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.141+0000 7f74c87cbb80 1 Processor -- start 2024-05-08T21:25:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74c87cbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3540 0x5587ac4dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74c87cbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3ea0 0x5587ac4dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74c87cbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d4860 0x5587ac4e2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74c87cbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ac4e3410 con 0x5587ac4d4860 2024-05-08T21:25:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74c87cbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ac4e3580 con 0x5587ac4d3ea0 2024-05-08T21:25:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74c87cbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587ac4e3740 con 0x5587ac4d3540 2024-05-08T21:25:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74c87cbb80 10 client.0 _ll_register_callbacks cb 0x5587ac3c8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74babae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d4860 0x5587ac4e2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74ba3ad700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3540 0x5587ac4dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74babae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d4860 0x5587ac4e2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44744/0 (socket says 172.21.15.33:44744) 2024-05-08T21:25:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74babae700 1 -- 172.21.15.33:0/2830592703 learned_addr learned my addr 172.21.15.33:0/2830592703 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74b9bac700 1 --2- 172.21.15.33:0/2830592703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3ea0 0x5587ac4dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74b9bac700 1 -- 172.21.15.33:0/2830592703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3540 msgr2=0x5587ac4dfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74b9bac700 1 --2- 172.21.15.33:0/2830592703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3540 0x5587ac4dfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74b9bac700 1 -- 172.21.15.33:0/2830592703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d4860 msgr2=0x5587ac4e2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74b9bac700 1 --2- 172.21.15.33:0/2830592703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d4860 0x5587ac4e2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74b9bac700 1 -- 172.21.15.33:0/2830592703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587ac2a43b0 con 0x5587ac4d3ea0 2024-05-08T21:25:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.142+0000 7f74b9bac700 1 --2- 172.21.15.33:0/2830592703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3ea0 0x5587ac4dffa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f74a40b8ce0 tx=0x7f74a40964c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec48492d4a82b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f749dffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f749dffb700 1 -- 172.21.15.33:0/2830592703 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40b1040 con 0x5587ac4d3ea0 2024-05-08T21:25:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f749dffb700 1 -- 172.21.15.33:0/2830592703 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f74a409a050 con 0x5587ac4d3ea0 2024-05-08T21:25:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2830592703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74b001e040 con 0x5587ac4d3ea0 2024-05-08T21:25:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f749dffb700 1 -- 172.21.15.33:0/2830592703 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74a409de80 con 0x5587ac4d3ea0 2024-05-08T21:25:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 10 client.27179 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2830592703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5587ac3e9560 con 0x5587ac4d3ea0 2024-05-08T21:25:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f749dffb700 1 -- 172.21.15.33:0/2830592703 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74b001e040 con 0x5587ac4d3ea0 2024-05-08T21:25:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f749dffb700 1 -- 172.21.15.33:0/2830592703 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74a4095e40 con 0x5587ac4d3ea0 2024-05-08T21:25:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 10 client.27179 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 10 client.27179 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:25:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 20 client.27179 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/2830592703 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5587ac550580 0x5587ac5529c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 4 client.27179 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:25:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2830592703 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5587ac41bec0 con 0x5587ac550580 2024-05-08T21:25:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74ba3ad700 1 --2- 172.21.15.33:0/2830592703 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5587ac550580 0x5587ac5529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f749dffb700 10 client.27179 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:25:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74ba3ad700 1 --2- 172.21.15.33:0/2830592703 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5587ac550580 0x5587ac5529c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=662dcf57fa59a87e server_cookie=eaacdcf9cb0f4f79 in_seq=0 out_seq=0 2024-05-08T21:25:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f749dffb700 1 -- 172.21.15.33:0/2830592703 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5587ac41bec0 con 0x5587ac550580 2024-05-08T21:25:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f749dffb700 10 client.27179 handle_command_reply: tid=0 2024-05-08T21:25:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 client.27179 shutdown 2024-05-08T21:25:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2830592703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3ea0 msgr2=0x5587ac4dffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/2830592703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3ea0 0x5587ac4dffa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f74a40b8ce0 tx=0x7f74a40964c0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2830592703 shutdown_connections 2024-05-08T21:25:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/2830592703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587ac4d4860 0x5587ac4e2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/2830592703 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x5587ac550580 0x5587ac5529c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/2830592703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587ac4d3540 0x5587ac4dfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 --2- 172.21.15.33:0/2830592703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587ac4d3ea0 0x5587ac4dffa0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f749dffb700 0 client.27179 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:25:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.143+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2830592703 >> 172.21.15.33:0/2830592703 conn(0x5587ac431210 msgr2=0x5587ac4d7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2830592703 shutdown_connections 2024-05-08T21:25:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74c87cbb80 1 -- 172.21.15.33:0/2830592703 wait complete. 2024-05-08T21:25:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74c87cbb80 20 client.27179 trim_cache size 0 max 16384 2024-05-08T21:25:35.153 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:25:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74bc611700 1 -- 172.21.15.33:0/351953167 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f74a00a1120 msgr2=0x7f74a00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74bc611700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f74a00a1120 0x7f74a00a35e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f74b000b2e0 tx=0x7f74b000b630 comp rx=0 tx=0).stop 2024-05-08T21:25:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74bc611700 1 -- 172.21.15.33:0/351953167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74b4104270 msgr2=0x7f74b4197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74bc611700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74b4104270 0x7f74b4197230 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f74a400e9f0 tx=0x7f74a400ed00 comp rx=0 tx=0).stop 2024-05-08T21:25:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74bc611700 1 -- 172.21.15.33:0/351953167 shutdown_connections 2024-05-08T21:25:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74bc611700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f74a00a1120 0x7f74a00a35e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74bc611700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74b4106ba0 0x7f74b419e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74bc611700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74b4101940 0x7f74b4196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74bc611700 1 --2- 172.21.15.33:0/351953167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74b4104270 0x7f74b4197230 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74bc611700 1 -- 172.21.15.33:0/351953167 >> 172.21.15.33:0/351953167 conn(0x7f74b40fb330 msgr2=0x7f74b4102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74bc611700 1 -- 172.21.15.33:0/351953167 shutdown_connections 2024-05-08T21:25:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.144+0000 7f74bc611700 1 -- 172.21.15.33:0/351953167 wait complete. 2024-05-08T21:25:35.163 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:25:35.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.340+0000 7fbcf16e2700 1 -- 172.21.15.33:0/2571188876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec101390 msgr2=0x7fbcec1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.340+0000 7fbcf16e2700 1 --2- 172.21.15.33:0/2571188876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec101390 0x7fbcec1017b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009990 tx=0x7fbcdc009ca0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.340+0000 7fbcf16e2700 1 -- 172.21.15.33:0/2571188876 shutdown_connections 2024-05-08T21:25:35.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.340+0000 7fbcf16e2700 1 --2- 172.21.15.33:0/2571188876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec1009a0 0x7fbcec108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.340+0000 7fbcf16e2700 1 --2- 172.21.15.33:0/2571188876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec101390 0x7fbcec1017b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.340+0000 7fbcf16e2700 1 --2- 172.21.15.33:0/2571188876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcec0fffe0 0x7fbcec100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.340+0000 7fbcf16e2700 1 -- 172.21.15.33:0/2571188876 >> 172.21.15.33:0/2571188876 conn(0x7fbcec0faad0 msgr2=0x7fbcec0fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.340+0000 7fbcf16e2700 1 -- 172.21.15.33:0/2571188876 shutdown_connections 2024-05-08T21:25:35.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.340+0000 7fbcf16e2700 1 -- 172.21.15.33:0/2571188876 wait complete. 2024-05-08T21:25:35.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.341+0000 7fbcf16e2700 1 Processor -- start 2024-05-08T21:25:35.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.341+0000 7fbcf16e2700 1 -- start start 2024-05-08T21:25:35.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.341+0000 7fbcf16e2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec0fffe0 0x7fbcec0ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.341+0000 7fbcf16e2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcec1009a0 0x7fbcec19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.341+0000 7fbcf16e2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec101390 0x7fbcec19e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.341+0000 7fbcf16e2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec19e7e0 con 0x7fbcec101390 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.341+0000 7fbcf16e2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec19e950 con 0x7fbcec0fffe0 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.341+0000 7fbcf16e2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec19eb10 con 0x7fbcec1009a0 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbceb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec101390 0x7fbcec19e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcea7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcec1009a0 0x7fbcec19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbceaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec0fffe0 0x7fbcec0ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcea7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcec1009a0 0x7fbcec19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49502/0 (socket says 172.21.15.33:49502) 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbceb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec101390 0x7fbcec19e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44752/0 (socket says 172.21.15.33:44752) 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcea7fc700 1 -- 172.21.15.33:0/3308485046 learned_addr learned my addr 172.21.15.33:0/3308485046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcea7fc700 1 -- 172.21.15.33:0/3308485046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec0fffe0 msgr2=0x7fbcec0ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcea7fc700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec0fffe0 0x7fbcec0ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcea7fc700 1 -- 172.21.15.33:0/3308485046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec101390 msgr2=0x7fbcec19e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcea7fc700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec101390 0x7fbcec19e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcea7fc700 1 -- 172.21.15.33:0/3308485046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc0095f0 con 0x7fbcec1009a0 2024-05-08T21:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbceb7fe700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec101390 0x7fbcec19e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcea7fc700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcec1009a0 0x7fbcec19ba50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbce000e9a0 tx=0x7fbce000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcd3fff700 1 -- 172.21.15.33:0/3308485046 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0009790 con 0x7fbcec1009a0 2024-05-08T21:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcd3fff700 1 -- 172.21.15.33:0/3308485046 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbce0010430 con 0x7fbcec1009a0 2024-05-08T21:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcf16e2700 1 -- 172.21.15.33:0/3308485046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec19edf0 con 0x7fbcec1009a0 2024-05-08T21:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcd3fff700 1 -- 172.21.15.33:0/3308485046 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce000cea0 con 0x7fbcec1009a0 2024-05-08T21:25:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.342+0000 7fbcf16e2700 1 -- 172.21.15.33:0/3308485046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec19f270 con 0x7fbcec1009a0 2024-05-08T21:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.345+0000 7fbcd3fff700 1 -- 172.21.15.33:0/3308485046 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbce00105a0 con 0x7fbcec1009a0 2024-05-08T21:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.346+0000 7fbcd3fff700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbcd40a1120 0x7fbcd40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.346+0000 7fbcd3fff700 1 -- 172.21.15.33:0/3308485046 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbce0014070 con 0x7fbcec1009a0 2024-05-08T21:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.346+0000 7fbceaffd700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbcd40a1120 0x7fbcd40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbceaffd700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbcd40a1120 0x7fbcd40a35e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc006010 tx=0x7fbcdc00b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbcfd89cb80 1 Processor -- start 2024-05-08T21:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbcfd89cb80 1 -- start start 2024-05-08T21:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbcfd89cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 0x55b236f03960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbcfd89cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f04320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbcfd89cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f04ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbcfd89cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b236e5a150 con 0x55b236f03ea0 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbceaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 0x55b236f03960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbcfd89cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b236e57b10 con 0x55b236f03540 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbceaffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 0x55b236f03960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56110/0 (socket says 172.21.15.33:56110) 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbceaffd700 1 -- 172.21.15.33:0/2762773701 learned_addr learned my addr 172.21.15.33:0/2762773701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/2762773701 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b236e53400 con 0x55b236f04860 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.347+0000 7fbcea7fc700 1 --2- 172.21.15.33:0/2762773701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f04320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.348+0000 7fbceb7fe700 1 --2- 172.21.15.33:0/2762773701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f04ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.348+0000 7fbceaffd700 1 -- 172.21.15.33:0/2762773701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 msgr2=0x55b236f04ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.348+0000 7fbceaffd700 1 --2- 172.21.15.33:0/2762773701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f04ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.348+0000 7fbceaffd700 1 -- 172.21.15.33:0/2762773701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 msgr2=0x55b236f04320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.348+0000 7fbceaffd700 1 --2- 172.21.15.33:0/2762773701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f04320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.348+0000 7fbceaffd700 1 -- 172.21.15.33:0/2762773701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b236cd43b0 con 0x55b236f03540 2024-05-08T21:25:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.348+0000 7fbcea7fc700 1 --2- 172.21.15.33:0/2762773701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f04320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.348+0000 7fbceaffd700 1 --2- 172.21.15.33:0/2762773701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 0x55b236f03960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc005470 tx=0x7fbcdc020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a8e6f72903638ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.348+0000 7fbcd17fa700 1 -- 172.21.15.33:0/2762773701 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc003d10 con 0x55b236f03540 2024-05-08T21:25:35.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.348+0000 7fbcd17fa700 1 -- 172.21.15.33:0/2762773701 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcdc01e040 con 0x55b236f03540 2024-05-08T21:25:35.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.348+0000 7fbcd17fa700 1 -- 172.21.15.33:0/2762773701 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0048e0 con 0x55b236f03540 2024-05-08T21:25:35.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.350+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/2762773701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 msgr2=0x55b236f03960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.350+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/2762773701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 0x55b236f03960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc005470 tx=0x7fbcdc020000 comp rx=0 tx=0).stop 2024-05-08T21:25:35.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.350+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/2762773701 shutdown_connections 2024-05-08T21:25:35.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.350+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/2762773701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f04ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.350+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/2762773701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 0x55b236f03960 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.350+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/2762773701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f04320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.350+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/2762773701 >> 172.21.15.33:0/2762773701 conn(0x55b236e61210 msgr2=0x55b236efdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.350+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/2762773701 shutdown_connections 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.350+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/2762773701 wait complete. 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.350+0000 7fbcfd89cb80 1 -- start start 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcfd89cb80 1 Processor -- start 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcfd89cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03540 0x55b236f0cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcfd89cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03ea0 0x55b236f0d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcfd89cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f14320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcfd89cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b236def560 con 0x55b236f03540 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcfd89cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b236def200 con 0x55b236f03ea0 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcfd89cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b236cd66a0 con 0x55b236f04860 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcfd89cb80 10 client.0 _ll_register_callbacks cb 0x55b236df8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbceaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03540 0x55b236f0cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcea7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03ea0 0x55b236f0d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbceb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f14320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcea7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03ea0 0x55b236f0d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56122/0 (socket says 172.21.15.33:56122) 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcea7fc700 1 -- 172.21.15.33:0/3164635216 learned_addr learned my addr 172.21.15.33:0/3164635216 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcea7fc700 1 -- 172.21.15.33:0/3164635216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 msgr2=0x55b236f14320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcea7fc700 1 --2- 172.21.15.33:0/3164635216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f14320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcea7fc700 1 -- 172.21.15.33:0/3164635216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03540 msgr2=0x55b236f0cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcea7fc700 1 --2- 172.21.15.33:0/3164635216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03540 0x55b236f0cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.351+0000 7fbcea7fc700 1 -- 172.21.15.33:0/3164635216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec19f270 con 0x55b236f03ea0 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcea7fc700 1 --2- 172.21.15.33:0/3164635216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03ea0 0x55b236f0d2e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbce00ca5b0 tx=0x7fbce009a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d130f85687595a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcceff5700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcceff5700 1 -- 172.21.15.33:0/3164635216 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce00a20f0 con 0x55b236f03ea0 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcceff5700 1 -- 172.21.15.33:0/3164635216 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbce0099f00 con 0x55b236f03ea0 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/3164635216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b236dac270 con 0x55b236f03ea0 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcceff5700 1 -- 172.21.15.33:0/3164635216 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbce009dd30 con 0x55b236f03ea0 2024-05-08T21:25:35.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcfd89cb80 10 client.27191 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/3164635216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fbcec108b70 con 0x55b236f03ea0 2024-05-08T21:25:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcceff5700 1 -- 172.21.15.33:0/3164635216 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b236dac270 con 0x55b236f03ea0 2024-05-08T21:25:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcceff5700 1 -- 172.21.15.33:0/3164635216 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbce0095cf0 con 0x55b236f03ea0 2024-05-08T21:25:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcfd89cb80 10 client.27191 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcfd89cb80 10 client.27191 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:25:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcfd89cb80 20 client.27191 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/3164635216 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55b236f18ae0 0x55b236f1af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcfd89cb80 4 client.27191 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.352+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/3164635216 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b236ed2810 con 0x55b236f18ae0 2024-05-08T21:25:35.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.353+0000 7fbceaffd700 1 --2- 172.21.15.33:0/3164635216 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55b236f18ae0 0x55b236f1af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.353+0000 7fbcceff5700 10 client.27191 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:25:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.353+0000 7fbceaffd700 1 --2- 172.21.15.33:0/3164635216 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55b236f18ae0 0x55b236f1af00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=70ccff6801f5c803 server_cookie=985cec4a97c8d4dd in_seq=0 out_seq=0 2024-05-08T21:25:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.354+0000 7fbcceff5700 1 -- 172.21.15.33:0/3164635216 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b236ed2810 con 0x55b236f18ae0 2024-05-08T21:25:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.354+0000 7fbcceff5700 10 client.27191 handle_command_reply: tid=0 2024-05-08T21:25:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.354+0000 7fbcfd89cb80 1 client.27191 shutdown 2024-05-08T21:25:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.354+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/3164635216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03ea0 msgr2=0x55b236f0d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.354+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/3164635216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03ea0 0x55b236f0d2e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbce00ca5b0 tx=0x7fbce009a440 comp rx=0 tx=0).stop 2024-05-08T21:25:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.357+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/3164635216 shutdown_connections 2024-05-08T21:25:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.357+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/3164635216 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55b236f18ae0 0x55b236f1af00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.357+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/3164635216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f14320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.357+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/3164635216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03540 0x55b236f0cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.357+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/3164635216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03ea0 0x55b236f0d2e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.357+0000 7fbcceff5700 0 client.27191 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:25:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.357+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/3164635216 >> 172.21.15.33:0/3164635216 conn(0x55b236e61210 msgr2=0x55b236efd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.357+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/3164635216 shutdown_connections 2024-05-08T21:25:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.357+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/3164635216 wait complete. 2024-05-08T21:25:35.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.357+0000 7fbcfd89cb80 20 client.27191 trim_cache size 0 max 16384 2024-05-08T21:25:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbcfd89cb80 1 Processor -- start 2024-05-08T21:25:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbcfd89cb80 1 -- start start 2024-05-08T21:25:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbcfd89cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 0x55b236f1aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbcfd89cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f03920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbcfd89cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f149d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbcfd89cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b236f0c940 con 0x55b236f03ea0 2024-05-08T21:25:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbcfd89cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b236efadc0 con 0x55b236f03540 2024-05-08T21:25:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbcfd89cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b236f18910 con 0x55b236f04860 2024-05-08T21:25:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbcea7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f03920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbceb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f149d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbceb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f149d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49532/0 (socket says 172.21.15.33:49532) 2024-05-08T21:25:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbcea7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f03920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44784/0 (socket says 172.21.15.33:44784) 2024-05-08T21:25:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbceb7fe700 1 -- 172.21.15.33:0/1511334626 learned_addr learned my addr 172.21.15.33:0/1511334626 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.374+0000 7fbceaffd700 1 --2- 172.21.15.33:0/1511334626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 0x55b236f1aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.375+0000 7fbceaffd700 1 -- 172.21.15.33:0/1511334626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 msgr2=0x55b236f149d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.375+0000 7fbceaffd700 1 --2- 172.21.15.33:0/1511334626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f149d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.375+0000 7fbceaffd700 1 -- 172.21.15.33:0/1511334626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 msgr2=0x55b236f03920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.375+0000 7fbceaffd700 1 --2- 172.21.15.33:0/1511334626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f03920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.375+0000 7fbceaffd700 1 -- 172.21.15.33:0/1511334626 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b236c80b60 con 0x55b236f03540 2024-05-08T21:25:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.375+0000 7fbcea7fc700 1 --2- 172.21.15.33:0/1511334626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f03920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.375+0000 7fbceb7fe700 1 --2- 172.21.15.33:0/1511334626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.375+0000 7fbceaffd700 1 --2- 172.21.15.33:0/1511334626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 0x55b236f1aed0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc000c00 tx=0x7fbcdc01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae383d5123a4ec4f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.375+0000 7fbcd1ffb700 1 -- 172.21.15.33:0/1511334626 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc029040 con 0x55b236f03540 2024-05-08T21:25:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.375+0000 7fbcd1ffb700 1 -- 172.21.15.33:0/1511334626 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcdc00bb40 con 0x55b236f03540 2024-05-08T21:25:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.375+0000 7fbcd1ffb700 1 -- 172.21.15.33:0/1511334626 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc03d630 con 0x55b236f03540 2024-05-08T21:25:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.376+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1511334626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 msgr2=0x55b236f1aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.376+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/1511334626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 0x55b236f1aed0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc000c00 tx=0x7fbcdc01a040 comp rx=0 tx=0).stop 2024-05-08T21:25:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.376+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1511334626 shutdown_connections 2024-05-08T21:25:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.376+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/1511334626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f04860 0x55b236f149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.376+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/1511334626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f03540 0x55b236f1aed0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.376+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/1511334626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f03920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.376+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1511334626 >> 172.21.15.33:0/1511334626 conn(0x55b236e61210 msgr2=0x55b236f04c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.376+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1511334626 shutdown_connections 2024-05-08T21:25:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.376+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1511334626 wait complete. 2024-05-08T21:25:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.376+0000 7fbcfd89cb80 1 -- start start 2024-05-08T21:25:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbcfd89cb80 1 Processor -- start 2024-05-08T21:25:35.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbcfd89cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f03540 0x55b236f122d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbcfd89cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f6afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbcfd89cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f04860 0x55b236f6d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbcfd89cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b236f0a540 con 0x55b236f03ea0 2024-05-08T21:25:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbcfd89cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b236f6da40 con 0x55b236f04860 2024-05-08T21:25:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbcfd89cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b236f6db80 con 0x55b236f03540 2024-05-08T21:25:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbcfd89cb80 10 client.0 _ll_register_callbacks cb 0x55b236df8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbcea7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f6afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbcea7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f6afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44800/0 (socket says 172.21.15.33:44800) 2024-05-08T21:25:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbcea7fc700 1 -- 172.21.15.33:0/1099574767 learned_addr learned my addr 172.21.15.33:0/1099574767 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbceaffd700 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f03540 0x55b236f122d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbceb7fe700 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f04860 0x55b236f6d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbceaffd700 1 -- 172.21.15.33:0/1099574767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f04860 msgr2=0x55b236f6d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbceaffd700 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f04860 0x55b236f6d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbceaffd700 1 -- 172.21.15.33:0/1099574767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 msgr2=0x55b236f6afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbceaffd700 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f6afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbceaffd700 1 -- 172.21.15.33:0/1099574767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec108b70 con 0x55b236f03540 2024-05-08T21:25:35.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbceb7fe700 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f04860 0x55b236f6d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:25:35.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.377+0000 7fbcea7fc700 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f6afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbceaffd700 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f03540 0x55b236f122d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55b236dac3b0 tx=0x7fbcdc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a24dee38cb707981 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcceff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcceff5700 1 -- 172.21.15.33:0/1099574767 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc029070 con 0x55b236f03540 2024-05-08T21:25:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcceff5700 1 -- 172.21.15.33:0/1099574767 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbcdc00b8b0 con 0x55b236f03540 2024-05-08T21:25:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1099574767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcdc01e040 con 0x55b236f03540 2024-05-08T21:25:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcceff5700 1 -- 172.21.15.33:0/1099574767 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcdc03d3b0 con 0x55b236f03540 2024-05-08T21:25:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcfd89cb80 10 client.27202 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1099574767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fbce4003a80 con 0x55b236f03540 2024-05-08T21:25:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcceff5700 1 -- 172.21.15.33:0/1099574767 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcdc01e040 con 0x55b236f03540 2024-05-08T21:25:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcceff5700 1 -- 172.21.15.33:0/1099574767 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbcdc046070 con 0x55b236f03540 2024-05-08T21:25:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcfd89cb80 10 client.27202 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcfd89cb80 10 client.27202 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:25:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcfd89cb80 20 client.27202 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:35.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55b236f6f0a0 0x55b236f714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcfd89cb80 4 client.27202 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:25:35.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.378+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1099574767 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b236e4bec0 con 0x55b236f6f0a0 2024-05-08T21:25:35.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.379+0000 7fbcea7fc700 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55b236f6f0a0 0x55b236f714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.379+0000 7fbcea7fc700 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55b236f6f0a0 0x55b236f714c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2231591322d91edd server_cookie=90364d23792dd63b in_seq=0 out_seq=0 2024-05-08T21:25:35.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.379+0000 7fbcceff5700 10 client.27202 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.379+0000 7fbcceff5700 1 -- 172.21.15.33:0/1099574767 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b236e4bec0 con 0x55b236f6f0a0 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.379+0000 7fbcceff5700 10 client.27202 handle_command_reply: tid=0 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.379+0000 7fbcfd89cb80 1 client.27202 shutdown 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.379+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1099574767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f03540 msgr2=0x55b236f122d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.379+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f03540 0x55b236f122d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55b236dac3b0 tx=0x7fbcdc003c10 comp rx=0 tx=0).stop 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1099574767 shutdown_connections 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b236f04860 0x55b236f6d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x55b236f6f0a0 0x55b236f714c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b236f03540 0x55b236f122d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcfd89cb80 1 --2- 172.21.15.33:0/1099574767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b236f03ea0 0x55b236f6afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcceff5700 0 client.27202 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1099574767 >> 172.21.15.33:0/1099574767 conn(0x55b236e61210 msgr2=0x55b236f04c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1099574767 shutdown_connections 2024-05-08T21:25:35.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcfd89cb80 1 -- 172.21.15.33:0/1099574767 wait complete. 2024-05-08T21:25:35.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcfd89cb80 20 client.27202 trim_cache size 0 max 16384 2024-05-08T21:25:35.395 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:25:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcf16e2700 1 -- 172.21.15.33:0/3308485046 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbcd40a1120 msgr2=0x7fbcd40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcf16e2700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbcd40a1120 0x7fbcd40a35e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc006010 tx=0x7fbcdc00b5d0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcf16e2700 1 -- 172.21.15.33:0/3308485046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcec1009a0 msgr2=0x7fbcec19ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcf16e2700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcec1009a0 0x7fbcec19ba50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbce000e9a0 tx=0x7fbce000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcf16e2700 1 -- 172.21.15.33:0/3308485046 shutdown_connections 2024-05-08T21:25:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcf16e2700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec101390 0x7fbcec19e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcf16e2700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbcd40a1120 0x7fbcd40a35e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcf16e2700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec0fffe0 0x7fbcec0ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcf16e2700 1 --2- 172.21.15.33:0/3308485046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcec1009a0 0x7fbcec19ba50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.380+0000 7fbcf16e2700 1 -- 172.21.15.33:0/3308485046 >> 172.21.15.33:0/3308485046 conn(0x7fbcec0faad0 msgr2=0x7fbcec1043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.381+0000 7fbcf16e2700 1 -- 172.21.15.33:0/3308485046 shutdown_connections 2024-05-08T21:25:35.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.381+0000 7fbcf16e2700 1 -- 172.21.15.33:0/3308485046 wait complete. 2024-05-08T21:25:35.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:25:35.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.567+0000 7f60430a2700 1 -- 172.21.15.33:0/3414380805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c104270 msgr2=0x7f603c106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.567+0000 7f60430a2700 1 --2- 172.21.15.33:0/3414380805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c104270 0x7f603c106660 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6028009990 tx=0x7f6028009ca0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.567+0000 7f60430a2700 1 -- 172.21.15.33:0/3414380805 shutdown_connections 2024-05-08T21:25:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.567+0000 7f60430a2700 1 --2- 172.21.15.33:0/3414380805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603c106ba0 0x7f603c108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.567+0000 7f60430a2700 1 --2- 172.21.15.33:0/3414380805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101940 0x7f603c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.567+0000 7f60430a2700 1 --2- 172.21.15.33:0/3414380805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c104270 0x7f603c106660 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.567+0000 7f60430a2700 1 -- 172.21.15.33:0/3414380805 >> 172.21.15.33:0/3414380805 conn(0x7f603c0fb290 msgr2=0x7f603c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.567+0000 7f60430a2700 1 -- 172.21.15.33:0/3414380805 shutdown_connections 2024-05-08T21:25:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.567+0000 7f60430a2700 1 -- 172.21.15.33:0/3414380805 wait complete. 2024-05-08T21:25:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f60430a2700 1 Processor -- start 2024-05-08T21:25:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f60430a2700 1 -- start start 2024-05-08T21:25:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f60430a2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101940 0x7f603c101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f60430a2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c104270 0x7f603c0ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f60430a2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603c106ba0 0x7f603c0ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f60430a2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c1005d0 con 0x7f603c101940 2024-05-08T21:25:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f60430a2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c100740 con 0x7f603c106ba0 2024-05-08T21:25:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f60430a2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c1a0db0 con 0x7f603c104270 2024-05-08T21:25:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f6040e3e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101940 0x7f603c101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f6040e3e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101940 0x7f603c101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44814/0 (socket says 172.21.15.33:44814) 2024-05-08T21:25:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f604163f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603c106ba0 0x7f603c0ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f6040e3e700 1 -- 172.21.15.33:0/2033459512 learned_addr learned my addr 172.21.15.33:0/2033459512 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.568+0000 7f603bfff700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c104270 0x7f603c0ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.569+0000 7f6040e3e700 1 -- 172.21.15.33:0/2033459512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c104270 msgr2=0x7f603c0ff8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.569+0000 7f6040e3e700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c104270 0x7f603c0ff8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.569+0000 7f6040e3e700 1 -- 172.21.15.33:0/2033459512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603c106ba0 msgr2=0x7f603c0ffe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.569+0000 7f6040e3e700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603c106ba0 0x7f603c0ffe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.569+0000 7f6040e3e700 1 -- 172.21.15.33:0/2033459512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60280095f0 con 0x7f603c101940 2024-05-08T21:25:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.569+0000 7f604163f700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603c106ba0 0x7f603c0ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:25:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.569+0000 7f6040e3e700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101940 0x7f603c101260 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f603000e9a0 tx=0x7f603000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.569+0000 7f6039ffb700 1 -- 172.21.15.33:0/2033459512 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030009790 con 0x7f603c101940 2024-05-08T21:25:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.569+0000 7f60430a2700 1 -- 172.21.15.33:0/2033459512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603c1a0fe0 con 0x7f603c101940 2024-05-08T21:25:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.569+0000 7f6039ffb700 1 -- 172.21.15.33:0/2033459512 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6030010430 con 0x7f603c101940 2024-05-08T21:25:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.569+0000 7f6039ffb700 1 -- 172.21.15.33:0/2033459512 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030018680 con 0x7f603c101940 2024-05-08T21:25:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.570+0000 7f60430a2700 1 -- 172.21.15.33:0/2033459512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c1a14b0 con 0x7f603c101940 2024-05-08T21:25:35.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.573+0000 7f6039ffb700 1 -- 172.21.15.33:0/2033459512 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f60300105a0 con 0x7f603c101940 2024-05-08T21:25:35.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.574+0000 7f6039ffb700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f602c0a1120 0x7f602c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.574+0000 7f6039ffb700 1 -- 172.21.15.33:0/2033459512 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f6030014070 con 0x7f603c101940 2024-05-08T21:25:35.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.574+0000 7f603bfff700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f602c0a1120 0x7f602c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.574+0000 7f604f25cb80 1 Processor -- start 2024-05-08T21:25:35.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.574+0000 7f603bfff700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f602c0a1120 0x7f602c0a35e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6028006010 tx=0x7f602800b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604f25cb80 1 -- start start 2024-05-08T21:25:35.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604f25cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec0540 0x55f057ec0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604f25cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0ea0 0x55f057ec1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604f25cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec1860 0x55f057ec1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604f25cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f057e17150 con 0x55f057ec0ea0 2024-05-08T21:25:35.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604f25cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f057e14b10 con 0x55f057ec1860 2024-05-08T21:25:35.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604f25cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f057e10400 con 0x55f057ec0540 2024-05-08T21:25:35.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f603bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0ea0 0x55f057ec1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f6040e3e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec0540 0x55f057ec0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604163f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec1860 0x55f057ec1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f6040e3e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec0540 0x55f057ec0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49564/0 (socket says 172.21.15.33:49564) 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f603bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0ea0 0x55f057ec1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44826/0 (socket says 172.21.15.33:44826) 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f6040e3e700 1 -- 172.21.15.33:0/2314353303 learned_addr learned my addr 172.21.15.33:0/2314353303 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604163f700 1 -- 172.21.15.33:0/2314353303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec0540 msgr2=0x55f057ec0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604163f700 1 --2- 172.21.15.33:0/2314353303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec0540 0x55f057ec0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604163f700 1 -- 172.21.15.33:0/2314353303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0ea0 msgr2=0x55f057ec1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604163f700 1 --2- 172.21.15.33:0/2314353303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0ea0 0x55f057ec1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.575+0000 7f604163f700 1 -- 172.21.15.33:0/2314353303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f057c913b0 con 0x55f057ec1860 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.576+0000 7f6040e3e700 1 --2- 172.21.15.33:0/2314353303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec0540 0x55f057ec0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.576+0000 7f604163f700 1 --2- 172.21.15.33:0/2314353303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec1860 0x55f057ec1ce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f60340093d0 tx=0x7f6034009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebfd0409b037cb72 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.576+0000 7f6026ffd700 1 -- 172.21.15.33:0/2314353303 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034019070 con 0x55f057ec1860 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.576+0000 7f6026ffd700 1 -- 172.21.15.33:0/2314353303 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f603400f040 con 0x55f057ec1860 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.576+0000 7f6026ffd700 1 -- 172.21.15.33:0/2314353303 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603400e030 con 0x55f057ec1860 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.577+0000 7f604f25cb80 1 -- 172.21.15.33:0/2314353303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec1860 msgr2=0x55f057ec1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.577+0000 7f604f25cb80 1 --2- 172.21.15.33:0/2314353303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec1860 0x55f057ec1ce0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f60340093d0 tx=0x7f6034009920 comp rx=0 tx=0).stop 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.577+0000 7f604f25cb80 1 -- 172.21.15.33:0/2314353303 shutdown_connections 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.577+0000 7f604f25cb80 1 --2- 172.21.15.33:0/2314353303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec1860 0x55f057ec1ce0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.577+0000 7f604f25cb80 1 --2- 172.21.15.33:0/2314353303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec0540 0x55f057ec0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.577+0000 7f604f25cb80 1 --2- 172.21.15.33:0/2314353303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0ea0 0x55f057ec1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.577+0000 7f604f25cb80 1 -- 172.21.15.33:0/2314353303 >> 172.21.15.33:0/2314353303 conn(0x55f057e1e210 msgr2=0x55f057ebade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.577+0000 7f604f25cb80 1 -- 172.21.15.33:0/2314353303 shutdown_connections 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.577+0000 7f604f25cb80 1 -- 172.21.15.33:0/2314353303 wait complete. 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.578+0000 7f604f25cb80 1 -- start start 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.578+0000 7f604f25cb80 1 Processor -- start 2024-05-08T21:25:35.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.578+0000 7f604f25cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 0x55f057ed2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.578+0000 7f604f25cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec0ea0 0x55f057ed2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.578+0000 7f604f25cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec1860 0x55f057ed9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.578+0000 7f604f25cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f057dac560 con 0x55f057ec1860 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.578+0000 7f604f25cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f057dac200 con 0x55f057ec0540 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.578+0000 7f604f25cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f057c936a0 con 0x55f057ec0ea0 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.578+0000 7f604f25cb80 10 client.0 _ll_register_callbacks cb 0x55f057db5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.579+0000 7f6040e3e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 0x55f057ed2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.579+0000 7f6040e3e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 0x55f057ed2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56178/0 (socket says 172.21.15.33:56178) 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.579+0000 7f6040e3e700 1 -- 172.21.15.33:0/4122392069 learned_addr learned my addr 172.21.15.33:0/4122392069 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.579+0000 7f6040e3e700 1 -- 172.21.15.33:0/4122392069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec0ea0 msgr2=0x55f057ed2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.579+0000 7f6040e3e700 1 --2- 172.21.15.33:0/4122392069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec0ea0 0x55f057ed2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.579+0000 7f6040e3e700 1 -- 172.21.15.33:0/4122392069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec1860 msgr2=0x55f057ed9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.579+0000 7f6040e3e700 1 --2- 172.21.15.33:0/4122392069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec1860 0x55f057ed9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.579+0000 7f6040e3e700 1 -- 172.21.15.33:0/4122392069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c1a14b0 con 0x55f057ec0540 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.579+0000 7f6040e3e700 1 --2- 172.21.15.33:0/4122392069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 0x55f057ed2620 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6030009230 tx=0x7f603009a4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95eb266aaad642c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.579+0000 7f601bfff700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.579+0000 7f601bfff700 1 -- 172.21.15.33:0/4122392069 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300b1070 con 0x55f057ec0540 2024-05-08T21:25:35.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.579+0000 7f601bfff700 1 -- 172.21.15.33:0/4122392069 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f603009ddd0 con 0x55f057ec0540 2024-05-08T21:25:35.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.580+0000 7f604f25cb80 1 -- 172.21.15.33:0/4122392069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f057d69270 con 0x55f057ec0540 2024-05-08T21:25:35.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.580+0000 7f601bfff700 1 -- 172.21.15.33:0/4122392069 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60300a2420 con 0x55f057ec0540 2024-05-08T21:25:35.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.581+0000 7f604f25cb80 10 client.27209 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:35.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.581+0000 7f604f25cb80 1 -- 172.21.15.33:0/4122392069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f6034009050 con 0x55f057ec0540 2024-05-08T21:25:35.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.581+0000 7f601bfff700 1 -- 172.21.15.33:0/4122392069 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f057d69270 con 0x55f057ec0540 2024-05-08T21:25:35.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.581+0000 7f601bfff700 1 -- 172.21.15.33:0/4122392069 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6030094bb0 con 0x55f057ec0540 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.581+0000 7f604f25cb80 10 client.27209 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.581+0000 7f604f25cb80 10 client.27209 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.581+0000 7f604f25cb80 20 client.27209 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.581+0000 7f604f25cb80 1 --2- 172.21.15.33:0/4122392069 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f057ecdd90 0x55f057ed01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.581+0000 7f604f25cb80 4 client.27209 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.581+0000 7f604f25cb80 1 -- 172.21.15.33:0/4122392069 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f057e8f810 con 0x55f057ecdd90 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.581+0000 7f603bfff700 1 --2- 172.21.15.33:0/4122392069 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f057ecdd90 0x55f057ed01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.582+0000 7f603bfff700 1 --2- 172.21.15.33:0/4122392069 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f057ecdd90 0x55f057ed01b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9cc52b36361dcd9b server_cookie=90dc82bab6a5998b in_seq=0 out_seq=0 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.582+0000 7f601bfff700 10 client.27209 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.582+0000 7f601bfff700 1 -- 172.21.15.33:0/4122392069 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f057e8f810 con 0x55f057ecdd90 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.582+0000 7f601bfff700 10 client.27209 handle_command_reply: tid=0 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.582+0000 7f604f25cb80 1 client.27209 shutdown 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.582+0000 7f604f25cb80 1 -- 172.21.15.33:0/4122392069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 msgr2=0x55f057ed2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.583+0000 7f604f25cb80 1 --2- 172.21.15.33:0/4122392069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 0x55f057ed2620 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6030009230 tx=0x7f603009a4e0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.583+0000 7f604f25cb80 1 -- 172.21.15.33:0/4122392069 shutdown_connections 2024-05-08T21:25:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.583+0000 7f604f25cb80 1 --2- 172.21.15.33:0/4122392069 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f057ecdd90 0x55f057ed01b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.583+0000 7f604f25cb80 1 --2- 172.21.15.33:0/4122392069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec1860 0x55f057ed9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.583+0000 7f601bfff700 0 client.27209 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:25:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.583+0000 7f604f25cb80 1 --2- 172.21.15.33:0/4122392069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 0x55f057ed2620 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.583+0000 7f604f25cb80 1 --2- 172.21.15.33:0/4122392069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec0ea0 0x55f057ed2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.583+0000 7f604f25cb80 1 -- 172.21.15.33:0/4122392069 >> 172.21.15.33:0/4122392069 conn(0x55f057e1e210 msgr2=0x55f057eba5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.583+0000 7f604f25cb80 1 -- 172.21.15.33:0/4122392069 shutdown_connections 2024-05-08T21:25:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.583+0000 7f604f25cb80 1 -- 172.21.15.33:0/4122392069 wait complete. 2024-05-08T21:25:35.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.583+0000 7f604f25cb80 20 client.27209 trim_cache size 0 max 16384 2024-05-08T21:25:35.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.598+0000 7f604f25cb80 1 Processor -- start 2024-05-08T21:25:35.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.598+0000 7f604f25cb80 1 -- start start 2024-05-08T21:25:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f604f25cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 0x55f057ec9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f604f25cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0ea0 0x55f057ed9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f604f25cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec1860 0x55f057ec1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f604f25cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f057eb7dc0 con 0x55f057ec0ea0 2024-05-08T21:25:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f604f25cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f057eb5800 con 0x55f057ec0540 2024-05-08T21:25:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f604f25cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f057ed3810 con 0x55f057ec1860 2024-05-08T21:25:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f6040e3e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 0x55f057ec9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f6040e3e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 0x55f057ec9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56186/0 (socket says 172.21.15.33:56186) 2024-05-08T21:25:35.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f6040e3e700 1 -- 172.21.15.33:0/3100388244 learned_addr learned my addr 172.21.15.33:0/3100388244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f603bfff700 1 --2- 172.21.15.33:0/3100388244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0ea0 0x55f057ed9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f6040e3e700 1 -- 172.21.15.33:0/3100388244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec1860 msgr2=0x55f057ec1c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:25:35.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f6040e3e700 1 --2- 172.21.15.33:0/3100388244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec1860 0x55f057ec1c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f6040e3e700 1 -- 172.21.15.33:0/3100388244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0ea0 msgr2=0x55f057ed9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f6040e3e700 1 --2- 172.21.15.33:0/3100388244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0ea0 0x55f057ed9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.599+0000 7f6040e3e700 1 -- 172.21.15.33:0/3100388244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f057c3db60 con 0x55f057ec0540 2024-05-08T21:25:35.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.600+0000 7f6040e3e700 1 --2- 172.21.15.33:0/3100388244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 0x55f057ec9bb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f60300964a0 tx=0x7f6030099ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ddb6a4cc8d377b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.600+0000 7f60277fe700 1 -- 172.21.15.33:0/3100388244 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300b1070 con 0x55f057ec0540 2024-05-08T21:25:35.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.600+0000 7f60277fe700 1 -- 172.21.15.33:0/3100388244 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6030020be0 con 0x55f057ec0540 2024-05-08T21:25:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.600+0000 7f60277fe700 1 -- 172.21.15.33:0/3100388244 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300b2740 con 0x55f057ec0540 2024-05-08T21:25:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.601+0000 7f604f25cb80 1 -- 172.21.15.33:0/3100388244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 msgr2=0x55f057ec9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.601+0000 7f604f25cb80 1 --2- 172.21.15.33:0/3100388244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 0x55f057ec9bb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f60300964a0 tx=0x7f6030099ea0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.601+0000 7f604f25cb80 1 -- 172.21.15.33:0/3100388244 shutdown_connections 2024-05-08T21:25:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.601+0000 7f604f25cb80 1 --2- 172.21.15.33:0/3100388244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec1860 0x55f057ec1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.601+0000 7f604f25cb80 1 --2- 172.21.15.33:0/3100388244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0540 0x55f057ec9bb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.601+0000 7f604f25cb80 1 --2- 172.21.15.33:0/3100388244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0ea0 0x55f057ed9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.601+0000 7f604f25cb80 1 -- 172.21.15.33:0/3100388244 >> 172.21.15.33:0/3100388244 conn(0x55f057e1e210 msgr2=0x55f057eb7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.601+0000 7f604f25cb80 1 -- 172.21.15.33:0/3100388244 shutdown_connections 2024-05-08T21:25:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.601+0000 7f604f25cb80 1 -- 172.21.15.33:0/3100388244 wait complete. 2024-05-08T21:25:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.601+0000 7f604f25cb80 1 -- start start 2024-05-08T21:25:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.602+0000 7f604f25cb80 1 Processor -- start 2024-05-08T21:25:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.602+0000 7f604f25cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0540 0x55f057ed3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.602+0000 7f604f25cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0ea0 0x55f057ed4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.602+0000 7f604f25cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec1860 0x55f057ed9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.602+0000 7f604f25cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f057ed9760 con 0x55f057ec0540 2024-05-08T21:25:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.602+0000 7f604f25cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f057ed98d0 con 0x55f057ec0ea0 2024-05-08T21:25:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.602+0000 7f604f25cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f057ed9a90 con 0x55f057ec1860 2024-05-08T21:25:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.602+0000 7f604f25cb80 10 client.0 _ll_register_callbacks cb 0x55f057db5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:35.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.603+0000 7f604163f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec1860 0x55f057ed9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.603+0000 7f604163f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec1860 0x55f057ed9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49582/0 (socket says 172.21.15.33:49582) 2024-05-08T21:25:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.603+0000 7f604163f700 1 -- 172.21.15.33:0/1129796837 learned_addr learned my addr 172.21.15.33:0/1129796837 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.603+0000 7f604163f700 1 -- 172.21.15.33:0/1129796837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0ea0 msgr2=0x55f057ed4290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:25:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.603+0000 7f604163f700 1 --2- 172.21.15.33:0/1129796837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0ea0 0x55f057ed4290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.603+0000 7f604163f700 1 -- 172.21.15.33:0/1129796837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0540 msgr2=0x55f057ed3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.603+0000 7f604163f700 1 --2- 172.21.15.33:0/1129796837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0540 0x55f057ed3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.603+0000 7f604163f700 1 -- 172.21.15.33:0/1129796837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6034009050 con 0x55f057ec1860 2024-05-08T21:25:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.603+0000 7f604163f700 1 --2- 172.21.15.33:0/1129796837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec1860 0x55f057ed9220 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6034007ac0 tx=0x7f60340148d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5cdd1ac1fcd0b84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.603+0000 7f601bfff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.603+0000 7f601bfff700 1 -- 172.21.15.33:0/1129796837 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603400e070 con 0x55f057ec1860 2024-05-08T21:25:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.604+0000 7f604f25cb80 1 -- 172.21.15.33:0/1129796837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f603400f040 con 0x55f057ec1860 2024-05-08T21:25:35.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.604+0000 7f601bfff700 1 -- 172.21.15.33:0/1129796837 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f6034014d80 con 0x55f057ec1860 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.604+0000 7f601bfff700 1 -- 172.21.15.33:0/1129796837 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6034019040 con 0x55f057ec1860 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.604+0000 7f604f25cb80 10 client.27214 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.604+0000 7f604f25cb80 1 -- 172.21.15.33:0/1129796837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f057dd6560 con 0x55f057ec1860 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.604+0000 7f601bfff700 1 -- 172.21.15.33:0/1129796837 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f603400f040 con 0x55f057ec1860 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.604+0000 7f601bfff700 1 -- 172.21.15.33:0/1129796837 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60340043c0 con 0x55f057ec1860 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.605+0000 7f604f25cb80 10 client.27214 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.605+0000 7f604f25cb80 10 client.27214 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.605+0000 7f604f25cb80 20 client.27214 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.605+0000 7f604f25cb80 1 --2- 172.21.15.33:0/1129796837 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f057eda7c0 0x55f057edcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.605+0000 7f604f25cb80 4 client.27214 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.605+0000 7f604f25cb80 1 -- 172.21.15.33:0/1129796837 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f057e08ec0 con 0x55f057eda7c0 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.605+0000 7f6040e3e700 1 --2- 172.21.15.33:0/1129796837 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f057eda7c0 0x55f057edcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.605+0000 7f601bfff700 10 client.27214 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:25:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.605+0000 7f6040e3e700 1 --2- 172.21.15.33:0/1129796837 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f057eda7c0 0x55f057edcc20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83b77982f6397b85 server_cookie=5609bbfb5a03680d in_seq=0 out_seq=0 2024-05-08T21:25:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.605+0000 7f601bfff700 1 -- 172.21.15.33:0/1129796837 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f057e08ec0 con 0x55f057eda7c0 2024-05-08T21:25:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.605+0000 7f601bfff700 10 client.27214 handle_command_reply: tid=0 2024-05-08T21:25:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.605+0000 7f604f25cb80 1 client.27214 shutdown 2024-05-08T21:25:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.606+0000 7f604f25cb80 1 -- 172.21.15.33:0/1129796837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec1860 msgr2=0x55f057ed9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.606+0000 7f604f25cb80 1 --2- 172.21.15.33:0/1129796837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec1860 0x55f057ed9220 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6034007ac0 tx=0x7f60340148d0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.607+0000 7f604f25cb80 1 -- 172.21.15.33:0/1129796837 shutdown_connections 2024-05-08T21:25:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.607+0000 7f604f25cb80 1 --2- 172.21.15.33:0/1129796837 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f057eda7c0 0x55f057edcc20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.607+0000 7f604f25cb80 1 --2- 172.21.15.33:0/1129796837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f057ec1860 0x55f057ed9220 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.607+0000 7f604f25cb80 1 --2- 172.21.15.33:0/1129796837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f057ec0540 0x55f057ed3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.607+0000 7f604f25cb80 1 --2- 172.21.15.33:0/1129796837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f057ec0ea0 0x55f057ed4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.607+0000 7f601bfff700 0 client.27214 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:25:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.607+0000 7f604f25cb80 1 -- 172.21.15.33:0/1129796837 >> 172.21.15.33:0/1129796837 conn(0x55f057e1e210 msgr2=0x55f057ec1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.607+0000 7f604f25cb80 1 -- 172.21.15.33:0/1129796837 shutdown_connections 2024-05-08T21:25:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.607+0000 7f604f25cb80 1 -- 172.21.15.33:0/1129796837 wait complete. 2024-05-08T21:25:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.607+0000 7f604f25cb80 20 client.27214 trim_cache size 0 max 16384 2024-05-08T21:25:35.612 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:25:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.608+0000 7f60430a2700 1 -- 172.21.15.33:0/2033459512 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f602c0a1120 msgr2=0x7f602c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.608+0000 7f60430a2700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f602c0a1120 0x7f602c0a35e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6028006010 tx=0x7f602800b610 comp rx=0 tx=0).stop 2024-05-08T21:25:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.608+0000 7f60430a2700 1 -- 172.21.15.33:0/2033459512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101940 msgr2=0x7f603c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.608+0000 7f60430a2700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101940 0x7f603c101260 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f603000e9a0 tx=0x7f603000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.608+0000 7f60430a2700 1 -- 172.21.15.33:0/2033459512 shutdown_connections 2024-05-08T21:25:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.608+0000 7f60430a2700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f602c0a1120 0x7f602c0a35e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.608+0000 7f60430a2700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f603c106ba0 0x7f603c0ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.608+0000 7f60430a2700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f603c101940 0x7f603c101260 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.608+0000 7f60430a2700 1 --2- 172.21.15.33:0/2033459512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f603c104270 0x7f603c0ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.608+0000 7f60430a2700 1 -- 172.21.15.33:0/2033459512 >> 172.21.15.33:0/2033459512 conn(0x7f603c0fb290 msgr2=0x7f603c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.608+0000 7f60430a2700 1 -- 172.21.15.33:0/2033459512 shutdown_connections 2024-05-08T21:25:35.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.608+0000 7f60430a2700 1 -- 172.21.15.33:0/2033459512 wait complete. 2024-05-08T21:25:35.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:35 smithi033 ceph-mon[42528]: pgmap v488: 129 pgs: 129 active+clean; 227 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.4 MiB/s wr, 371 op/s 2024-05-08T21:25:35.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:35 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2158554989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:25:35.625 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:25:35.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.792+0000 7ff93946c700 1 -- 172.21.15.33:0/4128862573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9340ffd90 msgr2=0x7ff9341001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.792+0000 7ff93946c700 1 --2- 172.21.15.33:0/4128862573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9340ffd90 0x7ff9341001b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff924009990 tx=0x7ff924009ca0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.792+0000 7ff93946c700 1 -- 172.21.15.33:0/4128862573 shutdown_connections 2024-05-08T21:25:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.792+0000 7ff93946c700 1 --2- 172.21.15.33:0/4128862573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff934101920 0x7ff9341081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.792+0000 7ff93946c700 1 --2- 172.21.15.33:0/4128862573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9340ffd90 0x7ff9341001b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.792+0000 7ff93946c700 1 --2- 172.21.15.33:0/4128862573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff934100f80 0x7ff9341013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.792+0000 7ff93946c700 1 -- 172.21.15.33:0/4128862573 >> 172.21.15.33:0/4128862573 conn(0x7ff9340fb300 msgr2=0x7ff9340fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.792+0000 7ff93946c700 1 -- 172.21.15.33:0/4128862573 shutdown_connections 2024-05-08T21:25:35.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.792+0000 7ff93946c700 1 -- 172.21.15.33:0/4128862573 wait complete. 2024-05-08T21:25:35.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.793+0000 7ff93946c700 1 Processor -- start 2024-05-08T21:25:35.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.793+0000 7ff93946c700 1 -- start start 2024-05-08T21:25:35.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.793+0000 7ff93946c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9340ffd90 0x7ff93419fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.793+0000 7ff93946c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff934100f80 0x7ff9341a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.793+0000 7ff93946c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff934101920 0x7ff9341a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.793+0000 7ff93946c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9341a4e50 con 0x7ff9340ffd90 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.793+0000 7ff93946c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9341a4fc0 con 0x7ff934101920 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.793+0000 7ff93946c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9341a5180 con 0x7ff934100f80 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.793+0000 7ff932ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9340ffd90 0x7ff93419fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.793+0000 7ff932ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9340ffd90 0x7ff93419fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44862/0 (socket says 172.21.15.33:44862) 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.793+0000 7ff932ffd700 1 -- 172.21.15.33:0/1255430347 learned_addr learned my addr 172.21.15.33:0/1255430347 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff9337fe700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff934101920 0x7ff9341a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff9327fc700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff934100f80 0x7ff9341a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff932ffd700 1 -- 172.21.15.33:0/1255430347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff934100f80 msgr2=0x7ff9341a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff932ffd700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff934100f80 0x7ff9341a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff932ffd700 1 -- 172.21.15.33:0/1255430347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff934101920 msgr2=0x7ff9341a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff932ffd700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff934101920 0x7ff9341a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff932ffd700 1 -- 172.21.15.33:0/1255430347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9240095f0 con 0x7ff9340ffd90 2024-05-08T21:25:35.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff9327fc700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff934100f80 0x7ff9341a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:25:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff932ffd700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9340ffd90 0x7ff93419fac0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff9240098e0 tx=0x7ff92400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff91bfff700 1 -- 172.21.15.33:0/1255430347 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92401d070 con 0x7ff9340ffd90 2024-05-08T21:25:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff93946c700 1 -- 172.21.15.33:0/1255430347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9341a53a0 con 0x7ff9340ffd90 2024-05-08T21:25:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff91bfff700 1 -- 172.21.15.33:0/1255430347 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff924003d20 con 0x7ff9340ffd90 2024-05-08T21:25:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff91bfff700 1 -- 172.21.15.33:0/1255430347 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924022e30 con 0x7ff9340ffd90 2024-05-08T21:25:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.794+0000 7ff93946c700 1 -- 172.21.15.33:0/1255430347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9341a5820 con 0x7ff9340ffd90 2024-05-08T21:25:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.797+0000 7ff91bfff700 1 -- 172.21.15.33:0/1255430347 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff92400f460 con 0x7ff9340ffd90 2024-05-08T21:25:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.797+0000 7ff91bfff700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff91c0a1120 0x7ff91c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.798+0000 7ff9327fc700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff91c0a1120 0x7ff91c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.798+0000 7ff91bfff700 1 -- 172.21.15.33:0/1255430347 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff9240cdf70 con 0x7ff9340ffd90 2024-05-08T21:25:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.798+0000 7ff945626b80 1 Processor -- start 2024-05-08T21:25:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.798+0000 7ff9327fc700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff91c0a1120 0x7ff91c0a35e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff9341086e0 tx=0x7ff928009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.798+0000 7ff945626b80 1 -- start start 2024-05-08T21:25:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.798+0000 7ff945626b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 0x561d79f99960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.798+0000 7ff945626b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 0x561d79f9a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.798+0000 7ff945626b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f9a860 0x561d79f9ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.798+0000 7ff945626b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d79ef0150 con 0x561d79f99540 2024-05-08T21:25:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.798+0000 7ff945626b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d79eedb10 con 0x561d79f9a860 2024-05-08T21:25:35.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff945626b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d79ee9400 con 0x561d79f99ea0 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9327fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 0x561d79f9a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff932ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 0x561d79f99960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9327fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 0x561d79f9a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49616/0 (socket says 172.21.15.33:49616) 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9327fc700 1 -- 172.21.15.33:0/2679657752 learned_addr learned my addr 172.21.15.33:0/2679657752 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9337fe700 1 --2- 172.21.15.33:0/2679657752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f9a860 0x561d79f9ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9327fc700 1 -- 172.21.15.33:0/2679657752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f9a860 msgr2=0x561d79f9ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9327fc700 1 --2- 172.21.15.33:0/2679657752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f9a860 0x561d79f9ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9327fc700 1 -- 172.21.15.33:0/2679657752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 msgr2=0x561d79f99960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9327fc700 1 --2- 172.21.15.33:0/2679657752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 0x561d79f99960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9327fc700 1 -- 172.21.15.33:0/2679657752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d79d6a3b0 con 0x561d79f99ea0 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff932ffd700 1 --2- 172.21.15.33:0/2679657752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 0x561d79f99960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9327fc700 1 --2- 172.21.15.33:0/2679657752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 0x561d79f9a320 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff928010d10 tx=0x7ff928013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b5a6b672d4a9cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9197fa700 1 -- 172.21.15.33:0/2679657752 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928012ae0 con 0x561d79f99ea0 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9197fa700 1 -- 172.21.15.33:0/2679657752 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff928012c40 con 0x561d79f99ea0 2024-05-08T21:25:35.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.799+0000 7ff9197fa700 1 -- 172.21.15.33:0/2679657752 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92801e610 con 0x561d79f99ea0 2024-05-08T21:25:35.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.800+0000 7ff945626b80 1 -- 172.21.15.33:0/2679657752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 msgr2=0x561d79f9a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.800+0000 7ff945626b80 1 --2- 172.21.15.33:0/2679657752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 0x561d79f9a320 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff928010d10 tx=0x7ff928013000 comp rx=0 tx=0).stop 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.800+0000 7ff945626b80 1 -- 172.21.15.33:0/2679657752 shutdown_connections 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.800+0000 7ff945626b80 1 --2- 172.21.15.33:0/2679657752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f9a860 0x561d79f9ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.800+0000 7ff945626b80 1 --2- 172.21.15.33:0/2679657752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 0x561d79f99960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.800+0000 7ff945626b80 1 --2- 172.21.15.33:0/2679657752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 0x561d79f9a320 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.800+0000 7ff945626b80 1 -- 172.21.15.33:0/2679657752 >> 172.21.15.33:0/2679657752 conn(0x561d79ef7210 msgr2=0x561d79f93de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.800+0000 7ff945626b80 1 -- 172.21.15.33:0/2679657752 shutdown_connections 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.800+0000 7ff945626b80 1 -- 172.21.15.33:0/2679657752 wait complete. 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff945626b80 1 -- start start 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff945626b80 1 Processor -- start 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff945626b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 0x561d79fa2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff945626b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99ea0 0x561d79fa3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff945626b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f9a860 0x561d79faa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff945626b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d79e85560 con 0x561d79f99540 2024-05-08T21:25:35.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff945626b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d79e85200 con 0x561d79f99ea0 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff945626b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d79d6c6a0 con 0x561d79f9a860 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff945626b80 10 client.0 _ll_register_callbacks cb 0x561d79e8eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff932ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 0x561d79fa2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff932ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 0x561d79fa2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44882/0 (socket says 172.21.15.33:44882) 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff932ffd700 1 -- 172.21.15.33:0/4043076191 learned_addr learned my addr 172.21.15.33:0/4043076191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff9337fe700 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f9a860 0x561d79faa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.801+0000 7ff9327fc700 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99ea0 0x561d79fa3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff9337fe700 1 -- 172.21.15.33:0/4043076191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99ea0 msgr2=0x561d79fa3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff9337fe700 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99ea0 0x561d79fa3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff9337fe700 1 -- 172.21.15.33:0/4043076191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 msgr2=0x561d79fa2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff9337fe700 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 0x561d79fa2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff9337fe700 1 -- 172.21.15.33:0/4043076191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9341a5820 con 0x561d79f9a860 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff932ffd700 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 0x561d79fa2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff9337fe700 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f9a860 0x561d79faa3b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff92c007af0 tx=0x7ff92c007b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bcc17d5793288f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff916ff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff916ff5700 1 -- 172.21.15.33:0/4043076191 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c00e040 con 0x561d79f9a860 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff916ff5700 1 -- 172.21.15.33:0/4043076191 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff92c004db0 con 0x561d79f9a860 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff945626b80 1 -- 172.21.15.33:0/4043076191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d79e42270 con 0x561d79f9a860 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff916ff5700 1 -- 172.21.15.33:0/4043076191 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff92c015720 con 0x561d79f9a860 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff945626b80 10 client.27223 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.802+0000 7ff945626b80 1 -- 172.21.15.33:0/4043076191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7ff928010990 con 0x561d79f9a860 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.803+0000 7ff916ff5700 1 -- 172.21.15.33:0/4043076191 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561d79e42270 con 0x561d79f9a860 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.803+0000 7ff916ff5700 1 -- 172.21.15.33:0/4043076191 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff92c0168f0 con 0x561d79f9a860 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.803+0000 7ff945626b80 10 client.27223 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.803+0000 7ff945626b80 10 client.27223 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.803+0000 7ff945626b80 20 client.27223 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.803+0000 7ff945626b80 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561d79faec10 0x561d79fb1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.803+0000 7ff945626b80 4 client.27223 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.803+0000 7ff945626b80 1 -- 172.21.15.33:0/4043076191 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d79f68810 con 0x561d79faec10 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.803+0000 7ff932ffd700 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561d79faec10 0x561d79fb1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff932ffd700 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561d79faec10 0x561d79fb1030 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18745d8e9d83516d server_cookie=2ab2f18469514ce6 in_seq=0 out_seq=0 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff916ff5700 10 client.27223 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff916ff5700 1 -- 172.21.15.33:0/4043076191 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d79f68810 con 0x561d79faec10 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff916ff5700 10 client.27223 handle_command_reply: tid=0 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff945626b80 1 client.27223 shutdown 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff945626b80 1 -- 172.21.15.33:0/4043076191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f9a860 msgr2=0x561d79faa3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff945626b80 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f9a860 0x561d79faa3b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff92c007af0 tx=0x7ff92c007b20 comp rx=0 tx=0).stop 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff945626b80 1 -- 172.21.15.33:0/4043076191 shutdown_connections 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff945626b80 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561d79faec10 0x561d79fb1030 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff945626b80 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f9a860 0x561d79faa3b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff945626b80 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99540 0x561d79fa2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff945626b80 1 --2- 172.21.15.33:0/4043076191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99ea0 0x561d79fa3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff945626b80 1 -- 172.21.15.33:0/4043076191 >> 172.21.15.33:0/4043076191 conn(0x561d79ef7210 msgr2=0x561d79f935a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.804+0000 7ff916ff5700 0 client.27223 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:25:35.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.805+0000 7ff945626b80 1 -- 172.21.15.33:0/4043076191 shutdown_connections 2024-05-08T21:25:35.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.805+0000 7ff945626b80 1 -- 172.21.15.33:0/4043076191 wait complete. 2024-05-08T21:25:35.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.805+0000 7ff945626b80 20 client.27223 trim_cache size 0 max 16384 2024-05-08T21:25:35.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:35 smithi071 ceph-mon[47074]: pgmap v488: 129 pgs: 129 active+clean; 227 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.4 MiB/s wr, 371 op/s 2024-05-08T21:25:35.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:35 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2158554989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:25:35.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff945626b80 1 Processor -- start 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff945626b80 1 -- start start 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff945626b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 0x561d79f9d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff945626b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99ea0 0x561d79f9dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff945626b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f9a860 0x561d79faa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff945626b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d79fa29a0 con 0x561d79f99ea0 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff945626b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d79f90dc0 con 0x561d79f99540 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff945626b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d79f8e800 con 0x561d79f9a860 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff9327fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99ea0 0x561d79f9dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff932ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 0x561d79f9d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff9327fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99ea0 0x561d79f9dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44884/0 (socket says 172.21.15.33:44884) 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff9327fc700 1 -- 172.21.15.33:0/3538976076 learned_addr learned my addr 172.21.15.33:0/3538976076 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.821+0000 7ff9337fe700 1 --2- 172.21.15.33:0/3538976076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f9a860 0x561d79faa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.822+0000 7ff9327fc700 1 -- 172.21.15.33:0/3538976076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f9a860 msgr2=0x561d79faa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.822+0000 7ff9327fc700 1 --2- 172.21.15.33:0/3538976076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f9a860 0x561d79faa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.822+0000 7ff9327fc700 1 -- 172.21.15.33:0/3538976076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 msgr2=0x561d79f9d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.822+0000 7ff9327fc700 1 --2- 172.21.15.33:0/3538976076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 0x561d79f9d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.822+0000 7ff9327fc700 1 -- 172.21.15.33:0/3538976076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d79d16b60 con 0x561d79f99ea0 2024-05-08T21:25:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.822+0000 7ff932ffd700 1 --2- 172.21.15.33:0/3538976076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 0x561d79f9d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.822+0000 7ff9327fc700 1 --2- 172.21.15.33:0/3538976076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99ea0 0x561d79f9dd10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff928009960 tx=0x7ff928009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835c301e254aea7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.822+0000 7ff919ffb700 1 -- 172.21.15.33:0/3538976076 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9280128e0 con 0x561d79f99ea0 2024-05-08T21:25:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.822+0000 7ff919ffb700 1 -- 172.21.15.33:0/3538976076 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff92802f040 con 0x561d79f99ea0 2024-05-08T21:25:35.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.822+0000 7ff919ffb700 1 -- 172.21.15.33:0/3538976076 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92801f7d0 con 0x561d79f99ea0 2024-05-08T21:25:35.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.823+0000 7ff945626b80 1 -- 172.21.15.33:0/3538976076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99ea0 msgr2=0x561d79f9dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.823+0000 7ff945626b80 1 --2- 172.21.15.33:0/3538976076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99ea0 0x561d79f9dd10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff928009960 tx=0x7ff928009990 comp rx=0 tx=0).stop 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.823+0000 7ff945626b80 1 -- 172.21.15.33:0/3538976076 shutdown_connections 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.823+0000 7ff945626b80 1 --2- 172.21.15.33:0/3538976076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f9a860 0x561d79faa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.823+0000 7ff945626b80 1 --2- 172.21.15.33:0/3538976076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 0x561d79f9d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.823+0000 7ff945626b80 1 --2- 172.21.15.33:0/3538976076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f99ea0 0x561d79f9dd10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.823+0000 7ff945626b80 1 -- 172.21.15.33:0/3538976076 >> 172.21.15.33:0/3538976076 conn(0x561d79ef7210 msgr2=0x561d79fb5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.823+0000 7ff945626b80 1 -- 172.21.15.33:0/3538976076 shutdown_connections 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.823+0000 7ff945626b80 1 -- 172.21.15.33:0/3538976076 wait complete. 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff945626b80 1 -- start start 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff945626b80 1 Processor -- start 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff945626b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 0x561d79fa5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff945626b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 0x561d79fa61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff945626b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f9a860 0x561d79fa9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff945626b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d79fa96b0 con 0x561d79f9a860 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff945626b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d79fa9820 con 0x561d79f99540 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff945626b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d79fa99e0 con 0x561d79f99ea0 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff945626b80 10 client.0 _ll_register_callbacks cb 0x561d79e8eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff932ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 0x561d79fa5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff9327fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 0x561d79fa61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff9337fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f9a860 0x561d79fa9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff9327fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 0x561d79fa61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49640/0 (socket says 172.21.15.33:49640) 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff9337fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f9a860 0x561d79fa9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44894/0 (socket says 172.21.15.33:44894) 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff932ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 0x561d79fa5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56256/0 (socket says 172.21.15.33:56256) 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff9327fc700 1 -- 172.21.15.33:0/3511745124 learned_addr learned my addr 172.21.15.33:0/3511745124 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff9337fe700 1 -- 172.21.15.33:0/3511745124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 msgr2=0x561d79fa61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff9337fe700 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 0x561d79fa61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff9337fe700 1 -- 172.21.15.33:0/3511745124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 msgr2=0x561d79fa5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:35.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff9337fe700 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 0x561d79fa5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff9337fe700 1 -- 172.21.15.33:0/3511745124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9341a53a0 con 0x561d79f9a860 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff9327fc700 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 0x561d79fa61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.824+0000 7ff932ffd700 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 0x561d79fa5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.825+0000 7ff9337fe700 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f9a860 0x561d79fa9170 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff92c016d00 tx=0x7ff92c004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8f71953c97bd4f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.825+0000 7ff916ff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.825+0000 7ff916ff5700 1 -- 172.21.15.33:0/3511745124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c00e040 con 0x561d79f9a860 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.825+0000 7ff916ff5700 1 -- 172.21.15.33:0/3511745124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff92c009990 con 0x561d79f9a860 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.825+0000 7ff945626b80 1 -- 172.21.15.33:0/3511745124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff928012c40 con 0x561d79f9a860 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.825+0000 7ff916ff5700 1 -- 172.21.15.33:0/3511745124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff92c004fe0 con 0x561d79f9a860 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.825+0000 7ff945626b80 10 client.17316 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.825+0000 7ff945626b80 1 -- 172.21.15.33:0/3511745124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561d79d6a3b0 con 0x561d79f9a860 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.825+0000 7ff916ff5700 1 -- 172.21.15.33:0/3511745124 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff928012c40 con 0x561d79f9a860 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.826+0000 7ff916ff5700 1 -- 172.21.15.33:0/3511745124 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff92c02f070 con 0x561d79f9a860 2024-05-08T21:25:35.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.826+0000 7ff945626b80 10 client.17316 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.826+0000 7ff945626b80 10 client.17316 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.826+0000 7ff945626b80 20 client.17316 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.826+0000 7ff945626b80 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561d7a005a70 0x561d7a007e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.826+0000 7ff945626b80 4 client.17316 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.826+0000 7ff945626b80 1 -- 172.21.15.33:0/3511745124 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561d79ee1ec0 con 0x561d7a005a70 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.826+0000 7ff932ffd700 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561d7a005a70 0x561d7a007e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.826+0000 7ff916ff5700 10 client.17316 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.826+0000 7ff932ffd700 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561d7a005a70 0x561d7a007e90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=acfb623523ee40f2 server_cookie=2a666cbeab41db1 in_seq=0 out_seq=0 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff916ff5700 1 -- 172.21.15.33:0/3511745124 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561d79ee1ec0 con 0x561d7a005a70 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff916ff5700 10 client.17316 handle_command_reply: tid=0 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff945626b80 1 client.17316 shutdown 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff945626b80 1 -- 172.21.15.33:0/3511745124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f9a860 msgr2=0x561d79fa9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff945626b80 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f9a860 0x561d79fa9170 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff92c016d00 tx=0x7ff92c004c60 comp rx=0 tx=0).stop 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff945626b80 1 -- 172.21.15.33:0/3511745124 shutdown_connections 2024-05-08T21:25:35.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff945626b80 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561d7a005a70 0x561d7a007e90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff945626b80 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d79f9a860 0x561d79fa9170 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff945626b80 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d79f99540 0x561d79fa5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff916ff5700 0 client.17316 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:25:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff945626b80 1 --2- 172.21.15.33:0/3511745124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561d79f99ea0 0x561d79fa61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff945626b80 1 -- 172.21.15.33:0/3511745124 >> 172.21.15.33:0/3511745124 conn(0x561d79ef7210 msgr2=0x561d79f9d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.827+0000 7ff945626b80 1 -- 172.21.15.33:0/3511745124 shutdown_connections 2024-05-08T21:25:35.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.828+0000 7ff945626b80 1 -- 172.21.15.33:0/3511745124 wait complete. 2024-05-08T21:25:35.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.828+0000 7ff945626b80 20 client.17316 trim_cache size 0 max 16384 2024-05-08T21:25:35.834 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:25:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.828+0000 7ff93946c700 1 -- 172.21.15.33:0/1255430347 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff91c0a1120 msgr2=0x7ff91c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.828+0000 7ff93946c700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff91c0a1120 0x7ff91c0a35e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff9341086e0 tx=0x7ff928009260 comp rx=0 tx=0).stop 2024-05-08T21:25:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.828+0000 7ff93946c700 1 -- 172.21.15.33:0/1255430347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9340ffd90 msgr2=0x7ff93419fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.828+0000 7ff93946c700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9340ffd90 0x7ff93419fac0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff9240098e0 tx=0x7ff92400b9c0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.828+0000 7ff93946c700 1 -- 172.21.15.33:0/1255430347 shutdown_connections 2024-05-08T21:25:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.828+0000 7ff93946c700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff91c0a1120 0x7ff91c0a35e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.828+0000 7ff93946c700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff934101920 0x7ff9341a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.828+0000 7ff93946c700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9340ffd90 0x7ff93419fac0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.828+0000 7ff93946c700 1 --2- 172.21.15.33:0/1255430347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff934100f80 0x7ff9341a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.828+0000 7ff93946c700 1 -- 172.21.15.33:0/1255430347 >> 172.21.15.33:0/1255430347 conn(0x7ff9340fb300 msgr2=0x7ff934069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.829+0000 7ff93946c700 1 -- 172.21.15.33:0/1255430347 shutdown_connections 2024-05-08T21:25:35.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:35.829+0000 7ff93946c700 1 -- 172.21.15.33:0/1255430347 wait complete. 2024-05-08T21:25:35.847 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:25:35.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:35 smithi082 ceph-mon[47512]: pgmap v488: 129 pgs: 129 active+clean; 227 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.4 MiB/s wr, 371 op/s 2024-05-08T21:25:35.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:35 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2158554989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:25:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.011+0000 7f1949937700 1 -- 172.21.15.33:0/1360316974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1944100840 msgr2=0x7f19441050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.011+0000 7f1949937700 1 --2- 172.21.15.33:0/1360316974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1944100840 0x7f19441050e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f193c009f90 tx=0x7f193c00b330 comp rx=0 tx=0).stop 2024-05-08T21:25:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.011+0000 7f1949937700 1 -- 172.21.15.33:0/1360316974 shutdown_connections 2024-05-08T21:25:36.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.011+0000 7f1949937700 1 --2- 172.21.15.33:0/1360316974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1944100840 0x7f19441050e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.011+0000 7f1949937700 1 --2- 172.21.15.33:0/1360316974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19440ff120 0x7f19440ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.011+0000 7f1949937700 1 --2- 172.21.15.33:0/1360316974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19440ffe80 0x7f1944100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.011+0000 7f1949937700 1 -- 172.21.15.33:0/1360316974 >> 172.21.15.33:0/1360316974 conn(0x7f19440faa50 msgr2=0x7f19440fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.011+0000 7f1949937700 1 -- 172.21.15.33:0/1360316974 shutdown_connections 2024-05-08T21:25:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.011+0000 7f1949937700 1 -- 172.21.15.33:0/1360316974 wait complete. 2024-05-08T21:25:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.012+0000 7f1949937700 1 Processor -- start 2024-05-08T21:25:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.012+0000 7f1949937700 1 -- start start 2024-05-08T21:25:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.012+0000 7f1949937700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19440ff120 0x7f194419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.012+0000 7f1949937700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19440ffe80 0x7f194419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f1949937700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1944100840 0x7f19441a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f1949937700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19441a4ee0 con 0x7f19440ff120 2024-05-08T21:25:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f1949937700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19441a5020 con 0x7f19440ffe80 2024-05-08T21:25:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f1949937700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19441a51b0 con 0x7f1944100840 2024-05-08T21:25:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f1942ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19440ff120 0x7f194419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f19437fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1944100840 0x7f19441a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f1942ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19440ff120 0x7f194419d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44904/0 (socket says 172.21.15.33:44904) 2024-05-08T21:25:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f19437fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1944100840 0x7f19441a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49652/0 (socket says 172.21.15.33:49652) 2024-05-08T21:25:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f1942ffd700 1 -- 172.21.15.33:0/4113358898 learned_addr learned my addr 172.21.15.33:0/4113358898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f19427fc700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19440ffe80 0x7f194419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f19437fe700 1 -- 172.21.15.33:0/4113358898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19440ffe80 msgr2=0x7f194419d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f19437fe700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19440ffe80 0x7f194419d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f19437fe700 1 -- 172.21.15.33:0/4113358898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19440ff120 msgr2=0x7f194419d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f19437fe700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19440ff120 0x7f194419d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f19437fe700 1 -- 172.21.15.33:0/4113358898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934009520 con 0x7f1944100840 2024-05-08T21:25:36.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f1942ffd700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19440ff120 0x7f194419d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:36.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f19427fc700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19440ffe80 0x7f194419d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:36.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.013+0000 7f19437fe700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1944100840 0x7f19441a4910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f193c00b6c0 tx=0x7f193c008d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:36.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.014+0000 7f1948935700 1 -- 172.21.15.33:0/4113358898 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c00e040 con 0x7f1944100840 2024-05-08T21:25:36.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.014+0000 7f1948935700 1 -- 172.21.15.33:0/4113358898 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f193c00bc00 con 0x7f1944100840 2024-05-08T21:25:36.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.014+0000 7f1949937700 1 -- 172.21.15.33:0/4113358898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193c009c40 con 0x7f1944100840 2024-05-08T21:25:36.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.014+0000 7f1948935700 1 -- 172.21.15.33:0/4113358898 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c005040 con 0x7f1944100840 2024-05-08T21:25:36.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.014+0000 7f1949937700 1 -- 172.21.15.33:0/4113358898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19441a5770 con 0x7f1944100840 2024-05-08T21:25:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.016+0000 7f1948935700 1 -- 172.21.15.33:0/4113358898 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f193c0051a0 con 0x7f1944100840 2024-05-08T21:25:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.017+0000 7f1948935700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f192c0a11a0 0x7f192c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.017+0000 7f1942ffd700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f192c0a11a0 0x7f192c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.017+0000 7f1948935700 1 -- 172.21.15.33:0/4113358898 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f193c0cdf10 con 0x7f1944100840 2024-05-08T21:25:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.018+0000 7f1942ffd700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f192c0a11a0 0x7f192c0a3660 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1934009fd0 tx=0x7f1934009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.018+0000 7f1955af1b80 1 Processor -- start 2024-05-08T21:25:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.018+0000 7f1955af1b80 1 -- start start 2024-05-08T21:25:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.018+0000 7f1955af1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 0x562a83089960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.018+0000 7f1955af1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a83089ea0 0x562a8308a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1955af1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a8308a860 0x562a8308ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1955af1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a82fe0150 con 0x562a83089ea0 2024-05-08T21:25:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1955af1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a82fddb10 con 0x562a8308a860 2024-05-08T21:25:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1955af1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a82fd9400 con 0x562a83089540 2024-05-08T21:25:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1942ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 0x562a83089960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1942ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 0x562a83089960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49662/0 (socket says 172.21.15.33:49662) 2024-05-08T21:25:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1942ffd700 1 -- 172.21.15.33:0/1388636857 learned_addr learned my addr 172.21.15.33:0/1388636857 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f19437fe700 1 --2- 172.21.15.33:0/1388636857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a8308a860 0x562a8308ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f19427fc700 1 --2- 172.21.15.33:0/1388636857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a83089ea0 0x562a8308a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1942ffd700 1 -- 172.21.15.33:0/1388636857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a8308a860 msgr2=0x562a8308ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1942ffd700 1 --2- 172.21.15.33:0/1388636857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a8308a860 0x562a8308ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1942ffd700 1 -- 172.21.15.33:0/1388636857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a83089ea0 msgr2=0x562a8308a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1942ffd700 1 --2- 172.21.15.33:0/1388636857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a83089ea0 0x562a8308a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1942ffd700 1 -- 172.21.15.33:0/1388636857 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a82e5a3b0 con 0x562a83089540 2024-05-08T21:25:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.019+0000 7f1942ffd700 1 --2- 172.21.15.33:0/1388636857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 0x562a83089960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f193400f800 tx=0x7f1934014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=758733ed28a90a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.020+0000 7f1929ffb700 1 -- 172.21.15.33:0/1388636857 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934026640 con 0x562a83089540 2024-05-08T21:25:36.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.020+0000 7f1929ffb700 1 -- 172.21.15.33:0/1388636857 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f193401c8a0 con 0x562a83089540 2024-05-08T21:25:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.020+0000 7f1929ffb700 1 -- 172.21.15.33:0/1388636857 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193401b870 con 0x562a83089540 2024-05-08T21:25:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.021+0000 7f1955af1b80 1 -- 172.21.15.33:0/1388636857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 msgr2=0x562a83089960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.021+0000 7f1955af1b80 1 --2- 172.21.15.33:0/1388636857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 0x562a83089960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f193400f800 tx=0x7f1934014000 comp rx=0 tx=0).stop 2024-05-08T21:25:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.021+0000 7f1955af1b80 1 -- 172.21.15.33:0/1388636857 shutdown_connections 2024-05-08T21:25:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.021+0000 7f1955af1b80 1 --2- 172.21.15.33:0/1388636857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a8308a860 0x562a8308ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.021+0000 7f1955af1b80 1 --2- 172.21.15.33:0/1388636857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 0x562a83089960 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.021+0000 7f1955af1b80 1 --2- 172.21.15.33:0/1388636857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a83089ea0 0x562a8308a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.021+0000 7f1955af1b80 1 -- 172.21.15.33:0/1388636857 >> 172.21.15.33:0/1388636857 conn(0x562a82fe7210 msgr2=0x562a83083de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.021+0000 7f1955af1b80 1 -- 172.21.15.33:0/1388636857 shutdown_connections 2024-05-08T21:25:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.021+0000 7f1955af1b80 1 -- 172.21.15.33:0/1388636857 wait complete. 2024-05-08T21:25:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.021+0000 7f1955af1b80 1 -- start start 2024-05-08T21:25:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f1955af1b80 1 Processor -- start 2024-05-08T21:25:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f1955af1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089540 0x562a83092e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f1955af1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089ea0 0x562a83093340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f1955af1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a8308a860 0x562a8309a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f1955af1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a82f75560 con 0x562a8308a860 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f1955af1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a82f75200 con 0x562a83089540 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f1955af1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a82e5c6a0 con 0x562a83089ea0 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f1955af1b80 10 client.0 _ll_register_callbacks cb 0x562a82f7eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f19427fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089ea0 0x562a83093340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f19427fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089ea0 0x562a83093340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49672/0 (socket says 172.21.15.33:49672) 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f19427fc700 1 -- 172.21.15.33:0/596361251 learned_addr learned my addr 172.21.15.33:0/596361251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f19437fe700 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a8308a860 0x562a8309a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f1942ffd700 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089540 0x562a83092e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f19427fc700 1 -- 172.21.15.33:0/596361251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089540 msgr2=0x562a83092e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:36.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f19427fc700 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089540 0x562a83092e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f19427fc700 1 -- 172.21.15.33:0/596361251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a8308a860 msgr2=0x562a8309a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f19427fc700 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a8308a860 0x562a8309a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.022+0000 7f19427fc700 1 -- 172.21.15.33:0/596361251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1934017050 con 0x562a83089ea0 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.023+0000 7f19427fc700 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089ea0 0x562a83093340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1938012cf0 tx=0x7f1938009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e741d0818760ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.023+0000 7f1922ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.023+0000 7f1922ffd700 1 -- 172.21.15.33:0/596361251 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938017070 con 0x562a83089ea0 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.023+0000 7f1955af1b80 1 -- 172.21.15.33:0/596361251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a82f32270 con 0x562a83089ea0 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.023+0000 7f1922ffd700 1 -- 172.21.15.33:0/596361251 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f193800f830 con 0x562a83089ea0 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.023+0000 7f1922ffd700 1 -- 172.21.15.33:0/596361251 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19380122e0 con 0x562a83089ea0 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.023+0000 7f1955af1b80 10 client.27241 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.023+0000 7f1955af1b80 1 -- 172.21.15.33:0/596361251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x562a82e06b60 con 0x562a83089ea0 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.023+0000 7f1922ffd700 1 -- 172.21.15.33:0/596361251 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562a82f32270 con 0x562a83089ea0 2024-05-08T21:25:36.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.024+0000 7f1922ffd700 1 -- 172.21.15.33:0/596361251 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1938010ae0 con 0x562a83089ea0 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.024+0000 7f1955af1b80 10 client.27241 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.024+0000 7f1955af1b80 10 client.27241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.024+0000 7f1955af1b80 20 client.27241 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.024+0000 7f1955af1b80 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562a8309ed60 0x562a830a1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.024+0000 7f1955af1b80 4 client.27241 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.024+0000 7f1955af1b80 1 -- 172.21.15.33:0/596361251 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a83058810 con 0x562a8309ed60 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.024+0000 7f1942ffd700 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562a8309ed60 0x562a830a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.024+0000 7f1922ffd700 10 client.27241 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.024+0000 7f1942ffd700 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562a8309ed60 0x562a830a1180 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56ff9713703205f9 server_cookie=335453bcfd3aa395 in_seq=0 out_seq=0 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.025+0000 7f1922ffd700 1 -- 172.21.15.33:0/596361251 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a83058810 con 0x562a8309ed60 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.025+0000 7f1922ffd700 10 client.27241 handle_command_reply: tid=0 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.025+0000 7f1955af1b80 1 client.27241 shutdown 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.026+0000 7f1955af1b80 1 -- 172.21.15.33:0/596361251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089ea0 msgr2=0x562a83093340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.026+0000 7f1955af1b80 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089ea0 0x562a83093340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1938012cf0 tx=0x7f1938009e70 comp rx=0 tx=0).stop 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.027+0000 7f1955af1b80 1 -- 172.21.15.33:0/596361251 shutdown_connections 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.027+0000 7f1955af1b80 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562a8309ed60 0x562a830a1180 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.027+0000 7f1955af1b80 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a8308a860 0x562a8309a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.027+0000 7f1955af1b80 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089540 0x562a83092e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.027+0000 7f1955af1b80 1 --2- 172.21.15.33:0/596361251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089ea0 0x562a83093340 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.027+0000 7f1955af1b80 1 -- 172.21.15.33:0/596361251 >> 172.21.15.33:0/596361251 conn(0x562a82fe7210 msgr2=0x562a830835a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.027+0000 7f1922ffd700 0 client.27241 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.027+0000 7f1955af1b80 1 -- 172.21.15.33:0/596361251 shutdown_connections 2024-05-08T21:25:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.027+0000 7f1955af1b80 1 -- 172.21.15.33:0/596361251 wait complete. 2024-05-08T21:25:36.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.027+0000 7f1955af1b80 20 client.27241 trim_cache size 0 max 16384 2024-05-08T21:25:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.044+0000 7f1955af1b80 1 Processor -- start 2024-05-08T21:25:36.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.044+0000 7f1955af1b80 1 -- start start 2024-05-08T21:25:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1955af1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a83089540 0x562a8309aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1955af1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 0x562a8309a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1955af1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8308a860 0x562a83090d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1955af1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a83080dc0 con 0x562a83089540 2024-05-08T21:25:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1955af1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8309a7e0 con 0x562a83089ea0 2024-05-08T21:25:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1955af1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a83085570 con 0x562a8308a860 2024-05-08T21:25:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1942ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a83089540 0x562a8309aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f19427fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 0x562a8309a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1942ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a83089540 0x562a8309aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44924/0 (socket says 172.21.15.33:44924) 2024-05-08T21:25:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f19427fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 0x562a8309a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56310/0 (socket says 172.21.15.33:56310) 2024-05-08T21:25:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1942ffd700 1 -- 172.21.15.33:0/1862432822 learned_addr learned my addr 172.21.15.33:0/1862432822 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:36.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f19437fe700 1 --2- 172.21.15.33:0/1862432822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8308a860 0x562a83090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1942ffd700 1 -- 172.21.15.33:0/1862432822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8308a860 msgr2=0x562a83090d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1942ffd700 1 --2- 172.21.15.33:0/1862432822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8308a860 0x562a83090d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1942ffd700 1 -- 172.21.15.33:0/1862432822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 msgr2=0x562a8309a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1942ffd700 1 --2- 172.21.15.33:0/1862432822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 0x562a8309a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1942ffd700 1 -- 172.21.15.33:0/1862432822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a82f9f560 con 0x562a83089540 2024-05-08T21:25:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.045+0000 7f1942ffd700 1 --2- 172.21.15.33:0/1862432822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a83089540 0x562a8309aa30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1934010010 tx=0x7f193401cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8328e617d7bc667 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.046+0000 7f192a7fc700 1 -- 172.21.15.33:0/1862432822 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193401c410 con 0x562a83089540 2024-05-08T21:25:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.046+0000 7f192a7fc700 1 -- 172.21.15.33:0/1862432822 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f193401c570 con 0x562a83089540 2024-05-08T21:25:36.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.046+0000 7f192a7fc700 1 -- 172.21.15.33:0/1862432822 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19340049c0 con 0x562a83089540 2024-05-08T21:25:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.046+0000 7f1955af1b80 1 -- 172.21.15.33:0/1862432822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a83089540 msgr2=0x562a8309aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.046+0000 7f1955af1b80 1 --2- 172.21.15.33:0/1862432822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a83089540 0x562a8309aa30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1934010010 tx=0x7f193401cea0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 -- 172.21.15.33:0/1862432822 shutdown_connections 2024-05-08T21:25:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 --2- 172.21.15.33:0/1862432822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a8308a860 0x562a83090d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 --2- 172.21.15.33:0/1862432822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a83089540 0x562a8309aa30 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 --2- 172.21.15.33:0/1862432822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 0x562a8309a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 -- 172.21.15.33:0/1862432822 >> 172.21.15.33:0/1862432822 conn(0x562a82fe7210 msgr2=0x562a8308ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 -- 172.21.15.33:0/1862432822 shutdown_connections 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 -- 172.21.15.33:0/1862432822 wait complete. 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 -- start start 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 Processor -- start 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 0x562a83095460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 0x562a830959a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a8308a860 0x562a83098970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a83098eb0 con 0x562a8308a860 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a83099020 con 0x562a83089ea0 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.047+0000 7f1955af1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a830991e0 con 0x562a83089540 2024-05-08T21:25:36.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1955af1b80 10 client.0 _ll_register_callbacks cb 0x562a82f7eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1942ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 0x562a83095460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f19427fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 0x562a830959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f19437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a8308a860 0x562a83098970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1942ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 0x562a83095460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49680/0 (socket says 172.21.15.33:49680) 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f19427fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 0x562a830959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56314/0 (socket says 172.21.15.33:56314) 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1942ffd700 1 -- 172.21.15.33:0/48757176 learned_addr learned my addr 172.21.15.33:0/48757176 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1942ffd700 1 -- 172.21.15.33:0/48757176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 msgr2=0x562a830959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1942ffd700 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 0x562a830959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1942ffd700 1 -- 172.21.15.33:0/48757176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a8308a860 msgr2=0x562a83098970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1942ffd700 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a8308a860 0x562a83098970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1942ffd700 1 -- 172.21.15.33:0/48757176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193402a040 con 0x562a83089540 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f19427fc700 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 0x562a830959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f19437fe700 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a8308a860 0x562a83098970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1942ffd700 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 0x562a83095460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1934006010 tx=0x7f193401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89f6bf49a53840ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1922ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1922ffd700 1 -- 172.21.15.33:0/48757176 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934005040 con 0x562a83089540 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1955af1b80 1 -- 172.21.15.33:0/48757176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f193401c8a0 con 0x562a83089540 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.048+0000 7f1922ffd700 1 -- 172.21.15.33:0/48757176 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f19340051a0 con 0x562a83089540 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.049+0000 7f1922ffd700 1 -- 172.21.15.33:0/48757176 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1934005360 con 0x562a83089540 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.049+0000 7f1955af1b80 10 client.27247 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.049+0000 7f1955af1b80 1 -- 172.21.15.33:0/48757176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x562a82e5ecd0 con 0x562a83089540 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.049+0000 7f1922ffd700 1 -- 172.21.15.33:0/48757176 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f193401c8a0 con 0x562a83089540 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.049+0000 7f1922ffd700 1 -- 172.21.15.33:0/48757176 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1934042030 con 0x562a83089540 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.049+0000 7f1955af1b80 10 client.27247 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.049+0000 7f1955af1b80 10 client.27247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.049+0000 7f1955af1b80 20 client.27247 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.049+0000 7f1955af1b80 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562a830f5250 0x562a830f7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.049+0000 7f1955af1b80 4 client.27247 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:25:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.049+0000 7f1955af1b80 1 -- 172.21.15.33:0/48757176 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a82fd1ec0 con 0x562a830f5250 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.049+0000 7f19427fc700 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562a830f5250 0x562a830f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1922ffd700 10 client.27247 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f19427fc700 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562a830f5250 0x562a830f7640 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5189ad32056e00b0 server_cookie=6e598ab9b7e0bf8e in_seq=0 out_seq=0 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1922ffd700 1 -- 172.21.15.33:0/48757176 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a82fd1ec0 con 0x562a830f5250 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1922ffd700 10 client.27247 handle_command_reply: tid=0 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1955af1b80 1 client.27247 shutdown 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1955af1b80 1 -- 172.21.15.33:0/48757176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 msgr2=0x562a83095460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1955af1b80 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 0x562a83095460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1934006010 tx=0x7f193401c410 comp rx=0 tx=0).stop 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1955af1b80 1 -- 172.21.15.33:0/48757176 shutdown_connections 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1955af1b80 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562a830f5250 0x562a830f7640 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1955af1b80 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a8308a860 0x562a83098970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1955af1b80 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a83089540 0x562a83095460 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1955af1b80 1 --2- 172.21.15.33:0/48757176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a83089ea0 0x562a830959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1922ffd700 0 client.27247 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:36.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1955af1b80 1 -- 172.21.15.33:0/48757176 >> 172.21.15.33:0/48757176 conn(0x562a82fe7210 msgr2=0x562a83091ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:36.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1955af1b80 1 -- 172.21.15.33:0/48757176 shutdown_connections 2024-05-08T21:25:36.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1955af1b80 1 -- 172.21.15.33:0/48757176 wait complete. 2024-05-08T21:25:36.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.050+0000 7f1955af1b80 20 client.27247 trim_cache size 0 max 16384 2024-05-08T21:25:36.059 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:25:36.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.051+0000 7f1949937700 1 -- 172.21.15.33:0/4113358898 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f192c0a11a0 msgr2=0x7f192c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:36.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.051+0000 7f1949937700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f192c0a11a0 0x7f192c0a3660 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1934009fd0 tx=0x7f1934009260 comp rx=0 tx=0).stop 2024-05-08T21:25:36.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.051+0000 7f1949937700 1 -- 172.21.15.33:0/4113358898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1944100840 msgr2=0x7f19441a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:36.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.051+0000 7f1949937700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1944100840 0x7f19441a4910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f193c00b6c0 tx=0x7f193c008d00 comp rx=0 tx=0).stop 2024-05-08T21:25:36.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.051+0000 7f1949937700 1 -- 172.21.15.33:0/4113358898 shutdown_connections 2024-05-08T21:25:36.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.051+0000 7f1949937700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1944100840 0x7f19441a4910 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.051+0000 7f1949937700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19440ff120 0x7f194419d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.051+0000 7f1949937700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f192c0a11a0 0x7f192c0a3660 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.051+0000 7f1949937700 1 --2- 172.21.15.33:0/4113358898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19440ffe80 0x7f194419d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:36.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.051+0000 7f1949937700 1 -- 172.21.15.33:0/4113358898 >> 172.21.15.33:0/4113358898 conn(0x7f19440faa50 msgr2=0x7f19441010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:36.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.051+0000 7f1949937700 1 -- 172.21.15.33:0/4113358898 shutdown_connections 2024-05-08T21:25:36.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:36.051+0000 7f1949937700 1 -- 172.21.15.33:0/4113358898 wait complete. 2024-05-08T21:25:37.068 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:25:37.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:25:37.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.235+0000 7fd064905700 1 -- 172.21.15.33:0/2956230006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd060104270 msgr2=0x7fd060106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.235+0000 7fd064905700 1 --2- 172.21.15.33:0/2956230006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd060104270 0x7fd060106660 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd054009990 tx=0x7fd054009ca0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.235+0000 7fd064905700 1 -- 172.21.15.33:0/2956230006 shutdown_connections 2024-05-08T21:25:37.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.235+0000 7fd064905700 1 --2- 172.21.15.33:0/2956230006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060106ba0 0x7fd060108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.235+0000 7fd064905700 1 --2- 172.21.15.33:0/2956230006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd060101940 0x7fd060103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.235+0000 7fd064905700 1 --2- 172.21.15.33:0/2956230006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd060104270 0x7fd060106660 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.235+0000 7fd064905700 1 -- 172.21.15.33:0/2956230006 >> 172.21.15.33:0/2956230006 conn(0x7fd0600fb290 msgr2=0x7fd0600fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:37.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.235+0000 7fd064905700 1 -- 172.21.15.33:0/2956230006 shutdown_connections 2024-05-08T21:25:37.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.235+0000 7fd064905700 1 -- 172.21.15.33:0/2956230006 wait complete. 2024-05-08T21:25:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.236+0000 7fd064905700 1 Processor -- start 2024-05-08T21:25:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.236+0000 7fd064905700 1 -- start start 2024-05-08T21:25:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd064905700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd060101940 0x7fd060196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd064905700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060104270 0x7fd060197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd064905700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd060106ba0 0x7fd06019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd064905700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06019e830 con 0x7fd060106ba0 2024-05-08T21:25:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd064905700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06019e9a0 con 0x7fd060104270 2024-05-08T21:25:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd064905700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06019eb60 con 0x7fd060101940 2024-05-08T21:25:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd05ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd060106ba0 0x7fd06019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd05dd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060104270 0x7fd060197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd05ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd060106ba0 0x7fd06019e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44960/0 (socket says 172.21.15.33:44960) 2024-05-08T21:25:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd05dd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060104270 0x7fd060197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56328/0 (socket says 172.21.15.33:56328) 2024-05-08T21:25:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd05ed9d700 1 -- 172.21.15.33:0/2360065525 learned_addr learned my addr 172.21.15.33:0/2360065525 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd05ed9d700 1 -- 172.21.15.33:0/2360065525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd060101940 msgr2=0x7fd060196d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T21:25:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd060101940 0x7fd060196d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd05ed9d700 1 -- 172.21.15.33:0/2360065525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060104270 msgr2=0x7fd060197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060104270 0x7fd060197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd05ed9d700 1 -- 172.21.15.33:0/2360065525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0540095f0 con 0x7fd060106ba0 2024-05-08T21:25:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd05dd9b700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060104270 0x7fd060197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.237+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd060106ba0 0x7fd06019e260 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd058007ee0 tx=0x7fd05800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.238+0000 7fd04f7fe700 1 -- 172.21.15.33:0/2360065525 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058017070 con 0x7fd060106ba0 2024-05-08T21:25:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.238+0000 7fd04f7fe700 1 -- 172.21.15.33:0/2360065525 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd05800f040 con 0x7fd060106ba0 2024-05-08T21:25:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.238+0000 7fd04f7fe700 1 -- 172.21.15.33:0/2360065525 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058005710 con 0x7fd060106ba0 2024-05-08T21:25:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.238+0000 7fd064905700 1 -- 172.21.15.33:0/2360065525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06019ee40 con 0x7fd060106ba0 2024-05-08T21:25:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.238+0000 7fd064905700 1 -- 172.21.15.33:0/2360065525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06019f270 con 0x7fd060106ba0 2024-05-08T21:25:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.241+0000 7fd04f7fe700 1 -- 172.21.15.33:0/2360065525 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd058005870 con 0x7fd060106ba0 2024-05-08T21:25:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.241+0000 7fd04f7fe700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd0480a1120 0x7fd0480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.242+0000 7fd05e59c700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd0480a1120 0x7fd0480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.242+0000 7fd04f7fe700 1 -- 172.21.15.33:0/2360065525 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd0580cc3b0 con 0x7fd060106ba0 2024-05-08T21:25:37.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.242+0000 7fd05e59c700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd0480a1120 0x7fd0480a35e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd0601981c0 tx=0x7fd050009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:37.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.242+0000 7fd070abfb80 1 Processor -- start 2024-05-08T21:25:37.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd070abfb80 1 -- start start 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd070abfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd09d0 0x55af59cd0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd070abfb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd1330 0x55af59cd17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd070abfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1cf0 0x55af59cd2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd070abfb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af59c231d0 con 0x55af59cd1330 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd070abfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af59c20b90 con 0x55af59cd1cf0 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd070abfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af59c1c480 con 0x55af59cd09d0 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd1330 0x55af59cd17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd09d0 0x55af59cd0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd09d0 0x55af59cd0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49698/0 (socket says 172.21.15.33:49698) 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd1330 0x55af59cd17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44970/0 (socket says 172.21.15.33:44970) 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05e59c700 1 -- 172.21.15.33:0/1749462774 learned_addr learned my addr 172.21.15.33:0/1749462774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1cf0 0x55af59cd2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05ed9d700 1 -- 172.21.15.33:0/1749462774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd09d0 msgr2=0x55af59cd0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/1749462774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd09d0 0x55af59cd0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05ed9d700 1 -- 172.21.15.33:0/1749462774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd1330 msgr2=0x55af59cd17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/1749462774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd1330 0x55af59cd17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05ed9d700 1 -- 172.21.15.33:0/1749462774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af59a9d430 con 0x55af59cd1cf0 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05e59c700 1 --2- 172.21.15.33:0/1749462774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd09d0 0x55af59cd0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.243+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/1749462774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1cf0 0x55af59cd2170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd0580c9b40 tx=0x7fd0580b9050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4209da4a21c7b30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.244+0000 7fd04cff9700 1 -- 172.21.15.33:0/1749462774 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05809d120 con 0x55af59cd1cf0 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.244+0000 7fd04cff9700 1 -- 172.21.15.33:0/1749462774 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd058099be0 con 0x55af59cd1cf0 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.244+0000 7fd04cff9700 1 -- 172.21.15.33:0/1749462774 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058097690 con 0x55af59cd1cf0 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.245+0000 7fd070abfb80 1 -- 172.21.15.33:0/1749462774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1cf0 msgr2=0x55af59cd2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.245+0000 7fd070abfb80 1 --2- 172.21.15.33:0/1749462774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1cf0 0x55af59cd2170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd0580c9b40 tx=0x7fd0580b9050 comp rx=0 tx=0).stop 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.245+0000 7fd070abfb80 1 -- 172.21.15.33:0/1749462774 shutdown_connections 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.245+0000 7fd070abfb80 1 --2- 172.21.15.33:0/1749462774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1cf0 0x55af59cd2170 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.245+0000 7fd070abfb80 1 --2- 172.21.15.33:0/1749462774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd09d0 0x55af59cd0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.245+0000 7fd070abfb80 1 --2- 172.21.15.33:0/1749462774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd1330 0x55af59cd17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.245+0000 7fd070abfb80 1 -- 172.21.15.33:0/1749462774 >> 172.21.15.33:0/1749462774 conn(0x55af59c2a290 msgr2=0x55af59cc6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.245+0000 7fd070abfb80 1 -- 172.21.15.33:0/1749462774 shutdown_connections 2024-05-08T21:25:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.245+0000 7fd070abfb80 1 -- 172.21.15.33:0/1749462774 wait complete. 2024-05-08T21:25:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.245+0000 7fd070abfb80 1 -- start start 2024-05-08T21:25:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.245+0000 7fd070abfb80 1 Processor -- start 2024-05-08T21:25:37.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.245+0000 7fd070abfb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 0x55af59cda1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd070abfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1330 0x55af59cda710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd070abfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1cf0 0x55af59ce1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd070abfb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af59bb85e0 con 0x55af59cd09d0 2024-05-08T21:25:37.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd070abfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af59bb8280 con 0x55af59cd1cf0 2024-05-08T21:25:37.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd070abfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af59a9f720 con 0x55af59cd1330 2024-05-08T21:25:37.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd070abfb80 10 client.0 _ll_register_callbacks cb 0x55af59bc1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd05e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 0x55af59cda1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd05e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 0x55af59cda1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44986/0 (socket says 172.21.15.33:44986) 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd05e59c700 1 -- 172.21.15.33:0/2134331758 learned_addr learned my addr 172.21.15.33:0/2134331758 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1cf0 0x55af59ce1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd05dd9b700 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1330 0x55af59cda710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd05ed9d700 1 -- 172.21.15.33:0/2134331758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1330 msgr2=0x55af59cda710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1330 0x55af59cda710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd05ed9d700 1 -- 172.21.15.33:0/2134331758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 msgr2=0x55af59cda1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 0x55af59cda1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd05ed9d700 1 -- 172.21.15.33:0/2134331758 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05000e820 con 0x55af59cd1cf0 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd05e59c700 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 0x55af59cda1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1cf0 0x55af59ce1700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd0580c9bc0 tx=0x7fd05800d8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd786bcaa438026 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd045ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.246+0000 7fd045ffb700 1 -- 172.21.15.33:0/2134331758 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0580994e0 con 0x55af59cd1cf0 2024-05-08T21:25:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd045ffb700 1 -- 172.21.15.33:0/2134331758 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd05809d310 con 0x55af59cd1cf0 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd070abfb80 1 -- 172.21.15.33:0/2134331758 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af59b752f0 con 0x55af59cd1cf0 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd045ffb700 1 -- 172.21.15.33:0/2134331758 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0580840c0 con 0x55af59cd1cf0 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd070abfb80 10 client.27260 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd070abfb80 1 -- 172.21.15.33:0/2134331758 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fd06019f270 con 0x55af59cd1cf0 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd045ffb700 1 -- 172.21.15.33:0/2134331758 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd05807d020 con 0x55af59cd1cf0 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd045ffb700 1 -- 172.21.15.33:0/2134331758 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd058099740 con 0x55af59cd1cf0 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd070abfb80 10 client.27260 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd070abfb80 10 client.27260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd070abfb80 20 client.27260 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd070abfb80 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55af59ce5d00 0x55af59ce80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd070abfb80 4 client.27260 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd070abfb80 1 -- 172.21.15.33:0/2134331758 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af59c9b890 con 0x55af59ce5d00 2024-05-08T21:25:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.247+0000 7fd05e59c700 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55af59ce5d00 0x55af59ce80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.248+0000 7fd05e59c700 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55af59ce5d00 0x55af59ce80f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8b4ebda59007bd9 server_cookie=c46b45a12ba6dd81 in_seq=0 out_seq=0 2024-05-08T21:25:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.248+0000 7fd045ffb700 10 client.27260 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.248+0000 7fd045ffb700 1 -- 172.21.15.33:0/2134331758 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af59c9b890 con 0x55af59ce5d00 2024-05-08T21:25:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.248+0000 7fd045ffb700 10 client.27260 handle_command_reply: tid=0 2024-05-08T21:25:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.248+0000 7fd070abfb80 1 client.27260 shutdown 2024-05-08T21:25:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.248+0000 7fd070abfb80 1 -- 172.21.15.33:0/2134331758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1cf0 msgr2=0x55af59ce1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.248+0000 7fd070abfb80 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1cf0 0x55af59ce1700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd0580c9bc0 tx=0x7fd05800d8e0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.249+0000 7fd070abfb80 1 -- 172.21.15.33:0/2134331758 shutdown_connections 2024-05-08T21:25:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.249+0000 7fd070abfb80 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55af59ce5d00 0x55af59ce80f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.249+0000 7fd070abfb80 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1cf0 0x55af59ce1700 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.249+0000 7fd070abfb80 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 0x55af59cda1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.249+0000 7fd070abfb80 1 --2- 172.21.15.33:0/2134331758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1330 0x55af59cda710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.249+0000 7fd045ffb700 0 client.27260 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:37.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.249+0000 7fd070abfb80 1 -- 172.21.15.33:0/2134331758 >> 172.21.15.33:0/2134331758 conn(0x55af59c2a290 msgr2=0x55af59cc6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:37.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.251+0000 7fd070abfb80 1 -- 172.21.15.33:0/2134331758 shutdown_connections 2024-05-08T21:25:37.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.251+0000 7fd070abfb80 1 -- 172.21.15.33:0/2134331758 wait complete. 2024-05-08T21:25:37.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.252+0000 7fd070abfb80 20 client.27260 trim_cache size 0 max 16384 2024-05-08T21:25:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.274+0000 7fd070abfb80 1 Processor -- start 2024-05-08T21:25:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.274+0000 7fd070abfb80 1 -- start start 2024-05-08T21:25:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.274+0000 7fd070abfb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 0x55af59cd4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.274+0000 7fd070abfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1330 0x55af59cd5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.274+0000 7fd070abfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1cf0 0x55af59ce16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.274+0000 7fd070abfb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af59cc3e40 con 0x55af59cd09d0 2024-05-08T21:25:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.274+0000 7fd070abfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af59cdb370 con 0x55af59cd1330 2024-05-08T21:25:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.274+0000 7fd070abfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af59ced2e0 con 0x55af59cd1cf0 2024-05-08T21:25:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd05e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 0x55af59cd4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd05e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 0x55af59cd4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45000/0 (socket says 172.21.15.33:45000) 2024-05-08T21:25:37.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd05ed9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1cf0 0x55af59ce16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd05e59c700 1 -- 172.21.15.33:0/2975219267 learned_addr learned my addr 172.21.15.33:0/2975219267 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd05dd9b700 1 --2- 172.21.15.33:0/2975219267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1330 0x55af59cd5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd05ed9d700 1 -- 172.21.15.33:0/2975219267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1330 msgr2=0x55af59cd5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/2975219267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1330 0x55af59cd5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd05ed9d700 1 -- 172.21.15.33:0/2975219267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 msgr2=0x55af59cd4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/2975219267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 0x55af59cd4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd05ed9d700 1 -- 172.21.15.33:0/2975219267 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af59a49be0 con 0x55af59cd1cf0 2024-05-08T21:25:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd05e59c700 1 --2- 172.21.15.33:0/2975219267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 0x55af59cd4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/2975219267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1cf0 0x55af59ce16c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd0580a1430 tx=0x7fd05800d950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e34c367033e6c7a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.275+0000 7fd04d7fa700 1 -- 172.21.15.33:0/2975219267 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05809d120 con 0x55af59cd1cf0 2024-05-08T21:25:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.276+0000 7fd04d7fa700 1 -- 172.21.15.33:0/2975219267 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd05807d070 con 0x55af59cd1cf0 2024-05-08T21:25:37.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.276+0000 7fd04d7fa700 1 -- 172.21.15.33:0/2975219267 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05809d3a0 con 0x55af59cd1cf0 2024-05-08T21:25:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.276+0000 7fd070abfb80 1 -- 172.21.15.33:0/2975219267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1cf0 msgr2=0x55af59ce16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.276+0000 7fd070abfb80 1 --2- 172.21.15.33:0/2975219267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1cf0 0x55af59ce16c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd0580a1430 tx=0x7fd05800d950 comp rx=0 tx=0).stop 2024-05-08T21:25:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.276+0000 7fd070abfb80 1 -- 172.21.15.33:0/2975219267 shutdown_connections 2024-05-08T21:25:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.276+0000 7fd070abfb80 1 --2- 172.21.15.33:0/2975219267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1cf0 0x55af59ce16c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.276+0000 7fd070abfb80 1 --2- 172.21.15.33:0/2975219267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd09d0 0x55af59cd4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.276+0000 7fd070abfb80 1 --2- 172.21.15.33:0/2975219267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd1330 0x55af59cd5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.276+0000 7fd070abfb80 1 -- 172.21.15.33:0/2975219267 >> 172.21.15.33:0/2975219267 conn(0x55af59c2a290 msgr2=0x55af59cc3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.276+0000 7fd070abfb80 1 -- 172.21.15.33:0/2975219267 shutdown_connections 2024-05-08T21:25:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.276+0000 7fd070abfb80 1 -- 172.21.15.33:0/2975219267 wait complete. 2024-05-08T21:25:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd070abfb80 1 -- start start 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd070abfb80 1 Processor -- start 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd070abfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd09d0 0x55af59cdf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd070abfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1330 0x55af59cdf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd070abfb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd1cf0 0x55af59ce09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd070abfb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af59ce0ee0 con 0x55af59cd1cf0 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd070abfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af59ce1050 con 0x55af59cd09d0 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd070abfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af59ce1210 con 0x55af59cd1330 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd070abfb80 10 client.0 _ll_register_callbacks cb 0x55af59bc1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd05dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1330 0x55af59cdf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd05dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1330 0x55af59cdf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49720/0 (socket says 172.21.15.33:49720) 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd05dd9b700 1 -- 172.21.15.33:0/1187919966 learned_addr learned my addr 172.21.15.33:0/1187919966 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.277+0000 7fd05ed9d700 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd1cf0 0x55af59ce09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd05e59c700 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd09d0 0x55af59cdf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd05dd9b700 1 -- 172.21.15.33:0/1187919966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd09d0 msgr2=0x55af59cdf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd05dd9b700 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd09d0 0x55af59cdf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd05dd9b700 1 -- 172.21.15.33:0/1187919966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd1cf0 msgr2=0x55af59ce09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd05dd9b700 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd1cf0 0x55af59ce09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd05dd9b700 1 -- 172.21.15.33:0/1187919966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05001d040 con 0x55af59cd1330 2024-05-08T21:25:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd05dd9b700 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1330 0x55af59cdf990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd054000c00 tx=0x7fd054005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ae931553ba8777a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd045ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:25:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd045ffb700 1 -- 172.21.15.33:0/1187919966 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05401d040 con 0x55af59cd1330 2024-05-08T21:25:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd070abfb80 1 -- 172.21.15.33:0/1187919966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd058099be0 con 0x55af59cd1330 2024-05-08T21:25:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd045ffb700 1 -- 172.21.15.33:0/1187919966 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd05400b710 con 0x55af59cd1330 2024-05-08T21:25:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd045ffb700 1 -- 172.21.15.33:0/1187919966 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd05400b8d0 con 0x55af59cd1330 2024-05-08T21:25:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd070abfb80 10 client.27262 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.278+0000 7fd070abfb80 1 -- 172.21.15.33:0/1187919966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fd06019f270 con 0x55af59cd1330 2024-05-08T21:25:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.279+0000 7fd045ffb700 1 -- 172.21.15.33:0/1187919966 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd058099be0 con 0x55af59cd1330 2024-05-08T21:25:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.279+0000 7fd045ffb700 1 -- 172.21.15.33:0/1187919966 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd05402c070 con 0x55af59cd1330 2024-05-08T21:25:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.279+0000 7fd070abfb80 10 client.27262 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.279+0000 7fd070abfb80 10 client.27262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.279+0000 7fd070abfb80 20 client.27262 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.279+0000 7fd070abfb80 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55af59d3d0f0 0x55af59d3f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.279+0000 7fd070abfb80 4 client.27262 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.279+0000 7fd070abfb80 1 -- 172.21.15.33:0/1187919966 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55af59c14f40 con 0x55af59d3d0f0 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.279+0000 7fd05e59c700 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55af59d3d0f0 0x55af59d3f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.280+0000 7fd045ffb700 10 client.27262 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.280+0000 7fd05e59c700 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55af59d3d0f0 0x55af59d3f510 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6e1efd67b0de6e1 server_cookie=efcde0872b6bc423 in_seq=0 out_seq=0 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd045ffb700 1 -- 172.21.15.33:0/1187919966 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55af59c14f40 con 0x55af59d3d0f0 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd045ffb700 10 client.27262 handle_command_reply: tid=0 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd070abfb80 1 client.27262 shutdown 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd070abfb80 1 -- 172.21.15.33:0/1187919966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1330 msgr2=0x55af59cdf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd070abfb80 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1330 0x55af59cdf990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd054000c00 tx=0x7fd054005270 comp rx=0 tx=0).stop 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd070abfb80 1 -- 172.21.15.33:0/1187919966 shutdown_connections 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd070abfb80 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55af59d3d0f0 0x55af59d3f510 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd070abfb80 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af59cd1cf0 0x55af59ce09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd070abfb80 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af59cd09d0 0x55af59cdf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd045ffb700 0 client.27262 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd070abfb80 1 --2- 172.21.15.33:0/1187919966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af59cd1330 0x55af59cdf990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd070abfb80 1 -- 172.21.15.33:0/1187919966 >> 172.21.15.33:0/1187919966 conn(0x55af59c2a290 msgr2=0x55af59cd4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd070abfb80 1 -- 172.21.15.33:0/1187919966 shutdown_connections 2024-05-08T21:25:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd070abfb80 1 -- 172.21.15.33:0/1187919966 wait complete. 2024-05-08T21:25:37.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.281+0000 7fd070abfb80 20 client.27262 trim_cache size 0 max 16384 2024-05-08T21:25:37.288 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:25:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:25:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "ea0c18f1-d682-4321-bf52-114825a506c3", 2024-05-08T21:25:37.288 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:25:37.289 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:25:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.282+0000 7fd064905700 1 -- 172.21.15.33:0/2360065525 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd0480a1120 msgr2=0x7fd0480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.282+0000 7fd064905700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd0480a1120 0x7fd0480a35e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd0601981c0 tx=0x7fd050009260 comp rx=0 tx=0).stop 2024-05-08T21:25:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.282+0000 7fd064905700 1 -- 172.21.15.33:0/2360065525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd060106ba0 msgr2=0x7fd06019e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.282+0000 7fd064905700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd060106ba0 0x7fd06019e260 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd058007ee0 tx=0x7fd05800d3b0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.282+0000 7fd064905700 1 -- 172.21.15.33:0/2360065525 shutdown_connections 2024-05-08T21:25:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.282+0000 7fd064905700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd060106ba0 0x7fd06019e260 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.282+0000 7fd064905700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd060101940 0x7fd060196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.282+0000 7fd064905700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd0480a1120 0x7fd0480a35e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.282+0000 7fd064905700 1 --2- 172.21.15.33:0/2360065525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd060104270 0x7fd060197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.282+0000 7fd064905700 1 -- 172.21.15.33:0/2360065525 >> 172.21.15.33:0/2360065525 conn(0x7fd0600fb290 msgr2=0x7fd060102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.282+0000 7fd064905700 1 -- 172.21.15.33:0/2360065525 shutdown_connections 2024-05-08T21:25:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.282+0000 7fd064905700 1 -- 172.21.15.33:0/2360065525 wait complete. 2024-05-08T21:25:37.299 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:25:37.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:37 smithi033 ceph-mon[42528]: pgmap v489: 129 pgs: 129 active+clean; 224 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.4 MiB/s wr, 534 op/s 2024-05-08T21:25:37.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.494+0000 7f175401a700 1 -- 172.21.15.33:0/739542981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f174c102040 msgr2=0x7f174c1044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.494+0000 7f175401a700 1 --2- 172.21.15.33:0/739542981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f174c102040 0x7f174c1044d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f173c0098f0 tx=0x7f173c009c00 comp rx=0 tx=0).stop 2024-05-08T21:25:37.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.494+0000 7f175401a700 1 -- 172.21.15.33:0/739542981 shutdown_connections 2024-05-08T21:25:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.494+0000 7f175401a700 1 --2- 172.21.15.33:0/739542981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f174c104a10 0x7f174c104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.494+0000 7f175401a700 1 --2- 172.21.15.33:0/739542981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f174c0ff6d0 0x7f174c101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.494+0000 7f175401a700 1 --2- 172.21.15.33:0/739542981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f174c102040 0x7f174c1044d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.494+0000 7f175401a700 1 -- 172.21.15.33:0/739542981 >> 172.21.15.33:0/739542981 conn(0x7f174c0fb290 msgr2=0x7f174c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.494+0000 7f175401a700 1 -- 172.21.15.33:0/739542981 shutdown_connections 2024-05-08T21:25:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.494+0000 7f175401a700 1 -- 172.21.15.33:0/739542981 wait complete. 2024-05-08T21:25:37.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.495+0000 7f175401a700 1 Processor -- start 2024-05-08T21:25:37.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.495+0000 7f175401a700 1 -- start start 2024-05-08T21:25:37.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.496+0000 7f175401a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f174c0ff6d0 0x7f174c19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.496+0000 7f175401a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f174c102040 0x7f174c19d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.496+0000 7f175401a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f174c104a10 0x7f174c1a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.496+0000 7f175401a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c1a4ee0 con 0x7f174c102040 2024-05-08T21:25:37.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.496+0000 7f175401a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c1a5020 con 0x7f174c0ff6d0 2024-05-08T21:25:37.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.496+0000 7f175401a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c1a51b0 con 0x7f174c104a10 2024-05-08T21:25:37.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.496+0000 7f17525b7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f174c104a10 0x7f174c1a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.496+0000 7f17525b7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f174c104a10 0x7f174c1a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49732/0 (socket says 172.21.15.33:49732) 2024-05-08T21:25:37.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.496+0000 7f17525b7700 1 -- 172.21.15.33:0/1985068811 learned_addr learned my addr 172.21.15.33:0/1985068811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:37.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.496+0000 7f1751db6700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f174c0ff6d0 0x7f174c19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.496+0000 7f17515b5700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f174c102040 0x7f174c19d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.497+0000 7f17525b7700 1 -- 172.21.15.33:0/1985068811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f174c0ff6d0 msgr2=0x7f174c19d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.497+0000 7f17525b7700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f174c0ff6d0 0x7f174c19d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.497+0000 7f17525b7700 1 -- 172.21.15.33:0/1985068811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f174c102040 msgr2=0x7f174c19d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.497+0000 7f17525b7700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f174c102040 0x7f174c19d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.497+0000 7f17525b7700 1 -- 172.21.15.33:0/1985068811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1744009c40 con 0x7f174c104a10 2024-05-08T21:25:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.497+0000 7f17525b7700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f174c104a10 0x7f174c1a4910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f174400c370 tx=0x7f174400c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.498+0000 7f1742ffd700 1 -- 172.21.15.33:0/1985068811 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1744017070 con 0x7f174c104a10 2024-05-08T21:25:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.498+0000 7f1742ffd700 1 -- 172.21.15.33:0/1985068811 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f174400f040 con 0x7f174c104a10 2024-05-08T21:25:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.498+0000 7f1742ffd700 1 -- 172.21.15.33:0/1985068811 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174400e050 con 0x7f174c104a10 2024-05-08T21:25:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.498+0000 7f175401a700 1 -- 172.21.15.33:0/1985068811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c009520 con 0x7f174c104a10 2024-05-08T21:25:37.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.498+0000 7f175401a700 1 -- 172.21.15.33:0/1985068811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c1a5760 con 0x7f174c104a10 2024-05-08T21:25:37.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.503+0000 7f1742ffd700 1 -- 172.21.15.33:0/1985068811 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1744003a00 con 0x7f174c104a10 2024-05-08T21:25:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.503+0000 7f1742ffd700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17380a1220 0x7f17380a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.504+0000 7f1742ffd700 1 -- 172.21.15.33:0/1985068811 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f17440cc5b0 con 0x7f174c104a10 2024-05-08T21:25:37.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.504+0000 7f17601d4b80 1 Processor -- start 2024-05-08T21:25:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.504+0000 7f1751db6700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17380a1220 0x7f17380a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.504+0000 7f17601d4b80 1 -- start start 2024-05-08T21:25:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.504+0000 7f17601d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111540 0x559b87111960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17601d4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b87112320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17601d4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87112860 0x559b87112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17601d4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b87068150 con 0x559b87112860 2024-05-08T21:25:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17601d4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b87065b10 con 0x559b87111ea0 2024-05-08T21:25:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17601d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b87061400 con 0x559b87111540 2024-05-08T21:25:37.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17515b5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b87112320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17515b5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b87112320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56382/0 (socket says 172.21.15.33:56382) 2024-05-08T21:25:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17515b5700 1 -- 172.21.15.33:0/1747712937 learned_addr learned my addr 172.21.15.33:0/1747712937 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17525b7700 1 --2- 172.21.15.33:0/1747712937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87112860 0x559b87112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17515b5700 1 -- 172.21.15.33:0/1747712937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111540 msgr2=0x559b87111960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:25:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17515b5700 1 --2- 172.21.15.33:0/1747712937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111540 0x559b87111960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17515b5700 1 -- 172.21.15.33:0/1747712937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87112860 msgr2=0x559b87112ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17515b5700 1 --2- 172.21.15.33:0/1747712937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87112860 0x559b87112ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17515b5700 1 -- 172.21.15.33:0/1747712937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b86ee23b0 con 0x559b87111ea0 2024-05-08T21:25:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.505+0000 7f17515b5700 1 --2- 172.21.15.33:0/1747712937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b87112320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f173c0098f0 tx=0x7f173c009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e259bbef5932961 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.506+0000 7f1751db6700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17380a1220 0x7f17380a36e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1748005fd0 tx=0x7f1748009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.507+0000 7f1737fff700 1 -- 172.21.15.33:0/1747712937 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c00fda0 con 0x559b87111ea0 2024-05-08T21:25:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.507+0000 7f1737fff700 1 -- 172.21.15.33:0/1747712937 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f173c01d040 con 0x559b87111ea0 2024-05-08T21:25:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.507+0000 7f1737fff700 1 -- 172.21.15.33:0/1747712937 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c005040 con 0x559b87111ea0 2024-05-08T21:25:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.508+0000 7f17601d4b80 1 -- 172.21.15.33:0/1747712937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 msgr2=0x559b87112320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.508+0000 7f17601d4b80 1 --2- 172.21.15.33:0/1747712937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b87112320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f173c0098f0 tx=0x7f173c009f50 comp rx=0 tx=0).stop 2024-05-08T21:25:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.508+0000 7f17601d4b80 1 -- 172.21.15.33:0/1747712937 shutdown_connections 2024-05-08T21:25:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.508+0000 7f17601d4b80 1 --2- 172.21.15.33:0/1747712937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87112860 0x559b87112ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.508+0000 7f17601d4b80 1 --2- 172.21.15.33:0/1747712937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111540 0x559b87111960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.508+0000 7f17601d4b80 1 --2- 172.21.15.33:0/1747712937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b87112320 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.508+0000 7f17601d4b80 1 -- 172.21.15.33:0/1747712937 >> 172.21.15.33:0/1747712937 conn(0x559b8706f210 msgr2=0x559b8710bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.508+0000 7f17601d4b80 1 -- 172.21.15.33:0/1747712937 shutdown_connections 2024-05-08T21:25:37.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.508+0000 7f17601d4b80 1 -- 172.21.15.33:0/1747712937 wait complete. 2024-05-08T21:25:37.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.508+0000 7f17601d4b80 1 -- start start 2024-05-08T21:25:37.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.509+0000 7f17601d4b80 1 Processor -- start 2024-05-08T21:25:37.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.509+0000 7f17601d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111540 0x559b8711ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.509+0000 7f17601d4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b8711b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.509+0000 7f17601d4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87112860 0x559b871222f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.509+0000 7f17601d4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b86ffd560 con 0x559b87112860 2024-05-08T21:25:37.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.509+0000 7f17601d4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b86ffd200 con 0x559b87111ea0 2024-05-08T21:25:37.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.509+0000 7f17601d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b86ee46a0 con 0x559b87111540 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.509+0000 7f17601d4b80 10 client.0 _ll_register_callbacks cb 0x559b87006b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.509+0000 7f1751db6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111540 0x559b8711ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.509+0000 7f17515b5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b8711b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.510+0000 7f1751db6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111540 0x559b8711ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:49738/0 (socket says 172.21.15.33:49738) 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.510+0000 7f17515b5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b8711b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56384/0 (socket says 172.21.15.33:56384) 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.510+0000 7f1751db6700 1 -- 172.21.15.33:0/714426741 learned_addr learned my addr 172.21.15.33:0/714426741 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.510+0000 7f17525b7700 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87112860 0x559b871222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.510+0000 7f17515b5700 1 -- 172.21.15.33:0/714426741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111540 msgr2=0x559b8711ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.510+0000 7f17515b5700 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111540 0x559b8711ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.510+0000 7f17515b5700 1 -- 172.21.15.33:0/714426741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87112860 msgr2=0x559b871222f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.510+0000 7f17515b5700 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87112860 0x559b871222f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.510+0000 7f17515b5700 1 -- 172.21.15.33:0/714426741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c1059c0 con 0x559b87111ea0 2024-05-08T21:25:37.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.511+0000 7f1751db6700 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111540 0x559b8711ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.511+0000 7f17515b5700 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b8711b2b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f173c009810 tx=0x7f173c00b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f760b82c8037d23f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.511+0000 7f17357fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.511+0000 7f17357fa700 1 -- 172.21.15.33:0/714426741 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c00f660 con 0x559b87111ea0 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.511+0000 7f17601d4b80 1 -- 172.21.15.33:0/714426741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b86fba270 con 0x559b87111ea0 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.511+0000 7f17357fa700 1 -- 172.21.15.33:0/714426741 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f173c005040 con 0x559b87111ea0 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.511+0000 7f17357fa700 1 -- 172.21.15.33:0/714426741 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f173c00fe90 con 0x559b87111ea0 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.511+0000 7f17601d4b80 10 client.27278 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.512+0000 7f17601d4b80 1 -- 172.21.15.33:0/714426741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559b86e8eb60 con 0x559b87111ea0 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.512+0000 7f17357fa700 1 -- 172.21.15.33:0/714426741 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559b86fba270 con 0x559b87111ea0 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.512+0000 7f17357fa700 1 -- 172.21.15.33:0/714426741 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f173c015020 con 0x559b87111ea0 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.512+0000 7f17601d4b80 10 client.27278 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.512+0000 7f17601d4b80 10 client.27278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.512+0000 7f17601d4b80 20 client.27278 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.512+0000 7f17601d4b80 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559b87126650 0x559b87128a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.512+0000 7f17601d4b80 4 client.27278 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:25:37.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.512+0000 7f17601d4b80 1 -- 172.21.15.33:0/714426741 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b870e0810 con 0x559b87126650 2024-05-08T21:25:37.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.512+0000 7f1751db6700 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559b87126650 0x559b87128a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.513+0000 7f17357fa700 10 client.27278 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:37.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.513+0000 7f1751db6700 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559b87126650 0x559b87128a40 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6176c157f16f646 server_cookie=203c1ad0131c6ba in_seq=0 out_seq=0 2024-05-08T21:25:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.514+0000 7f17357fa700 1 -- 172.21.15.33:0/714426741 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b870e0810 con 0x559b87126650 2024-05-08T21:25:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.514+0000 7f17357fa700 10 client.27278 handle_command_reply: tid=0 2024-05-08T21:25:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.514+0000 7f17601d4b80 1 client.27278 shutdown 2024-05-08T21:25:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.514+0000 7f17601d4b80 1 -- 172.21.15.33:0/714426741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 msgr2=0x559b8711b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.514+0000 7f17601d4b80 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b8711b2b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f173c009810 tx=0x7f173c00b720 comp rx=0 tx=0).stop 2024-05-08T21:25:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.515+0000 7f17601d4b80 1 -- 172.21.15.33:0/714426741 shutdown_connections 2024-05-08T21:25:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.515+0000 7f17601d4b80 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559b87126650 0x559b87128a40 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.515+0000 7f17601d4b80 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87112860 0x559b871222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.515+0000 7f17601d4b80 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111540 0x559b8711ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.515+0000 7f17601d4b80 1 --2- 172.21.15.33:0/714426741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b8711b2b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.515+0000 7f17601d4b80 1 -- 172.21.15.33:0/714426741 >> 172.21.15.33:0/714426741 conn(0x559b8706f210 msgr2=0x559b8710b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.515+0000 7f17357fa700 0 client.27278 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.515+0000 7f17601d4b80 1 -- 172.21.15.33:0/714426741 shutdown_connections 2024-05-08T21:25:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.515+0000 7f17601d4b80 1 -- 172.21.15.33:0/714426741 wait complete. 2024-05-08T21:25:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.515+0000 7f17601d4b80 20 client.27278 trim_cache size 0 max 16384 2024-05-08T21:25:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.537+0000 7f17601d4b80 1 Processor -- start 2024-05-08T21:25:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.537+0000 7f17601d4b80 1 -- start start 2024-05-08T21:25:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f17601d4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87111540 0x559b8711a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f17601d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111ea0 0x559b871222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f17601d4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87112860 0x559b8712d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f17601d4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b87108dc0 con 0x559b87111540 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f17601d4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b8710d570 con 0x559b87112860 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f17601d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b87126440 con 0x559b87111ea0 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f1751db6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87111540 0x559b8711a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f17525b7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87112860 0x559b8712d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f17515b5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111ea0 0x559b871222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f1751db6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87111540 0x559b8711a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45060/0 (socket says 172.21.15.33:45060) 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f1751db6700 1 -- 172.21.15.33:0/658937994 learned_addr learned my addr 172.21.15.33:0/658937994 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f1751db6700 1 -- 172.21.15.33:0/658937994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111ea0 msgr2=0x559b871222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f1751db6700 1 --2- 172.21.15.33:0/658937994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111ea0 0x559b871222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f1751db6700 1 -- 172.21.15.33:0/658937994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87112860 msgr2=0x559b8712d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f1751db6700 1 --2- 172.21.15.33:0/658937994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87112860 0x559b8712d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f1751db6700 1 -- 172.21.15.33:0/658937994 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b87027560 con 0x559b87111540 2024-05-08T21:25:37.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.538+0000 7f17525b7700 1 --2- 172.21.15.33:0/658937994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87112860 0x559b8712d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:25:37.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.539+0000 7f1751db6700 1 --2- 172.21.15.33:0/658937994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87111540 0x559b8711a280 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1748012580 tx=0x7f1748014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929a9648fb992e06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:37.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.539+0000 7f1740ff9700 1 -- 172.21.15.33:0/658937994 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748026070 con 0x559b87111540 2024-05-08T21:25:37.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.539+0000 7f1740ff9700 1 -- 172.21.15.33:0/658937994 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1748016070 con 0x559b87111540 2024-05-08T21:25:37.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.539+0000 7f1740ff9700 1 -- 172.21.15.33:0/658937994 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17480267d0 con 0x559b87111540 2024-05-08T21:25:37.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.540+0000 7f17601d4b80 1 -- 172.21.15.33:0/658937994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87111540 msgr2=0x559b8711a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.540+0000 7f17601d4b80 1 --2- 172.21.15.33:0/658937994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87111540 0x559b8711a280 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1748012580 tx=0x7f1748014000 comp rx=0 tx=0).stop 2024-05-08T21:25:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.540+0000 7f17601d4b80 1 -- 172.21.15.33:0/658937994 shutdown_connections 2024-05-08T21:25:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.540+0000 7f17601d4b80 1 --2- 172.21.15.33:0/658937994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87112860 0x559b8712d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.540+0000 7f17601d4b80 1 --2- 172.21.15.33:0/658937994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87111540 0x559b8711a280 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.540+0000 7f17601d4b80 1 --2- 172.21.15.33:0/658937994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87111ea0 0x559b871222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.540+0000 7f17601d4b80 1 -- 172.21.15.33:0/658937994 >> 172.21.15.33:0/658937994 conn(0x559b8706f210 msgr2=0x559b87112c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.540+0000 7f17601d4b80 1 -- 172.21.15.33:0/658937994 shutdown_connections 2024-05-08T21:25:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.540+0000 7f17601d4b80 1 -- 172.21.15.33:0/658937994 wait complete. 2024-05-08T21:25:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.540+0000 7f17601d4b80 1 -- start start 2024-05-08T21:25:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.540+0000 7f17601d4b80 1 Processor -- start 2024-05-08T21:25:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.541+0000 7f17601d4b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87111540 0x559b8711d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.541+0000 7f17601d4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b8711d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.541+0000 7f17601d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87112860 0x559b871208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.541+0000 7f17601d4b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b87120e20 con 0x559b87111540 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.541+0000 7f17601d4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b87120f90 con 0x559b87111ea0 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.541+0000 7f17601d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b87121100 con 0x559b87112860 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.541+0000 7f17601d4b80 10 client.0 _ll_register_callbacks cb 0x559b87006b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.541+0000 7f17515b5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b8711d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.541+0000 7f17515b5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b8711d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56398/0 (socket says 172.21.15.33:56398) 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.541+0000 7f17515b5700 1 -- 172.21.15.33:0/3464904346 learned_addr learned my addr 172.21.15.33:0/3464904346 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.541+0000 7f17525b7700 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87112860 0x559b871208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.541+0000 7f1751db6700 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87111540 0x559b8711d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17515b5700 1 -- 172.21.15.33:0/3464904346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87112860 msgr2=0x559b871208e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17515b5700 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87112860 0x559b871208e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17515b5700 1 -- 172.21.15.33:0/3464904346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87111540 msgr2=0x559b8711d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:25:37.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17515b5700 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87111540 0x559b8711d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17515b5700 1 -- 172.21.15.33:0/3464904346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c1a5760 con 0x559b87111ea0 2024-05-08T21:25:37.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17515b5700 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b8711d960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f173c00b6f0 tx=0x7f173c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ba25bcaf1e8bddc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:25:37.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17357fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:25:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17357fa700 1 -- 172.21.15.33:0/3464904346 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c0058e0 con 0x559b87111ea0 2024-05-08T21:25:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17357fa700 1 -- 172.21.15.33:0/3464904346 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f173c00f460 con 0x559b87111ea0 2024-05-08T21:25:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17601d4b80 1 -- 172.21.15.33:0/3464904346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f173c01d040 con 0x559b87111ea0 2024-05-08T21:25:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17357fa700 1 -- 172.21.15.33:0/3464904346 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f173c00f620 con 0x559b87111ea0 2024-05-08T21:25:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17601d4b80 10 client.27284 fetch_fsmap learned FSMap version 39 2024-05-08T21:25:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.542+0000 7f17601d4b80 1 -- 172.21.15.33:0/3464904346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559b86ee6cd0 con 0x559b87111ea0 2024-05-08T21:25:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.544+0000 7f17357fa700 1 -- 172.21.15.33:0/3464904346 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f173c01d040 con 0x559b87111ea0 2024-05-08T21:25:37.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.544+0000 7f17357fa700 1 -- 172.21.15.33:0/3464904346 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f173c009920 con 0x559b87111ea0 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.544+0000 7f17601d4b80 10 client.27284 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.544+0000 7f17601d4b80 10 client.27284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.544+0000 7f17601d4b80 20 client.27284 populate_metadata read hostname 'smithi033' 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.544+0000 7f17601d4b80 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559b8718dab0 0x559b8718fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.544+0000 7f17601d4b80 4 client.27284 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.544+0000 7f17601d4b80 1 -- 172.21.15.33:0/3464904346 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559b87059ec0 con 0x559b8718dab0 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.544+0000 7f1751db6700 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559b8718dab0 0x559b8718fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.545+0000 7f17357fa700 10 client.27284 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.545+0000 7f1751db6700 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559b8718dab0 0x559b8718fea0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=317a9c8c8b8ddc2a server_cookie=1ea3a7d19a2f59c9 in_seq=0 out_seq=0 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.545+0000 7f17357fa700 1 -- 172.21.15.33:0/3464904346 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559b87059ec0 con 0x559b8718dab0 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.545+0000 7f17357fa700 10 client.27284 handle_command_reply: tid=0 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.545+0000 7f17601d4b80 1 client.27284 shutdown 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.545+0000 7f17601d4b80 1 -- 172.21.15.33:0/3464904346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 msgr2=0x559b8711d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.545+0000 7f17601d4b80 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b8711d960 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f173c00b6f0 tx=0x7f173c00b790 comp rx=0 tx=0).stop 2024-05-08T21:25:37.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.546+0000 7f17601d4b80 1 -- 172.21.15.33:0/3464904346 shutdown_connections 2024-05-08T21:25:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.546+0000 7f17601d4b80 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559b8718dab0 0x559b8718fea0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.546+0000 7f17601d4b80 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b87112860 0x559b871208e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.546+0000 7f17601d4b80 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b87111540 0x559b8711d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.546+0000 7f17357fa700 0 client.27284 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:25:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.546+0000 7f17601d4b80 1 --2- 172.21.15.33:0/3464904346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b87111ea0 0x559b8711d960 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.546+0000 7f17601d4b80 1 -- 172.21.15.33:0/3464904346 >> 172.21.15.33:0/3464904346 conn(0x559b8706f210 msgr2=0x559b8712d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.546+0000 7f17601d4b80 1 -- 172.21.15.33:0/3464904346 shutdown_connections 2024-05-08T21:25:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.546+0000 7f17601d4b80 1 -- 172.21.15.33:0/3464904346 wait complete. 2024-05-08T21:25:37.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.546+0000 7f17601d4b80 20 client.27284 trim_cache size 0 max 16384 2024-05-08T21:25:37.550 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:25:37.551 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:25:37.551 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:25:37.551 INFO:teuthology.orchestra.run.smithi033.stdout: "ea0c18f1-d682-4321-bf52-114825a506c3": { 2024-05-08T21:25:37.551 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:25:37.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "ea0c18f1-d682-4321-bf52-114825a506c3", 2024-05-08T21:25:37.551 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:25:37.551 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:25:37.551 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:25:37.551 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:25:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.547+0000 7f175401a700 1 -- 172.21.15.33:0/1985068811 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17380a1220 msgr2=0x7f17380a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.547+0000 7f175401a700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17380a1220 0x7f17380a36e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1748005fd0 tx=0x7f1748009310 comp rx=0 tx=0).stop 2024-05-08T21:25:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.547+0000 7f175401a700 1 -- 172.21.15.33:0/1985068811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f174c104a10 msgr2=0x7f174c1a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:25:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.547+0000 7f175401a700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f174c104a10 0x7f174c1a4910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f174400c370 tx=0x7f174400c680 comp rx=0 tx=0).stop 2024-05-08T21:25:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.547+0000 7f175401a700 1 -- 172.21.15.33:0/1985068811 shutdown_connections 2024-05-08T21:25:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.547+0000 7f175401a700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f174c104a10 0x7f174c1a4910 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.547+0000 7f175401a700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f17380a1220 0x7f17380a36e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.547+0000 7f175401a700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f174c0ff6d0 0x7f174c19d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.547+0000 7f175401a700 1 --2- 172.21.15.33:0/1985068811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f174c102040 0x7f174c19d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:25:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.547+0000 7f175401a700 1 -- 172.21.15.33:0/1985068811 >> 172.21.15.33:0/1985068811 conn(0x7f174c0fb290 msgr2=0x7f174c101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:25:37.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.547+0000 7f175401a700 1 -- 172.21.15.33:0/1985068811 shutdown_connections 2024-05-08T21:25:37.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:25:37.547+0000 7f175401a700 1 -- 172.21.15.33:0/1985068811 wait complete. 2024-05-08T21:25:37.563 INFO:tasks.cephfs.filesystem:scrub status for tag:ea0c18f1-d682-4321-bf52-114825a506c3 - {'path': '/', 'tag': 'ea0c18f1-d682-4321-bf52-114825a506c3', 'options': 'recursive,force'} 2024-05-08T21:25:37.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:37 smithi071 ceph-mon[47074]: pgmap v489: 129 pgs: 129 active+clean; 224 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.4 MiB/s wr, 534 op/s 2024-05-08T21:25:37.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:37 smithi082 ceph-mon[47512]: pgmap v489: 129 pgs: 129 active+clean; 224 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.4 MiB/s wr, 534 op/s 2024-05-08T21:25:38.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:38 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:38 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:25:38.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:38.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:38.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:38.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:38.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:38.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:38.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:25:38.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:38.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:38 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:39.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:39 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:39.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:39 smithi033 ceph-mon[42528]: pgmap v490: 129 pgs: 129 active+clean; 224 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 2.7 MiB/s wr, 431 op/s 2024-05-08T21:25:39.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:39 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:39.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:39 smithi071 ceph-mon[47074]: pgmap v490: 129 pgs: 129 active+clean; 224 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 2.7 MiB/s wr, 431 op/s 2024-05-08T21:25:39.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:39 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:25:39.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:39 smithi082 ceph-mon[47512]: pgmap v490: 129 pgs: 129 active+clean; 224 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 2.7 MiB/s wr, 431 op/s 2024-05-08T21:25:41.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:41 smithi033 ceph-mon[42528]: pgmap v491: 129 pgs: 129 active+clean; 227 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.9 MiB/s wr, 426 op/s 2024-05-08T21:25:41.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:41 smithi071 ceph-mon[47074]: pgmap v491: 129 pgs: 129 active+clean; 227 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.9 MiB/s wr, 426 op/s 2024-05-08T21:25:41.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:41 smithi082 ceph-mon[47512]: pgmap v491: 129 pgs: 129 active+clean; 227 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.9 MiB/s wr, 426 op/s 2024-05-08T21:25:43.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:43 smithi033 ceph-mon[42528]: pgmap v492: 129 pgs: 129 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.2 MiB/s wr, 486 op/s 2024-05-08T21:25:43.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:43 smithi071 ceph-mon[47074]: pgmap v492: 129 pgs: 129 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.2 MiB/s wr, 486 op/s 2024-05-08T21:25:43.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:43 smithi082 ceph-mon[47512]: pgmap v492: 129 pgs: 129 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 3.2 MiB/s wr, 486 op/s 2024-05-08T21:25:45.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:45 smithi033 ceph-mon[42528]: pgmap v493: 129 pgs: 129 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.5 MiB/s wr, 365 op/s 2024-05-08T21:25:45.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:45 smithi071 ceph-mon[47074]: pgmap v493: 129 pgs: 129 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.5 MiB/s wr, 365 op/s 2024-05-08T21:25:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:45 smithi082 ceph-mon[47512]: pgmap v493: 129 pgs: 129 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.5 MiB/s wr, 365 op/s 2024-05-08T21:25:47.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:47 smithi033 ceph-mon[42528]: pgmap v494: 129 pgs: 129 active+clean; 229 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.5 MiB/s wr, 523 op/s 2024-05-08T21:25:47.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:47 smithi071 ceph-mon[47074]: pgmap v494: 129 pgs: 129 active+clean; 229 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.5 MiB/s wr, 523 op/s 2024-05-08T21:25:47.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:47 smithi082 ceph-mon[47512]: pgmap v494: 129 pgs: 129 active+clean; 229 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.5 MiB/s wr, 523 op/s 2024-05-08T21:25:48.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:48 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:25:48.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:48 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:25:48.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:48 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:25:48.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:48 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:25:48.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:48 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:25:48.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:48 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:25:48.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:48 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:25:48.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:48 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:25:48.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:48 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:25:48.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:48 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:25:48.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:48 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:25:48.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:48 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:25:48.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:48 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:25:48.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:48 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:25:48.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:48 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:25:49.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:49 smithi071 ceph-mon[47074]: pgmap v495: 129 pgs: 129 active+clean; 228 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 380 op/s 2024-05-08T21:25:49.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:49 smithi082 ceph-mon[47512]: pgmap v495: 129 pgs: 129 active+clean; 228 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 380 op/s 2024-05-08T21:25:49.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:49 smithi033 ceph-mon[42528]: pgmap v495: 129 pgs: 129 active+clean; 228 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.7 MiB/s wr, 380 op/s 2024-05-08T21:25:51.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:51 smithi071 ceph-mon[47074]: pgmap v496: 129 pgs: 129 active+clean; 227 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 2.9 MiB/s wr, 418 op/s 2024-05-08T21:25:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:51 smithi082 ceph-mon[47512]: pgmap v496: 129 pgs: 129 active+clean; 227 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 2.9 MiB/s wr, 418 op/s 2024-05-08T21:25:51.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:51 smithi033 ceph-mon[42528]: pgmap v496: 129 pgs: 129 active+clean; 227 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 2.9 MiB/s wr, 418 op/s 2024-05-08T21:25:53.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:53 smithi071 ceph-mon[47074]: pgmap v497: 129 pgs: 129 active+clean; 228 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.2 MiB/s wr, 480 op/s 2024-05-08T21:25:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:53 smithi082 ceph-mon[47512]: pgmap v497: 129 pgs: 129 active+clean; 228 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.2 MiB/s wr, 480 op/s 2024-05-08T21:25:53.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:53 smithi033 ceph-mon[42528]: pgmap v497: 129 pgs: 129 active+clean; 228 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.2 MiB/s wr, 480 op/s 2024-05-08T21:25:55.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:55 smithi071 ceph-mon[47074]: pgmap v498: 129 pgs: 129 active+clean; 228 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.5 MiB/s wr, 368 op/s 2024-05-08T21:25:55.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:55 smithi082 ceph-mon[47512]: pgmap v498: 129 pgs: 129 active+clean; 228 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.5 MiB/s wr, 368 op/s 2024-05-08T21:25:55.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:55 smithi033 ceph-mon[42528]: pgmap v498: 129 pgs: 129 active+clean; 228 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 2.5 MiB/s wr, 368 op/s 2024-05-08T21:25:57.446 INFO:tasks.workunit.client.0.smithi033.stdout:50000 mapread 0x648404 thru 0x648abe (0x6bb bytes) 2024-05-08T21:25:57.448 INFO:tasks.workunit.client.0.smithi033.stdout:All 50000 operations completed A-OK! 2024-05-08T21:25:57.448 INFO:tasks.workunit.client.0.smithi033.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-05-08T21:25:57.449 INFO:tasks.workunit.client.0.smithi033.stdout:Seed set to 1 2024-05-08T21:25:57.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:57 smithi071 ceph-mon[47074]: pgmap v499: 129 pgs: 129 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.4 MiB/s wr, 503 op/s 2024-05-08T21:25:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:57 smithi082 ceph-mon[47512]: pgmap v499: 129 pgs: 129 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.4 MiB/s wr, 503 op/s 2024-05-08T21:25:57.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:57 smithi033 ceph-mon[42528]: pgmap v499: 129 pgs: 129 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.4 MiB/s wr, 503 op/s 2024-05-08T21:25:58.685 INFO:tasks.workunit.client.0.smithi033.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-08T21:25:58.685 INFO:tasks.workunit.client.0.smithi033.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-08T21:25:58.686 INFO:tasks.workunit.client.0.smithi033.stderr:main: filesystem does not support clone range, disabling! 2024-05-08T21:25:58.686 INFO:tasks.workunit.client.0.smithi033.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-08T21:25:58.691 INFO:tasks.workunit.client.0.smithi033.stdout:skipping zero size read 2024-05-08T21:25:58.692 INFO:tasks.workunit.client.0.smithi033.stdout:truncating to largest ever: 0x7e0023 2024-05-08T21:25:58.694 INFO:tasks.workunit.client.0.smithi033.stdout:fallocating to largest ever: 0x16e26fa 2024-05-08T21:25:58.699 INFO:tasks.workunit.client.0.smithi033.stdout:copying to largest ever: 0x51314a1 2024-05-08T21:25:58.727 INFO:tasks.workunit.client.0.smithi033.stdout:copying to largest ever: 0x529f232 2024-05-08T21:25:58.858 INFO:tasks.workunit.client.0.smithi033.stdout:copying to largest ever: 0x53b012c 2024-05-08T21:25:58.899 INFO:tasks.workunit.client.0.smithi033.stdout:fallocating to largest ever: 0x548ef5b 2024-05-08T21:25:58.919 INFO:tasks.workunit.client.0.smithi033.stdout:copying to largest ever: 0x623914d 2024-05-08T21:25:59.030 INFO:tasks.workunit.client.0.smithi033.stdout:copying to largest ever: 0x628f369 2024-05-08T21:25:59.110 INFO:tasks.workunit.client.0.smithi033.stdout:copying to largest ever: 0x6328dee 2024-05-08T21:25:59.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:25:59 smithi071 ceph-mon[47074]: pgmap v500: 129 pgs: 129 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.6 MiB/s wr, 367 op/s 2024-05-08T21:25:59.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:25:59 smithi082 ceph-mon[47512]: pgmap v500: 129 pgs: 129 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.6 MiB/s wr, 367 op/s 2024-05-08T21:25:59.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:25:59 smithi033 ceph-mon[42528]: pgmap v500: 129 pgs: 129 active+clean; 231 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.6 MiB/s wr, 367 op/s 2024-05-08T21:26:00.057 INFO:tasks.workunit.client.0.smithi033.stdout:truncating to largest ever: 0x63b535e 2024-05-08T21:26:01.792 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:01 smithi082 ceph-mon[47512]: pgmap v501: 129 pgs: 129 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.8 MiB/s wr, 418 op/s 2024-05-08T21:26:01.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:01 smithi071 ceph-mon[47074]: pgmap v501: 129 pgs: 129 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.8 MiB/s wr, 418 op/s 2024-05-08T21:26:01.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:01 smithi033 ceph-mon[42528]: pgmap v501: 129 pgs: 129 active+clean; 235 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.8 MiB/s wr, 418 op/s 2024-05-08T21:26:03.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:03 smithi071 ceph-mon[47074]: pgmap v502: 129 pgs: 129 active+clean; 237 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.6 MiB/s wr, 398 op/s 2024-05-08T21:26:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:03 smithi082 ceph-mon[47512]: pgmap v502: 129 pgs: 129 active+clean; 237 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.6 MiB/s wr, 398 op/s 2024-05-08T21:26:03.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:03 smithi033 ceph-mon[42528]: pgmap v502: 129 pgs: 129 active+clean; 237 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.6 MiB/s wr, 398 op/s 2024-05-08T21:26:05.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:05 smithi071 ceph-mon[47074]: pgmap v503: 129 pgs: 129 active+clean; 245 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.6 MiB/s wr, 468 op/s 2024-05-08T21:26:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:05 smithi082 ceph-mon[47512]: pgmap v503: 129 pgs: 129 active+clean; 245 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.6 MiB/s wr, 468 op/s 2024-05-08T21:26:05.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:05 smithi033 ceph-mon[42528]: pgmap v503: 129 pgs: 129 active+clean; 245 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.6 MiB/s wr, 468 op/s 2024-05-08T21:26:07.565 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:26:07.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.743+0000 7fcfca667700 1 -- 172.21.15.33:0/2100141856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc41018e0 msgr2=0x7fcfc4103cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:07.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.743+0000 7fcfca667700 1 --2- 172.21.15.33:0/2100141856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc41018e0 0x7fcfc4103cd0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009990 tx=0x7fcfb4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.743+0000 7fcfca667700 1 -- 172.21.15.33:0/2100141856 shutdown_connections 2024-05-08T21:26:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.743+0000 7fcfca667700 1 --2- 172.21.15.33:0/2100141856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc4106b40 0x7fcfc4108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.743+0000 7fcfca667700 1 --2- 172.21.15.33:0/2100141856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc41018e0 0x7fcfc4103cd0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.743+0000 7fcfca667700 1 --2- 172.21.15.33:0/2100141856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc4104210 0x7fcfc4106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.743+0000 7fcfca667700 1 -- 172.21.15.33:0/2100141856 >> 172.21.15.33:0/2100141856 conn(0x7fcfc40fb290 msgr2=0x7fcfc40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.743+0000 7fcfca667700 1 -- 172.21.15.33:0/2100141856 shutdown_connections 2024-05-08T21:26:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.743+0000 7fcfca667700 1 -- 172.21.15.33:0/2100141856 wait complete. 2024-05-08T21:26:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfca667700 1 Processor -- start 2024-05-08T21:26:07.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfca667700 1 -- start start 2024-05-08T21:26:07.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfca667700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc41018e0 0x7fcfc4196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfca667700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc4104210 0x7fcfc4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfca667700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc4106b40 0x7fcfc419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfca667700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc419e7e0 con 0x7fcfc41018e0 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfca667700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc419e950 con 0x7fcfc4106b40 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfca667700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc419eb10 con 0x7fcfc4104210 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfc3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc41018e0 0x7fcfc4196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfc8c04700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc4106b40 0x7fcfc419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfc3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc41018e0 0x7fcfc4196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52786/0 (socket says 172.21.15.33:52786) 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfc3fff700 1 -- 172.21.15.33:0/482159787 learned_addr learned my addr 172.21.15.33:0/482159787 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.744+0000 7fcfc37fe700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc4104210 0x7fcfc4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.745+0000 7fcfc8c04700 1 -- 172.21.15.33:0/482159787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc4104210 msgr2=0x7fcfc4197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.745+0000 7fcfc8c04700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc4104210 0x7fcfc4197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.745+0000 7fcfc8c04700 1 -- 172.21.15.33:0/482159787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc41018e0 msgr2=0x7fcfc4196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:07.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.745+0000 7fcfc8c04700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc41018e0 0x7fcfc4196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.745+0000 7fcfc8c04700 1 -- 172.21.15.33:0/482159787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40095f0 con 0x7fcfc4106b40 2024-05-08T21:26:07.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.745+0000 7fcfc3fff700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc41018e0 0x7fcfc4196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:07.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.745+0000 7fcfc8c04700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc4106b40 0x7fcfc419e210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00efc0 tx=0x7fcfbc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:07.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.745+0000 7fcfc17fa700 1 -- 172.21.15.33:0/482159787 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc018070 con 0x7fcfc4106b40 2024-05-08T21:26:07.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.745+0000 7fcfca667700 1 -- 172.21.15.33:0/482159787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc419ed90 con 0x7fcfc4106b40 2024-05-08T21:26:07.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.745+0000 7fcfc17fa700 1 -- 172.21.15.33:0/482159787 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfbc00f040 con 0x7fcfc4106b40 2024-05-08T21:26:07.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.745+0000 7fcfc17fa700 1 -- 172.21.15.33:0/482159787 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc00cba0 con 0x7fcfc4106b40 2024-05-08T21:26:07.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.745+0000 7fcfca667700 1 -- 172.21.15.33:0/482159787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc419f1f0 con 0x7fcfc4106b40 2024-05-08T21:26:07.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.748+0000 7fcfc17fa700 1 -- 172.21.15.33:0/482159787 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcfbc004000 con 0x7fcfc4106b40 2024-05-08T21:26:07.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.749+0000 7fcfc17fa700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcfac0a1120 0x7fcfac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.750+0000 7fcfc17fa700 1 -- 172.21.15.33:0/482159787 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcfbc0cc7a0 con 0x7fcfc4106b40 2024-05-08T21:26:07.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.750+0000 7fcfc3fff700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcfac0a1120 0x7fcfac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfc3fff700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcfac0a1120 0x7fcfac0a35e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400b2e0 tx=0x7fcfb400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:07.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfd6821b80 1 Processor -- start 2024-05-08T21:26:07.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfd6821b80 1 -- start start 2024-05-08T21:26:07.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfd6821b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfd6821b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d72b0 0x559d594d7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfd6821b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d7c70 0x559d594d80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfd6821b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d59429150 con 0x559d594d7c70 2024-05-08T21:26:07.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfd6821b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d59426b10 con 0x559d594d72b0 2024-05-08T21:26:07.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfd6821b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d59422400 con 0x559d594d6950 2024-05-08T21:26:07.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfc8c04700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d7c70 0x559d594d80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfc37fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d72b0 0x559d594d7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfc8c04700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d7c70 0x559d594d80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52788/0 (socket says 172.21.15.33:52788) 2024-05-08T21:26:07.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfc3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.751+0000 7fcfc8c04700 1 -- 172.21.15.33:0/3133474489 learned_addr learned my addr 172.21.15.33:0/3133474489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:07.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.752+0000 7fcfc37fe700 1 -- 172.21.15.33:0/3133474489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 msgr2=0x559d594d6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:07.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.752+0000 7fcfc37fe700 1 --2- 172.21.15.33:0/3133474489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.752+0000 7fcfc37fe700 1 -- 172.21.15.33:0/3133474489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d7c70 msgr2=0x559d594d80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:07.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.752+0000 7fcfc37fe700 1 --2- 172.21.15.33:0/3133474489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d7c70 0x559d594d80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.752+0000 7fcfc37fe700 1 -- 172.21.15.33:0/3133474489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d592a33b0 con 0x559d594d72b0 2024-05-08T21:26:07.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.752+0000 7fcfc3fff700 1 --2- 172.21.15.33:0/3133474489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:07.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.752+0000 7fcfc37fe700 1 --2- 172.21.15.33:0/3133474489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d72b0 0x559d594d7730 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009cb0 tx=0x7fcfb8011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31cc30b4e1f369b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:07.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.752+0000 7fcfaa7fc700 1 -- 172.21.15.33:0/3133474489 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8011d20 con 0x559d594d72b0 2024-05-08T21:26:07.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.752+0000 7fcfaa7fc700 1 -- 172.21.15.33:0/3133474489 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfb8011e80 con 0x559d594d72b0 2024-05-08T21:26:07.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.752+0000 7fcfaa7fc700 1 -- 172.21.15.33:0/3133474489 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8005280 con 0x559d594d72b0 2024-05-08T21:26:07.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.753+0000 7fcfd6821b80 1 -- 172.21.15.33:0/3133474489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d72b0 msgr2=0x559d594d7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:07.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.753+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/3133474489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d72b0 0x559d594d7730 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009cb0 tx=0x7fcfb8011440 comp rx=0 tx=0).stop 2024-05-08T21:26:07.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.753+0000 7fcfd6821b80 1 -- 172.21.15.33:0/3133474489 shutdown_connections 2024-05-08T21:26:07.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.753+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/3133474489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d7c70 0x559d594d80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.753+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/3133474489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.753+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/3133474489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d72b0 0x559d594d7730 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.753+0000 7fcfd6821b80 1 -- 172.21.15.33:0/3133474489 >> 172.21.15.33:0/3133474489 conn(0x559d59430210 msgr2=0x559d594ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.753+0000 7fcfd6821b80 1 -- 172.21.15.33:0/3133474489 shutdown_connections 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.753+0000 7fcfd6821b80 1 -- 172.21.15.33:0/3133474489 wait complete. 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.754+0000 7fcfd6821b80 1 -- start start 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.754+0000 7fcfd6821b80 1 Processor -- start 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.754+0000 7fcfd6821b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d6950 0x559d594e0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.754+0000 7fcfd6821b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d72b0 0x559d594e07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.754+0000 7fcfd6821b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d7c70 0x559d594e7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.754+0000 7fcfd6821b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d593be560 con 0x559d594d6950 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.754+0000 7fcfd6821b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d593be200 con 0x559d594d72b0 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.754+0000 7fcfd6821b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d592a56a0 con 0x559d594d7c70 2024-05-08T21:26:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.754+0000 7fcfd6821b80 10 client.0 _ll_register_callbacks cb 0x559d593c7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfc3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d6950 0x559d594e0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfc8c04700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d7c70 0x559d594e7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfc3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d6950 0x559d594e0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52798/0 (socket says 172.21.15.33:52798) 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfc3fff700 1 -- 172.21.15.33:0/1205865423 learned_addr learned my addr 172.21.15.33:0/1205865423 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfc37fe700 1 --2- 172.21.15.33:0/1205865423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d72b0 0x559d594e07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfc8c04700 1 -- 172.21.15.33:0/1205865423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d72b0 msgr2=0x559d594e07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfc8c04700 1 --2- 172.21.15.33:0/1205865423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d72b0 0x559d594e07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfc8c04700 1 -- 172.21.15.33:0/1205865423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d6950 msgr2=0x559d594e0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfc8c04700 1 --2- 172.21.15.33:0/1205865423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d6950 0x559d594e0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfc8c04700 1 -- 172.21.15.33:0/1205865423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc4109ba0 con 0x559d594d7c70 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfc8c04700 1 --2- 172.21.15.33:0/1205865423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d7c70 0x559d594e7800 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0090a0 tx=0x7fcfbc005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c4c44d5b546cd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfa37fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.755+0000 7fcfa37fe700 1 -- 172.21.15.33:0/1205865423 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0b2200 con 0x559d594d7c70 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfa37fe700 1 -- 172.21.15.33:0/1205865423 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcfbc09a000 con 0x559d594d7c70 2024-05-08T21:26:07.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfd6821b80 1 -- 172.21.15.33:0/1205865423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d594a1810 con 0x559d594d7c70 2024-05-08T21:26:07.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfa37fe700 1 -- 172.21.15.33:0/1205865423 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfbc099900 con 0x559d594d7c70 2024-05-08T21:26:07.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfd6821b80 10 client.27289 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:07.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfd6821b80 1 -- 172.21.15.33:0/1205865423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fcfb80098e0 con 0x559d594d7c70 2024-05-08T21:26:07.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfa37fe700 1 -- 172.21.15.33:0/1205865423 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfbc064020 con 0x559d594d7c70 2024-05-08T21:26:07.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfa37fe700 1 -- 172.21.15.33:0/1205865423 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcfbc098df0 con 0x559d594d7c70 2024-05-08T21:26:07.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfd6821b80 10 client.27289 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:07.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfd6821b80 10 client.27289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:07.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfd6821b80 20 client.27289 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:07.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/1205865423 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559d594ebff0 0x559d594ee410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfd6821b80 4 client.27289 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.756+0000 7fcfd6821b80 1 -- 172.21.15.33:0/1205865423 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d5941aec0 con 0x559d594ebff0 2024-05-08T21:26:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.757+0000 7fcfc3fff700 1 --2- 172.21.15.33:0/1205865423 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559d594ebff0 0x559d594ee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.757+0000 7fcfc3fff700 1 --2- 172.21.15.33:0/1205865423 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559d594ebff0 0x559d594ee410 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e3879640273a055 server_cookie=ac1ae4a102c73744 in_seq=0 out_seq=0 2024-05-08T21:26:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.757+0000 7fcfa37fe700 10 client.27289 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfa37fe700 1 -- 172.21.15.33:0/1205865423 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559d5941aec0 con 0x559d594ebff0 2024-05-08T21:26:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfa37fe700 10 client.27289 handle_command_reply: tid=0 2024-05-08T21:26:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfd6821b80 1 client.27289 shutdown 2024-05-08T21:26:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfd6821b80 1 -- 172.21.15.33:0/1205865423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d7c70 msgr2=0x559d594e7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/1205865423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d7c70 0x559d594e7800 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0090a0 tx=0x7fcfbc005b20 comp rx=0 tx=0).stop 2024-05-08T21:26:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfd6821b80 1 -- 172.21.15.33:0/1205865423 shutdown_connections 2024-05-08T21:26:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/1205865423 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559d594ebff0 0x559d594ee410 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/1205865423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d7c70 0x559d594e7800 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfa37fe700 0 client.27289 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/1205865423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d6950 0x559d594e0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/1205865423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d72b0 0x559d594e07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfd6821b80 1 -- 172.21.15.33:0/1205865423 >> 172.21.15.33:0/1205865423 conn(0x559d59430210 msgr2=0x559d594cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfd6821b80 1 -- 172.21.15.33:0/1205865423 shutdown_connections 2024-05-08T21:26:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfd6821b80 1 -- 172.21.15.33:0/1205865423 wait complete. 2024-05-08T21:26:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.758+0000 7fcfd6821b80 20 client.27289 trim_cache size 0 max 16384 2024-05-08T21:26:07.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.783+0000 7fcfd6821b80 1 Processor -- start 2024-05-08T21:26:07.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.783+0000 7fcfd6821b80 1 -- start start 2024-05-08T21:26:07.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.783+0000 7fcfd6821b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.783+0000 7fcfd6821b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d72b0 0x559d594e7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.783+0000 7fcfd6821b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 0x559d594e77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.783+0000 7fcfd6821b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d594f3600 con 0x559d594d72b0 2024-05-08T21:26:07.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.783+0000 7fcfd6821b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d594db790 con 0x559d594d7c70 2024-05-08T21:26:07.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.783+0000 7fcfd6821b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d594e0030 con 0x559d594d6950 2024-05-08T21:26:07.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.784+0000 7fcfc8c04700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 0x559d594e77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.784+0000 7fcfc8c04700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 0x559d594e77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:58480/0 (socket says 172.21.15.33:58480) 2024-05-08T21:26:07.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.784+0000 7fcfc8c04700 1 -- 172.21.15.33:0/4205963520 learned_addr learned my addr 172.21.15.33:0/4205963520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:07.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.784+0000 7fcfc37fe700 1 --2- 172.21.15.33:0/4205963520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d72b0 0x559d594e7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.784+0000 7fcfc3fff700 1 --2- 172.21.15.33:0/4205963520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.784+0000 7fcfc8c04700 1 -- 172.21.15.33:0/4205963520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 msgr2=0x559d594d6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:07.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.784+0000 7fcfc8c04700 1 --2- 172.21.15.33:0/4205963520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.784+0000 7fcfc8c04700 1 -- 172.21.15.33:0/4205963520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d72b0 msgr2=0x559d594e7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:07.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.784+0000 7fcfc8c04700 1 --2- 172.21.15.33:0/4205963520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d72b0 0x559d594e7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.784+0000 7fcfc8c04700 1 -- 172.21.15.33:0/4205963520 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d5924fb60 con 0x559d594d7c70 2024-05-08T21:26:07.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.784+0000 7fcfc3fff700 1 --2- 172.21.15.33:0/4205963520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:26:07.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.785+0000 7fcfc8c04700 1 --2- 172.21.15.33:0/4205963520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 0x559d594e77c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc099600 tx=0x7fcfbc0a1860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca5f84bce08c75b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:07.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.786+0000 7fcfaaffd700 1 -- 172.21.15.33:0/4205963520 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0b2200 con 0x559d594d7c70 2024-05-08T21:26:07.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.786+0000 7fcfaaffd700 1 -- 172.21.15.33:0/4205963520 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfbc064070 con 0x559d594d7c70 2024-05-08T21:26:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.786+0000 7fcfaaffd700 1 -- 172.21.15.33:0/4205963520 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0a1980 con 0x559d594d7c70 2024-05-08T21:26:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.788+0000 7fcfd6821b80 1 -- 172.21.15.33:0/4205963520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 msgr2=0x559d594e77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.788+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/4205963520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 0x559d594e77c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc099600 tx=0x7fcfbc0a1860 comp rx=0 tx=0).stop 2024-05-08T21:26:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.788+0000 7fcfd6821b80 1 -- 172.21.15.33:0/4205963520 shutdown_connections 2024-05-08T21:26:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.788+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/4205963520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 0x559d594e77c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.788+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/4205963520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.788+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/4205963520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d72b0 0x559d594e7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.788+0000 7fcfd6821b80 1 -- 172.21.15.33:0/4205963520 >> 172.21.15.33:0/4205963520 conn(0x559d59430210 msgr2=0x559d594d8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:07.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.788+0000 7fcfd6821b80 1 -- 172.21.15.33:0/4205963520 shutdown_connections 2024-05-08T21:26:07.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.788+0000 7fcfd6821b80 1 -- 172.21.15.33:0/4205963520 wait complete. 2024-05-08T21:26:07.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.788+0000 7fcfd6821b80 1 -- start start 2024-05-08T21:26:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.789+0000 7fcfd6821b80 1 Processor -- start 2024-05-08T21:26:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.789+0000 7fcfd6821b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.789+0000 7fcfd6821b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d72b0 0x559d594d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.789+0000 7fcfd6821b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 0x559d594d50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.789+0000 7fcfd6821b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d594d5610 con 0x559d594d72b0 2024-05-08T21:26:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.789+0000 7fcfd6821b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d594d5780 con 0x559d594d7c70 2024-05-08T21:26:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.789+0000 7fcfd6821b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d594d5940 con 0x559d594d6950 2024-05-08T21:26:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.789+0000 7fcfd6821b80 10 client.0 _ll_register_callbacks cb 0x559d593c7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:07.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d72b0 0x559d594d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d72b0 0x559d594d20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52814/0 (socket says 172.21.15.33:52814) 2024-05-08T21:26:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc37fe700 1 -- 172.21.15.33:0/866166309 learned_addr learned my addr 172.21.15.33:0/866166309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc3fff700 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc8c04700 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 0x559d594d50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc37fe700 1 -- 172.21.15.33:0/866166309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 msgr2=0x559d594d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc37fe700 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc37fe700 1 -- 172.21.15.33:0/866166309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 msgr2=0x559d594d50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc37fe700 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 0x559d594d50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc37fe700 1 -- 172.21.15.33:0/866166309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb80098e0 con 0x559d594d72b0 2024-05-08T21:26:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc3fff700 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:26:07.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc8c04700 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 0x559d594d50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:26:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfc37fe700 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d72b0 0x559d594d20b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009cb0 tx=0x7fcfb8025cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f7f65db0c26d22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.790+0000 7fcfa37fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:26:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfa37fe700 1 -- 172.21.15.33:0/866166309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb802a070 con 0x559d594d72b0 2024-05-08T21:26:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfa37fe700 1 -- 172.21.15.33:0/866166309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcfb8005320 con 0x559d594d72b0 2024-05-08T21:26:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfd6821b80 1 -- 172.21.15.33:0/866166309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d5937b270 con 0x559d594d72b0 2024-05-08T21:26:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfa37fe700 1 -- 172.21.15.33:0/866166309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfb80054e0 con 0x559d594d72b0 2024-05-08T21:26:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfd6821b80 10 client.17388 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfd6821b80 1 -- 172.21.15.33:0/866166309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559d593e8560 con 0x559d594d72b0 2024-05-08T21:26:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfa37fe700 1 -- 172.21.15.33:0/866166309 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559d5937b270 con 0x559d594d72b0 2024-05-08T21:26:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfa37fe700 1 -- 172.21.15.33:0/866166309 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcfb802f4b0 con 0x559d594d72b0 2024-05-08T21:26:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfd6821b80 10 client.17388 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:07.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfd6821b80 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfd6821b80 20 client.17388 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559d594e6220 0x559d594e6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfd6821b80 4 client.17388 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:26:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.791+0000 7fcfd6821b80 1 -- 172.21.15.33:0/866166309 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559d59270cf0 con 0x559d594e6220 2024-05-08T21:26:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.793+0000 7fcfc3fff700 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559d594e6220 0x559d594e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:07.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.794+0000 7fcfa37fe700 10 client.17388 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.794+0000 7fcfc3fff700 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559d594e6220 0x559d594e6630 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0b611181f66407 server_cookie=8868ce300fd08157 in_seq=0 out_seq=0 2024-05-08T21:26:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.795+0000 7fcfa37fe700 1 -- 172.21.15.33:0/866166309 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559d59270cf0 con 0x559d594e6220 2024-05-08T21:26:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.795+0000 7fcfa37fe700 10 client.17388 handle_command_reply: tid=0 2024-05-08T21:26:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.795+0000 7fcfd6821b80 1 client.17388 shutdown 2024-05-08T21:26:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.795+0000 7fcfd6821b80 1 -- 172.21.15.33:0/866166309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d72b0 msgr2=0x559d594d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.795+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d72b0 0x559d594d20b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009cb0 tx=0x7fcfb8025cc0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.795+0000 7fcfd6821b80 1 -- 172.21.15.33:0/866166309 shutdown_connections 2024-05-08T21:26:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.795+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x559d594e6220 0x559d594e6630 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.795+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d594d7c70 0x559d594d50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.795+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559d594d6950 0x559d594d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.795+0000 7fcfa37fe700 0 client.17388 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:07.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.795+0000 7fcfd6821b80 1 --2- 172.21.15.33:0/866166309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d594d72b0 0x559d594d20b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.795+0000 7fcfd6821b80 1 -- 172.21.15.33:0/866166309 >> 172.21.15.33:0/866166309 conn(0x559d59430210 msgr2=0x559d594df6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.797+0000 7fcfd6821b80 1 -- 172.21.15.33:0/866166309 shutdown_connections 2024-05-08T21:26:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.797+0000 7fcfd6821b80 1 -- 172.21.15.33:0/866166309 wait complete. 2024-05-08T21:26:07.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.797+0000 7fcfd6821b80 20 client.17388 trim_cache size 0 max 16384 2024-05-08T21:26:07.800 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:26:07.800 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:26:07.800 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:26:07.800 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:26:07.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.798+0000 7fcfca667700 1 -- 172.21.15.33:0/482159787 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcfac0a1120 msgr2=0x7fcfac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:07.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.798+0000 7fcfca667700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcfac0a1120 0x7fcfac0a35e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400b2e0 tx=0x7fcfb400b610 comp rx=0 tx=0).stop 2024-05-08T21:26:07.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.798+0000 7fcfca667700 1 -- 172.21.15.33:0/482159787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc4106b40 msgr2=0x7fcfc419e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:07.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.798+0000 7fcfca667700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc4106b40 0x7fcfc419e210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00efc0 tx=0x7fcfbc00c3a0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.798+0000 7fcfca667700 1 -- 172.21.15.33:0/482159787 shutdown_connections 2024-05-08T21:26:07.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.798+0000 7fcfca667700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcfac0a1120 0x7fcfac0a35e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.798+0000 7fcfca667700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfc4106b40 0x7fcfc419e210 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.798+0000 7fcfca667700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc41018e0 0x7fcfc4196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.798+0000 7fcfca667700 1 --2- 172.21.15.33:0/482159787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc4104210 0x7fcfc4197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:07.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.798+0000 7fcfca667700 1 -- 172.21.15.33:0/482159787 >> 172.21.15.33:0/482159787 conn(0x7fcfc40fb290 msgr2=0x7fcfc4102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:07.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.798+0000 7fcfca667700 1 -- 172.21.15.33:0/482159787 shutdown_connections 2024-05-08T21:26:07.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:07.798+0000 7fcfca667700 1 -- 172.21.15.33:0/482159787 wait complete. 2024-05-08T21:26:07.818 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:26:07.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:26:07.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:07 smithi071 ceph-mon[47074]: pgmap v504: 129 pgs: 129 active+clean; 252 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.8 MiB/s wr, 518 op/s 2024-05-08T21:26:07.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:07 smithi033 ceph-mon[42528]: pgmap v504: 129 pgs: 129 active+clean; 252 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.8 MiB/s wr, 518 op/s 2024-05-08T21:26:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:07 smithi082 ceph-mon[47512]: pgmap v504: 129 pgs: 129 active+clean; 252 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.8 MiB/s wr, 518 op/s 2024-05-08T21:26:08.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.010+0000 7fe33081d700 1 -- 172.21.15.33:0/2323860469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c104230 msgr2=0x7fe32c106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.010+0000 7fe33081d700 1 --2- 172.21.15.33:0/2323860469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c104230 0x7fe32c106620 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe320009990 tx=0x7fe320009ca0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.010+0000 7fe33081d700 1 -- 172.21.15.33:0/2323860469 shutdown_connections 2024-05-08T21:26:08.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.010+0000 7fe33081d700 1 --2- 172.21.15.33:0/2323860469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32c106b60 0x7fe32c108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.010+0000 7fe33081d700 1 --2- 172.21.15.33:0/2323860469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe32c101900 0x7fe32c103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.010+0000 7fe33081d700 1 --2- 172.21.15.33:0/2323860469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c104230 0x7fe32c106620 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.010+0000 7fe33081d700 1 -- 172.21.15.33:0/2323860469 >> 172.21.15.33:0/2323860469 conn(0x7fe32c0fb2f0 msgr2=0x7fe32c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.010+0000 7fe33081d700 1 -- 172.21.15.33:0/2323860469 shutdown_connections 2024-05-08T21:26:08.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.010+0000 7fe33081d700 1 -- 172.21.15.33:0/2323860469 wait complete. 2024-05-08T21:26:08.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.011+0000 7fe33081d700 1 Processor -- start 2024-05-08T21:26:08.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.011+0000 7fe33081d700 1 -- start start 2024-05-08T21:26:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe33081d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe32c101900 0x7fe32c101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe33081d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c104230 0x7fe32c0ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe33081d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32c106b60 0x7fe32c0ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe33081d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c100590 con 0x7fe32c104230 2024-05-08T21:26:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe33081d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c100700 con 0x7fe32c101900 2024-05-08T21:26:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe33081d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c1a0d50 con 0x7fe32c106b60 2024-05-08T21:26:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe32ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32c106b60 0x7fe32c0ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe329d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c104230 0x7fe32c0ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe32a59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe32c101900 0x7fe32c101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe32ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32c106b60 0x7fe32c0ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39612/0 (socket says 172.21.15.33:39612) 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe329d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c104230 0x7fe32c0ff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52826/0 (socket says 172.21.15.33:52826) 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe32ad9d700 1 -- 172.21.15.33:0/1940971710 learned_addr learned my addr 172.21.15.33:0/1940971710 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe32ad9d700 1 -- 172.21.15.33:0/1940971710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe32c101900 msgr2=0x7fe32c101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe32ad9d700 1 --2- 172.21.15.33:0/1940971710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe32c101900 0x7fe32c101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe32ad9d700 1 -- 172.21.15.33:0/1940971710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c104230 msgr2=0x7fe32c0ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe32ad9d700 1 --2- 172.21.15.33:0/1940971710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c104230 0x7fe32c0ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe32ad9d700 1 -- 172.21.15.33:0/1940971710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c009520 con 0x7fe32c106b60 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.012+0000 7fe32ad9d700 1 --2- 172.21.15.33:0/1940971710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32c106b60 0x7fe32c0ffdf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe324007ee0 tx=0x7fe32400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.013+0000 7fe31b7fe700 1 -- 172.21.15.33:0/1940971710 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe324017070 con 0x7fe32c106b60 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.013+0000 7fe31b7fe700 1 -- 172.21.15.33:0/1940971710 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe32400f040 con 0x7fe32c106b60 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.013+0000 7fe33081d700 1 -- 172.21.15.33:0/1940971710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3200095f0 con 0x7fe32c106b60 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.013+0000 7fe31b7fe700 1 -- 172.21.15.33:0/1940971710 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe324005570 con 0x7fe32c106b60 2024-05-08T21:26:08.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.013+0000 7fe33081d700 1 -- 172.21.15.33:0/1940971710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c196ca0 con 0x7fe32c106b60 2024-05-08T21:26:08.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.014+0000 7fe33081d700 1 -- 172.21.15.33:0/1940971710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c0fce70 con 0x7fe32c106b60 2024-05-08T21:26:08.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.015+0000 7fe31b7fe700 1 -- 172.21.15.33:0/1940971710 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fe32400db20 con 0x7fe32c106b60 2024-05-08T21:26:08.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.016+0000 7fe31b7fe700 1 --2- 172.21.15.33:0/1940971710 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe3140a1120 0x7fe3140a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.016+0000 7fe31b7fe700 1 -- 172.21.15.33:0/1940971710 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fe3240cc3b0 con 0x7fe32c106b60 2024-05-08T21:26:08.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.016+0000 7fe32a59c700 1 --2- 172.21.15.33:0/1940971710 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe3140a1120 0x7fe3140a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.017+0000 7fe32a59c700 1 --2- 172.21.15.33:0/1940971710 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe3140a1120 0x7fe3140a35e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009df0 tx=0x7fe31c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.021+0000 7fe31b7fe700 1 -- 172.21.15.33:0/1940971710 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fe3240957c0 con 0x7fe32c106b60 2024-05-08T21:26:08.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.394+0000 7fe33081d700 1 -- 172.21.15.33:0/1940971710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe32c0fc850 con 0x7fe32c106b60 2024-05-08T21:26:08.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.395+0000 7fe31b7fe700 1 -- 172.21.15.33:0/1940971710 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10899 (secure 0 0 0) 0x7fe324098de0 con 0x7fe32c106b60 2024-05-08T21:26:08.397 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:26:08.397 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:25:13.388928+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:26:08.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.398+0000 7fe33081d700 1 -- 172.21.15.33:0/1940971710 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe3140a1120 msgr2=0x7fe3140a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.398+0000 7fe33081d700 1 --2- 172.21.15.33:0/1940971710 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe3140a1120 0x7fe3140a35e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009df0 tx=0x7fe31c009260 comp rx=0 tx=0).stop 2024-05-08T21:26:08.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.398+0000 7fe33081d700 1 -- 172.21.15.33:0/1940971710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32c106b60 msgr2=0x7fe32c0ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.398+0000 7fe33081d700 1 --2- 172.21.15.33:0/1940971710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32c106b60 0x7fe32c0ffdf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe324007ee0 tx=0x7fe32400d3b0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.398+0000 7fe33081d700 1 -- 172.21.15.33:0/1940971710 shutdown_connections 2024-05-08T21:26:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.398+0000 7fe33081d700 1 --2- 172.21.15.33:0/1940971710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe32c106b60 0x7fe32c0ffdf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.398+0000 7fe33081d700 1 --2- 172.21.15.33:0/1940971710 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fe3140a1120 0x7fe3140a35e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.398+0000 7fe33081d700 1 --2- 172.21.15.33:0/1940971710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe32c101900 0x7fe32c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.398+0000 7fe33081d700 1 --2- 172.21.15.33:0/1940971710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c104230 0x7fe32c0ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.398+0000 7fe33081d700 1 -- 172.21.15.33:0/1940971710 >> 172.21.15.33:0/1940971710 conn(0x7fe32c0fb2f0 msgr2=0x7fe32c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.399+0000 7fe33081d700 1 -- 172.21.15.33:0/1940971710 shutdown_connections 2024-05-08T21:26:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.399+0000 7fe33081d700 1 -- 172.21.15.33:0/1940971710 wait complete. 2024-05-08T21:26:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 39 2024-05-08T21:26:08.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:26:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.604+0000 7f001b329700 1 -- 172.21.15.33:0/4271911375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0014100a50 msgr2=0x7f0014100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.604+0000 7f001b329700 1 --2- 172.21.15.33:0/4271911375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0014100a50 0x7f0014100ed0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f000c009fc0 tx=0x7f000c00b330 comp rx=0 tx=0).stop 2024-05-08T21:26:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.604+0000 7f001b329700 1 -- 172.21.15.33:0/4271911375 shutdown_connections 2024-05-08T21:26:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.604+0000 7f001b329700 1 --2- 172.21.15.33:0/4271911375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0014100a50 0x7f0014100ed0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.604+0000 7f001b329700 1 --2- 172.21.15.33:0/4271911375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00140ff730 0x7f00140ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.604+0000 7f001b329700 1 --2- 172.21.15.33:0/4271911375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0014100090 0x7f0014100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.604+0000 7f001b329700 1 -- 172.21.15.33:0/4271911375 >> 172.21.15.33:0/4271911375 conn(0x7f00140fb2d0 msgr2=0x7f00140fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.605+0000 7f001b329700 1 -- 172.21.15.33:0/4271911375 shutdown_connections 2024-05-08T21:26:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.605+0000 7f001b329700 1 -- 172.21.15.33:0/4271911375 wait complete. 2024-05-08T21:26:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.605+0000 7f001b329700 1 Processor -- start 2024-05-08T21:26:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.606+0000 7f001b329700 1 -- start start 2024-05-08T21:26:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.606+0000 7f001b329700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00140ff730 0x7f001419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.606+0000 7f001b329700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014100090 0x7f001419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.606+0000 7f001b329700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0014100a50 0x7f00141a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.606+0000 7f001b329700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00141a4f60 con 0x7f0014100a50 2024-05-08T21:26:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.606+0000 7f001b329700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00141a50a0 con 0x7f00140ff730 2024-05-08T21:26:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.606+0000 7f001b329700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00141a5230 con 0x7f0014100090 2024-05-08T21:26:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.606+0000 7f00190c5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00140ff730 0x7f001419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.606+0000 7f00188c4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014100090 0x7f001419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.606+0000 7f00190c5700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00140ff730 0x7f001419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:58536/0 (socket says 172.21.15.33:58536) 2024-05-08T21:26:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.606+0000 7f00190c5700 1 -- 172.21.15.33:0/3729884250 learned_addr learned my addr 172.21.15.33:0/3729884250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.606+0000 7f00188c4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014100090 0x7f001419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39624/0 (socket says 172.21.15.33:39624) 2024-05-08T21:26:08.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.607+0000 7f00188c4700 1 -- 172.21.15.33:0/3729884250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00140ff730 msgr2=0x7f001419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.607+0000 7f00188c4700 1 --2- 172.21.15.33:0/3729884250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00140ff730 0x7f001419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.607+0000 7f00188c4700 1 -- 172.21.15.33:0/3729884250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0014100a50 msgr2=0x7f00141a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.607+0000 7f00188c4700 1 --2- 172.21.15.33:0/3729884250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0014100a50 0x7f00141a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.607+0000 7f00188c4700 1 -- 172.21.15.33:0/3729884250 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c009c40 con 0x7f0014100090 2024-05-08T21:26:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.610+0000 7f00188c4700 1 --2- 172.21.15.33:0/3729884250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014100090 0x7f001419d970 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f000400e9f0 tx=0x7f000400edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.610+0000 7f000a7fc700 1 -- 172.21.15.33:0/3729884250 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000400cbd0 con 0x7f0014100090 2024-05-08T21:26:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.610+0000 7f000a7fc700 1 -- 172.21.15.33:0/3729884250 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f000400cd30 con 0x7f0014100090 2024-05-08T21:26:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.610+0000 7f000a7fc700 1 -- 172.21.15.33:0/3729884250 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004018680 con 0x7f0014100090 2024-05-08T21:26:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.610+0000 7f001b329700 1 -- 172.21.15.33:0/3729884250 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00141a5510 con 0x7f0014100090 2024-05-08T21:26:08.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.610+0000 7f001b329700 1 -- 172.21.15.33:0/3729884250 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00141a5a20 con 0x7f0014100090 2024-05-08T21:26:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.615+0000 7f00274e3b80 1 Processor -- start 2024-05-08T21:26:08.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.617+0000 7f000a7fc700 1 -- 172.21.15.33:0/3729884250 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0004010430 con 0x7f0014100090 2024-05-08T21:26:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.617+0000 7f000a7fc700 1 --2- 172.21.15.33:0/3729884250 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f00000a1220 0x7f00000a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.618+0000 7f00274e3b80 1 -- start start 2024-05-08T21:26:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.618+0000 7f000a7fc700 1 -- 172.21.15.33:0/3729884250 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f0004014070 con 0x7f0014100090 2024-05-08T21:26:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.618+0000 7f00274e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134900540 0x55d134900960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.618+0000 7f00274e3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d134901320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.618+0000 7f00274e3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 0x55d134901ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.618+0000 7f00274e3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d134857150 con 0x55d134900ea0 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.618+0000 7f00274e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d134854b10 con 0x55d134900540 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.618+0000 7f00274e3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d134850400 con 0x55d134901860 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.618+0000 7f00198c6700 1 --2- 172.21.15.33:0/3729884250 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f00000a1220 0x7f00000a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.618+0000 7f00198c6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 0x55d134901ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.618+0000 7f00198c6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 0x55d134901ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39632/0 (socket says 172.21.15.33:39632) 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.618+0000 7f00198c6700 1 -- 172.21.15.33:0/3338751370 learned_addr learned my addr 172.21.15.33:0/3338751370 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.619+0000 7f00188c4700 1 --2- 172.21.15.33:0/3338751370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d134901320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.619+0000 7f00198c6700 1 -- 172.21.15.33:0/3338751370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134900540 msgr2=0x55d134900960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.619+0000 7f00198c6700 1 --2- 172.21.15.33:0/3338751370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134900540 0x55d134900960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.619+0000 7f00198c6700 1 -- 172.21.15.33:0/3338751370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 msgr2=0x55d134901320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.619+0000 7f00198c6700 1 --2- 172.21.15.33:0/3338751370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d134901320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.619+0000 7f00198c6700 1 -- 172.21.15.33:0/3338751370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1346d13b0 con 0x55d134901860 2024-05-08T21:26:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.619+0000 7f00198c6700 1 --2- 172.21.15.33:0/3729884250 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f00000a1220 0x7f00000a36e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f000c009550 tx=0x7f000c009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.619+0000 7f00198c6700 1 --2- 172.21.15.33:0/3338751370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 0x55d134901ce0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f000c00b6c0 tx=0x7f000c024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfd7b3a115490748 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.620+0000 7effff7fe700 1 -- 172.21.15.33:0/3338751370 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c00e050 con 0x55d134901860 2024-05-08T21:26:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.620+0000 7effff7fe700 1 -- 172.21.15.33:0/3338751370 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f000c00be80 con 0x55d134901860 2024-05-08T21:26:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.620+0000 7effff7fe700 1 -- 172.21.15.33:0/3338751370 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c01ea40 con 0x55d134901860 2024-05-08T21:26:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.620+0000 7f00274e3b80 1 -- 172.21.15.33:0/3338751370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 msgr2=0x55d134901ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.620+0000 7f00274e3b80 1 --2- 172.21.15.33:0/3338751370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 0x55d134901ce0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f000c00b6c0 tx=0x7f000c024000 comp rx=0 tx=0).stop 2024-05-08T21:26:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.621+0000 7f00274e3b80 1 -- 172.21.15.33:0/3338751370 shutdown_connections 2024-05-08T21:26:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.621+0000 7f00274e3b80 1 --2- 172.21.15.33:0/3338751370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 0x55d134901ce0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.621+0000 7f00274e3b80 1 --2- 172.21.15.33:0/3338751370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134900540 0x55d134900960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.621+0000 7f00274e3b80 1 --2- 172.21.15.33:0/3338751370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d134901320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.621+0000 7f00274e3b80 1 -- 172.21.15.33:0/3338751370 >> 172.21.15.33:0/3338751370 conn(0x55d13485e210 msgr2=0x55d1348fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.623+0000 7f00274e3b80 1 -- 172.21.15.33:0/3338751370 shutdown_connections 2024-05-08T21:26:08.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.623+0000 7f00274e3b80 1 -- 172.21.15.33:0/3338751370 wait complete. 2024-05-08T21:26:08.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.623+0000 7f00274e3b80 1 -- start start 2024-05-08T21:26:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.623+0000 7f00274e3b80 1 Processor -- start 2024-05-08T21:26:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.624+0000 7f00274e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134900ea0 0x55d134909d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.624+0000 7f00274e3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 0x55d13490a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.624+0000 7f00274e3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d13490f2c0 0x55d134911750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.624+0000 7f00274e3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1347ec560 con 0x55d13490f2c0 2024-05-08T21:26:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.624+0000 7f00274e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1347ec200 con 0x55d134900ea0 2024-05-08T21:26:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.624+0000 7f00274e3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1346d36a0 con 0x55d134901860 2024-05-08T21:26:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.624+0000 7f00274e3b80 10 client.0 _ll_register_callbacks cb 0x55d1347f5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.624+0000 7f00188c4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 0x55d13490a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7f00188c4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 0x55d13490a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39634/0 (socket says 172.21.15.33:39634) 2024-05-08T21:26:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7f00188c4700 1 -- 172.21.15.33:0/3415990930 learned_addr learned my addr 172.21.15.33:0/3415990930 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7f00198c6700 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d13490f2c0 0x55d134911750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7f00190c5700 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134900ea0 0x55d134909d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7f00190c5700 1 -- 172.21.15.33:0/3415990930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 msgr2=0x55d13490a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7f00190c5700 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 0x55d13490a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7f00190c5700 1 -- 172.21.15.33:0/3415990930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d13490f2c0 msgr2=0x55d134911750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7f00190c5700 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d13490f2c0 0x55d134911750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7f00190c5700 1 -- 172.21.15.33:0/3415990930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001410bdd0 con 0x55d134900ea0 2024-05-08T21:26:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7f00188c4700 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 0x55d13490a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7f00190c5700 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134900ea0 0x55d134909d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55d1348fb880 tx=0x7f00100118f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d0cf23505b0885d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7efffcff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:26:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7efffcff9700 1 -- 172.21.15.33:0/3415990930 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010004440 con 0x55d134900ea0 2024-05-08T21:26:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.625+0000 7efffcff9700 1 -- 172.21.15.33:0/3415990930 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f0010011e60 con 0x55d134900ea0 2024-05-08T21:26:08.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.626+0000 7f00274e3b80 1 -- 172.21.15.33:0/3415990930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1348cf810 con 0x55d134900ea0 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.626+0000 7efffcff9700 1 -- 172.21.15.33:0/3415990930 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00100045a0 con 0x55d134900ea0 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.626+0000 7f00274e3b80 10 client.27314 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.626+0000 7f00274e3b80 1 -- 172.21.15.33:0/3415990930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d13467db60 con 0x55d134900ea0 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.626+0000 7efffcff9700 1 -- 172.21.15.33:0/3415990930 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d1348cf810 con 0x55d134900ea0 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.626+0000 7efffcff9700 1 -- 172.21.15.33:0/3415990930 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f001001b7b0 con 0x55d134900ea0 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.627+0000 7f00274e3b80 10 client.27314 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.627+0000 7f00274e3b80 10 client.27314 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.627+0000 7f00274e3b80 20 client.27314 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.627+0000 7f00274e3b80 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55d134915e60 0x55d134918280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.627+0000 7f00274e3b80 4 client.27314 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.627+0000 7f00274e3b80 1 -- 172.21.15.33:0/3415990930 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d134848ec0 con 0x55d134915e60 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.628+0000 7f00188c4700 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55d134915e60 0x55d134918280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.628+0000 7f00188c4700 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55d134915e60 0x55d134918280 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1aea4f792e6682e2 server_cookie=9c9e0deca6d2b9b4 in_seq=0 out_seq=0 2024-05-08T21:26:08.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.628+0000 7efffcff9700 10 client.27314 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:26:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7efffcff9700 1 -- 172.21.15.33:0/3415990930 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d134848ec0 con 0x55d134915e60 2024-05-08T21:26:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7efffcff9700 10 client.27314 handle_command_reply: tid=0 2024-05-08T21:26:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7f00274e3b80 1 client.27314 shutdown 2024-05-08T21:26:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7f00274e3b80 1 -- 172.21.15.33:0/3415990930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134900ea0 msgr2=0x55d134909d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7f00274e3b80 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134900ea0 0x55d134909d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55d1348fb880 tx=0x7f00100118f0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7f00274e3b80 1 -- 172.21.15.33:0/3415990930 shutdown_connections 2024-05-08T21:26:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7f00274e3b80 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d13490f2c0 0x55d134911750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7f00274e3b80 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134900ea0 0x55d134909d50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7f00274e3b80 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55d134915e60 0x55d134918280 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7f00274e3b80 1 --2- 172.21.15.33:0/3415990930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d134901860 0x55d13490a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7efffcff9700 0 client.27314 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:26:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7f00274e3b80 1 -- 172.21.15.33:0/3415990930 >> 172.21.15.33:0/3415990930 conn(0x55d13485e210 msgr2=0x55d1348faca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7f00274e3b80 1 -- 172.21.15.33:0/3415990930 shutdown_connections 2024-05-08T21:26:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.629+0000 7f00274e3b80 1 -- 172.21.15.33:0/3415990930 wait complete. 2024-05-08T21:26:08.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.630+0000 7f00274e3b80 20 client.27314 trim_cache size 0 max 16384 2024-05-08T21:26:08.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.647+0000 7f00274e3b80 1 Processor -- start 2024-05-08T21:26:08.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.647+0000 7f00274e3b80 1 -- start start 2024-05-08T21:26:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.647+0000 7f00274e3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d134907f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.647+0000 7f00274e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134901860 0x55d134909260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.647+0000 7f00274e3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d13490f2c0 0x55d134909b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.647+0000 7f00274e3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d134912830 con 0x55d134900ea0 2024-05-08T21:26:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.647+0000 7f00274e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1348fc870 con 0x55d134901860 2024-05-08T21:26:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.647+0000 7f00274e3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d134901690 con 0x55d13490f2c0 2024-05-08T21:26:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.647+0000 7f00190c5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d134907f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.647+0000 7f00190c5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d134907f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52886/0 (socket says 172.21.15.33:52886) 2024-05-08T21:26:08.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.647+0000 7f00190c5700 1 -- 172.21.15.33:0/1502136226 learned_addr learned my addr 172.21.15.33:0/1502136226 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.648+0000 7f00190c5700 1 -- 172.21.15.33:0/1502136226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d13490f2c0 msgr2=0x55d134909b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.648+0000 7f00190c5700 1 --2- 172.21.15.33:0/1502136226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d13490f2c0 0x55d134909b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.648+0000 7f00190c5700 1 -- 172.21.15.33:0/1502136226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134901860 msgr2=0x55d134909260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:26:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.648+0000 7f00190c5700 1 --2- 172.21.15.33:0/1502136226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134901860 0x55d134909260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.648+0000 7f00190c5700 1 -- 172.21.15.33:0/1502136226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d134816560 con 0x55d134900ea0 2024-05-08T21:26:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.648+0000 7f00190c5700 1 --2- 172.21.15.33:0/1502136226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d134907f70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0010009540 tx=0x7f0010009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e2fb6d4647c755 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.648+0000 7efffffff700 1 -- 172.21.15.33:0/1502136226 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00100054b0 con 0x55d134900ea0 2024-05-08T21:26:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.648+0000 7efffffff700 1 -- 172.21.15.33:0/1502136226 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0010011e60 con 0x55d134900ea0 2024-05-08T21:26:08.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.648+0000 7efffffff700 1 -- 172.21.15.33:0/1502136226 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010003b40 con 0x55d134900ea0 2024-05-08T21:26:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.649+0000 7f00274e3b80 1 -- 172.21.15.33:0/1502136226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 msgr2=0x55d134907f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.649+0000 7f00274e3b80 1 --2- 172.21.15.33:0/1502136226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d134907f70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0010009540 tx=0x7f0010009820 comp rx=0 tx=0).stop 2024-05-08T21:26:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.649+0000 7f00274e3b80 1 -- 172.21.15.33:0/1502136226 shutdown_connections 2024-05-08T21:26:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.649+0000 7f00274e3b80 1 --2- 172.21.15.33:0/1502136226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d13490f2c0 0x55d134909b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.649+0000 7f00274e3b80 1 --2- 172.21.15.33:0/1502136226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d134907f70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.649+0000 7f00274e3b80 1 --2- 172.21.15.33:0/1502136226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134901860 0x55d134909260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.649+0000 7f00274e3b80 1 -- 172.21.15.33:0/1502136226 >> 172.21.15.33:0/1502136226 conn(0x55d13485e210 msgr2=0x55d1348f7210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.650+0000 7f00274e3b80 1 -- 172.21.15.33:0/1502136226 shutdown_connections 2024-05-08T21:26:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.650+0000 7f00274e3b80 1 -- 172.21.15.33:0/1502136226 wait complete. 2024-05-08T21:26:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.650+0000 7f00274e3b80 1 -- start start 2024-05-08T21:26:08.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.650+0000 7f00274e3b80 1 Processor -- start 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00274e3b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d13491c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00274e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134901860 0x55d13491cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00274e3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d13490f2c0 0x55d13491d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00274e3b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13496afe0 con 0x55d134900ea0 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00274e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13496b150 con 0x55d134901860 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00274e3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d13497c320 con 0x55d13490f2c0 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00274e3b80 10 client.0 _ll_register_callbacks cb 0x55d1347f5b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00190c5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d13491c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00190c5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d13491c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52888/0 (socket says 172.21.15.33:52888) 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00190c5700 1 -- 172.21.15.33:0/1012555789 learned_addr learned my addr 172.21.15.33:0/1012555789 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00198c6700 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d13490f2c0 0x55d13491d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00188c4700 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134901860 0x55d13491cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00188c4700 1 -- 172.21.15.33:0/1012555789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d13490f2c0 msgr2=0x55d13491d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00188c4700 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d13490f2c0 0x55d13491d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00188c4700 1 -- 172.21.15.33:0/1012555789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 msgr2=0x55d13491c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00188c4700 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d13491c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00188c4700 1 -- 172.21.15.33:0/1012555789 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d13467db60 con 0x55d134901860 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.651+0000 7f00190c5700 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d13491c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.652+0000 7f00188c4700 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134901860 0x55d13491cc20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00040a1f10 tx=0x7f00040109c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=497cadad0aeb0cfc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.652+0000 7efffcff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.652+0000 7efffcff9700 1 -- 172.21.15.33:0/1012555789 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004099fc0 con 0x55d134901860 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.652+0000 7efffcff9700 1 -- 172.21.15.33:0/1012555789 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f000409ddf0 con 0x55d134901860 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.652+0000 7f00274e3b80 1 -- 172.21.15.33:0/1012555789 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1347a9270 con 0x55d134901860 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.652+0000 7efffcff9700 1 -- 172.21.15.33:0/1012555789 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00040a2440 con 0x55d134901860 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.652+0000 7f00274e3b80 10 client.27317 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.652+0000 7f00274e3b80 1 -- 172.21.15.33:0/1012555789 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d1346d5cd0 con 0x55d134901860 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.652+0000 7efffcff9700 1 -- 172.21.15.33:0/1012555789 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f000407b080 con 0x55d134901860 2024-05-08T21:26:08.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.653+0000 7efffcff9700 1 -- 172.21.15.33:0/1012555789 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0004094c60 con 0x55d134901860 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.653+0000 7f00274e3b80 10 client.27317 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.653+0000 7f00274e3b80 10 client.27317 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.653+0000 7f00274e3b80 20 client.27317 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.653+0000 7f00274e3b80 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55d13497dca0 0x55d1349800c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.653+0000 7f00274e3b80 4 client.27317 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.653+0000 7f00274e3b80 1 -- 172.21.15.33:0/1012555789 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d13469ecf0 con 0x55d13497dca0 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.653+0000 7f00190c5700 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55d13497dca0 0x55d1349800c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.653+0000 7efffcff9700 10 client.27317 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.653+0000 7f00190c5700 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55d13497dca0 0x55d1349800c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=22f5f3f3f6ef07b1 server_cookie=109a2ec73281ad34 in_seq=0 out_seq=0 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7efffcff9700 1 -- 172.21.15.33:0/1012555789 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d13469ecf0 con 0x55d13497dca0 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7efffcff9700 10 client.27317 handle_command_reply: tid=0 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7f00274e3b80 1 client.27317 shutdown 2024-05-08T21:26:08.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7f00274e3b80 1 -- 172.21.15.33:0/1012555789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134901860 msgr2=0x55d13491cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7f00274e3b80 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134901860 0x55d13491cc20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00040a1f10 tx=0x7f00040109c0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7f00274e3b80 1 -- 172.21.15.33:0/1012555789 shutdown_connections 2024-05-08T21:26:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7f00274e3b80 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55d13497dca0 0x55d1349800c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7f00274e3b80 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d13490f2c0 0x55d13491d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7f00274e3b80 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d134900ea0 0x55d13491c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7efffcff9700 0 client.27317 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:26:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7f00274e3b80 1 --2- 172.21.15.33:0/1012555789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d134901860 0x55d13491cc20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7f00274e3b80 1 -- 172.21.15.33:0/1012555789 >> 172.21.15.33:0/1012555789 conn(0x55d13485e210 msgr2=0x55d134909260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7f00274e3b80 1 -- 172.21.15.33:0/1012555789 shutdown_connections 2024-05-08T21:26:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7f00274e3b80 1 -- 172.21.15.33:0/1012555789 wait complete. 2024-05-08T21:26:08.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.654+0000 7f00274e3b80 20 client.27317 trim_cache size 0 max 16384 2024-05-08T21:26:08.659 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:26:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.655+0000 7f001b329700 1 -- 172.21.15.33:0/3729884250 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f00000a1220 msgr2=0x7f00000a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.655+0000 7f001b329700 1 --2- 172.21.15.33:0/3729884250 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f00000a1220 0x7f00000a36e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f000c009550 tx=0x7f000c009360 comp rx=0 tx=0).stop 2024-05-08T21:26:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.655+0000 7f001b329700 1 -- 172.21.15.33:0/3729884250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014100090 msgr2=0x7f001419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.655+0000 7f001b329700 1 --2- 172.21.15.33:0/3729884250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014100090 0x7f001419d970 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f000400e9f0 tx=0x7f000400edb0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.655+0000 7f001b329700 1 -- 172.21.15.33:0/3729884250 shutdown_connections 2024-05-08T21:26:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.655+0000 7f001b329700 1 --2- 172.21.15.33:0/3729884250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0014100a50 0x7f00141a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.655+0000 7f001b329700 1 --2- 172.21.15.33:0/3729884250 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f00000a1220 0x7f00000a36e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.655+0000 7f001b329700 1 --2- 172.21.15.33:0/3729884250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00140ff730 0x7f001419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.655+0000 7f001b329700 1 --2- 172.21.15.33:0/3729884250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0014100090 0x7f001419d970 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.655+0000 7f001b329700 1 -- 172.21.15.33:0/3729884250 >> 172.21.15.33:0/3729884250 conn(0x7f00140fb2d0 msgr2=0x7f00140fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.655+0000 7f001b329700 1 -- 172.21.15.33:0/3729884250 shutdown_connections 2024-05-08T21:26:08.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.655+0000 7f001b329700 1 -- 172.21.15.33:0/3729884250 wait complete. 2024-05-08T21:26:08.672 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:08 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1940971710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:26:08.674 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:26:08.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:08 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1940971710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:26:08.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:08 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1940971710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:26:08.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.861+0000 7fc87ac44700 1 -- 172.21.15.33:0/875605844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8741086b0 msgr2=0x7fc874108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.861+0000 7fc87ac44700 1 --2- 172.21.15.33:0/875605844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8741086b0 0x7fc874108b10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc8600099c0 tx=0x7fc860009cd0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.861+0000 7fc87ac44700 1 -- 172.21.15.33:0/875605844 shutdown_connections 2024-05-08T21:26:08.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.861+0000 7fc87ac44700 1 --2- 172.21.15.33:0/875605844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc874109050 0x7fc87410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.861+0000 7fc87ac44700 1 --2- 172.21.15.33:0/875605844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8741074c0 0x7fc8741078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.861+0000 7fc87ac44700 1 --2- 172.21.15.33:0/875605844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8741086b0 0x7fc874108b10 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.861+0000 7fc87ac44700 1 -- 172.21.15.33:0/875605844 >> 172.21.15.33:0/875605844 conn(0x7fc874076460 msgr2=0x7fc8740788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.861+0000 7fc87ac44700 1 -- 172.21.15.33:0/875605844 shutdown_connections 2024-05-08T21:26:08.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.861+0000 7fc87ac44700 1 -- 172.21.15.33:0/875605844 wait complete. 2024-05-08T21:26:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc87ac44700 1 Processor -- start 2024-05-08T21:26:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc87ac44700 1 -- start start 2024-05-08T21:26:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc87ac44700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8741074c0 0x7fc87419d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc87ac44700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8741086b0 0x7fc87419d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc87ac44700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc874109050 0x7fc8741a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc87ac44700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741a4e30 con 0x7fc8741086b0 2024-05-08T21:26:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc87ac44700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741a4f70 con 0x7fc8741074c0 2024-05-08T21:26:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc87ac44700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741a5100 con 0x7fc874109050 2024-05-08T21:26:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc8789e0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8741074c0 0x7fc87419d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc8789e0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8741074c0 0x7fc87419d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:58578/0 (socket says 172.21.15.33:58578) 2024-05-08T21:26:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc8789e0700 1 -- 172.21.15.33:0/3445976900 learned_addr learned my addr 172.21.15.33:0/3445976900 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc8791e1700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc874109050 0x7fc8741a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.862+0000 7fc873fff700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8741086b0 0x7fc87419d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.863+0000 7fc8789e0700 1 -- 172.21.15.33:0/3445976900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc874109050 msgr2=0x7fc8741a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.863+0000 7fc8789e0700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc874109050 0x7fc8741a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.863+0000 7fc8789e0700 1 -- 172.21.15.33:0/3445976900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8741086b0 msgr2=0x7fc87419d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.863+0000 7fc8789e0700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8741086b0 0x7fc87419d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.863+0000 7fc8789e0700 1 -- 172.21.15.33:0/3445976900 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868009520 con 0x7fc8741074c0 2024-05-08T21:26:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.863+0000 7fc8791e1700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc874109050 0x7fc8741a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.863+0000 7fc8789e0700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8741074c0 0x7fc87419d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc86800ea50 tx=0x7fc86800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.863+0000 7fc871ffb700 1 -- 172.21.15.33:0/3445976900 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86800cbd0 con 0x7fc8741074c0 2024-05-08T21:26:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.863+0000 7fc871ffb700 1 -- 172.21.15.33:0/3445976900 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc86800cd30 con 0x7fc8741074c0 2024-05-08T21:26:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.863+0000 7fc87ac44700 1 -- 172.21.15.33:0/3445976900 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8600095f0 con 0x7fc8741074c0 2024-05-08T21:26:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.863+0000 7fc871ffb700 1 -- 172.21.15.33:0/3445976900 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868018680 con 0x7fc8741074c0 2024-05-08T21:26:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.863+0000 7fc87ac44700 1 -- 172.21.15.33:0/3445976900 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8741a5760 con 0x7fc8741074c0 2024-05-08T21:26:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.866+0000 7fc871ffb700 1 -- 172.21.15.33:0/3445976900 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc868010430 con 0x7fc8741074c0 2024-05-08T21:26:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.867+0000 7fc871ffb700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc8640a1120 0x7fc8640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.867+0000 7fc871ffb700 1 -- 172.21.15.33:0/3445976900 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc868014070 con 0x7fc8741074c0 2024-05-08T21:26:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.867+0000 7fc873fff700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc8640a1120 0x7fc8640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.868+0000 7fc873fff700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc8640a1120 0x7fc8640a35e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc860009990 tx=0x7fc860003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.868+0000 7fc886dfeb80 1 Processor -- start 2024-05-08T21:26:08.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.868+0000 7fc886dfeb80 1 -- start start 2024-05-08T21:26:08.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc886dfeb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011c540 0x55951011c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc886dfeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 0x55951011d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc886dfeb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011d860 0x55951011dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc886dfeb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559510073150 con 0x55951011d860 2024-05-08T21:26:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc886dfeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559510070b10 con 0x55951011cea0 2024-05-08T21:26:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc886dfeb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951006c400 con 0x55951011c540 2024-05-08T21:26:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc8789e0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011c540 0x55951011c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc8789e0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011c540 0x55951011c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39688/0 (socket says 172.21.15.33:39688) 2024-05-08T21:26:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc8789e0700 1 -- 172.21.15.33:0/3168691105 learned_addr learned my addr 172.21.15.33:0/3168691105 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc8791e1700 1 --2- 172.21.15.33:0/3168691105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011d860 0x55951011dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc8791e1700 1 -- 172.21.15.33:0/3168691105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011c540 msgr2=0x55951011c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc8791e1700 1 --2- 172.21.15.33:0/3168691105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011c540 0x55951011c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc8791e1700 1 -- 172.21.15.33:0/3168691105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 msgr2=0x55951011d320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:26:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.869+0000 7fc8791e1700 1 --2- 172.21.15.33:0/3168691105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 0x55951011d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.870+0000 7fc8791e1700 1 -- 172.21.15.33:0/3168691105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55950feed3b0 con 0x55951011d860 2024-05-08T21:26:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.870+0000 7fc8791e1700 1 --2- 172.21.15.33:0/3168691105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011d860 0x55951011dce0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc86c003db0 tx=0x7fc86c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89349b0014882648 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.870+0000 7fc85effd700 1 -- 172.21.15.33:0/3168691105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c004bb0 con 0x55951011d860 2024-05-08T21:26:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.870+0000 7fc85effd700 1 -- 172.21.15.33:0/3168691105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc86c00f040 con 0x55951011d860 2024-05-08T21:26:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.870+0000 7fc85effd700 1 -- 172.21.15.33:0/3168691105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c007bf0 con 0x55951011d860 2024-05-08T21:26:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.871+0000 7fc886dfeb80 1 -- 172.21.15.33:0/3168691105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011d860 msgr2=0x55951011dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.871+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/3168691105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011d860 0x55951011dce0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc86c003db0 tx=0x7fc86c009920 comp rx=0 tx=0).stop 2024-05-08T21:26:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.871+0000 7fc886dfeb80 1 -- 172.21.15.33:0/3168691105 shutdown_connections 2024-05-08T21:26:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.871+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/3168691105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011d860 0x55951011dce0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.871+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/3168691105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011c540 0x55951011c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.871+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/3168691105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 0x55951011d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.871+0000 7fc886dfeb80 1 -- 172.21.15.33:0/3168691105 >> 172.21.15.33:0/3168691105 conn(0x55951007a210 msgr2=0x559510116de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.871+0000 7fc886dfeb80 1 -- 172.21.15.33:0/3168691105 shutdown_connections 2024-05-08T21:26:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.872+0000 7fc886dfeb80 1 -- 172.21.15.33:0/3168691105 wait complete. 2024-05-08T21:26:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.872+0000 7fc886dfeb80 1 -- start start 2024-05-08T21:26:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.872+0000 7fc886dfeb80 1 Processor -- start 2024-05-08T21:26:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc886dfeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011c540 0x559510125e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc886dfeb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011cea0 0x5595101263a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc886dfeb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011d860 0x55951012d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc886dfeb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559510008560 con 0x55951011d860 2024-05-08T21:26:08.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc886dfeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559510008200 con 0x55951011c540 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc886dfeb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55950feef6a0 con 0x55951011cea0 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc886dfeb80 10 client.0 _ll_register_callbacks cb 0x559510011b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc873fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011cea0 0x5595101263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc873fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011cea0 0x5595101263a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39702/0 (socket says 172.21.15.33:39702) 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc873fff700 1 -- 172.21.15.33:0/2283301036 learned_addr learned my addr 172.21.15.33:0/2283301036 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc8791e1700 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011d860 0x55951012d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc8789e0700 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011c540 0x559510125e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc873fff700 1 -- 172.21.15.33:0/2283301036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011c540 msgr2=0x559510125e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc873fff700 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011c540 0x559510125e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc873fff700 1 -- 172.21.15.33:0/2283301036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011d860 msgr2=0x55951012d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc873fff700 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011d860 0x55951012d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc873fff700 1 -- 172.21.15.33:0/2283301036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c003a60 con 0x55951011cea0 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.873+0000 7fc8791e1700 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011d860 0x55951012d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:26:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.874+0000 7fc873fff700 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011cea0 0x5595101263a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc860006010 tx=0x7fc860009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d296441144480a0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.874+0000 7fc853fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.874+0000 7fc853fff700 1 -- 172.21.15.33:0/2283301036 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860029070 con 0x55951011cea0 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.874+0000 7fc853fff700 1 -- 172.21.15.33:0/2283301036 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc860003cd0 con 0x55951011cea0 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.874+0000 7fc886dfeb80 1 -- 172.21.15.33:0/2283301036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55950ffc5270 con 0x55951011cea0 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.874+0000 7fc853fff700 1 -- 172.21.15.33:0/2283301036 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc860003e90 con 0x55951011cea0 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.874+0000 7fc886dfeb80 10 client.27322 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.874+0000 7fc886dfeb80 1 -- 172.21.15.33:0/2283301036 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55950fe99b60 con 0x55951011cea0 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.874+0000 7fc853fff700 1 -- 172.21.15.33:0/2283301036 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55950ffc5270 con 0x55951011cea0 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.875+0000 7fc853fff700 1 -- 172.21.15.33:0/2283301036 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc860035070 con 0x55951011cea0 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.875+0000 7fc886dfeb80 10 client.27322 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.875+0000 7fc886dfeb80 10 client.27322 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.875+0000 7fc886dfeb80 20 client.27322 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.875+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x559510131b40 0x559510133f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.875+0000 7fc886dfeb80 4 client.27322 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.875+0000 7fc886dfeb80 1 -- 172.21.15.33:0/2283301036 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595100eb810 con 0x559510131b40 2024-05-08T21:26:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.876+0000 7fc8789e0700 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x559510131b40 0x559510133f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.876+0000 7fc8789e0700 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x559510131b40 0x559510133f60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b62d5215da1dc46b server_cookie=6db644497ba31287 in_seq=0 out_seq=0 2024-05-08T21:26:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.876+0000 7fc853fff700 10 client.27322 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:26:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc853fff700 1 -- 172.21.15.33:0/2283301036 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5595100eb810 con 0x559510131b40 2024-05-08T21:26:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc853fff700 10 client.27322 handle_command_reply: tid=0 2024-05-08T21:26:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc886dfeb80 1 client.27322 shutdown 2024-05-08T21:26:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc886dfeb80 1 -- 172.21.15.33:0/2283301036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011cea0 msgr2=0x5595101263a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011cea0 0x5595101263a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc860006010 tx=0x7fc860009310 comp rx=0 tx=0).stop 2024-05-08T21:26:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc886dfeb80 1 -- 172.21.15.33:0/2283301036 shutdown_connections 2024-05-08T21:26:08.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011d860 0x55951012d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011c540 0x559510125e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x559510131b40 0x559510133f60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/2283301036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011cea0 0x5595101263a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc886dfeb80 1 -- 172.21.15.33:0/2283301036 >> 172.21.15.33:0/2283301036 conn(0x55951007a210 msgr2=0x5595101165a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc853fff700 0 client.27322 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:26:08.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc886dfeb80 1 -- 172.21.15.33:0/2283301036 shutdown_connections 2024-05-08T21:26:08.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc886dfeb80 1 -- 172.21.15.33:0/2283301036 wait complete. 2024-05-08T21:26:08.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.877+0000 7fc886dfeb80 20 client.27322 trim_cache size 0 max 16384 2024-05-08T21:26:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc886dfeb80 1 Processor -- start 2024-05-08T21:26:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc886dfeb80 1 -- start start 2024-05-08T21:26:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc886dfeb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011c540 0x559510125370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc886dfeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 0x55951012d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc886dfeb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 0x559510123e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc886dfeb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559510117b60 con 0x55951011c540 2024-05-08T21:26:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc886dfeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559510127050 con 0x55951011cea0 2024-05-08T21:26:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc886dfeb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559510139150 con 0x55951011d860 2024-05-08T21:26:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc8789e0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011c540 0x559510125370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc8791e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 0x559510123e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc8791e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 0x559510123e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39708/0 (socket says 172.21.15.33:39708) 2024-05-08T21:26:08.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc8789e0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011c540 0x559510125370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52934/0 (socket says 172.21.15.33:52934) 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc8791e1700 1 -- 172.21.15.33:0/899752574 learned_addr learned my addr 172.21.15.33:0/899752574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc873fff700 1 --2- 172.21.15.33:0/899752574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 0x55951012d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc8791e1700 1 -- 172.21.15.33:0/899752574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 msgr2=0x55951012d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc8791e1700 1 --2- 172.21.15.33:0/899752574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 0x55951012d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc8791e1700 1 -- 172.21.15.33:0/899752574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011c540 msgr2=0x559510125370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc8791e1700 1 --2- 172.21.15.33:0/899752574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011c540 0x559510125370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.893+0000 7fc8791e1700 1 -- 172.21.15.33:0/899752574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559510032560 con 0x55951011d860 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.894+0000 7fc873fff700 1 --2- 172.21.15.33:0/899752574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 0x55951012d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.894+0000 7fc8789e0700 1 --2- 172.21.15.33:0/899752574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011c540 0x559510125370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.894+0000 7fc8791e1700 1 --2- 172.21.15.33:0/899752574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 0x559510123e70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc86c003d50 tx=0x7fc86c006070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6feb47f03fd4566b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.894+0000 7fc85f7fe700 1 -- 172.21.15.33:0/899752574 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0090d0 con 0x55951011d860 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.894+0000 7fc85f7fe700 1 -- 172.21.15.33:0/899752574 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc86c004e30 con 0x55951011d860 2024-05-08T21:26:08.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.894+0000 7fc85f7fe700 1 -- 172.21.15.33:0/899752574 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00e4f0 con 0x55951011d860 2024-05-08T21:26:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.895+0000 7fc886dfeb80 1 -- 172.21.15.33:0/899752574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 msgr2=0x559510123e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.895+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/899752574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 0x559510123e70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc86c003d50 tx=0x7fc86c006070 comp rx=0 tx=0).stop 2024-05-08T21:26:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.895+0000 7fc886dfeb80 1 -- 172.21.15.33:0/899752574 shutdown_connections 2024-05-08T21:26:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.895+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/899752574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 0x559510123e70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.895+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/899752574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011c540 0x559510125370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.895+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/899752574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 0x55951012d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.895+0000 7fc886dfeb80 1 -- 172.21.15.33:0/899752574 >> 172.21.15.33:0/899752574 conn(0x55951007a210 msgr2=0x559510113210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.895+0000 7fc886dfeb80 1 -- 172.21.15.33:0/899752574 shutdown_connections 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.895+0000 7fc886dfeb80 1 -- 172.21.15.33:0/899752574 wait complete. 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.895+0000 7fc886dfeb80 1 -- start start 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.895+0000 7fc886dfeb80 1 Processor -- start 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.896+0000 7fc886dfeb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011c540 0x559510128570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.896+0000 7fc886dfeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 0x559510128ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.896+0000 7fc886dfeb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 0x55951012b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.896+0000 7fc886dfeb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951012beb0 con 0x55951011c540 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.896+0000 7fc886dfeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951012c020 con 0x55951011cea0 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.896+0000 7fc886dfeb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55951012c1e0 con 0x55951011d860 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.896+0000 7fc886dfeb80 10 client.0 _ll_register_callbacks cb 0x559510011b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.896+0000 7fc8791e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 0x55951012b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.896+0000 7fc8791e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 0x55951012b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39720/0 (socket says 172.21.15.33:39720) 2024-05-08T21:26:08.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.896+0000 7fc8791e1700 1 -- 172.21.15.33:0/1160998755 learned_addr learned my addr 172.21.15.33:0/1160998755 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.896+0000 7fc8789e0700 1 --2- 172.21.15.33:0/1160998755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011c540 0x559510128570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.897+0000 7fc8791e1700 1 -- 172.21.15.33:0/1160998755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 msgr2=0x559510128ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.897+0000 7fc8791e1700 1 --2- 172.21.15.33:0/1160998755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 0x559510128ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.897+0000 7fc8791e1700 1 -- 172.21.15.33:0/1160998755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011c540 msgr2=0x559510128570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.897+0000 7fc8791e1700 1 --2- 172.21.15.33:0/1160998755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011c540 0x559510128570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.897+0000 7fc8791e1700 1 -- 172.21.15.33:0/1160998755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8741a5760 con 0x55951011d860 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.897+0000 7fc8791e1700 1 --2- 172.21.15.33:0/1160998755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 0x55951012b900 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc86c000f80 tx=0x7fc86c004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9452c7d33c68422 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.897+0000 7fc853fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.897+0000 7fc853fff700 1 -- 172.21.15.33:0/1160998755 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c007710 con 0x55951011d860 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.897+0000 7fc853fff700 1 -- 172.21.15.33:0/1160998755 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc86c00e4f0 con 0x55951011d860 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.897+0000 7fc886dfeb80 1 -- 172.21.15.33:0/1160998755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc86c00f040 con 0x55951011d860 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.898+0000 7fc853fff700 1 -- 172.21.15.33:0/1160998755 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc86c00e6b0 con 0x55951011d860 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.898+0000 7fc886dfeb80 10 client.27328 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.898+0000 7fc886dfeb80 1 -- 172.21.15.33:0/1160998755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55950feed3b0 con 0x55951011d860 2024-05-08T21:26:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.898+0000 7fc853fff700 1 -- 172.21.15.33:0/1160998755 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc86c00f040 con 0x55951011d860 2024-05-08T21:26:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.898+0000 7fc853fff700 1 -- 172.21.15.33:0/1160998755 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc86c024020 con 0x55951011d860 2024-05-08T21:26:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.898+0000 7fc886dfeb80 10 client.27328 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.898+0000 7fc886dfeb80 10 client.27328 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:26:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.898+0000 7fc886dfeb80 20 client.27328 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.898+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/1160998755 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x559510188030 0x55951018a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.898+0000 7fc886dfeb80 4 client.27328 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:26:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.898+0000 7fc886dfeb80 1 -- 172.21.15.33:0/1160998755 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559510064ec0 con 0x559510188030 2024-05-08T21:26:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.898+0000 7fc8789e0700 1 --2- 172.21.15.33:0/1160998755 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x559510188030 0x55951018a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.899+0000 7fc853fff700 10 client.27328 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:26:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.899+0000 7fc8789e0700 1 --2- 172.21.15.33:0/1160998755 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x559510188030 0x55951018a420 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=265dd3451a3c0b99 server_cookie=ce5116db2226106c in_seq=0 out_seq=0 2024-05-08T21:26:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc853fff700 1 -- 172.21.15.33:0/1160998755 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559510064ec0 con 0x559510188030 2024-05-08T21:26:08.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc853fff700 10 client.27328 handle_command_reply: tid=0 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc886dfeb80 1 client.27328 shutdown 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc886dfeb80 1 -- 172.21.15.33:0/1160998755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 msgr2=0x55951012b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/1160998755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 0x55951012b900 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc86c000f80 tx=0x7fc86c004d00 comp rx=0 tx=0).stop 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc886dfeb80 1 -- 172.21.15.33:0/1160998755 shutdown_connections 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/1160998755 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x559510188030 0x55951018a420 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/1160998755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55951011d860 0x55951012b900 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/1160998755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951011c540 0x559510128570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc853fff700 0 client.27328 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc886dfeb80 1 --2- 172.21.15.33:0/1160998755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55951011cea0 0x559510128ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc886dfeb80 1 -- 172.21.15.33:0/1160998755 >> 172.21.15.33:0/1160998755 conn(0x55951007a210 msgr2=0x559510120870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc886dfeb80 1 -- 172.21.15.33:0/1160998755 shutdown_connections 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.901+0000 7fc886dfeb80 1 -- 172.21.15.33:0/1160998755 wait complete. 2024-05-08T21:26:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.902+0000 7fc886dfeb80 20 client.27328 trim_cache size 0 max 16384 2024-05-08T21:26:08.908 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:26:08.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.902+0000 7fc87ac44700 1 -- 172.21.15.33:0/3445976900 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc8640a1120 msgr2=0x7fc8640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.902+0000 7fc87ac44700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc8640a1120 0x7fc8640a35e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc860009990 tx=0x7fc860003680 comp rx=0 tx=0).stop 2024-05-08T21:26:08.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.902+0000 7fc87ac44700 1 -- 172.21.15.33:0/3445976900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8741074c0 msgr2=0x7fc87419d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:08.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.902+0000 7fc87ac44700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8741074c0 0x7fc87419d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc86800ea50 tx=0x7fc86800ed60 comp rx=0 tx=0).stop 2024-05-08T21:26:08.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.903+0000 7fc87ac44700 1 -- 172.21.15.33:0/3445976900 shutdown_connections 2024-05-08T21:26:08.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.903+0000 7fc87ac44700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc874109050 0x7fc8741a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.903+0000 7fc87ac44700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc8640a1120 0x7fc8640a35e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.903+0000 7fc87ac44700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8741074c0 0x7fc87419d300 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.903+0000 7fc87ac44700 1 --2- 172.21.15.33:0/3445976900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8741086b0 0x7fc87419d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:08.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.903+0000 7fc87ac44700 1 -- 172.21.15.33:0/3445976900 >> 172.21.15.33:0/3445976900 conn(0x7fc874076460 msgr2=0x7fc87410ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:08.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.904+0000 7fc87ac44700 1 -- 172.21.15.33:0/3445976900 shutdown_connections 2024-05-08T21:26:08.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:08.904+0000 7fc87ac44700 1 -- 172.21.15.33:0/3445976900 wait complete. 2024-05-08T21:26:08.922 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:26:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.103+0000 7f1d6821c700 1 -- 172.21.15.33:0/3977209592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d601042d0 msgr2=0x7f1d601066c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.103+0000 7f1d6821c700 1 --2- 172.21.15.33:0/3977209592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d601042d0 0x7f1d601066c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009990 tx=0x7f1d50009ca0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.103+0000 7f1d6821c700 1 -- 172.21.15.33:0/3977209592 shutdown_connections 2024-05-08T21:26:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.103+0000 7f1d6821c700 1 --2- 172.21.15.33:0/3977209592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d60106c00 0x7f1d60109010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.103+0000 7f1d6821c700 1 --2- 172.21.15.33:0/3977209592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d601019a0 0x7f1d60103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.103+0000 7f1d6821c700 1 --2- 172.21.15.33:0/3977209592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d601042d0 0x7f1d601066c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.103+0000 7f1d6821c700 1 -- 172.21.15.33:0/3977209592 >> 172.21.15.33:0/3977209592 conn(0x7f1d600fb330 msgr2=0x7f1d600fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.103+0000 7f1d6821c700 1 -- 172.21.15.33:0/3977209592 shutdown_connections 2024-05-08T21:26:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.103+0000 7f1d6821c700 1 -- 172.21.15.33:0/3977209592 wait complete. 2024-05-08T21:26:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.104+0000 7f1d6821c700 1 Processor -- start 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.104+0000 7f1d6821c700 1 -- start start 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.104+0000 7f1d6821c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d601019a0 0x7f1d601012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.104+0000 7f1d6821c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d601042d0 0x7f1d600ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.104+0000 7f1d6821c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d60106c00 0x7f1d600ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.104+0000 7f1d6821c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6019ab70 con 0x7f1d601042d0 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.104+0000 7f1d6821c700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d601003d0 con 0x7f1d60106c00 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.104+0000 7f1d6821c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d60100590 con 0x7f1d601019a0 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d657b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d601042d0 0x7f1d600ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d657b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d601042d0 0x7f1d600ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52946/0 (socket says 172.21.15.33:52946) 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d657b7700 1 -- 172.21.15.33:0/382070322 learned_addr learned my addr 172.21.15.33:0/382070322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d65fb8700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d601019a0 0x7f1d601012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d667b9700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d60106c00 0x7f1d600ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d65fb8700 1 -- 172.21.15.33:0/382070322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d60106c00 msgr2=0x7f1d600ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d65fb8700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d60106c00 0x7f1d600ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d65fb8700 1 -- 172.21.15.33:0/382070322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d601042d0 msgr2=0x7f1d600ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d65fb8700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d601042d0 0x7f1d600ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d65fb8700 1 -- 172.21.15.33:0/382070322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c009520 con 0x7f1d601019a0 2024-05-08T21:26:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d657b7700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d601042d0 0x7f1d600ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d667b9700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d60106c00 0x7f1d600ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d65fb8700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d601019a0 0x7f1d601012c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e9f0 tx=0x7f1d5c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d56ffd700 1 -- 172.21.15.33:0/382070322 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00cb70 con 0x7f1d601019a0 2024-05-08T21:26:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d56ffd700 1 -- 172.21.15.33:0/382070322 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d5c00ee90 con 0x7f1d601019a0 2024-05-08T21:26:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d56ffd700 1 -- 172.21.15.33:0/382070322 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c003bc0 con 0x7f1d601019a0 2024-05-08T21:26:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.105+0000 7f1d6821c700 1 -- 172.21.15.33:0/382070322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d500095f0 con 0x7f1d601019a0 2024-05-08T21:26:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.106+0000 7f1d6821c700 1 -- 172.21.15.33:0/382070322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6019aec0 con 0x7f1d601019a0 2024-05-08T21:26:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.108+0000 7f1d56ffd700 1 -- 172.21.15.33:0/382070322 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1d5c003d80 con 0x7f1d601019a0 2024-05-08T21:26:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.109+0000 7f1d56ffd700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1d4c0a1120 0x7f1d4c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.109+0000 7f1d56ffd700 1 -- 172.21.15.33:0/382070322 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1d5c014070 con 0x7f1d601019a0 2024-05-08T21:26:09.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.109+0000 7f1d657b7700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1d4c0a1120 0x7f1d4c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.110+0000 7f1d657b7700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1d4c0a1120 0x7f1d4c0a35e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000b2e0 tx=0x7f1d5000b630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.111+0000 7f1d743d6b80 1 Processor -- start 2024-05-08T21:26:09.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.111+0000 7f1d743d6b80 1 -- start start 2024-05-08T21:26:09.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.112+0000 7f1d743d6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1540 0x55f579dd1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.112+0000 7f1d743d6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1ea0 0x55f579dd2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.112+0000 7f1d743d6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579dd2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.112+0000 7f1d743d6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f579d28150 con 0x55f579dd1ea0 2024-05-08T21:26:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.112+0000 7f1d743d6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f579d25b10 con 0x55f579dd2860 2024-05-08T21:26:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.112+0000 7f1d743d6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f579d21400 con 0x55f579dd1540 2024-05-08T21:26:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.112+0000 7f1d667b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579dd2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.112+0000 7f1d667b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579dd2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:58632/0 (socket says 172.21.15.33:58632) 2024-05-08T21:26:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.112+0000 7f1d667b9700 1 -- 172.21.15.33:0/1718604363 learned_addr learned my addr 172.21.15.33:0/1718604363 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.113+0000 7f1d667b9700 1 -- 172.21.15.33:0/1718604363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1540 msgr2=0x55f579dd1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.113+0000 7f1d667b9700 1 --2- 172.21.15.33:0/1718604363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1540 0x55f579dd1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.113+0000 7f1d667b9700 1 -- 172.21.15.33:0/1718604363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1ea0 msgr2=0x55f579dd2320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:26:09.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.113+0000 7f1d667b9700 1 --2- 172.21.15.33:0/1718604363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1ea0 0x55f579dd2320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.113+0000 7f1d667b9700 1 -- 172.21.15.33:0/1718604363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f579ba23b0 con 0x55f579dd2860 2024-05-08T21:26:09.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.113+0000 7f1d667b9700 1 --2- 172.21.15.33:0/1718604363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579dd2ce0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1d58003ee0 tx=0x7f1d58009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733a0aa18239759f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.114+0000 7f1d4bfff700 1 -- 172.21.15.33:0/1718604363 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58004900 con 0x55f579dd2860 2024-05-08T21:26:09.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.115+0000 7f1d4bfff700 1 -- 172.21.15.33:0/1718604363 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d5800f040 con 0x55f579dd2860 2024-05-08T21:26:09.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.115+0000 7f1d4bfff700 1 -- 172.21.15.33:0/1718604363 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800e8b0 con 0x55f579dd2860 2024-05-08T21:26:09.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.115+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1718604363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 msgr2=0x55f579dd2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.115+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/1718604363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579dd2ce0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1d58003ee0 tx=0x7f1d58009920 comp rx=0 tx=0).stop 2024-05-08T21:26:09.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.117+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1718604363 shutdown_connections 2024-05-08T21:26:09.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.117+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/1718604363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579dd2ce0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.117+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/1718604363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1540 0x55f579dd1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.117+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/1718604363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1ea0 0x55f579dd2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.117+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1718604363 >> 172.21.15.33:0/1718604363 conn(0x55f579d2f210 msgr2=0x55f579dcbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.117+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1718604363 shutdown_connections 2024-05-08T21:26:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.117+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1718604363 wait complete. 2024-05-08T21:26:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.118+0000 7f1d743d6b80 1 -- start start 2024-05-08T21:26:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.118+0000 7f1d743d6b80 1 Processor -- start 2024-05-08T21:26:09.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d743d6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1540 0x55f579debf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d743d6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1ea0 0x55f579dec450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d743d6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579df3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d743d6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f579cbd560 con 0x55f579dd1540 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d743d6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f579cbd200 con 0x55f579dd2860 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d743d6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f579ba46a0 con 0x55f579dd1ea0 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d743d6b80 10 client.0 _ll_register_callbacks cb 0x55f579cc6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d65fb8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1540 0x55f579debf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d65fb8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1540 0x55f579debf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52956/0 (socket says 172.21.15.33:52956) 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d65fb8700 1 -- 172.21.15.33:0/1400971583 learned_addr learned my addr 172.21.15.33:0/1400971583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d667b9700 1 --2- 172.21.15.33:0/1400971583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579df3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d65fb8700 1 -- 172.21.15.33:0/1400971583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1ea0 msgr2=0x55f579dec450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d65fb8700 1 --2- 172.21.15.33:0/1400971583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1ea0 0x55f579dec450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d65fb8700 1 -- 172.21.15.33:0/1400971583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 msgr2=0x55f579df3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d65fb8700 1 --2- 172.21.15.33:0/1400971583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579df3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.119+0000 7f1d65fb8700 1 -- 172.21.15.33:0/1400971583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d58003b90 con 0x55f579dd1540 2024-05-08T21:26:09.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.120+0000 7f1d65fb8700 1 --2- 172.21.15.33:0/1400971583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1540 0x55f579debf10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0a1fc0 tx=0x7f1d5c0999d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab95039659ebca32 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.120+0000 7f1d497fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.120+0000 7f1d497fa700 1 -- 172.21.15.33:0/1400971583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c09a070 con 0x55f579dd1540 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.120+0000 7f1d497fa700 1 -- 172.21.15.33:0/1400971583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1d5c09dea0 con 0x55f579dd1540 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.120+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1400971583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f579c7a270 con 0x55f579dd1540 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.120+0000 7f1d497fa700 1 -- 172.21.15.33:0/1400971583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d5c0a24f0 con 0x55f579dd1540 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.120+0000 7f1d743d6b80 10 client.17436 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.120+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1400971583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f579b4eb60 con 0x55f579dd1540 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.121+0000 7f1d497fa700 1 -- 172.21.15.33:0/1400971583 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f579c7a270 con 0x55f579dd1540 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.121+0000 7f1d497fa700 1 -- 172.21.15.33:0/1400971583 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1d5c0cabe0 con 0x55f579dd1540 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.121+0000 7f1d743d6b80 10 client.17436 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.121+0000 7f1d743d6b80 10 client.17436 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.121+0000 7f1d743d6b80 20 client.17436 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.121+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/1400971583 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f579df7c70 0x55f579dfa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.121+0000 7f1d743d6b80 4 client.17436 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:09.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.121+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1400971583 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f579da0810 con 0x55f579df7c70 2024-05-08T21:26:09.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.121+0000 7f1d657b7700 1 --2- 172.21.15.33:0/1400971583 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f579df7c70 0x55f579dfa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.123+0000 7f1d497fa700 10 client.17436 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:26:09.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.123+0000 7f1d657b7700 1 --2- 172.21.15.33:0/1400971583 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f579df7c70 0x55f579dfa090 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4d44eb249a68bb7 server_cookie=65afc42a5631e2b0 in_seq=0 out_seq=0 2024-05-08T21:26:09.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d497fa700 1 -- 172.21.15.33:0/1400971583 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f579da0810 con 0x55f579df7c70 2024-05-08T21:26:09.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d497fa700 10 client.17436 handle_command_reply: tid=0 2024-05-08T21:26:09.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d743d6b80 1 client.17436 shutdown 2024-05-08T21:26:09.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1400971583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1540 msgr2=0x55f579debf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/1400971583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1540 0x55f579debf10 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0a1fc0 tx=0x7f1d5c0999d0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1400971583 shutdown_connections 2024-05-08T21:26:09.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/1400971583 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f579df7c70 0x55f579dfa090 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/1400971583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579df3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/1400971583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1540 0x55f579debf10 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/1400971583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1ea0 0x55f579dec450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1400971583 >> 172.21.15.33:0/1400971583 conn(0x55f579d2f210 msgr2=0x55f579dcb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d497fa700 0 client.17436 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:26:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1400971583 shutdown_connections 2024-05-08T21:26:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d743d6b80 1 -- 172.21.15.33:0/1400971583 wait complete. 2024-05-08T21:26:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.124+0000 7f1d743d6b80 20 client.17436 trim_cache size 0 max 16384 2024-05-08T21:26:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.141+0000 7f1d743d6b80 1 Processor -- start 2024-05-08T21:26:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.141+0000 7f1d743d6b80 1 -- start start 2024-05-08T21:26:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.141+0000 7f1d743d6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1540 0x55f579df3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.141+0000 7f1d743d6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1ea0 0x55f579df3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.141+0000 7f1d743d6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579dd8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.141+0000 7f1d743d6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f579dcd830 con 0x55f579dd1ea0 2024-05-08T21:26:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d743d6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f579dff280 con 0x55f579dd2860 2024-05-08T21:26:09.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d743d6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f579dd93b0 con 0x55f579dd1540 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d657b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1ea0 0x55f579df3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d667b9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579dd8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d657b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1ea0 0x55f579df3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52962/0 (socket says 172.21.15.33:52962) 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d657b7700 1 -- 172.21.15.33:0/3967669508 learned_addr learned my addr 172.21.15.33:0/3967669508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d65fb8700 1 --2- 172.21.15.33:0/3967669508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1540 0x55f579df3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d667b9700 1 -- 172.21.15.33:0/3967669508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1540 msgr2=0x55f579df3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d667b9700 1 --2- 172.21.15.33:0/3967669508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1540 0x55f579df3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d667b9700 1 -- 172.21.15.33:0/3967669508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1ea0 msgr2=0x55f579df3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d667b9700 1 --2- 172.21.15.33:0/3967669508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1ea0 0x55f579df3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d667b9700 1 -- 172.21.15.33:0/3967669508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f579ce7560 con 0x55f579dd2860 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d65fb8700 1 --2- 172.21.15.33:0/3967669508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1540 0x55f579df3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.142+0000 7f1d667b9700 1 --2- 172.21.15.33:0/3967669508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579dd8e70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1d58003eb0 tx=0x7f1d58003fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1016cecdd26b83b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.143+0000 7f1d54ff9700 1 -- 172.21.15.33:0/3967669508 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58004b00 con 0x55f579dd2860 2024-05-08T21:26:09.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.143+0000 7f1d54ff9700 1 -- 172.21.15.33:0/3967669508 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d58004c60 con 0x55f579dd2860 2024-05-08T21:26:09.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.143+0000 7f1d54ff9700 1 -- 172.21.15.33:0/3967669508 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800e730 con 0x55f579dd2860 2024-05-08T21:26:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.143+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3967669508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 msgr2=0x55f579dd8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.143+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/3967669508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579dd8e70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1d58003eb0 tx=0x7f1d58003fd0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.144+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3967669508 shutdown_connections 2024-05-08T21:26:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.144+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/3967669508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd2860 0x55f579dd8e70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.144+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/3967669508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1540 0x55f579df3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.144+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/3967669508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd1ea0 0x55f579df3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.144+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3967669508 >> 172.21.15.33:0/3967669508 conn(0x55f579d2f210 msgr2=0x55f579dd2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.144+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3967669508 shutdown_connections 2024-05-08T21:26:09.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.144+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3967669508 wait complete. 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.144+0000 7f1d743d6b80 1 -- start start 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.144+0000 7f1d743d6b80 1 Processor -- start 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d743d6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd1540 0x55f579deee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d743d6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1ea0 0x55f579def3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d743d6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd2860 0x55f579df2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d743d6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f579df2860 con 0x55f579dd2860 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d743d6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f579df29d0 con 0x55f579dd1540 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d743d6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f579df2b90 con 0x55f579dd1ea0 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d743d6b80 10 client.0 _ll_register_callbacks cb 0x55f579cc6b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d65fb8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd1540 0x55f579deee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d65fb8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd1540 0x55f579deee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:58652/0 (socket says 172.21.15.33:58652) 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d65fb8700 1 -- 172.21.15.33:0/3102122427 learned_addr learned my addr 172.21.15.33:0/3102122427 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d657b7700 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1ea0 0x55f579def3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d65fb8700 1 -- 172.21.15.33:0/3102122427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1ea0 msgr2=0x55f579def3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d65fb8700 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1ea0 0x55f579def3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d65fb8700 1 -- 172.21.15.33:0/3102122427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd2860 msgr2=0x55f579df2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d65fb8700 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd2860 0x55f579df2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d65fb8700 1 -- 172.21.15.33:0/3102122427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f579b4eb60 con 0x55f579dd1540 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d657b7700 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1ea0 0x55f579def3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d65fb8700 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd1540 0x55f579deee60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c09a000 tx=0x7f1d5c005c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe32353f2bc6b9bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d497fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.145+0000 7f1d497fa700 1 -- 172.21.15.33:0/3102122427 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c09a070 con 0x55f579dd1540 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d497fa700 1 -- 172.21.15.33:0/3102122427 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1d5c0a24f0 con 0x55f579dd1540 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3102122427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d5800f040 con 0x55f579dd1540 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d497fa700 1 -- 172.21.15.33:0/3102122427 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d5c0cabe0 con 0x55f579dd1540 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d743d6b80 10 client.27347 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3102122427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f579ba23b0 con 0x55f579dd1540 2024-05-08T21:26:09.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d497fa700 1 -- 172.21.15.33:0/3102122427 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d5800f040 con 0x55f579dd1540 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d497fa700 1 -- 172.21.15.33:0/3102122427 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1d5c07c070 con 0x55f579dd1540 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d743d6b80 10 client.27347 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d743d6b80 10 client.27347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d743d6b80 20 client.27347 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f579e4e710 0x55f579e50b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d743d6b80 4 client.27347 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3102122427 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f579d19ec0 con 0x55f579e4e710 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.146+0000 7f1d657b7700 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f579e4e710 0x55f579e50b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d497fa700 10 client.27347 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d657b7700 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f579e4e710 0x55f579e50b30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe1e142259035045 server_cookie=82706d713c82c701 in_seq=0 out_seq=0 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d497fa700 1 -- 172.21.15.33:0/3102122427 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f579d19ec0 con 0x55f579e4e710 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d497fa700 10 client.27347 handle_command_reply: tid=0 2024-05-08T21:26:09.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d743d6b80 1 client.27347 shutdown 2024-05-08T21:26:09.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3102122427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd1540 msgr2=0x55f579deee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd1540 0x55f579deee60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c09a000 tx=0x7f1d5c005c60 comp rx=0 tx=0).stop 2024-05-08T21:26:09.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3102122427 shutdown_connections 2024-05-08T21:26:09.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55f579e4e710 0x55f579e50b30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f579dd2860 0x55f579df2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f579dd1540 0x55f579deee60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d743d6b80 1 --2- 172.21.15.33:0/3102122427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f579dd1ea0 0x55f579def3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d497fa700 0 client.27347 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:26:09.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3102122427 >> 172.21.15.33:0/3102122427 conn(0x55f579d2f210 msgr2=0x55f579dcbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3102122427 shutdown_connections 2024-05-08T21:26:09.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d743d6b80 1 -- 172.21.15.33:0/3102122427 wait complete. 2024-05-08T21:26:09.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.147+0000 7f1d743d6b80 20 client.27347 trim_cache size 0 max 16384 2024-05-08T21:26:09.154 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:26:09.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.148+0000 7f1d6821c700 1 -- 172.21.15.33:0/382070322 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1d4c0a1120 msgr2=0x7f1d4c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.148+0000 7f1d6821c700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1d4c0a1120 0x7f1d4c0a35e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000b2e0 tx=0x7f1d5000b630 comp rx=0 tx=0).stop 2024-05-08T21:26:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.148+0000 7f1d6821c700 1 -- 172.21.15.33:0/382070322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d601019a0 msgr2=0x7f1d601012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.148+0000 7f1d6821c700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d601019a0 0x7f1d601012c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e9f0 tx=0x7f1d5c00ed00 comp rx=0 tx=0).stop 2024-05-08T21:26:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.148+0000 7f1d6821c700 1 -- 172.21.15.33:0/382070322 shutdown_connections 2024-05-08T21:26:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.148+0000 7f1d6821c700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1d4c0a1120 0x7f1d4c0a35e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.148+0000 7f1d6821c700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d60106c00 0x7f1d600ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.148+0000 7f1d6821c700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d601019a0 0x7f1d601012c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.148+0000 7f1d6821c700 1 --2- 172.21.15.33:0/382070322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d601042d0 0x7f1d600ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.148+0000 7f1d6821c700 1 -- 172.21.15.33:0/382070322 >> 172.21.15.33:0/382070322 conn(0x7f1d600fb330 msgr2=0x7f1d60103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.148+0000 7f1d6821c700 1 -- 172.21.15.33:0/382070322 shutdown_connections 2024-05-08T21:26:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.148+0000 7f1d6821c700 1 -- 172.21.15.33:0/382070322 wait complete. 2024-05-08T21:26:09.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:26:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.345+0000 7f24f5bfc700 1 -- 172.21.15.33:0/930610202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24f0101fc0 msgr2=0x7f24f0102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.345+0000 7f24f5bfc700 1 --2- 172.21.15.33:0/930610202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24f0101fc0 0x7f24f0102440 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009990 tx=0x7f24e4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.345+0000 7f24f5bfc700 1 -- 172.21.15.33:0/930610202 shutdown_connections 2024-05-08T21:26:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.345+0000 7f24f5bfc700 1 --2- 172.21.15.33:0/930610202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f0102980 0x7f24f01092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.345+0000 7f24f5bfc700 1 --2- 172.21.15.33:0/930610202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f0100e70 0x7f24f0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.345+0000 7f24f5bfc700 1 --2- 172.21.15.33:0/930610202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24f0101fc0 0x7f24f0102440 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.345+0000 7f24f5bfc700 1 -- 172.21.15.33:0/930610202 >> 172.21.15.33:0/930610202 conn(0x7f24f00fc400 msgr2=0x7f24f00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.345+0000 7f24f5bfc700 1 -- 172.21.15.33:0/930610202 shutdown_connections 2024-05-08T21:26:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.345+0000 7f24f5bfc700 1 -- 172.21.15.33:0/930610202 wait complete. 2024-05-08T21:26:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.346+0000 7f24f5bfc700 1 Processor -- start 2024-05-08T21:26:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.346+0000 7f24f5bfc700 1 -- start start 2024-05-08T21:26:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.346+0000 7f24f5bfc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f0100e70 0x7f24f0196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.346+0000 7f24f5bfc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24f0101fc0 0x7f24f01972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.347+0000 7f24f5bfc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f0102980 0x7f24f019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.347+0000 7f24f5bfc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f019e850 con 0x7f24f0102980 2024-05-08T21:26:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.347+0000 7f24f5bfc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f006a9f0 con 0x7f24f0101fc0 2024-05-08T21:26:09.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.347+0000 7f24f5bfc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f006abb0 con 0x7f24f0100e70 2024-05-08T21:26:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.348+0000 7f24effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f0102980 0x7f24f019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.348+0000 7f24effff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f0102980 0x7f24f019e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52994/0 (socket says 172.21.15.33:52994) 2024-05-08T21:26:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.348+0000 7f24effff700 1 -- 172.21.15.33:0/1357672709 learned_addr learned my addr 172.21.15.33:0/1357672709 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.348+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f0100e70 0x7f24f0196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.349+0000 7f24ef7fe700 1 -- 172.21.15.33:0/1357672709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24f0101fc0 msgr2=0x7f24f01972a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:26:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.349+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24f0101fc0 0x7f24f01972a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.349+0000 7f24ef7fe700 1 -- 172.21.15.33:0/1357672709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f0102980 msgr2=0x7f24f019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.349+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f0102980 0x7f24f019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.349+0000 7f24ef7fe700 1 -- 172.21.15.33:0/1357672709 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e40095f0 con 0x7f24f0100e70 2024-05-08T21:26:09.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.350+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f0100e70 0x7f24f0196d60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f24e000e9f0 tx=0x7f24e000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.351+0000 7f24ecff9700 1 -- 172.21.15.33:0/1357672709 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000eee0 con 0x7f24f0100e70 2024-05-08T21:26:09.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.351+0000 7f24ecff9700 1 -- 172.21.15.33:0/1357672709 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24e000cbd0 con 0x7f24f0100e70 2024-05-08T21:26:09.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.351+0000 7f24f5bfc700 1 -- 172.21.15.33:0/1357672709 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f006ae90 con 0x7f24f0100e70 2024-05-08T21:26:09.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.351+0000 7f24ecff9700 1 -- 172.21.15.33:0/1357672709 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0018680 con 0x7f24f0100e70 2024-05-08T21:26:09.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.351+0000 7f24f5bfc700 1 -- 172.21.15.33:0/1357672709 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f006b3f0 con 0x7f24f0100e70 2024-05-08T21:26:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.356+0000 7f2501db6b80 1 Processor -- start 2024-05-08T21:26:09.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.357+0000 7f24ecff9700 1 -- 172.21.15.33:0/1357672709 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f24e00187e0 con 0x7f24f0100e70 2024-05-08T21:26:09.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.358+0000 7f24ecff9700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f24d80a11a0 0x7f24d80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.359+0000 7f24ecff9700 1 -- 172.21.15.33:0/1357672709 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f24e0014070 con 0x7f24f0100e70 2024-05-08T21:26:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.359+0000 7f2501db6b80 1 -- start start 2024-05-08T21:26:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.359+0000 7f2501db6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353540 0x56517f353960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.359+0000 7f2501db6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f353ea0 0x56517f354320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.359+0000 7f2501db6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f354860 0x56517f354ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.359+0000 7f2501db6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517f2aa150 con 0x56517f354860 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.359+0000 7f2501db6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517f2a7b10 con 0x56517f353540 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.359+0000 7f2501db6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517f2a3400 con 0x56517f353ea0 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.360+0000 7f24ef7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353540 0x56517f353960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.360+0000 7f24ef7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353540 0x56517f353960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:58670/0 (socket says 172.21.15.33:58670) 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.360+0000 7f24ef7fe700 1 -- 172.21.15.33:0/2097089264 learned_addr learned my addr 172.21.15.33:0/2097089264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.360+0000 7f24effff700 1 --2- 172.21.15.33:0/2097089264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f354860 0x56517f354ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.360+0000 7f24eeffd700 1 --2- 172.21.15.33:0/2097089264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f353ea0 0x56517f354320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.360+0000 7f24ef7fe700 1 -- 172.21.15.33:0/2097089264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f353ea0 msgr2=0x56517f354320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.360+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/2097089264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f353ea0 0x56517f354320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.360+0000 7f24ef7fe700 1 -- 172.21.15.33:0/2097089264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f354860 msgr2=0x56517f354ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.360+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/2097089264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f354860 0x56517f354ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.360+0000 7f24ef7fe700 1 -- 172.21.15.33:0/2097089264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56517f1243b0 con 0x56517f353540 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.360+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/2097089264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353540 0x56517f353960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f24e00a2180 tx=0x7f24e0099ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=585ecbc88dafe2b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.360+0000 7f24d5ffb700 1 -- 172.21.15.33:0/2097089264 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e009a230 con 0x56517f353540 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.361+0000 7f24effff700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f24d80a11a0 0x7f24d80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.361+0000 7f24d5ffb700 1 -- 172.21.15.33:0/2097089264 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24e009e060 con 0x56517f353540 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.361+0000 7f24d5ffb700 1 -- 172.21.15.33:0/2097089264 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0099980 con 0x56517f353540 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.362+0000 7f2501db6b80 1 -- 172.21.15.33:0/2097089264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353540 msgr2=0x56517f353960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.362+0000 7f24effff700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f24d80a11a0 0x7f24d80a3660 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009d30 tx=0x7f24e8004b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.362+0000 7f2501db6b80 1 --2- 172.21.15.33:0/2097089264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353540 0x56517f353960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f24e00a2180 tx=0x7f24e0099ea0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.362+0000 7f2501db6b80 1 -- 172.21.15.33:0/2097089264 shutdown_connections 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.362+0000 7f2501db6b80 1 --2- 172.21.15.33:0/2097089264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f354860 0x56517f354ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.362+0000 7f2501db6b80 1 --2- 172.21.15.33:0/2097089264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353540 0x56517f353960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.362+0000 7f2501db6b80 1 --2- 172.21.15.33:0/2097089264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f353ea0 0x56517f354320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.362+0000 7f2501db6b80 1 -- 172.21.15.33:0/2097089264 >> 172.21.15.33:0/2097089264 conn(0x56517f2b1210 msgr2=0x56517f34dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.362+0000 7f2501db6b80 1 -- 172.21.15.33:0/2097089264 shutdown_connections 2024-05-08T21:26:09.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.362+0000 7f2501db6b80 1 -- 172.21.15.33:0/2097089264 wait complete. 2024-05-08T21:26:09.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.362+0000 7f2501db6b80 1 -- start start 2024-05-08T21:26:09.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f2501db6b80 1 Processor -- start 2024-05-08T21:26:09.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f2501db6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f353540 0x56517f35ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f2501db6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 0x56517f35d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f2501db6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f354860 0x56517f3643c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f2501db6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517f23f560 con 0x56517f353540 2024-05-08T21:26:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f2501db6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517f23f200 con 0x56517f353ea0 2024-05-08T21:26:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f2501db6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517f1266a0 con 0x56517f354860 2024-05-08T21:26:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f2501db6b80 10 client.0 _ll_register_callbacks cb 0x56517f248b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f24ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f353540 0x56517f35ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f24ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f353540 0x56517f35ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53016/0 (socket says 172.21.15.33:53016) 2024-05-08T21:26:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f24ef7fe700 1 -- 172.21.15.33:0/993926661 learned_addr learned my addr 172.21.15.33:0/993926661 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f24eeffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 0x56517f35d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f24effff700 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f354860 0x56517f3643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f24ef7fe700 1 -- 172.21.15.33:0/993926661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f354860 msgr2=0x56517f3643c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.363+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f354860 0x56517f3643c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.364+0000 7f24ef7fe700 1 -- 172.21.15.33:0/993926661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 msgr2=0x56517f35d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.364+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 0x56517f35d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.364+0000 7f24ef7fe700 1 -- 172.21.15.33:0/993926661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f006b3f0 con 0x56517f353540 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.364+0000 7f24eeffd700 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 0x56517f35d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.364+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f353540 0x56517f35ce40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f24e0000c00 tx=0x7f24e00a26b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925aa2d1af86a6b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.364+0000 7f24ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.364+0000 7f24ceffd700 1 -- 172.21.15.33:0/993926661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e00a2420 con 0x56517f353540 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.364+0000 7f24ceffd700 1 -- 172.21.15.33:0/993926661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f24e0094e40 con 0x56517f353540 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.364+0000 7f2501db6b80 1 -- 172.21.15.33:0/993926661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56517f1fc270 con 0x56517f353540 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.364+0000 7f24ceffd700 1 -- 172.21.15.33:0/993926661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24e009a2b0 con 0x56517f353540 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.364+0000 7f2501db6b80 10 client.17451 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.364+0000 7f2501db6b80 1 -- 172.21.15.33:0/993926661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56517f0d0b60 con 0x56517f353540 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.365+0000 7f24ceffd700 1 -- 172.21.15.33:0/993926661 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56517f1fc270 con 0x56517f353540 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.365+0000 7f24ceffd700 1 -- 172.21.15.33:0/993926661 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f24e0018a90 con 0x56517f353540 2024-05-08T21:26:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.365+0000 7f2501db6b80 10 client.17451 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.365+0000 7f2501db6b80 10 client.17451 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:26:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.365+0000 7f2501db6b80 20 client.17451 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.365+0000 7f2501db6b80 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x56517f367220 0x56517f367630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.365+0000 7f2501db6b80 4 client.17451 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.365+0000 7f2501db6b80 1 -- 172.21.15.33:0/993926661 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56517f322810 con 0x56517f367220 2024-05-08T21:26:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.365+0000 7f24eeffd700 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x56517f367220 0x56517f367630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.366+0000 7f24ceffd700 10 client.17451 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:26:09.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.366+0000 7f24eeffd700 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x56517f367220 0x56517f367630 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79b6be40322fc53d server_cookie=89f2cd7b9f57f13e in_seq=0 out_seq=0 2024-05-08T21:26:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.366+0000 7f24ceffd700 1 -- 172.21.15.33:0/993926661 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56517f322810 con 0x56517f367220 2024-05-08T21:26:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.366+0000 7f24ceffd700 10 client.17451 handle_command_reply: tid=0 2024-05-08T21:26:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.366+0000 7f2501db6b80 1 client.17451 shutdown 2024-05-08T21:26:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.367+0000 7f2501db6b80 1 -- 172.21.15.33:0/993926661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f353540 msgr2=0x56517f35ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.367+0000 7f2501db6b80 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f353540 0x56517f35ce40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f24e0000c00 tx=0x7f24e00a26b0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.367+0000 7f2501db6b80 1 -- 172.21.15.33:0/993926661 shutdown_connections 2024-05-08T21:26:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.367+0000 7f2501db6b80 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x56517f367220 0x56517f367630 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.367+0000 7f2501db6b80 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f354860 0x56517f3643c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.367+0000 7f2501db6b80 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f353540 0x56517f35ce40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.367+0000 7f2501db6b80 1 --2- 172.21.15.33:0/993926661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 0x56517f35d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.367+0000 7f2501db6b80 1 -- 172.21.15.33:0/993926661 >> 172.21.15.33:0/993926661 conn(0x56517f2b1210 msgr2=0x56517f34d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.367+0000 7f24ceffd700 0 client.17451 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:26:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.367+0000 7f2501db6b80 1 -- 172.21.15.33:0/993926661 shutdown_connections 2024-05-08T21:26:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.367+0000 7f2501db6b80 1 -- 172.21.15.33:0/993926661 wait complete. 2024-05-08T21:26:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.368+0000 7f2501db6b80 20 client.17451 trim_cache size 0 max 16384 2024-05-08T21:26:09.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.386+0000 7f2501db6b80 1 Processor -- start 2024-05-08T21:26:09.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.386+0000 7f2501db6b80 1 -- start start 2024-05-08T21:26:09.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.386+0000 7f2501db6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f353ea0 0x56517f35c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.386+0000 7f2501db6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f354860 0x56517f364350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.386+0000 7f2501db6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f35bb40 0x56517f378d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.386+0000 7f2501db6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517f35b960 con 0x56517f354860 2024-05-08T21:26:09.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.386+0000 7f2501db6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517f34adf0 con 0x56517f35bb40 2024-05-08T21:26:09.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.386+0000 7f2501db6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517f34f5f0 con 0x56517f353ea0 2024-05-08T21:26:09.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.386+0000 7f24eeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f354860 0x56517f364350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.386+0000 7f24eeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f354860 0x56517f364350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53020/0 (socket says 172.21.15.33:53020) 2024-05-08T21:26:09.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.386+0000 7f24effff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f35bb40 0x56517f378d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.387+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/1264994002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f353ea0 0x56517f35c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.387+0000 7f24eeffd700 1 -- 172.21.15.33:0/1264994002 learned_addr learned my addr 172.21.15.33:0/1264994002 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.387+0000 7f24ef7fe700 1 -- 172.21.15.33:0/1264994002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f35bb40 msgr2=0x56517f378d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.387+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/1264994002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f35bb40 0x56517f378d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.387+0000 7f24ef7fe700 1 -- 172.21.15.33:0/1264994002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f354860 msgr2=0x56517f364350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.387+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/1264994002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f354860 0x56517f364350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.387+0000 7f24ef7fe700 1 -- 172.21.15.33:0/1264994002 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56517f269560 con 0x56517f353ea0 2024-05-08T21:26:09.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.387+0000 7f24eeffd700 1 --2- 172.21.15.33:0/1264994002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f354860 0x56517f364350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:09.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.387+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/1264994002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f353ea0 0x56517f35c380 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f24e009a1c0 tx=0x7f24e00987e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c282426761a53ec7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.387+0000 7f24d67fc700 1 -- 172.21.15.33:0/1264994002 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e00a2720 con 0x56517f353ea0 2024-05-08T21:26:09.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.387+0000 7f24d67fc700 1 -- 172.21.15.33:0/1264994002 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24e0010e90 con 0x56517f353ea0 2024-05-08T21:26:09.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.387+0000 7f24d67fc700 1 -- 172.21.15.33:0/1264994002 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e009f0a0 con 0x56517f353ea0 2024-05-08T21:26:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.388+0000 7f2501db6b80 1 -- 172.21.15.33:0/1264994002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f353ea0 msgr2=0x56517f35c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.388+0000 7f2501db6b80 1 --2- 172.21.15.33:0/1264994002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f353ea0 0x56517f35c380 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f24e009a1c0 tx=0x7f24e00987e0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.388+0000 7f2501db6b80 1 -- 172.21.15.33:0/1264994002 shutdown_connections 2024-05-08T21:26:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.388+0000 7f2501db6b80 1 --2- 172.21.15.33:0/1264994002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f35bb40 0x56517f378d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.388+0000 7f2501db6b80 1 --2- 172.21.15.33:0/1264994002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f353ea0 0x56517f35c380 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.388+0000 7f2501db6b80 1 --2- 172.21.15.33:0/1264994002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f354860 0x56517f364350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.388+0000 7f2501db6b80 1 -- 172.21.15.33:0/1264994002 >> 172.21.15.33:0/1264994002 conn(0x56517f2b1210 msgr2=0x56517f354c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.388+0000 7f2501db6b80 1 -- 172.21.15.33:0/1264994002 shutdown_connections 2024-05-08T21:26:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.388+0000 7f2501db6b80 1 -- 172.21.15.33:0/1264994002 wait complete. 2024-05-08T21:26:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.389+0000 7f2501db6b80 1 -- start start 2024-05-08T21:26:09.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.389+0000 7f2501db6b80 1 Processor -- start 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.389+0000 7f2501db6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 0x56517f35e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.389+0000 7f2501db6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f354860 0x56517f35eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f2501db6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f35bb40 0x56517f3639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f2501db6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517f363ef0 con 0x56517f35bb40 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f2501db6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517f364060 con 0x56517f353ea0 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f2501db6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517f364220 con 0x56517f354860 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f2501db6b80 10 client.0 _ll_register_callbacks cb 0x56517f248b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24eeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f354860 0x56517f35eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24ef7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 0x56517f35e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24ef7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 0x56517f35e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:58708/0 (socket says 172.21.15.33:58708) 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24eeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f354860 0x56517f35eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39798/0 (socket says 172.21.15.33:39798) 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24ef7fe700 1 -- 172.21.15.33:0/1455924979 learned_addr learned my addr 172.21.15.33:0/1455924979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24effff700 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f35bb40 0x56517f3639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24ef7fe700 1 -- 172.21.15.33:0/1455924979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f354860 msgr2=0x56517f35eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f354860 0x56517f35eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24ef7fe700 1 -- 172.21.15.33:0/1455924979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f35bb40 msgr2=0x56517f3639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f35bb40 0x56517f3639b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24ef7fe700 1 -- 172.21.15.33:0/1455924979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f0109bd0 con 0x56517f353ea0 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24eeffd700 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f354860 0x56517f35eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24ef7fe700 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 0x56517f35e560 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f24e000c5b0 tx=0x7f24e009ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c13601a57c778104 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.390+0000 7f24ceffd700 1 -- 172.21.15.33:0/1455924979 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0096020 con 0x56517f353ea0 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f24ceffd700 1 -- 172.21.15.33:0/1455924979 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f24e0094e40 con 0x56517f353ea0 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f2501db6b80 1 -- 172.21.15.33:0/1455924979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24e009e060 con 0x56517f353ea0 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f24ceffd700 1 -- 172.21.15.33:0/1455924979 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24e0096180 con 0x56517f353ea0 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f2501db6b80 10 client.27362 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f2501db6b80 1 -- 172.21.15.33:0/1455924979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56517f0d0b60 con 0x56517f353ea0 2024-05-08T21:26:09.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f24ceffd700 1 -- 172.21.15.33:0/1455924979 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24e009e060 con 0x56517f353ea0 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f24ceffd700 1 -- 172.21.15.33:0/1455924979 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f24e009abc0 con 0x56517f353ea0 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f2501db6b80 10 client.27362 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f2501db6b80 10 client.27362 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f2501db6b80 20 client.27362 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f2501db6b80 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x56517f3c7320 0x56517f3c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f2501db6b80 4 client.27362 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f2501db6b80 1 -- 172.21.15.33:0/1455924979 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56517f29bec0 con 0x56517f3c7320 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.391+0000 7f24eeffd700 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x56517f3c7320 0x56517f3c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.392+0000 7f24eeffd700 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x56517f3c7320 0x56517f3c9710 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0e831195441b50f server_cookie=fee692292b755419 in_seq=0 out_seq=0 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.392+0000 7f24ceffd700 10 client.27362 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.392+0000 7f24ceffd700 1 -- 172.21.15.33:0/1455924979 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56517f29bec0 con 0x56517f3c7320 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.392+0000 7f24ceffd700 10 client.27362 handle_command_reply: tid=0 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.392+0000 7f2501db6b80 1 client.27362 shutdown 2024-05-08T21:26:09.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.393+0000 7f2501db6b80 1 -- 172.21.15.33:0/1455924979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 msgr2=0x56517f35e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.393+0000 7f2501db6b80 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 0x56517f35e560 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f24e000c5b0 tx=0x7f24e009ef80 comp rx=0 tx=0).stop 2024-05-08T21:26:09.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.393+0000 7f2501db6b80 1 -- 172.21.15.33:0/1455924979 shutdown_connections 2024-05-08T21:26:09.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.393+0000 7f2501db6b80 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x56517f3c7320 0x56517f3c9710 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.393+0000 7f2501db6b80 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56517f35bb40 0x56517f3639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.393+0000 7f24ceffd700 0 client.27362 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:26:09.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.393+0000 7f2501db6b80 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56517f353ea0 0x56517f35e560 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.393+0000 7f2501db6b80 1 --2- 172.21.15.33:0/1455924979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56517f354860 0x56517f35eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.393+0000 7f2501db6b80 1 -- 172.21.15.33:0/1455924979 >> 172.21.15.33:0/1455924979 conn(0x56517f2b1210 msgr2=0x56517f35c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.393+0000 7f2501db6b80 1 -- 172.21.15.33:0/1455924979 shutdown_connections 2024-05-08T21:26:09.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f2501db6b80 1 -- 172.21.15.33:0/1455924979 wait complete. 2024-05-08T21:26:09.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f2501db6b80 20 client.27362 trim_cache size 0 max 16384 2024-05-08T21:26:09.401 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:26:09.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f24f5bfc700 1 -- 172.21.15.33:0/1357672709 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f24d80a11a0 msgr2=0x7f24d80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f24f5bfc700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f24d80a11a0 0x7f24d80a3660 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009d30 tx=0x7f24e8004b40 comp rx=0 tx=0).stop 2024-05-08T21:26:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f24f5bfc700 1 -- 172.21.15.33:0/1357672709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f0100e70 msgr2=0x7f24f0196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f24f5bfc700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f0100e70 0x7f24f0196d60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f24e000e9f0 tx=0x7f24e000ed00 comp rx=0 tx=0).stop 2024-05-08T21:26:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f24f5bfc700 1 -- 172.21.15.33:0/1357672709 shutdown_connections 2024-05-08T21:26:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f24f5bfc700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f0102980 0x7f24f019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f24f5bfc700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24f0100e70 0x7f24f0196d60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f24f5bfc700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f24d80a11a0 0x7f24d80a3660 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f24f5bfc700 1 --2- 172.21.15.33:0/1357672709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24f0101fc0 0x7f24f01972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f24f5bfc700 1 -- 172.21.15.33:0/1357672709 >> 172.21.15.33:0/1357672709 conn(0x7f24f00fc400 msgr2=0x7f24f0105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f24f5bfc700 1 -- 172.21.15.33:0/1357672709 shutdown_connections 2024-05-08T21:26:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.394+0000 7f24f5bfc700 1 -- 172.21.15.33:0/1357672709 wait complete. 2024-05-08T21:26:09.412 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:26:09.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.596+0000 7fcac75df700 1 -- 172.21.15.33:0/844475302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0100f80 msgr2=0x7fcac01013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.596+0000 7fcac75df700 1 --2- 172.21.15.33:0/844475302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0100f80 0x7fcac01013e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcab00099c0 tx=0x7fcab0009cd0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.596+0000 7fcac75df700 1 -- 172.21.15.33:0/844475302 shutdown_connections 2024-05-08T21:26:09.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.596+0000 7fcac75df700 1 --2- 172.21.15.33:0/844475302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac0101920 0x7fcac01081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.596+0000 7fcac75df700 1 --2- 172.21.15.33:0/844475302 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac00ffd90 0x7fcac01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.596+0000 7fcac75df700 1 --2- 172.21.15.33:0/844475302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0100f80 0x7fcac01013e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.596+0000 7fcac75df700 1 -- 172.21.15.33:0/844475302 >> 172.21.15.33:0/844475302 conn(0x7fcac00fb300 msgr2=0x7fcac00fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.596+0000 7fcac75df700 1 -- 172.21.15.33:0/844475302 shutdown_connections 2024-05-08T21:26:09.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.596+0000 7fcac75df700 1 -- 172.21.15.33:0/844475302 wait complete. 2024-05-08T21:26:09.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.597+0000 7fcac75df700 1 Processor -- start 2024-05-08T21:26:09.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.597+0000 7fcac75df700 1 -- start start 2024-05-08T21:26:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.597+0000 7fcac75df700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac00ffd90 0x7fcac0103e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.597+0000 7fcac75df700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac0100f80 0x7fcac0102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.597+0000 7fcac75df700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0101920 0x7fcac0102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.597+0000 7fcac75df700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac0103190 con 0x7fcac0101920 2024-05-08T21:26:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.597+0000 7fcac75df700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac0103300 con 0x7fcac00ffd90 2024-05-08T21:26:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.597+0000 7fcac75df700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac01a0dd0 con 0x7fcac0100f80 2024-05-08T21:26:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac537b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac00ffd90 0x7fcac0103e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac5b7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0101920 0x7fcac0102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac537b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac00ffd90 0x7fcac0103e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:58726/0 (socket says 172.21.15.33:58726) 2024-05-08T21:26:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac4b7a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac0100f80 0x7fcac0102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac5b7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0101920 0x7fcac0102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53030/0 (socket says 172.21.15.33:53030) 2024-05-08T21:26:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac537b700 1 -- 172.21.15.33:0/2833343022 learned_addr learned my addr 172.21.15.33:0/2833343022 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac537b700 1 -- 172.21.15.33:0/2833343022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac0100f80 msgr2=0x7fcac0102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac537b700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac0100f80 0x7fcac0102550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac537b700 1 -- 172.21.15.33:0/2833343022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0101920 msgr2=0x7fcac0102a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac537b700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0101920 0x7fcac0102a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac537b700 1 -- 172.21.15.33:0/2833343022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00095f0 con 0x7fcac00ffd90 2024-05-08T21:26:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac5b7c700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0101920 0x7fcac0102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac537b700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac00ffd90 0x7fcac0103e70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00e970 tx=0x7fcabc00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcab67fc700 1 -- 172.21.15.33:0/2833343022 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc009790 con 0x7fcac00ffd90 2024-05-08T21:26:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcab67fc700 1 -- 172.21.15.33:0/2833343022 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcabc00c7e0 con 0x7fcac00ffd90 2024-05-08T21:26:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac75df700 1 -- 172.21.15.33:0/2833343022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac01a0fd0 con 0x7fcac00ffd90 2024-05-08T21:26:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcab67fc700 1 -- 172.21.15.33:0/2833343022 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc003bd0 con 0x7fcac00ffd90 2024-05-08T21:26:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.598+0000 7fcac75df700 1 -- 172.21.15.33:0/2833343022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac01a1450 con 0x7fcac00ffd90 2024-05-08T21:26:09.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.601+0000 7fcab67fc700 1 -- 172.21.15.33:0/2833343022 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcabc00c950 con 0x7fcac00ffd90 2024-05-08T21:26:09.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.602+0000 7fcab67fc700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcaac0a1120 0x7fcaac0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.602+0000 7fcab67fc700 1 -- 172.21.15.33:0/2833343022 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcabc014070 con 0x7fcac00ffd90 2024-05-08T21:26:09.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.602+0000 7fcac4b7a700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcaac0a1120 0x7fcaac0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcac4b7a700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcaac0a1120 0x7fcaac0a35e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcab0000c00 tx=0x7fcab0003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcad3799b80 1 Processor -- start 2024-05-08T21:26:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcad3799b80 1 -- start start 2024-05-08T21:26:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcad3799b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4e540 0x55e2cdd4e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcad3799b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd4f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcad3799b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4f860 0x55e2cdd4fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcad3799b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cdca5150 con 0x55e2cdd4eea0 2024-05-08T21:26:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcad3799b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cdca2b10 con 0x55e2cdd4f860 2024-05-08T21:26:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcad3799b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cdc9e400 con 0x55e2cdd4e540 2024-05-08T21:26:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcac537b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4e540 0x55e2cdd4e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcac537b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4e540 0x55e2cdd4e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39832/0 (socket says 172.21.15.33:39832) 2024-05-08T21:26:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcac4b7a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd4f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.603+0000 7fcac537b700 1 -- 172.21.15.33:0/2479803547 learned_addr learned my addr 172.21.15.33:0/2479803547 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.604+0000 7fcac5b7c700 1 --2- 172.21.15.33:0/2479803547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4f860 0x55e2cdd4fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.604+0000 7fcac537b700 1 -- 172.21.15.33:0/2479803547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4f860 msgr2=0x55e2cdd4fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.604+0000 7fcac537b700 1 --2- 172.21.15.33:0/2479803547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4f860 0x55e2cdd4fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.604+0000 7fcac537b700 1 -- 172.21.15.33:0/2479803547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4eea0 msgr2=0x55e2cdd4f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.604+0000 7fcac537b700 1 --2- 172.21.15.33:0/2479803547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd4f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.604+0000 7fcac537b700 1 -- 172.21.15.33:0/2479803547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2cdb1f3b0 con 0x55e2cdd4e540 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.604+0000 7fcac537b700 1 --2- 172.21.15.33:0/2479803547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4e540 0x55e2cdd4e960 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0963d0 tx=0x7fcabc005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1ee1bcde7118950 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.604+0000 7fcaab7fe700 1 -- 172.21.15.33:0/2479803547 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc09a5d0 con 0x55e2cdd4e540 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.604+0000 7fcaab7fe700 1 -- 172.21.15.33:0/2479803547 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcabc099ed0 con 0x55e2cdd4e540 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.604+0000 7fcaab7fe700 1 -- 172.21.15.33:0/2479803547 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc09a130 con 0x55e2cdd4e540 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.605+0000 7fcad3799b80 1 -- 172.21.15.33:0/2479803547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4e540 msgr2=0x55e2cdd4e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.605+0000 7fcad3799b80 1 --2- 172.21.15.33:0/2479803547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4e540 0x55e2cdd4e960 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0963d0 tx=0x7fcabc005910 comp rx=0 tx=0).stop 2024-05-08T21:26:09.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.606+0000 7fcad3799b80 1 -- 172.21.15.33:0/2479803547 shutdown_connections 2024-05-08T21:26:09.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.606+0000 7fcad3799b80 1 --2- 172.21.15.33:0/2479803547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4f860 0x55e2cdd4fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.606+0000 7fcad3799b80 1 --2- 172.21.15.33:0/2479803547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4e540 0x55e2cdd4e960 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.606+0000 7fcad3799b80 1 --2- 172.21.15.33:0/2479803547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd4f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.606+0000 7fcad3799b80 1 -- 172.21.15.33:0/2479803547 >> 172.21.15.33:0/2479803547 conn(0x55e2cdcac210 msgr2=0x55e2cdd48de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.607+0000 7fcad3799b80 1 -- 172.21.15.33:0/2479803547 shutdown_connections 2024-05-08T21:26:09.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.607+0000 7fcad3799b80 1 -- 172.21.15.33:0/2479803547 wait complete. 2024-05-08T21:26:09.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.608+0000 7fcad3799b80 1 -- start start 2024-05-08T21:26:09.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.608+0000 7fcad3799b80 1 Processor -- start 2024-05-08T21:26:09.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.608+0000 7fcad3799b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4e540 0x55e2cdd57e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.608+0000 7fcad3799b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.608+0000 7fcad3799b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd5f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.608+0000 7fcad3799b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cdc3a560 con 0x55e2cdd4e540 2024-05-08T21:26:09.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.608+0000 7fcad3799b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cdc3a200 con 0x55e2cdd4eea0 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.608+0000 7fcad3799b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cdb216a0 con 0x55e2cdd4f860 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.609+0000 7fcad3799b80 10 client.0 _ll_register_callbacks cb 0x55e2cdc43b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.609+0000 7fcac5b7c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd5f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.609+0000 7fcac5b7c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd5f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39842/0 (socket says 172.21.15.33:39842) 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.609+0000 7fcac5b7c700 1 -- 172.21.15.33:0/3291763261 learned_addr learned my addr 172.21.15.33:0/3291763261 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.609+0000 7fcac537b700 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4e540 0x55e2cdd57e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.609+0000 7fcac4b7a700 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.610+0000 7fcac5b7c700 1 -- 172.21.15.33:0/3291763261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4eea0 msgr2=0x55e2cdd583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.610+0000 7fcac5b7c700 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.610+0000 7fcac5b7c700 1 -- 172.21.15.33:0/3291763261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4e540 msgr2=0x55e2cdd57e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.610+0000 7fcac5b7c700 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4e540 0x55e2cdd57e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.610+0000 7fcac5b7c700 1 -- 172.21.15.33:0/3291763261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac01a1450 con 0x55e2cdd4f860 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.611+0000 7fcac5b7c700 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd5f3f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcab80046e0 tx=0x7fcab80049f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36d3cd64bb54c9bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.611+0000 7fcaa8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:26:09.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.611+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/3291763261 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8004bb0 con 0x55e2cdd4f860 2024-05-08T21:26:09.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.611+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/3291763261 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcab8004d10 con 0x55e2cdd4f860 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.612+0000 7fcad3799b80 1 -- 172.21.15.33:0/3291763261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2cdbf7270 con 0x55e2cdd4f860 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.612+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/3291763261 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcab800eaa0 con 0x55e2cdd4f860 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.613+0000 7fcad3799b80 10 client.27370 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.613+0000 7fcad3799b80 1 -- 172.21.15.33:0/3291763261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55e2cdacbb60 con 0x55e2cdd4f860 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.613+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/3291763261 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e2cdbf7270 con 0x55e2cdd4f860 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.613+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/3291763261 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcab801c030 con 0x55e2cdd4f860 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.614+0000 7fcad3799b80 10 client.27370 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.614+0000 7fcad3799b80 10 client.27370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.614+0000 7fcad3799b80 20 client.27370 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.614+0000 7fcad3799b80 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cdd623f0 0x55e2cdd64810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.614+0000 7fcad3799b80 4 client.27370 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.614+0000 7fcad3799b80 1 -- 172.21.15.33:0/3291763261 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2cdd1d810 con 0x55e2cdd623f0 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.614+0000 7fcac537b700 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cdd623f0 0x55e2cdd64810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.614+0000 7fcaa8ff9700 10 client.27370 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.614+0000 7fcac537b700 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cdd623f0 0x55e2cdd64810 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d1c76297ab2ca9a server_cookie=69443c56d89b68b9 in_seq=0 out_seq=0 2024-05-08T21:26:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.615+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/3291763261 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e2cdd1d810 con 0x55e2cdd623f0 2024-05-08T21:26:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.615+0000 7fcaa8ff9700 10 client.27370 handle_command_reply: tid=0 2024-05-08T21:26:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.615+0000 7fcad3799b80 1 client.27370 shutdown 2024-05-08T21:26:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.615+0000 7fcad3799b80 1 -- 172.21.15.33:0/3291763261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 msgr2=0x55e2cdd5f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.615+0000 7fcad3799b80 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd5f3f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcab80046e0 tx=0x7fcab80049f0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.616+0000 7fcad3799b80 1 -- 172.21.15.33:0/3291763261 shutdown_connections 2024-05-08T21:26:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.616+0000 7fcad3799b80 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cdd623f0 0x55e2cdd64810 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.616+0000 7fcad3799b80 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd5f3f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.616+0000 7fcad3799b80 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4e540 0x55e2cdd57e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.616+0000 7fcaa8ff9700 0 client.27370 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.616+0000 7fcad3799b80 1 --2- 172.21.15.33:0/3291763261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.616+0000 7fcad3799b80 1 -- 172.21.15.33:0/3291763261 >> 172.21.15.33:0/3291763261 conn(0x55e2cdcac210 msgr2=0x55e2cdd485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.617+0000 7fcad3799b80 1 -- 172.21.15.33:0/3291763261 shutdown_connections 2024-05-08T21:26:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.617+0000 7fcad3799b80 1 -- 172.21.15.33:0/3291763261 wait complete. 2024-05-08T21:26:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.617+0000 7fcad3799b80 20 client.27370 trim_cache size 0 max 16384 2024-05-08T21:26:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.633+0000 7fcad3799b80 1 Processor -- start 2024-05-08T21:26:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.633+0000 7fcad3799b80 1 -- start start 2024-05-08T21:26:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.633+0000 7fcad3799b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4e540 0x55e2cdd647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.633+0000 7fcad3799b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd5f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.633+0000 7fcad3799b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd48db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.633+0000 7fcad3799b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cdd4a570 con 0x55e2cdd4e540 2024-05-08T21:26:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.633+0000 7fcad3799b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cdd69a00 con 0x55e2cdd4eea0 2024-05-08T21:26:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.633+0000 7fcad3799b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cdd55a30 con 0x55e2cdd4f860 2024-05-08T21:26:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcac537b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4e540 0x55e2cdd647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcac537b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4e540 0x55e2cdd647e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53066/0 (socket says 172.21.15.33:53066) 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcac537b700 1 -- 172.21.15.33:0/1802176321 learned_addr learned my addr 172.21.15.33:0/1802176321 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcac4b7a700 1 --2- 172.21.15.33:0/1802176321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd5f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcac5b7c700 1 --2- 172.21.15.33:0/1802176321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd48db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcac5b7c700 1 -- 172.21.15.33:0/1802176321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4eea0 msgr2=0x55e2cdd5f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcac5b7c700 1 --2- 172.21.15.33:0/1802176321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd5f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcac5b7c700 1 -- 172.21.15.33:0/1802176321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4e540 msgr2=0x55e2cdd647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcac5b7c700 1 --2- 172.21.15.33:0/1802176321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4e540 0x55e2cdd647e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcac5b7c700 1 -- 172.21.15.33:0/1802176321 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2cdc64560 con 0x55e2cdd4f860 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcac537b700 1 --2- 172.21.15.33:0/1802176321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4e540 0x55e2cdd647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcac5b7c700 1 --2- 172.21.15.33:0/1802176321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd48db0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcab8028fd0 tx=0x7fcab8007e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdd4ca624316be7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.634+0000 7fcaabfff700 1 -- 172.21.15.33:0/1802176321 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8003a00 con 0x55e2cdd4f860 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.635+0000 7fcaabfff700 1 -- 172.21.15.33:0/1802176321 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcab800f040 con 0x55e2cdd4f860 2024-05-08T21:26:09.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.635+0000 7fcaabfff700 1 -- 172.21.15.33:0/1802176321 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8007760 con 0x55e2cdd4f860 2024-05-08T21:26:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.635+0000 7fcad3799b80 1 -- 172.21.15.33:0/1802176321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 msgr2=0x55e2cdd48db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.635+0000 7fcad3799b80 1 --2- 172.21.15.33:0/1802176321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd48db0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcab8028fd0 tx=0x7fcab8007e50 comp rx=0 tx=0).stop 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.635+0000 7fcad3799b80 1 -- 172.21.15.33:0/1802176321 shutdown_connections 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.635+0000 7fcad3799b80 1 --2- 172.21.15.33:0/1802176321 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd48db0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.635+0000 7fcad3799b80 1 --2- 172.21.15.33:0/1802176321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4e540 0x55e2cdd647e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.635+0000 7fcad3799b80 1 --2- 172.21.15.33:0/1802176321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd5f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.635+0000 7fcad3799b80 1 -- 172.21.15.33:0/1802176321 >> 172.21.15.33:0/1802176321 conn(0x55e2cdcac210 msgr2=0x55e2cdd4fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.635+0000 7fcad3799b80 1 -- 172.21.15.33:0/1802176321 shutdown_connections 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.636+0000 7fcad3799b80 1 -- 172.21.15.33:0/1802176321 wait complete. 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.636+0000 7fcad3799b80 1 -- start start 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.636+0000 7fcad3799b80 1 Processor -- start 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.637+0000 7fcad3799b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4e540 0x55e2cdd5ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.637+0000 7fcad3799b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd5b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.637+0000 7fcad3799b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd5e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.637+0000 7fcad3799b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cdd5e780 con 0x55e2cdd4eea0 2024-05-08T21:26:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.637+0000 7fcad3799b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cdd5e8f0 con 0x55e2cdd4e540 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.637+0000 7fcad3799b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2cdd5eab0 con 0x55e2cdd4f860 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.637+0000 7fcad3799b80 10 client.0 _ll_register_callbacks cb 0x55e2cdc43b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.637+0000 7fcac5b7c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd5e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.637+0000 7fcac5b7c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd5e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39864/0 (socket says 172.21.15.33:39864) 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.637+0000 7fcac5b7c700 1 -- 172.21.15.33:0/2464951940 learned_addr learned my addr 172.21.15.33:0/2464951940 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.637+0000 7fcac4b7a700 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd5b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.637+0000 7fcac537b700 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4e540 0x55e2cdd5ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.638+0000 7fcac4b7a700 1 -- 172.21.15.33:0/2464951940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 msgr2=0x55e2cdd5e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.638+0000 7fcac4b7a700 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd5e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.638+0000 7fcac4b7a700 1 -- 172.21.15.33:0/2464951940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4e540 msgr2=0x55e2cdd5ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.638+0000 7fcac4b7a700 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4e540 0x55e2cdd5ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.638+0000 7fcac4b7a700 1 -- 172.21.15.33:0/2464951940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2cdb1f3b0 con 0x55e2cdd4eea0 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.638+0000 7fcac537b700 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4e540 0x55e2cdd5ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.638+0000 7fcac4b7a700 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd5b270 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098e0 tx=0x7fcab0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcae661048adef3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.638+0000 7fcaa8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:26:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.639+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/2464951940 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab000f870 con 0x55e2cdd4eea0 2024-05-08T21:26:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.639+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/2464951940 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcab000f9d0 con 0x55e2cdd4eea0 2024-05-08T21:26:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.639+0000 7fcad3799b80 1 -- 172.21.15.33:0/2464951940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcabc099ed0 con 0x55e2cdd4eea0 2024-05-08T21:26:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.640+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/2464951940 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcab000fb90 con 0x55e2cdd4eea0 2024-05-08T21:26:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.640+0000 7fcad3799b80 10 client.17475 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.640+0000 7fcad3799b80 1 -- 172.21.15.33:0/2464951940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55e2cdacbb60 con 0x55e2cdd4eea0 2024-05-08T21:26:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.640+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/2464951940 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcabc099ed0 con 0x55e2cdd4eea0 2024-05-08T21:26:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.640+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/2464951940 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcab005c070 con 0x55e2cdd4eea0 2024-05-08T21:26:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.640+0000 7fcad3799b80 10 client.17475 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.640+0000 7fcad3799b80 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.640+0000 7fcad3799b80 20 client.17475 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.641+0000 7fcad3799b80 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cddc36c0 0x55e2cddc5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.641+0000 7fcad3799b80 4 client.17475 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:26:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.641+0000 7fcad3799b80 1 -- 172.21.15.33:0/2464951940 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e2cdc96ec0 con 0x55e2cddc36c0 2024-05-08T21:26:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.641+0000 7fcac537b700 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cddc36c0 0x55e2cddc5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.642+0000 7fcac537b700 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cddc36c0 0x55e2cddc5ae0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3840237714ed677 server_cookie=a68c2e63e0450b49 in_seq=0 out_seq=0 2024-05-08T21:26:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.642+0000 7fcaa8ff9700 10 client.17475 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.642+0000 7fcaa8ff9700 1 -- 172.21.15.33:0/2464951940 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e2cdc96ec0 con 0x55e2cddc36c0 2024-05-08T21:26:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.642+0000 7fcaa8ff9700 10 client.17475 handle_command_reply: tid=0 2024-05-08T21:26:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.642+0000 7fcad3799b80 1 client.17475 shutdown 2024-05-08T21:26:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.643+0000 7fcad3799b80 1 -- 172.21.15.33:0/2464951940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4eea0 msgr2=0x55e2cdd5b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.643+0000 7fcad3799b80 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd5b270 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098e0 tx=0x7fcab0009cd0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.643+0000 7fcad3799b80 1 -- 172.21.15.33:0/2464951940 shutdown_connections 2024-05-08T21:26:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.643+0000 7fcad3799b80 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e2cddc36c0 0x55e2cddc5ae0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.643+0000 7fcad3799b80 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e2cdd4f860 0x55e2cdd5e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.643+0000 7fcad3799b80 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2cdd4e540 0x55e2cdd5ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.643+0000 7fcad3799b80 1 --2- 172.21.15.33:0/2464951940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2cdd4eea0 0x55e2cdd5b270 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.643+0000 7fcad3799b80 1 -- 172.21.15.33:0/2464951940 >> 172.21.15.33:0/2464951940 conn(0x55e2cdcac210 msgr2=0x55e2cdd48d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.643+0000 7fcaa8ff9700 0 client.17475 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.643+0000 7fcad3799b80 1 -- 172.21.15.33:0/2464951940 shutdown_connections 2024-05-08T21:26:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.643+0000 7fcad3799b80 1 -- 172.21.15.33:0/2464951940 wait complete. 2024-05-08T21:26:09.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.643+0000 7fcad3799b80 20 client.17475 trim_cache size 0 max 16384 2024-05-08T21:26:09.646 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:26:09.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.644+0000 7fcac75df700 1 -- 172.21.15.33:0/2833343022 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcaac0a1120 msgr2=0x7fcaac0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.644+0000 7fcac75df700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcaac0a1120 0x7fcaac0a35e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcab0000c00 tx=0x7fcab0003680 comp rx=0 tx=0).stop 2024-05-08T21:26:09.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.644+0000 7fcac75df700 1 -- 172.21.15.33:0/2833343022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac00ffd90 msgr2=0x7fcac0103e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:09.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.644+0000 7fcac75df700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac00ffd90 0x7fcac0103e70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00e970 tx=0x7fcabc00ec80 comp rx=0 tx=0).stop 2024-05-08T21:26:09.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.644+0000 7fcac75df700 1 -- 172.21.15.33:0/2833343022 shutdown_connections 2024-05-08T21:26:09.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.644+0000 7fcac75df700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcac0101920 0x7fcac0102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.644+0000 7fcac75df700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcaac0a1120 0x7fcaac0a35e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.644+0000 7fcac75df700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcac00ffd90 0x7fcac0103e70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.644+0000 7fcac75df700 1 --2- 172.21.15.33:0/2833343022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac0100f80 0x7fcac0102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:09.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.644+0000 7fcac75df700 1 -- 172.21.15.33:0/2833343022 >> 172.21.15.33:0/2833343022 conn(0x7fcac00fb300 msgr2=0x7fcac0069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:09.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.644+0000 7fcac75df700 1 -- 172.21.15.33:0/2833343022 shutdown_connections 2024-05-08T21:26:09.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:09.644+0000 7fcac75df700 1 -- 172.21.15.33:0/2833343022 wait complete. 2024-05-08T21:26:09.661 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:09 smithi033 ceph-mon[42528]: pgmap v505: 129 pgs: 129 active+clean; 252 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 396 op/s 2024-05-08T21:26:09.758 INFO:tasks.workunit.client.0.smithi033.stdout:truncating to largest ever: 0x63e4f5a 2024-05-08T21:26:09.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:09 smithi071 ceph-mon[47074]: pgmap v505: 129 pgs: 129 active+clean; 252 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 396 op/s 2024-05-08T21:26:09.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:09 smithi082 ceph-mon[47512]: pgmap v505: 129 pgs: 129 active+clean; 252 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 396 op/s 2024-05-08T21:26:10.663 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:26:10.663 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:26:10.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.859+0000 7f1372368700 1 -- 172.21.15.33:0/3751267464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f136c1018f0 msgr2=0x7f136c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.859+0000 7f1372368700 1 --2- 172.21.15.33:0/3751267464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f136c1018f0 0x7f136c1081a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f136400b0d0 tx=0x7f136400b3e0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.860+0000 7f1372368700 1 -- 172.21.15.33:0/3751267464 shutdown_connections 2024-05-08T21:26:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.860+0000 7f1372368700 1 --2- 172.21.15.33:0/3751267464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f136c1018f0 0x7f136c1081a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.860+0000 7f1372368700 1 --2- 172.21.15.33:0/3751267464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f136c0ffd60 0x7f136c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.860+0000 7f1372368700 1 --2- 172.21.15.33:0/3751267464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f136c100f50 0x7f136c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.860+0000 7f1372368700 1 -- 172.21.15.33:0/3751267464 >> 172.21.15.33:0/3751267464 conn(0x7f136c0fb290 msgr2=0x7f136c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.860+0000 7f1372368700 1 -- 172.21.15.33:0/3751267464 shutdown_connections 2024-05-08T21:26:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.860+0000 7f1372368700 1 -- 172.21.15.33:0/3751267464 wait complete. 2024-05-08T21:26:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.861+0000 7f1372368700 1 Processor -- start 2024-05-08T21:26:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.861+0000 7f1372368700 1 -- start start 2024-05-08T21:26:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.861+0000 7f1372368700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f136c0ffd60 0x7f136c070d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.861+0000 7f1372368700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f136c100f50 0x7f136c071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.861+0000 7f1372368700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f136c1018f0 0x7f136c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.861+0000 7f1372368700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c068730 con 0x7f136c0ffd60 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.861+0000 7f1372368700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0688a0 con 0x7f136c100f50 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.861+0000 7f1372368700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c068a60 con 0x7f136c1018f0 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.862+0000 7f1370905700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f136c1018f0 0x7f136c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.862+0000 7f1370905700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f136c1018f0 0x7f136c068160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:46432/0 (socket says 172.21.15.33:46432) 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.862+0000 7f1370905700 1 -- 172.21.15.33:0/1044166587 learned_addr learned my addr 172.21.15.33:0/1044166587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.862+0000 7f136b7fe700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f136c100f50 0x7f136c071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.862+0000 7f1370905700 1 -- 172.21.15.33:0/1044166587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f136c100f50 msgr2=0x7f136c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.862+0000 7f1370905700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f136c100f50 0x7f136c071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.862+0000 7f1370905700 1 -- 172.21.15.33:0/1044166587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f136c0ffd60 msgr2=0x7f136c070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.862+0000 7f1370905700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f136c0ffd60 0x7f136c070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.862+0000 7f1370905700 1 -- 172.21.15.33:0/1044166587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1364009d00 con 0x7f136c1018f0 2024-05-08T21:26:10.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.862+0000 7f1370905700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f136c1018f0 0x7f136c068160 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1364007b60 tx=0x7f13640094f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.863+0000 7f13697fa700 1 -- 172.21.15.33:0/1044166587 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136400e040 con 0x7f136c1018f0 2024-05-08T21:26:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.863+0000 7f1372368700 1 -- 172.21.15.33:0/1044166587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c068c80 con 0x7f136c1018f0 2024-05-08T21:26:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.863+0000 7f1372368700 1 -- 172.21.15.33:0/1044166587 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c072230 con 0x7f136c1018f0 2024-05-08T21:26:10.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.863+0000 7f13697fa700 1 -- 172.21.15.33:0/1044166587 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1364012e10 con 0x7f136c1018f0 2024-05-08T21:26:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.863+0000 7f13697fa700 1 -- 172.21.15.33:0/1044166587 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136401b450 con 0x7f136c1018f0 2024-05-08T21:26:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.865+0000 7f13697fa700 1 -- 172.21.15.33:0/1044166587 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f136401b5b0 con 0x7f136c1018f0 2024-05-08T21:26:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.866+0000 7f13697fa700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13540a11a0 0x7f13540a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.866+0000 7f13697fa700 1 -- 172.21.15.33:0/1044166587 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f13640cea80 con 0x7f136c1018f0 2024-05-08T21:26:10.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.866+0000 7f136bfff700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13540a11a0 0x7f13540a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.867+0000 7f137e522b80 1 Processor -- start 2024-05-08T21:26:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.867+0000 7f137e522b80 1 -- start start 2024-05-08T21:26:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.867+0000 7f136bfff700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13540a11a0 0x7f13540a3660 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f136c100db0 tx=0x7f135c009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.867+0000 7f137e522b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d2030361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.867+0000 7f137e522b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d203036730 0x55d203036bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.867+0000 7f137e522b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2030370f0 0x55d203037570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.867+0000 7f137e522b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d202f841d0 con 0x55d2030370f0 2024-05-08T21:26:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.867+0000 7f137e522b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d202f81b90 con 0x55d203035dd0 2024-05-08T21:26:10.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.867+0000 7f137e522b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d202f7d480 con 0x55d203036730 2024-05-08T21:26:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.868+0000 7f136bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d2030361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.868+0000 7f1370905700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2030370f0 0x55d203037570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.868+0000 7f136bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d2030361f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37724/0 (socket says 172.21.15.33:37724) 2024-05-08T21:26:10.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.868+0000 7f136bfff700 1 -- 172.21.15.33:0/2914311907 learned_addr learned my addr 172.21.15.33:0/2914311907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.868+0000 7f136bfff700 1 -- 172.21.15.33:0/2914311907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d203036730 msgr2=0x55d203036bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:26:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.868+0000 7f136bfff700 1 --2- 172.21.15.33:0/2914311907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d203036730 0x55d203036bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.868+0000 7f136bfff700 1 -- 172.21.15.33:0/2914311907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2030370f0 msgr2=0x55d203037570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.868+0000 7f136bfff700 1 --2- 172.21.15.33:0/2914311907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2030370f0 0x55d203037570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.868+0000 7f136bfff700 1 -- 172.21.15.33:0/2914311907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d202dfe430 con 0x55d203035dd0 2024-05-08T21:26:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.868+0000 7f1370905700 1 --2- 172.21.15.33:0/2914311907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2030370f0 0x55d203037570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.868+0000 7f136bfff700 1 --2- 172.21.15.33:0/2914311907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d2030361f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f135c00dbd0 tx=0x7f135c010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43fe7d5be1b75b4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.869+0000 7f13527fc700 1 -- 172.21.15.33:0/2914311907 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c010bb0 con 0x55d203035dd0 2024-05-08T21:26:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.869+0000 7f13527fc700 1 -- 172.21.15.33:0/2914311907 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f135c010d10 con 0x55d203035dd0 2024-05-08T21:26:10.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.869+0000 7f13527fc700 1 -- 172.21.15.33:0/2914311907 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c01dea0 con 0x55d203035dd0 2024-05-08T21:26:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.874+0000 7f137e522b80 1 -- 172.21.15.33:0/2914311907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 msgr2=0x55d2030361f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.874+0000 7f137e522b80 1 --2- 172.21.15.33:0/2914311907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d2030361f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f135c00dbd0 tx=0x7f135c010000 comp rx=0 tx=0).stop 2024-05-08T21:26:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.875+0000 7f137e522b80 1 -- 172.21.15.33:0/2914311907 shutdown_connections 2024-05-08T21:26:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.875+0000 7f137e522b80 1 --2- 172.21.15.33:0/2914311907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2030370f0 0x55d203037570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.875+0000 7f137e522b80 1 --2- 172.21.15.33:0/2914311907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d2030361f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.875+0000 7f137e522b80 1 --2- 172.21.15.33:0/2914311907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d203036730 0x55d203036bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.875+0000 7f137e522b80 1 -- 172.21.15.33:0/2914311907 >> 172.21.15.33:0/2914311907 conn(0x55d202f8b290 msgr2=0x55d203027e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.875+0000 7f137e522b80 1 -- 172.21.15.33:0/2914311907 shutdown_connections 2024-05-08T21:26:10.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.875+0000 7f137e522b80 1 -- 172.21.15.33:0/2914311907 wait complete. 2024-05-08T21:26:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.875+0000 7f137e522b80 1 -- start start 2024-05-08T21:26:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.876+0000 7f137e522b80 1 Processor -- start 2024-05-08T21:26:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.876+0000 7f137e522b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d20303f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.876+0000 7f137e522b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 0x55d20303fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.876+0000 7f137e522b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2030370f0 0x55d203046b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.876+0000 7f137e522b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d202f195e0 con 0x55d203036730 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.876+0000 7f137e522b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d202f19280 con 0x55d203035dd0 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.876+0000 7f137e522b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d202e00720 con 0x55d2030370f0 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.876+0000 7f137e522b80 10 client.0 _ll_register_callbacks cb 0x55d202f22bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.876+0000 7f136b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 0x55d20303fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.876+0000 7f136b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 0x55d20303fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59010/0 (socket says 172.21.15.33:59010) 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.877+0000 7f136b7fe700 1 -- 172.21.15.33:0/110550140 learned_addr learned my addr 172.21.15.33:0/110550140 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.877+0000 7f1370905700 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2030370f0 0x55d203046b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.877+0000 7f136bfff700 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d20303f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.877+0000 7f136b7fe700 1 -- 172.21.15.33:0/110550140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2030370f0 msgr2=0x55d203046b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.877+0000 7f136b7fe700 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2030370f0 0x55d203046b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.877+0000 7f136b7fe700 1 -- 172.21.15.33:0/110550140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 msgr2=0x55d20303f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.877+0000 7f136b7fe700 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d20303f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.877+0000 7f136b7fe700 1 -- 172.21.15.33:0/110550140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c072230 con 0x55d203036730 2024-05-08T21:26:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.877+0000 7f136b7fe700 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 0x55d20303fb40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f136000e3a0 tx=0x7f136000e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17ff00ec6d5d499 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.877+0000 7f134b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:26:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f134b7fe700 1 -- 172.21.15.33:0/110550140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360011670 con 0x55d203036730 2024-05-08T21:26:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f134b7fe700 1 -- 172.21.15.33:0/110550140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f13600117d0 con 0x55d203036730 2024-05-08T21:26:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f137e522b80 1 -- 172.21.15.33:0/110550140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d202ed62f0 con 0x55d203036730 2024-05-08T21:26:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f134b7fe700 1 -- 172.21.15.33:0/110550140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1360011990 con 0x55d203036730 2024-05-08T21:26:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f137e522b80 10 client.17484 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f137e522b80 1 -- 172.21.15.33:0/110550140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f135c00d850 con 0x55d203036730 2024-05-08T21:26:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f134b7fe700 1 -- 172.21.15.33:0/110550140 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d202ed62f0 con 0x55d203036730 2024-05-08T21:26:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f134b7fe700 1 -- 172.21.15.33:0/110550140 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f136000c070 con 0x55d203036730 2024-05-08T21:26:10.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f137e522b80 10 client.17484 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:10.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f137e522b80 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:10.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f137e522b80 20 client.17484 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:10.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f137e522b80 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d20304b4a0 0x55d20304d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f137e522b80 4 client.17484 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:10.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.878+0000 7f137e522b80 1 -- 172.21.15.33:0/110550140 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d202ffc890 con 0x55d20304b4a0 2024-05-08T21:26:10.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.879+0000 7f136bfff700 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d20304b4a0 0x55d20304d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f136bfff700 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d20304b4a0 0x55d20304d8c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea7df97a3f48cc2 server_cookie=95d57fd690241cbd in_seq=0 out_seq=0 2024-05-08T21:26:10.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f134b7fe700 10 client.17484 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:10.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f134b7fe700 1 -- 172.21.15.33:0/110550140 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d202ffc890 con 0x55d20304b4a0 2024-05-08T21:26:10.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f134b7fe700 10 client.17484 handle_command_reply: tid=0 2024-05-08T21:26:10.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f137e522b80 1 client.17484 shutdown 2024-05-08T21:26:10.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f137e522b80 1 -- 172.21.15.33:0/110550140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 msgr2=0x55d20303fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:10.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f137e522b80 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 0x55d20303fb40 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f136000e3a0 tx=0x7f136000e6b0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f137e522b80 1 -- 172.21.15.33:0/110550140 shutdown_connections 2024-05-08T21:26:10.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f137e522b80 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d20304b4a0 0x55d20304d8c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f137e522b80 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2030370f0 0x55d203046b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f137e522b80 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d20303f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f134b7fe700 0 client.17484 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:10.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f137e522b80 1 --2- 172.21.15.33:0/110550140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 0x55d20303fb40 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.880+0000 7f137e522b80 1 -- 172.21.15.33:0/110550140 >> 172.21.15.33:0/110550140 conn(0x55d202f8b290 msgr2=0x55d203027620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:10.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.881+0000 7f137e522b80 1 -- 172.21.15.33:0/110550140 shutdown_connections 2024-05-08T21:26:10.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.881+0000 7f137e522b80 1 -- 172.21.15.33:0/110550140 wait complete. 2024-05-08T21:26:10.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.881+0000 7f137e522b80 20 client.17484 trim_cache size 0 max 16384 2024-05-08T21:26:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.906+0000 7f137e522b80 1 Processor -- start 2024-05-08T21:26:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.906+0000 7f137e522b80 1 -- start start 2024-05-08T21:26:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.907+0000 7f137e522b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d20303f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.907+0000 7f137e522b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 0x55d203046aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.907+0000 7f137e522b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2030370f0 0x55d20303f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.907+0000 7f137e522b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d203022880 con 0x55d203036730 2024-05-08T21:26:10.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.907+0000 7f137e522b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d203048910 con 0x55d203035dd0 2024-05-08T21:26:10.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.907+0000 7f137e522b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d20303e5c0 con 0x55d2030370f0 2024-05-08T21:26:10.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.907+0000 7f136b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 0x55d203046aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.907+0000 7f136b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 0x55d203046aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59022/0 (socket says 172.21.15.33:59022) 2024-05-08T21:26:10.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.907+0000 7f136b7fe700 1 -- 172.21.15.33:0/3382139691 learned_addr learned my addr 172.21.15.33:0/3382139691 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:10.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.908+0000 7f136b7fe700 1 -- 172.21.15.33:0/3382139691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2030370f0 msgr2=0x55d20303f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:10.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.908+0000 7f136b7fe700 1 --2- 172.21.15.33:0/3382139691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2030370f0 0x55d20303f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.908+0000 7f136b7fe700 1 -- 172.21.15.33:0/3382139691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 msgr2=0x55d20303f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:10.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.908+0000 7f136b7fe700 1 --2- 172.21.15.33:0/3382139691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d20303f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.908+0000 7f136b7fe700 1 -- 172.21.15.33:0/3382139691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d202daabe0 con 0x55d203036730 2024-05-08T21:26:10.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.908+0000 7f136b7fe700 1 --2- 172.21.15.33:0/3382139691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 0x55d203046aa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f13600096f0 tx=0x7f1360005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7df6fe0fcd943d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:10.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.909+0000 7f1352ffd700 1 -- 172.21.15.33:0/3382139691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13600105d0 con 0x55d203036730 2024-05-08T21:26:10.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.909+0000 7f1352ffd700 1 -- 172.21.15.33:0/3382139691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f136000eab0 con 0x55d203036730 2024-05-08T21:26:10.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.909+0000 7f1352ffd700 1 -- 172.21.15.33:0/3382139691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360010990 con 0x55d203036730 2024-05-08T21:26:10.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.910+0000 7f137e522b80 1 -- 172.21.15.33:0/3382139691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 msgr2=0x55d203046aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:10.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.910+0000 7f137e522b80 1 --2- 172.21.15.33:0/3382139691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 0x55d203046aa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f13600096f0 tx=0x7f1360005900 comp rx=0 tx=0).stop 2024-05-08T21:26:10.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.910+0000 7f137e522b80 1 -- 172.21.15.33:0/3382139691 shutdown_connections 2024-05-08T21:26:10.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.910+0000 7f137e522b80 1 --2- 172.21.15.33:0/3382139691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2030370f0 0x55d20303f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.910+0000 7f137e522b80 1 --2- 172.21.15.33:0/3382139691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d20303f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.910+0000 7f137e522b80 1 --2- 172.21.15.33:0/3382139691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d203036730 0x55d203046aa0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.910+0000 7f137e522b80 1 -- 172.21.15.33:0/3382139691 >> 172.21.15.33:0/3382139691 conn(0x55d202f8b290 msgr2=0x55d203039f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:10.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.911+0000 7f137e522b80 1 -- 172.21.15.33:0/3382139691 shutdown_connections 2024-05-08T21:26:10.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.911+0000 7f137e522b80 1 -- 172.21.15.33:0/3382139691 wait complete. 2024-05-08T21:26:10.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.911+0000 7f137e522b80 1 -- start start 2024-05-08T21:26:10.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.911+0000 7f137e522b80 1 Processor -- start 2024-05-08T21:26:10.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f137e522b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d203043770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f137e522b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d203036730 0x55d203043cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f137e522b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2030370f0 0x55d2030a1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f137e522b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d203046b90 con 0x55d2030370f0 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f137e522b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d203046d00 con 0x55d203035dd0 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f137e522b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2030a1ed0 con 0x55d203036730 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f137e522b80 10 client.0 _ll_register_callbacks cb 0x55d202f22bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f136bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d203043770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f136bfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d203043770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37752/0 (socket says 172.21.15.33:37752) 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f136bfff700 1 -- 172.21.15.33:0/1368865931 learned_addr learned my addr 172.21.15.33:0/1368865931 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f136b7fe700 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d203036730 0x55d203043cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f1370905700 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2030370f0 0x55d2030a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f136bfff700 1 -- 172.21.15.33:0/1368865931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d203036730 msgr2=0x55d203043cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f136bfff700 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d203036730 0x55d203043cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f136bfff700 1 -- 172.21.15.33:0/1368865931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2030370f0 msgr2=0x55d2030a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f136bfff700 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2030370f0 0x55d2030a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f136bfff700 1 -- 172.21.15.33:0/1368865931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135c00d850 con 0x55d203035dd0 2024-05-08T21:26:10.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.912+0000 7f1370905700 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2030370f0 0x55d2030a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:26:10.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.913+0000 7f136bfff700 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d203043770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f135c010740 tx=0x7f135c010f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b5707a3302937a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:10.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.913+0000 7f134b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:26:10.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.913+0000 7f134b7fe700 1 -- 172.21.15.33:0/1368865931 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c02c040 con 0x55d203035dd0 2024-05-08T21:26:10.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.913+0000 7f134b7fe700 1 -- 172.21.15.33:0/1368865931 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f135c01de40 con 0x55d203035dd0 2024-05-08T21:26:10.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.913+0000 7f137e522b80 1 -- 172.21.15.33:0/1368865931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f135c010d10 con 0x55d203035dd0 2024-05-08T21:26:10.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.913+0000 7f134b7fe700 1 -- 172.21.15.33:0/1368865931 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f135c033610 con 0x55d203035dd0 2024-05-08T21:26:10.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.914+0000 7f137e522b80 10 client.27392 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:10.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.914+0000 7f137e522b80 1 -- 172.21.15.33:0/1368865931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d202f435e0 con 0x55d203035dd0 2024-05-08T21:26:10.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.914+0000 7f134b7fe700 1 -- 172.21.15.33:0/1368865931 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f135c010d10 con 0x55d203035dd0 2024-05-08T21:26:10.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.914+0000 7f134b7fe700 1 -- 172.21.15.33:0/1368865931 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f135c027400 con 0x55d203035dd0 2024-05-08T21:26:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.914+0000 7f137e522b80 10 client.27392 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.914+0000 7f137e522b80 10 client.27392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.914+0000 7f137e522b80 20 client.27392 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.914+0000 7f137e522b80 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d2030a3790 0x55d2030a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.914+0000 7f137e522b80 4 client.27392 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:26:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.914+0000 7f137e522b80 1 -- 172.21.15.33:0/1368865931 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d202f75f40 con 0x55d2030a3790 2024-05-08T21:26:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.914+0000 7f136b7fe700 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d2030a3790 0x55d2030a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.915+0000 7f134b7fe700 10 client.27392 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.915+0000 7f136b7fe700 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d2030a3790 0x55d2030a5bb0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33622c0e1724857 server_cookie=9ecf93fa20bd6b0 in_seq=0 out_seq=0 2024-05-08T21:26:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.916+0000 7f134b7fe700 1 -- 172.21.15.33:0/1368865931 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d202f75f40 con 0x55d2030a3790 2024-05-08T21:26:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.916+0000 7f134b7fe700 10 client.27392 handle_command_reply: tid=0 2024-05-08T21:26:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.916+0000 7f137e522b80 1 client.27392 shutdown 2024-05-08T21:26:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.916+0000 7f137e522b80 1 -- 172.21.15.33:0/1368865931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 msgr2=0x55d203043770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.916+0000 7f137e522b80 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d203043770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f135c010740 tx=0x7f135c010f30 comp rx=0 tx=0).stop 2024-05-08T21:26:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f137e522b80 1 -- 172.21.15.33:0/1368865931 shutdown_connections 2024-05-08T21:26:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f137e522b80 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d2030a3790 0x55d2030a5bb0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f137e522b80 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2030370f0 0x55d2030a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f137e522b80 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d203035dd0 0x55d203043770 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f137e522b80 1 --2- 172.21.15.33:0/1368865931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d203036730 0x55d203043cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f137e522b80 1 -- 172.21.15.33:0/1368865931 >> 172.21.15.33:0/1368865931 conn(0x55d202f8b290 msgr2=0x55d20303f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f134b7fe700 0 client.27392 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f137e522b80 1 -- 172.21.15.33:0/1368865931 shutdown_connections 2024-05-08T21:26:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f137e522b80 1 -- 172.21.15.33:0/1368865931 wait complete. 2024-05-08T21:26:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f137e522b80 20 client.27392 trim_cache size 0 max 16384 2024-05-08T21:26:10.920 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:26:10.920 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:26:10.920 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "fadea047-00b5-4441-af1d-cde6a5ced7f4", 2024-05-08T21:26:10.921 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:26:10.921 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:26:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f1372368700 1 -- 172.21.15.33:0/1044166587 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13540a11a0 msgr2=0x7f13540a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f1372368700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13540a11a0 0x7f13540a3660 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f136c100db0 tx=0x7f135c009470 comp rx=0 tx=0).stop 2024-05-08T21:26:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f1372368700 1 -- 172.21.15.33:0/1044166587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f136c1018f0 msgr2=0x7f136c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.917+0000 7f1372368700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f136c1018f0 0x7f136c068160 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1364007b60 tx=0x7f13640094f0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.918+0000 7f1372368700 1 -- 172.21.15.33:0/1044166587 shutdown_connections 2024-05-08T21:26:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.918+0000 7f1372368700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f136c1018f0 0x7f136c068160 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.918+0000 7f1372368700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f136c0ffd60 0x7f136c070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.918+0000 7f1372368700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13540a11a0 0x7f13540a3660 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.918+0000 7f1372368700 1 --2- 172.21.15.33:0/1044166587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f136c100f50 0x7f136c071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:10.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.918+0000 7f1372368700 1 -- 172.21.15.33:0/1044166587 >> 172.21.15.33:0/1044166587 conn(0x7f136c0fb290 msgr2=0x7f136c1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:10.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.918+0000 7f1372368700 1 -- 172.21.15.33:0/1044166587 shutdown_connections 2024-05-08T21:26:10.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:10.918+0000 7f1372368700 1 -- 172.21.15.33:0/1044166587 wait complete. 2024-05-08T21:26:10.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:26:11.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.149+0000 7f12f5e9b700 1 -- 172.21.15.33:0/1140661973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12f0101fc0 msgr2=0x7f12f0102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.149+0000 7f12f5e9b700 1 --2- 172.21.15.33:0/1140661973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12f0101fc0 0x7f12f0102440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f12e40098f0 tx=0x7f12e4009c00 comp rx=0 tx=0).stop 2024-05-08T21:26:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.149+0000 7f12f5e9b700 1 -- 172.21.15.33:0/1140661973 shutdown_connections 2024-05-08T21:26:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.149+0000 7f12f5e9b700 1 --2- 172.21.15.33:0/1140661973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f0102980 0x7f12f01092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.149+0000 7f12f5e9b700 1 --2- 172.21.15.33:0/1140661973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12f0100e70 0x7f12f0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.149+0000 7f12f5e9b700 1 --2- 172.21.15.33:0/1140661973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12f0101fc0 0x7f12f0102440 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.149+0000 7f12f5e9b700 1 -- 172.21.15.33:0/1140661973 >> 172.21.15.33:0/1140661973 conn(0x7f12f00fc400 msgr2=0x7f12f00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.149+0000 7f12f5e9b700 1 -- 172.21.15.33:0/1140661973 shutdown_connections 2024-05-08T21:26:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.149+0000 7f12f5e9b700 1 -- 172.21.15.33:0/1140661973 wait complete. 2024-05-08T21:26:11.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12f5e9b700 1 Processor -- start 2024-05-08T21:26:11.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12f5e9b700 1 -- start start 2024-05-08T21:26:11.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12f5e9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12f0100e70 0x7f12f0074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12f5e9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f0101fc0 0x7f12f0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12f5e9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12f0102980 0x7f12f00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12f5e9b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f0074070 con 0x7f12f0100e70 2024-05-08T21:26:11.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12f5e9b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f00741e0 con 0x7f12f0101fc0 2024-05-08T21:26:11.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12f5e9b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f01a0d90 con 0x7f12f0102980 2024-05-08T21:26:11.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12eeffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f0101fc0 0x7f12f0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12eeffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f0101fc0 0x7f12f0073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37776/0 (socket says 172.21.15.33:37776) 2024-05-08T21:26:11.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12effff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12f0102980 0x7f12f00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12eeffd700 1 -- 172.21.15.33:0/767663830 learned_addr learned my addr 172.21.15.33:0/767663830 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:11.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.150+0000 7f12ef7fe700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12f0100e70 0x7f12f0074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.151+0000 7f12effff700 1 -- 172.21.15.33:0/767663830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f0101fc0 msgr2=0x7f12f0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:11.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.151+0000 7f12effff700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f0101fc0 0x7f12f0073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.151+0000 7f12effff700 1 -- 172.21.15.33:0/767663830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12f0100e70 msgr2=0x7f12f0074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:11.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.151+0000 7f12effff700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12f0100e70 0x7f12f0074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.151+0000 7f12effff700 1 -- 172.21.15.33:0/767663830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e4009520 con 0x7f12f0102980 2024-05-08T21:26:11.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.151+0000 7f12effff700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12f0102980 0x7f12f00738d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12e8007f10 tx=0x7f12e800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:11.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.151+0000 7f12ecff9700 1 -- 172.21.15.33:0/767663830 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e800dbd0 con 0x7f12f0102980 2024-05-08T21:26:11.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.151+0000 7f12ecff9700 1 -- 172.21.15.33:0/767663830 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12e800f040 con 0x7f12f0102980 2024-05-08T21:26:11.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.151+0000 7f12ecff9700 1 -- 172.21.15.33:0/767663830 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e80058f0 con 0x7f12f0102980 2024-05-08T21:26:11.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.151+0000 7f12f5e9b700 1 -- 172.21.15.33:0/767663830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f01a0fc0 con 0x7f12f0102980 2024-05-08T21:26:11.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.151+0000 7f12f5e9b700 1 -- 172.21.15.33:0/767663830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f01a1390 con 0x7f12f0102980 2024-05-08T21:26:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.156+0000 7f12ecff9700 1 -- 172.21.15.33:0/767663830 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f12e8005000 con 0x7f12f0102980 2024-05-08T21:26:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.156+0000 7f1302055b80 1 Processor -- start 2024-05-08T21:26:11.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.156+0000 7f1302055b80 1 -- start start 2024-05-08T21:26:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.156+0000 7f1302055b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fc540 0x55ed3e7fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.156+0000 7f1302055b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fcea0 0x55ed3e7fd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.156+0000 7f1302055b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fd860 0x55ed3e7fdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.156+0000 7f1302055b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3e753150 con 0x55ed3e7fcea0 2024-05-08T21:26:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.156+0000 7f1302055b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3e750b10 con 0x55ed3e7fd860 2024-05-08T21:26:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.156+0000 7f1302055b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3e74c400 con 0x55ed3e7fc540 2024-05-08T21:26:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12eeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fc540 0x55ed3e7fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12eeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fc540 0x55ed3e7fc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:46468/0 (socket says 172.21.15.33:46468) 2024-05-08T21:26:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12eeffd700 1 -- 172.21.15.33:0/1756615997 learned_addr learned my addr 172.21.15.33:0/1756615997 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12eeffd700 1 --2- 172.21.15.33:0/1756615997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fd860 0x55ed3e7fdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12ef7fe700 1 --2- 172.21.15.33:0/1756615997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fcea0 0x55ed3e7fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12eeffd700 1 -- 172.21.15.33:0/1756615997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fd860 msgr2=0x55ed3e7fdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12eeffd700 1 --2- 172.21.15.33:0/1756615997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fd860 0x55ed3e7fdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12eeffd700 1 -- 172.21.15.33:0/1756615997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fcea0 msgr2=0x55ed3e7fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12eeffd700 1 --2- 172.21.15.33:0/1756615997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fcea0 0x55ed3e7fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12ecff9700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f12d80a11a0 0x7f12d80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12eeffd700 1 -- 172.21.15.33:0/1756615997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed3e5cd3b0 con 0x55ed3e7fc540 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12ecff9700 1 -- 172.21.15.33:0/767663830 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f12e80cc4f0 con 0x7f12f0102980 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12eeffd700 1 --2- 172.21.15.33:0/1756615997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fc540 0x55ed3e7fc960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f12e400fb00 tx=0x7f12e4019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d054455357ef7a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.157+0000 7f12eeffd700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f12d80a11a0 0x7f12d80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.158+0000 7f12d5ffb700 1 -- 172.21.15.33:0/1756615997 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e400f510 con 0x55ed3e7fc540 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.158+0000 7f12d5ffb700 1 -- 172.21.15.33:0/1756615997 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12e400f670 con 0x55ed3e7fc540 2024-05-08T21:26:11.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.158+0000 7f12d5ffb700 1 -- 172.21.15.33:0/1756615997 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e4005670 con 0x55ed3e7fc540 2024-05-08T21:26:11.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.158+0000 7f12eeffd700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f12d80a11a0 0x7f12d80a3660 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f12e4005850 tx=0x7f12e4003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:11.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.158+0000 7f1302055b80 1 -- 172.21.15.33:0/1756615997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fc540 msgr2=0x55ed3e7fc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:11.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.159+0000 7f1302055b80 1 --2- 172.21.15.33:0/1756615997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fc540 0x55ed3e7fc960 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f12e400fb00 tx=0x7f12e4019040 comp rx=0 tx=0).stop 2024-05-08T21:26:11.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.159+0000 7f1302055b80 1 -- 172.21.15.33:0/1756615997 shutdown_connections 2024-05-08T21:26:11.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.159+0000 7f1302055b80 1 --2- 172.21.15.33:0/1756615997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fd860 0x55ed3e7fdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.159+0000 7f1302055b80 1 --2- 172.21.15.33:0/1756615997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fc540 0x55ed3e7fc960 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.159+0000 7f1302055b80 1 --2- 172.21.15.33:0/1756615997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fcea0 0x55ed3e7fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.159+0000 7f1302055b80 1 -- 172.21.15.33:0/1756615997 >> 172.21.15.33:0/1756615997 conn(0x55ed3e75a210 msgr2=0x55ed3e7f6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.159+0000 7f1302055b80 1 -- 172.21.15.33:0/1756615997 shutdown_connections 2024-05-08T21:26:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.159+0000 7f1302055b80 1 -- 172.21.15.33:0/1756615997 wait complete. 2024-05-08T21:26:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.159+0000 7f1302055b80 1 -- start start 2024-05-08T21:26:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.160+0000 7f1302055b80 1 Processor -- start 2024-05-08T21:26:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.160+0000 7f1302055b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 0x55ed3e80e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.160+0000 7f1302055b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fcea0 0x55ed3e80ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.160+0000 7f1302055b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 0x55ed3e815b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.160+0000 7f1302055b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3e6e8560 con 0x55ed3e7fc540 2024-05-08T21:26:11.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f1302055b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3e6e8200 con 0x55ed3e7fcea0 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f1302055b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3e5cf6a0 con 0x55ed3e7fd860 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f1302055b80 10 client.0 _ll_register_callbacks cb 0x55ed3e6f1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f12ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 0x55ed3e80e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f12ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 0x55ed3e80e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59076/0 (socket says 172.21.15.33:59076) 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f12ef7fe700 1 -- 172.21.15.33:0/3405236100 learned_addr learned my addr 172.21.15.33:0/3405236100 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f12eeffd700 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fcea0 0x55ed3e80ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f12effff700 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 0x55ed3e815b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f12eeffd700 1 -- 172.21.15.33:0/3405236100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 msgr2=0x55ed3e815b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f12eeffd700 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 0x55ed3e815b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f12eeffd700 1 -- 172.21.15.33:0/3405236100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 msgr2=0x55ed3e80e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f12eeffd700 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 0x55ed3e80e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f12eeffd700 1 -- 172.21.15.33:0/3405236100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e401f040 con 0x55ed3e7fcea0 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f12effff700 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 0x55ed3e815b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:11.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.161+0000 7f12ef7fe700 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 0x55ed3e80e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.162+0000 7f12eeffd700 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fcea0 0x55ed3e80ead0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009fd0 tx=0x7f12e4030000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a1b844160a384ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.162+0000 7f12ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.162+0000 7f12ceffd700 1 -- 172.21.15.33:0/3405236100 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e400f510 con 0x55ed3e7fcea0 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.162+0000 7f12ceffd700 1 -- 172.21.15.33:0/3405236100 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f12e40055a0 con 0x55ed3e7fcea0 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.162+0000 7f1302055b80 1 -- 172.21.15.33:0/3405236100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed3e6a5270 con 0x55ed3e7fcea0 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.162+0000 7f12ceffd700 1 -- 172.21.15.33:0/3405236100 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12e400b770 con 0x55ed3e7fcea0 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.162+0000 7f1302055b80 10 client.27404 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.162+0000 7f1302055b80 1 -- 172.21.15.33:0/3405236100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ed3e579b60 con 0x55ed3e7fcea0 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.162+0000 7f12ceffd700 1 -- 172.21.15.33:0/3405236100 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ed3e6a5270 con 0x55ed3e7fcea0 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.163+0000 7f12ceffd700 1 -- 172.21.15.33:0/3405236100 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f12e40289a0 con 0x55ed3e7fcea0 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.163+0000 7f1302055b80 10 client.27404 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.163+0000 7f1302055b80 10 client.27404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.163+0000 7f1302055b80 20 client.27404 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:11.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.163+0000 7f1302055b80 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ed3e819e20 0x55ed3e81c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.163+0000 7f1302055b80 4 client.27404 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.163+0000 7f1302055b80 1 -- 172.21.15.33:0/3405236100 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed3e7cb810 con 0x55ed3e819e20 2024-05-08T21:26:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.163+0000 7f12ef7fe700 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ed3e819e20 0x55ed3e81c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.163+0000 7f12ef7fe700 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ed3e819e20 0x55ed3e81c240 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5633c81bea5f8b1d server_cookie=9596405f20619d71 in_seq=0 out_seq=0 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.163+0000 7f12ceffd700 10 client.27404 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.164+0000 7f12ceffd700 1 -- 172.21.15.33:0/3405236100 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ed3e7cb810 con 0x55ed3e819e20 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.164+0000 7f12ceffd700 10 client.27404 handle_command_reply: tid=0 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.164+0000 7f1302055b80 1 client.27404 shutdown 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.165+0000 7f1302055b80 1 -- 172.21.15.33:0/3405236100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fcea0 msgr2=0x55ed3e80ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.165+0000 7f1302055b80 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fcea0 0x55ed3e80ead0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009fd0 tx=0x7f12e4030000 comp rx=0 tx=0).stop 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.166+0000 7f1302055b80 1 -- 172.21.15.33:0/3405236100 shutdown_connections 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.166+0000 7f1302055b80 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ed3e819e20 0x55ed3e81c240 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.166+0000 7f1302055b80 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 0x55ed3e815b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.166+0000 7f1302055b80 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 0x55ed3e80e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.166+0000 7f1302055b80 1 --2- 172.21.15.33:0/3405236100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fcea0 0x55ed3e80ead0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.166+0000 7f1302055b80 1 -- 172.21.15.33:0/3405236100 >> 172.21.15.33:0/3405236100 conn(0x55ed3e75a210 msgr2=0x55ed3e7f65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:11.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.166+0000 7f12ceffd700 0 client.27404 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:11.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.167+0000 7f1302055b80 1 -- 172.21.15.33:0/3405236100 shutdown_connections 2024-05-08T21:26:11.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.167+0000 7f1302055b80 1 -- 172.21.15.33:0/3405236100 wait complete. 2024-05-08T21:26:11.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.168+0000 7f1302055b80 20 client.27404 trim_cache size 0 max 16384 2024-05-08T21:26:11.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f1302055b80 1 Processor -- start 2024-05-08T21:26:11.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f1302055b80 1 -- start start 2024-05-08T21:26:11.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f1302055b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fc540 0x55ed3e80e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f1302055b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fcea0 0x55ed3e80ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f1302055b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 0x55ed3e815ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f1302055b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3e7cb810 con 0x55ed3e7fcea0 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f1302055b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3e6a5270 con 0x55ed3e7fc540 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f1302055b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3e7ef110 con 0x55ed3e7fd860 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f12ef7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fc540 0x55ed3e80e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f12effff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 0x55ed3e815ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f12ef7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fc540 0x55ed3e80e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:37800/0 (socket says 172.21.15.33:37800) 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f12effff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 0x55ed3e815ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:46480/0 (socket says 172.21.15.33:46480) 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.194+0000 7f12ef7fe700 1 -- 172.21.15.33:0/909232808 learned_addr learned my addr 172.21.15.33:0/909232808 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.195+0000 7f12effff700 1 -- 172.21.15.33:0/909232808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fc540 msgr2=0x55ed3e80e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.195+0000 7f12effff700 1 --2- 172.21.15.33:0/909232808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fc540 0x55ed3e80e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.195+0000 7f12effff700 1 -- 172.21.15.33:0/909232808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fcea0 msgr2=0x55ed3e80ea90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.195+0000 7f12effff700 1 --2- 172.21.15.33:0/909232808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fcea0 0x55ed3e80ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.195+0000 7f12effff700 1 -- 172.21.15.33:0/909232808 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed3e712560 con 0x55ed3e7fd860 2024-05-08T21:26:11.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.195+0000 7f12ef7fe700 1 --2- 172.21.15.33:0/909232808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fc540 0x55ed3e80e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:11.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.195+0000 7f12effff700 1 --2- 172.21.15.33:0/909232808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 0x55ed3e815ad0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f12e8099280 tx=0x7f12e8084040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10fd2d65f76aa490 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.195+0000 7f12d67fc700 1 -- 172.21.15.33:0/909232808 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e80b3070 con 0x55ed3e7fd860 2024-05-08T21:26:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.195+0000 7f12d67fc700 1 -- 172.21.15.33:0/909232808 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12e8098ee0 con 0x55ed3e7fd860 2024-05-08T21:26:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.195+0000 7f12d67fc700 1 -- 172.21.15.33:0/909232808 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e80a1a00 con 0x55ed3e7fd860 2024-05-08T21:26:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.196+0000 7f1302055b80 1 -- 172.21.15.33:0/909232808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 msgr2=0x55ed3e815ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.196+0000 7f1302055b80 1 --2- 172.21.15.33:0/909232808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 0x55ed3e815ad0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f12e8099280 tx=0x7f12e8084040 comp rx=0 tx=0).stop 2024-05-08T21:26:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.196+0000 7f1302055b80 1 -- 172.21.15.33:0/909232808 shutdown_connections 2024-05-08T21:26:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.196+0000 7f1302055b80 1 --2- 172.21.15.33:0/909232808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fd860 0x55ed3e815ad0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.196+0000 7f1302055b80 1 --2- 172.21.15.33:0/909232808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fc540 0x55ed3e80e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.196+0000 7f1302055b80 1 --2- 172.21.15.33:0/909232808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fcea0 0x55ed3e80ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.196+0000 7f1302055b80 1 -- 172.21.15.33:0/909232808 >> 172.21.15.33:0/909232808 conn(0x55ed3e75a210 msgr2=0x55ed3e800750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:11.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.197+0000 7f1302055b80 1 -- 172.21.15.33:0/909232808 shutdown_connections 2024-05-08T21:26:11.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.197+0000 7f1302055b80 1 -- 172.21.15.33:0/909232808 wait complete. 2024-05-08T21:26:11.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.198+0000 7f1302055b80 1 -- start start 2024-05-08T21:26:11.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.198+0000 7f1302055b80 1 Processor -- start 2024-05-08T21:26:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.201+0000 7f1302055b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 0x55ed3e811880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.201+0000 7f1302055b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fcea0 0x55ed3e811dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.201+0000 7f1302055b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fd860 0x55ed3e814d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.201+0000 7f1302055b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3e8152d0 con 0x55ed3e7fc540 2024-05-08T21:26:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.201+0000 7f1302055b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3e815440 con 0x55ed3e7fd860 2024-05-08T21:26:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.201+0000 7f1302055b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed3e8155b0 con 0x55ed3e7fcea0 2024-05-08T21:26:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.201+0000 7f1302055b80 10 client.0 _ll_register_callbacks cb 0x55ed3e6f1b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.201+0000 7f12ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 0x55ed3e811880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.201+0000 7f12ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 0x55ed3e811880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59082/0 (socket says 172.21.15.33:59082) 2024-05-08T21:26:11.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.201+0000 7f12ef7fe700 1 -- 172.21.15.33:0/3757886014 learned_addr learned my addr 172.21.15.33:0/3757886014 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:11.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.202+0000 7f12eeffd700 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fcea0 0x55ed3e811dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.202+0000 7f12effff700 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fd860 0x55ed3e814d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.202+0000 7f12ef7fe700 1 -- 172.21.15.33:0/3757886014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fcea0 msgr2=0x55ed3e811dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.202+0000 7f12ef7fe700 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fcea0 0x55ed3e811dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.202+0000 7f12ef7fe700 1 -- 172.21.15.33:0/3757886014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fd860 msgr2=0x55ed3e814d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.202+0000 7f12ef7fe700 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fd860 0x55ed3e814d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.202+0000 7f12ef7fe700 1 -- 172.21.15.33:0/3757886014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e0012050 con 0x55ed3e7fc540 2024-05-08T21:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.203+0000 7f12ef7fe700 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 0x55ed3e811880 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f12e0009840 tx=0x7f12e001f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff33ac0e35feff9a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.203+0000 7f12ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.203+0000 7f12ceffd700 1 -- 172.21.15.33:0/3757886014 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e0017030 con 0x55ed3e7fc540 2024-05-08T21:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.203+0000 7f12ceffd700 1 -- 172.21.15.33:0/3757886014 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f12e0012760 con 0x55ed3e7fc540 2024-05-08T21:26:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.203+0000 7f1302055b80 1 -- 172.21.15.33:0/3757886014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12e400f670 con 0x55ed3e7fc540 2024-05-08T21:26:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.204+0000 7f12ceffd700 1 -- 172.21.15.33:0/3757886014 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12e0012920 con 0x55ed3e7fc540 2024-05-08T21:26:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.204+0000 7f1302055b80 10 client.17502 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.204+0000 7f1302055b80 1 -- 172.21.15.33:0/3757886014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f12f01a1390 con 0x55ed3e7fc540 2024-05-08T21:26:11.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.205+0000 7f12ceffd700 1 -- 172.21.15.33:0/3757886014 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12e400f670 con 0x55ed3e7fc540 2024-05-08T21:26:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f12ceffd700 1 -- 172.21.15.33:0/3757886014 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f12e0003cc0 con 0x55ed3e7fc540 2024-05-08T21:26:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f1302055b80 10 client.17502 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f1302055b80 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f1302055b80 20 client.17502 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f1302055b80 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ed3e8823b0 0x55ed3e8847d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f1302055b80 4 client.17502 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:26:11.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f1302055b80 1 -- 172.21.15.33:0/3757886014 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ed3e744ec0 con 0x55ed3e8823b0 2024-05-08T21:26:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f12eeffd700 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ed3e8823b0 0x55ed3e8847d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f12eeffd700 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ed3e8823b0 0x55ed3e8847d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6710a90fd1557ef1 server_cookie=e70f9891c5984db4 in_seq=0 out_seq=0 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f12ceffd700 10 client.17502 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f12ceffd700 1 -- 172.21.15.33:0/3757886014 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ed3e744ec0 con 0x55ed3e8823b0 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f12ceffd700 10 client.17502 handle_command_reply: tid=0 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.206+0000 7f1302055b80 1 client.17502 shutdown 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.207+0000 7f1302055b80 1 -- 172.21.15.33:0/3757886014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 msgr2=0x55ed3e811880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.207+0000 7f1302055b80 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 0x55ed3e811880 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f12e0009840 tx=0x7f12e001f6e0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.207+0000 7f1302055b80 1 -- 172.21.15.33:0/3757886014 shutdown_connections 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.207+0000 7f1302055b80 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ed3e8823b0 0x55ed3e8847d0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.207+0000 7f1302055b80 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed3e7fd860 0x55ed3e814d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.207+0000 7f1302055b80 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed3e7fc540 0x55ed3e811880 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.207+0000 7f1302055b80 1 --2- 172.21.15.33:0/3757886014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed3e7fcea0 0x55ed3e811dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.207+0000 7f1302055b80 1 -- 172.21.15.33:0/3757886014 >> 172.21.15.33:0/3757886014 conn(0x55ed3e75a210 msgr2=0x55ed3e800750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.207+0000 7f12ceffd700 0 client.17502 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:11.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.208+0000 7f1302055b80 1 -- 172.21.15.33:0/3757886014 shutdown_connections 2024-05-08T21:26:11.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.208+0000 7f1302055b80 1 -- 172.21.15.33:0/3757886014 wait complete. 2024-05-08T21:26:11.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.208+0000 7f1302055b80 20 client.17502 trim_cache size 0 max 16384 2024-05-08T21:26:11.211 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:26:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:26:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:26:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "fadea047-00b5-4441-af1d-cde6a5ced7f4": { 2024-05-08T21:26:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:26:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "fadea047-00b5-4441-af1d-cde6a5ced7f4", 2024-05-08T21:26:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:26:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:26:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:26:11.211 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.208+0000 7f12f5e9b700 1 -- 172.21.15.33:0/767663830 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f12d80a11a0 msgr2=0x7f12d80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.208+0000 7f12f5e9b700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f12d80a11a0 0x7f12d80a3660 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f12e4005850 tx=0x7f12e4003680 comp rx=0 tx=0).stop 2024-05-08T21:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.208+0000 7f12f5e9b700 1 -- 172.21.15.33:0/767663830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12f0102980 msgr2=0x7f12f00738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.208+0000 7f12f5e9b700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12f0102980 0x7f12f00738d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f12e8007f10 tx=0x7f12e800d3b0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.209+0000 7f12f5e9b700 1 -- 172.21.15.33:0/767663830 shutdown_connections 2024-05-08T21:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.209+0000 7f12f5e9b700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12f0102980 0x7f12f00738d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.209+0000 7f12f5e9b700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12f0100e70 0x7f12f0074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.209+0000 7f12f5e9b700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f12d80a11a0 0x7f12d80a3660 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.209+0000 7f12f5e9b700 1 --2- 172.21.15.33:0/767663830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f0101fc0 0x7f12f0073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:11.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.209+0000 7f12f5e9b700 1 -- 172.21.15.33:0/767663830 >> 172.21.15.33:0/767663830 conn(0x7f12f00fc400 msgr2=0x7f12f0105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:11.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.209+0000 7f12f5e9b700 1 -- 172.21.15.33:0/767663830 shutdown_connections 2024-05-08T21:26:11.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:11.209+0000 7f12f5e9b700 1 -- 172.21.15.33:0/767663830 wait complete. 2024-05-08T21:26:11.234 INFO:tasks.cephfs.filesystem:scrub status for tag:fadea047-00b5-4441-af1d-cde6a5ced7f4 - {'path': '/', 'tag': 'fadea047-00b5-4441-af1d-cde6a5ced7f4', 'options': 'recursive,force'} 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: pgmap v506: 129 pgs: 129 active+clean; 281 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.6 MiB/s wr, 575 op/s 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:11.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:11.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:11 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: pgmap v506: 129 pgs: 129 active+clean; 281 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.6 MiB/s wr, 575 op/s 2024-05-08T21:26:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:26:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:11 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: pgmap v506: 129 pgs: 129 active+clean; 281 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.6 MiB/s wr, 575 op/s 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:11.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:11.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:11.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:11 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:12.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:12 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:12.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:12 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:12.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:12 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:12.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:12 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:12.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:12 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:12.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:12 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:12.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:12 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:12.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:12 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:12.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:12 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:13.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:13 smithi071 ceph-mon[47074]: pgmap v507: 129 pgs: 129 active+clean; 277 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.3 MiB/s wr, 538 op/s 2024-05-08T21:26:13.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:13 smithi082 ceph-mon[47512]: pgmap v507: 129 pgs: 129 active+clean; 277 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.3 MiB/s wr, 538 op/s 2024-05-08T21:26:13.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:13 smithi033 ceph-mon[42528]: pgmap v507: 129 pgs: 129 active+clean; 277 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.3 MiB/s wr, 538 op/s 2024-05-08T21:26:14.676 INFO:tasks.workunit.client.0.smithi033.stdout:truncating to largest ever: 0x63e65d6 2024-05-08T21:26:15.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:15 smithi071 ceph-mon[47074]: pgmap v508: 129 pgs: 129 active+clean; 255 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.9 MiB/s wr, 697 op/s 2024-05-08T21:26:15.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:15 smithi082 ceph-mon[47512]: pgmap v508: 129 pgs: 129 active+clean; 255 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.9 MiB/s wr, 697 op/s 2024-05-08T21:26:15.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:15 smithi033 ceph-mon[42528]: pgmap v508: 129 pgs: 129 active+clean; 255 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.9 MiB/s wr, 697 op/s 2024-05-08T21:26:17.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:17 smithi071 ceph-mon[47074]: pgmap v509: 129 pgs: 129 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.5 MiB/s wr, 566 op/s 2024-05-08T21:26:17.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:17 smithi082 ceph-mon[47512]: pgmap v509: 129 pgs: 129 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.5 MiB/s wr, 566 op/s 2024-05-08T21:26:17.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:17 smithi033 ceph-mon[42528]: pgmap v509: 129 pgs: 129 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.5 MiB/s wr, 566 op/s 2024-05-08T21:26:18.611 INFO:tasks.workunit.client.0.smithi033.stdout:truncating to largest ever: 0x63f1359 2024-05-08T21:26:18.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:18 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:26:18.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:18 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:26:18.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:18 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:26:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: pgmap v510: 129 pgs: 129 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 520 op/s 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi071 to 182.4M 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: Unable to set osd_memory_target on smithi071 to 191357644: error parsing value: Value '191357644' is below minimum 939524096 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi033 to 183.5M 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: Unable to set osd_memory_target on smithi033 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:26:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:26:19.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:26:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: pgmap v510: 129 pgs: 129 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 520 op/s 2024-05-08T21:26:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi071 to 182.4M 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: Unable to set osd_memory_target on smithi071 to 191357644: error parsing value: Value '191357644' is below minimum 939524096 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi033 to 183.5M 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: Unable to set osd_memory_target on smithi033 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:26:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:26:19.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: pgmap v510: 129 pgs: 129 active+clean; 251 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 520 op/s 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi071 to 182.4M 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: Unable to set osd_memory_target on smithi071 to 191357644: error parsing value: Value '191357644' is below minimum 939524096 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi033 to 183.5M 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: Unable to set osd_memory_target on smithi033 to 192483020: error parsing value: Value '192483020' is below minimum 939524096 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:26:19.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:26:20.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:20 smithi071 ceph-mon[47074]: pgmap v511: 129 pgs: 129 active+clean; 242 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.1 MiB/s wr, 722 op/s 2024-05-08T21:26:20.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:20 smithi082 ceph-mon[47512]: pgmap v511: 129 pgs: 129 active+clean; 242 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.1 MiB/s wr, 722 op/s 2024-05-08T21:26:20.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:20 smithi033 ceph-mon[42528]: pgmap v511: 129 pgs: 129 active+clean; 242 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.1 MiB/s wr, 722 op/s 2024-05-08T21:26:21.669 INFO:tasks.workunit.client.0.smithi033.stdout:fallocating to largest ever: 0x6400000 2024-05-08T21:26:21.893 INFO:tasks.workunit.client.0.smithi033.stdout:10000 punch from 0xc44d4b to 0xc4b890, (0x6b45 bytes) 2024-05-08T21:26:23.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:23 smithi082 ceph-mon[47512]: pgmap v512: 129 pgs: 129 active+clean; 241 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.4 MiB/s wr, 554 op/s 2024-05-08T21:26:23.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:23 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:26:23.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:23 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:26:23.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:23 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:26:23.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:23 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:26:23.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:23 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:26:23.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:23 smithi033 ceph-mon[42528]: pgmap v512: 129 pgs: 129 active+clean; 241 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.4 MiB/s wr, 554 op/s 2024-05-08T21:26:23.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:23 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:26:23.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:23 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:26:23.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:23 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:26:23.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:23 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:26:23.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:23 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:26:23.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:23 smithi071 ceph-mon[47074]: pgmap v512: 129 pgs: 129 active+clean; 241 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.4 MiB/s wr, 554 op/s 2024-05-08T21:26:23.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:23 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:26:23.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:23 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:26:23.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:23 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:26:23.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:23 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:26:23.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:23 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:26:25.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:25 smithi082 ceph-mon[47512]: pgmap v513: 129 pgs: 129 active+clean; 241 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 688 op/s 2024-05-08T21:26:25.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:25 smithi033 ceph-mon[42528]: pgmap v513: 129 pgs: 129 active+clean; 241 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 688 op/s 2024-05-08T21:26:25.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:25 smithi071 ceph-mon[47074]: pgmap v513: 129 pgs: 129 active+clean; 241 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 688 op/s 2024-05-08T21:26:27.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:27 smithi082 ceph-mon[47512]: pgmap v514: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.6 MiB/s wr, 569 op/s 2024-05-08T21:26:27.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:27 smithi033 ceph-mon[42528]: pgmap v514: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.6 MiB/s wr, 569 op/s 2024-05-08T21:26:27.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:27 smithi071 ceph-mon[47074]: pgmap v514: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.6 MiB/s wr, 569 op/s 2024-05-08T21:26:29.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:29 smithi033 ceph-mon[42528]: pgmap v515: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.4 MiB/s wr, 528 op/s 2024-05-08T21:26:29.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:29 smithi071 ceph-mon[47074]: pgmap v515: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.4 MiB/s wr, 528 op/s 2024-05-08T21:26:29.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:29 smithi082 ceph-mon[47512]: pgmap v515: 129 pgs: 129 active+clean; 246 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.4 MiB/s wr, 528 op/s 2024-05-08T21:26:31.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:31 smithi033 ceph-mon[42528]: pgmap v516: 129 pgs: 129 active+clean; 254 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.3 MiB/s wr, 725 op/s 2024-05-08T21:26:31.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:31 smithi071 ceph-mon[47074]: pgmap v516: 129 pgs: 129 active+clean; 254 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.3 MiB/s wr, 725 op/s 2024-05-08T21:26:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:31 smithi082 ceph-mon[47512]: pgmap v516: 129 pgs: 129 active+clean; 254 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.3 MiB/s wr, 725 op/s 2024-05-08T21:26:33.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:33 smithi033 ceph-mon[42528]: pgmap v517: 129 pgs: 129 active+clean; 254 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.5 MiB/s wr, 545 op/s 2024-05-08T21:26:33.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:33 smithi071 ceph-mon[47074]: pgmap v517: 129 pgs: 129 active+clean; 254 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.5 MiB/s wr, 545 op/s 2024-05-08T21:26:33.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:33 smithi082 ceph-mon[47512]: pgmap v517: 129 pgs: 129 active+clean; 254 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.5 MiB/s wr, 545 op/s 2024-05-08T21:26:35.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:35 smithi033 ceph-mon[42528]: pgmap v518: 129 pgs: 129 active+clean; 254 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.1 MiB/s wr, 672 op/s 2024-05-08T21:26:35.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:35 smithi071 ceph-mon[47074]: pgmap v518: 129 pgs: 129 active+clean; 254 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.1 MiB/s wr, 672 op/s 2024-05-08T21:26:35.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:35 smithi082 ceph-mon[47512]: pgmap v518: 129 pgs: 129 active+clean; 254 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.1 MiB/s wr, 672 op/s 2024-05-08T21:26:37.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:37 smithi033 ceph-mon[42528]: pgmap v519: 129 pgs: 129 active+clean; 262 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.6 MiB/s wr, 586 op/s 2024-05-08T21:26:37.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:37 smithi071 ceph-mon[47074]: pgmap v519: 129 pgs: 129 active+clean; 262 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.6 MiB/s wr, 586 op/s 2024-05-08T21:26:37.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:37 smithi082 ceph-mon[47512]: pgmap v519: 129 pgs: 129 active+clean; 262 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.6 MiB/s wr, 586 op/s 2024-05-08T21:26:39.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:39 smithi033 ceph-mon[42528]: pgmap v520: 129 pgs: 129 active+clean; 262 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 520 op/s 2024-05-08T21:26:39.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:39 smithi071 ceph-mon[47074]: pgmap v520: 129 pgs: 129 active+clean; 262 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 520 op/s 2024-05-08T21:26:39.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:39 smithi082 ceph-mon[47512]: pgmap v520: 129 pgs: 129 active+clean; 262 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 520 op/s 2024-05-08T21:26:41.236 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:26:41.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.427+0000 7f53904cc700 1 -- 172.21.15.33:0/4275742990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5388104c30 msgr2=0x7f538810b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:41.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.427+0000 7f53904cc700 1 --2- 172.21.15.33:0/4275742990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5388104c30 0x7f538810b4e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f538000b0a0 tx=0x7f538000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.429+0000 7f53904cc700 1 -- 172.21.15.33:0/4275742990 shutdown_connections 2024-05-08T21:26:41.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.429+0000 7f53904cc700 1 --2- 172.21.15.33:0/4275742990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5388104c30 0x7f538810b4e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.429+0000 7f53904cc700 1 --2- 172.21.15.33:0/4275742990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53881030a0 0x7f53881034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.429+0000 7f53904cc700 1 --2- 172.21.15.33:0/4275742990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5388104290 0x7f53881046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.429+0000 7f53904cc700 1 -- 172.21.15.33:0/4275742990 >> 172.21.15.33:0/4275742990 conn(0x7f53880fe610 msgr2=0x7f5388100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.430+0000 7f53904cc700 1 -- 172.21.15.33:0/4275742990 shutdown_connections 2024-05-08T21:26:41.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.430+0000 7f53904cc700 1 -- 172.21.15.33:0/4275742990 wait complete. 2024-05-08T21:26:41.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.431+0000 7f53904cc700 1 Processor -- start 2024-05-08T21:26:41.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.431+0000 7f53904cc700 1 -- start start 2024-05-08T21:26:41.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.431+0000 7f53904cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53881030a0 0x7f5388199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.431+0000 7f53904cc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5388104290 0x7f5388199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.431+0000 7f53904cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5388104c30 0x7f53881a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.431+0000 7f53904cc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53881a0b80 con 0x7f53881030a0 2024-05-08T21:26:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.431+0000 7f53904cc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53881a0cc0 con 0x7f5388104290 2024-05-08T21:26:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.431+0000 7f53904cc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53881a0e50 con 0x7f5388104c30 2024-05-08T21:26:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.432+0000 7f538e268700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53881030a0 0x7f5388199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.432+0000 7f538e268700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53881030a0 0x7f5388199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57632/0 (socket says 172.21.15.33:57632) 2024-05-08T21:26:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.432+0000 7f538e268700 1 -- 172.21.15.33:0/587425776 learned_addr learned my addr 172.21.15.33:0/587425776 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.432+0000 7f538ea69700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5388104c30 0x7f53881a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.432+0000 7f538e268700 1 -- 172.21.15.33:0/587425776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5388104c30 msgr2=0x7f53881a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:41.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.432+0000 7f538e268700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5388104c30 0x7f53881a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.432+0000 7f538da67700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5388104290 0x7f5388199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.432+0000 7f538e268700 1 -- 172.21.15.33:0/587425776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5388104290 msgr2=0x7f5388199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.432+0000 7f538e268700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5388104290 0x7f5388199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.432+0000 7f538e268700 1 -- 172.21.15.33:0/587425776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5380009d00 con 0x7f53881030a0 2024-05-08T21:26:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.432+0000 7f538da67700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5388104290 0x7f5388199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:26:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.432+0000 7f538e268700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53881030a0 0x7f5388199050 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f538400d980 tx=0x7f538400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.433+0000 7f537f7fe700 1 -- 172.21.15.33:0/587425776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538400de60 con 0x7f53881030a0 2024-05-08T21:26:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.433+0000 7f537f7fe700 1 -- 172.21.15.33:0/587425776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53840044b0 con 0x7f53881030a0 2024-05-08T21:26:41.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.433+0000 7f537f7fe700 1 -- 172.21.15.33:0/587425776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384003b40 con 0x7f53881030a0 2024-05-08T21:26:41.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.434+0000 7f53904cc700 1 -- 172.21.15.33:0/587425776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53881a1130 con 0x7f53881030a0 2024-05-08T21:26:41.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.434+0000 7f53904cc700 1 -- 172.21.15.33:0/587425776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53881a15f0 con 0x7f53881030a0 2024-05-08T21:26:41.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.438+0000 7f539c686b80 1 Processor -- start 2024-05-08T21:26:41.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.439+0000 7f537f7fe700 1 -- 172.21.15.33:0/587425776 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5384003ca0 con 0x7f53881030a0 2024-05-08T21:26:41.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.439+0000 7f539c686b80 1 -- start start 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.439+0000 7f539c686b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237450960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.439+0000 7f539c686b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237450ea0 0x562237451320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.439+0000 7f539c686b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237451860 0x562237451ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.439+0000 7f539c686b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622373a7150 con 0x562237450ea0 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.439+0000 7f539c686b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622373a4b10 con 0x562237450540 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.439+0000 7f539c686b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622373a0400 con 0x562237451860 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.439+0000 7f538da67700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237451860 0x562237451ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.439+0000 7f538da67700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237451860 0x562237451ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:50874/0 (socket says 172.21.15.33:50874) 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.439+0000 7f538da67700 1 -- 172.21.15.33:0/4162792255 learned_addr learned my addr 172.21.15.33:0/4162792255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.440+0000 7f538da67700 1 -- 172.21.15.33:0/4162792255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 msgr2=0x562237450960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.440+0000 7f538da67700 1 --2- 172.21.15.33:0/4162792255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237450960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.440+0000 7f537f7fe700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53740a11a0 0x7f53740a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.440+0000 7f538da67700 1 -- 172.21.15.33:0/4162792255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237450ea0 msgr2=0x562237451320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.440+0000 7f538da67700 1 --2- 172.21.15.33:0/4162792255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237450ea0 0x562237451320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.440+0000 7f538da67700 1 -- 172.21.15.33:0/4162792255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622372213b0 con 0x562237451860 2024-05-08T21:26:41.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.440+0000 7f537f7fe700 1 -- 172.21.15.33:0/587425776 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f53840cc900 con 0x7f53881030a0 2024-05-08T21:26:41.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.440+0000 7f538da67700 1 --2- 172.21.15.33:0/4162792255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237451860 0x562237451ce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5378009d10 tx=0x7f537800e5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed428a91a4f8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:41.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.440+0000 7f537cff9700 1 -- 172.21.15.33:0/4162792255 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537800ede0 con 0x562237451860 2024-05-08T21:26:41.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.440+0000 7f537cff9700 1 -- 172.21.15.33:0/4162792255 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5378009e60 con 0x562237451860 2024-05-08T21:26:41.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.440+0000 7f537cff9700 1 -- 172.21.15.33:0/4162792255 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53780106c0 con 0x562237451860 2024-05-08T21:26:41.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.440+0000 7f538ea69700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53740a11a0 0x7f53740a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.441+0000 7f539c686b80 1 -- 172.21.15.33:0/4162792255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237451860 msgr2=0x562237451ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:41.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.441+0000 7f538ea69700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53740a11a0 0x7f53740a3660 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f53800062a0 tx=0x7f53800061f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:41.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.441+0000 7f539c686b80 1 --2- 172.21.15.33:0/4162792255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237451860 0x562237451ce0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5378009d10 tx=0x7f537800e5d0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.441+0000 7f539c686b80 1 -- 172.21.15.33:0/4162792255 shutdown_connections 2024-05-08T21:26:41.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.441+0000 7f539c686b80 1 --2- 172.21.15.33:0/4162792255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237451860 0x562237451ce0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.441+0000 7f539c686b80 1 --2- 172.21.15.33:0/4162792255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237450960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.441+0000 7f539c686b80 1 --2- 172.21.15.33:0/4162792255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237450ea0 0x562237451320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.441+0000 7f539c686b80 1 -- 172.21.15.33:0/4162792255 >> 172.21.15.33:0/4162792255 conn(0x5622373ae210 msgr2=0x56223744ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.441+0000 7f539c686b80 1 -- 172.21.15.33:0/4162792255 shutdown_connections 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.441+0000 7f539c686b80 1 -- 172.21.15.33:0/4162792255 wait complete. 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f539c686b80 1 -- start start 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f539c686b80 1 Processor -- start 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f539c686b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237459ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f539c686b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450ea0 0x56223745a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f539c686b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237451860 0x562237461470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f539c686b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223733c560 con 0x562237451860 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f539c686b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223733c200 con 0x562237450540 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f539c686b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622372236a0 con 0x562237450ea0 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f539c686b80 10 client.0 _ll_register_callbacks cb 0x562237345b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f538ea69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237451860 0x562237461470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f538ea69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237451860 0x562237461470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57646/0 (socket says 172.21.15.33:57646) 2024-05-08T21:26:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f538ea69700 1 -- 172.21.15.33:0/1580943286 learned_addr learned my addr 172.21.15.33:0/1580943286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.442+0000 7f538da67700 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450ea0 0x56223745a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f538e268700 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237459ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f538da67700 1 -- 172.21.15.33:0/1580943286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 msgr2=0x562237459ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f538da67700 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237459ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f538da67700 1 -- 172.21.15.33:0/1580943286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237451860 msgr2=0x562237461470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f538da67700 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237451860 0x562237461470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f538da67700 1 -- 172.21.15.33:0/1580943286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5378009940 con 0x562237450ea0 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f538e268700 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237459ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f538da67700 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450ea0 0x56223745a430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5378009ce0 tx=0x7f53780052e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e5cf1cad69a2d12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f5371ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f5371ffb700 1 -- 172.21.15.33:0/1580943286 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378020070 con 0x562237450ea0 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f539c686b80 1 -- 172.21.15.33:0/1580943286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622372f9270 con 0x562237450ea0 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f5371ffb700 1 -- 172.21.15.33:0/1580943286 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f5378004020 con 0x562237450ea0 2024-05-08T21:26:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f5371ffb700 1 -- 172.21.15.33:0/1580943286 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5378005750 con 0x562237450ea0 2024-05-08T21:26:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.443+0000 7f539c686b80 10 client.27418 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.444+0000 7f539c686b80 1 -- 172.21.15.33:0/1580943286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5622371cdb60 con 0x562237450ea0 2024-05-08T21:26:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.444+0000 7f5371ffb700 1 -- 172.21.15.33:0/1580943286 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5622372f9270 con 0x562237450ea0 2024-05-08T21:26:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.444+0000 7f5371ffb700 1 -- 172.21.15.33:0/1580943286 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f537800c070 con 0x562237450ea0 2024-05-08T21:26:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.444+0000 7f539c686b80 10 client.27418 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.444+0000 7f539c686b80 10 client.27418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.444+0000 7f539c686b80 20 client.27418 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.444+0000 7f539c686b80 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562237465b70 0x562237467f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.444+0000 7f539c686b80 4 client.27418 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.444+0000 7f539c686b80 1 -- 172.21.15.33:0/1580943286 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56223741f810 con 0x562237465b70 2024-05-08T21:26:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.445+0000 7f538e268700 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562237465b70 0x562237467f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.446+0000 7f538e268700 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562237465b70 0x562237467f90 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f68c5d091853e6a server_cookie=f9460ebae7cdc4e2 in_seq=0 out_seq=0 2024-05-08T21:26:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.446+0000 7f5371ffb700 10 client.27418 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.448+0000 7f5371ffb700 1 -- 172.21.15.33:0/1580943286 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56223741f810 con 0x562237465b70 2024-05-08T21:26:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.448+0000 7f5371ffb700 10 client.27418 handle_command_reply: tid=0 2024-05-08T21:26:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.448+0000 7f539c686b80 1 client.27418 shutdown 2024-05-08T21:26:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.448+0000 7f539c686b80 1 -- 172.21.15.33:0/1580943286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450ea0 msgr2=0x56223745a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.448+0000 7f539c686b80 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450ea0 0x56223745a430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5378009ce0 tx=0x7f53780052e0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.448+0000 7f539c686b80 1 -- 172.21.15.33:0/1580943286 shutdown_connections 2024-05-08T21:26:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.448+0000 7f539c686b80 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562237465b70 0x562237467f90 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.448+0000 7f5371ffb700 0 client.27418 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.448+0000 7f539c686b80 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237451860 0x562237461470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.448+0000 7f539c686b80 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237459ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.448+0000 7f539c686b80 1 --2- 172.21.15.33:0/1580943286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450ea0 0x56223745a430 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.448+0000 7f539c686b80 1 -- 172.21.15.33:0/1580943286 >> 172.21.15.33:0/1580943286 conn(0x5622373ae210 msgr2=0x56223744a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.449+0000 7f539c686b80 1 -- 172.21.15.33:0/1580943286 shutdown_connections 2024-05-08T21:26:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.449+0000 7f539c686b80 1 -- 172.21.15.33:0/1580943286 wait complete. 2024-05-08T21:26:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.449+0000 7f539c686b80 20 client.27418 trim_cache size 0 max 16384 2024-05-08T21:26:41.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.471+0000 7f539c686b80 1 Processor -- start 2024-05-08T21:26:41.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.472+0000 7f539c686b80 1 -- start start 2024-05-08T21:26:41.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.472+0000 7f539c686b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237459400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.472+0000 7f539c686b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450ea0 0x562237461430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.472+0000 7f539c686b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237451860 0x562237459ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.472+0000 7f539c686b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223744ba70 con 0x562237451860 2024-05-08T21:26:41.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.472+0000 7f539c686b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223745b130 con 0x562237450540 2024-05-08T21:26:41.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.472+0000 7f539c686b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223744b860 con 0x562237450ea0 2024-05-08T21:26:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.472+0000 7f538e268700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237459400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.472+0000 7f538e268700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237459400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56112/0 (socket says 172.21.15.33:56112) 2024-05-08T21:26:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.472+0000 7f538e268700 1 -- 172.21.15.33:0/3700734086 learned_addr learned my addr 172.21.15.33:0/3700734086 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.473+0000 7f538e268700 1 -- 172.21.15.33:0/3700734086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450ea0 msgr2=0x562237461430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:26:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.473+0000 7f538e268700 1 --2- 172.21.15.33:0/3700734086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450ea0 0x562237461430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.473+0000 7f538e268700 1 -- 172.21.15.33:0/3700734086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237451860 msgr2=0x562237459ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.473+0000 7f538e268700 1 --2- 172.21.15.33:0/3700734086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237451860 0x562237459ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.473+0000 7f538e268700 1 -- 172.21.15.33:0/3700734086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562237366560 con 0x562237450540 2024-05-08T21:26:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.473+0000 7f538e268700 1 --2- 172.21.15.33:0/3700734086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237459400 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5384095e90 tx=0x7f5384004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4afaee7214c90e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.473+0000 7f537d7fa700 1 -- 172.21.15.33:0/3700734086 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384099990 con 0x562237450540 2024-05-08T21:26:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.473+0000 7f537d7fa700 1 -- 172.21.15.33:0/3700734086 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f538409d7c0 con 0x562237450540 2024-05-08T21:26:41.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.473+0000 7f537d7fa700 1 -- 172.21.15.33:0/3700734086 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384097e50 con 0x562237450540 2024-05-08T21:26:41.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.474+0000 7f539c686b80 1 -- 172.21.15.33:0/3700734086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 msgr2=0x562237459400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:41.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.474+0000 7f539c686b80 1 --2- 172.21.15.33:0/3700734086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237459400 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5384095e90 tx=0x7f5384004750 comp rx=0 tx=0).stop 2024-05-08T21:26:41.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.475+0000 7f539c686b80 1 -- 172.21.15.33:0/3700734086 shutdown_connections 2024-05-08T21:26:41.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.475+0000 7f539c686b80 1 --2- 172.21.15.33:0/3700734086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237451860 0x562237459ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.475+0000 7f539c686b80 1 --2- 172.21.15.33:0/3700734086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237450540 0x562237459400 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.475+0000 7f539c686b80 1 --2- 172.21.15.33:0/3700734086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450ea0 0x562237461430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.475+0000 7f539c686b80 1 -- 172.21.15.33:0/3700734086 >> 172.21.15.33:0/3700734086 conn(0x5622373ae210 msgr2=0x562237451c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:41.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.475+0000 7f539c686b80 1 -- 172.21.15.33:0/3700734086 shutdown_connections 2024-05-08T21:26:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.475+0000 7f539c686b80 1 -- 172.21.15.33:0/3700734086 wait complete. 2024-05-08T21:26:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.475+0000 7f539c686b80 1 -- start start 2024-05-08T21:26:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.475+0000 7f539c686b80 1 Processor -- start 2024-05-08T21:26:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f539c686b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450540 0x562237460d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f539c686b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237450ea0 0x562237461290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f539c686b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237451860 0x56223745f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f539c686b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223745fa90 con 0x562237450ea0 2024-05-08T21:26:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f539c686b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223745fc00 con 0x562237451860 2024-05-08T21:26:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f539c686b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223745fdc0 con 0x562237450540 2024-05-08T21:26:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f539c686b80 10 client.0 _ll_register_callbacks cb 0x562237345b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:41.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f538ea69700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237451860 0x56223745f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f538da67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237450ea0 0x562237461290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f538ea69700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237451860 0x56223745f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56126/0 (socket says 172.21.15.33:56126) 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f538ea69700 1 -- 172.21.15.33:0/2574114875 learned_addr learned my addr 172.21.15.33:0/2574114875 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f538e268700 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450540 0x562237460d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f538ea69700 1 -- 172.21.15.33:0/2574114875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450540 msgr2=0x562237460d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f538ea69700 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450540 0x562237460d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f538ea69700 1 -- 172.21.15.33:0/2574114875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237450ea0 msgr2=0x562237461290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f538ea69700 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237450ea0 0x562237461290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f538ea69700 1 -- 172.21.15.33:0/2574114875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53881a15f0 con 0x562237451860 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f538e268700 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450540 0x562237460d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.476+0000 7f538da67700 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237450ea0 0x562237461290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.477+0000 7f538ea69700 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237451860 0x56223745f3b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5380009fd0 tx=0x7f5380004cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25562091ddcabe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.477+0000 7f5371ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:26:41.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.477+0000 7f5371ffb700 1 -- 172.21.15.33:0/2574114875 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380005620 con 0x562237451860 2024-05-08T21:26:41.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.477+0000 7f5371ffb700 1 -- 172.21.15.33:0/2574114875 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f5380023040 con 0x562237451860 2024-05-08T21:26:41.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.477+0000 7f539c686b80 1 -- 172.21.15.33:0/2574114875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5378009e60 con 0x562237451860 2024-05-08T21:26:41.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.477+0000 7f5371ffb700 1 -- 172.21.15.33:0/2574114875 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5380005780 con 0x562237451860 2024-05-08T21:26:41.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.477+0000 7f539c686b80 10 client.27419 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:41.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.477+0000 7f539c686b80 1 -- 172.21.15.33:0/2574114875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x562237225cd0 con 0x562237451860 2024-05-08T21:26:41.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.477+0000 7f5371ffb700 1 -- 172.21.15.33:0/2574114875 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5378009e60 con 0x562237451860 2024-05-08T21:26:41.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.478+0000 7f5371ffb700 1 -- 172.21.15.33:0/2574114875 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f538001b430 con 0x562237451860 2024-05-08T21:26:41.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.478+0000 7f539c686b80 10 client.27419 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:41.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.478+0000 7f539c686b80 10 client.27419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:41.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.478+0000 7f539c686b80 20 client.27419 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.478+0000 7f539c686b80 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562237465b70 0x5622374bf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.478+0000 7f539c686b80 4 client.27419 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.478+0000 7f539c686b80 1 -- 172.21.15.33:0/2574114875 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562237398ec0 con 0x562237465b70 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.478+0000 7f538e268700 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562237465b70 0x5622374bf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.478+0000 7f538e268700 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562237465b70 0x5622374bf520 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26648d410d59c278 server_cookie=164195037e3635fe in_seq=0 out_seq=0 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.478+0000 7f5371ffb700 10 client.27419 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f5371ffb700 1 -- 172.21.15.33:0/2574114875 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562237398ec0 con 0x562237465b70 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f5371ffb700 10 client.27419 handle_command_reply: tid=0 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f539c686b80 1 client.27419 shutdown 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f539c686b80 1 -- 172.21.15.33:0/2574114875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237451860 msgr2=0x56223745f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f539c686b80 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237451860 0x56223745f3b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5380009fd0 tx=0x7f5380004cd0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f539c686b80 1 -- 172.21.15.33:0/2574114875 shutdown_connections 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f539c686b80 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x562237465b70 0x5622374bf520 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f539c686b80 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562237451860 0x56223745f3b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f539c686b80 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562237450540 0x562237460d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f539c686b80 1 --2- 172.21.15.33:0/2574114875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562237450ea0 0x562237461290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f539c686b80 1 -- 172.21.15.33:0/2574114875 >> 172.21.15.33:0/2574114875 conn(0x5622373ae210 msgr2=0x562237459400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f5371ffb700 0 client.27419 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f539c686b80 1 -- 172.21.15.33:0/2574114875 shutdown_connections 2024-05-08T21:26:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.479+0000 7f539c686b80 1 -- 172.21.15.33:0/2574114875 wait complete. 2024-05-08T21:26:41.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f539c686b80 20 client.27419 trim_cache size 0 max 16384 2024-05-08T21:26:41.484 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:26:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:26:41.484 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:26:41.484 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:26:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f53904cc700 1 -- 172.21.15.33:0/587425776 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53740a11a0 msgr2=0x7f53740a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f53904cc700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53740a11a0 0x7f53740a3660 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f53800062a0 tx=0x7f53800061f0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f53904cc700 1 -- 172.21.15.33:0/587425776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53881030a0 msgr2=0x7f5388199050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f53904cc700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53881030a0 0x7f5388199050 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f538400d980 tx=0x7f538400dd40 comp rx=0 tx=0).stop 2024-05-08T21:26:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f53904cc700 1 -- 172.21.15.33:0/587425776 shutdown_connections 2024-05-08T21:26:41.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f53904cc700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5388104c30 0x7f53881a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f53904cc700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53881030a0 0x7f5388199050 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f53904cc700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53740a11a0 0x7f53740a3660 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f53904cc700 1 --2- 172.21.15.33:0/587425776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5388104290 0x7f5388199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f53904cc700 1 -- 172.21.15.33:0/587425776 >> 172.21.15.33:0/587425776 conn(0x7f53880fe610 msgr2=0x7f5388107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f53904cc700 1 -- 172.21.15.33:0/587425776 shutdown_connections 2024-05-08T21:26:41.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.480+0000 7f53904cc700 1 -- 172.21.15.33:0/587425776 wait complete. 2024-05-08T21:26:41.503 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:26:41.503 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:26:41.539 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:41 smithi033 ceph-mon[42528]: pgmap v521: 129 pgs: 129 active+clean; 265 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.1 MiB/s wr, 713 op/s 2024-05-08T21:26:41.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.687+0000 7f7310168700 1 -- 172.21.15.33:0/2046174910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7308100e80 msgr2=0x7f73081012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:41.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.687+0000 7f7310168700 1 --2- 172.21.15.33:0/2046174910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7308100e80 0x7f73081012a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7304009990 tx=0x7f7304009ca0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.687+0000 7f7310168700 1 -- 172.21.15.33:0/2046174910 shutdown_connections 2024-05-08T21:26:41.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.687+0000 7f7310168700 1 --2- 172.21.15.33:0/2046174910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7308102990 0x7f73081092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.687+0000 7f7310168700 1 --2- 172.21.15.33:0/2046174910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7308100e80 0x7f73081012a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.687+0000 7f7310168700 1 --2- 172.21.15.33:0/2046174910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7308101fd0 0x7f7308102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.687+0000 7f7310168700 1 -- 172.21.15.33:0/2046174910 >> 172.21.15.33:0/2046174910 conn(0x7f73080fc410 msgr2=0x7f73080fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:41.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.689+0000 7f7310168700 1 -- 172.21.15.33:0/2046174910 shutdown_connections 2024-05-08T21:26:41.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.689+0000 7f7310168700 1 -- 172.21.15.33:0/2046174910 wait complete. 2024-05-08T21:26:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.690+0000 7f7310168700 1 Processor -- start 2024-05-08T21:26:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.690+0000 7f7310168700 1 -- start start 2024-05-08T21:26:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.691+0000 7f7310168700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7308100e80 0x7f7308194ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.691+0000 7f7310168700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7308101fd0 0x7f7308195000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.691+0000 7f7310168700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7308102990 0x7f730819c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.691+0000 7f7310168700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730819c660 con 0x7f7308100e80 2024-05-08T21:26:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.691+0000 7f7310168700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730819c7d0 con 0x7f7308101fd0 2024-05-08T21:26:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.691+0000 7f7310168700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730819c940 con 0x7f7308102990 2024-05-08T21:26:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.691+0000 7f730df04700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7308100e80 0x7f7308194ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.691+0000 7f730df04700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7308100e80 0x7f7308194ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57690/0 (socket says 172.21.15.33:57690) 2024-05-08T21:26:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.691+0000 7f730df04700 1 -- 172.21.15.33:0/1249599047 learned_addr learned my addr 172.21.15.33:0/1249599047 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.691+0000 7f730e705700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7308102990 0x7f730819c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.692+0000 7f730df04700 1 -- 172.21.15.33:0/1249599047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7308102990 msgr2=0x7f730819c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.692+0000 7f730df04700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7308102990 0x7f730819c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.692+0000 7f730df04700 1 -- 172.21.15.33:0/1249599047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7308101fd0 msgr2=0x7f7308195000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:41.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.692+0000 7f730df04700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7308101fd0 0x7f7308195000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:41.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.692+0000 7f730df04700 1 -- 172.21.15.33:0/1249599047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73040095f0 con 0x7f7308100e80 2024-05-08T21:26:41.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.692+0000 7f730df04700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7308100e80 0x7f7308194ac0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f73040092e0 tx=0x7f7304005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:41.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.693+0000 7f72feffd700 1 -- 172.21.15.33:0/1249599047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730401d070 con 0x7f7308100e80 2024-05-08T21:26:41.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.693+0000 7f72feffd700 1 -- 172.21.15.33:0/1249599047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f730400f460 con 0x7f7308100e80 2024-05-08T21:26:41.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.693+0000 7f72feffd700 1 -- 172.21.15.33:0/1249599047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304021670 con 0x7f7308100e80 2024-05-08T21:26:41.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.693+0000 7f7310168700 1 -- 172.21.15.33:0/1249599047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730819cbc0 con 0x7f7308100e80 2024-05-08T21:26:41.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.693+0000 7f7310168700 1 -- 172.21.15.33:0/1249599047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730819d020 con 0x7f7308100e80 2024-05-08T21:26:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.696+0000 7f7310168700 1 -- 172.21.15.33:0/1249599047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7308067d10 con 0x7f7308100e80 2024-05-08T21:26:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.697+0000 7f72feffd700 1 -- 172.21.15.33:0/1249599047 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f730400f890 con 0x7f7308100e80 2024-05-08T21:26:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.697+0000 7f72feffd700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f72f40a1270 0x7f72f40a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.697+0000 7f72feffd700 1 -- 172.21.15.33:0/1249599047 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f73040cdd60 con 0x7f7308100e80 2024-05-08T21:26:41.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.697+0000 7f730d703700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f72f40a1270 0x7f72f40a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:41.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.698+0000 7f730d703700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f72f40a1270 0x7f72f40a3730 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f72f8005950 tx=0x7f72f80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:41.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:41.704+0000 7f72feffd700 1 -- 172.21.15.33:0/1249599047 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7304097200 con 0x7f7308100e80 2024-05-08T21:26:41.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:41 smithi071 ceph-mon[47074]: pgmap v521: 129 pgs: 129 active+clean; 265 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.1 MiB/s wr, 713 op/s 2024-05-08T21:26:41.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:41 smithi082 ceph-mon[47512]: pgmap v521: 129 pgs: 129 active+clean; 265 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.1 MiB/s wr, 713 op/s 2024-05-08T21:26:42.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.084+0000 7f7310168700 1 -- 172.21.15.33:0/1249599047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f730810c530 con 0x7f7308100e80 2024-05-08T21:26:42.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.086+0000 7f72feffd700 1 -- 172.21.15.33:0/1249599047 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10899 (secure 0 0 0) 0x7f730409a820 con 0x7f7308100e80 2024-05-08T21:26:42.087 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:26:42.087 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:25:13.388928+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:26:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.089+0000 7f7310168700 1 -- 172.21.15.33:0/1249599047 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f72f40a1270 msgr2=0x7f72f40a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.089+0000 7f7310168700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f72f40a1270 0x7f72f40a3730 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f72f8005950 tx=0x7f72f80058e0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.089+0000 7f7310168700 1 -- 172.21.15.33:0/1249599047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7308100e80 msgr2=0x7f7308194ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.089+0000 7f7310168700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7308100e80 0x7f7308194ac0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f73040092e0 tx=0x7f7304005e80 comp rx=0 tx=0).stop 2024-05-08T21:26:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.089+0000 7f7310168700 1 -- 172.21.15.33:0/1249599047 shutdown_connections 2024-05-08T21:26:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.089+0000 7f7310168700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7308102990 0x7f730819c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.089+0000 7f7310168700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7308100e80 0x7f7308194ac0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.089+0000 7f7310168700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f72f40a1270 0x7f72f40a3730 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.089+0000 7f7310168700 1 --2- 172.21.15.33:0/1249599047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7308101fd0 0x7f7308195000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.089+0000 7f7310168700 1 -- 172.21.15.33:0/1249599047 >> 172.21.15.33:0/1249599047 conn(0x7f73080fc410 msgr2=0x7f7308107830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.089+0000 7f7310168700 1 -- 172.21.15.33:0/1249599047 shutdown_connections 2024-05-08T21:26:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.089+0000 7f7310168700 1 -- 172.21.15.33:0/1249599047 wait complete. 2024-05-08T21:26:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 39 2024-05-08T21:26:42.115 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:26:42.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.341+0000 7f831f684700 1 -- 172.21.15.33:0/944238986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83180ffd60 msgr2=0x7f8318100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.341+0000 7f831f684700 1 --2- 172.21.15.33:0/944238986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83180ffd60 0x7f8318100180 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f83080099c0 tx=0x7f8308009cd0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.343+0000 7f831f684700 1 -- 172.21.15.33:0/944238986 shutdown_connections 2024-05-08T21:26:42.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.343+0000 7f831f684700 1 --2- 172.21.15.33:0/944238986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83181018f0 0x7f83181081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.343+0000 7f831f684700 1 --2- 172.21.15.33:0/944238986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83180ffd60 0x7f8318100180 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.343+0000 7f831f684700 1 --2- 172.21.15.33:0/944238986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8318100f50 0x7f83181013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.343+0000 7f831f684700 1 -- 172.21.15.33:0/944238986 >> 172.21.15.33:0/944238986 conn(0x7f83180fb290 msgr2=0x7f83180fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.344+0000 7f831f684700 1 -- 172.21.15.33:0/944238986 shutdown_connections 2024-05-08T21:26:42.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.344+0000 7f831f684700 1 -- 172.21.15.33:0/944238986 wait complete. 2024-05-08T21:26:42.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.344+0000 7f831f684700 1 Processor -- start 2024-05-08T21:26:42.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.344+0000 7f831f684700 1 -- start start 2024-05-08T21:26:42.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831f684700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83180ffd60 0x7f8318194b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831f684700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8318100f50 0x7f8318195050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831f684700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83181018f0 0x7f831819c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831f684700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831819c640 con 0x7f8318100f50 2024-05-08T21:26:42.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831f684700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831819c7b0 con 0x7f83180ffd60 2024-05-08T21:26:42.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831f684700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831819c970 con 0x7f83181018f0 2024-05-08T21:26:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831d420700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83180ffd60 0x7f8318194b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831d420700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83180ffd60 0x7f8318194b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56162/0 (socket says 172.21.15.33:56162) 2024-05-08T21:26:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831d420700 1 -- 172.21.15.33:0/3724661835 learned_addr learned my addr 172.21.15.33:0/3724661835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831dc21700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83181018f0 0x7f831819c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831cc1f700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8318100f50 0x7f8318195050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831d420700 1 -- 172.21.15.33:0/3724661835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83181018f0 msgr2=0x7f831819c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831d420700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83181018f0 0x7f831819c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831d420700 1 -- 172.21.15.33:0/3724661835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8318100f50 msgr2=0x7f8318195050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831d420700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8318100f50 0x7f8318195050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.345+0000 7f831d420700 1 -- 172.21.15.33:0/3724661835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8314009520 con 0x7f83180ffd60 2024-05-08T21:26:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.346+0000 7f831d420700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83180ffd60 0x7f8318194b10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8308000c00 tx=0x7f8308009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.346+0000 7f830e7fc700 1 -- 172.21.15.33:0/3724661835 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830801d070 con 0x7f83180ffd60 2024-05-08T21:26:42.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.346+0000 7f830e7fc700 1 -- 172.21.15.33:0/3724661835 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8308005030 con 0x7f83180ffd60 2024-05-08T21:26:42.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.346+0000 7f831f684700 1 -- 172.21.15.33:0/3724661835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83080095f0 con 0x7f83180ffd60 2024-05-08T21:26:42.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.346+0000 7f830e7fc700 1 -- 172.21.15.33:0/3724661835 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308004b50 con 0x7f83180ffd60 2024-05-08T21:26:42.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.346+0000 7f831f684700 1 -- 172.21.15.33:0/3724661835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831819ce20 con 0x7f83180ffd60 2024-05-08T21:26:42.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.349+0000 7f830e7fc700 1 -- 172.21.15.33:0/3724661835 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8308004cb0 con 0x7f83180ffd60 2024-05-08T21:26:42.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.349+0000 7f830e7fc700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f83040a11f0 0x7f83040a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.349+0000 7f830e7fc700 1 -- 172.21.15.33:0/3724661835 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f83080cd7e0 con 0x7f83180ffd60 2024-05-08T21:26:42.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.349+0000 7f831cc1f700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f83040a11f0 0x7f83040a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.350+0000 7f831cc1f700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f83040a11f0 0x7f83040a36b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8314009d80 tx=0x7f8314009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f832b83eb80 1 Processor -- start 2024-05-08T21:26:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f832b83eb80 1 -- start start 2024-05-08T21:26:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f832b83eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 0x55600fae3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f832b83eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3ea0 0x55600fae4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f832b83eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae4860 0x55600fae4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f832b83eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600fa3a150 con 0x55600fae3ea0 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f832b83eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600fa37b10 con 0x55600fae4860 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f832b83eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600fa33400 con 0x55600fae3540 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f831d420700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 0x55600fae3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f831d420700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 0x55600fae3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:50926/0 (socket says 172.21.15.33:50926) 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f831d420700 1 -- 172.21.15.33:0/4095207405 learned_addr learned my addr 172.21.15.33:0/4095207405 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f831cc1f700 1 --2- 172.21.15.33:0/4095207405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3ea0 0x55600fae4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.351+0000 7f831dc21700 1 --2- 172.21.15.33:0/4095207405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae4860 0x55600fae4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.352+0000 7f831d420700 1 -- 172.21.15.33:0/4095207405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae4860 msgr2=0x55600fae4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.352+0000 7f831d420700 1 --2- 172.21.15.33:0/4095207405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae4860 0x55600fae4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.352+0000 7f831d420700 1 -- 172.21.15.33:0/4095207405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3ea0 msgr2=0x55600fae4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.352+0000 7f831d420700 1 --2- 172.21.15.33:0/4095207405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3ea0 0x55600fae4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.352+0000 7f831d420700 1 -- 172.21.15.33:0/4095207405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55600f8b43b0 con 0x55600fae3540 2024-05-08T21:26:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.352+0000 7f831d420700 1 --2- 172.21.15.33:0/4095207405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 0x55600fae3960 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f830809a890 tx=0x7f830809a800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a546481cbd5fd6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.352+0000 7f83037fe700 1 -- 172.21.15.33:0/4095207405 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830809a900 con 0x55600fae3540 2024-05-08T21:26:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.352+0000 7f83037fe700 1 -- 172.21.15.33:0/4095207405 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f830809e730 con 0x55600fae3540 2024-05-08T21:26:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.352+0000 7f83037fe700 1 -- 172.21.15.33:0/4095207405 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830809ab60 con 0x55600fae3540 2024-05-08T21:26:42.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.353+0000 7f832b83eb80 1 -- 172.21.15.33:0/4095207405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 msgr2=0x55600fae3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.353+0000 7f832b83eb80 1 --2- 172.21.15.33:0/4095207405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 0x55600fae3960 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f830809a890 tx=0x7f830809a800 comp rx=0 tx=0).stop 2024-05-08T21:26:42.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.353+0000 7f832b83eb80 1 -- 172.21.15.33:0/4095207405 shutdown_connections 2024-05-08T21:26:42.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.353+0000 7f832b83eb80 1 --2- 172.21.15.33:0/4095207405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae4860 0x55600fae4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.353+0000 7f832b83eb80 1 --2- 172.21.15.33:0/4095207405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 0x55600fae3960 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.353+0000 7f832b83eb80 1 --2- 172.21.15.33:0/4095207405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3ea0 0x55600fae4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.353+0000 7f832b83eb80 1 -- 172.21.15.33:0/4095207405 >> 172.21.15.33:0/4095207405 conn(0x55600fa41210 msgr2=0x55600faddde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.353+0000 7f832b83eb80 1 -- 172.21.15.33:0/4095207405 shutdown_connections 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.353+0000 7f832b83eb80 1 -- 172.21.15.33:0/4095207405 wait complete. 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.353+0000 7f832b83eb80 1 -- start start 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.354+0000 7f832b83eb80 1 Processor -- start 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.354+0000 7f832b83eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3540 0x55600faecb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.354+0000 7f832b83eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 0x55600faed040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.354+0000 7f832b83eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae4860 0x55600faf4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.354+0000 7f832b83eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600f9cf560 con 0x55600fae3540 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.354+0000 7f832b83eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600f9cf200 con 0x55600fae3ea0 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.354+0000 7f832b83eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600f8b66a0 con 0x55600fae4860 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.354+0000 7f832b83eb80 10 client.0 _ll_register_callbacks cb 0x55600f9d8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.354+0000 7f831cc1f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 0x55600faed040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.354+0000 7f831cc1f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 0x55600faed040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56182/0 (socket says 172.21.15.33:56182) 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.354+0000 7f831cc1f700 1 -- 172.21.15.33:0/2721790854 learned_addr learned my addr 172.21.15.33:0/2721790854 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.354+0000 7f831dc21700 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae4860 0x55600faf4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f831d420700 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3540 0x55600faecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f831dc21700 1 -- 172.21.15.33:0/2721790854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 msgr2=0x55600faed040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f831dc21700 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 0x55600faed040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f831dc21700 1 -- 172.21.15.33:0/2721790854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3540 msgr2=0x55600faecb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f831dc21700 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3540 0x55600faecb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f831dc21700 1 -- 172.21.15.33:0/2721790854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f831819ce20 con 0x55600fae4860 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f831dc21700 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae4860 0x55600faf4080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8310007af0 tx=0x7f8310007b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1212ae004870e1d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f8300ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f8300ff9700 1 -- 172.21.15.33:0/2721790854 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831000e040 con 0x55600fae4860 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f8300ff9700 1 -- 172.21.15.33:0/2721790854 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8310015920 con 0x55600fae4860 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f832b83eb80 1 -- 172.21.15.33:0/2721790854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55600f98c270 con 0x55600fae4860 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f8300ff9700 1 -- 172.21.15.33:0/2721790854 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8310015ae0 con 0x55600fae4860 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f832b83eb80 10 client.27439 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.355+0000 7f832b83eb80 1 -- 172.21.15.33:0/2721790854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55600f860b60 con 0x55600fae4860 2024-05-08T21:26:42.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.356+0000 7f8300ff9700 1 -- 172.21.15.33:0/2721790854 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55600f98c270 con 0x55600fae4860 2024-05-08T21:26:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.356+0000 7f8300ff9700 1 -- 172.21.15.33:0/2721790854 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8310016c50 con 0x55600fae4860 2024-05-08T21:26:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.356+0000 7f832b83eb80 10 client.27439 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.356+0000 7f832b83eb80 10 client.27439 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:26:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.356+0000 7f832b83eb80 20 client.27439 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.356+0000 7f832b83eb80 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55600faf8890 0x55600fafacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.356+0000 7f832b83eb80 4 client.27439 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:42.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.356+0000 7f832b83eb80 1 -- 172.21.15.33:0/2721790854 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55600fab2810 con 0x55600faf8890 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.356+0000 7f831d420700 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55600faf8890 0x55600fafacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f831d420700 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55600faf8890 0x55600fafacb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8474c8830c5f5247 server_cookie=b747a705a53122ed in_seq=0 out_seq=0 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f8300ff9700 10 client.27439 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f8300ff9700 1 -- 172.21.15.33:0/2721790854 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55600fab2810 con 0x55600faf8890 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f8300ff9700 10 client.27439 handle_command_reply: tid=0 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f832b83eb80 1 client.27439 shutdown 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f832b83eb80 1 -- 172.21.15.33:0/2721790854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae4860 msgr2=0x55600faf4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f832b83eb80 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae4860 0x55600faf4080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8310007af0 tx=0x7f8310007b20 comp rx=0 tx=0).stop 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f832b83eb80 1 -- 172.21.15.33:0/2721790854 shutdown_connections 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f832b83eb80 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55600faf8890 0x55600fafacb0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f832b83eb80 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae4860 0x55600faf4080 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f832b83eb80 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3540 0x55600faecb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f8300ff9700 0 client.27439 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f832b83eb80 1 --2- 172.21.15.33:0/2721790854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 0x55600faed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.357+0000 7f832b83eb80 1 -- 172.21.15.33:0/2721790854 >> 172.21.15.33:0/2721790854 conn(0x55600fa41210 msgr2=0x55600fadd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.358+0000 7f832b83eb80 1 -- 172.21.15.33:0/2721790854 shutdown_connections 2024-05-08T21:26:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.358+0000 7f832b83eb80 1 -- 172.21.15.33:0/2721790854 wait complete. 2024-05-08T21:26:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.358+0000 7f832b83eb80 20 client.27439 trim_cache size 0 max 16384 2024-05-08T21:26:42.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.373+0000 7f832b83eb80 1 Processor -- start 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.373+0000 7f832b83eb80 1 -- start start 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.373+0000 7f832b83eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 0x55600faf4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.373+0000 7f832b83eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 0x55600faf4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.373+0000 7f832b83eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae4860 0x55600faead70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f832b83eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600faf5240 con 0x55600fae4860 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f832b83eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600faec670 con 0x55600fae3ea0 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f832b83eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600fadadf0 con 0x55600fae3540 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f831cc1f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 0x55600faf4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f831dc21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae4860 0x55600faead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f831cc1f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 0x55600faf4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56196/0 (socket says 172.21.15.33:56196) 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f831cc1f700 1 -- 172.21.15.33:0/1397008360 learned_addr learned my addr 172.21.15.33:0/1397008360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f831dc21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae4860 0x55600faead70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57740/0 (socket says 172.21.15.33:57740) 2024-05-08T21:26:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f831d420700 1 --2- 172.21.15.33:0/1397008360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 0x55600faf4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f831cc1f700 1 -- 172.21.15.33:0/1397008360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 msgr2=0x55600faf4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f831cc1f700 1 --2- 172.21.15.33:0/1397008360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 0x55600faf4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f831cc1f700 1 -- 172.21.15.33:0/1397008360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae4860 msgr2=0x55600faead70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f831cc1f700 1 --2- 172.21.15.33:0/1397008360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae4860 0x55600faead70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f831cc1f700 1 -- 172.21.15.33:0/1397008360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55600f9f9560 con 0x55600fae3ea0 2024-05-08T21:26:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.374+0000 7f831d420700 1 --2- 172.21.15.33:0/1397008360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 0x55600faf4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.375+0000 7f831cc1f700 1 --2- 172.21.15.33:0/1397008360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 0x55600faf4040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8314012550 tx=0x7f8314014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cafedaad17951db8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.375+0000 7f8303fff700 1 -- 172.21.15.33:0/1397008360 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314029440 con 0x55600fae3ea0 2024-05-08T21:26:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.375+0000 7f8303fff700 1 -- 172.21.15.33:0/1397008360 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8314016070 con 0x55600fae3ea0 2024-05-08T21:26:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.375+0000 7f8303fff700 1 -- 172.21.15.33:0/1397008360 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831401f740 con 0x55600fae3ea0 2024-05-08T21:26:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.376+0000 7f832b83eb80 1 -- 172.21.15.33:0/1397008360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 msgr2=0x55600faf4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.376+0000 7f832b83eb80 1 --2- 172.21.15.33:0/1397008360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 0x55600faf4040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8314012550 tx=0x7f8314014000 comp rx=0 tx=0).stop 2024-05-08T21:26:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.376+0000 7f832b83eb80 1 -- 172.21.15.33:0/1397008360 shutdown_connections 2024-05-08T21:26:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.376+0000 7f832b83eb80 1 --2- 172.21.15.33:0/1397008360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae4860 0x55600faead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.376+0000 7f832b83eb80 1 --2- 172.21.15.33:0/1397008360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae3540 0x55600faf4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.376+0000 7f832b83eb80 1 --2- 172.21.15.33:0/1397008360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3ea0 0x55600faf4040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.376+0000 7f832b83eb80 1 -- 172.21.15.33:0/1397008360 >> 172.21.15.33:0/1397008360 conn(0x55600fa41210 msgr2=0x55600fae4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.376+0000 7f832b83eb80 1 -- 172.21.15.33:0/1397008360 shutdown_connections 2024-05-08T21:26:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.376+0000 7f832b83eb80 1 -- 172.21.15.33:0/1397008360 wait complete. 2024-05-08T21:26:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.376+0000 7f832b83eb80 1 -- start start 2024-05-08T21:26:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.376+0000 7f832b83eb80 1 Processor -- start 2024-05-08T21:26:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f832b83eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3540 0x55600faf01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f832b83eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3ea0 0x55600faf0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f832b83eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae4860 0x55600faf3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f832b83eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600faf3680 con 0x55600fae3ea0 2024-05-08T21:26:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f832b83eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600faf37f0 con 0x55600fae3540 2024-05-08T21:26:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f832b83eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600faf39b0 con 0x55600fae4860 2024-05-08T21:26:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f832b83eb80 10 client.0 _ll_register_callbacks cb 0x55600f9d8b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f831d420700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3540 0x55600faf01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f831d420700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3540 0x55600faf01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56208/0 (socket says 172.21.15.33:56208) 2024-05-08T21:26:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f831d420700 1 -- 172.21.15.33:0/3078037787 learned_addr learned my addr 172.21.15.33:0/3078037787 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f831cc1f700 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3ea0 0x55600faf0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f831dc21700 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae4860 0x55600faf3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f831d420700 1 -- 172.21.15.33:0/3078037787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae4860 msgr2=0x55600faf3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f831d420700 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae4860 0x55600faf3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f831d420700 1 -- 172.21.15.33:0/3078037787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3ea0 msgr2=0x55600faf0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f831d420700 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3ea0 0x55600faf0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.377+0000 7f831d420700 1 -- 172.21.15.33:0/3078037787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55600f8b43b0 con 0x55600fae3540 2024-05-08T21:26:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.378+0000 7f831d420700 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3540 0x55600faf01f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8308009990 tx=0x7f830809e8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc06ca03d35a45cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.378+0000 7f8300ff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:26:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.378+0000 7f8300ff9700 1 -- 172.21.15.33:0/3078037787 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83080b2040 con 0x55600fae3540 2024-05-08T21:26:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.378+0000 7f8300ff9700 1 -- 172.21.15.33:0/3078037787 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f83080a2cf0 con 0x55600fae3540 2024-05-08T21:26:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.378+0000 7f832b83eb80 1 -- 172.21.15.33:0/3078037787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f830809e730 con 0x55600fae3540 2024-05-08T21:26:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.383+0000 7f8300ff9700 1 -- 172.21.15.33:0/3078037787 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f830809a980 con 0x55600fae3540 2024-05-08T21:26:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.383+0000 7f832b83eb80 10 client.27440 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.383+0000 7f832b83eb80 1 -- 172.21.15.33:0/3078037787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f8314011df0 con 0x55600fae3540 2024-05-08T21:26:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.383+0000 7f8300ff9700 1 -- 172.21.15.33:0/3078037787 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f830809e730 con 0x55600fae3540 2024-05-08T21:26:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.384+0000 7f8300ff9700 1 -- 172.21.15.33:0/3078037787 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f830809ab60 con 0x55600fae3540 2024-05-08T21:26:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.384+0000 7f832b83eb80 10 client.27440 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.384+0000 7f832b83eb80 10 client.27440 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:26:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.384+0000 7f832b83eb80 20 client.27440 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.384+0000 7f832b83eb80 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55600fb4f830 0x55600fb51c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.384+0000 7f832b83eb80 4 client.27440 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:26:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.384+0000 7f832b83eb80 1 -- 172.21.15.33:0/3078037787 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55600fa2bec0 con 0x55600fb4f830 2024-05-08T21:26:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.384+0000 7f831cc1f700 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55600fb4f830 0x55600fb51c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.384+0000 7f8300ff9700 10 client.27440 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:26:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.384+0000 7f831cc1f700 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55600fb4f830 0x55600fb51c50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=61f50dcdb6e7d2a0 server_cookie=b29a0cacf506f566 in_seq=0 out_seq=0 2024-05-08T21:26:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f8300ff9700 1 -- 172.21.15.33:0/3078037787 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55600fa2bec0 con 0x55600fb4f830 2024-05-08T21:26:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f8300ff9700 10 client.27440 handle_command_reply: tid=0 2024-05-08T21:26:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f832b83eb80 1 client.27440 shutdown 2024-05-08T21:26:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f832b83eb80 1 -- 172.21.15.33:0/3078037787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3540 msgr2=0x55600faf01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f832b83eb80 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3540 0x55600faf01f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8308009990 tx=0x7f830809e8a0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f832b83eb80 1 -- 172.21.15.33:0/3078037787 shutdown_connections 2024-05-08T21:26:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f832b83eb80 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55600fb4f830 0x55600fb51c50 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f832b83eb80 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55600fae4860 0x55600faf3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f832b83eb80 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600fae3540 0x55600faf01f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f832b83eb80 1 --2- 172.21.15.33:0/3078037787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55600fae3ea0 0x55600faf0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f832b83eb80 1 -- 172.21.15.33:0/3078037787 >> 172.21.15.33:0/3078037787 conn(0x55600fa41210 msgr2=0x55600faff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f8300ff9700 0 client.27440 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:26:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f832b83eb80 1 -- 172.21.15.33:0/3078037787 shutdown_connections 2024-05-08T21:26:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.385+0000 7f832b83eb80 1 -- 172.21.15.33:0/3078037787 wait complete. 2024-05-08T21:26:42.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.386+0000 7f832b83eb80 20 client.27440 trim_cache size 0 max 16384 2024-05-08T21:26:42.394 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:26:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.387+0000 7f831f684700 1 -- 172.21.15.33:0/3724661835 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f83040a11f0 msgr2=0x7f83040a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.387+0000 7f831f684700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f83040a11f0 0x7f83040a36b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8314009d80 tx=0x7f8314009260 comp rx=0 tx=0).stop 2024-05-08T21:26:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.387+0000 7f831f684700 1 -- 172.21.15.33:0/3724661835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83180ffd60 msgr2=0x7f8318194b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.387+0000 7f831f684700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83180ffd60 0x7f8318194b10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8308000c00 tx=0x7f8308009580 comp rx=0 tx=0).stop 2024-05-08T21:26:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.388+0000 7f831f684700 1 -- 172.21.15.33:0/3724661835 shutdown_connections 2024-05-08T21:26:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.388+0000 7f831f684700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83181018f0 0x7f831819c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.388+0000 7f831f684700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f83040a11f0 0x7f83040a36b0 secure :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8314009d80 tx=0x7f8314009260 comp rx=0 tx=0).stop 2024-05-08T21:26:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.388+0000 7f831f684700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83180ffd60 0x7f8318194b10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.388+0000 7f831f684700 1 --2- 172.21.15.33:0/3724661835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8318100f50 0x7f8318195050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.388+0000 7f831f684700 1 -- 172.21.15.33:0/3724661835 >> 172.21.15.33:0/3724661835 conn(0x7f83180fb290 msgr2=0x7f8318068250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.391+0000 7f831f684700 1 -- 172.21.15.33:0/3724661835 shutdown_connections 2024-05-08T21:26:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.393+0000 7f831f684700 1 -- 172.21.15.33:0/3724661835 wait complete. 2024-05-08T21:26:42.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:26:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.603+0000 7fec3c086700 1 -- 172.21.15.33:0/3390570261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec34100f50 msgr2=0x7fec341013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.603+0000 7fec3c086700 1 --2- 172.21.15.33:0/3390570261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec34100f50 0x7fec341013b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fec240099c0 tx=0x7fec24009cd0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.603+0000 7fec3c086700 1 -- 172.21.15.33:0/3390570261 shutdown_connections 2024-05-08T21:26:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.603+0000 7fec3c086700 1 --2- 172.21.15.33:0/3390570261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec341018f0 0x7fec341081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.603+0000 7fec3c086700 1 --2- 172.21.15.33:0/3390570261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec340ffd60 0x7fec34100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.603+0000 7fec3c086700 1 --2- 172.21.15.33:0/3390570261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec34100f50 0x7fec341013b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.603+0000 7fec3c086700 1 -- 172.21.15.33:0/3390570261 >> 172.21.15.33:0/3390570261 conn(0x7fec340fb290 msgr2=0x7fec340fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.603+0000 7fec3c086700 1 -- 172.21.15.33:0/3390570261 shutdown_connections 2024-05-08T21:26:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.603+0000 7fec3c086700 1 -- 172.21.15.33:0/3390570261 wait complete. 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec3c086700 1 Processor -- start 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec3c086700 1 -- start start 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec3c086700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec340ffd60 0x7fec34196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec3c086700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec34100f50 0x7fec341972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec3c086700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec341018f0 0x7fec3419e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec3c086700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3419e890 con 0x7fec340ffd60 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec3c086700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3419ea00 con 0x7fec341018f0 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec3c086700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3419ebc0 con 0x7fec34100f50 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec39e22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec340ffd60 0x7fec34196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec39e22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec340ffd60 0x7fec34196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57776/0 (socket says 172.21.15.33:57776) 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec39e22700 1 -- 172.21.15.33:0/378267142 learned_addr learned my addr 172.21.15.33:0/378267142 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec39621700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec34100f50 0x7fec341972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.604+0000 7fec3a623700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec341018f0 0x7fec3419e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.605+0000 7fec39e22700 1 -- 172.21.15.33:0/378267142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec34100f50 msgr2=0x7fec341972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.605+0000 7fec39e22700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec34100f50 0x7fec341972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.605+0000 7fec39e22700 1 -- 172.21.15.33:0/378267142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec341018f0 msgr2=0x7fec3419e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.605+0000 7fec39e22700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec341018f0 0x7fec3419e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.605+0000 7fec39e22700 1 -- 172.21.15.33:0/378267142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240095f0 con 0x7fec340ffd60 2024-05-08T21:26:42.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.605+0000 7fec39e22700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec340ffd60 0x7fec34196da0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fec3000e970 tx=0x7fec3000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.605+0000 7fec2affd700 1 -- 172.21.15.33:0/378267142 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30009790 con 0x7fec340ffd60 2024-05-08T21:26:42.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.605+0000 7fec2affd700 1 -- 172.21.15.33:0/378267142 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec3000eec0 con 0x7fec340ffd60 2024-05-08T21:26:42.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.605+0000 7fec3c086700 1 -- 172.21.15.33:0/378267142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3419eea0 con 0x7fec340ffd60 2024-05-08T21:26:42.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.605+0000 7fec3c086700 1 -- 172.21.15.33:0/378267142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3419f2d0 con 0x7fec340ffd60 2024-05-08T21:26:42.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.605+0000 7fec2affd700 1 -- 172.21.15.33:0/378267142 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30003c10 con 0x7fec340ffd60 2024-05-08T21:26:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.610+0000 7fec48240b80 1 Processor -- start 2024-05-08T21:26:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.610+0000 7fec2affd700 1 -- 172.21.15.33:0/378267142 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fec3000cbd0 con 0x7fec340ffd60 2024-05-08T21:26:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.610+0000 7fec48240b80 1 -- start start 2024-05-08T21:26:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.610+0000 7fec48240b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x55976598f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.610+0000 7fec48240b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598fea0 0x559765990320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.610+0000 7fec48240b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x559765990ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.610+0000 7fec48240b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597658e6150 con 0x55976598f540 2024-05-08T21:26:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.610+0000 7fec48240b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597658e3b10 con 0x559765990860 2024-05-08T21:26:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.610+0000 7fec48240b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597658df400 con 0x55976598fea0 2024-05-08T21:26:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec3a623700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x55976598f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec3a623700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x55976598f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57788/0 (socket says 172.21.15.33:57788) 2024-05-08T21:26:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec3a623700 1 -- 172.21.15.33:0/156265094 learned_addr learned my addr 172.21.15.33:0/156265094 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec39621700 1 --2- 172.21.15.33:0/156265094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x559765990ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec2affd700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fec200a1220 0x7fec200a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec39e22700 1 --2- 172.21.15.33:0/156265094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598fea0 0x559765990320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec3a623700 1 -- 172.21.15.33:0/156265094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598fea0 msgr2=0x559765990320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec3a623700 1 --2- 172.21.15.33:0/156265094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598fea0 0x559765990320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec3a623700 1 -- 172.21.15.33:0/156265094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 msgr2=0x559765990ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec3a623700 1 --2- 172.21.15.33:0/156265094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x559765990ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec3a623700 1 -- 172.21.15.33:0/156265094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597657603b0 con 0x55976598f540 2024-05-08T21:26:42.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec3a623700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fec200a1220 0x7fec200a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec2affd700 1 -- 172.21.15.33:0/378267142 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fec30014070 con 0x7fec340ffd60 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.611+0000 7fec3a623700 1 --2- 172.21.15.33:0/156265094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x55976598f960 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fec2c011790 tx=0x7fec2c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e146e4886574a8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.612+0000 7fec1ffff700 1 -- 172.21.15.33:0/156265094 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00e040 con 0x55976598f540 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.612+0000 7fec3a623700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fec200a1220 0x7fec200a36e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fec2c022000 tx=0x7fec2c01f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.612+0000 7fec1ffff700 1 -- 172.21.15.33:0/156265094 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec2c00f040 con 0x55976598f540 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.612+0000 7fec1ffff700 1 -- 172.21.15.33:0/156265094 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0238c0 con 0x55976598f540 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.612+0000 7fec48240b80 1 -- 172.21.15.33:0/156265094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 msgr2=0x55976598f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.612+0000 7fec48240b80 1 --2- 172.21.15.33:0/156265094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x55976598f960 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fec2c011790 tx=0x7fec2c011aa0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.613+0000 7fec48240b80 1 -- 172.21.15.33:0/156265094 shutdown_connections 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.613+0000 7fec48240b80 1 --2- 172.21.15.33:0/156265094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x559765990ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.613+0000 7fec48240b80 1 --2- 172.21.15.33:0/156265094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x55976598f960 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.613+0000 7fec48240b80 1 --2- 172.21.15.33:0/156265094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598fea0 0x559765990320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.613+0000 7fec48240b80 1 -- 172.21.15.33:0/156265094 >> 172.21.15.33:0/156265094 conn(0x5597658ed210 msgr2=0x559765989de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.613+0000 7fec48240b80 1 -- 172.21.15.33:0/156265094 shutdown_connections 2024-05-08T21:26:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.613+0000 7fec48240b80 1 -- 172.21.15.33:0/156265094 wait complete. 2024-05-08T21:26:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.613+0000 7fec48240b80 1 -- start start 2024-05-08T21:26:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.613+0000 7fec48240b80 1 Processor -- start 2024-05-08T21:26:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec48240b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x559765998df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec48240b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598fea0 0x559765999330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec48240b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x5597659a0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec48240b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55976587b560 con 0x55976598f540 2024-05-08T21:26:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec48240b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55976587b200 con 0x559765990860 2024-05-08T21:26:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec48240b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597657626a0 con 0x55976598fea0 2024-05-08T21:26:42.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec48240b80 10 client.0 _ll_register_callbacks cb 0x559765884b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec39e22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x559765998df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec39e22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x559765998df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57794/0 (socket says 172.21.15.33:57794) 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec39621700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598fea0 0x559765999330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec39e22700 1 -- 172.21.15.33:0/2552276833 learned_addr learned my addr 172.21.15.33:0/2552276833 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec3a623700 1 --2- 172.21.15.33:0/2552276833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x5597659a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec39621700 1 -- 172.21.15.33:0/2552276833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 msgr2=0x5597659a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec39621700 1 --2- 172.21.15.33:0/2552276833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x5597659a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec39621700 1 -- 172.21.15.33:0/2552276833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 msgr2=0x559765998df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec39621700 1 --2- 172.21.15.33:0/2552276833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x559765998df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.614+0000 7fec39621700 1 -- 172.21.15.33:0/2552276833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c011380 con 0x55976598fea0 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.615+0000 7fec39621700 1 --2- 172.21.15.33:0/2552276833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598fea0 0x559765999330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fec240098e0 tx=0x7fec24009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47a700c72bad5e9e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.615+0000 7fec1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:26:42.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.615+0000 7fec1d7fa700 1 -- 172.21.15.33:0/2552276833 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2401d040 con 0x55976598fea0 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.615+0000 7fec1d7fa700 1 -- 172.21.15.33:0/2552276833 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fec2400bcd0 con 0x55976598fea0 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.615+0000 7fec48240b80 1 -- 172.21.15.33:0/2552276833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559765838270 con 0x55976598fea0 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.615+0000 7fec1d7fa700 1 -- 172.21.15.33:0/2552276833 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec240054e0 con 0x55976598fea0 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.615+0000 7fec48240b80 10 client.27457 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.615+0000 7fec48240b80 1 -- 172.21.15.33:0/2552276833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55976570cb60 con 0x55976598fea0 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.615+0000 7fec1d7fa700 1 -- 172.21.15.33:0/2552276833 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559765838270 con 0x55976598fea0 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.616+0000 7fec1d7fa700 1 -- 172.21.15.33:0/2552276833 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fec2400f690 con 0x55976598fea0 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.616+0000 7fec48240b80 10 client.27457 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.616+0000 7fec48240b80 10 client.27457 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.616+0000 7fec48240b80 20 client.27457 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.616+0000 7fec48240b80 1 --2- 172.21.15.33:0/2552276833 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5597659a4c00 0x5597659a7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.616+0000 7fec48240b80 4 client.27457 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.616+0000 7fec48240b80 1 -- 172.21.15.33:0/2552276833 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55976595e810 con 0x5597659a4c00 2024-05-08T21:26:42.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.616+0000 7fec39e22700 1 --2- 172.21.15.33:0/2552276833 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5597659a4c00 0x5597659a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.616+0000 7fec1d7fa700 10 client.27457 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:26:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec39e22700 1 --2- 172.21.15.33:0/2552276833 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5597659a4c00 0x5597659a7020 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3e6d7eb0937bf580 server_cookie=541fc39012429483 in_seq=0 out_seq=0 2024-05-08T21:26:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec1d7fa700 1 -- 172.21.15.33:0/2552276833 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55976595e810 con 0x5597659a4c00 2024-05-08T21:26:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec1d7fa700 10 client.27457 handle_command_reply: tid=0 2024-05-08T21:26:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec48240b80 1 client.27457 shutdown 2024-05-08T21:26:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec48240b80 1 -- 172.21.15.33:0/2552276833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598fea0 msgr2=0x559765999330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec48240b80 1 --2- 172.21.15.33:0/2552276833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598fea0 0x559765999330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fec240098e0 tx=0x7fec24009510 comp rx=0 tx=0).stop 2024-05-08T21:26:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec48240b80 1 -- 172.21.15.33:0/2552276833 shutdown_connections 2024-05-08T21:26:42.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec48240b80 1 --2- 172.21.15.33:0/2552276833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x5597659a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec48240b80 1 --2- 172.21.15.33:0/2552276833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x559765998df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec48240b80 1 --2- 172.21.15.33:0/2552276833 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5597659a4c00 0x5597659a7020 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec48240b80 1 --2- 172.21.15.33:0/2552276833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598fea0 0x559765999330 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec1d7fa700 0 client.27457 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:26:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.617+0000 7fec48240b80 1 -- 172.21.15.33:0/2552276833 >> 172.21.15.33:0/2552276833 conn(0x5597658ed210 msgr2=0x5597659895a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.618+0000 7fec48240b80 1 -- 172.21.15.33:0/2552276833 shutdown_connections 2024-05-08T21:26:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.618+0000 7fec48240b80 1 -- 172.21.15.33:0/2552276833 wait complete. 2024-05-08T21:26:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.618+0000 7fec48240b80 20 client.27457 trim_cache size 0 max 16384 2024-05-08T21:26:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.640+0000 7fec48240b80 1 Processor -- start 2024-05-08T21:26:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.640+0000 7fec48240b80 1 -- start start 2024-05-08T21:26:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.640+0000 7fec48240b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x5597659a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.640+0000 7fec48240b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55976598fea0 0x5597659a0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.640+0000 7fec48240b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559765990860 0x559765998300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.640+0000 7fec48240b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55976599a030 con 0x55976598f540 2024-05-08T21:26:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.640+0000 7fec48240b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559765998960 con 0x55976598fea0 2024-05-08T21:26:42.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.640+0000 7fec48240b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559765986dc0 con 0x559765990860 2024-05-08T21:26:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.640+0000 7fec39e22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x5597659a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.640+0000 7fec39e22700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x5597659a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57796/0 (socket says 172.21.15.33:57796) 2024-05-08T21:26:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.640+0000 7fec39e22700 1 -- 172.21.15.33:0/2997466655 learned_addr learned my addr 172.21.15.33:0/2997466655 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.641+0000 7fec3a623700 1 --2- 172.21.15.33:0/2997466655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559765990860 0x559765998300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.641+0000 7fec39e22700 1 -- 172.21.15.33:0/2997466655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559765990860 msgr2=0x559765998300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.641+0000 7fec39e22700 1 --2- 172.21.15.33:0/2997466655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559765990860 0x559765998300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.641+0000 7fec39e22700 1 -- 172.21.15.33:0/2997466655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55976598fea0 msgr2=0x5597659a0290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:26:42.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.641+0000 7fec39e22700 1 --2- 172.21.15.33:0/2997466655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55976598fea0 0x5597659a0290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.641+0000 7fec39e22700 1 -- 172.21.15.33:0/2997466655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597658a5560 con 0x55976598f540 2024-05-08T21:26:42.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.642+0000 7fec39e22700 1 --2- 172.21.15.33:0/2997466655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x5597659a0a20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fec30099e20 tx=0x7fec30096300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8145da989d0f89ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.642+0000 7fec28ff9700 1 -- 172.21.15.33:0/2997466655 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3009aea0 con 0x55976598f540 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.642+0000 7fec28ff9700 1 -- 172.21.15.33:0/2997466655 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec30099e90 con 0x55976598f540 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.642+0000 7fec28ff9700 1 -- 172.21.15.33:0/2997466655 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3009a0f0 con 0x55976598f540 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.643+0000 7fec48240b80 1 -- 172.21.15.33:0/2997466655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 msgr2=0x5597659a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.643+0000 7fec48240b80 1 --2- 172.21.15.33:0/2997466655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x5597659a0a20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fec30099e20 tx=0x7fec30096300 comp rx=0 tx=0).stop 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.643+0000 7fec48240b80 1 -- 172.21.15.33:0/2997466655 shutdown_connections 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.643+0000 7fec48240b80 1 --2- 172.21.15.33:0/2997466655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559765990860 0x559765998300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.643+0000 7fec48240b80 1 --2- 172.21.15.33:0/2997466655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598f540 0x5597659a0a20 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.643+0000 7fec48240b80 1 --2- 172.21.15.33:0/2997466655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55976598fea0 0x5597659a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.643+0000 7fec48240b80 1 -- 172.21.15.33:0/2997466655 >> 172.21.15.33:0/2997466655 conn(0x5597658ed210 msgr2=0x559765990c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.643+0000 7fec48240b80 1 -- 172.21.15.33:0/2997466655 shutdown_connections 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.643+0000 7fec48240b80 1 -- 172.21.15.33:0/2997466655 wait complete. 2024-05-08T21:26:42.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.644+0000 7fec48240b80 1 -- start start 2024-05-08T21:26:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.644+0000 7fec48240b80 1 Processor -- start 2024-05-08T21:26:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.644+0000 7fec48240b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598f540 0x55976599bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.644+0000 7fec48240b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598fea0 0x55976599c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.644+0000 7fec48240b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x55976599f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.644+0000 7fec48240b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55976599f8b0 con 0x55976598fea0 2024-05-08T21:26:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.644+0000 7fec48240b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55976599fa20 con 0x559765990860 2024-05-08T21:26:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.644+0000 7fec48240b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55976599fbe0 con 0x55976598f540 2024-05-08T21:26:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.644+0000 7fec48240b80 10 client.0 _ll_register_callbacks cb 0x559765884b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:42.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec39e22700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598f540 0x55976599bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec39e22700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598f540 0x55976599bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51008/0 (socket says 172.21.15.33:51008) 2024-05-08T21:26:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec39e22700 1 -- 172.21.15.33:0/2906248774 learned_addr learned my addr 172.21.15.33:0/2906248774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec39621700 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598fea0 0x55976599c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec3a623700 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x55976599f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec39e22700 1 -- 172.21.15.33:0/2906248774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 msgr2=0x55976599f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec39e22700 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x55976599f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec39e22700 1 -- 172.21.15.33:0/2906248774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598fea0 msgr2=0x55976599c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec39e22700 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598fea0 0x55976599c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec39e22700 1 -- 172.21.15.33:0/2906248774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3419f2d0 con 0x55976598f540 2024-05-08T21:26:42.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec3a623700 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x55976599f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec39621700 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598fea0 0x55976599c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec39e22700 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598f540 0x55976599bf00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fec30009230 tx=0x7fec30098440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86b1d5e8efe0660f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec1d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec1d7fa700 1 -- 172.21.15.33:0/2906248774 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3009aea0 con 0x55976598f540 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec48240b80 1 -- 172.21.15.33:0/2906248774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec2c00f040 con 0x55976598f540 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec1d7fa700 1 -- 172.21.15.33:0/2906248774 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fec300a2310 con 0x55976598f540 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec1d7fa700 1 -- 172.21.15.33:0/2906248774 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec30099590 con 0x55976598f540 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec48240b80 10 client.27463 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.645+0000 7fec48240b80 1 -- 172.21.15.33:0/2906248774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559765764cd0 con 0x55976598f540 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.646+0000 7fec1d7fa700 1 -- 172.21.15.33:0/2906248774 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec2c00f040 con 0x55976598f540 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.646+0000 7fec1d7fa700 1 -- 172.21.15.33:0/2906248774 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fec300a1f10 con 0x55976598f540 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.646+0000 7fec48240b80 10 client.27463 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.646+0000 7fec48240b80 10 client.27463 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:26:42.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.646+0000 7fec48240b80 20 client.27463 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.646+0000 7fec48240b80 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5597659fbb90 0x5597659fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.646+0000 7fec48240b80 4 client.27463 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:26:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.646+0000 7fec48240b80 1 -- 172.21.15.33:0/2906248774 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5597658d7ec0 con 0x5597659fbb90 2024-05-08T21:26:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.646+0000 7fec39621700 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5597659fbb90 0x5597659fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.646+0000 7fec1d7fa700 10 client.27463 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:26:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.646+0000 7fec39621700 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5597659fbb90 0x5597659fdfb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=16e9c87c412c02a5 server_cookie=33416871495afd2f in_seq=0 out_seq=0 2024-05-08T21:26:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec1d7fa700 1 -- 172.21.15.33:0/2906248774 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5597658d7ec0 con 0x5597659fbb90 2024-05-08T21:26:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec1d7fa700 10 client.27463 handle_command_reply: tid=0 2024-05-08T21:26:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec48240b80 1 client.27463 shutdown 2024-05-08T21:26:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec48240b80 1 -- 172.21.15.33:0/2906248774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598f540 msgr2=0x55976599bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec48240b80 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598f540 0x55976599bf00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fec30009230 tx=0x7fec30098440 comp rx=0 tx=0).stop 2024-05-08T21:26:42.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec48240b80 1 -- 172.21.15.33:0/2906248774 shutdown_connections 2024-05-08T21:26:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec48240b80 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559765990860 0x55976599f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec48240b80 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5597659fbb90 0x5597659fdfb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec48240b80 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55976598f540 0x55976599bf00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec48240b80 1 --2- 172.21.15.33:0/2906248774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55976598fea0 0x55976599c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec48240b80 1 -- 172.21.15.33:0/2906248774 >> 172.21.15.33:0/2906248774 conn(0x5597658ed210 msgr2=0x559765993750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec1d7fa700 0 client.27463 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:26:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec48240b80 1 -- 172.21.15.33:0/2906248774 shutdown_connections 2024-05-08T21:26:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec48240b80 1 -- 172.21.15.33:0/2906248774 wait complete. 2024-05-08T21:26:42.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.647+0000 7fec48240b80 20 client.27463 trim_cache size 0 max 16384 2024-05-08T21:26:42.658 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:26:42.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.648+0000 7fec3c086700 1 -- 172.21.15.33:0/378267142 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fec200a1220 msgr2=0x7fec200a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.648+0000 7fec3c086700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fec200a1220 0x7fec200a36e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fec2c022000 tx=0x7fec2c01f000 comp rx=0 tx=0).stop 2024-05-08T21:26:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.648+0000 7fec3c086700 1 -- 172.21.15.33:0/378267142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec340ffd60 msgr2=0x7fec34196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.648+0000 7fec3c086700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec340ffd60 0x7fec34196da0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fec3000e970 tx=0x7fec3000ec80 comp rx=0 tx=0).stop 2024-05-08T21:26:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.648+0000 7fec3c086700 1 -- 172.21.15.33:0/378267142 shutdown_connections 2024-05-08T21:26:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.648+0000 7fec3c086700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fec200a1220 0x7fec200a36e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.648+0000 7fec3c086700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec341018f0 0x7fec3419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.648+0000 7fec3c086700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec340ffd60 0x7fec34196da0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.648+0000 7fec3c086700 1 --2- 172.21.15.33:0/378267142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec34100f50 0x7fec341972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.648+0000 7fec3c086700 1 -- 172.21.15.33:0/378267142 >> 172.21.15.33:0/378267142 conn(0x7fec340fb290 msgr2=0x7fec340681d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.648+0000 7fec3c086700 1 -- 172.21.15.33:0/378267142 shutdown_connections 2024-05-08T21:26:42.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.648+0000 7fec3c086700 1 -- 172.21.15.33:0/378267142 wait complete. 2024-05-08T21:26:42.667 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:42 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1249599047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:26:42.671 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:26:42.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:42 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1249599047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:26:42.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:42 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1249599047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:26:42.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.860+0000 7f70dc155700 1 -- 172.21.15.33:0/3632671511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d41018f0 msgr2=0x7f70d41081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.860+0000 7f70dc155700 1 --2- 172.21.15.33:0/3632671511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d41018f0 0x7f70d41081a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00b0d0 tx=0x7f70cc00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.860+0000 7f70dc155700 1 -- 172.21.15.33:0/3632671511 shutdown_connections 2024-05-08T21:26:42.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.860+0000 7f70dc155700 1 --2- 172.21.15.33:0/3632671511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d41018f0 0x7f70d41081a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.860+0000 7f70dc155700 1 --2- 172.21.15.33:0/3632671511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70d40ffd60 0x7f70d4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.860+0000 7f70dc155700 1 --2- 172.21.15.33:0/3632671511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d4100f50 0x7f70d41013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.860+0000 7f70dc155700 1 -- 172.21.15.33:0/3632671511 >> 172.21.15.33:0/3632671511 conn(0x7f70d40fb290 msgr2=0x7f70d40fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.860+0000 7f70dc155700 1 -- 172.21.15.33:0/3632671511 shutdown_connections 2024-05-08T21:26:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.860+0000 7f70dc155700 1 -- 172.21.15.33:0/3632671511 wait complete. 2024-05-08T21:26:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70dc155700 1 Processor -- start 2024-05-08T21:26:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70dc155700 1 -- start start 2024-05-08T21:26:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70dc155700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d40ffd60 0x7f70d4196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70dc155700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d4100f50 0x7f70d4197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70dc155700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70d41018f0 0x7f70d419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70dc155700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d419e900 con 0x7f70d40ffd60 2024-05-08T21:26:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70dc155700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d419ea70 con 0x7f70d4100f50 2024-05-08T21:26:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70dc155700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d419ec30 con 0x7f70d41018f0 2024-05-08T21:26:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70d9ef1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d40ffd60 0x7f70d4196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70d9ef1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d40ffd60 0x7f70d4196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57828/0 (socket says 172.21.15.33:57828) 2024-05-08T21:26:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70d9ef1700 1 -- 172.21.15.33:0/3743190237 learned_addr learned my addr 172.21.15.33:0/3743190237 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70d96f0700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d4100f50 0x7f70d4197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.861+0000 7f70da6f2700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70d41018f0 0x7f70d419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70d9ef1700 1 -- 172.21.15.33:0/3743190237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70d41018f0 msgr2=0x7f70d419e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70d9ef1700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70d41018f0 0x7f70d419e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70d9ef1700 1 -- 172.21.15.33:0/3743190237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d4100f50 msgr2=0x7f70d4197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70d9ef1700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d4100f50 0x7f70d4197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70d9ef1700 1 -- 172.21.15.33:0/3743190237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc009d00 con 0x7f70d40ffd60 2024-05-08T21:26:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70da6f2700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70d41018f0 0x7f70d419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70d96f0700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d4100f50 0x7f70d4197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70d9ef1700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d40ffd60 0x7f70d4196e10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f70d000e970 tx=0x7f70d000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70caffd700 1 -- 172.21.15.33:0/3743190237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0009790 con 0x7f70d40ffd60 2024-05-08T21:26:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70caffd700 1 -- 172.21.15.33:0/3743190237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70d000c7e0 con 0x7f70d40ffd60 2024-05-08T21:26:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70dc155700 1 -- 172.21.15.33:0/3743190237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d419ef10 con 0x7f70d40ffd60 2024-05-08T21:26:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70caffd700 1 -- 172.21.15.33:0/3743190237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0003bd0 con 0x7f70d40ffd60 2024-05-08T21:26:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.862+0000 7f70dc155700 1 -- 172.21.15.33:0/3743190237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d419f340 con 0x7f70d40ffd60 2024-05-08T21:26:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.868+0000 7f70caffd700 1 -- 172.21.15.33:0/3743190237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f70d000c950 con 0x7f70d40ffd60 2024-05-08T21:26:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.869+0000 7f70e830fb80 1 Processor -- start 2024-05-08T21:26:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.869+0000 7f70e830fb80 1 -- start start 2024-05-08T21:26:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.870+0000 7f70caffd700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f70c00a1220 0x7f70c00a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.870+0000 7f70e830fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa28950 0x562a7fa28d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.870+0000 7f70e830fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa292b0 0x562a7fa29730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.870+0000 7f70e830fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa29c70 0x562a7fa2a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.870+0000 7f70e830fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7f97b150 con 0x562a7fa29c70 2024-05-08T21:26:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.870+0000 7f70e830fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7f978b10 con 0x562a7fa28950 2024-05-08T21:26:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.870+0000 7f70e830fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7f974400 con 0x562a7fa292b0 2024-05-08T21:26:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.870+0000 7f70caffd700 1 -- 172.21.15.33:0/3743190237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f70d0014070 con 0x7f70d40ffd60 2024-05-08T21:26:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.870+0000 7f70da6f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa29c70 0x562a7fa2a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.870+0000 7f70da6f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa29c70 0x562a7fa2a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57834/0 (socket says 172.21.15.33:57834) 2024-05-08T21:26:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.870+0000 7f70da6f2700 1 -- 172.21.15.33:0/1360267260 learned_addr learned my addr 172.21.15.33:0/1360267260 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.870+0000 7f70d9ef1700 1 --2- 172.21.15.33:0/1360267260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa28950 0x562a7fa28d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.871+0000 7f70da6f2700 1 -- 172.21.15.33:0/1360267260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa292b0 msgr2=0x562a7fa29730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.871+0000 7f70da6f2700 1 --2- 172.21.15.33:0/1360267260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa292b0 0x562a7fa29730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.871+0000 7f70da6f2700 1 -- 172.21.15.33:0/1360267260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa28950 msgr2=0x562a7fa28d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.871+0000 7f70da6f2700 1 --2- 172.21.15.33:0/1360267260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa28950 0x562a7fa28d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.871+0000 7f70da6f2700 1 -- 172.21.15.33:0/1360267260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a7f7f53b0 con 0x562a7fa29c70 2024-05-08T21:26:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.871+0000 7f70da6f2700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f70c00a1220 0x7f70c00a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.872+0000 7f70da6f2700 1 --2- 172.21.15.33:0/1360267260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa29c70 0x562a7fa2a0f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00b0d0 tx=0x7f70cc0093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538d5756fa8e2073 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.872+0000 7f70da6f2700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f70c00a1220 0x7f70c00a36e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f70cc029000 tx=0x7f70cc02a000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.872+0000 7f70bffff700 1 -- 172.21.15.33:0/1360267260 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc00e030 con 0x562a7fa29c70 2024-05-08T21:26:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.872+0000 7f70bffff700 1 -- 172.21.15.33:0/1360267260 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70cc00bda0 con 0x562a7fa29c70 2024-05-08T21:26:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.872+0000 7f70bffff700 1 -- 172.21.15.33:0/1360267260 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc026670 con 0x562a7fa29c70 2024-05-08T21:26:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.873+0000 7f70e830fb80 1 -- 172.21.15.33:0/1360267260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa29c70 msgr2=0x562a7fa2a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.873+0000 7f70e830fb80 1 --2- 172.21.15.33:0/1360267260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa29c70 0x562a7fa2a0f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00b0d0 tx=0x7f70cc0093b0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.876+0000 7f70e830fb80 1 -- 172.21.15.33:0/1360267260 shutdown_connections 2024-05-08T21:26:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.876+0000 7f70e830fb80 1 --2- 172.21.15.33:0/1360267260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa29c70 0x562a7fa2a0f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.876+0000 7f70e830fb80 1 --2- 172.21.15.33:0/1360267260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa28950 0x562a7fa28d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.876+0000 7f70e830fb80 1 --2- 172.21.15.33:0/1360267260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa292b0 0x562a7fa29730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.876+0000 7f70e830fb80 1 -- 172.21.15.33:0/1360267260 >> 172.21.15.33:0/1360267260 conn(0x562a7f982210 msgr2=0x562a7fa1ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.876+0000 7f70e830fb80 1 -- 172.21.15.33:0/1360267260 shutdown_connections 2024-05-08T21:26:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.876+0000 7f70e830fb80 1 -- 172.21.15.33:0/1360267260 wait complete. 2024-05-08T21:26:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.877+0000 7f70e830fb80 1 -- start start 2024-05-08T21:26:42.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.877+0000 7f70e830fb80 1 Processor -- start 2024-05-08T21:26:42.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70e830fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa28950 0x562a7fa27c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70e830fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa292b0 0x562a7fa281c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70e830fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa29c70 0x562a7fa24810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70e830fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7f910560 con 0x562a7fa29c70 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70e830fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7f910200 con 0x562a7fa292b0 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70e830fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7f7f76a0 con 0x562a7fa28950 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70e830fb80 10 client.0 _ll_register_callbacks cb 0x562a7f919b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70d9ef1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa28950 0x562a7fa27c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70d96f0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa292b0 0x562a7fa281c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70d96f0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa292b0 0x562a7fa281c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56270/0 (socket says 172.21.15.33:56270) 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70d96f0700 1 -- 172.21.15.33:0/1091028457 learned_addr learned my addr 172.21.15.33:0/1091028457 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70da6f2700 1 --2- 172.21.15.33:0/1091028457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa29c70 0x562a7fa24810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70d96f0700 1 -- 172.21.15.33:0/1091028457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa28950 msgr2=0x562a7fa27c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70d96f0700 1 --2- 172.21.15.33:0/1091028457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa28950 0x562a7fa27c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70d96f0700 1 -- 172.21.15.33:0/1091028457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa29c70 msgr2=0x562a7fa24810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70d96f0700 1 --2- 172.21.15.33:0/1091028457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa29c70 0x562a7fa24810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.878+0000 7f70d96f0700 1 -- 172.21.15.33:0/1091028457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d4108af0 con 0x562a7fa292b0 2024-05-08T21:26:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.879+0000 7f70d96f0700 1 --2- 172.21.15.33:0/1091028457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa292b0 0x562a7fa281c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f70c4005950 tx=0x7f70c4010a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c22e649cdee88268 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.879+0000 7f70bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:26:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.879+0000 7f70bd7fa700 1 -- 172.21.15.33:0/1091028457 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4004460 con 0x562a7fa292b0 2024-05-08T21:26:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.879+0000 7f70bd7fa700 1 -- 172.21.15.33:0/1091028457 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f70c4003680 con 0x562a7fa292b0 2024-05-08T21:26:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.879+0000 7f70e830fb80 1 -- 172.21.15.33:0/1091028457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a7f8cd270 con 0x562a7fa292b0 2024-05-08T21:26:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.879+0000 7f70bd7fa700 1 -- 172.21.15.33:0/1091028457 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70c4003840 con 0x562a7fa292b0 2024-05-08T21:26:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.879+0000 7f70e830fb80 10 client.27464 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.880+0000 7f70e830fb80 1 -- 172.21.15.33:0/1091028457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x562a7f7a1b60 con 0x562a7fa292b0 2024-05-08T21:26:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.880+0000 7f70bd7fa700 1 -- 172.21.15.33:0/1091028457 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562a7f8cd270 con 0x562a7fa292b0 2024-05-08T21:26:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.880+0000 7f70bd7fa700 1 -- 172.21.15.33:0/1091028457 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f70c401a9c0 con 0x562a7fa292b0 2024-05-08T21:26:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.880+0000 7f70e830fb80 10 client.27464 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.880+0000 7f70e830fb80 10 client.27464 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:26:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.880+0000 7f70e830fb80 20 client.27464 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.880+0000 7f70e830fb80 1 --2- 172.21.15.33:0/1091028457 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562a7fa423b0 0x562a7fa447d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.880+0000 7f70e830fb80 4 client.27464 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.880+0000 7f70e830fb80 1 -- 172.21.15.33:0/1091028457 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a7f9f3810 con 0x562a7fa423b0 2024-05-08T21:26:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.880+0000 7f70d9ef1700 1 --2- 172.21.15.33:0/1091028457 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562a7fa423b0 0x562a7fa447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.881+0000 7f70bd7fa700 10 client.27464 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:26:42.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.881+0000 7f70d9ef1700 1 --2- 172.21.15.33:0/1091028457 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562a7fa423b0 0x562a7fa447d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45152a04cc9fc042 server_cookie=abae0cb430d28552 in_seq=0 out_seq=0 2024-05-08T21:26:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70bd7fa700 1 -- 172.21.15.33:0/1091028457 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a7f9f3810 con 0x562a7fa423b0 2024-05-08T21:26:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70bd7fa700 10 client.27464 handle_command_reply: tid=0 2024-05-08T21:26:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70e830fb80 1 client.27464 shutdown 2024-05-08T21:26:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70e830fb80 1 -- 172.21.15.33:0/1091028457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa292b0 msgr2=0x562a7fa281c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70e830fb80 1 --2- 172.21.15.33:0/1091028457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa292b0 0x562a7fa281c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f70c4005950 tx=0x7f70c4010a00 comp rx=0 tx=0).stop 2024-05-08T21:26:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70e830fb80 1 -- 172.21.15.33:0/1091028457 shutdown_connections 2024-05-08T21:26:42.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70e830fb80 1 --2- 172.21.15.33:0/1091028457 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562a7fa423b0 0x562a7fa447d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70e830fb80 1 --2- 172.21.15.33:0/1091028457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa29c70 0x562a7fa24810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70e830fb80 1 --2- 172.21.15.33:0/1091028457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa28950 0x562a7fa27c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70e830fb80 1 --2- 172.21.15.33:0/1091028457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa292b0 0x562a7fa281c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70bd7fa700 0 client.27464 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:26:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70e830fb80 1 -- 172.21.15.33:0/1091028457 >> 172.21.15.33:0/1091028457 conn(0x562a7f982210 msgr2=0x562a7fa1e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70e830fb80 1 -- 172.21.15.33:0/1091028457 shutdown_connections 2024-05-08T21:26:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70e830fb80 1 -- 172.21.15.33:0/1091028457 wait complete. 2024-05-08T21:26:42.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.882+0000 7f70e830fb80 20 client.27464 trim_cache size 0 max 16384 2024-05-08T21:26:42.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70e830fb80 1 Processor -- start 2024-05-08T21:26:42.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70e830fb80 1 -- start start 2024-05-08T21:26:42.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70e830fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa28950 0x562a7fa447a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70e830fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 0x562a7fa28d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70e830fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa29c70 0x562a7fa31780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70e830fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7fa1bdc0 con 0x562a7fa292b0 2024-05-08T21:26:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70e830fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7fa3e6d0 con 0x562a7fa29c70 2024-05-08T21:26:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70e830fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7fa499e0 con 0x562a7fa28950 2024-05-08T21:26:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70d96f0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 0x562a7fa28d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70d9ef1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa28950 0x562a7fa447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70d9ef1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa28950 0x562a7fa447a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51060/0 (socket says 172.21.15.33:51060) 2024-05-08T21:26:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70d9ef1700 1 -- 172.21.15.33:0/2618602343 learned_addr learned my addr 172.21.15.33:0/2618602343 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.901+0000 7f70da6f2700 1 --2- 172.21.15.33:0/2618602343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa29c70 0x562a7fa31780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.902+0000 7f70da6f2700 1 -- 172.21.15.33:0/2618602343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa28950 msgr2=0x562a7fa447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.902+0000 7f70da6f2700 1 --2- 172.21.15.33:0/2618602343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa28950 0x562a7fa447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.902+0000 7f70da6f2700 1 -- 172.21.15.33:0/2618602343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 msgr2=0x562a7fa28d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.902+0000 7f70da6f2700 1 --2- 172.21.15.33:0/2618602343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 0x562a7fa28d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.902+0000 7f70da6f2700 1 -- 172.21.15.33:0/2618602343 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a7f93a560 con 0x562a7fa29c70 2024-05-08T21:26:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.902+0000 7f70d96f0700 1 --2- 172.21.15.33:0/2618602343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 0x562a7fa28d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.902+0000 7f70da6f2700 1 --2- 172.21.15.33:0/2618602343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa29c70 0x562a7fa31780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f70cc007af0 tx=0x7f70cc00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c32d3cd691422f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.902+0000 7f70c8ff9700 1 -- 172.21.15.33:0/2618602343 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc00e040 con 0x562a7fa29c70 2024-05-08T21:26:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.902+0000 7f70c8ff9700 1 -- 172.21.15.33:0/2618602343 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70cc036070 con 0x562a7fa29c70 2024-05-08T21:26:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.902+0000 7f70c8ff9700 1 -- 172.21.15.33:0/2618602343 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc01d8b0 con 0x562a7fa29c70 2024-05-08T21:26:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.903+0000 7f70e830fb80 1 -- 172.21.15.33:0/2618602343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa29c70 msgr2=0x562a7fa31780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.903+0000 7f70e830fb80 1 --2- 172.21.15.33:0/2618602343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa29c70 0x562a7fa31780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f70cc007af0 tx=0x7f70cc00bf10 comp rx=0 tx=0).stop 2024-05-08T21:26:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.903+0000 7f70e830fb80 1 -- 172.21.15.33:0/2618602343 shutdown_connections 2024-05-08T21:26:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.903+0000 7f70e830fb80 1 --2- 172.21.15.33:0/2618602343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa29c70 0x562a7fa31780 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.903+0000 7f70e830fb80 1 --2- 172.21.15.33:0/2618602343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa28950 0x562a7fa447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.903+0000 7f70e830fb80 1 --2- 172.21.15.33:0/2618602343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 0x562a7fa28d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.903+0000 7f70e830fb80 1 -- 172.21.15.33:0/2618602343 >> 172.21.15.33:0/2618602343 conn(0x562a7f982210 msgr2=0x562a7fa1b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.903+0000 7f70e830fb80 1 -- 172.21.15.33:0/2618602343 shutdown_connections 2024-05-08T21:26:42.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.903+0000 7f70e830fb80 1 -- 172.21.15.33:0/2618602343 wait complete. 2024-05-08T21:26:42.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.904+0000 7f70e830fb80 1 -- start start 2024-05-08T21:26:42.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.904+0000 7f70e830fb80 1 Processor -- start 2024-05-08T21:26:42.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.905+0000 7f70e830fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa28950 0x562a7fa4f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.905+0000 7f70e830fb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 0x562a7fa4fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.905+0000 7f70e830fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa29c70 0x562a7fa97330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.905+0000 7f70e830fb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7fa502f0 con 0x562a7fa292b0 2024-05-08T21:26:42.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.905+0000 7f70e830fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7fa50460 con 0x562a7fa28950 2024-05-08T21:26:42.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.905+0000 7f70e830fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a7fa97870 con 0x562a7fa29c70 2024-05-08T21:26:42.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.905+0000 7f70e830fb80 10 client.0 _ll_register_callbacks cb 0x562a7f919b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:42.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.905+0000 7f70d96f0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 0x562a7fa4fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.905+0000 7f70d96f0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 0x562a7fa4fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57856/0 (socket says 172.21.15.33:57856) 2024-05-08T21:26:42.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.905+0000 7f70d96f0700 1 -- 172.21.15.33:0/2181671687 learned_addr learned my addr 172.21.15.33:0/2181671687 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:42.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.905+0000 7f70d9ef1700 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa28950 0x562a7fa4f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.905+0000 7f70da6f2700 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa29c70 0x562a7fa97330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70d96f0700 1 -- 172.21.15.33:0/2181671687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa29c70 msgr2=0x562a7fa97330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70d96f0700 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa29c70 0x562a7fa97330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70d96f0700 1 -- 172.21.15.33:0/2181671687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa28950 msgr2=0x562a7fa4f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:42.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70d96f0700 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa28950 0x562a7fa4f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70d96f0700 1 -- 172.21.15.33:0/2181671687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a7f7a1b60 con 0x562a7fa292b0 2024-05-08T21:26:42.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70d9ef1700 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa28950 0x562a7fa4f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:42.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70d96f0700 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 0x562a7fa4fdb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f70c4005950 tx=0x7f70c40052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d29424beaf2b9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:42.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70bd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:26:42.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70bd7fa700 1 -- 172.21.15.33:0/2181671687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c401aea0 con 0x562a7fa292b0 2024-05-08T21:26:42.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70bd7fa700 1 -- 172.21.15.33:0/2181671687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f70c4000e40 con 0x562a7fa292b0 2024-05-08T21:26:42.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70e830fb80 1 -- 172.21.15.33:0/2181671687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70cc00bda0 con 0x562a7fa292b0 2024-05-08T21:26:42.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70bd7fa700 1 -- 172.21.15.33:0/2181671687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70c4005c20 con 0x562a7fa292b0 2024-05-08T21:26:42.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70e830fb80 10 client.17574 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:42.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70e830fb80 1 -- 172.21.15.33:0/2181671687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x562a7f7f53b0 con 0x562a7fa292b0 2024-05-08T21:26:42.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.906+0000 7f70bd7fa700 1 -- 172.21.15.33:0/2181671687 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70cc00bda0 con 0x562a7fa292b0 2024-05-08T21:26:42.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.907+0000 7f70bd7fa700 1 -- 172.21.15.33:0/2181671687 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f70c4004750 con 0x562a7fa292b0 2024-05-08T21:26:42.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.907+0000 7f70e830fb80 10 client.17574 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:42.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.907+0000 7f70e830fb80 10 client.17574 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:26:42.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.907+0000 7f70e830fb80 20 client.17574 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:42.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.907+0000 7f70e830fb80 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562a7fa99260 0x562a7fa9b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:42.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.907+0000 7f70e830fb80 4 client.17574 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:26:42.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.907+0000 7f70e830fb80 1 -- 172.21.15.33:0/2181671687 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a7f96cec0 con 0x562a7fa99260 2024-05-08T21:26:42.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.907+0000 7f70d9ef1700 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562a7fa99260 0x562a7fa9b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:42.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.907+0000 7f70bd7fa700 10 client.17574 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:26:42.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70d9ef1700 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562a7fa99260 0x562a7fa9b680 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b121bee7788720b server_cookie=10554c2d5ea3a00f in_seq=0 out_seq=0 2024-05-08T21:26:42.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70bd7fa700 1 -- 172.21.15.33:0/2181671687 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a7f96cec0 con 0x562a7fa99260 2024-05-08T21:26:42.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70bd7fa700 10 client.17574 handle_command_reply: tid=0 2024-05-08T21:26:42.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70e830fb80 1 client.17574 shutdown 2024-05-08T21:26:42.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70e830fb80 1 -- 172.21.15.33:0/2181671687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 msgr2=0x562a7fa4fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70e830fb80 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 0x562a7fa4fdb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f70c4005950 tx=0x7f70c40052f0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70e830fb80 1 -- 172.21.15.33:0/2181671687 shutdown_connections 2024-05-08T21:26:42.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70e830fb80 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x562a7fa99260 0x562a7fa9b680 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70e830fb80 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a7fa29c70 0x562a7fa97330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70e830fb80 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a7fa28950 0x562a7fa4f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70e830fb80 1 --2- 172.21.15.33:0/2181671687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a7fa292b0 0x562a7fa4fdb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70bd7fa700 0 client.17574 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:26:42.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.908+0000 7f70e830fb80 1 -- 172.21.15.33:0/2181671687 >> 172.21.15.33:0/2181671687 conn(0x562a7f982210 msgr2=0x562a7fa30170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.909+0000 7f70e830fb80 1 -- 172.21.15.33:0/2181671687 shutdown_connections 2024-05-08T21:26:42.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.909+0000 7f70e830fb80 1 -- 172.21.15.33:0/2181671687 wait complete. 2024-05-08T21:26:42.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.909+0000 7f70e830fb80 20 client.17574 trim_cache size 0 max 16384 2024-05-08T21:26:42.922 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:26:42.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.909+0000 7f70dc155700 1 -- 172.21.15.33:0/3743190237 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f70c00a1220 msgr2=0x7f70c00a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.909+0000 7f70dc155700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f70c00a1220 0x7f70c00a36e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f70cc029000 tx=0x7f70cc02a000 comp rx=0 tx=0).stop 2024-05-08T21:26:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.909+0000 7f70dc155700 1 -- 172.21.15.33:0/3743190237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d40ffd60 msgr2=0x7f70d4196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.909+0000 7f70dc155700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d40ffd60 0x7f70d4196e10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f70d000e970 tx=0x7f70d000ec80 comp rx=0 tx=0).stop 2024-05-08T21:26:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.910+0000 7f70dc155700 1 -- 172.21.15.33:0/3743190237 shutdown_connections 2024-05-08T21:26:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.910+0000 7f70dc155700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70d41018f0 0x7f70d419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.910+0000 7f70dc155700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d40ffd60 0x7f70d4196e10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.910+0000 7f70dc155700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f70c00a1220 0x7f70c00a36e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.910+0000 7f70dc155700 1 --2- 172.21.15.33:0/3743190237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d4100f50 0x7f70d4197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.910+0000 7f70dc155700 1 -- 172.21.15.33:0/3743190237 >> 172.21.15.33:0/3743190237 conn(0x7f70d40fb290 msgr2=0x7f70d4068160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.910+0000 7f70dc155700 1 -- 172.21.15.33:0/3743190237 shutdown_connections 2024-05-08T21:26:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:42.910+0000 7f70dc155700 1 -- 172.21.15.33:0/3743190237 wait complete. 2024-05-08T21:26:42.936 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:26:43.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.121+0000 7fa2634f2700 1 -- 172.21.15.33:0/589018683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c0ffd60 msgr2=0x7fa25c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.121+0000 7fa2634f2700 1 --2- 172.21.15.33:0/589018683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c0ffd60 0x7fa25c100180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa258009920 tx=0x7fa258009c30 comp rx=0 tx=0).stop 2024-05-08T21:26:43.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.121+0000 7fa2634f2700 1 -- 172.21.15.33:0/589018683 shutdown_connections 2024-05-08T21:26:43.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.121+0000 7fa2634f2700 1 --2- 172.21.15.33:0/589018683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa25c1018f0 0x7fa25c1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.121+0000 7fa2634f2700 1 --2- 172.21.15.33:0/589018683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c0ffd60 0x7fa25c100180 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.121+0000 7fa2634f2700 1 --2- 172.21.15.33:0/589018683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25c100f50 0x7fa25c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.121+0000 7fa2634f2700 1 -- 172.21.15.33:0/589018683 >> 172.21.15.33:0/589018683 conn(0x7fa25c0fb290 msgr2=0x7fa25c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:43.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.121+0000 7fa2634f2700 1 -- 172.21.15.33:0/589018683 shutdown_connections 2024-05-08T21:26:43.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.121+0000 7fa2634f2700 1 -- 172.21.15.33:0/589018683 wait complete. 2024-05-08T21:26:43.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.122+0000 7fa2634f2700 1 Processor -- start 2024-05-08T21:26:43.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.122+0000 7fa2634f2700 1 -- start start 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa2634f2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c0ffd60 0x7fa25c199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa2634f2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa25c100f50 0x7fa25c199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa2634f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25c1018f0 0x7fa25c1a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa2634f2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c1a0b30 con 0x7fa25c1018f0 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa2634f2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c1a0c70 con 0x7fa25c0ffd60 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa2634f2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c1a0e00 con 0x7fa25c100f50 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa26128e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c0ffd60 0x7fa25c199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa261a8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25c1018f0 0x7fa25c1a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa26128e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c0ffd60 0x7fa25c199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56304/0 (socket says 172.21.15.33:56304) 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa26128e700 1 -- 172.21.15.33:0/2654071813 learned_addr learned my addr 172.21.15.33:0/2654071813 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa260a8d700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa25c100f50 0x7fa25c199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa261a8f700 1 -- 172.21.15.33:0/2654071813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa25c100f50 msgr2=0x7fa25c199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa261a8f700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa25c100f50 0x7fa25c199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa261a8f700 1 -- 172.21.15.33:0/2654071813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c0ffd60 msgr2=0x7fa25c199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa261a8f700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c0ffd60 0x7fa25c199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa261a8f700 1 -- 172.21.15.33:0/2654071813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c0095f0 con 0x7fa25c1018f0 2024-05-08T21:26:43.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa26128e700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c0ffd60 0x7fa25c199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:43.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.123+0000 7fa261a8f700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25c1018f0 0x7fa25c1a0560 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa25400e340 tx=0x7fa25400e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:43.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.124+0000 7fa2527fc700 1 -- 172.21.15.33:0/2654071813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254010040 con 0x7fa25c1018f0 2024-05-08T21:26:43.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.124+0000 7fa2634f2700 1 -- 172.21.15.33:0/2654071813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa258009520 con 0x7fa25c1018f0 2024-05-08T21:26:43.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.124+0000 7fa2527fc700 1 -- 172.21.15.33:0/2654071813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa254003e90 con 0x7fa25c1018f0 2024-05-08T21:26:43.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.124+0000 7fa2527fc700 1 -- 172.21.15.33:0/2654071813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa254009970 con 0x7fa25c1018f0 2024-05-08T21:26:43.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.124+0000 7fa2634f2700 1 -- 172.21.15.33:0/2654071813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c1a1410 con 0x7fa25c1018f0 2024-05-08T21:26:43.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.127+0000 7fa2527fc700 1 -- 172.21.15.33:0/2654071813 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa254003a00 con 0x7fa25c1018f0 2024-05-08T21:26:43.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.128+0000 7fa2527fc700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa2480a1120 0x7fa2480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.128+0000 7fa2527fc700 1 -- 172.21.15.33:0/2654071813 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa2540cc9c0 con 0x7fa25c1018f0 2024-05-08T21:26:43.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.128+0000 7fa26128e700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa2480a1120 0x7fa2480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa26128e700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa2480a1120 0x7fa2480a35e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa258011040 tx=0x7fa258019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:43.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa26f6acb80 1 Processor -- start 2024-05-08T21:26:43.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa26f6acb80 1 -- start start 2024-05-08T21:26:43.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa26f6acb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809540 0x55d0bd809960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa26f6acb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd80a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa26f6acb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd80a860 0x55d0bd80ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa26f6acb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bd760150 con 0x55d0bd80a860 2024-05-08T21:26:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa26f6acb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bd75db10 con 0x55d0bd809ea0 2024-05-08T21:26:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa26f6acb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bd759400 con 0x55d0bd809540 2024-05-08T21:26:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa26128e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809540 0x55d0bd809960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa26128e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809540 0x55d0bd809960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51100/0 (socket says 172.21.15.33:51100) 2024-05-08T21:26:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa260a8d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd80a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa26128e700 1 -- 172.21.15.33:0/2894511339 learned_addr learned my addr 172.21.15.33:0/2894511339 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.129+0000 7fa261a8f700 1 --2- 172.21.15.33:0/2894511339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd80a860 0x55d0bd80ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.130+0000 7fa26128e700 1 -- 172.21.15.33:0/2894511339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 msgr2=0x55d0bd80a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:43.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.130+0000 7fa26128e700 1 --2- 172.21.15.33:0/2894511339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd80a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.130+0000 7fa26128e700 1 -- 172.21.15.33:0/2894511339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd80a860 msgr2=0x55d0bd80ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.130+0000 7fa26128e700 1 --2- 172.21.15.33:0/2894511339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd80a860 0x55d0bd80ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.130+0000 7fa26128e700 1 -- 172.21.15.33:0/2894511339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0bd5da3b0 con 0x55d0bd809540 2024-05-08T21:26:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.130+0000 7fa260a8d700 1 --2- 172.21.15.33:0/2894511339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd80a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.130+0000 7fa26128e700 1 --2- 172.21.15.33:0/2894511339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809540 0x55d0bd809960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa25800f6a0 tx=0x7fa25801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c0228d0f38486dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.130+0000 7fa2477fe700 1 -- 172.21.15.33:0/2894511339 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25800b9f0 con 0x55d0bd809540 2024-05-08T21:26:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.130+0000 7fa2477fe700 1 -- 172.21.15.33:0/2894511339 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa25801d040 con 0x55d0bd809540 2024-05-08T21:26:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.130+0000 7fa2477fe700 1 -- 172.21.15.33:0/2894511339 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa258003a60 con 0x55d0bd809540 2024-05-08T21:26:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.131+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2894511339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809540 msgr2=0x55d0bd809960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.131+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/2894511339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809540 0x55d0bd809960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa25800f6a0 tx=0x7fa25801f000 comp rx=0 tx=0).stop 2024-05-08T21:26:43.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.131+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2894511339 shutdown_connections 2024-05-08T21:26:43.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.131+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/2894511339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd80a860 0x55d0bd80ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.131+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/2894511339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809540 0x55d0bd809960 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.131+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/2894511339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd80a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.131+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2894511339 >> 172.21.15.33:0/2894511339 conn(0x55d0bd767210 msgr2=0x55d0bd803de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:43.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.132+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2894511339 shutdown_connections 2024-05-08T21:26:43.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.132+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2894511339 wait complete. 2024-05-08T21:26:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.132+0000 7fa26f6acb80 1 -- start start 2024-05-08T21:26:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.132+0000 7fa26f6acb80 1 Processor -- start 2024-05-08T21:26:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.134+0000 7fa26f6acb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809540 0x55d0bd812e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.134+0000 7fa26f6acb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd813340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.134+0000 7fa26f6acb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd80a860 0x55d0bd81a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.134+0000 7fa26f6acb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bd6f5560 con 0x55d0bd80a860 2024-05-08T21:26:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.134+0000 7fa26f6acb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bd6f5200 con 0x55d0bd809ea0 2024-05-08T21:26:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.134+0000 7fa26f6acb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bd5dc6a0 con 0x55d0bd809540 2024-05-08T21:26:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.135+0000 7fa26f6acb80 10 client.0 _ll_register_callbacks cb 0x55d0bd6feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.135+0000 7fa260a8d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd813340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.135+0000 7fa260a8d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd813340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56314/0 (socket says 172.21.15.33:56314) 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.135+0000 7fa260a8d700 1 -- 172.21.15.33:0/2987281613 learned_addr learned my addr 172.21.15.33:0/2987281613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.135+0000 7fa261a8f700 1 --2- 172.21.15.33:0/2987281613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd80a860 0x55d0bd81a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.135+0000 7fa260a8d700 1 -- 172.21.15.33:0/2987281613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809540 msgr2=0x55d0bd812e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.135+0000 7fa260a8d700 1 --2- 172.21.15.33:0/2987281613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809540 0x55d0bd812e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.135+0000 7fa260a8d700 1 -- 172.21.15.33:0/2987281613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd80a860 msgr2=0x55d0bd81a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.135+0000 7fa260a8d700 1 --2- 172.21.15.33:0/2987281613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd80a860 0x55d0bd81a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.135+0000 7fa260a8d700 1 -- 172.21.15.33:0/2987281613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24c009ad0 con 0x55d0bd809ea0 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.136+0000 7fa260a8d700 1 --2- 172.21.15.33:0/2987281613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd813340 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa24c012cf0 tx=0x7fa24c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=194be86f27e29a55 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.136+0000 7fa244ff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.136+0000 7fa244ff9700 1 -- 172.21.15.33:0/2987281613 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c017070 con 0x55d0bd809ea0 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.136+0000 7fa244ff9700 1 -- 172.21.15.33:0/2987281613 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa24c011cb0 con 0x55d0bd809ea0 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.136+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2987281613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0bd6b2270 con 0x55d0bd809ea0 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.136+0000 7fa244ff9700 1 -- 172.21.15.33:0/2987281613 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa24c011e70 con 0x55d0bd809ea0 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.136+0000 7fa26f6acb80 10 client.27482 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:43.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.136+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2987281613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fa25c108af0 con 0x55d0bd809ea0 2024-05-08T21:26:43.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.137+0000 7fa244ff9700 1 -- 172.21.15.33:0/2987281613 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d0bd6b2270 con 0x55d0bd809ea0 2024-05-08T21:26:43.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.137+0000 7fa244ff9700 1 -- 172.21.15.33:0/2987281613 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa24c005440 con 0x55d0bd809ea0 2024-05-08T21:26:43.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.137+0000 7fa26f6acb80 10 client.27482 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:43.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.137+0000 7fa26f6acb80 10 client.27482 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:26:43.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.137+0000 7fa26f6acb80 20 client.27482 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:43.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.137+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/2987281613 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d0bd827470 0x55d0bd829890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.137+0000 7fa26f6acb80 4 client.27482 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:43.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.137+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2987281613 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0bd7d8810 con 0x55d0bd827470 2024-05-08T21:26:43.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.137+0000 7fa26128e700 1 --2- 172.21.15.33:0/2987281613 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d0bd827470 0x55d0bd829890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.138+0000 7fa244ff9700 10 client.27482 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:26:43.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.138+0000 7fa26128e700 1 --2- 172.21.15.33:0/2987281613 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d0bd827470 0x55d0bd829890 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae26c66d2ec1e255 server_cookie=4262543a01875175 in_seq=0 out_seq=0 2024-05-08T21:26:43.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.138+0000 7fa244ff9700 1 -- 172.21.15.33:0/2987281613 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0bd7d8810 con 0x55d0bd827470 2024-05-08T21:26:43.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.138+0000 7fa244ff9700 10 client.27482 handle_command_reply: tid=0 2024-05-08T21:26:43.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa26f6acb80 1 client.27482 shutdown 2024-05-08T21:26:43.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2987281613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 msgr2=0x55d0bd813340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/2987281613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd813340 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa24c012cf0 tx=0x7fa24c011720 comp rx=0 tx=0).stop 2024-05-08T21:26:43.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2987281613 shutdown_connections 2024-05-08T21:26:43.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/2987281613 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d0bd827470 0x55d0bd829890 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/2987281613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd80a860 0x55d0bd81a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/2987281613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809540 0x55d0bd812e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa244ff9700 0 client.27482 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:26:43.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/2987281613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd813340 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2987281613 >> 172.21.15.33:0/2987281613 conn(0x55d0bd767210 msgr2=0x55d0bd8035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:43.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2987281613 shutdown_connections 2024-05-08T21:26:43.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa26f6acb80 1 -- 172.21.15.33:0/2987281613 wait complete. 2024-05-08T21:26:43.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.139+0000 7fa26f6acb80 20 client.27482 trim_cache size 0 max 16384 2024-05-08T21:26:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.161+0000 7fa26f6acb80 1 Processor -- start 2024-05-08T21:26:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.161+0000 7fa26f6acb80 1 -- start start 2024-05-08T21:26:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.162+0000 7fa26f6acb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 0x55d0bd809920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.162+0000 7fa26f6acb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd823240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.162+0000 7fa26f6acb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd80a860 0x55d0bd81a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.162+0000 7fa26f6acb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bd81a7e0 con 0x55d0bd809540 2024-05-08T21:26:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.162+0000 7fa26f6acb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bd82eaa0 con 0x55d0bd809ea0 2024-05-08T21:26:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.162+0000 7fa26f6acb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bd813a80 con 0x55d0bd80a860 2024-05-08T21:26:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.162+0000 7fa26128e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 0x55d0bd809920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.162+0000 7fa26128e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 0x55d0bd809920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57894/0 (socket says 172.21.15.33:57894) 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.162+0000 7fa26128e700 1 -- 172.21.15.33:0/358528670 learned_addr learned my addr 172.21.15.33:0/358528670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.163+0000 7fa26128e700 1 -- 172.21.15.33:0/358528670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd80a860 msgr2=0x55d0bd81a2a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.163+0000 7fa26128e700 1 --2- 172.21.15.33:0/358528670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd80a860 0x55d0bd81a2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.163+0000 7fa26128e700 1 -- 172.21.15.33:0/358528670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 msgr2=0x55d0bd823240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.163+0000 7fa26128e700 1 --2- 172.21.15.33:0/358528670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd823240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.163+0000 7fa26128e700 1 -- 172.21.15.33:0/358528670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0bd586b60 con 0x55d0bd809540 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.163+0000 7fa26128e700 1 --2- 172.21.15.33:0/358528670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 0x55d0bd809920 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa258005850 tx=0x7fa258009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0f112a9224b856 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.163+0000 7fa247fff700 1 -- 172.21.15.33:0/358528670 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa258028070 con 0x55d0bd809540 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.163+0000 7fa247fff700 1 -- 172.21.15.33:0/358528670 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa258003b10 con 0x55d0bd809540 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.163+0000 7fa247fff700 1 -- 172.21.15.33:0/358528670 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2580055e0 con 0x55d0bd809540 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.164+0000 7fa26f6acb80 1 -- 172.21.15.33:0/358528670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 msgr2=0x55d0bd809920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.164+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/358528670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 0x55d0bd809920 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa258005850 tx=0x7fa258009c30 comp rx=0 tx=0).stop 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.165+0000 7fa26f6acb80 1 -- 172.21.15.33:0/358528670 shutdown_connections 2024-05-08T21:26:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.165+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/358528670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd80a860 0x55d0bd81a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.165+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/358528670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 0x55d0bd809920 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.165+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/358528670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd809ea0 0x55d0bd823240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.165+0000 7fa26f6acb80 1 -- 172.21.15.33:0/358528670 >> 172.21.15.33:0/358528670 conn(0x55d0bd767210 msgr2=0x55d0bd80ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.165+0000 7fa26f6acb80 1 -- 172.21.15.33:0/358528670 shutdown_connections 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.165+0000 7fa26f6acb80 1 -- 172.21.15.33:0/358528670 wait complete. 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.165+0000 7fa26f6acb80 1 -- start start 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.166+0000 7fa26f6acb80 1 Processor -- start 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.166+0000 7fa26f6acb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 0x55d0bd816280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.166+0000 7fa26f6acb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809ea0 0x55d0bd8187d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.166+0000 7fa26f6acb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd80a860 0x55d0bd818d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.166+0000 7fa26f6acb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bd819250 con 0x55d0bd809540 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.166+0000 7fa26f6acb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bd8193c0 con 0x55d0bd80a860 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.166+0000 7fa26f6acb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bd819580 con 0x55d0bd809ea0 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.166+0000 7fa26f6acb80 10 client.0 _ll_register_callbacks cb 0x55d0bd6feb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.167+0000 7fa26128e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 0x55d0bd816280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.167+0000 7fa26128e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 0x55d0bd816280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57900/0 (socket says 172.21.15.33:57900) 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.167+0000 7fa26128e700 1 -- 172.21.15.33:0/1884259949 learned_addr learned my addr 172.21.15.33:0/1884259949 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.167+0000 7fa260a8d700 1 --2- 172.21.15.33:0/1884259949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809ea0 0x55d0bd8187d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.167+0000 7fa26128e700 1 -- 172.21.15.33:0/1884259949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809ea0 msgr2=0x55d0bd8187d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.167+0000 7fa26128e700 1 --2- 172.21.15.33:0/1884259949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809ea0 0x55d0bd8187d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.167+0000 7fa26128e700 1 -- 172.21.15.33:0/1884259949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd80a860 msgr2=0x55d0bd818d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.167+0000 7fa26128e700 1 --2- 172.21.15.33:0/1884259949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd80a860 0x55d0bd818d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.167+0000 7fa26128e700 1 -- 172.21.15.33:0/1884259949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0bd5da3b0 con 0x55d0bd809540 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.168+0000 7fa26128e700 1 --2- 172.21.15.33:0/1884259949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 0x55d0bd816280 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa25800a010 tx=0x7fa258003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84936579bf8ad527 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.168+0000 7fa244ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.168+0000 7fa244ff9700 1 -- 172.21.15.33:0/1884259949 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa258028040 con 0x55d0bd809540 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.168+0000 7fa244ff9700 1 -- 172.21.15.33:0/1884259949 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa258003880 con 0x55d0bd809540 2024-05-08T21:26:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.168+0000 7fa26f6acb80 1 -- 172.21.15.33:0/1884259949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa25801d040 con 0x55d0bd809540 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.169+0000 7fa244ff9700 1 -- 172.21.15.33:0/1884259949 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa25800bb80 con 0x55d0bd809540 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.169+0000 7fa26f6acb80 10 client.17592 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.169+0000 7fa26f6acb80 1 -- 172.21.15.33:0/1884259949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d0bd71f560 con 0x55d0bd809540 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.169+0000 7fa244ff9700 1 -- 172.21.15.33:0/1884259949 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa25801d040 con 0x55d0bd809540 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.169+0000 7fa244ff9700 1 -- 172.21.15.33:0/1884259949 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa258042070 con 0x55d0bd809540 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.169+0000 7fa26f6acb80 10 client.17592 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.169+0000 7fa26f6acb80 10 client.17592 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.169+0000 7fa26f6acb80 20 client.17592 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.169+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/1884259949 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d0bd81a980 0x55d0bd81cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.169+0000 7fa26f6acb80 4 client.17592 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.169+0000 7fa26f6acb80 1 -- 172.21.15.33:0/1884259949 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d0bd751ec0 con 0x55d0bd81a980 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.170+0000 7fa260a8d700 1 --2- 172.21.15.33:0/1884259949 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d0bd81a980 0x55d0bd81cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.170+0000 7fa244ff9700 10 client.17592 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.170+0000 7fa260a8d700 1 --2- 172.21.15.33:0/1884259949 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d0bd81a980 0x55d0bd81cda0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df66ed1d3f1a18ac server_cookie=32ffb00714fd2ea2 in_seq=0 out_seq=0 2024-05-08T21:26:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.170+0000 7fa244ff9700 1 -- 172.21.15.33:0/1884259949 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d0bd751ec0 con 0x55d0bd81a980 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.170+0000 7fa244ff9700 10 client.17592 handle_command_reply: tid=0 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.170+0000 7fa26f6acb80 1 client.17592 shutdown 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.171+0000 7fa26f6acb80 1 -- 172.21.15.33:0/1884259949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 msgr2=0x55d0bd816280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.171+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/1884259949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 0x55d0bd816280 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa25800a010 tx=0x7fa258003c80 comp rx=0 tx=0).stop 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.171+0000 7fa26f6acb80 1 -- 172.21.15.33:0/1884259949 shutdown_connections 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.171+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/1884259949 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55d0bd81a980 0x55d0bd81cda0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.171+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/1884259949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0bd80a860 0x55d0bd818d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.171+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/1884259949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0bd809540 0x55d0bd816280 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.171+0000 7fa26f6acb80 1 --2- 172.21.15.33:0/1884259949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bd809ea0 0x55d0bd8187d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.171+0000 7fa26f6acb80 1 -- 172.21.15.33:0/1884259949 >> 172.21.15.33:0/1884259949 conn(0x55d0bd767210 msgr2=0x55d0bd8122e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.171+0000 7fa244ff9700 0 client.17592 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.171+0000 7fa26f6acb80 1 -- 172.21.15.33:0/1884259949 shutdown_connections 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.171+0000 7fa26f6acb80 1 -- 172.21.15.33:0/1884259949 wait complete. 2024-05-08T21:26:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.171+0000 7fa26f6acb80 20 client.17592 trim_cache size 0 max 16384 2024-05-08T21:26:43.181 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:26:43.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.172+0000 7fa2634f2700 1 -- 172.21.15.33:0/2654071813 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa2480a1120 msgr2=0x7fa2480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.172+0000 7fa2634f2700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa2480a1120 0x7fa2480a35e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa258011040 tx=0x7fa258019040 comp rx=0 tx=0).stop 2024-05-08T21:26:43.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.172+0000 7fa2634f2700 1 -- 172.21.15.33:0/2654071813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25c1018f0 msgr2=0x7fa25c1a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.172+0000 7fa2634f2700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25c1018f0 0x7fa25c1a0560 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa25400e340 tx=0x7fa25400e650 comp rx=0 tx=0).stop 2024-05-08T21:26:43.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.172+0000 7fa2634f2700 1 -- 172.21.15.33:0/2654071813 shutdown_connections 2024-05-08T21:26:43.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.172+0000 7fa2634f2700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa25c1018f0 0x7fa25c1a0560 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.172+0000 7fa2634f2700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa2480a1120 0x7fa2480a35e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.172+0000 7fa2634f2700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa25c0ffd60 0x7fa25c199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.172+0000 7fa2634f2700 1 --2- 172.21.15.33:0/2654071813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa25c100f50 0x7fa25c199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.172+0000 7fa2634f2700 1 -- 172.21.15.33:0/2654071813 >> 172.21.15.33:0/2654071813 conn(0x7fa25c0fb290 msgr2=0x7fa25c104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:43.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.172+0000 7fa2634f2700 1 -- 172.21.15.33:0/2654071813 shutdown_connections 2024-05-08T21:26:43.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.172+0000 7fa2634f2700 1 -- 172.21.15.33:0/2654071813 wait complete. 2024-05-08T21:26:43.190 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:26:43.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.374+0000 7f5c74fb4700 1 -- 172.21.15.33:0/2286339856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c70100f50 msgr2=0x7f5c701013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.374+0000 7f5c74fb4700 1 --2- 172.21.15.33:0/2286339856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c70100f50 0x7f5c701013b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009990 tx=0x7f5c64009ca0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.375+0000 7f5c74fb4700 1 -- 172.21.15.33:0/2286339856 shutdown_connections 2024-05-08T21:26:43.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.375+0000 7f5c74fb4700 1 --2- 172.21.15.33:0/2286339856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c701018f0 0x7f5c701081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.375+0000 7f5c74fb4700 1 --2- 172.21.15.33:0/2286339856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c700ffd60 0x7f5c70100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.375+0000 7f5c74fb4700 1 --2- 172.21.15.33:0/2286339856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c70100f50 0x7f5c701013b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.375+0000 7f5c74fb4700 1 -- 172.21.15.33:0/2286339856 >> 172.21.15.33:0/2286339856 conn(0x7f5c700fb290 msgr2=0x7f5c700fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:43.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.375+0000 7f5c74fb4700 1 -- 172.21.15.33:0/2286339856 shutdown_connections 2024-05-08T21:26:43.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.375+0000 7f5c74fb4700 1 -- 172.21.15.33:0/2286339856 wait complete. 2024-05-08T21:26:43.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c74fb4700 1 Processor -- start 2024-05-08T21:26:43.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c74fb4700 1 -- start start 2024-05-08T21:26:43.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c74fb4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c700ffd60 0x7f5c70068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c74fb4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c70100f50 0x7f5c7019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c74fb4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c701018f0 0x7f5c7019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c74fb4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7019e8b0 con 0x7f5c700ffd60 2024-05-08T21:26:43.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c74fb4700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7019ea20 con 0x7f5c701018f0 2024-05-08T21:26:43.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c74fb4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7019ebe0 con 0x7f5c70100f50 2024-05-08T21:26:43.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c6dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c70100f50 0x7f5c7019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c6dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c70100f50 0x7f5c7019baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51150/0 (socket says 172.21.15.33:51150) 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c6dd9b700 1 -- 172.21.15.33:0/4212828981 learned_addr learned my addr 172.21.15.33:0/4212828981 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c6ed9d700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c701018f0 0x7f5c7019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.376+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c700ffd60 0x7f5c70068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.377+0000 7f5c6dd9b700 1 -- 172.21.15.33:0/4212828981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c701018f0 msgr2=0x7f5c7019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.377+0000 7f5c6dd9b700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c701018f0 0x7f5c7019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.377+0000 7f5c6dd9b700 1 -- 172.21.15.33:0/4212828981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c700ffd60 msgr2=0x7f5c70068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.377+0000 7f5c6dd9b700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c700ffd60 0x7f5c70068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.377+0000 7f5c6dd9b700 1 -- 172.21.15.33:0/4212828981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c640095f0 con 0x7f5c70100f50 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.377+0000 7f5c6ed9d700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c701018f0 0x7f5c7019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.377+0000 7f5c6dd9b700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c70100f50 0x7f5c7019baa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5c64005910 tx=0x7f5c6400b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.377+0000 7f5c5f7fe700 1 -- 172.21.15.33:0/4212828981 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6401d070 con 0x7f5c70100f50 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.377+0000 7f5c5f7fe700 1 -- 172.21.15.33:0/4212828981 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c6400bed0 con 0x7f5c70100f50 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.377+0000 7f5c74fb4700 1 -- 172.21.15.33:0/4212828981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7019ee00 con 0x7f5c70100f50 2024-05-08T21:26:43.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.377+0000 7f5c5f7fe700 1 -- 172.21.15.33:0/4212828981 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6400fdc0 con 0x7f5c70100f50 2024-05-08T21:26:43.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.377+0000 7f5c74fb4700 1 -- 172.21.15.33:0/4212828981 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7019f220 con 0x7f5c70100f50 2024-05-08T21:26:43.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.380+0000 7f5c5f7fe700 1 -- 172.21.15.33:0/4212828981 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5c6400f560 con 0x7f5c70100f50 2024-05-08T21:26:43.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.381+0000 7f5c5f7fe700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5c580a11a0 0x7f5c580a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.382+0000 7f5c5f7fe700 1 -- 172.21.15.33:0/4212828981 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f5c640cde70 con 0x7f5c70100f50 2024-05-08T21:26:43.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.382+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5c580a11a0 0x7f5c580a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.382+0000 7f5c8116eb80 1 Processor -- start 2024-05-08T21:26:43.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.382+0000 7f5c8116eb80 1 -- start start 2024-05-08T21:26:43.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.382+0000 7f5c8116eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa69540 0x56449aa69960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.382+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5c580a11a0 0x7f5c580a3660 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5c70100db0 tx=0x7f5c60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:43.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.382+0000 7f5c8116eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69ea0 0x56449aa6a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.382+0000 7f5c8116eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa6a860 0x56449aa6ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.382+0000 7f5c8116eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56449a9c0150 con 0x56449aa6a860 2024-05-08T21:26:43.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.382+0000 7f5c8116eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56449a9bdb10 con 0x56449aa69540 2024-05-08T21:26:43.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.382+0000 7f5c8116eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56449a9b9400 con 0x56449aa69ea0 2024-05-08T21:26:43.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.383+0000 7f5c6ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa6a860 0x56449aa6ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.383+0000 7f5c6ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa6a860 0x56449aa6ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57938/0 (socket says 172.21.15.33:57938) 2024-05-08T21:26:43.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.383+0000 7f5c6ed9d700 1 -- 172.21.15.33:0/2284114415 learned_addr learned my addr 172.21.15.33:0/2284114415 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.383+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/2284114415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa69540 0x56449aa69960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.383+0000 7f5c6dd9b700 1 --2- 172.21.15.33:0/2284114415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69ea0 0x56449aa6a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.383+0000 7f5c6e59c700 1 -- 172.21.15.33:0/2284114415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69ea0 msgr2=0x56449aa6a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.383+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/2284114415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69ea0 0x56449aa6a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.383+0000 7f5c6e59c700 1 -- 172.21.15.33:0/2284114415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa6a860 msgr2=0x56449aa6ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.383+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/2284114415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa6a860 0x56449aa6ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.383+0000 7f5c6e59c700 1 -- 172.21.15.33:0/2284114415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56449a83a3b0 con 0x56449aa69540 2024-05-08T21:26:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.383+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/2284114415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa69540 0x56449aa69960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5c60011cb0 tx=0x7f5c60014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68ce4b98dc529754 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.384+0000 7f5c5cff9700 1 -- 172.21.15.33:0/2284114415 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60004220 con 0x56449aa69540 2024-05-08T21:26:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.384+0000 7f5c5cff9700 1 -- 172.21.15.33:0/2284114415 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c60021070 con 0x56449aa69540 2024-05-08T21:26:43.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.384+0000 7f5c5cff9700 1 -- 172.21.15.33:0/2284114415 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60003a30 con 0x56449aa69540 2024-05-08T21:26:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.385+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2284114415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa69540 msgr2=0x56449aa69960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.385+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/2284114415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa69540 0x56449aa69960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5c60011cb0 tx=0x7f5c60014000 comp rx=0 tx=0).stop 2024-05-08T21:26:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.385+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2284114415 shutdown_connections 2024-05-08T21:26:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.385+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/2284114415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa6a860 0x56449aa6ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.385+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/2284114415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa69540 0x56449aa69960 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.385+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/2284114415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69ea0 0x56449aa6a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.385+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2284114415 >> 172.21.15.33:0/2284114415 conn(0x56449a9c7210 msgr2=0x56449aa63de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.385+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2284114415 shutdown_connections 2024-05-08T21:26:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.385+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2284114415 wait complete. 2024-05-08T21:26:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.385+0000 7f5c8116eb80 1 -- start start 2024-05-08T21:26:43.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.386+0000 7f5c8116eb80 1 Processor -- start 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.386+0000 7f5c8116eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 0x56449aa7b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.386+0000 7f5c8116eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69ea0 0x56449aa7bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.386+0000 7f5c8116eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa82bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.386+0000 7f5c8116eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56449a955560 con 0x56449aa69ea0 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.386+0000 7f5c8116eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56449a955200 con 0x56449aa6a860 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.386+0000 7f5c8116eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56449a83c6a0 con 0x56449aa69540 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.386+0000 7f5c8116eb80 10 client.0 _ll_register_callbacks cb 0x56449a95eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.386+0000 7f5c6e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 0x56449aa7b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.386+0000 7f5c6e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 0x56449aa7b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51178/0 (socket says 172.21.15.33:51178) 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.386+0000 7f5c6e59c700 1 -- 172.21.15.33:0/2453102803 learned_addr learned my addr 172.21.15.33:0/2453102803 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.387+0000 7f5c6ed9d700 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa82bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.387+0000 7f5c6e59c700 1 -- 172.21.15.33:0/2453102803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 msgr2=0x56449aa82bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.387+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa82bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.387+0000 7f5c6e59c700 1 -- 172.21.15.33:0/2453102803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69ea0 msgr2=0x56449aa7bb90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:26:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.387+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69ea0 0x56449aa7bb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.387+0000 7f5c6e59c700 1 -- 172.21.15.33:0/2453102803 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c60017070 con 0x56449aa69540 2024-05-08T21:26:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.387+0000 7f5c6ed9d700 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa82bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:26:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.387+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 0x56449aa7b650 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5c600045b0 tx=0x7f5c6000eb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e21cc6e0162c8f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:43.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.387+0000 7f5c55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:26:43.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.393+0000 7f5c55ffb700 1 -- 172.21.15.33:0/2453102803 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60005460 con 0x56449aa69540 2024-05-08T21:26:43.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.393+0000 7f5c55ffb700 1 -- 172.21.15.33:0/2453102803 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f5c600055c0 con 0x56449aa69540 2024-05-08T21:26:43.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.393+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2453102803 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56449a912270 con 0x56449aa69540 2024-05-08T21:26:43.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.393+0000 7f5c55ffb700 1 -- 172.21.15.33:0/2453102803 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c60004750 con 0x56449aa69540 2024-05-08T21:26:43.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.393+0000 7f5c8116eb80 10 client.27502 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:43.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.393+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2453102803 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56449a7e6b60 con 0x56449aa69540 2024-05-08T21:26:43.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.393+0000 7f5c55ffb700 1 -- 172.21.15.33:0/2453102803 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56449a912270 con 0x56449aa69540 2024-05-08T21:26:43.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.393+0000 7f5c55ffb700 1 -- 172.21.15.33:0/2453102803 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c60026940 con 0x56449aa69540 2024-05-08T21:26:43.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.393+0000 7f5c8116eb80 10 client.27502 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:43.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.393+0000 7f5c8116eb80 10 client.27502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:43.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.393+0000 7f5c8116eb80 20 client.27502 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:43.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.394+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56449aa98360 0x56449aa9a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.394+0000 7f5c8116eb80 4 client.27502 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:43.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.394+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2453102803 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56449aa38810 con 0x56449aa98360 2024-05-08T21:26:43.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.394+0000 7f5c6dd9b700 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56449aa98360 0x56449aa9a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.394+0000 7f5c55ffb700 10 client.27502 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:43.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.394+0000 7f5c6dd9b700 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56449aa98360 0x56449aa9a780 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c77e2daf3bdf2bf server_cookie=84631ff37e032a3a in_seq=0 out_seq=0 2024-05-08T21:26:43.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c55ffb700 1 -- 172.21.15.33:0/2453102803 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56449aa38810 con 0x56449aa98360 2024-05-08T21:26:43.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c55ffb700 10 client.27502 handle_command_reply: tid=0 2024-05-08T21:26:43.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c8116eb80 1 client.27502 shutdown 2024-05-08T21:26:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2453102803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 msgr2=0x56449aa7b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 0x56449aa7b650 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5c600045b0 tx=0x7f5c6000eb40 comp rx=0 tx=0).stop 2024-05-08T21:26:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2453102803 shutdown_connections 2024-05-08T21:26:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56449aa98360 0x56449aa9a780 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa82bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 0x56449aa7b650 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/2453102803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69ea0 0x56449aa7bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c55ffb700 0 client.27502 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2453102803 >> 172.21.15.33:0/2453102803 conn(0x56449a9c7210 msgr2=0x56449aa635a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2453102803 shutdown_connections 2024-05-08T21:26:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c8116eb80 1 -- 172.21.15.33:0/2453102803 wait complete. 2024-05-08T21:26:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.395+0000 7f5c8116eb80 20 client.27502 trim_cache size 0 max 16384 2024-05-08T21:26:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c8116eb80 1 Processor -- start 2024-05-08T21:26:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c8116eb80 1 -- start start 2024-05-08T21:26:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c8116eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 0x56449aa9a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c8116eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69ea0 0x56449aa82b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c8116eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa7ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c8116eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56449aa655e0 con 0x56449aa69ea0 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c8116eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56449aa9f9a0 con 0x56449aa6a860 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c8116eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56449aa70a60 con 0x56449aa69540 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c6dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69ea0 0x56449aa82b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c6ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa7ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c6e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 0x56449aa9a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c6ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa7ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56382/0 (socket says 172.21.15.33:56382) 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c6e59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 0x56449aa9a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51182/0 (socket says 172.21.15.33:51182) 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c6ed9d700 1 -- 172.21.15.33:0/3115677125 learned_addr learned my addr 172.21.15.33:0/3115677125 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c6dd9b700 1 -- 172.21.15.33:0/3115677125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 msgr2=0x56449aa9a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c6dd9b700 1 --2- 172.21.15.33:0/3115677125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 0x56449aa9a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c6dd9b700 1 -- 172.21.15.33:0/3115677125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 msgr2=0x56449aa7ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c6dd9b700 1 --2- 172.21.15.33:0/3115677125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa7ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.420+0000 7f5c6dd9b700 1 -- 172.21.15.33:0/3115677125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56449a97f560 con 0x56449aa69ea0 2024-05-08T21:26:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.421+0000 7f5c6dd9b700 1 --2- 172.21.15.33:0/3115677125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69ea0 0x56449aa82b90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5c640cb5f0 tx=0x7f5c64009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab80b0906d97ab8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.421+0000 7f5c5d7fa700 1 -- 172.21.15.33:0/3115677125 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640a10b0 con 0x56449aa69ea0 2024-05-08T21:26:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.421+0000 7f5c5d7fa700 1 -- 172.21.15.33:0/3115677125 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c6409afd0 con 0x56449aa69ea0 2024-05-08T21:26:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.421+0000 7f5c5d7fa700 1 -- 172.21.15.33:0/3115677125 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64095e40 con 0x56449aa69ea0 2024-05-08T21:26:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.421+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3115677125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69ea0 msgr2=0x56449aa82b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.421+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/3115677125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69ea0 0x56449aa82b90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5c640cb5f0 tx=0x7f5c64009310 comp rx=0 tx=0).stop 2024-05-08T21:26:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3115677125 shutdown_connections 2024-05-08T21:26:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/3115677125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa7ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/3115677125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69540 0x56449aa9a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/3115677125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69ea0 0x56449aa82b90 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3115677125 >> 172.21.15.33:0/3115677125 conn(0x56449a9c7210 msgr2=0x56449aa6ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3115677125 shutdown_connections 2024-05-08T21:26:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3115677125 wait complete. 2024-05-08T21:26:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 -- start start 2024-05-08T21:26:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 Processor -- start 2024-05-08T21:26:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69540 0x56449aa7de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69ea0 0x56449aa7e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa813f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56449aa81930 con 0x56449aa69540 2024-05-08T21:26:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56449aa81aa0 con 0x56449aa6a860 2024-05-08T21:26:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.422+0000 7f5c8116eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56449aa81c60 con 0x56449aa69ea0 2024-05-08T21:26:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c8116eb80 10 client.0 _ll_register_callbacks cb 0x56449a95eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69540 0x56449aa7de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69540 0x56449aa7de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57956/0 (socket says 172.21.15.33:57956) 2024-05-08T21:26:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6e59c700 1 -- 172.21.15.33:0/3488595464 learned_addr learned my addr 172.21.15.33:0/3488595464 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:43.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6dd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69ea0 0x56449aa7e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6ed9d700 1 -- 172.21.15.33:0/3488595464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69ea0 msgr2=0x56449aa7e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:43.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6ed9d700 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69ea0 0x56449aa7e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6ed9d700 1 -- 172.21.15.33:0/3488595464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69540 msgr2=0x56449aa7de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:43.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6ed9d700 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69540 0x56449aa7de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6ed9d700 1 -- 172.21.15.33:0/3488595464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7019f220 con 0x56449aa6a860 2024-05-08T21:26:43.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6dd9b700 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69ea0 0x56449aa7e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:43.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69540 0x56449aa7de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c6ed9d700 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa813f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c68007b30 tx=0x7f5c68007b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=952021140c39304a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:26:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c55ffb700 1 -- 172.21.15.33:0/3488595464 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800ea80 con 0x56449aa6a860 2024-05-08T21:26:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c55ffb700 1 -- 172.21.15.33:0/3488595464 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f5c6800ebe0 con 0x56449aa6a860 2024-05-08T21:26:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.423+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3488595464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c60021070 con 0x56449aa6a860 2024-05-08T21:26:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.424+0000 7f5c55ffb700 1 -- 172.21.15.33:0/3488595464 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c6800eda0 con 0x56449aa6a860 2024-05-08T21:26:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.424+0000 7f5c8116eb80 10 client.27497 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.424+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3488595464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56449a83ecd0 con 0x56449aa6a860 2024-05-08T21:26:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.424+0000 7f5c55ffb700 1 -- 172.21.15.33:0/3488595464 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c60021070 con 0x56449aa6a860 2024-05-08T21:26:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.424+0000 7f5c55ffb700 1 -- 172.21.15.33:0/3488595464 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c68017410 con 0x56449aa6a860 2024-05-08T21:26:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.424+0000 7f5c8116eb80 10 client.27497 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.424+0000 7f5c8116eb80 10 client.27497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.424+0000 7f5c8116eb80 20 client.27497 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.424+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56449aaeeec0 0x56449aaf12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.424+0000 7f5c8116eb80 4 client.27497 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:26:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.424+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3488595464 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56449a9b1ec0 con 0x56449aaeeec0 2024-05-08T21:26:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.424+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56449aaeeec0 0x56449aaf12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c55ffb700 10 client.27497 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c6e59c700 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56449aaeeec0 0x56449aaf12e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc41cbed5665bfc server_cookie=6f195cd317d68832 in_seq=0 out_seq=0 2024-05-08T21:26:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c55ffb700 1 -- 172.21.15.33:0/3488595464 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56449a9b1ec0 con 0x56449aaeeec0 2024-05-08T21:26:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c55ffb700 10 client.27497 handle_command_reply: tid=0 2024-05-08T21:26:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c8116eb80 1 client.27497 shutdown 2024-05-08T21:26:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3488595464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 msgr2=0x56449aa813f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa813f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5c68007b30 tx=0x7f5c68007b60 comp rx=0 tx=0).stop 2024-05-08T21:26:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3488595464 shutdown_connections 2024-05-08T21:26:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x56449aaeeec0 0x56449aaf12e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56449aa6a860 0x56449aa813f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56449aa69540 0x56449aa7de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c55ffb700 0 client.27497 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c8116eb80 1 --2- 172.21.15.33:0/3488595464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56449aa69ea0 0x56449aa7e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3488595464 >> 172.21.15.33:0/3488595464 conn(0x56449a9c7210 msgr2=0x56449aa6d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3488595464 shutdown_connections 2024-05-08T21:26:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c8116eb80 1 -- 172.21.15.33:0/3488595464 wait complete. 2024-05-08T21:26:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.425+0000 7f5c8116eb80 20 client.27497 trim_cache size 0 max 16384 2024-05-08T21:26:43.436 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:26:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.426+0000 7f5c74fb4700 1 -- 172.21.15.33:0/4212828981 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5c580a11a0 msgr2=0x7f5c580a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.426+0000 7f5c74fb4700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5c580a11a0 0x7f5c580a3660 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5c70100db0 tx=0x7f5c60009260 comp rx=0 tx=0).stop 2024-05-08T21:26:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.426+0000 7f5c74fb4700 1 -- 172.21.15.33:0/4212828981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c70100f50 msgr2=0x7f5c7019baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.426+0000 7f5c74fb4700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c70100f50 0x7f5c7019baa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5c64005910 tx=0x7f5c6400b9c0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.426+0000 7f5c74fb4700 1 -- 172.21.15.33:0/4212828981 shutdown_connections 2024-05-08T21:26:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.426+0000 7f5c74fb4700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5c580a11a0 0x7f5c580a3660 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.426+0000 7f5c74fb4700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c701018f0 0x7f5c7019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.426+0000 7f5c74fb4700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c700ffd60 0x7f5c70068ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.426+0000 7f5c74fb4700 1 --2- 172.21.15.33:0/4212828981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c70100f50 0x7f5c7019baa0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.426+0000 7f5c74fb4700 1 -- 172.21.15.33:0/4212828981 >> 172.21.15.33:0/4212828981 conn(0x7f5c700fb290 msgr2=0x7f5c70104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.426+0000 7f5c74fb4700 1 -- 172.21.15.33:0/4212828981 shutdown_connections 2024-05-08T21:26:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:43.426+0000 7f5c74fb4700 1 -- 172.21.15.33:0/4212828981 wait complete. 2024-05-08T21:26:43.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:43 smithi033 ceph-mon[42528]: pgmap v522: 129 pgs: 129 active+clean; 257 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 537 op/s 2024-05-08T21:26:43.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:43 smithi071 ceph-mon[47074]: pgmap v522: 129 pgs: 129 active+clean; 257 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 537 op/s 2024-05-08T21:26:43.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:43 smithi082 ceph-mon[47512]: pgmap v522: 129 pgs: 129 active+clean; 257 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 537 op/s 2024-05-08T21:26:44.444 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:26:44.444 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:26:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.628+0000 7f72ce5a7700 1 -- 172.21.15.33:0/2765115136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72c80ffdc0 msgr2=0x7f72c81001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.628+0000 7f72ce5a7700 1 --2- 172.21.15.33:0/2765115136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72c80ffdc0 0x7f72c81001e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009920 tx=0x7f72b8009c30 comp rx=0 tx=0).stop 2024-05-08T21:26:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.628+0000 7f72ce5a7700 1 -- 172.21.15.33:0/2765115136 shutdown_connections 2024-05-08T21:26:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.628+0000 7f72ce5a7700 1 --2- 172.21.15.33:0/2765115136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c8101950 0x7f72c8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.628+0000 7f72ce5a7700 1 --2- 172.21.15.33:0/2765115136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72c80ffdc0 0x7f72c81001e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.628+0000 7f72ce5a7700 1 --2- 172.21.15.33:0/2765115136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c8100fb0 0x7f72c8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.628+0000 7f72ce5a7700 1 -- 172.21.15.33:0/2765115136 >> 172.21.15.33:0/2765115136 conn(0x7f72c80fb330 msgr2=0x7f72c80fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.628+0000 7f72ce5a7700 1 -- 172.21.15.33:0/2765115136 shutdown_connections 2024-05-08T21:26:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.628+0000 7f72ce5a7700 1 -- 172.21.15.33:0/2765115136 wait complete. 2024-05-08T21:26:44.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.629+0000 7f72ce5a7700 1 Processor -- start 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.629+0000 7f72ce5a7700 1 -- start start 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72ce5a7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c80ffdc0 0x7f72c8103d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72ce5a7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c8100fb0 0x7f72c8102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72ce5a7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72c8101950 0x7f72c81028d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72ce5a7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c8103070 con 0x7f72c80ffdc0 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72ce5a7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c81031e0 con 0x7f72c8101950 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72ce5a7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c81a0ee0 con 0x7f72c8100fb0 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72c77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c8100fb0 0x7f72c8102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c80ffdc0 0x7f72c8103d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72c77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c8100fb0 0x7f72c8102390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51212/0 (socket says 172.21.15.33:51212) 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72c77fe700 1 -- 172.21.15.33:0/1476588069 learned_addr learned my addr 172.21.15.33:0/1476588069 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72ccb44700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72c8101950 0x7f72c81028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72c77fe700 1 -- 172.21.15.33:0/1476588069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72c8101950 msgr2=0x7f72c81028d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72c77fe700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72c8101950 0x7f72c81028d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72c77fe700 1 -- 172.21.15.33:0/1476588069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c80ffdc0 msgr2=0x7f72c8103d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72c77fe700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c80ffdc0 0x7f72c8103d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72c77fe700 1 -- 172.21.15.33:0/1476588069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c0009d00 con 0x7f72c8100fb0 2024-05-08T21:26:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.630+0000 7f72c77fe700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c8100fb0 0x7f72c8102390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00e9a0 tx=0x7f72bc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.631+0000 7f72c57fa700 1 -- 172.21.15.33:0/1476588069 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc009790 con 0x7f72c8100fb0 2024-05-08T21:26:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.631+0000 7f72ce5a7700 1 -- 172.21.15.33:0/1476588069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b8009520 con 0x7f72c8100fb0 2024-05-08T21:26:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.631+0000 7f72c57fa700 1 -- 172.21.15.33:0/1476588069 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72bc010430 con 0x7f72c8100fb0 2024-05-08T21:26:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.631+0000 7f72c57fa700 1 -- 172.21.15.33:0/1476588069 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc00cea0 con 0x7f72c8100fb0 2024-05-08T21:26:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.631+0000 7f72ce5a7700 1 -- 172.21.15.33:0/1476588069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c81a1480 con 0x7f72c8100fb0 2024-05-08T21:26:44.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.635+0000 7f72da761b80 1 Processor -- start 2024-05-08T21:26:44.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.635+0000 7f72c57fa700 1 -- 172.21.15.33:0/1476588069 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f72bc0105a0 con 0x7f72c8100fb0 2024-05-08T21:26:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.635+0000 7f72c57fa700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f72b00a11a0 0x7f72b00a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72da761b80 1 -- start start 2024-05-08T21:26:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72c57fa700 1 -- 172.21.15.33:0/1476588069 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f72bc014070 con 0x7f72c8100fb0 2024-05-08T21:26:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72ccb44700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f72b00a11a0 0x7f72b00a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72da761b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32555c0 0x55e9f32559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72da761b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f3255f20 0x55e9f32563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72da761b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f32568e0 0x55e9f3256d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72da761b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f31ac1d0 con 0x55e9f32568e0 2024-05-08T21:26:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72da761b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f31a9b90 con 0x55e9f32555c0 2024-05-08T21:26:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72da761b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f31a5480 con 0x55e9f3255f20 2024-05-08T21:26:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72c7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32555c0 0x55e9f32559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72c7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32555c0 0x55e9f32559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:56418/0 (socket says 172.21.15.33:56418) 2024-05-08T21:26:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72c7fff700 1 -- 172.21.15.33:0/3743393904 learned_addr learned my addr 172.21.15.33:0/3743393904 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72ccb44700 1 --2- 172.21.15.33:0/3743393904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f32568e0 0x55e9f3256d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72c7fff700 1 -- 172.21.15.33:0/3743393904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f3255f20 msgr2=0x55e9f32563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.636+0000 7f72c7fff700 1 --2- 172.21.15.33:0/3743393904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f3255f20 0x55e9f32563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.637+0000 7f72c7fff700 1 -- 172.21.15.33:0/3743393904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f32568e0 msgr2=0x55e9f3256d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.637+0000 7f72c7fff700 1 --2- 172.21.15.33:0/3743393904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f32568e0 0x55e9f3256d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.637+0000 7f72c7fff700 1 -- 172.21.15.33:0/3743393904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9f3026430 con 0x55e9f32555c0 2024-05-08T21:26:44.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.637+0000 7f72c7fff700 1 --2- 172.21.15.33:0/3743393904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32555c0 0x55e9f32559e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009920 tx=0x7f72b8009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28a947bb60a45bb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:44.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.637+0000 7f72ae7fc700 1 -- 172.21.15.33:0/3743393904 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800fda0 con 0x55e9f32555c0 2024-05-08T21:26:44.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.637+0000 7f72ccb44700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f72b00a11a0 0x7f72b00a3660 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009fd0 tx=0x7f72c00093b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:44.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.637+0000 7f72ae7fc700 1 -- 172.21.15.33:0/3743393904 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72b801d040 con 0x55e9f32555c0 2024-05-08T21:26:44.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.637+0000 7f72ae7fc700 1 -- 172.21.15.33:0/3743393904 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b80052a0 con 0x55e9f32555c0 2024-05-08T21:26:44.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.638+0000 7f72da761b80 1 -- 172.21.15.33:0/3743393904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32555c0 msgr2=0x55e9f32559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.638+0000 7f72da761b80 1 --2- 172.21.15.33:0/3743393904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32555c0 0x55e9f32559e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009920 tx=0x7f72b8009f80 comp rx=0 tx=0).stop 2024-05-08T21:26:44.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.638+0000 7f72da761b80 1 -- 172.21.15.33:0/3743393904 shutdown_connections 2024-05-08T21:26:44.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.638+0000 7f72da761b80 1 --2- 172.21.15.33:0/3743393904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f32568e0 0x55e9f3256d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.638+0000 7f72da761b80 1 --2- 172.21.15.33:0/3743393904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32555c0 0x55e9f32559e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.638+0000 7f72da761b80 1 --2- 172.21.15.33:0/3743393904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f3255f20 0x55e9f32563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.638+0000 7f72da761b80 1 -- 172.21.15.33:0/3743393904 >> 172.21.15.33:0/3743393904 conn(0x55e9f31b3290 msgr2=0x55e9f324fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.638+0000 7f72da761b80 1 -- 172.21.15.33:0/3743393904 shutdown_connections 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.638+0000 7f72da761b80 1 -- 172.21.15.33:0/3743393904 wait complete. 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.638+0000 7f72da761b80 1 -- start start 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.639+0000 7f72da761b80 1 Processor -- start 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.639+0000 7f72da761b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f32555c0 0x55e9f325ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.639+0000 7f72da761b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f3255f20 0x55e9f325f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.639+0000 7f72da761b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32568e0 0x55e9f32663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.639+0000 7f72da761b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f31415e0 con 0x55e9f32555c0 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.639+0000 7f72da761b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f3141280 con 0x55e9f32568e0 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.639+0000 7f72da761b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f3028720 con 0x55e9f3255f20 2024-05-08T21:26:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.639+0000 7f72da761b80 10 client.0 _ll_register_callbacks cb 0x55e9f314abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:44.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.639+0000 7f72c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f32555c0 0x55e9f325ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.639+0000 7f72c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f32555c0 0x55e9f325ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57996/0 (socket says 172.21.15.33:57996) 2024-05-08T21:26:44.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.639+0000 7f72c7fff700 1 -- 172.21.15.33:0/1863112621 learned_addr learned my addr 172.21.15.33:0/1863112621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:44.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.640+0000 7f72ccb44700 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32568e0 0x55e9f32663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.640+0000 7f72c77fe700 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f3255f20 0x55e9f325f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.640+0000 7f72ccb44700 1 -- 172.21.15.33:0/1863112621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f3255f20 msgr2=0x55e9f325f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.640+0000 7f72ccb44700 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f3255f20 0x55e9f325f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.640+0000 7f72ccb44700 1 -- 172.21.15.33:0/1863112621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f32555c0 msgr2=0x55e9f325ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.640+0000 7f72ccb44700 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f32555c0 0x55e9f325ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.640+0000 7f72ccb44700 1 -- 172.21.15.33:0/1863112621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c8108b50 con 0x55e9f32568e0 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.640+0000 7f72ccb44700 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32568e0 0x55e9f32663d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55e9f3250dc0 tx=0x7f72c0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a3c35aba51d55b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.640+0000 7f72a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72a77fe700 1 -- 172.21.15.33:0/1863112621 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0028070 con 0x55e9f32568e0 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72a77fe700 1 -- 172.21.15.33:0/1863112621 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f72c001ab80 con 0x55e9f32568e0 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72da761b80 1 -- 172.21.15.33:0/1863112621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9f30fe2f0 con 0x55e9f32568e0 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72a77fe700 1 -- 172.21.15.33:0/1863112621 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72c0020040 con 0x55e9f32568e0 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72da761b80 10 client.27509 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72da761b80 1 -- 172.21.15.33:0/1863112621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55e9f2fd2be0 con 0x55e9f32568e0 2024-05-08T21:26:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72a77fe700 1 -- 172.21.15.33:0/1863112621 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e9f30fe2f0 con 0x55e9f32568e0 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72a77fe700 1 -- 172.21.15.33:0/1863112621 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f72c002b020 con 0x55e9f32568e0 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72da761b80 10 client.27509 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72da761b80 10 client.27509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72da761b80 20 client.27509 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72da761b80 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9f327bc90 0x55e9f327e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72da761b80 4 client.27509 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.641+0000 7f72da761b80 1 -- 172.21.15.33:0/1863112621 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9f3224890 con 0x55e9f327bc90 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.642+0000 7f72c7fff700 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9f327bc90 0x55e9f327e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.642+0000 7f72a77fe700 10 client.27509 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.642+0000 7f72c7fff700 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9f327bc90 0x55e9f327e0b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c5640dcb7d3c server_cookie=777014dfe1166b82 in_seq=0 out_seq=0 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72a77fe700 1 -- 172.21.15.33:0/1863112621 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e9f3224890 con 0x55e9f327bc90 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72a77fe700 10 client.27509 handle_command_reply: tid=0 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72da761b80 1 client.27509 shutdown 2024-05-08T21:26:44.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72da761b80 1 -- 172.21.15.33:0/1863112621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32568e0 msgr2=0x55e9f32663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72da761b80 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32568e0 0x55e9f32663d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55e9f3250dc0 tx=0x7f72c0004b40 comp rx=0 tx=0).stop 2024-05-08T21:26:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72da761b80 1 -- 172.21.15.33:0/1863112621 shutdown_connections 2024-05-08T21:26:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72da761b80 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9f327bc90 0x55e9f327e0b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72da761b80 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32568e0 0x55e9f32663d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72da761b80 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f32555c0 0x55e9f325ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72da761b80 1 --2- 172.21.15.33:0/1863112621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f3255f20 0x55e9f325f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72da761b80 1 -- 172.21.15.33:0/1863112621 >> 172.21.15.33:0/1863112621 conn(0x55e9f31b3290 msgr2=0x55e9f324f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72a77fe700 0 client.27509 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72da761b80 1 -- 172.21.15.33:0/1863112621 shutdown_connections 2024-05-08T21:26:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72da761b80 1 -- 172.21.15.33:0/1863112621 wait complete. 2024-05-08T21:26:44.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.643+0000 7f72da761b80 20 client.27509 trim_cache size 0 max 16384 2024-05-08T21:26:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72da761b80 1 Processor -- start 2024-05-08T21:26:44.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72da761b80 1 -- start start 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72da761b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32555c0 0x55e9f325f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72da761b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f3255f20 0x55e9f3266390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72da761b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32568e0 0x55e9f325db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72da761b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f3278560 con 0x55e9f3255f20 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72da761b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f325e9c0 con 0x55e9f32568e0 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72da761b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f324ce40 con 0x55e9f32555c0 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f3255f20 0x55e9f3266390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f3255f20 0x55e9f3266390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57998/0 (socket says 172.21.15.33:57998) 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72c7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32555c0 0x55e9f325f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72c77fe700 1 -- 172.21.15.33:0/3646133715 learned_addr learned my addr 172.21.15.33:0/3646133715 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.667+0000 7f72ccb44700 1 --2- 172.21.15.33:0/3646133715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32568e0 0x55e9f325db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.668+0000 7f72c7fff700 1 -- 172.21.15.33:0/3646133715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32568e0 msgr2=0x55e9f325db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.668+0000 7f72c7fff700 1 --2- 172.21.15.33:0/3646133715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32568e0 0x55e9f325db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.668+0000 7f72c7fff700 1 -- 172.21.15.33:0/3646133715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f3255f20 msgr2=0x55e9f3266390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.668+0000 7f72c7fff700 1 --2- 172.21.15.33:0/3646133715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f3255f20 0x55e9f3266390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.668+0000 7f72c7fff700 1 -- 172.21.15.33:0/3646133715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9f316b5e0 con 0x55e9f32555c0 2024-05-08T21:26:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.668+0000 7f72ccb44700 1 --2- 172.21.15.33:0/3646133715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32568e0 0x55e9f325db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.668+0000 7f72c7fff700 1 --2- 172.21.15.33:0/3646133715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32555c0 0x55e9f325f310 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f72b800ba60 tx=0x7f72b800fac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acbd74cf618cb0dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.668+0000 7f72aeffd700 1 -- 172.21.15.33:0/3646133715 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b80097f0 con 0x55e9f32555c0 2024-05-08T21:26:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.668+0000 7f72aeffd700 1 -- 172.21.15.33:0/3646133715 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72b800b740 con 0x55e9f32555c0 2024-05-08T21:26:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.668+0000 7f72aeffd700 1 -- 172.21.15.33:0/3646133715 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8005570 con 0x55e9f32555c0 2024-05-08T21:26:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.669+0000 7f72da761b80 1 -- 172.21.15.33:0/3646133715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32555c0 msgr2=0x55e9f325f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.669+0000 7f72da761b80 1 --2- 172.21.15.33:0/3646133715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32555c0 0x55e9f325f310 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f72b800ba60 tx=0x7f72b800fac0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.669+0000 7f72da761b80 1 -- 172.21.15.33:0/3646133715 shutdown_connections 2024-05-08T21:26:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.669+0000 7f72da761b80 1 --2- 172.21.15.33:0/3646133715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32568e0 0x55e9f325db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.669+0000 7f72da761b80 1 --2- 172.21.15.33:0/3646133715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32555c0 0x55e9f325f310 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.669+0000 7f72da761b80 1 --2- 172.21.15.33:0/3646133715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f3255f20 0x55e9f3266390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.669+0000 7f72da761b80 1 -- 172.21.15.33:0/3646133715 >> 172.21.15.33:0/3646133715 conn(0x55e9f31b3290 msgr2=0x55e9f325ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:44.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.669+0000 7f72da761b80 1 -- 172.21.15.33:0/3646133715 shutdown_connections 2024-05-08T21:26:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.670+0000 7f72da761b80 1 -- 172.21.15.33:0/3646133715 wait complete. 2024-05-08T21:26:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.671+0000 7f72da761b80 1 -- start start 2024-05-08T21:26:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.671+0000 7f72da761b80 1 Processor -- start 2024-05-08T21:26:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.671+0000 7f72da761b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32555c0 0x55e9f3265fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.671+0000 7f72da761b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f3255f20 0x55e9f3262a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.671+0000 7f72da761b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32568e0 0x55e9f3262fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.671+0000 7f72da761b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f32634e0 con 0x55e9f3255f20 2024-05-08T21:26:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.671+0000 7f72da761b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f3263650 con 0x55e9f32555c0 2024-05-08T21:26:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.671+0000 7f72da761b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9f32d2180 con 0x55e9f32568e0 2024-05-08T21:26:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.671+0000 7f72da761b80 10 client.0 _ll_register_callbacks cb 0x55e9f314abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:44.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72ccb44700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32568e0 0x55e9f3262fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72ccb44700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32568e0 0x55e9f3262fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51250/0 (socket says 172.21.15.33:51250) 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72ccb44700 1 -- 172.21.15.33:0/1001429969 learned_addr learned my addr 172.21.15.33:0/1001429969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72c7fff700 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32555c0 0x55e9f3265fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72c77fe700 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f3255f20 0x55e9f3262a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72ccb44700 1 -- 172.21.15.33:0/1001429969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32555c0 msgr2=0x55e9f3265fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72ccb44700 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32555c0 0x55e9f3265fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72ccb44700 1 -- 172.21.15.33:0/1001429969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f3255f20 msgr2=0x55e9f3262a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72ccb44700 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f3255f20 0x55e9f3262a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72ccb44700 1 -- 172.21.15.33:0/1001429969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9f2fd2be0 con 0x55e9f32568e0 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72c77fe700 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f3255f20 0x55e9f3262a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72ccb44700 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32568e0 0x55e9f3262fa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f72c0024eb0 tx=0x7f72c0007630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d543df19625fda server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.672+0000 7f72a77fe700 1 -- 172.21.15.33:0/1001429969 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0028070 con 0x55e9f32568e0 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.673+0000 7f72a77fe700 1 -- 172.21.15.33:0/1001429969 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f72c001ac10 con 0x55e9f32568e0 2024-05-08T21:26:44.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.673+0000 7f72da761b80 1 -- 172.21.15.33:0/1001429969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72b801d040 con 0x55e9f32568e0 2024-05-08T21:26:44.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.673+0000 7f72a77fe700 1 -- 172.21.15.33:0/1001429969 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72c0006000 con 0x55e9f32568e0 2024-05-08T21:26:44.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.673+0000 7f72da761b80 10 client.27523 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:44.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.673+0000 7f72da761b80 1 -- 172.21.15.33:0/1001429969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55e9f3026430 con 0x55e9f32568e0 2024-05-08T21:26:44.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.674+0000 7f72a77fe700 1 -- 172.21.15.33:0/1001429969 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72b801d040 con 0x55e9f32568e0 2024-05-08T21:26:44.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.674+0000 7f72a77fe700 1 -- 172.21.15.33:0/1001429969 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f72c001a440 con 0x55e9f32568e0 2024-05-08T21:26:44.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.674+0000 7f72da761b80 10 client.27523 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:44.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.674+0000 7f72da761b80 10 client.27523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:44.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.674+0000 7f72da761b80 20 client.27523 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:44.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.674+0000 7f72da761b80 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9f32dc310 0x55e9f32de730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.674+0000 7f72da761b80 4 client.27523 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:26:44.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.674+0000 7f72da761b80 1 -- 172.21.15.33:0/1001429969 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e9f319df40 con 0x55e9f32dc310 2024-05-08T21:26:44.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.675+0000 7f72c7fff700 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9f32dc310 0x55e9f32de730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.675+0000 7f72a77fe700 10 client.27523 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:44.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.675+0000 7f72c7fff700 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9f32dc310 0x55e9f32de730 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc9906e5731fce server_cookie=432d52fb9c93430 in_seq=0 out_seq=0 2024-05-08T21:26:44.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.676+0000 7f72a77fe700 1 -- 172.21.15.33:0/1001429969 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e9f319df40 con 0x55e9f32dc310 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.676+0000 7f72a77fe700 10 client.27523 handle_command_reply: tid=0 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.676+0000 7f72da761b80 1 client.27523 shutdown 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72da761b80 1 -- 172.21.15.33:0/1001429969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32568e0 msgr2=0x55e9f3262fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72da761b80 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32568e0 0x55e9f3262fa0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f72c0024eb0 tx=0x7f72c0007630 comp rx=0 tx=0).stop 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72da761b80 1 -- 172.21.15.33:0/1001429969 shutdown_connections 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72da761b80 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55e9f32dc310 0x55e9f32de730 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72da761b80 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e9f32568e0 0x55e9f3262fa0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72da761b80 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e9f32555c0 0x55e9f3265fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72da761b80 1 --2- 172.21.15.33:0/1001429969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9f3255f20 0x55e9f3262a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72a77fe700 0 client.27523 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72da761b80 1 -- 172.21.15.33:0/1001429969 >> 172.21.15.33:0/1001429969 conn(0x55e9f31b3290 msgr2=0x55e9f32543c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72da761b80 1 -- 172.21.15.33:0/1001429969 shutdown_connections 2024-05-08T21:26:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72da761b80 1 -- 172.21.15.33:0/1001429969 wait complete. 2024-05-08T21:26:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72da761b80 20 client.27523 trim_cache size 0 max 16384 2024-05-08T21:26:44.680 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:26:44.680 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:26:44.680 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "1c9c4d96-a8b0-4a91-a214-3529753d11fe", 2024-05-08T21:26:44.680 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:26:44.680 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:26:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72ce5a7700 1 -- 172.21.15.33:0/1476588069 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f72b00a11a0 msgr2=0x7f72b00a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72ce5a7700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f72b00a11a0 0x7f72b00a3660 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009fd0 tx=0x7f72c00093b0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72ce5a7700 1 -- 172.21.15.33:0/1476588069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c8100fb0 msgr2=0x7f72c8102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.677+0000 7f72ce5a7700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c8100fb0 0x7f72c8102390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00e9a0 tx=0x7f72bc00ecb0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.678+0000 7f72ce5a7700 1 -- 172.21.15.33:0/1476588069 shutdown_connections 2024-05-08T21:26:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.678+0000 7f72ce5a7700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f72b00a11a0 0x7f72b00a3660 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.678+0000 7f72ce5a7700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72c8101950 0x7f72c81028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.678+0000 7f72ce5a7700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c80ffdc0 0x7f72c8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.678+0000 7f72ce5a7700 1 --2- 172.21.15.33:0/1476588069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72c8100fb0 0x7f72c8102390 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.678+0000 7f72ce5a7700 1 -- 172.21.15.33:0/1476588069 >> 172.21.15.33:0/1476588069 conn(0x7f72c80fb330 msgr2=0x7f72c8069060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:44.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.678+0000 7f72ce5a7700 1 -- 172.21.15.33:0/1476588069 shutdown_connections 2024-05-08T21:26:44.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.678+0000 7f72ce5a7700 1 -- 172.21.15.33:0/1476588069 wait complete. 2024-05-08T21:26:44.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:26:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.909+0000 7f9615e80700 1 -- 172.21.15.33:0/1012699751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9610107fa0 msgr2=0x7f961010a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.909+0000 7f9615e80700 1 --2- 172.21.15.33:0/1012699751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9610107fa0 0x7f961010a3d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9600009990 tx=0x7f9600009ca0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.909+0000 7f9615e80700 1 -- 172.21.15.33:0/1012699751 shutdown_connections 2024-05-08T21:26:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.909+0000 7f9615e80700 1 --2- 172.21.15.33:0/1012699751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f961010d2e0 0x7f961010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.909+0000 7f9615e80700 1 --2- 172.21.15.33:0/1012699751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9610107fa0 0x7f961010a3d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.909+0000 7f9615e80700 1 --2- 172.21.15.33:0/1012699751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f961010a910 0x7f961010cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.909+0000 7f9615e80700 1 -- 172.21.15.33:0/1012699751 >> 172.21.15.33:0/1012699751 conn(0x7f961006dcb0 msgr2=0x7f96100700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.909+0000 7f9615e80700 1 -- 172.21.15.33:0/1012699751 shutdown_connections 2024-05-08T21:26:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.909+0000 7f9615e80700 1 -- 172.21.15.33:0/1012699751 wait complete. 2024-05-08T21:26:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.910+0000 7f9615e80700 1 Processor -- start 2024-05-08T21:26:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.911+0000 7f9615e80700 1 -- start start 2024-05-08T21:26:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.911+0000 7f9615e80700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9610107fa0 0x7f96101a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.911+0000 7f9615e80700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f961010a910 0x7f96101a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.911+0000 7f9615e80700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f961010d2e0 0x7f96101ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.911+0000 7f9615e80700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96101ad610 con 0x7f961010d2e0 2024-05-08T21:26:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.911+0000 7f9615e80700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961010f6d0 con 0x7f961010a910 2024-05-08T21:26:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.911+0000 7f9615e80700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961010f860 con 0x7f9610107fa0 2024-05-08T21:26:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.911+0000 7f960f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9610107fa0 0x7f96101a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.911+0000 7f960f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9610107fa0 0x7f96101a5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51262/0 (socket says 172.21.15.33:51262) 2024-05-08T21:26:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.911+0000 7f960f7fe700 1 -- 172.21.15.33:0/1012878348 learned_addr learned my addr 172.21.15.33:0/1012878348 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.911+0000 7f960effd700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f961010a910 0x7f96101a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.912+0000 7f960effd700 1 -- 172.21.15.33:0/1012878348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9610107fa0 msgr2=0x7f96101a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.912+0000 7f960effd700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9610107fa0 0x7f96101a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.912+0000 7f960effd700 1 -- 172.21.15.33:0/1012878348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f961010d2e0 msgr2=0x7f96101ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.912+0000 7f960effd700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f961010d2e0 0x7f96101ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.912+0000 7f960effd700 1 -- 172.21.15.33:0/1012878348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96000095f0 con 0x7f961010a910 2024-05-08T21:26:44.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.912+0000 7f960f7fe700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9610107fa0 0x7f96101a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.912+0000 7f960effd700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f961010a910 0x7f96101a6070 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f960400e9f0 tx=0x7f960400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.913+0000 7f960cff9700 1 -- 172.21.15.33:0/1012878348 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960400eee0 con 0x7f961010a910 2024-05-08T21:26:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.913+0000 7f960cff9700 1 -- 172.21.15.33:0/1012878348 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9604010430 con 0x7f961010a910 2024-05-08T21:26:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.913+0000 7f960cff9700 1 -- 172.21.15.33:0/1012878348 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604018680 con 0x7f961010a910 2024-05-08T21:26:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.913+0000 7f9615e80700 1 -- 172.21.15.33:0/1012878348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961010fb40 con 0x7f961010a910 2024-05-08T21:26:44.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.913+0000 7f9615e80700 1 -- 172.21.15.33:0/1012878348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9610110000 con 0x7f961010a910 2024-05-08T21:26:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.916+0000 7f960cff9700 1 -- 172.21.15.33:0/1012878348 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f96040187e0 con 0x7f961010a910 2024-05-08T21:26:44.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.917+0000 7f960cff9700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95f80a12f0 0x7f95f80a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.917+0000 7f960cff9700 1 -- 172.21.15.33:0/1012878348 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9604014070 con 0x7f961010a910 2024-05-08T21:26:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.919+0000 7f962203ab80 1 Processor -- start 2024-05-08T21:26:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.919+0000 7f962203ab80 1 -- start start 2024-05-08T21:26:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.919+0000 7f962203ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aa540 0x561b6a2aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.919+0000 7f962203ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aaea0 0x561b6a2ab320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.919+0000 7f962203ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2ab860 0x561b6a2abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.919+0000 7f962203ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6a201150 con 0x561b6a2aaea0 2024-05-08T21:26:44.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.919+0000 7f962203ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6a1feb10 con 0x561b6a2aa540 2024-05-08T21:26:44.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.919+0000 7f962203ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6a1fa400 con 0x561b6a2ab860 2024-05-08T21:26:44.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.919+0000 7f960f7fe700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95f80a12f0 0x7f95f80a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.921+0000 7f960effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aaea0 0x561b6a2ab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.921+0000 7f960effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aaea0 0x561b6a2ab320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58028/0 (socket says 172.21.15.33:58028) 2024-05-08T21:26:44.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.921+0000 7f960effd700 1 -- 172.21.15.33:0/1054185511 learned_addr learned my addr 172.21.15.33:0/1054185511 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:44.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.921+0000 7f960ffff700 1 --2- 172.21.15.33:0/1054185511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2ab860 0x561b6a2abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.921+0000 7f960effd700 1 -- 172.21.15.33:0/1054185511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2ab860 msgr2=0x561b6a2abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.921+0000 7f960effd700 1 --2- 172.21.15.33:0/1054185511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2ab860 0x561b6a2abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.921+0000 7f960effd700 1 -- 172.21.15.33:0/1054185511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aa540 msgr2=0x561b6a2aa960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.921+0000 7f960effd700 1 --2- 172.21.15.33:0/1054185511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aa540 0x561b6a2aa960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.921+0000 7f960effd700 1 -- 172.21.15.33:0/1054185511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b6a07b3b0 con 0x561b6a2aaea0 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.921+0000 7f960effd700 1 --2- 172.21.15.33:0/1054185511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aaea0 0x561b6a2ab320 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f96040a1d20 tx=0x7f96040999e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a34360e2ac21cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.922+0000 7f95f5ffb700 1 -- 172.21.15.33:0/1054185511 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604010860 con 0x561b6a2aaea0 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.922+0000 7f95f5ffb700 1 -- 172.21.15.33:0/1054185511 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f960409a4d0 con 0x561b6a2aaea0 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.922+0000 7f95f5ffb700 1 -- 172.21.15.33:0/1054185511 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96040992c0 con 0x561b6a2aaea0 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.923+0000 7f962203ab80 1 -- 172.21.15.33:0/1054185511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aaea0 msgr2=0x561b6a2ab320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.923+0000 7f962203ab80 1 --2- 172.21.15.33:0/1054185511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aaea0 0x561b6a2ab320 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f96040a1d20 tx=0x7f96040999e0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.923+0000 7f960f7fe700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95f80a12f0 0x7f95f80a37b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f960000b2e0 tx=0x7f960000b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.923+0000 7f962203ab80 1 -- 172.21.15.33:0/1054185511 shutdown_connections 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.923+0000 7f962203ab80 1 --2- 172.21.15.33:0/1054185511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2ab860 0x561b6a2abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.923+0000 7f962203ab80 1 --2- 172.21.15.33:0/1054185511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aa540 0x561b6a2aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.923+0000 7f962203ab80 1 --2- 172.21.15.33:0/1054185511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aaea0 0x561b6a2ab320 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.923+0000 7f962203ab80 1 -- 172.21.15.33:0/1054185511 >> 172.21.15.33:0/1054185511 conn(0x561b6a208210 msgr2=0x561b6a2a4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.923+0000 7f962203ab80 1 -- 172.21.15.33:0/1054185511 shutdown_connections 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.923+0000 7f962203ab80 1 -- 172.21.15.33:0/1054185511 wait complete. 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.924+0000 7f962203ab80 1 -- start start 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.924+0000 7f962203ab80 1 Processor -- start 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.924+0000 7f962203ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aa540 0x561b6a2b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.924+0000 7f962203ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aaea0 0x561b6a2b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.924+0000 7f962203ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2ab860 0x561b6a2bb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.924+0000 7f962203ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6a196560 con 0x561b6a2ab860 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.924+0000 7f962203ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6a196200 con 0x561b6a2aa540 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.924+0000 7f962203ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6a07d6a0 con 0x561b6a2aaea0 2024-05-08T21:26:44.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.924+0000 7f962203ab80 10 client.0 _ll_register_callbacks cb 0x561b6a19fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.924+0000 7f960effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aaea0 0x561b6a2b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.924+0000 7f960effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aaea0 0x561b6a2b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51288/0 (socket says 172.21.15.33:51288) 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.924+0000 7f960effd700 1 -- 172.21.15.33:0/3013463936 learned_addr learned my addr 172.21.15.33:0/3013463936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.925+0000 7f960ffff700 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2ab860 0x561b6a2bb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.925+0000 7f960f7fe700 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aa540 0x561b6a2b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.925+0000 7f960ffff700 1 -- 172.21.15.33:0/3013463936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aaea0 msgr2=0x561b6a2b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.925+0000 7f960ffff700 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aaea0 0x561b6a2b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.925+0000 7f960ffff700 1 -- 172.21.15.33:0/3013463936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aa540 msgr2=0x561b6a2b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.925+0000 7f960ffff700 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aa540 0x561b6a2b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.925+0000 7f960ffff700 1 -- 172.21.15.33:0/3013463936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9610110000 con 0x561b6a2ab860 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.925+0000 7f960effd700 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aaea0 0x561b6a2b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.926+0000 7f960ffff700 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2ab860 0x561b6a2bb420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9608011ce0 tx=0x7f9608009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5af45791546cb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:44.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.926+0000 7f95eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:26:44.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.926+0000 7f95eeffd700 1 -- 172.21.15.33:0/3013463936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960800e040 con 0x561b6a2ab860 2024-05-08T21:26:44.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.926+0000 7f95eeffd700 1 -- 172.21.15.33:0/3013463936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f96080093a0 con 0x561b6a2ab860 2024-05-08T21:26:44.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.926+0000 7f962203ab80 1 -- 172.21.15.33:0/3013463936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b6a153270 con 0x561b6a2ab860 2024-05-08T21:26:44.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.926+0000 7f95eeffd700 1 -- 172.21.15.33:0/3013463936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9608004db0 con 0x561b6a2ab860 2024-05-08T21:26:44.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.926+0000 7f962203ab80 10 client.17634 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:44.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.926+0000 7f962203ab80 1 -- 172.21.15.33:0/3013463936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561b6a027b60 con 0x561b6a2ab860 2024-05-08T21:26:44.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.927+0000 7f95eeffd700 1 -- 172.21.15.33:0/3013463936 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561b6a153270 con 0x561b6a2ab860 2024-05-08T21:26:44.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.927+0000 7f95eeffd700 1 -- 172.21.15.33:0/3013463936 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f960801b020 con 0x561b6a2ab860 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.928+0000 7f962203ab80 10 client.17634 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.928+0000 7f962203ab80 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.928+0000 7f962203ab80 20 client.17634 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.928+0000 7f962203ab80 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561b6a2bfd50 0x561b6a2c2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.928+0000 7f962203ab80 4 client.17634 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.928+0000 7f962203ab80 1 -- 172.21.15.33:0/3013463936 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b6a279810 con 0x561b6a2bfd50 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.928+0000 7f960f7fe700 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561b6a2bfd50 0x561b6a2c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.929+0000 7f960f7fe700 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561b6a2bfd50 0x561b6a2c2170 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce9a547cb1eb2a8 server_cookie=decf1be56327f5e0 in_seq=0 out_seq=0 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.929+0000 7f95eeffd700 10 client.17634 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.929+0000 7f95eeffd700 1 -- 172.21.15.33:0/3013463936 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b6a279810 con 0x561b6a2bfd50 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f95eeffd700 10 client.17634 handle_command_reply: tid=0 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f962203ab80 1 client.17634 shutdown 2024-05-08T21:26:44.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f962203ab80 1 -- 172.21.15.33:0/3013463936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2ab860 msgr2=0x561b6a2bb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f962203ab80 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2ab860 0x561b6a2bb420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9608011ce0 tx=0x7f9608009f00 comp rx=0 tx=0).stop 2024-05-08T21:26:44.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f962203ab80 1 -- 172.21.15.33:0/3013463936 shutdown_connections 2024-05-08T21:26:44.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f962203ab80 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561b6a2bfd50 0x561b6a2c2170 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f962203ab80 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2ab860 0x561b6a2bb420 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f962203ab80 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aa540 0x561b6a2b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f962203ab80 1 --2- 172.21.15.33:0/3013463936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aaea0 0x561b6a2b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f962203ab80 1 -- 172.21.15.33:0/3013463936 >> 172.21.15.33:0/3013463936 conn(0x561b6a208210 msgr2=0x561b6a2a45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:44.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f95eeffd700 0 client.17634 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:44.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f962203ab80 1 -- 172.21.15.33:0/3013463936 shutdown_connections 2024-05-08T21:26:44.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.930+0000 7f962203ab80 1 -- 172.21.15.33:0/3013463936 wait complete. 2024-05-08T21:26:44.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.931+0000 7f962203ab80 20 client.17634 trim_cache size 0 max 16384 2024-05-08T21:26:44.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.952+0000 7f962203ab80 1 Processor -- start 2024-05-08T21:26:44.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.952+0000 7f962203ab80 1 -- start start 2024-05-08T21:26:44.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f962203ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aa540 0x561b6a2bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f962203ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aaea0 0x561b6a2bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f962203ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2ab860 0x561b6a2b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f962203ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6a2bd1f0 con 0x561b6a2aaea0 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f962203ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6a2b3a10 con 0x561b6a2ab860 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f962203ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6a2a6860 con 0x561b6a2aa540 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f960effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aaea0 0x561b6a2bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f960f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aa540 0x561b6a2bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f960f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aa540 0x561b6a2bbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:51296/0 (socket says 172.21.15.33:51296) 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f960f7fe700 1 -- 172.21.15.33:0/418131903 learned_addr learned my addr 172.21.15.33:0/418131903 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f960ffff700 1 --2- 172.21.15.33:0/418131903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2ab860 0x561b6a2b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f960f7fe700 1 -- 172.21.15.33:0/418131903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2ab860 msgr2=0x561b6a2b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f960f7fe700 1 --2- 172.21.15.33:0/418131903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2ab860 0x561b6a2b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f960f7fe700 1 -- 172.21.15.33:0/418131903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aaea0 msgr2=0x561b6a2bb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f960f7fe700 1 --2- 172.21.15.33:0/418131903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aaea0 0x561b6a2bb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.953+0000 7f960f7fe700 1 -- 172.21.15.33:0/418131903 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b6a1c0560 con 0x561b6a2aa540 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.954+0000 7f960f7fe700 1 --2- 172.21.15.33:0/418131903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aa540 0x561b6a2bbad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9600005910 tx=0x7f9600020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87d9f2f9cc583f83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.954+0000 7f95f67fc700 1 -- 172.21.15.33:0/418131903 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960002b040 con 0x561b6a2aa540 2024-05-08T21:26:44.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.954+0000 7f95f67fc700 1 -- 172.21.15.33:0/418131903 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f960002c040 con 0x561b6a2aa540 2024-05-08T21:26:44.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.954+0000 7f95f67fc700 1 -- 172.21.15.33:0/418131903 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600004c30 con 0x561b6a2aa540 2024-05-08T21:26:44.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.955+0000 7f962203ab80 1 -- 172.21.15.33:0/418131903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aa540 msgr2=0x561b6a2bbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.955+0000 7f962203ab80 1 --2- 172.21.15.33:0/418131903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aa540 0x561b6a2bbad0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9600005910 tx=0x7f9600020000 comp rx=0 tx=0).stop 2024-05-08T21:26:44.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.955+0000 7f962203ab80 1 -- 172.21.15.33:0/418131903 shutdown_connections 2024-05-08T21:26:44.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.955+0000 7f962203ab80 1 --2- 172.21.15.33:0/418131903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2ab860 0x561b6a2b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.955+0000 7f962203ab80 1 --2- 172.21.15.33:0/418131903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2aa540 0x561b6a2bbad0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.955+0000 7f962203ab80 1 --2- 172.21.15.33:0/418131903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aaea0 0x561b6a2bb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.955+0000 7f962203ab80 1 -- 172.21.15.33:0/418131903 >> 172.21.15.33:0/418131903 conn(0x561b6a208210 msgr2=0x561b6a2abc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:44.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.955+0000 7f962203ab80 1 -- 172.21.15.33:0/418131903 shutdown_connections 2024-05-08T21:26:44.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.955+0000 7f962203ab80 1 -- 172.21.15.33:0/418131903 wait complete. 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.955+0000 7f962203ab80 1 -- start start 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.955+0000 7f962203ab80 1 Processor -- start 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.956+0000 7f962203ab80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aa540 0x561b6a2b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.956+0000 7f962203ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aaea0 0x561b6a2b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.956+0000 7f962203ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2ab860 0x561b6a2b9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.956+0000 7f962203ab80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6a2ba150 con 0x561b6a2aa540 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.956+0000 7f962203ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6a2ba2c0 con 0x561b6a2aaea0 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.956+0000 7f962203ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b6a2ba480 con 0x561b6a2ab860 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.956+0000 7f962203ab80 10 client.0 _ll_register_callbacks cb 0x561b6a19fb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.956+0000 7f960f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aa540 0x561b6a2b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.956+0000 7f960f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aa540 0x561b6a2b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58050/0 (socket says 172.21.15.33:58050) 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.956+0000 7f960effd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aaea0 0x561b6a2b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.956+0000 7f960f7fe700 1 -- 172.21.15.33:0/4192441170 learned_addr learned my addr 172.21.15.33:0/4192441170 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:26:44.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.956+0000 7f960ffff700 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2ab860 0x561b6a2b9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f960f7fe700 1 -- 172.21.15.33:0/4192441170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2ab860 msgr2=0x561b6a2b9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f960f7fe700 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2ab860 0x561b6a2b9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f960f7fe700 1 -- 172.21.15.33:0/4192441170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aaea0 msgr2=0x561b6a2b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f960f7fe700 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aaea0 0x561b6a2b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f960f7fe700 1 -- 172.21.15.33:0/4192441170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961010e1c0 con 0x561b6a2aa540 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f960effd700 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aaea0 0x561b6a2b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f960ffff700 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2ab860 0x561b6a2b9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f960f7fe700 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aa540 0x561b6a2b6700 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9600004360 tx=0x7f960000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36de3b39148c050e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f95eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f95eeffd700 1 -- 172.21.15.33:0/4192441170 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600033230 con 0x561b6a2aa540 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f95eeffd700 1 -- 172.21.15.33:0/4192441170 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9600003c40 con 0x561b6a2aa540 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f962203ab80 1 -- 172.21.15.33:0/4192441170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f960409a4d0 con 0x561b6a2aa540 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f95eeffd700 1 -- 172.21.15.33:0/4192441170 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9600033390 con 0x561b6a2aa540 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f962203ab80 10 client.17640 fetch_fsmap learned FSMap version 39 2024-05-08T21:26:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.957+0000 7f962203ab80 1 -- 172.21.15.33:0/4192441170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561b6a07fcd0 con 0x561b6a2aa540 2024-05-08T21:26:44.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.959+0000 7f95eeffd700 1 -- 172.21.15.33:0/4192441170 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9600045020 con 0x561b6a2aa540 2024-05-08T21:26:44.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.959+0000 7f95eeffd700 1 -- 172.21.15.33:0/4192441170 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f960000fbd0 con 0x561b6a2aa540 2024-05-08T21:26:44.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.959+0000 7f962203ab80 10 client.17640 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:26:44.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.959+0000 7f962203ab80 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:26:44.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.959+0000 7f962203ab80 20 client.17640 populate_metadata read hostname 'smithi033' 2024-05-08T21:26:44.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.959+0000 7f962203ab80 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561b6a316370 0x561b6a318760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:26:44.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.959+0000 7f962203ab80 4 client.17640 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:26:44.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.959+0000 7f962203ab80 1 -- 172.21.15.33:0/4192441170 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561b6a1f2ec0 con 0x561b6a316370 2024-05-08T21:26:44.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.960+0000 7f960effd700 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561b6a316370 0x561b6a318760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.961+0000 7f960effd700 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561b6a316370 0x561b6a318760 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54025b657d74fa5 server_cookie=114cc5b0c95b0b90 in_seq=0 out_seq=0 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.961+0000 7f95eeffd700 10 client.17640 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.961+0000 7f95eeffd700 1 -- 172.21.15.33:0/4192441170 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561b6a1f2ec0 con 0x561b6a316370 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.961+0000 7f95eeffd700 10 client.17640 handle_command_reply: tid=0 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.961+0000 7f962203ab80 1 client.17640 shutdown 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.962+0000 7f962203ab80 1 -- 172.21.15.33:0/4192441170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aa540 msgr2=0x561b6a2b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.962+0000 7f962203ab80 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aa540 0x561b6a2b6700 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9600004360 tx=0x7f960000b390 comp rx=0 tx=0).stop 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.962+0000 7f962203ab80 1 -- 172.21.15.33:0/4192441170 shutdown_connections 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.962+0000 7f962203ab80 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x561b6a316370 0x561b6a318760 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.962+0000 7f962203ab80 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b6a2ab860 0x561b6a2b9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.962+0000 7f962203ab80 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b6a2aa540 0x561b6a2b6700 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.962+0000 7f962203ab80 1 --2- 172.21.15.33:0/4192441170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b6a2aaea0 0x561b6a2b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.962+0000 7f962203ab80 1 -- 172.21.15.33:0/4192441170 >> 172.21.15.33:0/4192441170 conn(0x561b6a208210 msgr2=0x561b6a2b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.962+0000 7f95eeffd700 0 client.17640 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:26:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.962+0000 7f962203ab80 1 -- 172.21.15.33:0/4192441170 shutdown_connections 2024-05-08T21:26:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.962+0000 7f962203ab80 1 -- 172.21.15.33:0/4192441170 wait complete. 2024-05-08T21:26:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.962+0000 7f962203ab80 20 client.17640 trim_cache size 0 max 16384 2024-05-08T21:26:44.965 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:26:44.965 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:26:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:26:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "1c9c4d96-a8b0-4a91-a214-3529753d11fe": { 2024-05-08T21:26:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:26:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "1c9c4d96-a8b0-4a91-a214-3529753d11fe", 2024-05-08T21:26:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:26:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:26:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:26:44.966 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:26:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.963+0000 7f9615e80700 1 -- 172.21.15.33:0/1012878348 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95f80a12f0 msgr2=0x7f95f80a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.963+0000 7f9615e80700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95f80a12f0 0x7f95f80a37b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f960000b2e0 tx=0x7f960000b610 comp rx=0 tx=0).stop 2024-05-08T21:26:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.963+0000 7f9615e80700 1 -- 172.21.15.33:0/1012878348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f961010a910 msgr2=0x7f96101a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:26:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.963+0000 7f9615e80700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f961010a910 0x7f96101a6070 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f960400e9f0 tx=0x7f960400ed00 comp rx=0 tx=0).stop 2024-05-08T21:26:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.963+0000 7f9615e80700 1 -- 172.21.15.33:0/1012878348 shutdown_connections 2024-05-08T21:26:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.963+0000 7f9615e80700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f961010d2e0 0x7f96101ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.963+0000 7f9615e80700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9610107fa0 0x7f96101a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.963+0000 7f9615e80700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f95f80a12f0 0x7f95f80a37b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:26:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.963+0000 7f9615e80700 1 --2- 172.21.15.33:0/1012878348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f961010a910 0x7f96101a6070 secure :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f960400e9f0 tx=0x7f960400ed00 comp rx=0 tx=0).stop 2024-05-08T21:26:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.963+0000 7f9615e80700 1 -- 172.21.15.33:0/1012878348 >> 172.21.15.33:0/1012878348 conn(0x7f961006dcb0 msgr2=0x7f961010ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:26:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.964+0000 7f9615e80700 1 -- 172.21.15.33:0/1012878348 shutdown_connections 2024-05-08T21:26:44.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:26:44.964+0000 7f9615e80700 1 -- 172.21.15.33:0/1012878348 wait complete. 2024-05-08T21:26:44.983 INFO:tasks.cephfs.filesystem:scrub status for tag:1c9c4d96-a8b0-4a91-a214-3529753d11fe - {'path': '/', 'tag': '1c9c4d96-a8b0-4a91-a214-3529753d11fe', 'options': 'recursive,force'} 2024-05-08T21:26:45.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:45 smithi033 ceph-mon[42528]: pgmap v523: 129 pgs: 129 active+clean; 253 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 655 op/s 2024-05-08T21:26:45.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:45 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:45.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:45 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:45.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:45 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:45.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:45 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:45.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:45 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:45.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:45 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:45.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:45 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:45.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:45 smithi071 ceph-mon[47074]: pgmap v523: 129 pgs: 129 active+clean; 253 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 655 op/s 2024-05-08T21:26:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:45 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:45 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:45 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:45 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:45 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:45 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:45.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:45 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:45 smithi082 ceph-mon[47512]: pgmap v523: 129 pgs: 129 active+clean; 253 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 655 op/s 2024-05-08T21:26:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:45 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:45 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:45 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:45 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:45 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:45 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:45 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.566 INFO:tasks.workunit.client.0.smithi033.stdout:20000 punch from 0x167a970 to 0x1681166, (0x67f6 bytes) 2024-05-08T21:26:46.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:46 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:26:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:46 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:46 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:46 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:46 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:46 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:46 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:46 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:26:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:46 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:46 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:46 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:46 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:26:46.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:46 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:46 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:46.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:46 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:46.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:46 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:46.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:46 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:46 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:46 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:26:46.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:46 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:46 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:46 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:46 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:26:46.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:46 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:46 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:46.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:46 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:46.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:46 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:46.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:46 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:46 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:46 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:26:46.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:46 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:46 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:46.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:46 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:26:47.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:47 smithi033 ceph-mon[42528]: pgmap v524: 129 pgs: 129 active+clean; 261 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.4 MiB/s wr, 577 op/s 2024-05-08T21:26:47.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:47 smithi071 ceph-mon[47074]: pgmap v524: 129 pgs: 129 active+clean; 261 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.4 MiB/s wr, 577 op/s 2024-05-08T21:26:47.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:47 smithi082 ceph-mon[47512]: pgmap v524: 129 pgs: 129 active+clean; 261 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.4 MiB/s wr, 577 op/s 2024-05-08T21:26:49.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:49 smithi033 ceph-mon[42528]: pgmap v525: 129 pgs: 129 active+clean; 261 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 510 op/s 2024-05-08T21:26:49.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:49 smithi071 ceph-mon[47074]: pgmap v525: 129 pgs: 129 active+clean; 261 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 510 op/s 2024-05-08T21:26:49.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:49 smithi082 ceph-mon[47512]: pgmap v525: 129 pgs: 129 active+clean; 261 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 510 op/s 2024-05-08T21:26:51.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:51 smithi033 ceph-mon[42528]: pgmap v526: 129 pgs: 129 active+clean; 260 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.9 MiB/s wr, 701 op/s 2024-05-08T21:26:51.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:51 smithi071 ceph-mon[47074]: pgmap v526: 129 pgs: 129 active+clean; 260 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.9 MiB/s wr, 701 op/s 2024-05-08T21:26:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:51 smithi082 ceph-mon[47512]: pgmap v526: 129 pgs: 129 active+clean; 260 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.9 MiB/s wr, 701 op/s 2024-05-08T21:26:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:53 smithi033 ceph-mon[42528]: pgmap v527: 129 pgs: 129 active+clean; 258 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 528 op/s 2024-05-08T21:26:53.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:53 smithi071 ceph-mon[47074]: pgmap v527: 129 pgs: 129 active+clean; 258 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 528 op/s 2024-05-08T21:26:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:53 smithi082 ceph-mon[47512]: pgmap v527: 129 pgs: 129 active+clean; 258 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.2 MiB/s wr, 528 op/s 2024-05-08T21:26:55.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:55 smithi033 ceph-mon[42528]: pgmap v528: 129 pgs: 129 active+clean; 255 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.8 MiB/s wr, 655 op/s 2024-05-08T21:26:55.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:55 smithi071 ceph-mon[47074]: pgmap v528: 129 pgs: 129 active+clean; 255 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.8 MiB/s wr, 655 op/s 2024-05-08T21:26:55.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:55 smithi082 ceph-mon[47512]: pgmap v528: 129 pgs: 129 active+clean; 255 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.8 MiB/s wr, 655 op/s 2024-05-08T21:26:57.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:57 smithi033 ceph-mon[42528]: pgmap v529: 129 pgs: 129 active+clean; 259 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.4 MiB/s wr, 573 op/s 2024-05-08T21:26:57.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:57 smithi071 ceph-mon[47074]: pgmap v529: 129 pgs: 129 active+clean; 259 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.4 MiB/s wr, 573 op/s 2024-05-08T21:26:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:57 smithi082 ceph-mon[47512]: pgmap v529: 129 pgs: 129 active+clean; 259 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.4 MiB/s wr, 573 op/s 2024-05-08T21:26:58.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:58 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:26:58.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:58 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:26:58.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:58 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:26:58.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:58 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:26:58.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:58 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:26:58.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:58 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:26:58.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:58 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:26:58.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:58 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:26:58.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:58 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:26:58.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:58 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:26:58.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:58 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:26:58.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:58 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:26:58.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:58 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:26:58.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:58 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:26:58.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:58 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:26:59.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:26:59 smithi033 ceph-mon[42528]: pgmap v530: 129 pgs: 129 active+clean; 255 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 503 op/s 2024-05-08T21:26:59.975 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:26:59 smithi082 ceph-mon[47512]: pgmap v530: 129 pgs: 129 active+clean; 255 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 503 op/s 2024-05-08T21:26:59.976 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:26:59 smithi071 ceph-mon[47074]: pgmap v530: 129 pgs: 129 active+clean; 255 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 503 op/s 2024-05-08T21:27:01.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:01 smithi033 ceph-mon[42528]: pgmap v531: 129 pgs: 129 active+clean; 269 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.0 MiB/s wr, 694 op/s 2024-05-08T21:27:01.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:01 smithi071 ceph-mon[47074]: pgmap v531: 129 pgs: 129 active+clean; 269 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.0 MiB/s wr, 694 op/s 2024-05-08T21:27:01.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:01 smithi082 ceph-mon[47512]: pgmap v531: 129 pgs: 129 active+clean; 269 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.0 MiB/s wr, 694 op/s 2024-05-08T21:27:03.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:03 smithi033 ceph-mon[42528]: pgmap v532: 129 pgs: 129 active+clean; 273 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.4 MiB/s wr, 522 op/s 2024-05-08T21:27:03.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:03 smithi071 ceph-mon[47074]: pgmap v532: 129 pgs: 129 active+clean; 273 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.4 MiB/s wr, 522 op/s 2024-05-08T21:27:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:03 smithi082 ceph-mon[47512]: pgmap v532: 129 pgs: 129 active+clean; 273 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.4 MiB/s wr, 522 op/s 2024-05-08T21:27:05.710 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:05 smithi033 ceph-mon[42528]: pgmap v533: 129 pgs: 129 active+clean; 276 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.1 MiB/s wr, 640 op/s 2024-05-08T21:27:05.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:05 smithi071 ceph-mon[47074]: pgmap v533: 129 pgs: 129 active+clean; 276 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.1 MiB/s wr, 640 op/s 2024-05-08T21:27:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:05 smithi082 ceph-mon[47512]: pgmap v533: 129 pgs: 129 active+clean; 276 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.1 MiB/s wr, 640 op/s 2024-05-08T21:27:07.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:07 smithi033 ceph-mon[42528]: pgmap v534: 129 pgs: 129 active+clean; 275 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.5 MiB/s wr, 553 op/s 2024-05-08T21:27:07.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:07 smithi071 ceph-mon[47074]: pgmap v534: 129 pgs: 129 active+clean; 275 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.5 MiB/s wr, 553 op/s 2024-05-08T21:27:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:07 smithi082 ceph-mon[47512]: pgmap v534: 129 pgs: 129 active+clean; 275 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.5 MiB/s wr, 553 op/s 2024-05-08T21:27:09.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:09 smithi033 ceph-mon[42528]: pgmap v535: 129 pgs: 129 active+clean; 278 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.4 MiB/s wr, 496 op/s 2024-05-08T21:27:09.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:09 smithi071 ceph-mon[47074]: pgmap v535: 129 pgs: 129 active+clean; 278 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.4 MiB/s wr, 496 op/s 2024-05-08T21:27:09.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:09 smithi082 ceph-mon[47512]: pgmap v535: 129 pgs: 129 active+clean; 278 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.4 MiB/s wr, 496 op/s 2024-05-08T21:27:11.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:11 smithi033 ceph-mon[42528]: pgmap v536: 129 pgs: 129 active+clean; 259 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.1 MiB/s wr, 682 op/s 2024-05-08T21:27:11.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:11 smithi071 ceph-mon[47074]: pgmap v536: 129 pgs: 129 active+clean; 259 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.1 MiB/s wr, 682 op/s 2024-05-08T21:27:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:11 smithi082 ceph-mon[47512]: pgmap v536: 129 pgs: 129 active+clean; 259 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.1 MiB/s wr, 682 op/s 2024-05-08T21:27:13.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:13 smithi033 ceph-mon[42528]: pgmap v537: 129 pgs: 129 active+clean; 253 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.2 MiB/s wr, 508 op/s 2024-05-08T21:27:13.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:13 smithi071 ceph-mon[47074]: pgmap v537: 129 pgs: 129 active+clean; 253 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.2 MiB/s wr, 508 op/s 2024-05-08T21:27:13.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:13 smithi082 ceph-mon[47512]: pgmap v537: 129 pgs: 129 active+clean; 253 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.2 MiB/s wr, 508 op/s 2024-05-08T21:27:14.984 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:27:15.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.171+0000 7f22a8f1b700 1 -- 172.21.15.33:0/525179232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a40fff40 msgr2=0x7f22a41003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.171+0000 7f22a8f1b700 1 --2- 172.21.15.33:0/525179232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a40fff40 0x7f22a41003a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2298009990 tx=0x7f2298009ca0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.171+0000 7f22a8f1b700 1 -- 172.21.15.33:0/525179232 shutdown_connections 2024-05-08T21:27:15.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.171+0000 7f22a8f1b700 1 --2- 172.21.15.33:0/525179232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22a41008e0 0x7f22a41081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.171+0000 7f22a8f1b700 1 --2- 172.21.15.33:0/525179232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a4101340 0x7f22a4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.171+0000 7f22a8f1b700 1 --2- 172.21.15.33:0/525179232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a40fff40 0x7f22a41003a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.171+0000 7f22a8f1b700 1 -- 172.21.15.33:0/525179232 >> 172.21.15.33:0/525179232 conn(0x7f22a40faa50 msgr2=0x7f22a40fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:15.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.171+0000 7f22a8f1b700 1 -- 172.21.15.33:0/525179232 shutdown_connections 2024-05-08T21:27:15.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.171+0000 7f22a8f1b700 1 -- 172.21.15.33:0/525179232 wait complete. 2024-05-08T21:27:15.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.172+0000 7f22a8f1b700 1 Processor -- start 2024-05-08T21:27:15.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.172+0000 7f22a8f1b700 1 -- start start 2024-05-08T21:27:15.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.172+0000 7f22a8f1b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a40fff40 0x7f22a40ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.172+0000 7f22a8f1b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22a41008e0 0x7f22a419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.172+0000 7f22a8f1b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a4101340 0x7f22a419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.172+0000 7f22a8f1b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419e8d0 con 0x7f22a41008e0 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.172+0000 7f22a8f1b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419ea40 con 0x7f22a4101340 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.172+0000 7f22a8f1b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a419ec00 con 0x7f22a40fff40 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.173+0000 7f22a1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22a41008e0 0x7f22a419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.173+0000 7f22a1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22a41008e0 0x7f22a419bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40062/0 (socket says 172.21.15.33:40062) 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.173+0000 7f22a1d9b700 1 -- 172.21.15.33:0/1965051718 learned_addr learned my addr 172.21.15.33:0/1965051718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.173+0000 7f22a259c700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a40fff40 0x7f22a40ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.173+0000 7f22a2d9d700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a4101340 0x7f22a419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.173+0000 7f22a259c700 1 -- 172.21.15.33:0/1965051718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a4101340 msgr2=0x7f22a419e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.173+0000 7f22a259c700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a4101340 0x7f22a419e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.173+0000 7f22a259c700 1 -- 172.21.15.33:0/1965051718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22a41008e0 msgr2=0x7f22a419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.173+0000 7f22a259c700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22a41008e0 0x7f22a419bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.173+0000 7f22a259c700 1 -- 172.21.15.33:0/1965051718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22980095f0 con 0x7f22a40fff40 2024-05-08T21:27:15.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.173+0000 7f22a259c700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a40fff40 0x7f22a40ff950 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f22a4067fc0 tx=0x7f229400b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:15.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.173+0000 7f22937fe700 1 -- 172.21.15.33:0/1965051718 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294011830 con 0x7f22a40fff40 2024-05-08T21:27:15.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.174+0000 7f22937fe700 1 -- 172.21.15.33:0/1965051718 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f229400be30 con 0x7f22a40fff40 2024-05-08T21:27:15.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.174+0000 7f22a8f1b700 1 -- 172.21.15.33:0/1965051718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a419eee0 con 0x7f22a40fff40 2024-05-08T21:27:15.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.174+0000 7f22937fe700 1 -- 172.21.15.33:0/1965051718 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294010cd0 con 0x7f22a40fff40 2024-05-08T21:27:15.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.174+0000 7f22a8f1b700 1 -- 172.21.15.33:0/1965051718 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a419f3e0 con 0x7f22a40fff40 2024-05-08T21:27:15.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.176+0000 7f22937fe700 1 -- 172.21.15.33:0/1965051718 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f229401a400 con 0x7f22a40fff40 2024-05-08T21:27:15.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.177+0000 7f22937fe700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f228c0a11a0 0x7f228c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.177+0000 7f22937fe700 1 -- 172.21.15.33:0/1965051718 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f22940cc500 con 0x7f22a40fff40 2024-05-08T21:27:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.177+0000 7f22a1d9b700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f228c0a11a0 0x7f228c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.177+0000 7f22a1d9b700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f228c0a11a0 0x7f228c0a3660 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f22a40fff10 tx=0x7f229800b6b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.178+0000 7f22b50d5b80 1 Processor -- start 2024-05-08T21:27:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.178+0000 7f22b50d5b80 1 -- start start 2024-05-08T21:27:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.178+0000 7f22b50d5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 0x564977dd2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.178+0000 7f22b50d5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2ea0 0x564977dd3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.178+0000 7f22b50d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977dd3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22b50d5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564977d29150 con 0x564977dd2540 2024-05-08T21:27:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22b50d5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564977d26b10 con 0x564977dd2ea0 2024-05-08T21:27:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22b50d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564977d22400 con 0x564977dd3860 2024-05-08T21:27:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22a259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 0x564977dd2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22a259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 0x564977dd2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40064/0 (socket says 172.21.15.33:40064) 2024-05-08T21:27:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22a259c700 1 -- 172.21.15.33:0/1677761129 learned_addr learned my addr 172.21.15.33:0/1677761129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22a1d9b700 1 --2- 172.21.15.33:0/1677761129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2ea0 0x564977dd3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22a2d9d700 1 --2- 172.21.15.33:0/1677761129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977dd3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22a259c700 1 -- 172.21.15.33:0/1677761129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 msgr2=0x564977dd3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:15.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22a259c700 1 --2- 172.21.15.33:0/1677761129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977dd3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22a259c700 1 -- 172.21.15.33:0/1677761129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2ea0 msgr2=0x564977dd3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:15.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22a259c700 1 --2- 172.21.15.33:0/1677761129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2ea0 0x564977dd3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22a259c700 1 -- 172.21.15.33:0/1677761129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564977ba33b0 con 0x564977dd2540 2024-05-08T21:27:15.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f22a259c700 1 --2- 172.21.15.33:0/1677761129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 0x564977dd2960 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f22940c9bc0 tx=0x7f2294099690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b933ff4d278df3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:15.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.179+0000 7f2290ff9700 1 -- 172.21.15.33:0/1677761129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294018310 con 0x564977dd2540 2024-05-08T21:27:15.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.180+0000 7f2290ff9700 1 -- 172.21.15.33:0/1677761129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22940cb070 con 0x564977dd2540 2024-05-08T21:27:15.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.180+0000 7f2290ff9700 1 -- 172.21.15.33:0/1677761129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22940990f0 con 0x564977dd2540 2024-05-08T21:27:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.180+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1677761129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 msgr2=0x564977dd2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.180+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1677761129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 0x564977dd2960 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f22940c9bc0 tx=0x7f2294099690 comp rx=0 tx=0).stop 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.180+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1677761129 shutdown_connections 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.180+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1677761129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977dd3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.180+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1677761129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 0x564977dd2960 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.180+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1677761129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2ea0 0x564977dd3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.180+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1677761129 >> 172.21.15.33:0/1677761129 conn(0x564977d30210 msgr2=0x564977dccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.181+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1677761129 shutdown_connections 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.181+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1677761129 wait complete. 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.181+0000 7f22b50d5b80 1 -- start start 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.181+0000 7f22b50d5b80 1 Processor -- start 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.181+0000 7f22b50d5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 0x564977de46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.181+0000 7f22b50d5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2ea0 0x564977de4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.181+0000 7f22b50d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977debc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.181+0000 7f22b50d5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564977cbe560 con 0x564977dd2ea0 2024-05-08T21:27:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.181+0000 7f22b50d5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564977cbe200 con 0x564977dd2540 2024-05-08T21:27:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.181+0000 7f22b50d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564977ba56a0 con 0x564977dd3860 2024-05-08T21:27:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22b50d5b80 10 client.0 _ll_register_callbacks cb 0x564977cc7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2ea0 0x564977de4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977debc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 0x564977de46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2ea0 0x564977de4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40074/0 (socket says 172.21.15.33:40074) 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977debc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:56330/0 (socket says 172.21.15.33:56330) 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a1d9b700 1 -- 172.21.15.33:0/1955747243 learned_addr learned my addr 172.21.15.33:0/1955747243 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a259c700 1 -- 172.21.15.33:0/1955747243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 msgr2=0x564977debc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a259c700 1 --2- 172.21.15.33:0/1955747243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977debc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a259c700 1 -- 172.21.15.33:0/1955747243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2ea0 msgr2=0x564977de4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a259c700 1 --2- 172.21.15.33:0/1955747243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2ea0 0x564977de4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a259c700 1 -- 172.21.15.33:0/1955747243 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a419f3e0 con 0x564977dd2540 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a2d9d700 1 --2- 172.21.15.33:0/1955747243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977debc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22a259c700 1 --2- 172.21.15.33:0/1955747243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 0x564977de46b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f22940969a0 tx=0x7f229400bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0dadc5f9e0c5b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f2289ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:27:15.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f2289ffb700 1 -- 172.21.15.33:0/1955747243 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294099cd0 con 0x564977dd2540 2024-05-08T21:27:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f2289ffb700 1 -- 172.21.15.33:0/1955747243 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f22940990f0 con 0x564977dd2540 2024-05-08T21:27:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.182+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1955747243 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564977c7b270 con 0x564977dd2540 2024-05-08T21:27:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.183+0000 7f2289ffb700 1 -- 172.21.15.33:0/1955747243 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22940850c0 con 0x564977dd2540 2024-05-08T21:27:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.183+0000 7f22b50d5b80 10 client.27539 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.183+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1955747243 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564977b4fb60 con 0x564977dd2540 2024-05-08T21:27:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.183+0000 7f2289ffb700 1 -- 172.21.15.33:0/1955747243 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564977c7b270 con 0x564977dd2540 2024-05-08T21:27:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.183+0000 7f2289ffb700 1 -- 172.21.15.33:0/1955747243 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f22940a1710 con 0x564977dd2540 2024-05-08T21:27:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.183+0000 7f22b50d5b80 10 client.27539 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.183+0000 7f22b50d5b80 10 client.27539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.183+0000 7f22b50d5b80 20 client.27539 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:15.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.183+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1955747243 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564977df0140 0x564977df2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.183+0000 7f22b50d5b80 4 client.27539 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.183+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1955747243 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564977da1810 con 0x564977df0140 2024-05-08T21:27:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.183+0000 7f22a1d9b700 1 --2- 172.21.15.33:0/1955747243 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564977df0140 0x564977df2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.184+0000 7f2289ffb700 10 client.27539 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.184+0000 7f22a1d9b700 1 --2- 172.21.15.33:0/1955747243 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564977df0140 0x564977df2530 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363d3440ff722c9 server_cookie=2ddbbcd8b328126b in_seq=0 out_seq=0 2024-05-08T21:27:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.184+0000 7f2289ffb700 1 -- 172.21.15.33:0/1955747243 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564977da1810 con 0x564977df0140 2024-05-08T21:27:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.184+0000 7f2289ffb700 10 client.27539 handle_command_reply: tid=0 2024-05-08T21:27:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.184+0000 7f22b50d5b80 1 client.27539 shutdown 2024-05-08T21:27:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.185+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1955747243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 msgr2=0x564977de46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.185+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1955747243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 0x564977de46b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f22940969a0 tx=0x7f229400bc30 comp rx=0 tx=0).stop 2024-05-08T21:27:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.185+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1955747243 shutdown_connections 2024-05-08T21:27:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.185+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1955747243 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564977df0140 0x564977df2530 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.185+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1955747243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977debc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.185+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1955747243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 0x564977de46b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.185+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1955747243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2ea0 0x564977de4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.185+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1955747243 >> 172.21.15.33:0/1955747243 conn(0x564977d30210 msgr2=0x564977dcc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.185+0000 7f2289ffb700 0 client.27539 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.185+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1955747243 shutdown_connections 2024-05-08T21:27:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.185+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1955747243 wait complete. 2024-05-08T21:27:15.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.185+0000 7f22b50d5b80 20 client.27539 trim_cache size 0 max 16384 2024-05-08T21:27:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22b50d5b80 1 Processor -- start 2024-05-08T21:27:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22b50d5b80 1 -- start start 2024-05-08T21:27:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22b50d5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 0x564977de28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22b50d5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2ea0 0x564977debbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22b50d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977de3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22b50d5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564977dc5040 con 0x564977dd2540 2024-05-08T21:27:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22b50d5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564977df7720 con 0x564977dd2ea0 2024-05-08T21:27:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22b50d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564977de2df0 con 0x564977dd3860 2024-05-08T21:27:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22a259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 0x564977de28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22a1d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2ea0 0x564977debbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22a259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 0x564977de28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40076/0 (socket says 172.21.15.33:40076) 2024-05-08T21:27:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22a259c700 1 -- 172.21.15.33:0/1584988633 learned_addr learned my addr 172.21.15.33:0/1584988633 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22a2d9d700 1 --2- 172.21.15.33:0/1584988633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977de3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22a259c700 1 -- 172.21.15.33:0/1584988633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 msgr2=0x564977de3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22a259c700 1 --2- 172.21.15.33:0/1584988633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977de3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22a259c700 1 -- 172.21.15.33:0/1584988633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2ea0 msgr2=0x564977debbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22a259c700 1 --2- 172.21.15.33:0/1584988633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2ea0 0x564977debbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22a259c700 1 -- 172.21.15.33:0/1584988633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564977ce8560 con 0x564977dd2540 2024-05-08T21:27:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.206+0000 7f22a1d9b700 1 --2- 172.21.15.33:0/1584988633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2ea0 0x564977debbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.207+0000 7f22a259c700 1 --2- 172.21.15.33:0/1584988633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 0x564977de28b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f22940048a0 tx=0x7f229400bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c4f70c555b8991 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.207+0000 7f22917fa700 1 -- 172.21.15.33:0/1584988633 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294099cd0 con 0x564977dd2540 2024-05-08T21:27:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.207+0000 7f22917fa700 1 -- 172.21.15.33:0/1584988633 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2294018310 con 0x564977dd2540 2024-05-08T21:27:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.207+0000 7f22917fa700 1 -- 172.21.15.33:0/1584988633 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22940a15c0 con 0x564977dd2540 2024-05-08T21:27:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1584988633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 msgr2=0x564977de28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1584988633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 0x564977de28b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f22940048a0 tx=0x7f229400bca0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1584988633 shutdown_connections 2024-05-08T21:27:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1584988633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd3860 0x564977de3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1584988633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd2540 0x564977de28b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/1584988633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2ea0 0x564977debbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1584988633 >> 172.21.15.33:0/1584988633 conn(0x564977d30210 msgr2=0x564977dd3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1584988633 shutdown_connections 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 -- 172.21.15.33:0/1584988633 wait complete. 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 -- start start 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 Processor -- start 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 0x564977de9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd2ea0 0x564977de9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd3860 0x564977dea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564977dea960 con 0x564977dd3860 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564977deaad0 con 0x564977dd2540 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564977deac90 con 0x564977dd2ea0 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22b50d5b80 10 client.0 _ll_register_callbacks cb 0x564977cc7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22a2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd3860 0x564977dea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.208+0000 7f22a259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 0x564977de9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd2ea0 0x564977de9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 0x564977de9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41236/0 (socket says 172.21.15.33:41236) 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd3860 0x564977dea420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40078/0 (socket says 172.21.15.33:40078) 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a259c700 1 -- 172.21.15.33:0/2581747001 learned_addr learned my addr 172.21.15.33:0/2581747001 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd2ea0 0x564977de9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:56340/0 (socket says 172.21.15.33:56340) 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a2d9d700 1 -- 172.21.15.33:0/2581747001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd2ea0 msgr2=0x564977de9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a2d9d700 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd2ea0 0x564977de9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a2d9d700 1 -- 172.21.15.33:0/2581747001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 msgr2=0x564977de9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a2d9d700 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 0x564977de9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a2d9d700 1 -- 172.21.15.33:0/2581747001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a4108af0 con 0x564977dd3860 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a1d9b700 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd2ea0 0x564977de9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a259c700 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 0x564977de9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22a2d9d700 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd3860 0x564977dea420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x564977dce1f0 tx=0x7f229c018a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c640ab5c6664f870 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f2289ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f2289ffb700 1 -- 172.21.15.33:0/2581747001 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c01d070 con 0x564977dd3860 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f2289ffb700 1 -- 172.21.15.33:0/2581747001 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f229c004bb0 con 0x564977dd3860 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.209+0000 7f22b50d5b80 1 -- 172.21.15.33:0/2581747001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22940cb070 con 0x564977dd3860 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.210+0000 7f2289ffb700 1 -- 172.21.15.33:0/2581747001 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f229c00e040 con 0x564977dd3860 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.210+0000 7f22b50d5b80 10 client.17658 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.210+0000 7f22b50d5b80 1 -- 172.21.15.33:0/2581747001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564977b4fb60 con 0x564977dd3860 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.210+0000 7f2289ffb700 1 -- 172.21.15.33:0/2581747001 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22940cb070 con 0x564977dd3860 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.210+0000 7f2289ffb700 1 -- 172.21.15.33:0/2581747001 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f229c016400 con 0x564977dd3860 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.210+0000 7f22b50d5b80 10 client.17658 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.210+0000 7f22b50d5b80 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.210+0000 7f22b50d5b80 20 client.17658 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.210+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564977e46bc0 0x564977e48fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.210+0000 7f22b50d5b80 4 client.17658 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.210+0000 7f22b50d5b80 1 -- 172.21.15.33:0/2581747001 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564977d1aec0 con 0x564977e46bc0 2024-05-08T21:27:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.210+0000 7f22a259c700 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564977e46bc0 0x564977e48fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f2289ffb700 10 client.17658 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f22a259c700 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564977e46bc0 0x564977e48fe0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c5978b75aa42075 server_cookie=f446520e9854f064 in_seq=0 out_seq=0 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f2289ffb700 1 -- 172.21.15.33:0/2581747001 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564977d1aec0 con 0x564977e46bc0 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f2289ffb700 10 client.17658 handle_command_reply: tid=0 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f22b50d5b80 1 client.17658 shutdown 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f22b50d5b80 1 -- 172.21.15.33:0/2581747001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd3860 msgr2=0x564977dea420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd3860 0x564977dea420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x564977dce1f0 tx=0x7f229c018a50 comp rx=0 tx=0).stop 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f22b50d5b80 1 -- 172.21.15.33:0/2581747001 shutdown_connections 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564977e46bc0 0x564977e48fe0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564977dd3860 0x564977dea420 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564977dd2540 0x564977de9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f22b50d5b80 1 --2- 172.21.15.33:0/2581747001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564977dd2ea0 0x564977de9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f2289ffb700 0 client.17658 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f22b50d5b80 1 -- 172.21.15.33:0/2581747001 >> 172.21.15.33:0/2581747001 conn(0x564977d30210 msgr2=0x564977de3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f22b50d5b80 1 -- 172.21.15.33:0/2581747001 shutdown_connections 2024-05-08T21:27:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.211+0000 7f22b50d5b80 1 -- 172.21.15.33:0/2581747001 wait complete. 2024-05-08T21:27:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.212+0000 7f22b50d5b80 20 client.17658 trim_cache size 0 max 16384 2024-05-08T21:27:15.222 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:27:15.222 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:27:15.222 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:27:15.223 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:27:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.219+0000 7f22a8f1b700 1 -- 172.21.15.33:0/1965051718 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f228c0a11a0 msgr2=0x7f228c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.219+0000 7f22a8f1b700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f228c0a11a0 0x7f228c0a3660 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f22a40fff10 tx=0x7f229800b6b0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.219+0000 7f22a8f1b700 1 -- 172.21.15.33:0/1965051718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a40fff40 msgr2=0x7f22a40ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.219+0000 7f22a8f1b700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a40fff40 0x7f22a40ff950 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f22a4067fc0 tx=0x7f229400b730 comp rx=0 tx=0).stop 2024-05-08T21:27:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.219+0000 7f22a8f1b700 1 -- 172.21.15.33:0/1965051718 shutdown_connections 2024-05-08T21:27:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.219+0000 7f22a8f1b700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f228c0a11a0 0x7f228c0a3660 secure :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f22a40fff10 tx=0x7f229800b6b0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.219+0000 7f22a8f1b700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22a4101340 0x7f22a419e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.219+0000 7f22a8f1b700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a40fff40 0x7f22a40ff950 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.219+0000 7f22a8f1b700 1 --2- 172.21.15.33:0/1965051718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22a41008e0 0x7f22a419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.219+0000 7f22a8f1b700 1 -- 172.21.15.33:0/1965051718 >> 172.21.15.33:0/1965051718 conn(0x7f22a40faa50 msgr2=0x7f22a4104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.220+0000 7f22a8f1b700 1 -- 172.21.15.33:0/1965051718 shutdown_connections 2024-05-08T21:27:15.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.222+0000 7f22a8f1b700 1 -- 172.21.15.33:0/1965051718 wait complete. 2024-05-08T21:27:15.246 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:27:15.247 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:27:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.427+0000 7f91d07bb700 1 -- 172.21.15.33:0/2078605540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91c8101380 msgr2=0x7f91c81017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.427+0000 7f91d07bb700 1 --2- 172.21.15.33:0/2078605540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91c8101380 0x7f91c81017a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009990 tx=0x7f91c4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.427+0000 7f91d07bb700 1 -- 172.21.15.33:0/2078605540 shutdown_connections 2024-05-08T21:27:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.427+0000 7f91d07bb700 1 --2- 172.21.15.33:0/2078605540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91c81009c0 0x7f91c8108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.427+0000 7f91d07bb700 1 --2- 172.21.15.33:0/2078605540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91c8101380 0x7f91c81017a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.427+0000 7f91d07bb700 1 --2- 172.21.15.33:0/2078605540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c8100020 0x7f91c8100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.427+0000 7f91d07bb700 1 -- 172.21.15.33:0/2078605540 >> 172.21.15.33:0/2078605540 conn(0x7f91c80faa90 msgr2=0x7f91c80fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:15.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.427+0000 7f91d07bb700 1 -- 172.21.15.33:0/2078605540 shutdown_connections 2024-05-08T21:27:15.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.427+0000 7f91d07bb700 1 -- 172.21.15.33:0/2078605540 wait complete. 2024-05-08T21:27:15.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.428+0000 7f91d07bb700 1 Processor -- start 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91d07bb700 1 -- start start 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91d07bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91c8100020 0x7f91c80ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91d07bb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c81009c0 0x7f91c819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91d07bb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91c8101380 0x7f91c819e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91d07bb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c819e8e0 con 0x7f91c8100020 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91d07bb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c819ea50 con 0x7f91c81009c0 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91d07bb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c819ec10 con 0x7f91c8101380 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91ce557700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91c8100020 0x7f91c80ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91cdd56700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c81009c0 0x7f91c819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91ced58700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91c8101380 0x7f91c819e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91cdd56700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c81009c0 0x7f91c819bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41258/0 (socket says 172.21.15.33:41258) 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91ce557700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91c8100020 0x7f91c80ffa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40088/0 (socket says 172.21.15.33:40088) 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91ced58700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91c8101380 0x7f91c819e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:56358/0 (socket says 172.21.15.33:56358) 2024-05-08T21:27:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.429+0000 7f91ce557700 1 -- 172.21.15.33:0/503475596 learned_addr learned my addr 172.21.15.33:0/503475596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91cdd56700 1 -- 172.21.15.33:0/503475596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91c8101380 msgr2=0x7f91c819e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91cdd56700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91c8101380 0x7f91c819e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91cdd56700 1 -- 172.21.15.33:0/503475596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91c8100020 msgr2=0x7f91c80ffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91cdd56700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91c8100020 0x7f91c80ffa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91cdd56700 1 -- 172.21.15.33:0/503475596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c40095f0 con 0x7f91c81009c0 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91ce557700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91c8100020 0x7f91c80ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91ced58700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91c8101380 0x7f91c819e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91cdd56700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c81009c0 0x7f91c819bad0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f91b800be10 tx=0x7f91b800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91bf7fe700 1 -- 172.21.15.33:0/503475596 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8012c90 con 0x7f91c81009c0 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91bf7fe700 1 -- 172.21.15.33:0/503475596 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f91b8016070 con 0x7f91c81009c0 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91bf7fe700 1 -- 172.21.15.33:0/503475596 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b801b610 con 0x7f91c81009c0 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91d07bb700 1 -- 172.21.15.33:0/503475596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c8195900 con 0x7f91c81009c0 2024-05-08T21:27:15.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.430+0000 7f91d07bb700 1 -- 172.21.15.33:0/503475596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c8195dc0 con 0x7f91c81009c0 2024-05-08T21:27:15.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.431+0000 7f91d07bb700 1 -- 172.21.15.33:0/503475596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c8107360 con 0x7f91c81009c0 2024-05-08T21:27:15.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.433+0000 7f91bf7fe700 1 -- 172.21.15.33:0/503475596 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f91b801c400 con 0x7f91c81009c0 2024-05-08T21:27:15.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.434+0000 7f91bf7fe700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f91b40a11a0 0x7f91b40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:15.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.434+0000 7f91bf7fe700 1 -- 172.21.15.33:0/503475596 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f91b80d0420 con 0x7f91c81009c0 2024-05-08T21:27:15.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.434+0000 7f91ce557700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f91b40a11a0 0x7f91b40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:15.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.435+0000 7f91ce557700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f91b40a11a0 0x7f91b40a3660 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f91c4005910 tx=0x7f91c400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:15.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.438+0000 7f91bf7fe700 1 -- 172.21.15.33:0/503475596 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f91b8099830 con 0x7f91c81009c0 2024-05-08T21:27:15.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:15 smithi033 ceph-mon[42528]: pgmap v538: 129 pgs: 129 active+clean; 276 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 630 op/s 2024-05-08T21:27:15.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:15 smithi071 ceph-mon[47074]: pgmap v538: 129 pgs: 129 active+clean; 276 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 630 op/s 2024-05-08T21:27:15.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.829+0000 7f91d07bb700 1 -- 172.21.15.33:0/503475596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91c8067480 con 0x7f91c81009c0 2024-05-08T21:27:15.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.830+0000 7f91bf7fe700 1 -- 172.21.15.33:0/503475596 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10899 (secure 0 0 0) 0x7f91b809ce50 con 0x7f91c81009c0 2024-05-08T21:27:15.834 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:27:15.834 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:25:13.388928+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:27:15.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.833+0000 7f91d07bb700 1 -- 172.21.15.33:0/503475596 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f91b40a11a0 msgr2=0x7f91b40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.833+0000 7f91d07bb700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f91b40a11a0 0x7f91b40a3660 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f91c4005910 tx=0x7f91c400b610 comp rx=0 tx=0).stop 2024-05-08T21:27:15.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.833+0000 7f91d07bb700 1 -- 172.21.15.33:0/503475596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c81009c0 msgr2=0x7f91c819bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:15.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.833+0000 7f91d07bb700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c81009c0 0x7f91c819bad0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f91b800be10 tx=0x7f91b800dc30 comp rx=0 tx=0).stop 2024-05-08T21:27:15.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.833+0000 7f91d07bb700 1 -- 172.21.15.33:0/503475596 shutdown_connections 2024-05-08T21:27:15.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.833+0000 7f91d07bb700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91c8101380 0x7f91c819e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.833+0000 7f91d07bb700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91c8100020 0x7f91c80ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.833+0000 7f91d07bb700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f91b40a11a0 0x7f91b40a3660 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.833+0000 7f91d07bb700 1 --2- 172.21.15.33:0/503475596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91c81009c0 0x7f91c819bad0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:15.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.833+0000 7f91d07bb700 1 -- 172.21.15.33:0/503475596 >> 172.21.15.33:0/503475596 conn(0x7f91c80faa90 msgr2=0x7f91c81044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:15.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.833+0000 7f91d07bb700 1 -- 172.21.15.33:0/503475596 shutdown_connections 2024-05-08T21:27:15.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:15.833+0000 7f91d07bb700 1 -- 172.21.15.33:0/503475596 wait complete. 2024-05-08T21:27:15.836 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 39 2024-05-08T21:27:15.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:15 smithi082 ceph-mon[47512]: pgmap v538: 129 pgs: 129 active+clean; 276 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 630 op/s 2024-05-08T21:27:15.853 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:27:16.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.047+0000 7f34c26a6700 1 -- 172.21.15.33:0/1421093274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34bc102040 msgr2=0x7f34bc1044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.047+0000 7f34c26a6700 1 --2- 172.21.15.33:0/1421093274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34bc102040 0x7f34bc1044d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f34b00098f0 tx=0x7f34b0009c00 comp rx=0 tx=0).stop 2024-05-08T21:27:16.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.049+0000 7f34c26a6700 1 -- 172.21.15.33:0/1421093274 shutdown_connections 2024-05-08T21:27:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.049+0000 7f34c26a6700 1 --2- 172.21.15.33:0/1421093274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34bc104a10 0x7f34bc104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.049+0000 7f34c26a6700 1 --2- 172.21.15.33:0/1421093274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc0ff6d0 0x7f34bc101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.049+0000 7f34c26a6700 1 --2- 172.21.15.33:0/1421093274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34bc102040 0x7f34bc1044d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.049+0000 7f34c26a6700 1 -- 172.21.15.33:0/1421093274 >> 172.21.15.33:0/1421093274 conn(0x7f34bc0fb290 msgr2=0x7f34bc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.049+0000 7f34c26a6700 1 -- 172.21.15.33:0/1421093274 shutdown_connections 2024-05-08T21:27:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.050+0000 7f34c26a6700 1 -- 172.21.15.33:0/1421093274 wait complete. 2024-05-08T21:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.051+0000 7f34c26a6700 1 Processor -- start 2024-05-08T21:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.051+0000 7f34c26a6700 1 -- start start 2024-05-08T21:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.051+0000 7f34c26a6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34bc0ff6d0 0x7f34bc070be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.051+0000 7f34c26a6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34bc102040 0x7f34bc071120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.051+0000 7f34c26a6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc104a10 0x7f34bc072110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.051+0000 7f34c26a6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc0726e0 con 0x7f34bc104a10 2024-05-08T21:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.051+0000 7f34c26a6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc072850 con 0x7f34bc102040 2024-05-08T21:27:16.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.051+0000 7f34c26a6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc072a10 con 0x7f34bc0ff6d0 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.051+0000 7f34bbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34bc0ff6d0 0x7f34bc070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.051+0000 7f34bbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34bc0ff6d0 0x7f34bc070be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:56374/0 (socket says 172.21.15.33:56374) 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.051+0000 7f34bbfff700 1 -- 172.21.15.33:0/3562147082 learned_addr learned my addr 172.21.15.33:0/3562147082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.052+0000 7f34c0c43700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc104a10 0x7f34bc072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.052+0000 7f34bb7fe700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34bc102040 0x7f34bc071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.052+0000 7f34bbfff700 1 -- 172.21.15.33:0/3562147082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34bc102040 msgr2=0x7f34bc071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.052+0000 7f34bbfff700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34bc102040 0x7f34bc071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.052+0000 7f34bbfff700 1 -- 172.21.15.33:0/3562147082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc104a10 msgr2=0x7f34bc072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.052+0000 7f34bbfff700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc104a10 0x7f34bc072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.052+0000 7f34bbfff700 1 -- 172.21.15.33:0/3562147082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b0009520 con 0x7f34bc0ff6d0 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.052+0000 7f34bbfff700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34bc0ff6d0 0x7f34bc070be0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f34b400ba50 tx=0x7f34b400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.052+0000 7f34b97fa700 1 -- 172.21.15.33:0/3562147082 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b400c930 con 0x7f34bc0ff6d0 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.052+0000 7f34b97fa700 1 -- 172.21.15.33:0/3562147082 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34b4014440 con 0x7f34bc0ff6d0 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.052+0000 7f34c26a6700 1 -- 172.21.15.33:0/3562147082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34bc072c90 con 0x7f34bc0ff6d0 2024-05-08T21:27:16.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.052+0000 7f34b97fa700 1 -- 172.21.15.33:0/3562147082 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b40075d0 con 0x7f34bc0ff6d0 2024-05-08T21:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.053+0000 7f34c26a6700 1 -- 172.21.15.33:0/3562147082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34bc1ae390 con 0x7f34bc0ff6d0 2024-05-08T21:27:16.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.055+0000 7f34b97fa700 1 -- 172.21.15.33:0/3562147082 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f34b4007730 con 0x7f34bc0ff6d0 2024-05-08T21:27:16.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.056+0000 7f34b97fa700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34a40a11f0 0x7f34a40a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.056+0000 7f34b97fa700 1 -- 172.21.15.33:0/3562147082 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f34b4006e40 con 0x7f34bc0ff6d0 2024-05-08T21:27:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.056+0000 7f34bb7fe700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34a40a11f0 0x7f34a40a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.057+0000 7f34bb7fe700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34a40a11f0 0x7f34a40a36b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f34b000fb90 tx=0x7f34b0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.057+0000 7f34ce860b80 1 Processor -- start 2024-05-08T21:27:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.057+0000 7f34ce860b80 1 -- start start 2024-05-08T21:27:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.057+0000 7f34ce860b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978c540 0x563c1978c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.057+0000 7f34ce860b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c1978d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.057+0000 7f34ce860b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978d860 0x563c1978dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.057+0000 7f34ce860b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c196e3150 con 0x563c1978c540 2024-05-08T21:27:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.057+0000 7f34ce860b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c196e0b10 con 0x563c1978cea0 2024-05-08T21:27:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.057+0000 7f34ce860b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c196dc400 con 0x563c1978d860 2024-05-08T21:27:16.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.057+0000 7f34bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978c540 0x563c1978c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.057+0000 7f34bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978c540 0x563c1978c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40100/0 (socket says 172.21.15.33:40100) 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.057+0000 7f34bbfff700 1 -- 172.21.15.33:0/844362795 learned_addr learned my addr 172.21.15.33:0/844362795 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.058+0000 7f34bb7fe700 1 --2- 172.21.15.33:0/844362795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c1978d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.058+0000 7f34bbfff700 1 -- 172.21.15.33:0/844362795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978d860 msgr2=0x563c1978dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.058+0000 7f34bbfff700 1 --2- 172.21.15.33:0/844362795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978d860 0x563c1978dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.058+0000 7f34bbfff700 1 -- 172.21.15.33:0/844362795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 msgr2=0x563c1978d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.058+0000 7f34bbfff700 1 --2- 172.21.15.33:0/844362795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c1978d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.058+0000 7f34bbfff700 1 -- 172.21.15.33:0/844362795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c1955d3b0 con 0x563c1978c540 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.058+0000 7f34bb7fe700 1 --2- 172.21.15.33:0/844362795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c1978d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.058+0000 7f34bbfff700 1 --2- 172.21.15.33:0/844362795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978c540 0x563c1978c960 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f34b40a14e0 tx=0x7f34b4098fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a824cea4f064e45c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.058+0000 7f34a27fc700 1 -- 172.21.15.33:0/844362795 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b40b9070 con 0x563c1978c540 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.058+0000 7f34a27fc700 1 -- 172.21.15.33:0/844362795 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34b409d3c0 con 0x563c1978c540 2024-05-08T21:27:16.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.058+0000 7f34a27fc700 1 -- 172.21.15.33:0/844362795 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b40979c0 con 0x563c1978c540 2024-05-08T21:27:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.059+0000 7f34ce860b80 1 -- 172.21.15.33:0/844362795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978c540 msgr2=0x563c1978c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.059+0000 7f34ce860b80 1 --2- 172.21.15.33:0/844362795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978c540 0x563c1978c960 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f34b40a14e0 tx=0x7f34b4098fc0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.059+0000 7f34ce860b80 1 -- 172.21.15.33:0/844362795 shutdown_connections 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.059+0000 7f34ce860b80 1 --2- 172.21.15.33:0/844362795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978d860 0x563c1978dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.059+0000 7f34ce860b80 1 --2- 172.21.15.33:0/844362795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978c540 0x563c1978c960 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.059+0000 7f34ce860b80 1 --2- 172.21.15.33:0/844362795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c1978d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.059+0000 7f34ce860b80 1 -- 172.21.15.33:0/844362795 >> 172.21.15.33:0/844362795 conn(0x563c196ea210 msgr2=0x563c19786de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.059+0000 7f34ce860b80 1 -- 172.21.15.33:0/844362795 shutdown_connections 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.059+0000 7f34ce860b80 1 -- 172.21.15.33:0/844362795 wait complete. 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.060+0000 7f34ce860b80 1 -- start start 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.060+0000 7f34ce860b80 1 Processor -- start 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.060+0000 7f34ce860b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978c540 0x563c19795e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.060+0000 7f34ce860b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978cea0 0x563c19796380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.060+0000 7f34ce860b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.060+0000 7f34ce860b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c19678560 con 0x563c1978d860 2024-05-08T21:27:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.060+0000 7f34ce860b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c19678200 con 0x563c1978c540 2024-05-08T21:27:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.060+0000 7f34ce860b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1955f6a0 con 0x563c1978cea0 2024-05-08T21:27:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.060+0000 7f34ce860b80 10 client.0 _ll_register_callbacks cb 0x563c19681b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34c0c43700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34bb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978cea0 0x563c19796380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34c0c43700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40108/0 (socket says 172.21.15.33:40108) 2024-05-08T21:27:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34c0c43700 1 -- 172.21.15.33:0/497876603 learned_addr learned my addr 172.21.15.33:0/497876603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34bbfff700 1 --2- 172.21.15.33:0/497876603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978c540 0x563c19795e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34bb7fe700 1 -- 172.21.15.33:0/497876603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978c540 msgr2=0x563c19795e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34bb7fe700 1 --2- 172.21.15.33:0/497876603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978c540 0x563c19795e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34bb7fe700 1 -- 172.21.15.33:0/497876603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 msgr2=0x563c1979d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34bb7fe700 1 --2- 172.21.15.33:0/497876603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34bb7fe700 1 -- 172.21.15.33:0/497876603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34bc1059c0 con 0x563c1978cea0 2024-05-08T21:27:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34bb7fe700 1 --2- 172.21.15.33:0/497876603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978cea0 0x563c19796380 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f34b000b2e0 tx=0x7f34b001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e945d4d82a1dcfda server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f349b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f349b7fe700 1 -- 172.21.15.33:0/497876603 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b002c040 con 0x563c1978cea0 2024-05-08T21:27:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f349b7fe700 1 -- 172.21.15.33:0/497876603 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f34b0025730 con 0x563c1978cea0 2024-05-08T21:27:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34ce860b80 1 -- 172.21.15.33:0/497876603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c19635270 con 0x563c1978cea0 2024-05-08T21:27:16.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f349b7fe700 1 -- 172.21.15.33:0/497876603 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34b00258f0 con 0x563c1978cea0 2024-05-08T21:27:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34ce860b80 10 client.27574 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.061+0000 7f34ce860b80 1 -- 172.21.15.33:0/497876603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f34bc1ae390 con 0x563c1978cea0 2024-05-08T21:27:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.062+0000 7f349b7fe700 1 -- 172.21.15.33:0/497876603 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563c19635270 con 0x563c1978cea0 2024-05-08T21:27:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.062+0000 7f349b7fe700 1 -- 172.21.15.33:0/497876603 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f34b0031830 con 0x563c1978cea0 2024-05-08T21:27:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.062+0000 7f34ce860b80 10 client.27574 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.062+0000 7f34ce860b80 10 client.27574 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:27:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.062+0000 7f34ce860b80 20 client.27574 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.062+0000 7f34ce860b80 1 --2- 172.21.15.33:0/497876603 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563c197a1cc0 0x563c197a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.062+0000 7f34ce860b80 4 client.27574 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.062+0000 7f34ce860b80 1 -- 172.21.15.33:0/497876603 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c1975b810 con 0x563c197a1cc0 2024-05-08T21:27:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.062+0000 7f34bbfff700 1 --2- 172.21.15.33:0/497876603 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563c197a1cc0 0x563c197a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.063+0000 7f349b7fe700 10 client.27574 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:27:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.063+0000 7f34bbfff700 1 --2- 172.21.15.33:0/497876603 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563c197a1cc0 0x563c197a40e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1be89bed99083b14 server_cookie=5e9680f7db81974f in_seq=0 out_seq=0 2024-05-08T21:27:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.063+0000 7f349b7fe700 1 -- 172.21.15.33:0/497876603 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c1975b810 con 0x563c197a1cc0 2024-05-08T21:27:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.063+0000 7f349b7fe700 10 client.27574 handle_command_reply: tid=0 2024-05-08T21:27:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.063+0000 7f34ce860b80 1 client.27574 shutdown 2024-05-08T21:27:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.064+0000 7f34ce860b80 1 -- 172.21.15.33:0/497876603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978cea0 msgr2=0x563c19796380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.064+0000 7f34ce860b80 1 --2- 172.21.15.33:0/497876603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978cea0 0x563c19796380 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f34b000b2e0 tx=0x7f34b001f000 comp rx=0 tx=0).stop 2024-05-08T21:27:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.064+0000 7f34ce860b80 1 -- 172.21.15.33:0/497876603 shutdown_connections 2024-05-08T21:27:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.064+0000 7f34ce860b80 1 --2- 172.21.15.33:0/497876603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.064+0000 7f34ce860b80 1 --2- 172.21.15.33:0/497876603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978c540 0x563c19795e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.064+0000 7f34ce860b80 1 --2- 172.21.15.33:0/497876603 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563c197a1cc0 0x563c197a40e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.064+0000 7f34ce860b80 1 --2- 172.21.15.33:0/497876603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978cea0 0x563c19796380 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.064+0000 7f34ce860b80 1 -- 172.21.15.33:0/497876603 >> 172.21.15.33:0/497876603 conn(0x563c196ea210 msgr2=0x563c197865a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.064+0000 7f349b7fe700 0 client.27574 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:27:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.064+0000 7f34ce860b80 1 -- 172.21.15.33:0/497876603 shutdown_connections 2024-05-08T21:27:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.064+0000 7f34ce860b80 1 -- 172.21.15.33:0/497876603 wait complete. 2024-05-08T21:27:16.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.064+0000 7f34ce860b80 20 client.27574 trim_cache size 0 max 16384 2024-05-08T21:27:16.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.086+0000 7f34ce860b80 1 Processor -- start 2024-05-08T21:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.086+0000 7f34ce860b80 1 -- start start 2024-05-08T21:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.086+0000 7f34ce860b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978c540 0x563c1978c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.086+0000 7f34ce860b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c19795350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.086+0000 7f34ce860b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.086+0000 7f34ce860b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c197959b0 con 0x563c1978d860 2024-05-08T21:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.086+0000 7f34ce860b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c19783dc0 con 0x563c1978cea0 2024-05-08T21:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.086+0000 7f34ce860b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c19788830 con 0x563c1978c540 2024-05-08T21:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.087+0000 7f34c0c43700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.087+0000 7f34c0c43700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40118/0 (socket says 172.21.15.33:40118) 2024-05-08T21:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.087+0000 7f34c0c43700 1 -- 172.21.15.33:0/526470730 learned_addr learned my addr 172.21.15.33:0/526470730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.087+0000 7f34bb7fe700 1 --2- 172.21.15.33:0/526470730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c19795350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.087+0000 7f34c0c43700 1 -- 172.21.15.33:0/526470730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978c540 msgr2=0x563c1978c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.087+0000 7f34c0c43700 1 --2- 172.21.15.33:0/526470730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978c540 0x563c1978c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.087+0000 7f34c0c43700 1 -- 172.21.15.33:0/526470730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 msgr2=0x563c19795350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.087+0000 7f34c0c43700 1 --2- 172.21.15.33:0/526470730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c19795350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.087+0000 7f34c0c43700 1 -- 172.21.15.33:0/526470730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c19509b60 con 0x563c1978d860 2024-05-08T21:27:16.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.087+0000 7f34bb7fe700 1 --2- 172.21.15.33:0/526470730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c19795350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.088+0000 7f34c0c43700 1 --2- 172.21.15.33:0/526470730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979d380 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0046e0 tx=0x7f34ac003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc5b26965744e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.088+0000 7f34a2ffd700 1 -- 172.21.15.33:0/526470730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac016070 con 0x563c1978d860 2024-05-08T21:27:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.088+0000 7f34a2ffd700 1 -- 172.21.15.33:0/526470730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34ac004cc0 con 0x563c1978d860 2024-05-08T21:27:16.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.088+0000 7f34a2ffd700 1 -- 172.21.15.33:0/526470730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00b670 con 0x563c1978d860 2024-05-08T21:27:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.089+0000 7f34ce860b80 1 -- 172.21.15.33:0/526470730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 msgr2=0x563c1979d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.089+0000 7f34ce860b80 1 --2- 172.21.15.33:0/526470730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979d380 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0046e0 tx=0x7f34ac003960 comp rx=0 tx=0).stop 2024-05-08T21:27:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.089+0000 7f34ce860b80 1 -- 172.21.15.33:0/526470730 shutdown_connections 2024-05-08T21:27:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.089+0000 7f34ce860b80 1 --2- 172.21.15.33:0/526470730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979d380 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.089+0000 7f34ce860b80 1 --2- 172.21.15.33:0/526470730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978c540 0x563c1978c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.089+0000 7f34ce860b80 1 --2- 172.21.15.33:0/526470730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c19795350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.089+0000 7f34ce860b80 1 -- 172.21.15.33:0/526470730 >> 172.21.15.33:0/526470730 conn(0x563c196ea210 msgr2=0x563c1978dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.089+0000 7f34ce860b80 1 -- 172.21.15.33:0/526470730 shutdown_connections 2024-05-08T21:27:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.089+0000 7f34ce860b80 1 -- 172.21.15.33:0/526470730 wait complete. 2024-05-08T21:27:16.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.089+0000 7f34ce860b80 1 -- start start 2024-05-08T21:27:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.089+0000 7f34ce860b80 1 Processor -- start 2024-05-08T21:27:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34ce860b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978c540 0x563c19799510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34ce860b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c19799a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34ce860b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34ce860b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1979cb70 con 0x563c1978d860 2024-05-08T21:27:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34ce860b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1979cce0 con 0x563c1978cea0 2024-05-08T21:27:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34ce860b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c1979ce50 con 0x563c1978c540 2024-05-08T21:27:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34ce860b80 10 client.0 _ll_register_callbacks cb 0x563c19681b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34c0c43700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34c0c43700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40122/0 (socket says 172.21.15.33:40122) 2024-05-08T21:27:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34c0c43700 1 -- 172.21.15.33:0/1696249528 learned_addr learned my addr 172.21.15.33:0/1696249528 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34bb7fe700 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c19799a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34bbfff700 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978c540 0x563c19799510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34c0c43700 1 -- 172.21.15.33:0/1696249528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978c540 msgr2=0x563c19799510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34c0c43700 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978c540 0x563c19799510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34c0c43700 1 -- 172.21.15.33:0/1696249528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 msgr2=0x563c19799a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34c0c43700 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c19799a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34c0c43700 1 -- 172.21.15.33:0/1696249528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34bc1ae390 con 0x563c1978d860 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34bb7fe700 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c19799a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f34c0c43700 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979c630 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0053a0 tx=0x7f34ac0052c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8582743e595eff0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.090+0000 7f349b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f349b7fe700 1 -- 172.21.15.33:0/1696249528 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac016030 con 0x563c1978d860 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f349b7fe700 1 -- 172.21.15.33:0/1696249528 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f34ac017070 con 0x563c1978d860 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f34ce860b80 1 -- 172.21.15.33:0/1696249528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34b409d3c0 con 0x563c1978d860 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f349b7fe700 1 -- 172.21.15.33:0/1696249528 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34ac020070 con 0x563c1978d860 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f34ce860b80 10 client.17679 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f34ce860b80 1 -- 172.21.15.33:0/1696249528 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f34ac004390 con 0x563c1978d860 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f349b7fe700 1 -- 172.21.15.33:0/1696249528 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34b409d3c0 con 0x563c1978d860 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f349b7fe700 1 -- 172.21.15.33:0/1696249528 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f34ac02f470 con 0x563c1978d860 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f34ce860b80 10 client.17679 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f34ce860b80 10 client.17679 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:27:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f34ce860b80 20 client.17679 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f34ce860b80 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563c197f9050 0x563c197fb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f34ce860b80 4 client.17679 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.091+0000 7f34ce860b80 1 -- 172.21.15.33:0/1696249528 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c196d4ec0 con 0x563c197f9050 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f34bbfff700 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563c197f9050 0x563c197fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f34bbfff700 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563c197f9050 0x563c197fb470 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1bccb31548d022e2 server_cookie=8669e7888c1a1c60 in_seq=0 out_seq=0 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f349b7fe700 10 client.17679 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f349b7fe700 1 -- 172.21.15.33:0/1696249528 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563c196d4ec0 con 0x563c197f9050 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f349b7fe700 10 client.17679 handle_command_reply: tid=0 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f34ce860b80 1 client.17679 shutdown 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f34ce860b80 1 -- 172.21.15.33:0/1696249528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 msgr2=0x563c1979c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f34ce860b80 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979c630 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0053a0 tx=0x7f34ac0052c0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f34ce860b80 1 -- 172.21.15.33:0/1696249528 shutdown_connections 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f34ce860b80 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c1978d860 0x563c1979c630 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f34ce860b80 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x563c197f9050 0x563c197fb470 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f34ce860b80 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c1978c540 0x563c19799510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f34ce860b80 1 --2- 172.21.15.33:0/1696249528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c1978cea0 0x563c19799a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f34ce860b80 1 -- 172.21.15.33:0/1696249528 >> 172.21.15.33:0/1696249528 conn(0x563c196ea210 msgr2=0x563c1978dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.092+0000 7f349b7fe700 0 client.17679 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:27:16.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.093+0000 7f34ce860b80 1 -- 172.21.15.33:0/1696249528 shutdown_connections 2024-05-08T21:27:16.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.093+0000 7f34ce860b80 1 -- 172.21.15.33:0/1696249528 wait complete. 2024-05-08T21:27:16.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.093+0000 7f34ce860b80 20 client.17679 trim_cache size 0 max 16384 2024-05-08T21:27:16.102 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:27:16.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.093+0000 7f34c26a6700 1 -- 172.21.15.33:0/3562147082 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34a40a11f0 msgr2=0x7f34a40a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.093+0000 7f34c26a6700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34a40a11f0 0x7f34a40a36b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f34b000fb90 tx=0x7f34b0019040 comp rx=0 tx=0).stop 2024-05-08T21:27:16.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.093+0000 7f34c26a6700 1 -- 172.21.15.33:0/3562147082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34bc0ff6d0 msgr2=0x7f34bc070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.093+0000 7f34c26a6700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34bc0ff6d0 0x7f34bc070be0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f34b400ba50 tx=0x7f34b400bd60 comp rx=0 tx=0).stop 2024-05-08T21:27:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.094+0000 7f34c26a6700 1 -- 172.21.15.33:0/3562147082 shutdown_connections 2024-05-08T21:27:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.094+0000 7f34c26a6700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc104a10 0x7f34bc072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.094+0000 7f34c26a6700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34bc0ff6d0 0x7f34bc070be0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.094+0000 7f34c26a6700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f34a40a11f0 0x7f34a40a36b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.094+0000 7f34c26a6700 1 --2- 172.21.15.33:0/3562147082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34bc102040 0x7f34bc071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.094+0000 7f34c26a6700 1 -- 172.21.15.33:0/3562147082 >> 172.21.15.33:0/3562147082 conn(0x7f34bc0fb290 msgr2=0x7f34bc100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.094+0000 7f34c26a6700 1 -- 172.21.15.33:0/3562147082 shutdown_connections 2024-05-08T21:27:16.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.094+0000 7f34c26a6700 1 -- 172.21.15.33:0/3562147082 wait complete. 2024-05-08T21:27:16.120 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:27:16.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.314+0000 7f535d802700 1 -- 172.21.15.33:0/527954051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535810a910 msgr2=0x7f535810cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.314+0000 7f535d802700 1 --2- 172.21.15.33:0/527954051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535810a910 0x7f535810cda0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5350009f90 tx=0x7f535000b330 comp rx=0 tx=0).stop 2024-05-08T21:27:16.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.315+0000 7f535d802700 1 -- 172.21.15.33:0/527954051 shutdown_connections 2024-05-08T21:27:16.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.315+0000 7f535d802700 1 --2- 172.21.15.33:0/527954051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f535810d2e0 0x7f535810d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.315+0000 7f535d802700 1 --2- 172.21.15.33:0/527954051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5358107fa0 0x7f535810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.315+0000 7f535d802700 1 --2- 172.21.15.33:0/527954051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535810a910 0x7f535810cda0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.315+0000 7f535d802700 1 -- 172.21.15.33:0/527954051 >> 172.21.15.33:0/527954051 conn(0x7f535806dcb0 msgr2=0x7f53580700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.315+0000 7f535d802700 1 -- 172.21.15.33:0/527954051 shutdown_connections 2024-05-08T21:27:16.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.317+0000 7f535d802700 1 -- 172.21.15.33:0/527954051 wait complete. 2024-05-08T21:27:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.318+0000 7f535d802700 1 Processor -- start 2024-05-08T21:27:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.319+0000 7f535d802700 1 -- start start 2024-05-08T21:27:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f535d802700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5358107fa0 0x7f53581a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f535d802700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535810a910 0x7f53581a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f535d802700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f535810d2e0 0x7f53581116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f535d802700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358111cb0 con 0x7f5358107fa0 2024-05-08T21:27:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f535d802700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358111e20 con 0x7f535810d2e0 2024-05-08T21:27:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f535d802700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358111fe0 con 0x7f535810a910 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f53567fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535810a910 0x7f53581a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f53567fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535810a910 0x7f53581a60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:56418/0 (socket says 172.21.15.33:56418) 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f53567fc700 1 -- 172.21.15.33:0/641705405 learned_addr learned my addr 172.21.15.33:0/641705405 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f53577fe700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f535810d2e0 0x7f53581116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f53567fc700 1 -- 172.21.15.33:0/641705405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f535810d2e0 msgr2=0x7f53581116e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f53567fc700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f535810d2e0 0x7f53581116e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f53567fc700 1 -- 172.21.15.33:0/641705405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5358107fa0 msgr2=0x7f53581a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f53567fc700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5358107fa0 0x7f53581a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f53567fc700 1 -- 172.21.15.33:0/641705405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5350009c40 con 0x7f535810a910 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.320+0000 7f53567fc700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535810a910 0x7f53581a60e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f535000bcd0 tx=0x7f535000bdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.321+0000 7f535c800700 1 -- 172.21.15.33:0/641705405 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535000e040 con 0x7f535810a910 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.321+0000 7f535c800700 1 -- 172.21.15.33:0/641705405 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5350004e40 con 0x7f535810a910 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.321+0000 7f535c800700 1 -- 172.21.15.33:0/641705405 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535001b790 con 0x7f535810a910 2024-05-08T21:27:16.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.321+0000 7f535d802700 1 -- 172.21.15.33:0/641705405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5358112200 con 0x7f535810a910 2024-05-08T21:27:16.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.321+0000 7f535d802700 1 -- 172.21.15.33:0/641705405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53581126e0 con 0x7f535810a910 2024-05-08T21:27:16.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.324+0000 7f535c800700 1 -- 172.21.15.33:0/641705405 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f535001da90 con 0x7f535810a910 2024-05-08T21:27:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.325+0000 7f535c800700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53400a1170 0x7f53400a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.325+0000 7f535c800700 1 -- 172.21.15.33:0/641705405 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f53500ce000 con 0x7f535810a910 2024-05-08T21:27:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.325+0000 7f5356ffd700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53400a1170 0x7f53400a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.325+0000 7f53699bcb80 1 Processor -- start 2024-05-08T21:27:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.325+0000 7f53699bcb80 1 -- start start 2024-05-08T21:27:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53699bcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 0x5619f56cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53699bcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56cfea0 0x5619f56d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53699bcb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56d0860 0x5619f56d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53699bcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5626150 con 0x5619f56cf540 2024-05-08T21:27:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53699bcb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f5623b10 con 0x5619f56d0860 2024-05-08T21:27:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53699bcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f561f400 con 0x5619f56cfea0 2024-05-08T21:27:16.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f5356ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 0x5619f56cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f5356ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 0x5619f56cf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40134/0 (socket says 172.21.15.33:40134) 2024-05-08T21:27:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f5356ffd700 1 -- 172.21.15.33:0/679803989 learned_addr learned my addr 172.21.15.33:0/679803989 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53567fc700 1 --2- 172.21.15.33:0/679803989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56cfea0 0x5619f56d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f5356ffd700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53400a1170 0x7f53400a3630 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f534c009770 tx=0x7f534c006cd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53577fe700 1 --2- 172.21.15.33:0/679803989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56d0860 0x5619f56d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53567fc700 1 -- 172.21.15.33:0/679803989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56d0860 msgr2=0x5619f56d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53567fc700 1 --2- 172.21.15.33:0/679803989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56d0860 0x5619f56d0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53567fc700 1 -- 172.21.15.33:0/679803989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 msgr2=0x5619f56cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53567fc700 1 --2- 172.21.15.33:0/679803989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 0x5619f56cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53567fc700 1 -- 172.21.15.33:0/679803989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619f54a03b0 con 0x5619f56cfea0 2024-05-08T21:27:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f5356ffd700 1 --2- 172.21.15.33:0/679803989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 0x5619f56cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.326+0000 7f53567fc700 1 --2- 172.21.15.33:0/679803989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56cfea0 0x5619f56d0320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f53500cb750 tx=0x7f535000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77ab3630b43a8029 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.327+0000 7f533dffb700 1 -- 172.21.15.33:0/679803989 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535009ef90 con 0x5619f56cfea0 2024-05-08T21:27:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.327+0000 7f533dffb700 1 -- 172.21.15.33:0/679803989 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53500a3550 con 0x5619f56cfea0 2024-05-08T21:27:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.327+0000 7f533dffb700 1 -- 172.21.15.33:0/679803989 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53500a3320 con 0x5619f56cfea0 2024-05-08T21:27:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.327+0000 7f53699bcb80 1 -- 172.21.15.33:0/679803989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56cfea0 msgr2=0x5619f56d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.327+0000 7f53699bcb80 1 --2- 172.21.15.33:0/679803989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56cfea0 0x5619f56d0320 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f53500cb750 tx=0x7f535000b330 comp rx=0 tx=0).stop 2024-05-08T21:27:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.328+0000 7f53699bcb80 1 -- 172.21.15.33:0/679803989 shutdown_connections 2024-05-08T21:27:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.328+0000 7f53699bcb80 1 --2- 172.21.15.33:0/679803989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56d0860 0x5619f56d0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.328+0000 7f53699bcb80 1 --2- 172.21.15.33:0/679803989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 0x5619f56cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.328+0000 7f53699bcb80 1 --2- 172.21.15.33:0/679803989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56cfea0 0x5619f56d0320 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.328+0000 7f53699bcb80 1 -- 172.21.15.33:0/679803989 >> 172.21.15.33:0/679803989 conn(0x5619f562d210 msgr2=0x5619f56c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.328+0000 7f53699bcb80 1 -- 172.21.15.33:0/679803989 shutdown_connections 2024-05-08T21:27:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.328+0000 7f53699bcb80 1 -- 172.21.15.33:0/679803989 wait complete. 2024-05-08T21:27:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.328+0000 7f53699bcb80 1 -- start start 2024-05-08T21:27:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.328+0000 7f53699bcb80 1 Processor -- start 2024-05-08T21:27:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f53699bcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 0x5619f56d8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f53699bcb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cfea0 0x5619f56d9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f53699bcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56d0860 0x5619f56e0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f53699bcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f55bb560 con 0x5619f56cf540 2024-05-08T21:27:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f53699bcb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f55bb200 con 0x5619f56cfea0 2024-05-08T21:27:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f53699bcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f54a26a0 con 0x5619f56d0860 2024-05-08T21:27:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f53699bcb80 10 client.0 _ll_register_callbacks cb 0x5619f55c4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f5356ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 0x5619f56d8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f5356ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 0x5619f56d8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40146/0 (socket says 172.21.15.33:40146) 2024-05-08T21:27:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f5356ffd700 1 -- 172.21.15.33:0/2157575222 learned_addr learned my addr 172.21.15.33:0/2157575222 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f53577fe700 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56d0860 0x5619f56e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f53567fc700 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cfea0 0x5619f56d9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f5356ffd700 1 -- 172.21.15.33:0/2157575222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56d0860 msgr2=0x5619f56e0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f5356ffd700 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56d0860 0x5619f56e0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f5356ffd700 1 -- 172.21.15.33:0/2157575222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cfea0 msgr2=0x5619f56d9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f5356ffd700 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cfea0 0x5619f56d9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.329+0000 7f5356ffd700 1 -- 172.21.15.33:0/2157575222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c015680 con 0x5619f56cf540 2024-05-08T21:27:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.330+0000 7f5356ffd700 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 0x5619f56d8be0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f534c012960 tx=0x7f534c004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4839c3f2702c146 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.330+0000 7f5336ffd700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:27:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.330+0000 7f5336ffd700 1 -- 172.21.15.33:0/2157575222 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c01a050 con 0x5619f56cf540 2024-05-08T21:27:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.330+0000 7f5336ffd700 1 -- 172.21.15.33:0/2157575222 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f534c013620 con 0x5619f56cf540 2024-05-08T21:27:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.330+0000 7f53699bcb80 1 -- 172.21.15.33:0/2157575222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619f5578270 con 0x5619f56cf540 2024-05-08T21:27:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.330+0000 7f5336ffd700 1 -- 172.21.15.33:0/2157575222 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f534c0152e0 con 0x5619f56cf540 2024-05-08T21:27:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.330+0000 7f53699bcb80 10 client.17685 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.330+0000 7f53699bcb80 1 -- 172.21.15.33:0/2157575222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f53581126e0 con 0x5619f56cf540 2024-05-08T21:27:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.331+0000 7f5336ffd700 1 -- 172.21.15.33:0/2157575222 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5619f5578270 con 0x5619f56cf540 2024-05-08T21:27:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.331+0000 7f5336ffd700 1 -- 172.21.15.33:0/2157575222 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f534c006020 con 0x5619f56cf540 2024-05-08T21:27:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.331+0000 7f53699bcb80 10 client.17685 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.331+0000 7f53699bcb80 10 client.17685 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:27:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.331+0000 7f53699bcb80 20 client.17685 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.331+0000 7f53699bcb80 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5619f56e49d0 0x5619f56e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.331+0000 7f53699bcb80 4 client.17685 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.331+0000 7f53699bcb80 1 -- 172.21.15.33:0/2157575222 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619f569e810 con 0x5619f56e49d0 2024-05-08T21:27:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.331+0000 7f53567fc700 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5619f56e49d0 0x5619f56e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.332+0000 7f53567fc700 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5619f56e49d0 0x5619f56e6df0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21c82cae32ba9a3f server_cookie=6532921ba163fe7f in_seq=0 out_seq=0 2024-05-08T21:27:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.332+0000 7f5336ffd700 10 client.17685 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:27:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.332+0000 7f5336ffd700 1 -- 172.21.15.33:0/2157575222 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619f569e810 con 0x5619f56e49d0 2024-05-08T21:27:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.332+0000 7f5336ffd700 10 client.17685 handle_command_reply: tid=0 2024-05-08T21:27:16.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f53699bcb80 1 client.17685 shutdown 2024-05-08T21:27:16.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f53699bcb80 1 -- 172.21.15.33:0/2157575222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 msgr2=0x5619f56d8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f53699bcb80 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 0x5619f56d8be0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f534c012960 tx=0x7f534c004870 comp rx=0 tx=0).stop 2024-05-08T21:27:16.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f53699bcb80 1 -- 172.21.15.33:0/2157575222 shutdown_connections 2024-05-08T21:27:16.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f53699bcb80 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5619f56e49d0 0x5619f56e6df0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f53699bcb80 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56d0860 0x5619f56e0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f53699bcb80 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cf540 0x5619f56d8be0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f53699bcb80 1 --2- 172.21.15.33:0/2157575222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cfea0 0x5619f56d9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f53699bcb80 1 -- 172.21.15.33:0/2157575222 >> 172.21.15.33:0/2157575222 conn(0x5619f562d210 msgr2=0x5619f56c95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f5336ffd700 0 client.17685 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:27:16.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f53699bcb80 1 -- 172.21.15.33:0/2157575222 shutdown_connections 2024-05-08T21:27:16.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f53699bcb80 1 -- 172.21.15.33:0/2157575222 wait complete. 2024-05-08T21:27:16.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.333+0000 7f53699bcb80 20 client.17685 trim_cache size 0 max 16384 2024-05-08T21:27:16.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.349+0000 7f53699bcb80 1 Processor -- start 2024-05-08T21:27:16.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.349+0000 7f53699bcb80 1 -- start start 2024-05-08T21:27:16.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53699bcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56cf540 0x5619f56d80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53699bcb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cfea0 0x5619f56e0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53699bcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56d0860 0x5619f56eb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53699bcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f56cb160 con 0x5619f56d0860 2024-05-08T21:27:16.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53699bcb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f56c2040 con 0x5619f56cfea0 2024-05-08T21:27:16.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53699bcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f56d8630 con 0x5619f56cf540 2024-05-08T21:27:16.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56d0860 0x5619f56eb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53567fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cfea0 0x5619f56e0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53577fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56d0860 0x5619f56eb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40156/0 (socket says 172.21.15.33:40156) 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53567fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cfea0 0x5619f56e0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41374/0 (socket says 172.21.15.33:41374) 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53577fe700 1 -- 172.21.15.33:0/3103651841 learned_addr learned my addr 172.21.15.33:0/3103651841 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f5356ffd700 1 --2- 172.21.15.33:0/3103651841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56cf540 0x5619f56d80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53577fe700 1 -- 172.21.15.33:0/3103651841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56cf540 msgr2=0x5619f56d80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53577fe700 1 --2- 172.21.15.33:0/3103651841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56cf540 0x5619f56d80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53577fe700 1 -- 172.21.15.33:0/3103651841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cfea0 msgr2=0x5619f56e0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53577fe700 1 --2- 172.21.15.33:0/3103651841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cfea0 0x5619f56e0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53577fe700 1 -- 172.21.15.33:0/3103651841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619f544cb60 con 0x5619f56d0860 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53567fc700 1 --2- 172.21.15.33:0/3103651841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cfea0 0x5619f56e0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f53577fe700 1 --2- 172.21.15.33:0/3103651841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56d0860 0x5619f56eb750 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f534800beb0 tx=0x7f5348012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b6f40d23c78a89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.350+0000 7f533e7fc700 1 -- 172.21.15.33:0/3103651841 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348017070 con 0x5619f56d0860 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.351+0000 7f533e7fc700 1 -- 172.21.15.33:0/3103651841 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5348012b90 con 0x5619f56d0860 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.351+0000 7f533e7fc700 1 -- 172.21.15.33:0/3103651841 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348010630 con 0x5619f56d0860 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.351+0000 7f53699bcb80 1 -- 172.21.15.33:0/3103651841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56d0860 msgr2=0x5619f56eb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.351+0000 7f53699bcb80 1 --2- 172.21.15.33:0/3103651841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56d0860 0x5619f56eb750 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f534800beb0 tx=0x7f5348012730 comp rx=0 tx=0).stop 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.351+0000 7f53699bcb80 1 -- 172.21.15.33:0/3103651841 shutdown_connections 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.351+0000 7f53699bcb80 1 --2- 172.21.15.33:0/3103651841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56d0860 0x5619f56eb750 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.351+0000 7f53699bcb80 1 --2- 172.21.15.33:0/3103651841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56cf540 0x5619f56d80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.351+0000 7f53699bcb80 1 --2- 172.21.15.33:0/3103651841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cfea0 0x5619f56e0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.351+0000 7f53699bcb80 1 -- 172.21.15.33:0/3103651841 >> 172.21.15.33:0/3103651841 conn(0x5619f562d210 msgr2=0x5619f56d6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.351+0000 7f53699bcb80 1 -- 172.21.15.33:0/3103651841 shutdown_connections 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.351+0000 7f53699bcb80 1 -- 172.21.15.33:0/3103651841 wait complete. 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.352+0000 7f53699bcb80 1 -- start start 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.352+0000 7f53699bcb80 1 Processor -- start 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.352+0000 7f53699bcb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cf540 0x5619f56dc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.352+0000 7f53699bcb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cfea0 0x5619f56dc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.352+0000 7f53699bcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56d0860 0x5619f56df8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.352+0000 7f53699bcb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f56dfde0 con 0x5619f56cfea0 2024-05-08T21:27:16.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.352+0000 7f53699bcb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f56dff50 con 0x5619f56cf540 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.352+0000 7f53699bcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619f56e0110 con 0x5619f56d0860 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.352+0000 7f53699bcb80 10 client.0 _ll_register_callbacks cb 0x5619f55c4b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.352+0000 7f5356ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cf540 0x5619f56dc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.352+0000 7f53567fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cfea0 0x5619f56dc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.352+0000 7f5356ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cf540 0x5619f56dc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41378/0 (socket says 172.21.15.33:41378) 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f5356ffd700 1 -- 172.21.15.33:0/3793274541 learned_addr learned my addr 172.21.15.33:0/3793274541 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f53577fe700 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56d0860 0x5619f56df8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f53577fe700 1 -- 172.21.15.33:0/3793274541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cf540 msgr2=0x5619f56dc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f53577fe700 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cf540 0x5619f56dc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f53577fe700 1 -- 172.21.15.33:0/3793274541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cfea0 msgr2=0x5619f56dc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f53577fe700 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cfea0 0x5619f56dc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f53577fe700 1 -- 172.21.15.33:0/3793274541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619f54a03b0 con 0x5619f56d0860 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f5356ffd700 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cf540 0x5619f56dc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f53567fc700 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cfea0 0x5619f56dc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:16.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f53577fe700 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56d0860 0x5619f56df8a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5348000c00 tx=0x7f5348012f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f1711848aa3ea0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f5336ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:16.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f5336ffd700 1 -- 172.21.15.33:0/3793274541 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348029070 con 0x5619f56d0860 2024-05-08T21:27:16.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f5336ffd700 1 -- 172.21.15.33:0/3793274541 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f534801b620 con 0x5619f56d0860 2024-05-08T21:27:16.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.353+0000 7f53699bcb80 1 -- 172.21.15.33:0/3793274541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53500a3550 con 0x5619f56d0860 2024-05-08T21:27:16.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.354+0000 7f5336ffd700 1 -- 172.21.15.33:0/3793274541 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5348017040 con 0x5619f56d0860 2024-05-08T21:27:16.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.354+0000 7f53699bcb80 10 client.27595 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:16.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.354+0000 7f53699bcb80 1 -- 172.21.15.33:0/3793274541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f534800bb60 con 0x5619f56d0860 2024-05-08T21:27:16.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.354+0000 7f5336ffd700 1 -- 172.21.15.33:0/3793274541 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53500a3550 con 0x5619f56d0860 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.354+0000 7f5336ffd700 1 -- 172.21.15.33:0/3793274541 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f534802c020 con 0x5619f56d0860 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.354+0000 7f53699bcb80 10 client.27595 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.354+0000 7f53699bcb80 10 client.27595 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.354+0000 7f53699bcb80 20 client.27595 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.354+0000 7f53699bcb80 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5619f573c230 0x5619f573e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.354+0000 7f53699bcb80 4 client.27595 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.354+0000 7f53699bcb80 1 -- 172.21.15.33:0/3793274541 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5619f5617ec0 con 0x5619f573c230 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.354+0000 7f5356ffd700 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5619f573c230 0x5619f573e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f5336ffd700 10 client.27595 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f5356ffd700 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5619f573c230 0x5619f573e650 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a2c293409d9d11e server_cookie=1f327f8fdbe3ca1 in_seq=0 out_seq=0 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f5336ffd700 1 -- 172.21.15.33:0/3793274541 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5619f5617ec0 con 0x5619f573c230 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f5336ffd700 10 client.27595 handle_command_reply: tid=0 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f53699bcb80 1 client.27595 shutdown 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f53699bcb80 1 -- 172.21.15.33:0/3793274541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56d0860 msgr2=0x5619f56df8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f53699bcb80 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56d0860 0x5619f56df8a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5348000c00 tx=0x7f5348012f30 comp rx=0 tx=0).stop 2024-05-08T21:27:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f53699bcb80 1 -- 172.21.15.33:0/3793274541 shutdown_connections 2024-05-08T21:27:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f53699bcb80 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x5619f573c230 0x5619f573e650 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f53699bcb80 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5619f56d0860 0x5619f56df8a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f53699bcb80 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619f56cf540 0x5619f56dc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f5336ffd700 0 client.27595 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:27:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f53699bcb80 1 --2- 172.21.15.33:0/3793274541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619f56cfea0 0x5619f56dc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f53699bcb80 1 -- 172.21.15.33:0/3793274541 >> 172.21.15.33:0/3793274541 conn(0x5619f562d210 msgr2=0x5619f56eb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f53699bcb80 1 -- 172.21.15.33:0/3793274541 shutdown_connections 2024-05-08T21:27:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f53699bcb80 1 -- 172.21.15.33:0/3793274541 wait complete. 2024-05-08T21:27:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.355+0000 7f53699bcb80 20 client.27595 trim_cache size 0 max 16384 2024-05-08T21:27:16.364 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:27:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.356+0000 7f535d802700 1 -- 172.21.15.33:0/641705405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53400a1170 msgr2=0x7f53400a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.356+0000 7f535d802700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53400a1170 0x7f53400a3630 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f534c009770 tx=0x7f534c006cd0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.356+0000 7f535d802700 1 -- 172.21.15.33:0/641705405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535810a910 msgr2=0x7f53581a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.356+0000 7f535d802700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535810a910 0x7f53581a60e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f535000bcd0 tx=0x7f535000bdb0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.356+0000 7f535d802700 1 -- 172.21.15.33:0/641705405 shutdown_connections 2024-05-08T21:27:16.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.356+0000 7f535d802700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f53400a1170 0x7f53400a3630 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.356+0000 7f535d802700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f535810d2e0 0x7f53581116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.356+0000 7f535d802700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5358107fa0 0x7f53581a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.356+0000 7f535d802700 1 --2- 172.21.15.33:0/641705405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f535810a910 0x7f53581a60e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.356+0000 7f535d802700 1 -- 172.21.15.33:0/641705405 >> 172.21.15.33:0/641705405 conn(0x7f535806dcb0 msgr2=0x7f53581083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.356+0000 7f535d802700 1 -- 172.21.15.33:0/641705405 shutdown_connections 2024-05-08T21:27:16.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.356+0000 7f535d802700 1 -- 172.21.15.33:0/641705405 wait complete. 2024-05-08T21:27:16.379 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:27:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.552+0000 7fa7af8f3700 1 -- 172.21.15.33:0/2717333581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a8100ab0 msgr2=0x7fa7a8100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.552+0000 7fa7af8f3700 1 --2- 172.21.15.33:0/2717333581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a8100ab0 0x7fa7a8100f30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000b0d0 tx=0x7fa7a000b3e0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.552+0000 7fa7af8f3700 1 -- 172.21.15.33:0/2717333581 shutdown_connections 2024-05-08T21:27:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.552+0000 7fa7af8f3700 1 --2- 172.21.15.33:0/2717333581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a8100ab0 0x7fa7a8100f30 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.552+0000 7fa7af8f3700 1 --2- 172.21.15.33:0/2717333581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a80ff790 0x7fa7a80ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.552+0000 7fa7af8f3700 1 --2- 172.21.15.33:0/2717333581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a81000f0 0x7fa7a8100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.552+0000 7fa7af8f3700 1 -- 172.21.15.33:0/2717333581 >> 172.21.15.33:0/2717333581 conn(0x7fa7a80fb330 msgr2=0x7fa7a80fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.552+0000 7fa7af8f3700 1 -- 172.21.15.33:0/2717333581 shutdown_connections 2024-05-08T21:27:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.552+0000 7fa7af8f3700 1 -- 172.21.15.33:0/2717333581 wait complete. 2024-05-08T21:27:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7af8f3700 1 Processor -- start 2024-05-08T21:27:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7af8f3700 1 -- start start 2024-05-08T21:27:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7af8f3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a80ff790 0x7fa7a8199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7af8f3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a81000f0 0x7fa7a81995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7af8f3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a8100ab0 0x7fa7a81a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7af8f3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a81a0b00 con 0x7fa7a80ff790 2024-05-08T21:27:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7af8f3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a81a0c40 con 0x7fa7a81000f0 2024-05-08T21:27:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7af8f3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a81a0dd0 con 0x7fa7a8100ab0 2024-05-08T21:27:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7ad68f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a80ff790 0x7fa7a8199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7ade90700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a8100ab0 0x7fa7a81a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7ad68f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a80ff790 0x7fa7a8199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40182/0 (socket says 172.21.15.33:40182) 2024-05-08T21:27:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7ad68f700 1 -- 172.21.15.33:0/3637088191 learned_addr learned my addr 172.21.15.33:0/3637088191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.553+0000 7fa7ace8e700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a81000f0 0x7fa7a81995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.554+0000 7fa7ade90700 1 -- 172.21.15.33:0/3637088191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a81000f0 msgr2=0x7fa7a81995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.554+0000 7fa7ade90700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a81000f0 0x7fa7a81995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.554+0000 7fa7ade90700 1 -- 172.21.15.33:0/3637088191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a80ff790 msgr2=0x7fa7a8199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.554+0000 7fa7ade90700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a80ff790 0x7fa7a8199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.554+0000 7fa7ade90700 1 -- 172.21.15.33:0/3637088191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa798009520 con 0x7fa7a8100ab0 2024-05-08T21:27:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.554+0000 7fa7ad68f700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a80ff790 0x7fa7a8199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:27:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.554+0000 7fa7ade90700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a8100ab0 0x7fa7a81a0530 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000b770 tx=0x7fa7a000cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.554+0000 7fa79e7fc700 1 -- 172.21.15.33:0/3637088191 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a000e040 con 0x7fa7a8100ab0 2024-05-08T21:27:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.554+0000 7fa7af8f3700 1 -- 172.21.15.33:0/3637088191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a0009d00 con 0x7fa7a8100ab0 2024-05-08T21:27:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.554+0000 7fa79e7fc700 1 -- 172.21.15.33:0/3637088191 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7a0004c50 con 0x7fa7a8100ab0 2024-05-08T21:27:16.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.554+0000 7fa79e7fc700 1 -- 172.21.15.33:0/3637088191 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a001b6b0 con 0x7fa7a8100ab0 2024-05-08T21:27:16.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.554+0000 7fa7af8f3700 1 -- 172.21.15.33:0/3637088191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a81a1340 con 0x7fa7a8100ab0 2024-05-08T21:27:16.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.556+0000 7fa79e7fc700 1 -- 172.21.15.33:0/3637088191 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa7a001d480 con 0x7fa7a8100ab0 2024-05-08T21:27:16.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.557+0000 7fa79e7fc700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7940a1120 0x7fa7940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.558+0000 7fa7ad68f700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7940a1120 0x7fa7940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.558+0000 7fa79e7fc700 1 -- 172.21.15.33:0/3637088191 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa7a00ce4d0 con 0x7fa7a8100ab0 2024-05-08T21:27:16.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.558+0000 7fa7ad68f700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7940a1120 0x7fa7940a35e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009d70 tx=0x7fa7a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.558+0000 7fa7bbaadb80 1 Processor -- start 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.558+0000 7fa7bbaadb80 1 -- start start 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7bbaadb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ce540 0x55c8422ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7bbaadb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ceea0 0x55c8422cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7bbaadb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422cf860 0x55c8422cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7bbaadb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c842225150 con 0x55c8422ce540 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7bbaadb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c842222b10 con 0x55c8422ceea0 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7bbaadb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c84221e400 con 0x55c8422cf860 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7ad68f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ce540 0x55c8422ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7ade90700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422cf860 0x55c8422cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7ad68f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ce540 0x55c8422ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40198/0 (socket says 172.21.15.33:40198) 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7ace8e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ceea0 0x55c8422cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7ad68f700 1 -- 172.21.15.33:0/3870367706 learned_addr learned my addr 172.21.15.33:0/3870367706 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7ade90700 1 -- 172.21.15.33:0/3870367706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ceea0 msgr2=0x55c8422cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7ade90700 1 --2- 172.21.15.33:0/3870367706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ceea0 0x55c8422cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7ade90700 1 -- 172.21.15.33:0/3870367706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ce540 msgr2=0x55c8422ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7ade90700 1 --2- 172.21.15.33:0/3870367706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ce540 0x55c8422ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7ade90700 1 -- 172.21.15.33:0/3870367706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c84209f3b0 con 0x55c8422cf860 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.559+0000 7fa7ade90700 1 --2- 172.21.15.33:0/3870367706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422cf860 0x55c8422cfce0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00cbbe0 tx=0x7fa7a000cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b856343365ca715 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.560+0000 7fa7937fe700 1 -- 172.21.15.33:0/3870367706 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a009bcf0 con 0x55c8422cf860 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.560+0000 7fa7937fe700 1 -- 172.21.15.33:0/3870367706 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7a009b5f0 con 0x55c8422cf860 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.560+0000 7fa7937fe700 1 -- 172.21.15.33:0/3870367706 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0004e30 con 0x55c8422cf860 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.561+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3870367706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422cf860 msgr2=0x55c8422cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.561+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/3870367706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422cf860 0x55c8422cfce0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00cbbe0 tx=0x7fa7a000cee0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.561+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3870367706 shutdown_connections 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.561+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/3870367706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422cf860 0x55c8422cfce0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.561+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/3870367706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ce540 0x55c8422ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.561+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/3870367706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ceea0 0x55c8422cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.561+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3870367706 >> 172.21.15.33:0/3870367706 conn(0x55c84222c210 msgr2=0x55c8422c8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.561+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3870367706 shutdown_connections 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.561+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3870367706 wait complete. 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.561+0000 7fa7bbaadb80 1 -- start start 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.561+0000 7fa7bbaadb80 1 Processor -- start 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.562+0000 7fa7bbaadb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 0x55c8422d7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.562+0000 7fa7bbaadb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ceea0 0x55c8422d8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.562+0000 7fa7bbaadb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422cf860 0x55c8422df1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.562+0000 7fa7bbaadb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8421ba560 con 0x55c8422ceea0 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.562+0000 7fa7bbaadb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8421ba200 con 0x55c8422cf860 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.562+0000 7fa7bbaadb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8420a16a0 con 0x55c8422ce540 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.562+0000 7fa7bbaadb80 10 client.0 _ll_register_callbacks cb 0x55c8421c3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.562+0000 7fa7ad68f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 0x55c8422d7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.562+0000 7fa7ad68f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 0x55c8422d7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:56494/0 (socket says 172.21.15.33:56494) 2024-05-08T21:27:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.562+0000 7fa7ad68f700 1 -- 172.21.15.33:0/989000106 learned_addr learned my addr 172.21.15.33:0/989000106 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.562+0000 7fa7ade90700 1 --2- 172.21.15.33:0/989000106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422cf860 0x55c8422df1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa7ad68f700 1 -- 172.21.15.33:0/989000106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422cf860 msgr2=0x55c8422df1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa7ad68f700 1 --2- 172.21.15.33:0/989000106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422cf860 0x55c8422df1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa7ad68f700 1 -- 172.21.15.33:0/989000106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ceea0 msgr2=0x55c8422d8170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa7ad68f700 1 --2- 172.21.15.33:0/989000106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ceea0 0x55c8422d8170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa7ad68f700 1 -- 172.21.15.33:0/989000106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a81a1340 con 0x55c8422ce540 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa7ad68f700 1 --2- 172.21.15.33:0/989000106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 0x55c8422d7c30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4011a50 tx=0x7fa7a40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14cee683e3c48527 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa790ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa790ff9700 1 -- 172.21.15.33:0/989000106 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4020900 con 0x55c8422ce540 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa790ff9700 1 -- 172.21.15.33:0/989000106 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa7a4018070 con 0x55c8422ce540 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/989000106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c842177270 con 0x55c8422ce540 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa790ff9700 1 -- 172.21.15.33:0/989000106 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7a4020a60 con 0x55c8422ce540 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa7bbaadb80 10 client.27607 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:16.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/989000106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c84204bb60 con 0x55c8422ce540 2024-05-08T21:27:16.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.563+0000 7fa790ff9700 1 -- 172.21.15.33:0/989000106 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c842177270 con 0x55c8422ce540 2024-05-08T21:27:16.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.564+0000 7fa790ff9700 1 -- 172.21.15.33:0/989000106 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa7a401f430 con 0x55c8422ce540 2024-05-08T21:27:16.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.564+0000 7fa7bbaadb80 10 client.27607 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:16.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.564+0000 7fa7bbaadb80 10 client.27607 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:27:16.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.564+0000 7fa7bbaadb80 20 client.27607 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:16.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.564+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/989000106 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c8422e3a40 0x55c8422e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.564+0000 7fa7bbaadb80 4 client.27607 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:16.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.564+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/989000106 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c84229d810 con 0x55c8422e3a40 2024-05-08T21:27:16.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.564+0000 7fa7ace8e700 1 --2- 172.21.15.33:0/989000106 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c8422e3a40 0x55c8422e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.564+0000 7fa790ff9700 10 client.27607 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.564+0000 7fa7ace8e700 1 --2- 172.21.15.33:0/989000106 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c8422e3a40 0x55c8422e5e60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=780eef1ad8adb233 server_cookie=28b5e420d8513ff3 in_seq=0 out_seq=0 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.565+0000 7fa790ff9700 1 -- 172.21.15.33:0/989000106 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c84229d810 con 0x55c8422e3a40 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.565+0000 7fa790ff9700 10 client.27607 handle_command_reply: tid=0 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.565+0000 7fa7bbaadb80 1 client.27607 shutdown 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.566+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/989000106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 msgr2=0x55c8422d7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.566+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/989000106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 0x55c8422d7c30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4011a50 tx=0x7fa7a40047c0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.566+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/989000106 shutdown_connections 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.566+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/989000106 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c8422e3a40 0x55c8422e5e60 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.566+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/989000106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422cf860 0x55c8422df1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.566+0000 7fa790ff9700 0 client.27607 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.566+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/989000106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 0x55c8422d7c30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.566+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/989000106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ceea0 0x55c8422d8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.566+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/989000106 >> 172.21.15.33:0/989000106 conn(0x55c84222c210 msgr2=0x55c8422c85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.567+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/989000106 shutdown_connections 2024-05-08T21:27:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.567+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/989000106 wait complete. 2024-05-08T21:27:16.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.567+0000 7fa7bbaadb80 20 client.27607 trim_cache size 0 max 16384 2024-05-08T21:27:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.584+0000 7fa7bbaadb80 1 Processor -- start 2024-05-08T21:27:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.584+0000 7fa7bbaadb80 1 -- start start 2024-05-08T21:27:16.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7bbaadb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 0x55c8422ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7bbaadb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ceea0 0x55c8422df170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7bbaadb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422cf860 0x55c8422d5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7bbaadb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8422c97d0 con 0x55c8422cf860 2024-05-08T21:27:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7bbaadb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8422c1040 con 0x55c8422ceea0 2024-05-08T21:27:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7bbaadb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8422d6350 con 0x55c8422ce540 2024-05-08T21:27:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7ad68f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 0x55c8422ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7ade90700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422cf860 0x55c8422d5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7ad68f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 0x55c8422ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:56496/0 (socket says 172.21.15.33:56496) 2024-05-08T21:27:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7ace8e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ceea0 0x55c8422df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7ad68f700 1 -- 172.21.15.33:0/2028521891 learned_addr learned my addr 172.21.15.33:0/2028521891 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7ad68f700 1 -- 172.21.15.33:0/2028521891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ceea0 msgr2=0x55c8422df170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7ad68f700 1 --2- 172.21.15.33:0/2028521891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ceea0 0x55c8422df170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7ad68f700 1 -- 172.21.15.33:0/2028521891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422cf860 msgr2=0x55c8422d5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7ad68f700 1 --2- 172.21.15.33:0/2028521891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422cf860 0x55c8422d5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7ad68f700 1 -- 172.21.15.33:0/2028521891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8421e4560 con 0x55c8422ce540 2024-05-08T21:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.585+0000 7fa7ace8e700 1 --2- 172.21.15.33:0/2028521891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ceea0 0x55c8422df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.586+0000 7fa7ad68f700 1 --2- 172.21.15.33:0/2028521891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 0x55c8422ce920 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009c30 tx=0x7fa7a4011ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f10a1e98da98ec9c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.586+0000 7fa793fff700 1 -- 172.21.15.33:0/2028521891 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4011e30 con 0x55c8422ce540 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.586+0000 7fa793fff700 1 -- 172.21.15.33:0/2028521891 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7a4016070 con 0x55c8422ce540 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.586+0000 7fa793fff700 1 -- 172.21.15.33:0/2028521891 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4020df0 con 0x55c8422ce540 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.587+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/2028521891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 msgr2=0x55c8422ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.587+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/2028521891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 0x55c8422ce920 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009c30 tx=0x7fa7a4011ce0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.587+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/2028521891 shutdown_connections 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.587+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/2028521891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422cf860 0x55c8422d5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.587+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/2028521891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422ce540 0x55c8422ce920 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.587+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/2028521891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ceea0 0x55c8422df170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.587+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/2028521891 >> 172.21.15.33:0/2028521891 conn(0x55c84222c210 msgr2=0x55c8422c5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.587+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/2028521891 shutdown_connections 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.587+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/2028521891 wait complete. 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.587+0000 7fa7bbaadb80 1 -- start start 2024-05-08T21:27:16.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.587+0000 7fa7bbaadb80 1 Processor -- start 2024-05-08T21:27:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7bbaadb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ce540 0x55c8422dac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7bbaadb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ceea0 0x55c8422db190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7bbaadb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422cf860 0x55c8422de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7bbaadb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8422de6f0 con 0x55c8422ceea0 2024-05-08T21:27:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7bbaadb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8422de860 con 0x55c8422ce540 2024-05-08T21:27:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7bbaadb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8422dea20 con 0x55c8422cf860 2024-05-08T21:27:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7bbaadb80 10 client.0 _ll_register_callbacks cb 0x55c8421c3b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7ace8e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ceea0 0x55c8422db190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7ade90700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422cf860 0x55c8422de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7ace8e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ceea0 0x55c8422db190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40220/0 (socket says 172.21.15.33:40220) 2024-05-08T21:27:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7ace8e700 1 -- 172.21.15.33:0/3935575131 learned_addr learned my addr 172.21.15.33:0/3935575131 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7ad68f700 1 --2- 172.21.15.33:0/3935575131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ce540 0x55c8422dac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7ade90700 1 -- 172.21.15.33:0/3935575131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ce540 msgr2=0x55c8422dac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7ade90700 1 --2- 172.21.15.33:0/3935575131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ce540 0x55c8422dac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7ade90700 1 -- 172.21.15.33:0/3935575131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ceea0 msgr2=0x55c8422db190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7ade90700 1 --2- 172.21.15.33:0/3935575131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ceea0 0x55c8422db190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.588+0000 7fa7ade90700 1 -- 172.21.15.33:0/3935575131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa798009bb0 con 0x55c8422cf860 2024-05-08T21:27:16.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa7ade90700 1 --2- 172.21.15.33:0/3935575131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422cf860 0x55c8422de1b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa7a009f5e0 tx=0x7fa7a0099730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bba54455cfe57df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa790ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa790ff9700 1 -- 172.21.15.33:0/3935575131 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a009bcf0 con 0x55c8422cf860 2024-05-08T21:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa790ff9700 1 -- 172.21.15.33:0/3935575131 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa7a0084e40 con 0x55c8422cf860 2024-05-08T21:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3935575131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7a009b5f0 con 0x55c8422cf860 2024-05-08T21:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa790ff9700 1 -- 172.21.15.33:0/3935575131 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7a009f420 con 0x55c8422cf860 2024-05-08T21:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa7bbaadb80 10 client.27613 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3935575131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c84204bb60 con 0x55c8422cf860 2024-05-08T21:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa790ff9700 1 -- 172.21.15.33:0/3935575131 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7a009b5f0 con 0x55c8422cf860 2024-05-08T21:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa790ff9700 1 -- 172.21.15.33:0/3935575131 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa7a00860c0 con 0x55c8422cf860 2024-05-08T21:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa7bbaadb80 10 client.27613 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:16.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa7bbaadb80 10 client.27613 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa7bbaadb80 20 client.27613 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/3935575131 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c84233a740 0x55c84233cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa7bbaadb80 4 client.27613 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.589+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3935575131 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c842216ec0 con 0x55c84233a740 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa7ad68f700 1 --2- 172.21.15.33:0/3935575131 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c84233a740 0x55c84233cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa7ad68f700 1 --2- 172.21.15.33:0/3935575131 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c84233a740 0x55c84233cb30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b07903f8f006532f server_cookie=ba2a387eeb201ddc in_seq=0 out_seq=0 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa790ff9700 10 client.27613 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa790ff9700 1 -- 172.21.15.33:0/3935575131 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c842216ec0 con 0x55c84233a740 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa790ff9700 10 client.27613 handle_command_reply: tid=0 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa7bbaadb80 1 client.27613 shutdown 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3935575131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422cf860 msgr2=0x55c8422de1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/3935575131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422cf860 0x55c8422de1b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa7a009f5e0 tx=0x7fa7a0099730 comp rx=0 tx=0).stop 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3935575131 shutdown_connections 2024-05-08T21:27:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/3935575131 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x55c84233a740 0x55c84233cb30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/3935575131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c8422cf860 0x55c8422de1b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/3935575131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8422ce540 0x55c8422dac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa790ff9700 0 client.27613 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:27:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa7bbaadb80 1 --2- 172.21.15.33:0/3935575131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8422ceea0 0x55c8422db190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.590+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3935575131 >> 172.21.15.33:0/3935575131 conn(0x55c84222c210 msgr2=0x55c8422d27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3935575131 shutdown_connections 2024-05-08T21:27:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7bbaadb80 1 -- 172.21.15.33:0/3935575131 wait complete. 2024-05-08T21:27:16.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7bbaadb80 20 client.27613 trim_cache size 0 max 16384 2024-05-08T21:27:16.599 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:27:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7af8f3700 1 -- 172.21.15.33:0/3637088191 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7940a1120 msgr2=0x7fa7940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7af8f3700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7940a1120 0x7fa7940a35e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009d70 tx=0x7fa7a4009260 comp rx=0 tx=0).stop 2024-05-08T21:27:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7af8f3700 1 -- 172.21.15.33:0/3637088191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a8100ab0 msgr2=0x7fa7a81a0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7af8f3700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a8100ab0 0x7fa7a81a0530 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa7a000b770 tx=0x7fa7a000cab0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7af8f3700 1 -- 172.21.15.33:0/3637088191 shutdown_connections 2024-05-08T21:27:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7af8f3700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7a8100ab0 0x7fa7a81a0530 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7af8f3700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7a80ff790 0x7fa7a8199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7af8f3700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa7940a1120 0x7fa7940a35e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7af8f3700 1 --2- 172.21.15.33:0/3637088191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7a81000f0 0x7fa7a81995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7af8f3700 1 -- 172.21.15.33:0/3637088191 >> 172.21.15.33:0/3637088191 conn(0x7fa7a80fb330 msgr2=0x7fa7a80fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7af8f3700 1 -- 172.21.15.33:0/3637088191 shutdown_connections 2024-05-08T21:27:16.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.591+0000 7fa7af8f3700 1 -- 172.21.15.33:0/3637088191 wait complete. 2024-05-08T21:27:16.615 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:27:16.652 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:16 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/503475596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:27:16.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.813+0000 7f9b8fc21700 1 -- 172.21.15.33:0/3805870370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b88107fa0 msgr2=0x7f9b8810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.813+0000 7f9b8fc21700 1 --2- 172.21.15.33:0/3805870370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b88107fa0 0x7f9b8810a3d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9b7801a9a0 tx=0x7f9b7801acb0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.815+0000 7f9b8fc21700 1 -- 172.21.15.33:0/3805870370 shutdown_connections 2024-05-08T21:27:16.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.815+0000 7f9b8fc21700 1 --2- 172.21.15.33:0/3805870370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b8810d2e0 0x7f9b8810d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.815+0000 7f9b8fc21700 1 --2- 172.21.15.33:0/3805870370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b88107fa0 0x7f9b8810a3d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.815+0000 7f9b8fc21700 1 --2- 172.21.15.33:0/3805870370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b8810a910 0x7f9b8810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.815+0000 7f9b8fc21700 1 -- 172.21.15.33:0/3805870370 >> 172.21.15.33:0/3805870370 conn(0x7f9b8806dcb0 msgr2=0x7f9b880700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.815+0000 7f9b8fc21700 1 -- 172.21.15.33:0/3805870370 shutdown_connections 2024-05-08T21:27:16.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.815+0000 7f9b8fc21700 1 -- 172.21.15.33:0/3805870370 wait complete. 2024-05-08T21:27:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.816+0000 7f9b8fc21700 1 Processor -- start 2024-05-08T21:27:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.816+0000 7f9b8fc21700 1 -- start start 2024-05-08T21:27:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.816+0000 7f9b8fc21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b88107fa0 0x7f9b8819d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.816+0000 7f9b8fc21700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b8810a910 0x7f9b8819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.816+0000 7f9b8fc21700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b8810d2e0 0x7f9b881a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.816+0000 7f9b8fc21700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b881a4ea0 con 0x7f9b88107fa0 2024-05-08T21:27:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.816+0000 7f9b8fc21700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b881a4fe0 con 0x7f9b8810d2e0 2024-05-08T21:27:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.816+0000 7f9b8fc21700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b881a5170 con 0x7f9b8810a910 2024-05-08T21:27:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8e1be700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b8810d2e0 0x7f9b881a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8d1bc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b8810a910 0x7f9b8819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8d1bc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b8810a910 0x7f9b8819d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:56502/0 (socket says 172.21.15.33:56502) 2024-05-08T21:27:16.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8e1be700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b8810d2e0 0x7f9b881a48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41458/0 (socket says 172.21.15.33:41458) 2024-05-08T21:27:16.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8d1bc700 1 -- 172.21.15.33:0/2338859785 learned_addr learned my addr 172.21.15.33:0/2338859785 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8d9bd700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b88107fa0 0x7f9b8819d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8e1be700 1 -- 172.21.15.33:0/2338859785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b8810a910 msgr2=0x7f9b8819d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8e1be700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b8810a910 0x7f9b8819d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8e1be700 1 -- 172.21.15.33:0/2338859785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b88107fa0 msgr2=0x7f9b8819d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8e1be700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b88107fa0 0x7f9b8819d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8e1be700 1 -- 172.21.15.33:0/2338859785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7801a600 con 0x7f9b8810d2e0 2024-05-08T21:27:16.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8e1be700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b8810d2e0 0x7f9b881a48d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400b3c0 tx=0x7f9b8400b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.817+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b8810a910 0x7f9b8819d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:16.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.818+0000 7f9b7effd700 1 -- 172.21.15.33:0/2338859785 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84011620 con 0x7f9b8810d2e0 2024-05-08T21:27:16.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.818+0000 7f9b7effd700 1 -- 172.21.15.33:0/2338859785 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b84011c10 con 0x7f9b8810d2e0 2024-05-08T21:27:16.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.818+0000 7f9b7effd700 1 -- 172.21.15.33:0/2338859785 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84010be0 con 0x7f9b8810d2e0 2024-05-08T21:27:16.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:16 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/503475596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:27:16.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.818+0000 7f9b8fc21700 1 -- 172.21.15.33:0/2338859785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b881a5450 con 0x7f9b8810d2e0 2024-05-08T21:27:16.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.818+0000 7f9b8fc21700 1 -- 172.21.15.33:0/2338859785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b881a5910 con 0x7f9b8810d2e0 2024-05-08T21:27:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.822+0000 7f9b9bddbb80 1 Processor -- start 2024-05-08T21:27:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.822+0000 7f9b7effd700 1 -- 172.21.15.33:0/2338859785 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9b84010d40 con 0x7f9b8810d2e0 2024-05-08T21:27:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.823+0000 7f9b7effd700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9b740a1270 0x7f9b740a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.823+0000 7f9b7effd700 1 -- 172.21.15.33:0/2338859785 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9b840cc490 con 0x7f9b8810d2e0 2024-05-08T21:27:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b9bddbb80 1 -- start start 2024-05-08T21:27:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b9bddbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273f540 0x55c77273f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b9bddbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273fea0 0x55c772740320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9b740a1270 0x7f9b740a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b9bddbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c772740ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b9bddbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c772696150 con 0x55c77273fea0 2024-05-08T21:27:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b9bddbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c772693b10 con 0x55c772740860 2024-05-08T21:27:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b9bddbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77268f400 con 0x55c77273f540 2024-05-08T21:27:16.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b8e1be700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c772740ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b8d9bd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273f540 0x55c77273f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b8e1be700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c772740ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41466/0 (socket says 172.21.15.33:41466) 2024-05-08T21:27:16.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b8d9bd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273f540 0x55c77273f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:56506/0 (socket says 172.21.15.33:56506) 2024-05-08T21:27:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b8e1be700 1 -- 172.21.15.33:0/3732177754 learned_addr learned my addr 172.21.15.33:0/3732177754 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/3732177754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273fea0 0x55c772740320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b8e1be700 1 -- 172.21.15.33:0/3732177754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273f540 msgr2=0x55c77273f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b8e1be700 1 --2- 172.21.15.33:0/3732177754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273f540 0x55c77273f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b8e1be700 1 -- 172.21.15.33:0/3732177754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273fea0 msgr2=0x55c772740320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b8e1be700 1 --2- 172.21.15.33:0/3732177754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273fea0 0x55c772740320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.824+0000 7f9b8e1be700 1 -- 172.21.15.33:0/3732177754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7725103b0 con 0x55c772740860 2024-05-08T21:27:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.825+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9b740a1270 0x7f9b740a3730 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9b800094f0 tx=0x7f9b800175d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.825+0000 7f9b8e1be700 1 --2- 172.21.15.33:0/3732177754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c772740ce0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9b840994b0 tx=0x7f9b84099130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa8604ff7f500377 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.825+0000 7f9b73fff700 1 -- 172.21.15.33:0/3732177754 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b840cc070 con 0x55c772740860 2024-05-08T21:27:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.825+0000 7f9b73fff700 1 -- 172.21.15.33:0/3732177754 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b84098e80 con 0x55c772740860 2024-05-08T21:27:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.825+0000 7f9b73fff700 1 -- 172.21.15.33:0/3732177754 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84099590 con 0x55c772740860 2024-05-08T21:27:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.826+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/3732177754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 msgr2=0x55c772740ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.826+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/3732177754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c772740ce0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9b840994b0 tx=0x7f9b84099130 comp rx=0 tx=0).stop 2024-05-08T21:27:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.826+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/3732177754 shutdown_connections 2024-05-08T21:27:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.826+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/3732177754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c772740ce0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.826+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/3732177754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273f540 0x55c77273f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.826+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/3732177754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273fea0 0x55c772740320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.826+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/3732177754 >> 172.21.15.33:0/3732177754 conn(0x55c77269d210 msgr2=0x55c772739de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.826+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/3732177754 shutdown_connections 2024-05-08T21:27:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.826+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/3732177754 wait complete. 2024-05-08T21:27:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.826+0000 7f9b9bddbb80 1 -- start start 2024-05-08T21:27:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b9bddbb80 1 Processor -- start 2024-05-08T21:27:16.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b9bddbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c772748e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b9bddbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c77273fea0 0x55c772749370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b9bddbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c772740860 0x55c772750360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b9bddbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77262b560 con 0x55c77273f540 2024-05-08T21:27:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b9bddbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77262b200 con 0x55c77273fea0 2024-05-08T21:27:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b9bddbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7725126a0 con 0x55c772740860 2024-05-08T21:27:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b9bddbb80 10 client.0 _ll_register_callbacks cb 0x55c772634b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b8d9bd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c772748e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b8e1be700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c772740860 0x55c772750360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b8d9bd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c772748e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40254/0 (socket says 172.21.15.33:40254) 2024-05-08T21:27:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b8d9bd700 1 -- 172.21.15.33:0/596751722 learned_addr learned my addr 172.21.15.33:0/596751722 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.827+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c77273fea0 0x55c772749370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b8e1be700 1 -- 172.21.15.33:0/596751722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c77273fea0 msgr2=0x55c772749370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b8e1be700 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c77273fea0 0x55c772749370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b8e1be700 1 -- 172.21.15.33:0/596751722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 msgr2=0x55c772748e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b8e1be700 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c772748e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b8e1be700 1 -- 172.21.15.33:0/596751722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b881a5910 con 0x55c772740860 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c77273fea0 0x55c772749370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b8e1be700 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c772740860 0x55c772750360 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400ba20 tx=0x7f9b84095900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79f2e97a0db2b099 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b717fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b717fa700 1 -- 172.21.15.33:0/596751722 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b840cc040 con 0x55c772740860 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b717fa700 1 -- 172.21.15.33:0/596751722 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9b8409d350 con 0x55c772740860 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/596751722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7725e8270 con 0x55c772740860 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b717fa700 1 -- 172.21.15.33:0/596751722 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b84095040 con 0x55c772740860 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b9bddbb80 10 client.27625 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/596751722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c7724bcb60 con 0x55c772740860 2024-05-08T21:27:16.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.828+0000 7f9b717fa700 1 -- 172.21.15.33:0/596751722 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b84081020 con 0x55c772740860 2024-05-08T21:27:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.829+0000 7f9b717fa700 1 -- 172.21.15.33:0/596751722 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9b840c9c50 con 0x55c772740860 2024-05-08T21:27:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.829+0000 7f9b9bddbb80 10 client.27625 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.829+0000 7f9b9bddbb80 10 client.27625 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:27:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.829+0000 7f9b9bddbb80 20 client.27625 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.829+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55c772754910 0x55c772756d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.829+0000 7f9b9bddbb80 4 client.27625 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.829+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/596751722 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c77270e810 con 0x55c772754910 2024-05-08T21:27:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.829+0000 7f9b8d9bd700 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55c772754910 0x55c772756d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.829+0000 7f9b717fa700 10 client.27625 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:27:16.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.829+0000 7f9b8d9bd700 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55c772754910 0x55c772756d30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=411208d589bd178f server_cookie=81428935e9745a18 in_seq=0 out_seq=0 2024-05-08T21:27:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.830+0000 7f9b717fa700 1 -- 172.21.15.33:0/596751722 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c77270e810 con 0x55c772754910 2024-05-08T21:27:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.830+0000 7f9b717fa700 10 client.27625 handle_command_reply: tid=0 2024-05-08T21:27:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.830+0000 7f9b9bddbb80 1 client.27625 shutdown 2024-05-08T21:27:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.830+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/596751722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c772740860 msgr2=0x55c772750360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.830+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c772740860 0x55c772750360 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400ba20 tx=0x7f9b84095900 comp rx=0 tx=0).stop 2024-05-08T21:27:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.830+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/596751722 shutdown_connections 2024-05-08T21:27:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.830+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55c772754910 0x55c772756d30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.830+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c772740860 0x55c772750360 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.830+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c772748e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.830+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/596751722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c77273fea0 0x55c772749370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.830+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/596751722 >> 172.21.15.33:0/596751722 conn(0x55c77269d210 msgr2=0x55c7727395a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.831+0000 7f9b717fa700 0 client.27625 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:27:16.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.831+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/596751722 shutdown_connections 2024-05-08T21:27:16.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.831+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/596751722 wait complete. 2024-05-08T21:27:16.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.831+0000 7f9b9bddbb80 20 client.27625 trim_cache size 0 max 16384 2024-05-08T21:27:16.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.848+0000 7f9b9bddbb80 1 Processor -- start 2024-05-08T21:27:16.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.848+0000 7f9b9bddbb80 1 -- start start 2024-05-08T21:27:16.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.849+0000 7f9b9bddbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c77273f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.849+0000 7f9b9bddbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273fea0 0x55c7727483b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.849+0000 7f9b9bddbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c772750320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.849+0000 7f9b9bddbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c772736dc0 con 0x55c77273f540 2024-05-08T21:27:16.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.849+0000 7f9b9bddbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77275bee0 con 0x55c772740860 2024-05-08T21:27:16.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.849+0000 7f9b9bddbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c772746b70 con 0x55c77273fea0 2024-05-08T21:27:16.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.849+0000 7f9b8d9bd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c77273f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.849+0000 7f9b8d9bd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c77273f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40258/0 (socket says 172.21.15.33:40258) 2024-05-08T21:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.849+0000 7f9b8d9bd700 1 -- 172.21.15.33:0/3341462382 learned_addr learned my addr 172.21.15.33:0/3341462382 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.849+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/3341462382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273fea0 0x55c7727483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.850+0000 7f9b8d1bc700 1 -- 172.21.15.33:0/3341462382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 msgr2=0x55c772750320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.850+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/3341462382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c772750320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.850+0000 7f9b8d1bc700 1 -- 172.21.15.33:0/3341462382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 msgr2=0x55c77273f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.850+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/3341462382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c77273f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.850+0000 7f9b8d1bc700 1 -- 172.21.15.33:0/3341462382 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c772655560 con 0x55c77273fea0 2024-05-08T21:27:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.850+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/3341462382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273fea0 0x55c7727483b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9b80017ec0 tx=0x7f9b8001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da7df8cf9ebfccbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:16 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/503475596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.851+0000 7f9b7cda4700 1 -- 172.21.15.33:0/3341462382 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b800073e0 con 0x55c77273fea0 2024-05-08T21:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.851+0000 7f9b7cda4700 1 -- 172.21.15.33:0/3341462382 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b8000f040 con 0x55c77273fea0 2024-05-08T21:27:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.851+0000 7f9b7cda4700 1 -- 172.21.15.33:0/3341462382 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b800198d0 con 0x55c77273fea0 2024-05-08T21:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.853+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/3341462382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273fea0 msgr2=0x55c7727483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.853+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/3341462382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273fea0 0x55c7727483b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9b80017ec0 tx=0x7f9b8001a000 comp rx=0 tx=0).stop 2024-05-08T21:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.853+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/3341462382 shutdown_connections 2024-05-08T21:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.853+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/3341462382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c772750320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.853+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/3341462382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c77273f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.853+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/3341462382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273fea0 0x55c7727483b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.853+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/3341462382 >> 172.21.15.33:0/3341462382 conn(0x55c77269d210 msgr2=0x55c772736210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.854+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/3341462382 shutdown_connections 2024-05-08T21:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.854+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/3341462382 wait complete. 2024-05-08T21:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.854+0000 7f9b9bddbb80 1 -- start start 2024-05-08T21:27:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.854+0000 7f9b9bddbb80 1 Processor -- start 2024-05-08T21:27:16.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.858+0000 7f9b9bddbb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c77274daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.858+0000 7f9b9bddbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273fea0 0x55c77274dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.858+0000 7f9b9bddbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c77274eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.858+0000 7f9b9bddbb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77274f530 con 0x55c77273f540 2024-05-08T21:27:16.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.858+0000 7f9b9bddbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77274f6a0 con 0x55c772740860 2024-05-08T21:27:16.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.858+0000 7f9b9bddbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c77274f860 con 0x55c77273fea0 2024-05-08T21:27:16.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b9bddbb80 10 client.0 _ll_register_callbacks cb 0x55c772634b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:16.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b8d9bd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c77274daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b8d1bc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273fea0 0x55c77274dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b8d9bd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c77274daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40266/0 (socket says 172.21.15.33:40266) 2024-05-08T21:27:16.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b8d9bd700 1 -- 172.21.15.33:0/1474095770 learned_addr learned my addr 172.21.15.33:0/1474095770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:16.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b8e1be700 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c77274eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b8d1bc700 1 -- 172.21.15.33:0/1474095770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 msgr2=0x55c77274eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c77274eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b8d1bc700 1 -- 172.21.15.33:0/1474095770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 msgr2=0x55c77274daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c77274daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b8d1bc700 1 -- 172.21.15.33:0/1474095770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b80017bb0 con 0x55c77273fea0 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b8e1be700 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c77274eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b8d1bc700 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273fea0 0x55c77274dfe0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9b800079e0 tx=0x7f9b80026900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9250a2c18ecd388 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.859+0000 7f9b717fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.860+0000 7f9b717fa700 1 -- 172.21.15.33:0/1474095770 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b800073e0 con 0x55c77273fea0 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.860+0000 7f9b717fa700 1 -- 172.21.15.33:0/1474095770 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9b80004a90 con 0x55c77273fea0 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.860+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/1474095770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b84098e80 con 0x55c77273fea0 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.860+0000 7f9b717fa700 1 -- 172.21.15.33:0/1474095770 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b80004c50 con 0x55c77273fea0 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.860+0000 7f9b9bddbb80 10 client.27631 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.860+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/1474095770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c772514cd0 con 0x55c77273fea0 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.861+0000 7f9b717fa700 1 -- 172.21.15.33:0/1474095770 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b8003c030 con 0x55c77273fea0 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.861+0000 7f9b717fa700 1 -- 172.21.15.33:0/1474095770 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9b80019cd0 con 0x55c77273fea0 2024-05-08T21:27:16.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.861+0000 7f9b9bddbb80 10 client.27631 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:16.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.861+0000 7f9b9bddbb80 10 client.27631 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:27:16.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.861+0000 7f9b9bddbb80 20 client.27631 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:16.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.861+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55c7727ab680 0x55c7727adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:16.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.861+0000 7f9b9bddbb80 4 client.27631 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:27:16.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.861+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/1474095770 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c772687ec0 con 0x55c7727ab680 2024-05-08T21:27:16.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.862+0000 7f9b8d9bd700 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55c7727ab680 0x55c7727adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.863+0000 7f9b717fa700 10 client.27631 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.863+0000 7f9b8d9bd700 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55c7727ab680 0x55c7727adaa0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6df02c522e1c7eb7 server_cookie=5f02ffe67fd2542f in_seq=0 out_seq=0 2024-05-08T21:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b717fa700 1 -- 172.21.15.33:0/1474095770 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c772687ec0 con 0x55c7727ab680 2024-05-08T21:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b717fa700 10 client.27631 handle_command_reply: tid=0 2024-05-08T21:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b9bddbb80 1 client.27631 shutdown 2024-05-08T21:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/1474095770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273fea0 msgr2=0x55c77274dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273fea0 0x55c77274dfe0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9b800079e0 tx=0x7f9b80026900 comp rx=0 tx=0).stop 2024-05-08T21:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/1474095770 shutdown_connections 2024-05-08T21:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x55c7727ab680 0x55c7727adaa0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c772740860 0x55c77274eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c77273f540 0x55c77274daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b9bddbb80 1 --2- 172.21.15.33:0/1474095770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c77273fea0 0x55c77274dfe0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b717fa700 0 client.27631 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/1474095770 >> 172.21.15.33:0/1474095770 conn(0x55c77269d210 msgr2=0x55c772746d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/1474095770 shutdown_connections 2024-05-08T21:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.864+0000 7f9b9bddbb80 1 -- 172.21.15.33:0/1474095770 wait complete. 2024-05-08T21:27:16.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b9bddbb80 20 client.27631 trim_cache size 0 max 16384 2024-05-08T21:27:16.868 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:27:16.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b8fc21700 1 -- 172.21.15.33:0/2338859785 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9b740a1270 msgr2=0x7f9b740a3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b8fc21700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9b740a1270 0x7f9b740a3730 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9b800094f0 tx=0x7f9b800175d0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b8fc21700 1 -- 172.21.15.33:0/2338859785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b8810d2e0 msgr2=0x7f9b881a48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:16.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b8fc21700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b8810d2e0 0x7f9b881a48d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400b3c0 tx=0x7f9b8400b6d0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b8fc21700 1 -- 172.21.15.33:0/2338859785 shutdown_connections 2024-05-08T21:27:16.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b8fc21700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9b740a1270 0x7f9b740a3730 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b8fc21700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b8810d2e0 0x7f9b881a48d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b8fc21700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b88107fa0 0x7f9b8819d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b8fc21700 1 --2- 172.21.15.33:0/2338859785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b8810a910 0x7f9b8819d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:16.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b8fc21700 1 -- 172.21.15.33:0/2338859785 >> 172.21.15.33:0/2338859785 conn(0x7f9b8806dcb0 msgr2=0x7f9b881087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:16.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b8fc21700 1 -- 172.21.15.33:0/2338859785 shutdown_connections 2024-05-08T21:27:16.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:16.865+0000 7f9b8fc21700 1 -- 172.21.15.33:0/2338859785 wait complete. 2024-05-08T21:27:16.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:27:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.115+0000 7fcaaed06700 1 -- 172.21.15.33:0/428854512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa810f330 msgr2=0x7fcaa81117a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.115+0000 7fcaaed06700 1 --2- 172.21.15.33:0/428854512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa810f330 0x7fcaa81117a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000b0a0 tx=0x7fcaa000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.115+0000 7fcaaed06700 1 -- 172.21.15.33:0/428854512 shutdown_connections 2024-05-08T21:27:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.115+0000 7fcaaed06700 1 --2- 172.21.15.33:0/428854512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa810f330 0x7fcaa81117a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.115+0000 7fcaaed06700 1 --2- 172.21.15.33:0/428854512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaa8072ce0 0x7fcaa8107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.115+0000 7fcaaed06700 1 --2- 172.21.15.33:0/428854512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaa81084e0 0x7fcaa8108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.115+0000 7fcaaed06700 1 -- 172.21.15.33:0/428854512 >> 172.21.15.33:0/428854512 conn(0x7fcaa806dcb0 msgr2=0x7fcaa80700f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.115+0000 7fcaaed06700 1 -- 172.21.15.33:0/428854512 shutdown_connections 2024-05-08T21:27:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.115+0000 7fcaaed06700 1 -- 172.21.15.33:0/428854512 wait complete. 2024-05-08T21:27:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaaed06700 1 Processor -- start 2024-05-08T21:27:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaaed06700 1 -- start start 2024-05-08T21:27:17.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaaed06700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8072ce0 0x7fcaa819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaaed06700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaa81084e0 0x7fcaa819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaaed06700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaa810f330 0x7fcaa81a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaaed06700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa81a7070 con 0x7fcaa8072ce0 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaaed06700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa81a71b0 con 0x7fcaa81084e0 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaaed06700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa81a7340 con 0x7fcaa810f330 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaacaa2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8072ce0 0x7fcaa819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaa7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaa81084e0 0x7fcaa819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaacaa2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8072ce0 0x7fcaa819f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40276/0 (socket says 172.21.15.33:40276) 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaacaa2700 1 -- 172.21.15.33:0/2536010279 learned_addr learned my addr 172.21.15.33:0/2536010279 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaad2a3700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaa810f330 0x7fcaa81a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaa7fff700 1 -- 172.21.15.33:0/2536010279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaa810f330 msgr2=0x7fcaa81a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaa7fff700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaa810f330 0x7fcaa81a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaa7fff700 1 -- 172.21.15.33:0/2536010279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8072ce0 msgr2=0x7fcaa819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:17.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaa7fff700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8072ce0 0x7fcaa819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaa7fff700 1 -- 172.21.15.33:0/2536010279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa0009d00 con 0x7fcaa81084e0 2024-05-08T21:27:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.116+0000 7fcaacaa2700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8072ce0 0x7fcaa819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.117+0000 7fcaa7fff700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaa81084e0 0x7fcaa819fa80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fca94009c50 tx=0x7fca94009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.117+0000 7fcaa5ffb700 1 -- 172.21.15.33:0/2536010279 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94015070 con 0x7fcaa81084e0 2024-05-08T21:27:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.117+0000 7fcaaed06700 1 -- 172.21.15.33:0/2536010279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa8109110 con 0x7fcaa81084e0 2024-05-08T21:27:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.117+0000 7fcaaed06700 1 -- 172.21.15.33:0/2536010279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa8109670 con 0x7fcaa81084e0 2024-05-08T21:27:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.117+0000 7fcaa5ffb700 1 -- 172.21.15.33:0/2536010279 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca9400be60 con 0x7fcaa81084e0 2024-05-08T21:27:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.117+0000 7fcaa5ffb700 1 -- 172.21.15.33:0/2536010279 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca940183b0 con 0x7fcaa81084e0 2024-05-08T21:27:17.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.120+0000 7fcaa5ffb700 1 -- 172.21.15.33:0/2536010279 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fca94018510 con 0x7fcaa81084e0 2024-05-08T21:27:17.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.120+0000 7fcaa5ffb700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fca980a1170 0x7fca980a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.121+0000 7fcaa5ffb700 1 -- 172.21.15.33:0/2536010279 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fca940d01b0 con 0x7fcaa81084e0 2024-05-08T21:27:17.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.121+0000 7fcaacaa2700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fca980a1170 0x7fca980a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.121+0000 7fcabaec0b80 1 Processor -- start 2024-05-08T21:27:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.121+0000 7fcabaec0b80 1 -- start start 2024-05-08T21:27:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.122+0000 7fcabaec0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8d540 0x55c4b8c8d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.122+0000 7fcabaec0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c8e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.122+0000 7fcabaec0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8e860 0x55c4b8c8ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.122+0000 7fcabaec0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b8be4150 con 0x55c4b8c8d540 2024-05-08T21:27:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.122+0000 7fcabaec0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b8be1b10 con 0x55c4b8c8e860 2024-05-08T21:27:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.122+0000 7fcabaec0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b8bdd400 con 0x55c4b8c8dea0 2024-05-08T21:27:17.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.122+0000 7fcaad2a3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8e860 0x55c4b8c8ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.122+0000 7fcaad2a3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8e860 0x55c4b8c8ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41528/0 (socket says 172.21.15.33:41528) 2024-05-08T21:27:17.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.122+0000 7fcaad2a3700 1 -- 172.21.15.33:0/32405555 learned_addr learned my addr 172.21.15.33:0/32405555 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:17.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.122+0000 7fcaacaa2700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fca980a1170 0x7fca980a3630 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00f4d0 tx=0x7fca9c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:17.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.123+0000 7fcaacaa2700 1 --2- 172.21.15.33:0/32405555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8d540 0x55c4b8c8d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.123+0000 7fcaad2a3700 1 -- 172.21.15.33:0/32405555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8dea0 msgr2=0x55c4b8c8e320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:27:17.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.123+0000 7fcaad2a3700 1 --2- 172.21.15.33:0/32405555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c8e320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.123+0000 7fcaad2a3700 1 -- 172.21.15.33:0/32405555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8d540 msgr2=0x55c4b8c8d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:17.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.123+0000 7fcaad2a3700 1 --2- 172.21.15.33:0/32405555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8d540 0x55c4b8c8d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.123+0000 7fcaad2a3700 1 -- 172.21.15.33:0/32405555 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4b8a5e3b0 con 0x55c4b8c8e860 2024-05-08T21:27:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.123+0000 7fcaad2a3700 1 --2- 172.21.15.33:0/32405555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8e860 0x55c4b8c8ece0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000b0a0 tx=0x7fcaa00094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=805b77f5e8d59cab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.124+0000 7fca92ffd700 1 -- 172.21.15.33:0/32405555 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000e040 con 0x55c4b8c8e860 2024-05-08T21:27:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.124+0000 7fca92ffd700 1 -- 172.21.15.33:0/32405555 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcaa00076e0 con 0x55c4b8c8e860 2024-05-08T21:27:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.124+0000 7fca92ffd700 1 -- 172.21.15.33:0/32405555 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa001c440 con 0x55c4b8c8e860 2024-05-08T21:27:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.125+0000 7fcabaec0b80 1 -- 172.21.15.33:0/32405555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8e860 msgr2=0x55c4b8c8ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.125+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/32405555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8e860 0x55c4b8c8ece0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000b0a0 tx=0x7fcaa00094b0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.125+0000 7fcabaec0b80 1 -- 172.21.15.33:0/32405555 shutdown_connections 2024-05-08T21:27:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.125+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/32405555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8e860 0x55c4b8c8ece0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.125+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/32405555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8d540 0x55c4b8c8d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.125+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/32405555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c8e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.125+0000 7fcabaec0b80 1 -- 172.21.15.33:0/32405555 >> 172.21.15.33:0/32405555 conn(0x55c4b8beb210 msgr2=0x55c4b8c87de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.126+0000 7fcabaec0b80 1 -- 172.21.15.33:0/32405555 shutdown_connections 2024-05-08T21:27:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.126+0000 7fcabaec0b80 1 -- 172.21.15.33:0/32405555 wait complete. 2024-05-08T21:27:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.126+0000 7fcabaec0b80 1 -- start start 2024-05-08T21:27:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcabaec0b80 1 Processor -- start 2024-05-08T21:27:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcabaec0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8c96e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcabaec0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c973a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcabaec0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 0x55c4b8c9e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcabaec0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b8b79560 con 0x55c4b8c8e860 2024-05-08T21:27:17.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcabaec0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b8b79200 con 0x55c4b8c8dea0 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcabaec0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b8a606a0 con 0x55c4b8c8d540 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcabaec0b80 10 client.0 _ll_register_callbacks cb 0x55c4b8b82b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcaa7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcaacaa2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8c96e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcaa7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41540/0 (socket says 172.21.15.33:41540) 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcaa7fff700 1 -- 172.21.15.33:0/152640024 learned_addr learned my addr 172.21.15.33:0/152640024 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.127+0000 7fcaad2a3700 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 0x55c4b8c9e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.128+0000 7fcaacaa2700 1 -- 172.21.15.33:0/152640024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 msgr2=0x55c4b8c973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.128+0000 7fcaacaa2700 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.128+0000 7fcaacaa2700 1 -- 172.21.15.33:0/152640024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 msgr2=0x55c4b8c9e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.128+0000 7fcaacaa2700 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 0x55c4b8c9e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.128+0000 7fcaacaa2700 1 -- 172.21.15.33:0/152640024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa81123c0 con 0x55c4b8c8d540 2024-05-08T21:27:17.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.128+0000 7fcaa7fff700 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.128+0000 7fcaacaa2700 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8c96e60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00abf0 tx=0x7fca9c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b53052bd9a51aa52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.128+0000 7fca87fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.128+0000 7fca87fff700 1 -- 172.21.15.33:0/152640024 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c021070 con 0x55c4b8c8d540 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.128+0000 7fcabaec0b80 1 -- 172.21.15.33:0/152640024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4b8b36270 con 0x55c4b8c8d540 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.128+0000 7fca87fff700 1 -- 172.21.15.33:0/152640024 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fca9c01c990 con 0x55c4b8c8d540 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.129+0000 7fca87fff700 1 -- 172.21.15.33:0/152640024 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca9c01cb50 con 0x55c4b8c8d540 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.129+0000 7fcabaec0b80 10 client.27640 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.129+0000 7fcabaec0b80 1 -- 172.21.15.33:0/152640024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c4b8a0ab60 con 0x55c4b8c8d540 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.129+0000 7fca87fff700 1 -- 172.21.15.33:0/152640024 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c4b8b36270 con 0x55c4b8c8d540 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.129+0000 7fca87fff700 1 -- 172.21.15.33:0/152640024 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca9c02d070 con 0x55c4b8c8d540 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.129+0000 7fcabaec0b80 10 client.27640 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.129+0000 7fcabaec0b80 10 client.27640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:17.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.129+0000 7fcabaec0b80 20 client.27640 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:17.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.129+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c4b8ca2cb0 0x55c4b8ca50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.129+0000 7fcabaec0b80 4 client.27640 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:17.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.129+0000 7fcabaec0b80 1 -- 172.21.15.33:0/152640024 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4b8c5c810 con 0x55c4b8ca2cb0 2024-05-08T21:27:17.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.129+0000 7fcaa7fff700 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c4b8ca2cb0 0x55c4b8ca50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.130+0000 7fca87fff700 10 client.27640 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:17.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.130+0000 7fcaa7fff700 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c4b8ca2cb0 0x55c4b8ca50d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eddea1c02f23c7a1 server_cookie=b3a046e4f09e72a3 in_seq=0 out_seq=0 2024-05-08T21:27:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.130+0000 7fca87fff700 1 -- 172.21.15.33:0/152640024 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c4b8c5c810 con 0x55c4b8ca2cb0 2024-05-08T21:27:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.130+0000 7fca87fff700 10 client.27640 handle_command_reply: tid=0 2024-05-08T21:27:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.130+0000 7fcabaec0b80 1 client.27640 shutdown 2024-05-08T21:27:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.130+0000 7fcabaec0b80 1 -- 172.21.15.33:0/152640024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 msgr2=0x55c4b8c96e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.130+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8c96e60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00abf0 tx=0x7fca9c0047c0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.130+0000 7fcabaec0b80 1 -- 172.21.15.33:0/152640024 shutdown_connections 2024-05-08T21:27:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.130+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c4b8ca2cb0 0x55c4b8ca50d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.131+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 0x55c4b8c9e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.131+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8c96e60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.131+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/152640024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.131+0000 7fca87fff700 0 client.27640 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:17.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.131+0000 7fcabaec0b80 1 -- 172.21.15.33:0/152640024 >> 172.21.15.33:0/152640024 conn(0x55c4b8beb210 msgr2=0x55c4b8c875a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:17.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.131+0000 7fcabaec0b80 1 -- 172.21.15.33:0/152640024 shutdown_connections 2024-05-08T21:27:17.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.131+0000 7fcabaec0b80 1 -- 172.21.15.33:0/152640024 wait complete. 2024-05-08T21:27:17.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.131+0000 7fcabaec0b80 20 client.27640 trim_cache size 0 max 16384 2024-05-08T21:27:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.151+0000 7fcabaec0b80 1 Processor -- start 2024-05-08T21:27:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.151+0000 7fcabaec0b80 1 -- start start 2024-05-08T21:27:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.151+0000 7fcabaec0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8ca50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.151+0000 7fcabaec0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c9e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.151+0000 7fcabaec0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 0x55c4b8c94dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.151+0000 7fcabaec0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b8c82870 con 0x55c4b8c8e860 2024-05-08T21:27:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.151+0000 7fcabaec0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b8ca01a0 con 0x55c4b8c8dea0 2024-05-08T21:27:17.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.151+0000 7fcabaec0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b8c94a50 con 0x55c4b8c8d540 2024-05-08T21:27:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.151+0000 7fcaa7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c9e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.151+0000 7fcaa7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c9e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41542/0 (socket says 172.21.15.33:41542) 2024-05-08T21:27:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.151+0000 7fcaa7fff700 1 -- 172.21.15.33:0/1581301727 learned_addr learned my addr 172.21.15.33:0/1581301727 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.151+0000 7fcaacaa2700 1 --2- 172.21.15.33:0/1581301727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8ca50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.152+0000 7fcaa7fff700 1 -- 172.21.15.33:0/1581301727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 msgr2=0x55c4b8ca50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.152+0000 7fcaa7fff700 1 --2- 172.21.15.33:0/1581301727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8ca50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.152+0000 7fcaa7fff700 1 -- 172.21.15.33:0/1581301727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 msgr2=0x55c4b8c94dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.152+0000 7fcaa7fff700 1 --2- 172.21.15.33:0/1581301727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 0x55c4b8c94dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.152+0000 7fcaa7fff700 1 -- 172.21.15.33:0/1581301727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4b8ba3560 con 0x55c4b8c8dea0 2024-05-08T21:27:17.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.152+0000 7fcaa7fff700 1 --2- 172.21.15.33:0/1581301727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c9e3a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fca94099820 tx=0x7fca940cd7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=846701ad7ba49802 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.152+0000 7fca937fe700 1 -- 172.21.15.33:0/1581301727 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca940a56c0 con 0x55c4b8c8dea0 2024-05-08T21:27:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.152+0000 7fca937fe700 1 -- 172.21.15.33:0/1581301727 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca9409b460 con 0x55c4b8c8dea0 2024-05-08T21:27:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.152+0000 7fca937fe700 1 -- 172.21.15.33:0/1581301727 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94099110 con 0x55c4b8c8dea0 2024-05-08T21:27:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.153+0000 7fcabaec0b80 1 -- 172.21.15.33:0/1581301727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 msgr2=0x55c4b8c9e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.153+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/1581301727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c9e3a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fca94099820 tx=0x7fca940cd7c0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.153+0000 7fcabaec0b80 1 -- 172.21.15.33:0/1581301727 shutdown_connections 2024-05-08T21:27:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.153+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/1581301727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 0x55c4b8c94dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.153+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/1581301727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8ca50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.153+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/1581301727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c9e3a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.153+0000 7fcabaec0b80 1 -- 172.21.15.33:0/1581301727 >> 172.21.15.33:0/1581301727 conn(0x55c4b8beb210 msgr2=0x55c4b8c8ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.153+0000 7fcabaec0b80 1 -- 172.21.15.33:0/1581301727 shutdown_connections 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.153+0000 7fcabaec0b80 1 -- 172.21.15.33:0/1581301727 wait complete. 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.153+0000 7fcabaec0b80 1 -- start start 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcabaec0b80 1 Processor -- start 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcabaec0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8c99c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcabaec0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c9a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcabaec0b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 0x55c4b8c9d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcabaec0b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b8c9d5c0 con 0x55c4b8c8e860 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcabaec0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b8c9d730 con 0x55c4b8c8dea0 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcabaec0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4b8c9d8f0 con 0x55c4b8c8d540 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcabaec0b80 10 client.0 _ll_register_callbacks cb 0x55c4b8b82b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:17.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcaacaa2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8c99c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcaa7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c9a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcaacaa2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8c99c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:56584/0 (socket says 172.21.15.33:56584) 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcaacaa2700 1 -- 172.21.15.33:0/775980877 learned_addr learned my addr 172.21.15.33:0/775980877 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcaa7fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c9a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41548/0 (socket says 172.21.15.33:41548) 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.154+0000 7fcaad2a3700 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 0x55c4b8c9d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fcaacaa2700 1 -- 172.21.15.33:0/775980877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 msgr2=0x55c4b8c9a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fcaacaa2700 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c9a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fcaacaa2700 1 -- 172.21.15.33:0/775980877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 msgr2=0x55c4b8c9d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fcaacaa2700 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 0x55c4b8c9d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fcaacaa2700 1 -- 172.21.15.33:0/775980877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa8109670 con 0x55c4b8c8d540 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fcaa7fff700 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c9a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fcaacaa2700 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8c99c30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00abf0 tx=0x7fca9c0046f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17182dbe0c92acc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fca87fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fca87fff700 1 -- 172.21.15.33:0/775980877 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c021070 con 0x55c4b8c8d540 2024-05-08T21:27:17.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fcabaec0b80 1 -- 172.21.15.33:0/775980877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaa00076e0 con 0x55c4b8c8d540 2024-05-08T21:27:17.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fca87fff700 1 -- 172.21.15.33:0/775980877 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fca9c00ad40 con 0x55c4b8c8d540 2024-05-08T21:27:17.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fca87fff700 1 -- 172.21.15.33:0/775980877 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca9c02e3a0 con 0x55c4b8c8d540 2024-05-08T21:27:17.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fcabaec0b80 10 client.27646 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:17.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fcabaec0b80 1 -- 172.21.15.33:0/775980877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55c4b8a62cd0 con 0x55c4b8c8d540 2024-05-08T21:27:17.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.155+0000 7fca87fff700 1 -- 172.21.15.33:0/775980877 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcaa00076e0 con 0x55c4b8c8d540 2024-05-08T21:27:17.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.156+0000 7fca87fff700 1 -- 172.21.15.33:0/775980877 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca9c018040 con 0x55c4b8c8d540 2024-05-08T21:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.156+0000 7fcabaec0b80 10 client.27646 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.156+0000 7fcabaec0b80 10 client.27646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.156+0000 7fcabaec0b80 20 client.27646 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.156+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c4b8cf9b30 0x55c4b8cfbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.156+0000 7fcabaec0b80 4 client.27646 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.156+0000 7fcabaec0b80 1 -- 172.21.15.33:0/775980877 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c4b8bd5ec0 con 0x55c4b8cf9b30 2024-05-08T21:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.156+0000 7fcaa7fff700 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c4b8cf9b30 0x55c4b8cfbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fca87fff700 10 client.27646 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcaa7fff700 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c4b8cf9b30 0x55c4b8cfbf50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=507a065e34894a51 server_cookie=4ac99e63ca560692 in_seq=0 out_seq=0 2024-05-08T21:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fca87fff700 1 -- 172.21.15.33:0/775980877 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c4b8bd5ec0 con 0x55c4b8cf9b30 2024-05-08T21:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fca87fff700 10 client.27646 handle_command_reply: tid=0 2024-05-08T21:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcabaec0b80 1 client.27646 shutdown 2024-05-08T21:27:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcabaec0b80 1 -- 172.21.15.33:0/775980877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 msgr2=0x55c4b8c99c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8c99c30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00abf0 tx=0x7fca9c0046f0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcabaec0b80 1 -- 172.21.15.33:0/775980877 shutdown_connections 2024-05-08T21:27:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55c4b8cf9b30 0x55c4b8cfbf50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4b8c8e860 0x55c4b8c9d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4b8c8d540 0x55c4b8c99c30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcabaec0b80 1 --2- 172.21.15.33:0/775980877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4b8c8dea0 0x55c4b8c9a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcabaec0b80 1 -- 172.21.15.33:0/775980877 >> 172.21.15.33:0/775980877 conn(0x55c4b8beb210 msgr2=0x55c4b8c94c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fca87fff700 0 client.27646 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcabaec0b80 1 -- 172.21.15.33:0/775980877 shutdown_connections 2024-05-08T21:27:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcabaec0b80 1 -- 172.21.15.33:0/775980877 wait complete. 2024-05-08T21:27:17.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.157+0000 7fcabaec0b80 20 client.27646 trim_cache size 0 max 16384 2024-05-08T21:27:17.164 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:27:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.158+0000 7fcaaed06700 1 -- 172.21.15.33:0/2536010279 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fca980a1170 msgr2=0x7fca980a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.158+0000 7fcaaed06700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fca980a1170 0x7fca980a3630 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00f4d0 tx=0x7fca9c005f90 comp rx=0 tx=0).stop 2024-05-08T21:27:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.158+0000 7fcaaed06700 1 -- 172.21.15.33:0/2536010279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaa81084e0 msgr2=0x7fcaa819fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:17.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.158+0000 7fcaaed06700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaa81084e0 0x7fcaa819fa80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fca94009c50 tx=0x7fca94009f60 comp rx=0 tx=0).stop 2024-05-08T21:27:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.158+0000 7fcaaed06700 1 -- 172.21.15.33:0/2536010279 shutdown_connections 2024-05-08T21:27:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.158+0000 7fcaaed06700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcaa810f330 0x7fcaa81a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.158+0000 7fcaaed06700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaa8072ce0 0x7fcaa819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.158+0000 7fcaaed06700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fca980a1170 0x7fca980a3630 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.158+0000 7fcaaed06700 1 --2- 172.21.15.33:0/2536010279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcaa81084e0 0x7fcaa819fa80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.158+0000 7fcaaed06700 1 -- 172.21.15.33:0/2536010279 >> 172.21.15.33:0/2536010279 conn(0x7fcaa806dcb0 msgr2=0x7fcaa810b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.158+0000 7fcaaed06700 1 -- 172.21.15.33:0/2536010279 shutdown_connections 2024-05-08T21:27:17.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:17.158+0000 7fcaaed06700 1 -- 172.21.15.33:0/2536010279 wait complete. 2024-05-08T21:27:17.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:17 smithi071 ceph-mon[47074]: pgmap v539: 129 pgs: 129 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 545 op/s 2024-05-08T21:27:17.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:17 smithi082 ceph-mon[47512]: pgmap v539: 129 pgs: 129 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 545 op/s 2024-05-08T21:27:17.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:17 smithi033 ceph-mon[42528]: pgmap v539: 129 pgs: 129 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 545 op/s 2024-05-08T21:27:18.182 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:27:18.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:27:18.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.367+0000 7f34379d9700 1 -- 172.21.15.33:0/1352554569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3430100f50 msgr2=0x7f34301013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.367+0000 7f34379d9700 1 --2- 172.21.15.33:0/1352554569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3430100f50 0x7f34301013b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3420009990 tx=0x7f3420009ca0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.367+0000 7f34379d9700 1 -- 172.21.15.33:0/1352554569 shutdown_connections 2024-05-08T21:27:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.367+0000 7f34379d9700 1 --2- 172.21.15.33:0/1352554569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34301018f0 0x7f34301081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.367+0000 7f34379d9700 1 --2- 172.21.15.33:0/1352554569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34300ffd60 0x7f3430100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.367+0000 7f34379d9700 1 --2- 172.21.15.33:0/1352554569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3430100f50 0x7f34301013b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.367+0000 7f34379d9700 1 -- 172.21.15.33:0/1352554569 >> 172.21.15.33:0/1352554569 conn(0x7f34300fb290 msgr2=0x7f34300fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.367+0000 7f34379d9700 1 -- 172.21.15.33:0/1352554569 shutdown_connections 2024-05-08T21:27:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.367+0000 7f34379d9700 1 -- 172.21.15.33:0/1352554569 wait complete. 2024-05-08T21:27:18.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.368+0000 7f34379d9700 1 Processor -- start 2024-05-08T21:27:18.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.368+0000 7f34379d9700 1 -- start start 2024-05-08T21:27:18.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.368+0000 7f34379d9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34300ffd60 0x7f3430198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.368+0000 7f34379d9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3430100f50 0x7f3430199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.368+0000 7f34379d9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34301018f0 0x7f34301a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.368+0000 7f34379d9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34301a0a40 con 0x7f3430100f50 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.368+0000 7f34379d9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34301a0b80 con 0x7f34300ffd60 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.368+0000 7f34379d9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34301a0d10 con 0x7f34301018f0 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.368+0000 7f3435775700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34300ffd60 0x7f3430198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.368+0000 7f3435775700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34300ffd60 0x7f3430198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41554/0 (socket says 172.21.15.33:41554) 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.368+0000 7f3435775700 1 -- 172.21.15.33:0/300656256 learned_addr learned my addr 172.21.15.33:0/300656256 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.368+0000 7f3434f74700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3430100f50 0x7f3430199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f3435f76700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34301018f0 0x7f34301a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f3435775700 1 -- 172.21.15.33:0/300656256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34301018f0 msgr2=0x7f34301a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f3435775700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34301018f0 0x7f34301a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f3435775700 1 -- 172.21.15.33:0/300656256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3430100f50 msgr2=0x7f3430199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f3435775700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3430100f50 0x7f3430199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f3435775700 1 -- 172.21.15.33:0/300656256 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34200095f0 con 0x7f34300ffd60 2024-05-08T21:27:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f3435f76700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34301018f0 0x7f34301a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:27:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f3435775700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34300ffd60 0x7f3430198f10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d980 tx=0x7f342c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f34267fc700 1 -- 172.21.15.33:0/300656256 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c004020 con 0x7f34300ffd60 2024-05-08T21:27:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f34267fc700 1 -- 172.21.15.33:0/300656256 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f342c009730 con 0x7f34300ffd60 2024-05-08T21:27:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f34379d9700 1 -- 172.21.15.33:0/300656256 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34301a0ff0 con 0x7f34300ffd60 2024-05-08T21:27:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f34267fc700 1 -- 172.21.15.33:0/300656256 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c003a00 con 0x7f34300ffd60 2024-05-08T21:27:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.369+0000 7f34379d9700 1 -- 172.21.15.33:0/300656256 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34301a1410 con 0x7f34300ffd60 2024-05-08T21:27:18.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.372+0000 7f34267fc700 1 -- 172.21.15.33:0/300656256 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f342c0098a0 con 0x7f34300ffd60 2024-05-08T21:27:18.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.373+0000 7f34267fc700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f341c0a1120 0x7f341c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.373+0000 7f34267fc700 1 -- 172.21.15.33:0/300656256 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f342c0cc910 con 0x7f34300ffd60 2024-05-08T21:27:18.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.373+0000 7f3434f74700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f341c0a1120 0x7f341c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.374+0000 7f3434f74700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f341c0a1120 0x7f341c0a35e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f342000b5c0 tx=0x7f3420003720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:18.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.374+0000 7f3443b93b80 1 Processor -- start 2024-05-08T21:27:18.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.374+0000 7f3443b93b80 1 -- start start 2024-05-08T21:27:18.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.374+0000 7f3443b93b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5a5c0 0x55ad46a5a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.374+0000 7f3443b93b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 0x55ad46a5b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.374+0000 7f3443b93b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 0x55ad46a5bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.374+0000 7f3443b93b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad469b11d0 con 0x55ad46a5b8e0 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.374+0000 7f3443b93b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad469aeb90 con 0x55ad46a5a5c0 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.374+0000 7f3443b93b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad469aa480 con 0x55ad46a5af20 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.374+0000 7f3435f76700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 0x55ad46a5bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.374+0000 7f3435f76700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 0x55ad46a5bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40354/0 (socket says 172.21.15.33:40354) 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.374+0000 7f3435f76700 1 -- 172.21.15.33:0/1334162028 learned_addr learned my addr 172.21.15.33:0/1334162028 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.375+0000 7f3434f74700 1 --2- 172.21.15.33:0/1334162028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 0x55ad46a5b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.375+0000 7f3435775700 1 --2- 172.21.15.33:0/1334162028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5a5c0 0x55ad46a5a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.375+0000 7f3435f76700 1 -- 172.21.15.33:0/1334162028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 msgr2=0x55ad46a5b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.375+0000 7f3435f76700 1 --2- 172.21.15.33:0/1334162028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 0x55ad46a5b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.375+0000 7f3435f76700 1 -- 172.21.15.33:0/1334162028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5a5c0 msgr2=0x55ad46a5a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.375+0000 7f3435f76700 1 --2- 172.21.15.33:0/1334162028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5a5c0 0x55ad46a5a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.375+0000 7f3435f76700 1 -- 172.21.15.33:0/1334162028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad4682b430 con 0x55ad46a5b8e0 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.375+0000 7f3434f74700 1 --2- 172.21.15.33:0/1334162028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 0x55ad46a5b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.375+0000 7f3435f76700 1 --2- 172.21.15.33:0/1334162028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 0x55ad46a5bd60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3428009e50 tx=0x7f34280077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7860c2f6cfbe841f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.375+0000 7f341b7fe700 1 -- 172.21.15.33:0/1334162028 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428014610 con 0x55ad46a5b8e0 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.375+0000 7f341b7fe700 1 -- 172.21.15.33:0/1334162028 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f342800f050 con 0x55ad46a5b8e0 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.375+0000 7f341b7fe700 1 -- 172.21.15.33:0/1334162028 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428013990 con 0x55ad46a5b8e0 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.376+0000 7f3443b93b80 1 -- 172.21.15.33:0/1334162028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 msgr2=0x55ad46a5bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.376+0000 7f3443b93b80 1 --2- 172.21.15.33:0/1334162028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 0x55ad46a5bd60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3428009e50 tx=0x7f34280077e0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.376+0000 7f3443b93b80 1 -- 172.21.15.33:0/1334162028 shutdown_connections 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.376+0000 7f3443b93b80 1 --2- 172.21.15.33:0/1334162028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 0x55ad46a5bd60 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.376+0000 7f3443b93b80 1 --2- 172.21.15.33:0/1334162028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5a5c0 0x55ad46a5a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.376+0000 7f3443b93b80 1 --2- 172.21.15.33:0/1334162028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 0x55ad46a5b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.376+0000 7f3443b93b80 1 -- 172.21.15.33:0/1334162028 >> 172.21.15.33:0/1334162028 conn(0x55ad469b8290 msgr2=0x55ad46a54e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.376+0000 7f3443b93b80 1 -- 172.21.15.33:0/1334162028 shutdown_connections 2024-05-08T21:27:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.376+0000 7f3443b93b80 1 -- 172.21.15.33:0/1334162028 wait complete. 2024-05-08T21:27:18.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.376+0000 7f3443b93b80 1 -- start start 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3443b93b80 1 Processor -- start 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3443b93b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5a5c0 0x55ad46a63e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3443b93b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 0x55ad46a64380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3443b93b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5b8e0 0x55ad46a6b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3443b93b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad469465e0 con 0x55ad46a5a5c0 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3443b93b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad46946280 con 0x55ad46a5b8e0 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3443b93b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad4682d720 con 0x55ad46a5af20 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3443b93b80 10 client.0 _ll_register_callbacks cb 0x55ad4694fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3435775700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5a5c0 0x55ad46a63e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3435775700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5a5c0 0x55ad46a63e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40370/0 (socket says 172.21.15.33:40370) 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3435775700 1 -- 172.21.15.33:0/3111369738 learned_addr learned my addr 172.21.15.33:0/3111369738 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3434f74700 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 0x55ad46a64380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3435f76700 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5b8e0 0x55ad46a6b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3435775700 1 -- 172.21.15.33:0/3111369738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 msgr2=0x55ad46a64380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.377+0000 7f3435775700 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 0x55ad46a64380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3435775700 1 -- 172.21.15.33:0/3111369738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5b8e0 msgr2=0x55ad46a6b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3435775700 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5b8e0 0x55ad46a6b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3435775700 1 -- 172.21.15.33:0/3111369738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3430108af0 con 0x55ad46a5a5c0 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3435775700 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5a5c0 0x55ad46a63e40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f342c0046a0 tx=0x7f342c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8707ab28f4cfcae7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3418ff9700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3418ff9700 1 -- 172.21.15.33:0/3111369738 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c0999e0 con 0x55ad46a5a5c0 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3418ff9700 1 -- 172.21.15.33:0/3111369738 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f342c09d810 con 0x55ad46a5a5c0 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3443b93b80 1 -- 172.21.15.33:0/3111369738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad469032f0 con 0x55ad46a5a5c0 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3418ff9700 1 -- 172.21.15.33:0/3111369738 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f342c0a1e60 con 0x55ad46a5a5c0 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3443b93b80 10 client.17748 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3443b93b80 1 -- 172.21.15.33:0/3111369738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f3428009a80 con 0x55ad46a5a5c0 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3418ff9700 1 -- 172.21.15.33:0/3111369738 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f342c063030 con 0x55ad46a5a5c0 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.378+0000 7f3418ff9700 1 -- 172.21.15.33:0/3111369738 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f342c099cd0 con 0x55ad46a5a5c0 2024-05-08T21:27:18.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.379+0000 7f3443b93b80 10 client.17748 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.379+0000 7f3443b93b80 10 client.17748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.379+0000 7f3443b93b80 20 client.17748 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.379+0000 7f3443b93b80 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ad46a6fba0 0x55ad46a71fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.379+0000 7f3443b93b80 4 client.17748 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.379+0000 7f3443b93b80 1 -- 172.21.15.33:0/3111369738 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad46a29890 con 0x55ad46a6fba0 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.379+0000 7f3434f74700 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ad46a6fba0 0x55ad46a71fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.379+0000 7f3434f74700 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ad46a6fba0 0x55ad46a71fc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=602b2d03e4857866 server_cookie=9e647e0f3590ccff in_seq=0 out_seq=0 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.379+0000 7f3418ff9700 10 client.17748 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3418ff9700 1 -- 172.21.15.33:0/3111369738 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad46a29890 con 0x55ad46a6fba0 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3418ff9700 10 client.17748 handle_command_reply: tid=0 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3443b93b80 1 client.17748 shutdown 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3443b93b80 1 -- 172.21.15.33:0/3111369738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5a5c0 msgr2=0x55ad46a63e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3443b93b80 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5a5c0 0x55ad46a63e40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f342c0046a0 tx=0x7f342c009b80 comp rx=0 tx=0).stop 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3443b93b80 1 -- 172.21.15.33:0/3111369738 shutdown_connections 2024-05-08T21:27:18.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3443b93b80 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ad46a6fba0 0x55ad46a71fc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3443b93b80 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5b8e0 0x55ad46a6b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3443b93b80 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5a5c0 0x55ad46a63e40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3443b93b80 1 --2- 172.21.15.33:0/3111369738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 0x55ad46a64380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3443b93b80 1 -- 172.21.15.33:0/3111369738 >> 172.21.15.33:0/3111369738 conn(0x55ad469b8290 msgr2=0x55ad46a593c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3418ff9700 0 client.17748 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3443b93b80 1 -- 172.21.15.33:0/3111369738 shutdown_connections 2024-05-08T21:27:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3443b93b80 1 -- 172.21.15.33:0/3111369738 wait complete. 2024-05-08T21:27:18.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.380+0000 7f3443b93b80 20 client.17748 trim_cache size 0 max 16384 2024-05-08T21:27:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.412+0000 7f3443b93b80 1 Processor -- start 2024-05-08T21:27:18.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.412+0000 7f3443b93b80 1 -- start start 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3443b93b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5a5c0 0x55ad46a62b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3443b93b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5af20 0x55ad46a6b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3443b93b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5b8e0 0x55ad46a76920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3443b93b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad46a565f0 con 0x55ad46a5af20 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3443b93b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad46a6c540 con 0x55ad46a5b8e0 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3443b93b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad46a55c30 con 0x55ad46a5a5c0 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3434f74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5af20 0x55ad46a6b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3434f74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5af20 0x55ad46a6b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40372/0 (socket says 172.21.15.33:40372) 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3434f74700 1 -- 172.21.15.33:0/3263975448 learned_addr learned my addr 172.21.15.33:0/3263975448 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3435775700 1 --2- 172.21.15.33:0/3263975448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5a5c0 0x55ad46a62b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3435f76700 1 --2- 172.21.15.33:0/3263975448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5b8e0 0x55ad46a76920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3434f74700 1 -- 172.21.15.33:0/3263975448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5a5c0 msgr2=0x55ad46a62b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3434f74700 1 --2- 172.21.15.33:0/3263975448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5a5c0 0x55ad46a62b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3434f74700 1 -- 172.21.15.33:0/3263975448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5b8e0 msgr2=0x55ad46a76920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3434f74700 1 --2- 172.21.15.33:0/3263975448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5b8e0 0x55ad46a76920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3434f74700 1 -- 172.21.15.33:0/3263975448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad467d7be0 con 0x55ad46a5af20 2024-05-08T21:27:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.413+0000 7f3435f76700 1 --2- 172.21.15.33:0/3263975448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5b8e0 0x55ad46a76920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:27:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.414+0000 7f3434f74700 1 --2- 172.21.15.33:0/3263975448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5af20 0x55ad46a6b380 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3420009990 tx=0x7f3420020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b144e0e9fd0384aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.414+0000 7f341bfff700 1 -- 172.21.15.33:0/3263975448 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342002b040 con 0x55ad46a5af20 2024-05-08T21:27:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.414+0000 7f341bfff700 1 -- 172.21.15.33:0/3263975448 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f342002c040 con 0x55ad46a5af20 2024-05-08T21:27:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.414+0000 7f341bfff700 1 -- 172.21.15.33:0/3263975448 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420004c50 con 0x55ad46a5af20 2024-05-08T21:27:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.415+0000 7f3443b93b80 1 -- 172.21.15.33:0/3263975448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5af20 msgr2=0x55ad46a6b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.415+0000 7f3443b93b80 1 --2- 172.21.15.33:0/3263975448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5af20 0x55ad46a6b380 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3420009990 tx=0x7f3420020000 comp rx=0 tx=0).stop 2024-05-08T21:27:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.417+0000 7f3443b93b80 1 -- 172.21.15.33:0/3263975448 shutdown_connections 2024-05-08T21:27:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.417+0000 7f3443b93b80 1 --2- 172.21.15.33:0/3263975448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5b8e0 0x55ad46a76920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.417+0000 7f3443b93b80 1 --2- 172.21.15.33:0/3263975448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5a5c0 0x55ad46a62b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.417+0000 7f3443b93b80 1 --2- 172.21.15.33:0/3263975448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5af20 0x55ad46a6b380 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.417+0000 7f3443b93b80 1 -- 172.21.15.33:0/3263975448 >> 172.21.15.33:0/3263975448 conn(0x55ad469b8290 msgr2=0x55ad46a61e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.417+0000 7f3443b93b80 1 -- 172.21.15.33:0/3263975448 shutdown_connections 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.417+0000 7f3443b93b80 1 -- 172.21.15.33:0/3263975448 wait complete. 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.417+0000 7f3443b93b80 1 -- start start 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.417+0000 7f3443b93b80 1 Processor -- start 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.418+0000 7f3443b93b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5a5c0 0x55ad46a674e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.418+0000 7f3443b93b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 0x55ad46a67a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.418+0000 7f3443b93b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 0x55ad46a6aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.418+0000 7f3443b93b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad46a6af80 con 0x55ad46a5b8e0 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.418+0000 7f3443b93b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad46a6b0f0 con 0x55ad46a5a5c0 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.418+0000 7f3443b93b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad46a6b2b0 con 0x55ad46a5af20 2024-05-08T21:27:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.418+0000 7f3443b93b80 10 client.0 _ll_register_callbacks cb 0x55ad4694fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.418+0000 7f3435f76700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 0x55ad46a6aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.418+0000 7f3435775700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5a5c0 0x55ad46a674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.418+0000 7f3435775700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5a5c0 0x55ad46a674e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41602/0 (socket says 172.21.15.33:41602) 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.418+0000 7f3435775700 1 -- 172.21.15.33:0/1575137749 learned_addr learned my addr 172.21.15.33:0/1575137749 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.419+0000 7f3435775700 1 -- 172.21.15.33:0/1575137749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 msgr2=0x55ad46a67a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.419+0000 7f3435775700 1 --2- 172.21.15.33:0/1575137749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 0x55ad46a67a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.419+0000 7f3435775700 1 -- 172.21.15.33:0/1575137749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 msgr2=0x55ad46a6aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.419+0000 7f3435775700 1 --2- 172.21.15.33:0/1575137749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 0x55ad46a6aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.419+0000 7f3435775700 1 -- 172.21.15.33:0/1575137749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342000bd30 con 0x55ad46a5a5c0 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.419+0000 7f3435f76700 1 --2- 172.21.15.33:0/1575137749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 0x55ad46a6aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.419+0000 7f3435775700 1 --2- 172.21.15.33:0/1575137749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5a5c0 0x55ad46a674e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f342c009520 tx=0x7f342c065540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c966298bfece058b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.419+0000 7f3418ff9700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.419+0000 7f3418ff9700 1 -- 172.21.15.33:0/1575137749 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c099e50 con 0x55ad46a5a5c0 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.419+0000 7f3418ff9700 1 -- 172.21.15.33:0/1575137749 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f342c09d810 con 0x55ad46a5a5c0 2024-05-08T21:27:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.419+0000 7f3443b93b80 1 -- 172.21.15.33:0/1575137749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f342800f050 con 0x55ad46a5a5c0 2024-05-08T21:27:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.419+0000 7f3418ff9700 1 -- 172.21.15.33:0/1575137749 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f342c0999e0 con 0x55ad46a5a5c0 2024-05-08T21:27:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.420+0000 7f3443b93b80 10 client.27653 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.420+0000 7f3443b93b80 1 -- 172.21.15.33:0/1575137749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ad469705e0 con 0x55ad46a5a5c0 2024-05-08T21:27:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.420+0000 7f3418ff9700 1 -- 172.21.15.33:0/1575137749 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f342800f050 con 0x55ad46a5a5c0 2024-05-08T21:27:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.420+0000 7f3418ff9700 1 -- 172.21.15.33:0/1575137749 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f342c003b60 con 0x55ad46a5a5c0 2024-05-08T21:27:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.420+0000 7f3443b93b80 10 client.27653 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.420+0000 7f3443b93b80 10 client.27653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.420+0000 7f3443b93b80 20 client.27653 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.420+0000 7f3443b93b80 1 --2- 172.21.15.33:0/1575137749 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ad46ac70a0 0x55ad46ac94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.420+0000 7f3443b93b80 4 client.27653 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:27:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.420+0000 7f3443b93b80 1 -- 172.21.15.33:0/1575137749 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ad469a2f40 con 0x55ad46ac70a0 2024-05-08T21:27:18.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.420+0000 7f3434f74700 1 --2- 172.21.15.33:0/1575137749 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ad46ac70a0 0x55ad46ac94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.420+0000 7f3418ff9700 10 client.27653 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.421+0000 7f3434f74700 1 --2- 172.21.15.33:0/1575137749 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ad46ac70a0 0x55ad46ac94c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c3ccda9c6eb74a server_cookie=3f274a385c5721e9 in_seq=0 out_seq=0 2024-05-08T21:27:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3418ff9700 1 -- 172.21.15.33:0/1575137749 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ad469a2f40 con 0x55ad46ac70a0 2024-05-08T21:27:18.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3418ff9700 10 client.27653 handle_command_reply: tid=0 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3443b93b80 1 client.27653 shutdown 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3443b93b80 1 -- 172.21.15.33:0/1575137749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5a5c0 msgr2=0x55ad46a674e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3443b93b80 1 --2- 172.21.15.33:0/1575137749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5a5c0 0x55ad46a674e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f342c009520 tx=0x7f342c065540 comp rx=0 tx=0).stop 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3443b93b80 1 -- 172.21.15.33:0/1575137749 shutdown_connections 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3443b93b80 1 --2- 172.21.15.33:0/1575137749 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55ad46ac70a0 0x55ad46ac94c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3443b93b80 1 --2- 172.21.15.33:0/1575137749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad46a5b8e0 0x55ad46a6aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3443b93b80 1 --2- 172.21.15.33:0/1575137749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad46a5a5c0 0x55ad46a674e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3443b93b80 1 --2- 172.21.15.33:0/1575137749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad46a5af20 0x55ad46a67a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3443b93b80 1 -- 172.21.15.33:0/1575137749 >> 172.21.15.33:0/1575137749 conn(0x55ad469b8290 msgr2=0x55ad46a76920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3418ff9700 0 client.27653 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3443b93b80 1 -- 172.21.15.33:0/1575137749 shutdown_connections 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.422+0000 7f3443b93b80 1 -- 172.21.15.33:0/1575137749 wait complete. 2024-05-08T21:27:18.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.424+0000 7f3443b93b80 20 client.27653 trim_cache size 0 max 16384 2024-05-08T21:27:18.428 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:27:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:27:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "940f742e-3f9b-4ae5-83c7-866cf22f15ff", 2024-05-08T21:27:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:27:18.428 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:27:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.425+0000 7f34379d9700 1 -- 172.21.15.33:0/300656256 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f341c0a1120 msgr2=0x7f341c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.425+0000 7f34379d9700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f341c0a1120 0x7f341c0a35e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f342000b5c0 tx=0x7f3420003720 comp rx=0 tx=0).stop 2024-05-08T21:27:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.425+0000 7f34379d9700 1 -- 172.21.15.33:0/300656256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34300ffd60 msgr2=0x7f3430198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.425+0000 7f34379d9700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34300ffd60 0x7f3430198f10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d980 tx=0x7f342c00dc90 comp rx=0 tx=0).stop 2024-05-08T21:27:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.425+0000 7f34379d9700 1 -- 172.21.15.33:0/300656256 shutdown_connections 2024-05-08T21:27:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.425+0000 7f34379d9700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34301018f0 0x7f34301a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.425+0000 7f34379d9700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f341c0a1120 0x7f341c0a35e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.425+0000 7f34379d9700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34300ffd60 0x7f3430198f10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.425+0000 7f34379d9700 1 --2- 172.21.15.33:0/300656256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3430100f50 0x7f3430199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.425+0000 7f34379d9700 1 -- 172.21.15.33:0/300656256 >> 172.21.15.33:0/300656256 conn(0x7f34300fb290 msgr2=0x7f3430068190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:18.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.425+0000 7f34379d9700 1 -- 172.21.15.33:0/300656256 shutdown_connections 2024-05-08T21:27:18.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.425+0000 7f34379d9700 1 -- 172.21.15.33:0/300656256 wait complete. 2024-05-08T21:27:18.444 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:27:18.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.662+0000 7f0d3add6700 1 -- 172.21.15.33:0/4177638702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d341010e0 msgr2=0x7f0d34107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.662+0000 7f0d3add6700 1 --2- 172.21.15.33:0/4177638702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d341010e0 0x7f0d34107990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009fc0 tx=0x7f0d2c00b330 comp rx=0 tx=0).stop 2024-05-08T21:27:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.663+0000 7f0d3add6700 1 -- 172.21.15.33:0/4177638702 shutdown_connections 2024-05-08T21:27:18.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.663+0000 7f0d3add6700 1 --2- 172.21.15.33:0/4177638702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d341010e0 0x7f0d34107990 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.663+0000 7f0d3add6700 1 --2- 172.21.15.33:0/4177638702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d340ff550 0x7f0d340ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.663+0000 7f0d3add6700 1 --2- 172.21.15.33:0/4177638702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d34100740 0x7f0d34100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.663+0000 7f0d3add6700 1 -- 172.21.15.33:0/4177638702 >> 172.21.15.33:0/4177638702 conn(0x7f0d340faae0 msgr2=0x7f0d340fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:18.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.663+0000 7f0d3add6700 1 -- 172.21.15.33:0/4177638702 shutdown_connections 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.663+0000 7f0d3add6700 1 -- 172.21.15.33:0/4177638702 wait complete. 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.664+0000 7f0d3add6700 1 Processor -- start 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.664+0000 7f0d3add6700 1 -- start start 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.664+0000 7f0d3add6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d340ff550 0x7f0d34071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.664+0000 7f0d3add6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d34100740 0x7f0d34072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.664+0000 7f0d3add6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d341010e0 0x7f0d341ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.664+0000 7f0d3add6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34072710 con 0x7f0d34100740 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.664+0000 7f0d3add6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d341a4c70 con 0x7f0d340ff550 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.664+0000 7f0d3add6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d341a4e30 con 0x7f0d341010e0 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.664+0000 7f0d38b72700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d340ff550 0x7f0d34071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.664+0000 7f0d38b72700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d340ff550 0x7f0d34071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41620/0 (socket says 172.21.15.33:41620) 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.664+0000 7f0d38b72700 1 -- 172.21.15.33:0/1848287417 learned_addr learned my addr 172.21.15.33:0/1848287417 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:18.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.664+0000 7f0d39373700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d341010e0 0x7f0d341ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.665+0000 7f0d39373700 1 -- 172.21.15.33:0/1848287417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d340ff550 msgr2=0x7f0d34071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.665+0000 7f0d39373700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d340ff550 0x7f0d34071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.665+0000 7f0d39373700 1 -- 172.21.15.33:0/1848287417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d34100740 msgr2=0x7f0d34072100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.665+0000 7f0d33fff700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d34100740 0x7f0d34072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.665+0000 7f0d39373700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d34100740 0x7f0d34072100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.665+0000 7f0d39373700 1 -- 172.21.15.33:0/1848287417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d24009520 con 0x7f0d341010e0 2024-05-08T21:27:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.665+0000 7f0d33fff700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d34100740 0x7f0d34072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:27:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.665+0000 7f0d39373700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d341010e0 0x7f0d341ad870 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00d010 tx=0x7f0d2c007ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.665+0000 7f0d31ffb700 1 -- 172.21.15.33:0/1848287417 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00e040 con 0x7f0d341010e0 2024-05-08T21:27:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.665+0000 7f0d31ffb700 1 -- 172.21.15.33:0/1848287417 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d2c004c60 con 0x7f0d341010e0 2024-05-08T21:27:18.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.666+0000 7f0d31ffb700 1 -- 172.21.15.33:0/1848287417 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c01b6b0 con 0x7f0d341010e0 2024-05-08T21:27:18.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.666+0000 7f0d3add6700 1 -- 172.21.15.33:0/1848287417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2c009c40 con 0x7f0d341010e0 2024-05-08T21:27:18.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.666+0000 7f0d3add6700 1 -- 172.21.15.33:0/1848287417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d341a5290 con 0x7f0d341010e0 2024-05-08T21:27:18.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.669+0000 7f0d31ffb700 1 -- 172.21.15.33:0/1848287417 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0d2c01b810 con 0x7f0d341010e0 2024-05-08T21:27:18.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.670+0000 7f0d31ffb700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0d1c0a11a0 0x7f0d1c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.671+0000 7f0d31ffb700 1 -- 172.21.15.33:0/1848287417 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f0d2c0ce370 con 0x7f0d341010e0 2024-05-08T21:27:18.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.671+0000 7f0d38b72700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0d1c0a11a0 0x7f0d1c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.672+0000 7f0d38b72700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0d1c0a11a0 0x7f0d1c0a3660 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0d24005950 tx=0x7f0d24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.672+0000 7f0d46f90b80 1 Processor -- start 2024-05-08T21:27:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d46f90b80 1 -- start start 2024-05-08T21:27:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d46f90b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17e540 0x55a77b17e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d46f90b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17eea0 0x55a77b17f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d46f90b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17f860 0x55a77b17fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d46f90b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77b0d5150 con 0x55a77b17eea0 2024-05-08T21:27:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d46f90b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77b0d2b10 con 0x55a77b17e540 2024-05-08T21:27:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d46f90b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77b0ce400 con 0x55a77b17f860 2024-05-08T21:27:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d39373700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17f860 0x55a77b17fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d39373700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17f860 0x55a77b17fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:56654/0 (socket says 172.21.15.33:56654) 2024-05-08T21:27:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d39373700 1 -- 172.21.15.33:0/4239990734 learned_addr learned my addr 172.21.15.33:0/4239990734 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d39373700 1 -- 172.21.15.33:0/4239990734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17e540 msgr2=0x55a77b17e960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:27:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d39373700 1 --2- 172.21.15.33:0/4239990734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17e540 0x55a77b17e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d39373700 1 -- 172.21.15.33:0/4239990734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17eea0 msgr2=0x55a77b17f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d39373700 1 --2- 172.21.15.33:0/4239990734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17eea0 0x55a77b17f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.673+0000 7f0d39373700 1 -- 172.21.15.33:0/4239990734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a77af4f3b0 con 0x55a77b17f860 2024-05-08T21:27:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.674+0000 7f0d39373700 1 --2- 172.21.15.33:0/4239990734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17f860 0x55a77b17fce0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c0979d0 tx=0x7f0d2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19eea7bc1ed24e3e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.674+0000 7f0d1affd700 1 -- 172.21.15.33:0/4239990734 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c004dd0 con 0x55a77b17f860 2024-05-08T21:27:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.674+0000 7f0d1affd700 1 -- 172.21.15.33:0/4239990734 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d2c09b4d0 con 0x55a77b17f860 2024-05-08T21:27:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.674+0000 7f0d1affd700 1 -- 172.21.15.33:0/4239990734 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c099680 con 0x55a77b17f860 2024-05-08T21:27:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.675+0000 7f0d46f90b80 1 -- 172.21.15.33:0/4239990734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17f860 msgr2=0x55a77b17fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.675+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/4239990734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17f860 0x55a77b17fce0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c0979d0 tx=0x7f0d2c00b330 comp rx=0 tx=0).stop 2024-05-08T21:27:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.675+0000 7f0d46f90b80 1 -- 172.21.15.33:0/4239990734 shutdown_connections 2024-05-08T21:27:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.675+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/4239990734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17f860 0x55a77b17fce0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.675+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/4239990734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17e540 0x55a77b17e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.675+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/4239990734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17eea0 0x55a77b17f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.675+0000 7f0d46f90b80 1 -- 172.21.15.33:0/4239990734 >> 172.21.15.33:0/4239990734 conn(0x55a77b0dc210 msgr2=0x55a77b178de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.675+0000 7f0d46f90b80 1 -- 172.21.15.33:0/4239990734 shutdown_connections 2024-05-08T21:27:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.675+0000 7f0d46f90b80 1 -- 172.21.15.33:0/4239990734 wait complete. 2024-05-08T21:27:18.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d46f90b80 1 -- start start 2024-05-08T21:27:18.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d46f90b80 1 Processor -- start 2024-05-08T21:27:18.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d46f90b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17e540 0x55a77b187c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d46f90b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17eea0 0x55a77b1881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d46f90b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17f860 0x55a77b18f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d46f90b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77b06a560 con 0x55a77b17f860 2024-05-08T21:27:18.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d46f90b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77b06a200 con 0x55a77b17eea0 2024-05-08T21:27:18.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d46f90b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77af516a0 con 0x55a77b17e540 2024-05-08T21:27:18.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d46f90b80 10 client.0 _ll_register_callbacks cb 0x55a77b073b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:18.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d33fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17eea0 0x55a77b1881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d38b72700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17e540 0x55a77b187c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d33fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17eea0 0x55a77b1881c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41634/0 (socket says 172.21.15.33:41634) 2024-05-08T21:27:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.676+0000 7f0d33fff700 1 -- 172.21.15.33:0/861590752 learned_addr learned my addr 172.21.15.33:0/861590752 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.677+0000 7f0d39373700 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17f860 0x55a77b18f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.677+0000 7f0d33fff700 1 -- 172.21.15.33:0/861590752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17e540 msgr2=0x55a77b187c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.677+0000 7f0d33fff700 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17e540 0x55a77b187c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.677+0000 7f0d33fff700 1 -- 172.21.15.33:0/861590752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17f860 msgr2=0x55a77b18f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.677+0000 7f0d33fff700 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17f860 0x55a77b18f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.677+0000 7f0d33fff700 1 -- 172.21.15.33:0/861590752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d341a5290 con 0x55a77b17eea0 2024-05-08T21:27:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.677+0000 7f0d38b72700 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17e540 0x55a77b187c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.677+0000 7f0d33fff700 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17eea0 0x55a77b1881c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800fa50 tx=0x7f0d2800fd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=642d3b801a876200 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:18.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.677+0000 7f0d13fff700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:27:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.678+0000 7f0d13fff700 1 -- 172.21.15.33:0/861590752 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800ae10 con 0x55a77b17eea0 2024-05-08T21:27:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.678+0000 7f0d13fff700 1 -- 172.21.15.33:0/861590752 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f0d2800da60 con 0x55a77b17eea0 2024-05-08T21:27:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.678+0000 7f0d46f90b80 1 -- 172.21.15.33:0/861590752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a77b027270 con 0x55a77b17eea0 2024-05-08T21:27:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.678+0000 7f0d13fff700 1 -- 172.21.15.33:0/861590752 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d2800dc20 con 0x55a77b17eea0 2024-05-08T21:27:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.678+0000 7f0d46f90b80 10 client.27662 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.678+0000 7f0d46f90b80 1 -- 172.21.15.33:0/861590752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a77aefbb60 con 0x55a77b17eea0 2024-05-08T21:27:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.679+0000 7f0d13fff700 1 -- 172.21.15.33:0/861590752 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a77b027270 con 0x55a77b17eea0 2024-05-08T21:27:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.679+0000 7f0d13fff700 1 -- 172.21.15.33:0/861590752 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0d28003a60 con 0x55a77b17eea0 2024-05-08T21:27:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.679+0000 7f0d46f90b80 10 client.27662 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.679+0000 7f0d46f90b80 10 client.27662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.679+0000 7f0d46f90b80 20 client.27662 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.679+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a77b193b30 0x55a77b195f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.679+0000 7f0d46f90b80 4 client.27662 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.679+0000 7f0d46f90b80 1 -- 172.21.15.33:0/861590752 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a77b14d810 con 0x55a77b193b30 2024-05-08T21:27:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.679+0000 7f0d38b72700 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a77b193b30 0x55a77b195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.680+0000 7f0d38b72700 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a77b193b30 0x55a77b195f50 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c4bc63014528a9c server_cookie=3cff23db311a71d1 in_seq=0 out_seq=0 2024-05-08T21:27:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.680+0000 7f0d13fff700 10 client.27662 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:18.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.681+0000 7f0d13fff700 1 -- 172.21.15.33:0/861590752 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a77b14d810 con 0x55a77b193b30 2024-05-08T21:27:18.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.681+0000 7f0d13fff700 10 client.27662 handle_command_reply: tid=0 2024-05-08T21:27:18.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.681+0000 7f0d46f90b80 1 client.27662 shutdown 2024-05-08T21:27:18.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.682+0000 7f0d46f90b80 1 -- 172.21.15.33:0/861590752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17eea0 msgr2=0x55a77b1881c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.682+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17eea0 0x55a77b1881c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800fa50 tx=0x7f0d2800fd60 comp rx=0 tx=0).stop 2024-05-08T21:27:18.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.682+0000 7f0d46f90b80 1 -- 172.21.15.33:0/861590752 shutdown_connections 2024-05-08T21:27:18.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.682+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a77b193b30 0x55a77b195f50 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.682+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17f860 0x55a77b18f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.682+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b17e540 0x55a77b187c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.682+0000 7f0d13fff700 0 client.27662 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:18.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.682+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/861590752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17eea0 0x55a77b1881c0 secure :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800fa50 tx=0x7f0d2800fd60 comp rx=0 tx=0).stop 2024-05-08T21:27:18.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.682+0000 7f0d46f90b80 1 -- 172.21.15.33:0/861590752 >> 172.21.15.33:0/861590752 conn(0x55a77b0dc210 msgr2=0x55a77b1785a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:18.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.683+0000 7f0d46f90b80 1 -- 172.21.15.33:0/861590752 shutdown_connections 2024-05-08T21:27:18.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.683+0000 7f0d46f90b80 1 -- 172.21.15.33:0/861590752 wait complete. 2024-05-08T21:27:18.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.683+0000 7f0d46f90b80 20 client.27662 trim_cache size 0 max 16384 2024-05-08T21:27:18.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.705+0000 7f0d46f90b80 1 Processor -- start 2024-05-08T21:27:18.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.705+0000 7f0d46f90b80 1 -- start start 2024-05-08T21:27:18.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.705+0000 7f0d46f90b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17e540 0x55a77b187190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.705+0000 7f0d46f90b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17f860 0x55a77b18f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.705+0000 7f0d46f90b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b18f120 0x55a77b185e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.705+0000 7f0d46f90b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77b027270 con 0x55a77b17f860 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d46f90b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77b188ec0 con 0x55a77b17e540 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d46f90b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77b1877f0 con 0x55a77b18f120 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d33fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17f860 0x55a77b18f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d33fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17f860 0x55a77b18f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40422/0 (socket says 172.21.15.33:40422) 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d33fff700 1 -- 172.21.15.33:0/3217800250 learned_addr learned my addr 172.21.15.33:0/3217800250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d39373700 1 --2- 172.21.15.33:0/3217800250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b18f120 0x55a77b185e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d33fff700 1 -- 172.21.15.33:0/3217800250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b18f120 msgr2=0x55a77b185e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d33fff700 1 --2- 172.21.15.33:0/3217800250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b18f120 0x55a77b185e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d33fff700 1 -- 172.21.15.33:0/3217800250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17e540 msgr2=0x55a77b187190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d33fff700 1 --2- 172.21.15.33:0/3217800250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17e540 0x55a77b187190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d33fff700 1 -- 172.21.15.33:0/3217800250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a77b094560 con 0x55a77b17f860 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d33fff700 1 --2- 172.21.15.33:0/3217800250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17f860 0x55a77b18f8b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0d28003e10 tx=0x7f0d28005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b6c6b0f0846433 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:18.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.706+0000 7f0d1b7fe700 1 -- 172.21.15.33:0/3217800250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800ae10 con 0x55a77b17f860 2024-05-08T21:27:18.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.707+0000 7f0d1b7fe700 1 -- 172.21.15.33:0/3217800250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d28003680 con 0x55a77b17f860 2024-05-08T21:27:18.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.707+0000 7f0d1b7fe700 1 -- 172.21.15.33:0/3217800250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28005320 con 0x55a77b17f860 2024-05-08T21:27:18.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.707+0000 7f0d46f90b80 1 -- 172.21.15.33:0/3217800250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17f860 msgr2=0x55a77b18f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.707+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/3217800250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17f860 0x55a77b18f8b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0d28003e10 tx=0x7f0d28005da0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.707+0000 7f0d46f90b80 1 -- 172.21.15.33:0/3217800250 shutdown_connections 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.707+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/3217800250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b18f120 0x55a77b185e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.707+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/3217800250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17e540 0x55a77b187190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.707+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/3217800250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17f860 0x55a77b18f8b0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.707+0000 7f0d46f90b80 1 -- 172.21.15.33:0/3217800250 >> 172.21.15.33:0/3217800250 conn(0x55a77b0dc210 msgr2=0x55a77b175210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.707+0000 7f0d46f90b80 1 -- 172.21.15.33:0/3217800250 shutdown_connections 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.708+0000 7f0d46f90b80 1 -- 172.21.15.33:0/3217800250 wait complete. 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.708+0000 7f0d46f90b80 1 -- start start 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.708+0000 7f0d46f90b80 1 Processor -- start 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.708+0000 7f0d46f90b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17e540 0x55a77b18cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.708+0000 7f0d46f90b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17f860 0x55a77b18d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.708+0000 7f0d46f90b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b18f120 0x55a77b18df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.708+0000 7f0d46f90b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77b18e450 con 0x55a77b17e540 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.708+0000 7f0d46f90b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77b18e5c0 con 0x55a77b17f860 2024-05-08T21:27:18.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.708+0000 7f0d46f90b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a77b18e780 con 0x55a77b18f120 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d46f90b80 10 client.0 _ll_register_callbacks cb 0x55a77b073b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d33fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17f860 0x55a77b18d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d33fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17f860 0x55a77b18d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:41666/0 (socket says 172.21.15.33:41666) 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d33fff700 1 -- 172.21.15.33:0/2896633162 learned_addr learned my addr 172.21.15.33:0/2896633162 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d39373700 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b18f120 0x55a77b18df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d33fff700 1 -- 172.21.15.33:0/2896633162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b18f120 msgr2=0x55a77b18df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d33fff700 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b18f120 0x55a77b18df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d33fff700 1 -- 172.21.15.33:0/2896633162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17e540 msgr2=0x55a77b18cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d33fff700 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17e540 0x55a77b18cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d33fff700 1 -- 172.21.15.33:0/2896633162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a77aefbb60 con 0x55a77b17f860 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d39373700 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b18f120 0x55a77b18df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d33fff700 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17f860 0x55a77b18d040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800f7a0 tx=0x7f0d2800dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4465538a1f0b10a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:18.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d13fff700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:27:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.709+0000 7f0d13fff700 1 -- 172.21.15.33:0/2896633162 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800ad80 con 0x55a77b17f860 2024-05-08T21:27:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.710+0000 7f0d13fff700 1 -- 172.21.15.33:0/2896633162 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f0d28000e40 con 0x55a77b17f860 2024-05-08T21:27:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.710+0000 7f0d46f90b80 1 -- 172.21.15.33:0/2896633162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d2c09b4d0 con 0x55a77b17f860 2024-05-08T21:27:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.710+0000 7f0d13fff700 1 -- 172.21.15.33:0/2896633162 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d28005280 con 0x55a77b17f860 2024-05-08T21:27:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.710+0000 7f0d46f90b80 10 client.27665 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.710+0000 7f0d46f90b80 1 -- 172.21.15.33:0/2896633162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a77af53cd0 con 0x55a77b17f860 2024-05-08T21:27:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.710+0000 7f0d13fff700 1 -- 172.21.15.33:0/2896633162 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d2c09b4d0 con 0x55a77b17f860 2024-05-08T21:27:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.710+0000 7f0d13fff700 1 -- 172.21.15.33:0/2896633162 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0d28039070 con 0x55a77b17f860 2024-05-08T21:27:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.711+0000 7f0d46f90b80 10 client.27665 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.711+0000 7f0d46f90b80 10 client.27665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:18.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.711+0000 7f0d46f90b80 20 client.27665 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.711+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a77b1ead50 0x55a77b1ed170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.711+0000 7f0d46f90b80 4 client.27665 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:27:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.711+0000 7f0d46f90b80 1 -- 172.21.15.33:0/2896633162 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a77b0c6ec0 con 0x55a77b1ead50 2024-05-08T21:27:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.711+0000 7f0d38b72700 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a77b1ead50 0x55a77b1ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.712+0000 7f0d38b72700 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a77b1ead50 0x55a77b1ed170 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e3504f31d05922 server_cookie=2113edf0c2857e3f in_seq=0 out_seq=0 2024-05-08T21:27:18.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.712+0000 7f0d13fff700 10 client.27665 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:18.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d13fff700 1 -- 172.21.15.33:0/2896633162 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a77b0c6ec0 con 0x55a77b1ead50 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d13fff700 10 client.27665 handle_command_reply: tid=0 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d46f90b80 1 client.27665 shutdown 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d46f90b80 1 -- 172.21.15.33:0/2896633162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17f860 msgr2=0x55a77b18d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17f860 0x55a77b18d040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800f7a0 tx=0x7f0d2800dd10 comp rx=0 tx=0).stop 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d46f90b80 1 -- 172.21.15.33:0/2896633162 shutdown_connections 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55a77b1ead50 0x55a77b1ed170 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a77b18f120 0x55a77b18df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a77b17e540 0x55a77b18cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d46f90b80 1 --2- 172.21.15.33:0/2896633162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a77b17f860 0x55a77b18d040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d46f90b80 1 -- 172.21.15.33:0/2896633162 >> 172.21.15.33:0/2896633162 conn(0x55a77b0dc210 msgr2=0x55a77b185a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d13fff700 0 client.27665 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d46f90b80 1 -- 172.21.15.33:0/2896633162 shutdown_connections 2024-05-08T21:27:18.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.713+0000 7f0d46f90b80 1 -- 172.21.15.33:0/2896633162 wait complete. 2024-05-08T21:27:18.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d46f90b80 20 client.27665 trim_cache size 0 max 16384 2024-05-08T21:27:18.720 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:27:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:27:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:27:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: "940f742e-3f9b-4ae5-83c7-866cf22f15ff": { 2024-05-08T21:27:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:27:18.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "940f742e-3f9b-4ae5-83c7-866cf22f15ff", 2024-05-08T21:27:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:27:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:27:18.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:27:18.721 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:27:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d3add6700 1 -- 172.21.15.33:0/1848287417 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0d1c0a11a0 msgr2=0x7f0d1c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d3add6700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0d1c0a11a0 0x7f0d1c0a3660 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0d24005950 tx=0x7f0d24009260 comp rx=0 tx=0).stop 2024-05-08T21:27:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d3add6700 1 -- 172.21.15.33:0/1848287417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d341010e0 msgr2=0x7f0d341ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d3add6700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d341010e0 0x7f0d341ad870 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00d010 tx=0x7f0d2c007ce0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d3add6700 1 -- 172.21.15.33:0/1848287417 shutdown_connections 2024-05-08T21:27:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d3add6700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d341010e0 0x7f0d341ad870 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d3add6700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f0d1c0a11a0 0x7f0d1c0a3660 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d3add6700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d340ff550 0x7f0d34071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d3add6700 1 --2- 172.21.15.33:0/1848287417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d34100740 0x7f0d34072100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d3add6700 1 -- 172.21.15.33:0/1848287417 >> 172.21.15.33:0/1848287417 conn(0x7f0d340faae0 msgr2=0x7f0d34103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d3add6700 1 -- 172.21.15.33:0/1848287417 shutdown_connections 2024-05-08T21:27:18.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:18.714+0000 7f0d3add6700 1 -- 172.21.15.33:0/1848287417 wait complete. 2024-05-08T21:27:18.735 INFO:tasks.cephfs.filesystem:scrub status for tag:940f742e-3f9b-4ae5-83c7-866cf22f15ff - {'path': '/', 'tag': '940f742e-3f9b-4ae5-83c7-866cf22f15ff', 'options': 'recursive,force'} 2024-05-08T21:27:19.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: pgmap v540: 129 pgs: 129 active+clean; 271 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 484 op/s 2024-05-08T21:27:19.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:19 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:27:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: pgmap v540: 129 pgs: 129 active+clean; 271 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 484 op/s 2024-05-08T21:27:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:27:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:19 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:27:19.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: pgmap v540: 129 pgs: 129 active+clean; 271 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 484 op/s 2024-05-08T21:27:19.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:27:19.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:19.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:19.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:19.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:19.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:19.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:19.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:19.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:19.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:19.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:19 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:27:20.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:20 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:20.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:20 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:27:20.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:20 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:27:20.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:20 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:27:20.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:20 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:27:20.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:20 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:27:20.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:20 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:27:20.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:20 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:27:20.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:20 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:20.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:20 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:27:20.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:20 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:27:20.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:20 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:27:20.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:20 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:27:20.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:20 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:27:20.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:20 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:27:20.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:20 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:27:20.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:20 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:20.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:20 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:27:20.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:20 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:27:20.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:20 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:27:20.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:20 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T21:27:20.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:20 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:27:20.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:20 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:27:20.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:20 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:27:21.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:21 smithi071 ceph-mon[47074]: Adjusting osd_memory_target on smithi082 to 182.4M 2024-05-08T21:27:21.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:21 smithi071 ceph-mon[47074]: Unable to set osd_memory_target on smithi082 to 191349760: error parsing value: Value '191349760' is below minimum 939524096 2024-05-08T21:27:21.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:21 smithi071 ceph-mon[47074]: pgmap v541: 129 pgs: 129 active+clean; 260 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 654 op/s 2024-05-08T21:27:21.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:21 smithi082 ceph-mon[47512]: Adjusting osd_memory_target on smithi082 to 182.4M 2024-05-08T21:27:21.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:21 smithi082 ceph-mon[47512]: Unable to set osd_memory_target on smithi082 to 191349760: error parsing value: Value '191349760' is below minimum 939524096 2024-05-08T21:27:21.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:21 smithi082 ceph-mon[47512]: pgmap v541: 129 pgs: 129 active+clean; 260 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 654 op/s 2024-05-08T21:27:21.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:21 smithi033 ceph-mon[42528]: Adjusting osd_memory_target on smithi082 to 182.4M 2024-05-08T21:27:21.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:21 smithi033 ceph-mon[42528]: Unable to set osd_memory_target on smithi082 to 191349760: error parsing value: Value '191349760' is below minimum 939524096 2024-05-08T21:27:21.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:21 smithi033 ceph-mon[42528]: pgmap v541: 129 pgs: 129 active+clean; 260 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 654 op/s 2024-05-08T21:27:23.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:23 smithi071 ceph-mon[47074]: pgmap v542: 129 pgs: 129 active+clean; 266 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.3 MiB/s wr, 488 op/s 2024-05-08T21:27:23.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:23 smithi082 ceph-mon[47512]: pgmap v542: 129 pgs: 129 active+clean; 266 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.3 MiB/s wr, 488 op/s 2024-05-08T21:27:23.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:23 smithi033 ceph-mon[42528]: pgmap v542: 129 pgs: 129 active+clean; 266 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.3 MiB/s wr, 488 op/s 2024-05-08T21:27:25.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:25 smithi071 ceph-mon[47074]: pgmap v543: 129 pgs: 129 active+clean; 262 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 612 op/s 2024-05-08T21:27:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:25 smithi082 ceph-mon[47512]: pgmap v543: 129 pgs: 129 active+clean; 262 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 612 op/s 2024-05-08T21:27:25.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:25 smithi033 ceph-mon[42528]: pgmap v543: 129 pgs: 129 active+clean; 262 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 612 op/s 2024-05-08T21:27:27.824 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:27 smithi071 ceph-mon[47074]: pgmap v544: 129 pgs: 129 active+clean; 261 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.4 MiB/s wr, 535 op/s 2024-05-08T21:27:27.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:27 smithi082 ceph-mon[47512]: pgmap v544: 129 pgs: 129 active+clean; 261 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.4 MiB/s wr, 535 op/s 2024-05-08T21:27:27.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:27 smithi033 ceph-mon[42528]: pgmap v544: 129 pgs: 129 active+clean; 261 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.4 MiB/s wr, 535 op/s 2024-05-08T21:27:29.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:29 smithi071 ceph-mon[47074]: pgmap v545: 129 pgs: 129 active+clean; 261 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 480 op/s 2024-05-08T21:27:29.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:29 smithi082 ceph-mon[47512]: pgmap v545: 129 pgs: 129 active+clean; 261 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 480 op/s 2024-05-08T21:27:29.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:29 smithi033 ceph-mon[42528]: pgmap v545: 129 pgs: 129 active+clean; 261 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 480 op/s 2024-05-08T21:27:31.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:31 smithi071 ceph-mon[47074]: pgmap v546: 129 pgs: 129 active+clean; 275 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.9 MiB/s wr, 663 op/s 2024-05-08T21:27:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:31 smithi082 ceph-mon[47512]: pgmap v546: 129 pgs: 129 active+clean; 275 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.9 MiB/s wr, 663 op/s 2024-05-08T21:27:31.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:31 smithi033 ceph-mon[42528]: pgmap v546: 129 pgs: 129 active+clean; 275 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.9 MiB/s wr, 663 op/s 2024-05-08T21:27:32.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:32 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:27:32.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:32 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:27:32.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:32 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:27:32.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:32 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:27:32.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:32 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:27:32.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:32 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:27:33.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:33 smithi071 ceph-mon[47074]: pgmap v547: 129 pgs: 129 active+clean; 271 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 510 op/s 2024-05-08T21:27:33.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:33 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:27:33.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:33 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:27:33.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:33 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:27:33.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:33 smithi082 ceph-mon[47512]: pgmap v547: 129 pgs: 129 active+clean; 271 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 510 op/s 2024-05-08T21:27:33.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:33 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:27:33.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:33 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:27:33.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:33 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:27:33.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:33 smithi033 ceph-mon[42528]: pgmap v547: 129 pgs: 129 active+clean; 271 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.2 MiB/s wr, 510 op/s 2024-05-08T21:27:33.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:33 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:27:33.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:33 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:27:33.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:33 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:27:35.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:35 smithi071 ceph-mon[47074]: pgmap v548: 129 pgs: 129 active+clean; 277 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.7 MiB/s wr, 637 op/s 2024-05-08T21:27:35.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:35 smithi082 ceph-mon[47512]: pgmap v548: 129 pgs: 129 active+clean; 277 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.7 MiB/s wr, 637 op/s 2024-05-08T21:27:35.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:35 smithi033 ceph-mon[42528]: pgmap v548: 129 pgs: 129 active+clean; 277 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.7 MiB/s wr, 637 op/s 2024-05-08T21:27:36.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:36 smithi082 ceph-mon[47512]: pgmap v549: 129 pgs: 129 active+clean; 284 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.2 MiB/s wr, 549 op/s 2024-05-08T21:27:36.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:36 smithi033 ceph-mon[42528]: pgmap v549: 129 pgs: 129 active+clean; 284 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.2 MiB/s wr, 549 op/s 2024-05-08T21:27:37.069 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:36 smithi071 ceph-mon[47074]: pgmap v549: 129 pgs: 129 active+clean; 284 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.2 MiB/s wr, 549 op/s 2024-05-08T21:27:38.976 INFO:tasks.workunit.client.0.smithi033.stdout:40000 trunc from 0x2372fba to 0x3296874 2024-05-08T21:27:39.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:39 smithi033 ceph-mon[42528]: pgmap v550: 129 pgs: 129 active+clean; 284 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 483 op/s 2024-05-08T21:27:39.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:39 smithi071 ceph-mon[47074]: pgmap v550: 129 pgs: 129 active+clean; 284 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 483 op/s 2024-05-08T21:27:39.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:39 smithi082 ceph-mon[47512]: pgmap v550: 129 pgs: 129 active+clean; 284 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 483 op/s 2024-05-08T21:27:41.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:41 smithi033 ceph-mon[42528]: pgmap v551: 129 pgs: 129 active+clean; 261 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 660 op/s 2024-05-08T21:27:41.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:41 smithi071 ceph-mon[47074]: pgmap v551: 129 pgs: 129 active+clean; 261 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 660 op/s 2024-05-08T21:27:41.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:41 smithi082 ceph-mon[47512]: pgmap v551: 129 pgs: 129 active+clean; 261 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 660 op/s 2024-05-08T21:27:43.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:43 smithi033 ceph-mon[42528]: pgmap v552: 129 pgs: 129 active+clean; 266 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.2 MiB/s wr, 495 op/s 2024-05-08T21:27:43.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:43 smithi071 ceph-mon[47074]: pgmap v552: 129 pgs: 129 active+clean; 266 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.2 MiB/s wr, 495 op/s 2024-05-08T21:27:43.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:43 smithi082 ceph-mon[47512]: pgmap v552: 129 pgs: 129 active+clean; 266 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.2 MiB/s wr, 495 op/s 2024-05-08T21:27:45.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:45 smithi033 ceph-mon[42528]: pgmap v553: 129 pgs: 129 active+clean; 285 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.8 MiB/s wr, 615 op/s 2024-05-08T21:27:45.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:45 smithi071 ceph-mon[47074]: pgmap v553: 129 pgs: 129 active+clean; 285 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.8 MiB/s wr, 615 op/s 2024-05-08T21:27:45.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:45 smithi082 ceph-mon[47512]: pgmap v553: 129 pgs: 129 active+clean; 285 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.8 MiB/s wr, 615 op/s 2024-05-08T21:27:47.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:47 smithi033 ceph-mon[42528]: pgmap v554: 129 pgs: 129 active+clean; 284 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.3 MiB/s wr, 522 op/s 2024-05-08T21:27:47.823 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:47 smithi071 ceph-mon[47074]: pgmap v554: 129 pgs: 129 active+clean; 284 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.3 MiB/s wr, 522 op/s 2024-05-08T21:27:47.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:47 smithi082 ceph-mon[47512]: pgmap v554: 129 pgs: 129 active+clean; 284 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.3 MiB/s wr, 522 op/s 2024-05-08T21:27:48.736 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:27:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.932+0000 7f1e008ec700 1 -- 172.21.15.33:0/736812674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0ffd60 msgr2=0x7f1dfc100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.932+0000 7f1e008ec700 1 --2- 172.21.15.33:0/736812674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0ffd60 0x7f1dfc100180 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0099c0 tx=0x7f1dec009cd0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.932+0000 7f1e008ec700 1 -- 172.21.15.33:0/736812674 shutdown_connections 2024-05-08T21:27:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.932+0000 7f1e008ec700 1 --2- 172.21.15.33:0/736812674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dfc1018f0 0x7f1dfc1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.932+0000 7f1e008ec700 1 --2- 172.21.15.33:0/736812674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0ffd60 0x7f1dfc100180 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.932+0000 7f1e008ec700 1 --2- 172.21.15.33:0/736812674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc100f50 0x7f1dfc1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.932+0000 7f1e008ec700 1 -- 172.21.15.33:0/736812674 >> 172.21.15.33:0/736812674 conn(0x7f1dfc0fb290 msgr2=0x7f1dfc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.932+0000 7f1e008ec700 1 -- 172.21.15.33:0/736812674 shutdown_connections 2024-05-08T21:27:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.932+0000 7f1e008ec700 1 -- 172.21.15.33:0/736812674 wait complete. 2024-05-08T21:27:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.933+0000 7f1e008ec700 1 Processor -- start 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.933+0000 7f1e008ec700 1 -- start start 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.933+0000 7f1e008ec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0ffd60 0x7f1dfc196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.933+0000 7f1e008ec700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc100f50 0x7f1dfc197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.933+0000 7f1e008ec700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dfc1018f0 0x7f1dfc19e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.933+0000 7f1e008ec700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc102170 con 0x7f1dfc0ffd60 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.933+0000 7f1e008ec700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1022e0 con 0x7f1dfc100f50 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.933+0000 7f1e008ec700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1024a0 con 0x7f1dfc1018f0 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0ffd60 0x7f1dfc196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0ffd60 0x7f1dfc196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54144/0 (socket says 172.21.15.33:54144) 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1dfa59c700 1 -- 172.21.15.33:0/2961596344 learned_addr learned my addr 172.21.15.33:0/2961596344 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dfc1018f0 0x7f1dfc19e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1df9d9b700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc100f50 0x7f1dfc197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1dfad9d700 1 -- 172.21.15.33:0/2961596344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc100f50 msgr2=0x7f1dfc197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc100f50 0x7f1dfc197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1dfad9d700 1 -- 172.21.15.33:0/2961596344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0ffd60 msgr2=0x7f1dfc196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0ffd60 0x7f1dfc196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1dfad9d700 1 -- 172.21.15.33:0/2961596344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df4009c40 con 0x7f1dfc1018f0 2024-05-08T21:27:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1df9d9b700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc100f50 0x7f1dfc197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0ffd60 0x7f1dfc196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dfc1018f0 0x7f1dfc19e2e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1df400efb0 tx=0x7f1df400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1deb7fe700 1 -- 172.21.15.33:0/2961596344 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4003a00 con 0x7f1dfc1018f0 2024-05-08T21:27:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1deb7fe700 1 -- 172.21.15.33:0/2961596344 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1df400f040 con 0x7f1dfc1018f0 2024-05-08T21:27:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1e008ec700 1 -- 172.21.15.33:0/2961596344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dec0095f0 con 0x7f1dfc1018f0 2024-05-08T21:27:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.934+0000 7f1deb7fe700 1 -- 172.21.15.33:0/2961596344 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40044d0 con 0x7f1dfc1018f0 2024-05-08T21:27:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.935+0000 7f1e008ec700 1 -- 172.21.15.33:0/2961596344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc102ab0 con 0x7f1dfc1018f0 2024-05-08T21:27:48.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.937+0000 7f1deb7fe700 1 -- 172.21.15.33:0/2961596344 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1df400cbc0 con 0x7f1dfc1018f0 2024-05-08T21:27:48.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.938+0000 7f1deb7fe700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1de40a1120 0x7f1de40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.938+0000 7f1deb7fe700 1 -- 172.21.15.33:0/2961596344 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1df40cc8c0 con 0x7f1dfc1018f0 2024-05-08T21:27:48.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.938+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1de40a1120 0x7f1de40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.939+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1de40a1120 0x7f1de40a35e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00b580 tx=0x7f1dec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.939+0000 7f1e0caa6b80 1 Processor -- start 2024-05-08T21:27:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.939+0000 7f1e0caa6b80 1 -- start start 2024-05-08T21:27:48.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.939+0000 7f1e0caa6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851540 0x55d0d2851960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.939+0000 7f1e0caa6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d2852320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.939+0000 7f1e0caa6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2852860 0x55d0d2852ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.939+0000 7f1e0caa6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d27a8150 con 0x55d0d2851ea0 2024-05-08T21:27:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.939+0000 7f1e0caa6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d27a5b10 con 0x55d0d2851540 2024-05-08T21:27:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.939+0000 7f1e0caa6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d27a1400 con 0x55d0d2852860 2024-05-08T21:27:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851540 0x55d0d2851960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851540 0x55d0d2851960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57350/0 (socket says 172.21.15.33:57350) 2024-05-08T21:27:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1dfa59c700 1 -- 172.21.15.33:0/2430060194 learned_addr learned my addr 172.21.15.33:0/2430060194 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1df9d9b700 1 --2- 172.21.15.33:0/2430060194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d2852320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/2430060194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2852860 0x55d0d2852ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1dfa59c700 1 -- 172.21.15.33:0/2430060194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2852860 msgr2=0x55d0d2852ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:48.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/2430060194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2852860 0x55d0d2852ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1dfa59c700 1 -- 172.21.15.33:0/2430060194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 msgr2=0x55d0d2852320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:48.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/2430060194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d2852320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1dfa59c700 1 -- 172.21.15.33:0/2430060194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0d26223b0 con 0x55d0d2851540 2024-05-08T21:27:48.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/2430060194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2852860 0x55d0d2852ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:27:48.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/2430060194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851540 0x55d0d2851960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0098e0 tx=0x7f1dec01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd4de8c796a6712f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:48.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.940+0000 7f1de8ff9700 1 -- 172.21.15.33:0/2430060194 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec003760 con 0x55d0d2851540 2024-05-08T21:27:48.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.941+0000 7f1de8ff9700 1 -- 172.21.15.33:0/2430060194 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1dec01e040 con 0x55d0d2851540 2024-05-08T21:27:48.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.941+0000 7f1de8ff9700 1 -- 172.21.15.33:0/2430060194 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec00fae0 con 0x55d0d2851540 2024-05-08T21:27:48.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.942+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/2430060194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851540 msgr2=0x55d0d2851960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:48.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.942+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/2430060194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851540 0x55d0d2851960 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1dec0098e0 tx=0x7f1dec01f000 comp rx=0 tx=0).stop 2024-05-08T21:27:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.944+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/2430060194 shutdown_connections 2024-05-08T21:27:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.944+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/2430060194 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2852860 0x55d0d2852ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.944+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/2430060194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851540 0x55d0d2851960 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.944+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/2430060194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d2852320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.944+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/2430060194 >> 172.21.15.33:0/2430060194 conn(0x55d0d27af210 msgr2=0x55d0d284bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:48.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.948+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/2430060194 shutdown_connections 2024-05-08T21:27:48.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.948+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/2430060194 wait complete. 2024-05-08T21:27:48.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.949+0000 7f1e0caa6b80 1 -- start start 2024-05-08T21:27:48.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.949+0000 7f1e0caa6b80 1 Processor -- start 2024-05-08T21:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.949+0000 7f1e0caa6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 0x55d0d286bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.949+0000 7f1e0caa6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d286c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.949+0000 7f1e0caa6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2852860 0x55d0d28732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.949+0000 7f1e0caa6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d273d560 con 0x55d0d2851ea0 2024-05-08T21:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.949+0000 7f1e0caa6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d273d200 con 0x55d0d2852860 2024-05-08T21:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.949+0000 7f1e0caa6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d26246a0 con 0x55d0d2851540 2024-05-08T21:27:48.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.949+0000 7f1e0caa6b80 10 client.0 _ll_register_callbacks cb 0x55d0d2746b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.950+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 0x55d0d286bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.950+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 0x55d0d286bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39150/0 (socket says 172.21.15.33:39150) 2024-05-08T21:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.950+0000 7f1dfa59c700 1 -- 172.21.15.33:0/659412816 learned_addr learned my addr 172.21.15.33:0/659412816 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.950+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/659412816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2852860 0x55d0d28732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1dfa59c700 1 -- 172.21.15.33:0/659412816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2852860 msgr2=0x55d0d28732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/659412816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2852860 0x55d0d28732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1dfa59c700 1 -- 172.21.15.33:0/659412816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 msgr2=0x55d0d286c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/659412816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d286c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1dfa59c700 1 -- 172.21.15.33:0/659412816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc108af0 con 0x55d0d2851540 2024-05-08T21:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/659412816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 0x55d0d286bd20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1dec006010 tx=0x7f1dec00f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=577b3af8d8cdd74f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:48.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1de1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1de1ffb700 1 -- 172.21.15.33:0/659412816 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec035070 con 0x55d0d2851540 2024-05-08T21:27:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1de1ffb700 1 -- 172.21.15.33:0/659412816 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1dec0056c0 con 0x55d0d2851540 2024-05-08T21:27:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/659412816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0d26fa270 con 0x55d0d2851540 2024-05-08T21:27:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1de1ffb700 1 -- 172.21.15.33:0/659412816 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dec003970 con 0x55d0d2851540 2024-05-08T21:27:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1e0caa6b80 10 client.27691 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.951+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/659412816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55d0d25ceb60 con 0x55d0d2851540 2024-05-08T21:27:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.953+0000 7f1de1ffb700 1 -- 172.21.15.33:0/659412816 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d0d26fa270 con 0x55d0d2851540 2024-05-08T21:27:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.953+0000 7f1de1ffb700 1 -- 172.21.15.33:0/659412816 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1dec044cc0 con 0x55d0d2851540 2024-05-08T21:27:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.953+0000 7f1e0caa6b80 10 client.27691 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.953+0000 7f1e0caa6b80 10 client.27691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.953+0000 7f1e0caa6b80 20 client.27691 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.953+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/659412816 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d0d2877650 0x55d0d2879a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.953+0000 7f1e0caa6b80 4 client.27691 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.953+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/659412816 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0d2820810 con 0x55d0d2877650 2024-05-08T21:27:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.954+0000 7f1df9d9b700 1 --2- 172.21.15.33:0/659412816 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d0d2877650 0x55d0d2879a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.954+0000 7f1de1ffb700 10 client.27691 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:48.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.954+0000 7f1df9d9b700 1 --2- 172.21.15.33:0/659412816 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d0d2877650 0x55d0d2879a70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6416c4ac9938f9 server_cookie=ee63dd4c831fe447 in_seq=0 out_seq=0 2024-05-08T21:27:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1de1ffb700 1 -- 172.21.15.33:0/659412816 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0d2820810 con 0x55d0d2877650 2024-05-08T21:27:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1de1ffb700 10 client.27691 handle_command_reply: tid=0 2024-05-08T21:27:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1e0caa6b80 1 client.27691 shutdown 2024-05-08T21:27:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/659412816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 msgr2=0x55d0d286bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/659412816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 0x55d0d286bd20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1dec006010 tx=0x7f1dec00f510 comp rx=0 tx=0).stop 2024-05-08T21:27:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/659412816 shutdown_connections 2024-05-08T21:27:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/659412816 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d0d2877650 0x55d0d2879a70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/659412816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2852860 0x55d0d28732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/659412816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 0x55d0d286bd20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1de1ffb700 0 client.27691 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:48.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/659412816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d286c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/659412816 >> 172.21.15.33:0/659412816 conn(0x55d0d27af210 msgr2=0x55d0d284b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:48.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/659412816 shutdown_connections 2024-05-08T21:27:48.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.955+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/659412816 wait complete. 2024-05-08T21:27:48.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.956+0000 7f1e0caa6b80 20 client.27691 trim_cache size 0 max 16384 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1e0caa6b80 1 Processor -- start 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1e0caa6b80 1 -- start start 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1e0caa6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 0x55d0d286bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1e0caa6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d2873260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1e0caa6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2852860 0x55d0d286c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1e0caa6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d286b8b0 con 0x55d0d2851ea0 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1e0caa6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d2874420 con 0x55d0d2852860 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1e0caa6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d2848df0 con 0x55d0d2851540 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1df9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d2873260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1df9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d2873260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54170/0 (socket says 172.21.15.33:54170) 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1df9d9b700 1 -- 172.21.15.33:0/1058207503 learned_addr learned my addr 172.21.15.33:0/1058207503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 0x55d0d286bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/1058207503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2852860 0x55d0d286c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1dfa59c700 1 -- 172.21.15.33:0/1058207503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2852860 msgr2=0x55d0d286c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.981+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/1058207503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2852860 0x55d0d286c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.982+0000 7f1dfa59c700 1 -- 172.21.15.33:0/1058207503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 msgr2=0x55d0d2873260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.982+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/1058207503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d2873260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.982+0000 7f1dfa59c700 1 -- 172.21.15.33:0/1058207503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0d2767560 con 0x55d0d2851540 2024-05-08T21:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.982+0000 7f1df9d9b700 1 --2- 172.21.15.33:0/1058207503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d2873260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.982+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/1058207503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2852860 0x55d0d286c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.982+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/1058207503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 0x55d0d286bad0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1dec000c00 tx=0x7f1dec00bbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1897645294548d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.982+0000 7f1de97fa700 1 -- 172.21.15.33:0/1058207503 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec035030 con 0x55d0d2851540 2024-05-08T21:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.982+0000 7f1de97fa700 1 -- 172.21.15.33:0/1058207503 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1dec036040 con 0x55d0d2851540 2024-05-08T21:27:48.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.982+0000 7f1de97fa700 1 -- 172.21.15.33:0/1058207503 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec003cf0 con 0x55d0d2851540 2024-05-08T21:27:48.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.983+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/1058207503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 msgr2=0x55d0d286bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.983+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/1058207503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 0x55d0d286bad0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1dec000c00 tx=0x7f1dec00bbf0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.983+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/1058207503 shutdown_connections 2024-05-08T21:27:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.983+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/1058207503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2852860 0x55d0d286c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.983+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/1058207503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2851540 0x55d0d286bad0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.983+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/1058207503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851ea0 0x55d0d2873260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.983+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/1058207503 >> 172.21.15.33:0/1058207503 conn(0x55d0d27af210 msgr2=0x55d0d2855730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:48.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.983+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/1058207503 shutdown_connections 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.983+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/1058207503 wait complete. 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.983+0000 7f1e0caa6b80 1 -- start start 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.983+0000 7f1e0caa6b80 1 Processor -- start 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1e0caa6b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851540 0x55d0d286f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1e0caa6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851ea0 0x55d0d286fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1e0caa6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2852860 0x55d0d28729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1e0caa6b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d2872f10 con 0x55d0d2851540 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1e0caa6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d2873080 con 0x55d0d2851ea0 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1e0caa6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d2873240 con 0x55d0d2852860 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1e0caa6b80 10 client.0 _ll_register_callbacks cb 0x55d0d2746b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851540 0x55d0d286f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1dfa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851540 0x55d0d286f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54184/0 (socket says 172.21.15.33:54184) 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1dfa59c700 1 -- 172.21.15.33:0/3597125480 learned_addr learned my addr 172.21.15.33:0/3597125480 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:48.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2852860 0x55d0d28729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1df9d9b700 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851ea0 0x55d0d286fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1dfad9d700 1 -- 172.21.15.33:0/3597125480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851ea0 msgr2=0x55d0d286fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851ea0 0x55d0d286fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1dfad9d700 1 -- 172.21.15.33:0/3597125480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851540 msgr2=0x55d0d286f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851540 0x55d0d286f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1dfad9d700 1 -- 172.21.15.33:0/3597125480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df0012040 con 0x55d0d2852860 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1df9d9b700 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851ea0 0x55d0d286fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.984+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851540 0x55d0d286f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.985+0000 7f1dfad9d700 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2852860 0x55d0d28729d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c6f0 tx=0x7f1df40999e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6d484eef98a6c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.985+0000 7f1de1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.985+0000 7f1de1ffb700 1 -- 172.21.15.33:0/3597125480 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40a1dd0 con 0x55d0d2852860 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.985+0000 7f1de1ffb700 1 -- 172.21.15.33:0/3597125480 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1df4098ed0 con 0x55d0d2852860 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.985+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/3597125480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dec01e040 con 0x55d0d2852860 2024-05-08T21:27:48.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.985+0000 7f1de1ffb700 1 -- 172.21.15.33:0/3597125480 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1df4099090 con 0x55d0d2852860 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.985+0000 7f1e0caa6b80 10 client.27697 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.985+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/3597125480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f1dfc102ab0 con 0x55d0d2852860 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.985+0000 7f1de1ffb700 1 -- 172.21.15.33:0/3597125480 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1df40b6070 con 0x55d0d2852860 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.986+0000 7f1de1ffb700 1 -- 172.21.15.33:0/3597125480 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1df40a1940 con 0x55d0d2852860 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.986+0000 7f1e0caa6b80 10 client.27697 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.986+0000 7f1e0caa6b80 10 client.27697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.986+0000 7f1e0caa6b80 20 client.27697 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.986+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d0d28cefd0 0x55d0d28d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.986+0000 7f1e0caa6b80 4 client.27697 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.986+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/3597125480 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d0d2799ec0 con 0x55d0d28cefd0 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.986+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d0d28cefd0 0x55d0d28d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.986+0000 7f1de1ffb700 10 client.27697 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.986+0000 7f1dfa59c700 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d0d28cefd0 0x55d0d28d13f0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c1dc58bfcdaf38 server_cookie=5518ad2eb9c27a79 in_seq=0 out_seq=0 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1de1ffb700 1 -- 172.21.15.33:0/3597125480 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d0d2799ec0 con 0x55d0d28cefd0 2024-05-08T21:27:48.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1de1ffb700 10 client.27697 handle_command_reply: tid=0 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1e0caa6b80 1 client.27697 shutdown 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/3597125480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2852860 msgr2=0x55d0d28729d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2852860 0x55d0d28729d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c6f0 tx=0x7f1df40999e0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/3597125480 shutdown_connections 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55d0d28cefd0 0x55d0d28d13f0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d2852860 0x55d0d28729d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d0d2851540 0x55d0d286f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1de1ffb700 0 client.27697 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1e0caa6b80 1 --2- 172.21.15.33:0/3597125480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0d2851ea0 0x55d0d286fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/3597125480 >> 172.21.15.33:0/3597125480 conn(0x55d0d27af210 msgr2=0x55d0d286bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/3597125480 shutdown_connections 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1e0caa6b80 1 -- 172.21.15.33:0/3597125480 wait complete. 2024-05-08T21:27:48.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.987+0000 7f1e0caa6b80 20 client.27697 trim_cache size 0 max 16384 2024-05-08T21:27:48.995 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:27:48.995 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:27:48.995 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:27:48.995 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:27:48.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.988+0000 7f1e008ec700 1 -- 172.21.15.33:0/2961596344 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1de40a1120 msgr2=0x7f1de40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.988+0000 7f1e008ec700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1de40a1120 0x7f1de40a35e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00b580 tx=0x7f1dec01a040 comp rx=0 tx=0).stop 2024-05-08T21:27:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.988+0000 7f1e008ec700 1 -- 172.21.15.33:0/2961596344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dfc1018f0 msgr2=0x7f1dfc19e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.988+0000 7f1e008ec700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dfc1018f0 0x7f1dfc19e2e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1df400efb0 tx=0x7f1df400c3a0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.988+0000 7f1e008ec700 1 -- 172.21.15.33:0/2961596344 shutdown_connections 2024-05-08T21:27:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.988+0000 7f1e008ec700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dfc1018f0 0x7f1dfc19e2e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.988+0000 7f1e008ec700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dfc0ffd60 0x7f1dfc196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.988+0000 7f1e008ec700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1de40a1120 0x7f1de40a35e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.988+0000 7f1e008ec700 1 --2- 172.21.15.33:0/2961596344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc100f50 0x7f1dfc197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.988+0000 7f1e008ec700 1 -- 172.21.15.33:0/2961596344 >> 172.21.15.33:0/2961596344 conn(0x7f1dfc0fb290 msgr2=0x7f1dfc068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.988+0000 7f1e008ec700 1 -- 172.21.15.33:0/2961596344 shutdown_connections 2024-05-08T21:27:48.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:48.988+0000 7f1e008ec700 1 -- 172.21.15.33:0/2961596344 wait complete. 2024-05-08T21:27:49.007 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:27:49.007 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:27:49.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.212+0000 7fbef4de0700 1 -- 172.21.15.33:0/205145843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef0101fd0 msgr2=0x7fbef0102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.212+0000 7fbef4de0700 1 --2- 172.21.15.33:0/205145843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef0101fd0 0x7fbef0102450 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbee40098f0 tx=0x7fbee4009c00 comp rx=0 tx=0).stop 2024-05-08T21:27:49.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.212+0000 7fbef4de0700 1 -- 172.21.15.33:0/205145843 shutdown_connections 2024-05-08T21:27:49.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.212+0000 7fbef4de0700 1 --2- 172.21.15.33:0/205145843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0102990 0x7fbef01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.212+0000 7fbef4de0700 1 --2- 172.21.15.33:0/205145843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbef0100e80 0x7fbef01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.212+0000 7fbef4de0700 1 --2- 172.21.15.33:0/205145843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef0101fd0 0x7fbef0102450 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.212+0000 7fbef4de0700 1 -- 172.21.15.33:0/205145843 >> 172.21.15.33:0/205145843 conn(0x7fbef00fc410 msgr2=0x7fbef00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:49.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.212+0000 7fbef4de0700 1 -- 172.21.15.33:0/205145843 shutdown_connections 2024-05-08T21:27:49.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.212+0000 7fbef4de0700 1 -- 172.21.15.33:0/205145843 wait complete. 2024-05-08T21:27:49.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbef4de0700 1 Processor -- start 2024-05-08T21:27:49.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbef4de0700 1 -- start start 2024-05-08T21:27:49.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbef4de0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0100e80 0x7fbef0074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbef4de0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef0101fd0 0x7fbef0073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbef4de0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbef0102990 0x7fbef00738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbef4de0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0074070 con 0x7fbef0100e80 2024-05-08T21:27:49.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbef4de0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef00741e0 con 0x7fbef0101fd0 2024-05-08T21:27:49.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbef4de0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01a0df0 con 0x7fbef0102990 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbeee59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0100e80 0x7fbef0074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbeedd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef0101fd0 0x7fbef0073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbeedd9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef0101fd0 0x7fbef0073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57408/0 (socket says 172.21.15.33:57408) 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbeee59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0100e80 0x7fbef0074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54198/0 (socket says 172.21.15.33:54198) 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbeedd9b700 1 -- 172.21.15.33:0/4239385084 learned_addr learned my addr 172.21.15.33:0/4239385084 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbeeed9d700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbef0102990 0x7fbef00738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbeedd9b700 1 -- 172.21.15.33:0/4239385084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbef0102990 msgr2=0x7fbef00738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbeedd9b700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbef0102990 0x7fbef00738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbeedd9b700 1 -- 172.21.15.33:0/4239385084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0100e80 msgr2=0x7fbef0074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbeedd9b700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0100e80 0x7fbef0074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbeedd9b700 1 -- 172.21.15.33:0/4239385084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee8009d00 con 0x7fbef0101fd0 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.213+0000 7fbeeed9d700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbef0102990 0x7fbef00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.214+0000 7fbeedd9b700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef0101fd0 0x7fbef0073390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009810 tx=0x7fbee400bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.214+0000 7fbedf7fe700 1 -- 172.21.15.33:0/4239385084 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee401c070 con 0x7fbef0101fd0 2024-05-08T21:27:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.214+0000 7fbedf7fe700 1 -- 172.21.15.33:0/4239385084 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbee400ba90 con 0x7fbef0101fd0 2024-05-08T21:27:49.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.214+0000 7fbef4de0700 1 -- 172.21.15.33:0/4239385084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee4009520 con 0x7fbef0101fd0 2024-05-08T21:27:49.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.214+0000 7fbedf7fe700 1 -- 172.21.15.33:0/4239385084 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee40038e0 con 0x7fbef0101fd0 2024-05-08T21:27:49.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.214+0000 7fbef4de0700 1 -- 172.21.15.33:0/4239385084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef01a12b0 con 0x7fbef0101fd0 2024-05-08T21:27:49.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.217+0000 7fbedf7fe700 1 -- 172.21.15.33:0/4239385084 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbee4020030 con 0x7fbef0101fd0 2024-05-08T21:27:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.217+0000 7fbef4de0700 1 -- 172.21.15.33:0/4239385084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef01083e0 con 0x7fbef0101fd0 2024-05-08T21:27:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.218+0000 7fbedf7fe700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbed80a11a0 0x7fbed80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.218+0000 7fbedf7fe700 1 -- 172.21.15.33:0/4239385084 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbee40cdb40 con 0x7fbef0101fd0 2024-05-08T21:27:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.218+0000 7fbeee59c700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbed80a11a0 0x7fbed80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.219+0000 7fbeee59c700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbed80a11a0 0x7fbed80a3660 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009c60 tx=0x7fbee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.225+0000 7fbedf7fe700 1 -- 172.21.15.33:0/4239385084 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fbee4096fe0 con 0x7fbef0101fd0 2024-05-08T21:27:49.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.574+0000 7fbef4de0700 1 -- 172.21.15.33:0/4239385084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbef0107dc0 con 0x7fbef0101fd0 2024-05-08T21:27:49.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.578+0000 7fbedf7fe700 1 -- 172.21.15.33:0/4239385084 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+10899 (secure 0 0 0) 0x7fbee409a600 con 0x7fbef0101fd0 2024-05-08T21:27:49.580 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:27:49.580 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:25:13.388928+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.584+0000 7fbef4de0700 1 -- 172.21.15.33:0/4239385084 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbed80a11a0 msgr2=0x7fbed80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.584+0000 7fbef4de0700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbed80a11a0 0x7fbed80a3660 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009c60 tx=0x7fbee0009260 comp rx=0 tx=0).stop 2024-05-08T21:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.584+0000 7fbef4de0700 1 -- 172.21.15.33:0/4239385084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef0101fd0 msgr2=0x7fbef0073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.584+0000 7fbef4de0700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef0101fd0 0x7fbef0073390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009810 tx=0x7fbee400bef0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.585+0000 7fbef4de0700 1 -- 172.21.15.33:0/4239385084 shutdown_connections 2024-05-08T21:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.585+0000 7fbef4de0700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbef0102990 0x7fbef00738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.585+0000 7fbef4de0700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0100e80 0x7fbef0074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.585+0000 7fbef4de0700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbed80a11a0 0x7fbed80a3660 secure :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009c60 tx=0x7fbee0009260 comp rx=0 tx=0).stop 2024-05-08T21:27:49.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.585+0000 7fbef4de0700 1 --2- 172.21.15.33:0/4239385084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbef0101fd0 0x7fbef0073390 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.585+0000 7fbef4de0700 1 -- 172.21.15.33:0/4239385084 >> 172.21.15.33:0/4239385084 conn(0x7fbef00fc410 msgr2=0x7fbef0105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:49.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.585+0000 7fbef4de0700 1 -- 172.21.15.33:0/4239385084 shutdown_connections 2024-05-08T21:27:49.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.589+0000 7fbef4de0700 1 -- 172.21.15.33:0/4239385084 wait complete. 2024-05-08T21:27:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 39 2024-05-08T21:27:49.613 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:27:49.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:49 smithi033 ceph-mon[42528]: pgmap v555: 129 pgs: 129 active+clean; 287 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 468 op/s 2024-05-08T21:27:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.794+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/775169318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4104280 msgr2=0x7ff1e4106670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.794+0000 7ff1e8b6f700 1 --2- 172.21.15.33:0/775169318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4104280 0x7ff1e4106670 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009990 tx=0x7ff1d8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.794+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/775169318 shutdown_connections 2024-05-08T21:27:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.794+0000 7ff1e8b6f700 1 --2- 172.21.15.33:0/775169318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1e4106bb0 0x7ff1e4108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.794+0000 7ff1e8b6f700 1 --2- 172.21.15.33:0/775169318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1e4101950 0x7ff1e4103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.794+0000 7ff1e8b6f700 1 --2- 172.21.15.33:0/775169318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4104280 0x7ff1e4106670 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.794+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/775169318 >> 172.21.15.33:0/775169318 conn(0x7ff1e40fb2a0 msgr2=0x7ff1e40fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.794+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/775169318 shutdown_connections 2024-05-08T21:27:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.794+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/775169318 wait complete. 2024-05-08T21:27:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.795+0000 7ff1e8b6f700 1 Processor -- start 2024-05-08T21:27:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.795+0000 7ff1e8b6f700 1 -- start start 2024-05-08T21:27:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.795+0000 7ff1e8b6f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4101950 0x7ff1e4196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.795+0000 7ff1e8b6f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1e4104280 0x7ff1e41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.795+0000 7ff1e8b6f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1e4106bb0 0x7ff1e419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e8b6f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e419e880 con 0x7ff1e4101950 2024-05-08T21:27:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e8b6f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e419e9f0 con 0x7ff1e4106bb0 2024-05-08T21:27:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e8b6f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e419ebb0 con 0x7ff1e4104280 2024-05-08T21:27:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1e4104280 0x7ff1e41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4101950 0x7ff1e4196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e1d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1e4104280 0x7ff1e41972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39190/0 (socket says 172.21.15.33:39190) 2024-05-08T21:27:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e1d9b700 1 -- 172.21.15.33:0/4185476423 learned_addr learned my addr 172.21.15.33:0/4185476423 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e259c700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4101950 0x7ff1e4196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54222/0 (socket says 172.21.15.33:54222) 2024-05-08T21:27:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e2d9d700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1e4106bb0 0x7ff1e419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e1d9b700 1 -- 172.21.15.33:0/4185476423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1e4106bb0 msgr2=0x7ff1e419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e1d9b700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1e4106bb0 0x7ff1e419e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e1d9b700 1 -- 172.21.15.33:0/4185476423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4101950 msgr2=0x7ff1e4196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e1d9b700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4101950 0x7ff1e4196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e1d9b700 1 -- 172.21.15.33:0/4185476423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d80095f0 con 0x7ff1e4104280 2024-05-08T21:27:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e2d9d700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1e4106bb0 0x7ff1e419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:27:49.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1e1d9b700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1e4104280 0x7ff1e41972d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8003ec0 tx=0x7ff1d8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:49.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1d37fe700 1 -- 172.21.15.33:0/4185476423 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d801d070 con 0x7ff1e4104280 2024-05-08T21:27:49.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.796+0000 7ff1d37fe700 1 -- 172.21.15.33:0/4185476423 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1d800bdb0 con 0x7ff1e4104280 2024-05-08T21:27:49.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.797+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/4185476423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e419ee30 con 0x7ff1e4104280 2024-05-08T21:27:49.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.797+0000 7ff1d37fe700 1 -- 172.21.15.33:0/4185476423 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d800fda0 con 0x7ff1e4104280 2024-05-08T21:27:49.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.797+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/4185476423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e419f1f0 con 0x7ff1e4104280 2024-05-08T21:27:49.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.799+0000 7ff1d37fe700 1 -- 172.21.15.33:0/4185476423 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff1d800f460 con 0x7ff1e4104280 2024-05-08T21:27:49.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.799+0000 7ff1d37fe700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff1cc0a11a0 0x7ff1cc0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.800+0000 7ff1d37fe700 1 -- 172.21.15.33:0/4185476423 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff1d80cde90 con 0x7ff1e4104280 2024-05-08T21:27:49.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.800+0000 7ff1e259c700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff1cc0a11a0 0x7ff1cc0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.800+0000 7ff1e259c700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff1cc0a11a0 0x7ff1cc0a3660 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4198210 tx=0x7ff1d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.801+0000 7ff1f4d29b80 1 Processor -- start 2024-05-08T21:27:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.801+0000 7ff1f4d29b80 1 -- start start 2024-05-08T21:27:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.801+0000 7ff1f4d29b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024540 0x55fd96024960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.801+0000 7ff1f4d29b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96024ea0 0x55fd96025320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.801+0000 7ff1f4d29b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96025860 0x55fd96025ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.801+0000 7ff1f4d29b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd95f7b150 con 0x55fd96024540 2024-05-08T21:27:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.801+0000 7ff1f4d29b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd95f78b10 con 0x55fd96025860 2024-05-08T21:27:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.801+0000 7ff1f4d29b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd95f74400 con 0x55fd96024ea0 2024-05-08T21:27:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.801+0000 7ff1e259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024540 0x55fd96024960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.801+0000 7ff1e259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024540 0x55fd96024960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54238/0 (socket says 172.21.15.33:54238) 2024-05-08T21:27:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.801+0000 7ff1e259c700 1 -- 172.21.15.33:0/1993986919 learned_addr learned my addr 172.21.15.33:0/1993986919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.802+0000 7ff1e1d9b700 1 --2- 172.21.15.33:0/1993986919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96024ea0 0x55fd96025320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.802+0000 7ff1e259c700 1 -- 172.21.15.33:0/1993986919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96024ea0 msgr2=0x55fd96025320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.802+0000 7ff1e259c700 1 --2- 172.21.15.33:0/1993986919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96024ea0 0x55fd96025320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.802+0000 7ff1e2d9d700 1 --2- 172.21.15.33:0/1993986919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96025860 0x55fd96025ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.802+0000 7ff1e259c700 1 -- 172.21.15.33:0/1993986919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96025860 msgr2=0x55fd96025ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:49.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.802+0000 7ff1e259c700 1 --2- 172.21.15.33:0/1993986919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96025860 0x55fd96025ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.802+0000 7ff1e259c700 1 -- 172.21.15.33:0/1993986919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd95df53b0 con 0x55fd96024540 2024-05-08T21:27:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.802+0000 7ff1e1d9b700 1 --2- 172.21.15.33:0/1993986919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96024ea0 0x55fd96025320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.803+0000 7ff1e2d9d700 1 --2- 172.21.15.33:0/1993986919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96025860 0x55fd96025ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:27:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.803+0000 7ff1e259c700 1 --2- 172.21.15.33:0/1993986919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024540 0x55fd96024960 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4011d50 tx=0x7ff1d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becf7788d2267984 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.803+0000 7ff1d0ff9700 1 -- 172.21.15.33:0/1993986919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400eb20 con 0x55fd96024540 2024-05-08T21:27:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.803+0000 7ff1d0ff9700 1 -- 172.21.15.33:0/1993986919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1d4025070 con 0x55fd96024540 2024-05-08T21:27:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.803+0000 7ff1d0ff9700 1 -- 172.21.15.33:0/1993986919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4021de0 con 0x55fd96024540 2024-05-08T21:27:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.804+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1993986919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024540 msgr2=0x55fd96024960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.804+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/1993986919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024540 0x55fd96024960 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4011d50 tx=0x7ff1d4014000 comp rx=0 tx=0).stop 2024-05-08T21:27:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.804+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1993986919 shutdown_connections 2024-05-08T21:27:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.804+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/1993986919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96025860 0x55fd96025ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.804+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/1993986919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024540 0x55fd96024960 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.804+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/1993986919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96024ea0 0x55fd96025320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.804+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1993986919 >> 172.21.15.33:0/1993986919 conn(0x55fd95f82210 msgr2=0x55fd9601ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.804+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1993986919 shutdown_connections 2024-05-08T21:27:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.804+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1993986919 wait complete. 2024-05-08T21:27:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.805+0000 7ff1f4d29b80 1 -- start start 2024-05-08T21:27:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.805+0000 7ff1f4d29b80 1 Processor -- start 2024-05-08T21:27:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.805+0000 7ff1f4d29b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024540 0x55fd9602de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.805+0000 7ff1f4d29b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024ea0 0x55fd9602e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.805+0000 7ff1f4d29b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96025860 0x55fd96035380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.805+0000 7ff1f4d29b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd95f10560 con 0x55fd96024540 2024-05-08T21:27:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.805+0000 7ff1f4d29b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd95f10200 con 0x55fd96024ea0 2024-05-08T21:27:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.805+0000 7ff1f4d29b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd95df76a0 con 0x55fd96025860 2024-05-08T21:27:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.805+0000 7ff1f4d29b80 10 client.0 _ll_register_callbacks cb 0x55fd95f19b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.806+0000 7ff1e2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96025860 0x55fd96035380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.806+0000 7ff1e2d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96025860 0x55fd96035380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39202/0 (socket says 172.21.15.33:39202) 2024-05-08T21:27:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.806+0000 7ff1e2d9d700 1 -- 172.21.15.33:0/1825457082 learned_addr learned my addr 172.21.15.33:0/1825457082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:49.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.806+0000 7ff1e259c700 1 --2- 172.21.15.33:0/1825457082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024540 0x55fd9602de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.806+0000 7ff1e2d9d700 1 -- 172.21.15.33:0/1825457082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024ea0 msgr2=0x55fd9602e340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:27:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.806+0000 7ff1e2d9d700 1 --2- 172.21.15.33:0/1825457082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024ea0 0x55fd9602e340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.806+0000 7ff1e2d9d700 1 -- 172.21.15.33:0/1825457082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024540 msgr2=0x55fd9602de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.806+0000 7ff1e2d9d700 1 --2- 172.21.15.33:0/1825457082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024540 0x55fd9602de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.806+0000 7ff1e2d9d700 1 -- 172.21.15.33:0/1825457082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e419f1f0 con 0x55fd96025860 2024-05-08T21:27:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.806+0000 7ff1e2d9d700 1 --2- 172.21.15.33:0/1825457082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96025860 0x55fd96035380 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009690 tx=0x7ff1dc007d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=521a8dd652431e52 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.807+0000 7ff1c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.807+0000 7ff1c9ffb700 1 -- 172.21.15.33:0/1825457082 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc004db0 con 0x55fd96025860 2024-05-08T21:27:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.807+0000 7ff1c9ffb700 1 -- 172.21.15.33:0/1825457082 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff1dc010760 con 0x55fd96025860 2024-05-08T21:27:49.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.807+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1825457082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd95ecd270 con 0x55fd96025860 2024-05-08T21:27:49.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.807+0000 7ff1c9ffb700 1 -- 172.21.15.33:0/1825457082 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1dc010920 con 0x55fd96025860 2024-05-08T21:27:49.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.807+0000 7ff1f4d29b80 10 client.27715 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:49.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.807+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1825457082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7ff1d4017070 con 0x55fd96025860 2024-05-08T21:27:49.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.807+0000 7ff1c9ffb700 1 -- 172.21.15.33:0/1825457082 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fd95ecd270 con 0x55fd96025860 2024-05-08T21:27:49.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.808+0000 7ff1c9ffb700 1 -- 172.21.15.33:0/1825457082 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff1dc010a80 con 0x55fd96025860 2024-05-08T21:27:49.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.808+0000 7ff1f4d29b80 10 client.27715 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:49.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.808+0000 7ff1f4d29b80 10 client.27715 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:27:49.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.808+0000 7ff1f4d29b80 20 client.27715 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:49.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.808+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/1825457082 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55fd96039c00 0x55fd9603c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.808+0000 7ff1f4d29b80 4 client.27715 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:49.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.808+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1825457082 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd95ff3810 con 0x55fd96039c00 2024-05-08T21:27:49.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.809+0000 7ff1e259c700 1 --2- 172.21.15.33:0/1825457082 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55fd96039c00 0x55fd9603c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.809+0000 7ff1e259c700 1 --2- 172.21.15.33:0/1825457082 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55fd96039c00 0x55fd9603c020 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2db665498c8e95ef server_cookie=c1fba43ad7cabfe0 in_seq=0 out_seq=0 2024-05-08T21:27:49.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.809+0000 7ff1c9ffb700 10 client.27715 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:27:49.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1c9ffb700 1 -- 172.21.15.33:0/1825457082 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd95ff3810 con 0x55fd96039c00 2024-05-08T21:27:49.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1c9ffb700 10 client.27715 handle_command_reply: tid=0 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1f4d29b80 1 client.27715 shutdown 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1825457082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96025860 msgr2=0x55fd96035380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/1825457082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96025860 0x55fd96035380 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009690 tx=0x7ff1dc007d40 comp rx=0 tx=0).stop 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1825457082 shutdown_connections 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/1825457082 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55fd96039c00 0x55fd9603c020 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/1825457082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96025860 0x55fd96035380 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/1825457082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024540 0x55fd9602de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/1825457082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024ea0 0x55fd9602e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1825457082 >> 172.21.15.33:0/1825457082 conn(0x55fd95f82210 msgr2=0x55fd9601e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1c9ffb700 0 client.27715 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1825457082 shutdown_connections 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.810+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/1825457082 wait complete. 2024-05-08T21:27:49.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.811+0000 7ff1f4d29b80 20 client.27715 trim_cache size 0 max 16384 2024-05-08T21:27:49.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:49 smithi071 ceph-mon[47074]: pgmap v555: 129 pgs: 129 active+clean; 287 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 468 op/s 2024-05-08T21:27:49.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1f4d29b80 1 Processor -- start 2024-05-08T21:27:49.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1f4d29b80 1 -- start start 2024-05-08T21:27:49.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1f4d29b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024540 0x55fd96024920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1f4d29b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96024ea0 0x55fd96035340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1f4d29b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96025860 0x55fd9602d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1f4d29b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd9601bdc0 con 0x55fd96025860 2024-05-08T21:27:49.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1f4d29b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd96036500 con 0x55fd96024540 2024-05-08T21:27:49.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1f4d29b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd96020570 con 0x55fd96024ea0 2024-05-08T21:27:49.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1e2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96025860 0x55fd9602d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1e2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96025860 0x55fd9602d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54254/0 (socket says 172.21.15.33:54254) 2024-05-08T21:27:49.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1e259c700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024540 0x55fd96024920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1e2d9d700 1 -- 172.21.15.33:0/2189017449 learned_addr learned my addr 172.21.15.33:0/2189017449 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:49.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.828+0000 7ff1e1d9b700 1 --2- 172.21.15.33:0/2189017449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96024ea0 0x55fd96035340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.829+0000 7ff1e2d9d700 1 -- 172.21.15.33:0/2189017449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96024ea0 msgr2=0x55fd96035340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:49.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.829+0000 7ff1e2d9d700 1 --2- 172.21.15.33:0/2189017449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96024ea0 0x55fd96035340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.829+0000 7ff1e2d9d700 1 -- 172.21.15.33:0/2189017449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024540 msgr2=0x55fd96024920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:49.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.829+0000 7ff1e2d9d700 1 --2- 172.21.15.33:0/2189017449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024540 0x55fd96024920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.829+0000 7ff1e2d9d700 1 -- 172.21.15.33:0/2189017449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd95da1b60 con 0x55fd96025860 2024-05-08T21:27:49.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.829+0000 7ff1e2d9d700 1 --2- 172.21.15.33:0/2189017449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96025860 0x55fd9602d310 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc010e90 tx=0x7ff1dc010fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c933871b8711654 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:49.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.829+0000 7ff1d17fa700 1 -- 172.21.15.33:0/2189017449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc016420 con 0x55fd96025860 2024-05-08T21:27:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.829+0000 7ff1d17fa700 1 -- 172.21.15.33:0/2189017449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1dc00f070 con 0x55fd96025860 2024-05-08T21:27:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.829+0000 7ff1d17fa700 1 -- 172.21.15.33:0/2189017449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc016a90 con 0x55fd96025860 2024-05-08T21:27:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.830+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2189017449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96025860 msgr2=0x55fd9602d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.830+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/2189017449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96025860 0x55fd9602d310 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc010e90 tx=0x7ff1dc010fb0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.830+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2189017449 shutdown_connections 2024-05-08T21:27:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.830+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/2189017449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96025860 0x55fd9602d310 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.830+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/2189017449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024540 0x55fd96024920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.830+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/2189017449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96024ea0 0x55fd96035340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.830+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2189017449 >> 172.21.15.33:0/2189017449 conn(0x55fd95f82210 msgr2=0x55fd96025c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.831+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2189017449 shutdown_connections 2024-05-08T21:27:49.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.831+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2189017449 wait complete. 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.831+0000 7ff1f4d29b80 1 -- start start 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.831+0000 7ff1f4d29b80 1 Processor -- start 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.831+0000 7ff1f4d29b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024540 0x55fd96030900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.831+0000 7ff1f4d29b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024ea0 0x55fd96030e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.831+0000 7ff1f4d29b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96025860 0x55fd96033dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.831+0000 7ff1f4d29b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd96034300 con 0x55fd96024ea0 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.831+0000 7ff1f4d29b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd96034470 con 0x55fd96024540 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.831+0000 7ff1f4d29b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd96034630 con 0x55fd96025860 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.832+0000 7ff1f4d29b80 10 client.0 _ll_register_callbacks cb 0x55fd95f19b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.832+0000 7ff1e1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024ea0 0x55fd96030e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.832+0000 7ff1e1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024ea0 0x55fd96030e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54258/0 (socket says 172.21.15.33:54258) 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.832+0000 7ff1e1d9b700 1 -- 172.21.15.33:0/2761967832 learned_addr learned my addr 172.21.15.33:0/2761967832 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:49.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.832+0000 7ff1e2d9d700 1 --2- 172.21.15.33:0/2761967832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96025860 0x55fd96033dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.832+0000 7ff1e1d9b700 1 -- 172.21.15.33:0/2761967832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96025860 msgr2=0x55fd96033dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:49.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.832+0000 7ff1e1d9b700 1 --2- 172.21.15.33:0/2761967832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96025860 0x55fd96033dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.832+0000 7ff1e1d9b700 1 -- 172.21.15.33:0/2761967832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024540 msgr2=0x55fd96030900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:49.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.832+0000 7ff1e1d9b700 1 --2- 172.21.15.33:0/2761967832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024540 0x55fd96030900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.832+0000 7ff1e1d9b700 1 -- 172.21.15.33:0/2761967832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d4017070 con 0x55fd96024ea0 2024-05-08T21:27:49.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.832+0000 7ff1e1d9b700 1 --2- 172.21.15.33:0/2761967832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024ea0 0x55fd96030e40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8003f60 tx=0x7ff1d809b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1f5e62f85f57bc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:49.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.832+0000 7ff1c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:27:49.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.833+0000 7ff1c9ffb700 1 -- 172.21.15.33:0/2761967832 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80a3150 con 0x55fd96024ea0 2024-05-08T21:27:49.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.833+0000 7ff1c9ffb700 1 -- 172.21.15.33:0/2761967832 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff1d809ee20 con 0x55fd96024ea0 2024-05-08T21:27:49.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.833+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2761967832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd9602ce10 con 0x55fd96024ea0 2024-05-08T21:27:49.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.835+0000 7ff1c9ffb700 1 -- 172.21.15.33:0/2761967832 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d80a33e0 con 0x55fd96024ea0 2024-05-08T21:27:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.835+0000 7ff1f4d29b80 10 client.17802 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.835+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2761967832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55fd95f3a560 con 0x55fd96024ea0 2024-05-08T21:27:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.835+0000 7ff1c9ffb700 1 -- 172.21.15.33:0/2761967832 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fd9602ce10 con 0x55fd96024ea0 2024-05-08T21:27:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.835+0000 7ff1c9ffb700 1 -- 172.21.15.33:0/2761967832 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff1d8096de0 con 0x55fd96024ea0 2024-05-08T21:27:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.836+0000 7ff1f4d29b80 10 client.17802 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.836+0000 7ff1f4d29b80 10 client.17802 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:27:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.836+0000 7ff1f4d29b80 20 client.17802 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.836+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/2761967832 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55fd96090470 0x55fd96092860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.836+0000 7ff1f4d29b80 4 client.17802 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:27:49.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.836+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2761967832 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7ff1d4025070 con 0x55fd96090470 2024-05-08T21:27:49.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.836+0000 7ff1e259c700 1 --2- 172.21.15.33:0/2761967832 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55fd96090470 0x55fd96092860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:49.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.837+0000 7ff1e259c700 1 --2- 172.21.15.33:0/2761967832 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55fd96090470 0x55fd96092860 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8aaebad7002060f2 server_cookie=3ccb869438b3a616 in_seq=0 out_seq=0 2024-05-08T21:27:49.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.837+0000 7ff1c9ffb700 10 client.17802 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:27:49.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1c9ffb700 1 -- 172.21.15.33:0/2761967832 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7ff1d4025070 con 0x55fd96090470 2024-05-08T21:27:49.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1c9ffb700 10 client.17802 handle_command_reply: tid=0 2024-05-08T21:27:49.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1f4d29b80 1 client.17802 shutdown 2024-05-08T21:27:49.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2761967832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024ea0 msgr2=0x55fd96030e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/2761967832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024ea0 0x55fd96030e40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8003f60 tx=0x7ff1d809b530 comp rx=0 tx=0).stop 2024-05-08T21:27:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2761967832 shutdown_connections 2024-05-08T21:27:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/2761967832 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55fd96090470 0x55fd96092860 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/2761967832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fd96025860 0x55fd96033dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/2761967832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd96024540 0x55fd96030900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1f4d29b80 1 --2- 172.21.15.33:0/2761967832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fd96024ea0 0x55fd96030e40 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1c9ffb700 0 client.17802 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:27:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2761967832 >> 172.21.15.33:0/2761967832 conn(0x55fd95f82210 msgr2=0x55fd9603bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2761967832 shutdown_connections 2024-05-08T21:27:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1f4d29b80 1 -- 172.21.15.33:0/2761967832 wait complete. 2024-05-08T21:27:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.838+0000 7ff1f4d29b80 20 client.17802 trim_cache size 0 max 16384 2024-05-08T21:27:49.844 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:27:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.839+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/4185476423 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff1cc0a11a0 msgr2=0x7ff1cc0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.839+0000 7ff1e8b6f700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff1cc0a11a0 0x7ff1cc0a3660 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4198210 tx=0x7ff1d4009260 comp rx=0 tx=0).stop 2024-05-08T21:27:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.839+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/4185476423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1e4104280 msgr2=0x7ff1e41972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:49.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.839+0000 7ff1e8b6f700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1e4104280 0x7ff1e41972d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8003ec0 tx=0x7ff1d8004730 comp rx=0 tx=0).stop 2024-05-08T21:27:49.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.839+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/4185476423 shutdown_connections 2024-05-08T21:27:49.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.839+0000 7ff1e8b6f700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff1cc0a11a0 0x7ff1cc0a3660 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.839+0000 7ff1e8b6f700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1e4106bb0 0x7ff1e419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.839+0000 7ff1e8b6f700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e4101950 0x7ff1e4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.839+0000 7ff1e8b6f700 1 --2- 172.21.15.33:0/4185476423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1e4104280 0x7ff1e41972d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:49.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.839+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/4185476423 >> 172.21.15.33:0/4185476423 conn(0x7ff1e40fb2a0 msgr2=0x7ff1e4103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:49.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.839+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/4185476423 shutdown_connections 2024-05-08T21:27:49.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:49.839+0000 7ff1e8b6f700 1 -- 172.21.15.33:0/4185476423 wait complete. 2024-05-08T21:27:49.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:49 smithi082 ceph-mon[47512]: pgmap v555: 129 pgs: 129 active+clean; 287 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 468 op/s 2024-05-08T21:27:49.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:27:50.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.047+0000 7fc955162700 1 -- 172.21.15.33:0/1494468792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc950104320 msgr2=0x7fc950106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.047+0000 7fc955162700 1 --2- 172.21.15.33:0/1494468792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc950104320 0x7fc950106710 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc938009990 tx=0x7fc938009ca0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.047+0000 7fc955162700 1 -- 172.21.15.33:0/1494468792 shutdown_connections 2024-05-08T21:27:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.047+0000 7fc955162700 1 --2- 172.21.15.33:0/1494468792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc950106c50 0x7fc950109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.047+0000 7fc955162700 1 --2- 172.21.15.33:0/1494468792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9501019f0 0x7fc950103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.047+0000 7fc955162700 1 --2- 172.21.15.33:0/1494468792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc950104320 0x7fc950106710 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.047+0000 7fc955162700 1 -- 172.21.15.33:0/1494468792 >> 172.21.15.33:0/1494468792 conn(0x7fc9500fb340 msgr2=0x7fc9500fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.047+0000 7fc955162700 1 -- 172.21.15.33:0/1494468792 shutdown_connections 2024-05-08T21:27:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.047+0000 7fc955162700 1 -- 172.21.15.33:0/1494468792 wait complete. 2024-05-08T21:27:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.048+0000 7fc955162700 1 Processor -- start 2024-05-08T21:27:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.048+0000 7fc955162700 1 -- start start 2024-05-08T21:27:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc955162700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9501019f0 0x7fc9501012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc955162700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc950104320 0x7fc9500ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc955162700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc950106c50 0x7fc9500ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc955162700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501005e0 con 0x7fc9501019f0 2024-05-08T21:27:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc955162700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc950100750 con 0x7fc950106c50 2024-05-08T21:27:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc955162700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9501a0dc0 con 0x7fc950104320 2024-05-08T21:27:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc94659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc950104320 0x7fc9500ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc94ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9501019f0 0x7fc9501012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc94659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc950104320 0x7fc9500ff9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39238/0 (socket says 172.21.15.33:39238) 2024-05-08T21:27:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc94659c700 1 -- 172.21.15.33:0/1242043247 learned_addr learned my addr 172.21.15.33:0/1242043247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc94f59e700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc950106c50 0x7fc9500ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc94659c700 1 -- 172.21.15.33:0/1242043247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc950106c50 msgr2=0x7fc9500ffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc94659c700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc950106c50 0x7fc9500ffee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc94659c700 1 -- 172.21.15.33:0/1242043247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9501019f0 msgr2=0x7fc9501012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc94659c700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9501019f0 0x7fc9501012c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc94659c700 1 -- 172.21.15.33:0/1242043247 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948009c40 con 0x7fc950104320 2024-05-08T21:27:50.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.049+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9501019f0 0x7fc9501012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:50.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.050+0000 7fc94659c700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc950104320 0x7fc9500ff9a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc938003ec0 tx=0x7fc938004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.050+0000 7fc94cd99700 1 -- 172.21.15.33:0/1242043247 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93801d070 con 0x7fc950104320 2024-05-08T21:27:50.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.050+0000 7fc94cd99700 1 -- 172.21.15.33:0/1242043247 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc93800bdb0 con 0x7fc950104320 2024-05-08T21:27:50.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.050+0000 7fc955162700 1 -- 172.21.15.33:0/1242043247 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9380095f0 con 0x7fc950104320 2024-05-08T21:27:50.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.050+0000 7fc94cd99700 1 -- 172.21.15.33:0/1242043247 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93800fd20 con 0x7fc950104320 2024-05-08T21:27:50.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.050+0000 7fc955162700 1 -- 172.21.15.33:0/1242043247 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9501a11f0 con 0x7fc950104320 2024-05-08T21:27:50.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.055+0000 7fc94cd99700 1 -- 172.21.15.33:0/1242043247 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc93800f4f0 con 0x7fc950104320 2024-05-08T21:27:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.055+0000 7fc96131cb80 1 Processor -- start 2024-05-08T21:27:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.055+0000 7fc96131cb80 1 -- start start 2024-05-08T21:27:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.056+0000 7fc96131cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfd3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.056+0000 7fc96131cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3e80 0x56037dfd4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.056+0000 7fc96131cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037dfdb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.056+0000 7fc96131cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037df2d150 con 0x56037dfd3520 2024-05-08T21:27:50.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.056+0000 7fc96131cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037df2ab10 con 0x56037dfd3e80 2024-05-08T21:27:50.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.056+0000 7fc96131cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037df26400 con 0x56037dfd4840 2024-05-08T21:27:50.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.056+0000 7fc94cd99700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc93c0a11a0 0x7fc93c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.056+0000 7fc94ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3e80 0x56037dfd4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.056+0000 7fc94ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3e80 0x56037dfd4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57482/0 (socket says 172.21.15.33:57482) 2024-05-08T21:27:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.056+0000 7fc94ed9d700 1 -- 172.21.15.33:0/757018588 learned_addr learned my addr 172.21.15.33:0/757018588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.056+0000 7fc94cd99700 1 -- 172.21.15.33:0/1242043247 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc9380cdef0 con 0x7fc950104320 2024-05-08T21:27:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.057+0000 7fc94ed9d700 1 -- 172.21.15.33:0/757018588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 msgr2=0x56037dfdb5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:27:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.057+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/757018588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037dfdb5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.057+0000 7fc94ed9d700 1 -- 172.21.15.33:0/757018588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 msgr2=0x56037dfd3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.057+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/757018588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfd3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.057+0000 7fc94ed9d700 1 -- 172.21.15.33:0/757018588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56037dda73b0 con 0x56037dfd3e80 2024-05-08T21:27:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.057+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/757018588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3e80 0x56037dfd4300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc940033600 tx=0x7fc940033910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e04b190743aa27d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.057+0000 7fc94559a700 1 -- 172.21.15.33:0/757018588 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94003b410 con 0x56037dfd3e80 2024-05-08T21:27:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.057+0000 7fc94559a700 1 -- 172.21.15.33:0/757018588 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc94003ba00 con 0x56037dfd3e80 2024-05-08T21:27:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.057+0000 7fc94559a700 1 -- 172.21.15.33:0/757018588 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94003a9e0 con 0x56037dfd3e80 2024-05-08T21:27:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.058+0000 7fc96131cb80 1 -- 172.21.15.33:0/757018588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3e80 msgr2=0x56037dfd4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.059+0000 7fc96131cb80 1 --2- 172.21.15.33:0/757018588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3e80 0x56037dfd4300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc940033600 tx=0x7fc940033910 comp rx=0 tx=0).stop 2024-05-08T21:27:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.062+0000 7fc94f59e700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc93c0a11a0 0x7fc93c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.062+0000 7fc94f59e700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc93c0a11a0 0x7fc93c0a3660 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x56037dfd1cf0 tx=0x7fc948009920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.064+0000 7fc96131cb80 1 -- 172.21.15.33:0/757018588 shutdown_connections 2024-05-08T21:27:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.064+0000 7fc96131cb80 1 --2- 172.21.15.33:0/757018588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037dfdb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.064+0000 7fc96131cb80 1 --2- 172.21.15.33:0/757018588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfd3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.064+0000 7fc96131cb80 1 --2- 172.21.15.33:0/757018588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3e80 0x56037dfd4300 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.064+0000 7fc96131cb80 1 -- 172.21.15.33:0/757018588 >> 172.21.15.33:0/757018588 conn(0x56037df34210 msgr2=0x56037dfd0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.066+0000 7fc96131cb80 1 -- 172.21.15.33:0/757018588 shutdown_connections 2024-05-08T21:27:50.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.067+0000 7fc96131cb80 1 -- 172.21.15.33:0/757018588 wait complete. 2024-05-08T21:27:50.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.068+0000 7fc96131cb80 1 -- start start 2024-05-08T21:27:50.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.068+0000 7fc96131cb80 1 Processor -- start 2024-05-08T21:27:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.069+0000 7fc96131cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3520 0x56037dfdfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.069+0000 7fc96131cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3e80 0x56037dfe0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.069+0000 7fc96131cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037dfe7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.069+0000 7fc96131cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037dec2560 con 0x56037dfd3e80 2024-05-08T21:27:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.069+0000 7fc96131cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037dec2200 con 0x56037dfd3520 2024-05-08T21:27:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.069+0000 7fc96131cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037dda96a0 con 0x56037dfd4840 2024-05-08T21:27:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.069+0000 7fc96131cb80 10 client.0 _ll_register_callbacks cb 0x56037decbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.069+0000 7fc94f59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037dfe7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.069+0000 7fc94ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3520 0x56037dfdfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.069+0000 7fc94ed9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3520 0x56037dfdfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57484/0 (socket says 172.21.15.33:57484) 2024-05-08T21:27:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.069+0000 7fc94ed9d700 1 -- 172.21.15.33:0/3301360742 learned_addr learned my addr 172.21.15.33:0/3301360742 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.070+0000 7fc94ed9d700 1 -- 172.21.15.33:0/3301360742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 msgr2=0x56037dfe7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.070+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/3301360742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037dfe7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.070+0000 7fc94ed9d700 1 -- 172.21.15.33:0/3301360742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3e80 msgr2=0x56037dfe0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.070+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/3301360742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3e80 0x56037dfe0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.070+0000 7fc94ed9d700 1 -- 172.21.15.33:0/3301360742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc940009dd0 con 0x56037dfd3520 2024-05-08T21:27:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.073+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/3301360742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3520 0x56037dfdfec0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc940005920 tx=0x7fc94003b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b73656b9c430a09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.073+0000 7fc9367fc700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:27:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc9367fc700 1 -- 172.21.15.33:0/3301360742 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94003a3b0 con 0x56037dfd3520 2024-05-08T21:27:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc9367fc700 1 -- 172.21.15.33:0/3301360742 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc940005ce0 con 0x56037dfd3520 2024-05-08T21:27:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc96131cb80 1 -- 172.21.15.33:0/3301360742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56037de7f270 con 0x56037dfd3520 2024-05-08T21:27:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc9367fc700 1 -- 172.21.15.33:0/3301360742 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc940004240 con 0x56037dfd3520 2024-05-08T21:27:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc96131cb80 10 client.27707 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc96131cb80 1 -- 172.21.15.33:0/3301360742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56037dd53b60 con 0x56037dfd3520 2024-05-08T21:27:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc9367fc700 1 -- 172.21.15.33:0/3301360742 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56037de7f270 con 0x56037dfd3520 2024-05-08T21:27:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc9367fc700 1 -- 172.21.15.33:0/3301360742 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc940003710 con 0x56037dfd3520 2024-05-08T21:27:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc96131cb80 10 client.27707 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc96131cb80 10 client.27707 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:27:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc96131cb80 20 client.27707 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc96131cb80 1 --2- 172.21.15.33:0/3301360742 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x56037dfebc90 0x56037dfee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc96131cb80 4 client.27707 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.074+0000 7fc96131cb80 1 -- 172.21.15.33:0/3301360742 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56037dfa5810 con 0x56037dfebc90 2024-05-08T21:27:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.079+0000 7fc94659c700 1 --2- 172.21.15.33:0/3301360742 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x56037dfebc90 0x56037dfee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.080+0000 7fc9367fc700 10 client.27707 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:27:50.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.080+0000 7fc94659c700 1 --2- 172.21.15.33:0/3301360742 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x56037dfebc90 0x56037dfee0b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7f00afd5102edebc server_cookie=eacc2eb65af0a73c in_seq=0 out_seq=0 2024-05-08T21:27:50.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.081+0000 7fc9367fc700 1 -- 172.21.15.33:0/3301360742 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56037dfa5810 con 0x56037dfebc90 2024-05-08T21:27:50.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.081+0000 7fc9367fc700 10 client.27707 handle_command_reply: tid=0 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.081+0000 7fc96131cb80 1 client.27707 shutdown 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.081+0000 7fc96131cb80 1 -- 172.21.15.33:0/3301360742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3520 msgr2=0x56037dfdfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.081+0000 7fc96131cb80 1 --2- 172.21.15.33:0/3301360742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3520 0x56037dfdfec0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc940005920 tx=0x7fc94003b640 comp rx=0 tx=0).stop 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.081+0000 7fc96131cb80 1 -- 172.21.15.33:0/3301360742 shutdown_connections 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.081+0000 7fc96131cb80 1 --2- 172.21.15.33:0/3301360742 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x56037dfebc90 0x56037dfee0b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.081+0000 7fc96131cb80 1 --2- 172.21.15.33:0/3301360742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037dfe7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.081+0000 7fc96131cb80 1 --2- 172.21.15.33:0/3301360742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3520 0x56037dfdfec0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.081+0000 7fc96131cb80 1 --2- 172.21.15.33:0/3301360742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3e80 0x56037dfe0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.081+0000 7fc96131cb80 1 -- 172.21.15.33:0/3301360742 >> 172.21.15.33:0/3301360742 conn(0x56037df34210 msgr2=0x56037dfd05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.081+0000 7fc9367fc700 0 client.27707 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.082+0000 7fc96131cb80 1 -- 172.21.15.33:0/3301360742 shutdown_connections 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.082+0000 7fc96131cb80 1 -- 172.21.15.33:0/3301360742 wait complete. 2024-05-08T21:27:50.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.082+0000 7fc96131cb80 20 client.27707 trim_cache size 0 max 16384 2024-05-08T21:27:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.098+0000 7fc96131cb80 1 Processor -- start 2024-05-08T21:27:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc96131cb80 1 -- start start 2024-05-08T21:27:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc96131cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfe7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc96131cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd3e80 0x56037dfd0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc96131cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd4840 0x56037dfd1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc96131cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037de7f270 con 0x56037dfd3520 2024-05-08T21:27:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc96131cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037dfdbb80 con 0x56037dfd4840 2024-05-08T21:27:50.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc96131cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037dfd2870 con 0x56037dfd3e80 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc94ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfe7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc94659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd3e80 0x56037dfd0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc94ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfe7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54314/0 (socket says 172.21.15.33:54314) 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc94ed9d700 1 -- 172.21.15.33:0/4024659233 learned_addr learned my addr 172.21.15.33:0/4024659233 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc94659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd3e80 0x56037dfd0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39246/0 (socket says 172.21.15.33:39246) 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.099+0000 7fc94f59e700 1 --2- 172.21.15.33:0/4024659233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd4840 0x56037dfd1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.100+0000 7fc94ed9d700 1 -- 172.21.15.33:0/4024659233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd3e80 msgr2=0x56037dfd0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.100+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/4024659233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd3e80 0x56037dfd0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.100+0000 7fc94ed9d700 1 -- 172.21.15.33:0/4024659233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd4840 msgr2=0x56037dfd1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.100+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/4024659233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd4840 0x56037dfd1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.100+0000 7fc94ed9d700 1 -- 172.21.15.33:0/4024659233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56037deec560 con 0x56037dfd3520 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.100+0000 7fc94f59e700 1 --2- 172.21.15.33:0/4024659233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd4840 0x56037dfd1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.100+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/4024659233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfe7400 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc940003a70 tx=0x7fc940033910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a1e92a2521b342 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.100+0000 7fc945d9b700 1 -- 172.21.15.33:0/4024659233 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940003d60 con 0x56037dfd3520 2024-05-08T21:27:50.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.100+0000 7fc945d9b700 1 -- 172.21.15.33:0/4024659233 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc94003b760 con 0x56037dfd3520 2024-05-08T21:27:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.100+0000 7fc945d9b700 1 -- 172.21.15.33:0/4024659233 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940056860 con 0x56037dfd3520 2024-05-08T21:27:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.101+0000 7fc96131cb80 1 -- 172.21.15.33:0/4024659233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 msgr2=0x56037dfe7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.101+0000 7fc96131cb80 1 --2- 172.21.15.33:0/4024659233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfe7400 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc940003a70 tx=0x7fc940033910 comp rx=0 tx=0).stop 2024-05-08T21:27:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.103+0000 7fc96131cb80 1 -- 172.21.15.33:0/4024659233 shutdown_connections 2024-05-08T21:27:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.103+0000 7fc96131cb80 1 --2- 172.21.15.33:0/4024659233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd4840 0x56037dfd1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.103+0000 7fc96131cb80 1 --2- 172.21.15.33:0/4024659233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfe7400 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.103+0000 7fc96131cb80 1 --2- 172.21.15.33:0/4024659233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd3e80 0x56037dfd0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.103+0000 7fc96131cb80 1 -- 172.21.15.33:0/4024659233 >> 172.21.15.33:0/4024659233 conn(0x56037df34210 msgr2=0x56037dfdf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.104+0000 7fc96131cb80 1 -- 172.21.15.33:0/4024659233 shutdown_connections 2024-05-08T21:27:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.104+0000 7fc96131cb80 1 -- 172.21.15.33:0/4024659233 wait complete. 2024-05-08T21:27:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.104+0000 7fc96131cb80 1 -- start start 2024-05-08T21:27:50.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.104+0000 7fc96131cb80 1 Processor -- start 2024-05-08T21:27:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.105+0000 7fc96131cb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfd9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.105+0000 7fc96131cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3e80 0x56037dfd9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.105+0000 7fc96131cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037e0401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.105+0000 7fc96131cb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037e040720 con 0x56037dfd3520 2024-05-08T21:27:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.105+0000 7fc96131cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037e040890 con 0x56037dfd3e80 2024-05-08T21:27:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.105+0000 7fc96131cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56037e040a50 con 0x56037dfd4840 2024-05-08T21:27:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.105+0000 7fc96131cb80 10 client.0 _ll_register_callbacks cb 0x56037decbb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.105+0000 7fc94f59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037e0401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.105+0000 7fc94f59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037e0401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39250/0 (socket says 172.21.15.33:39250) 2024-05-08T21:27:50.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.105+0000 7fc94f59e700 1 -- 172.21.15.33:0/401457146 learned_addr learned my addr 172.21.15.33:0/401457146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.105+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfd9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.105+0000 7fc94659c700 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3e80 0x56037dfd9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc94f59e700 1 -- 172.21.15.33:0/401457146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3e80 msgr2=0x56037dfd9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc94f59e700 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3e80 0x56037dfd9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc94f59e700 1 -- 172.21.15.33:0/401457146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 msgr2=0x56037dfd9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc94f59e700 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfd9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc94f59e700 1 -- 172.21.15.33:0/401457146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56037dd53b60 con 0x56037dfd4840 2024-05-08T21:27:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc94f59e700 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037e0401b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc9480090a0 tx=0x7fc94800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d85f11073135311a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc9367fc700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:50.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc9367fc700 1 -- 172.21.15.33:0/401457146 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800e050 con 0x56037dfd4840 2024-05-08T21:27:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc9367fc700 1 -- 172.21.15.33:0/401457146 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc948015c80 con 0x56037dfd4840 2024-05-08T21:27:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc96131cb80 1 -- 172.21.15.33:0/401457146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc94003ba00 con 0x56037dfd4840 2024-05-08T21:27:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc9367fc700 1 -- 172.21.15.33:0/401457146 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc948015e80 con 0x56037dfd4840 2024-05-08T21:27:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc96131cb80 10 client.27736 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc96131cb80 1 -- 172.21.15.33:0/401457146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56037ddabcd0 con 0x56037dfd4840 2024-05-08T21:27:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.106+0000 7fc9367fc700 1 -- 172.21.15.33:0/401457146 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc94003ba00 con 0x56037dfd4840 2024-05-08T21:27:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.107+0000 7fc9367fc700 1 -- 172.21.15.33:0/401457146 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc948024460 con 0x56037dfd4840 2024-05-08T21:27:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.107+0000 7fc96131cb80 10 client.27736 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:50.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.107+0000 7fc96131cb80 10 client.27736 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:27:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.107+0000 7fc96131cb80 20 client.27736 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.107+0000 7fc96131cb80 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x56037e042500 0x56037e044920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.107+0000 7fc96131cb80 4 client.27736 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:27:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.107+0000 7fc96131cb80 1 -- 172.21.15.33:0/401457146 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56037df1eec0 con 0x56037e042500 2024-05-08T21:27:50.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.110+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x56037e042500 0x56037e044920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.110+0000 7fc9367fc700 10 client.27736 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:27:50.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.110+0000 7fc94ed9d700 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x56037e042500 0x56037e044920 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=933a6aaaf37ab8fe server_cookie=902bcb10959483b2 in_seq=0 out_seq=0 2024-05-08T21:27:50.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.110+0000 7fc9367fc700 1 -- 172.21.15.33:0/401457146 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56037df1eec0 con 0x56037e042500 2024-05-08T21:27:50.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.110+0000 7fc9367fc700 10 client.27736 handle_command_reply: tid=0 2024-05-08T21:27:50.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.110+0000 7fc96131cb80 1 client.27736 shutdown 2024-05-08T21:27:50.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc96131cb80 1 -- 172.21.15.33:0/401457146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 msgr2=0x56037e0401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc96131cb80 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037e0401b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc9480090a0 tx=0x7fc94800b040 comp rx=0 tx=0).stop 2024-05-08T21:27:50.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc96131cb80 1 -- 172.21.15.33:0/401457146 shutdown_connections 2024-05-08T21:27:50.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc96131cb80 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x56037e042500 0x56037e044920 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc96131cb80 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56037dfd4840 0x56037e0401b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc96131cb80 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56037dfd3520 0x56037dfd9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc9367fc700 0 client.27736 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:27:50.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc96131cb80 1 --2- 172.21.15.33:0/401457146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56037dfd3e80 0x56037dfd9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc96131cb80 1 -- 172.21.15.33:0/401457146 >> 172.21.15.33:0/401457146 conn(0x56037df34210 msgr2=0x56037dfdad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc96131cb80 1 -- 172.21.15.33:0/401457146 shutdown_connections 2024-05-08T21:27:50.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc96131cb80 1 -- 172.21.15.33:0/401457146 wait complete. 2024-05-08T21:27:50.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc96131cb80 20 client.27736 trim_cache size 0 max 16384 2024-05-08T21:27:50.121 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:27:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc955162700 1 -- 172.21.15.33:0/1242043247 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc93c0a11a0 msgr2=0x7fc93c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc955162700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc93c0a11a0 0x7fc93c0a3660 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x56037dfd1cf0 tx=0x7fc948009920 comp rx=0 tx=0).stop 2024-05-08T21:27:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc955162700 1 -- 172.21.15.33:0/1242043247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc950104320 msgr2=0x7fc9500ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc955162700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc950104320 0x7fc9500ff9a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc938003ec0 tx=0x7fc938004730 comp rx=0 tx=0).stop 2024-05-08T21:27:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.111+0000 7fc955162700 1 -- 172.21.15.33:0/1242043247 shutdown_connections 2024-05-08T21:27:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.112+0000 7fc955162700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc93c0a11a0 0x7fc93c0a3660 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.112+0000 7fc955162700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc950106c50 0x7fc9500ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.112+0000 7fc955162700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9501019f0 0x7fc9501012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.112+0000 7fc955162700 1 --2- 172.21.15.33:0/1242043247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc950104320 0x7fc9500ff9a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.112+0000 7fc955162700 1 -- 172.21.15.33:0/1242043247 >> 172.21.15.33:0/1242043247 conn(0x7fc9500fb340 msgr2=0x7fc9501038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.112+0000 7fc955162700 1 -- 172.21.15.33:0/1242043247 shutdown_connections 2024-05-08T21:27:50.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.112+0000 7fc955162700 1 -- 172.21.15.33:0/1242043247 wait complete. 2024-05-08T21:27:50.130 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:27:50.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.320+0000 7f7ab82ff700 1 -- 172.21.15.33:0/1162841362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ab00ffdd0 msgr2=0x7f7ab01001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.320+0000 7f7ab82ff700 1 --2- 172.21.15.33:0/1162841362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ab00ffdd0 0x7f7ab01001f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009990 tx=0x7f7aac009ca0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.320+0000 7f7ab82ff700 1 -- 172.21.15.33:0/1162841362 shutdown_connections 2024-05-08T21:27:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.320+0000 7f7ab82ff700 1 --2- 172.21.15.33:0/1162841362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab0101960 0x7f7ab0108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.320+0000 7f7ab82ff700 1 --2- 172.21.15.33:0/1162841362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ab00ffdd0 0x7f7ab01001f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.320+0000 7f7ab82ff700 1 --2- 172.21.15.33:0/1162841362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ab0100fc0 0x7f7ab0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.320+0000 7f7ab82ff700 1 -- 172.21.15.33:0/1162841362 >> 172.21.15.33:0/1162841362 conn(0x7f7ab00fb340 msgr2=0x7f7ab00fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.320+0000 7f7ab82ff700 1 -- 172.21.15.33:0/1162841362 shutdown_connections 2024-05-08T21:27:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.320+0000 7f7ab82ff700 1 -- 172.21.15.33:0/1162841362 wait complete. 2024-05-08T21:27:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab82ff700 1 Processor -- start 2024-05-08T21:27:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab82ff700 1 -- start start 2024-05-08T21:27:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab82ff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ab00ffdd0 0x7f7ab0196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab82ff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ab0100fc0 0x7f7ab01972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab82ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab0101960 0x7f7ab019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab82ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab019e830 con 0x7f7ab0101960 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab82ff700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab019e9a0 con 0x7f7ab00ffdd0 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab82ff700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab019eb60 con 0x7f7ab0100fc0 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab609b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ab00ffdd0 0x7f7ab0196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab589a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ab0100fc0 0x7f7ab01972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab689c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab0101960 0x7f7ab019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab609b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ab00ffdd0 0x7f7ab0196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:57508/0 (socket says 172.21.15.33:57508) 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.321+0000 7f7ab609b700 1 -- 172.21.15.33:0/896030588 learned_addr learned my addr 172.21.15.33:0/896030588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.322+0000 7f7ab589a700 1 -- 172.21.15.33:0/896030588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ab00ffdd0 msgr2=0x7f7ab0196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.322+0000 7f7ab589a700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ab00ffdd0 0x7f7ab0196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.322+0000 7f7ab589a700 1 -- 172.21.15.33:0/896030588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab0101960 msgr2=0x7f7ab019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.322+0000 7f7ab589a700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab0101960 0x7f7ab019e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.322+0000 7f7ab589a700 1 -- 172.21.15.33:0/896030588 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac0095f0 con 0x7f7ab0100fc0 2024-05-08T21:27:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.322+0000 7f7ab689c700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab0101960 0x7f7ab019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.322+0000 7f7ab589a700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ab0100fc0 0x7f7ab01972d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000e9f0 tx=0x7f7aa000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.322+0000 7f7aa77fe700 1 -- 172.21.15.33:0/896030588 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000c9c0 con 0x7f7ab0100fc0 2024-05-08T21:27:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.322+0000 7f7aa77fe700 1 -- 172.21.15.33:0/896030588 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7aa0010430 con 0x7f7ab0100fc0 2024-05-08T21:27:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.322+0000 7f7aa77fe700 1 -- 172.21.15.33:0/896030588 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0018680 con 0x7f7ab0100fc0 2024-05-08T21:27:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.322+0000 7f7ab82ff700 1 -- 172.21.15.33:0/896030588 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab019ee40 con 0x7f7ab0100fc0 2024-05-08T21:27:50.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.322+0000 7f7ab82ff700 1 -- 172.21.15.33:0/896030588 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab019f260 con 0x7f7ab0100fc0 2024-05-08T21:27:50.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.324+0000 7f7aa77fe700 1 -- 172.21.15.33:0/896030588 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7aa00105a0 con 0x7f7ab0100fc0 2024-05-08T21:27:50.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.325+0000 7f7aa77fe700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7a9c0a11a0 0x7f7a9c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.325+0000 7f7aa77fe700 1 -- 172.21.15.33:0/896030588 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7aa0014070 con 0x7f7ab0100fc0 2024-05-08T21:27:50.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.325+0000 7f7ab609b700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7a9c0a11a0 0x7f7a9c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.326+0000 7f7ac44b9b80 1 Processor -- start 2024-05-08T21:27:50.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.327+0000 7f7ac44b9b80 1 -- start start 2024-05-08T21:27:50.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.327+0000 7f7ac44b9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef77960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.327+0000 7f7ac44b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77ea0 0x561f7ef78320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.327+0000 7f7ac44b9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef78860 0x561f7ef78ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.327+0000 7f7ac44b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7eece150 con 0x561f7ef77ea0 2024-05-08T21:27:50.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.327+0000 7f7ac44b9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7eecbb10 con 0x561f7ef77540 2024-05-08T21:27:50.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.327+0000 7f7ac44b9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7eec7400 con 0x561f7ef78860 2024-05-08T21:27:50.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.330+0000 7f7ab689c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef78860 0x561f7ef78ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.330+0000 7f7ab689c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef78860 0x561f7ef78ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:39284/0 (socket says 172.21.15.33:39284) 2024-05-08T21:27:50.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.330+0000 7f7ab689c700 1 -- 172.21.15.33:0/1257974122 learned_addr learned my addr 172.21.15.33:0/1257974122 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.331+0000 7f7ab689c700 1 -- 172.21.15.33:0/1257974122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 msgr2=0x561f7ef77960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:27:50.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.331+0000 7f7ab689c700 1 --2- 172.21.15.33:0/1257974122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef77960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.331+0000 7f7ab689c700 1 -- 172.21.15.33:0/1257974122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77ea0 msgr2=0x561f7ef78320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:27:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.331+0000 7f7ab689c700 1 --2- 172.21.15.33:0/1257974122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77ea0 0x561f7ef78320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.331+0000 7f7ab689c700 1 -- 172.21.15.33:0/1257974122 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7ed483b0 con 0x561f7ef78860 2024-05-08T21:27:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.331+0000 7f7ab689c700 1 --2- 172.21.15.33:0/1257974122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef78860 0x561f7ef78ce0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8003ee0 tx=0x7f7aa8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a2469ef70bcecab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.332+0000 7f7aa4da4700 1 -- 172.21.15.33:0/1257974122 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8004900 con 0x561f7ef78860 2024-05-08T21:27:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.332+0000 7f7aa4da4700 1 -- 172.21.15.33:0/1257974122 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7aa800f040 con 0x561f7ef78860 2024-05-08T21:27:50.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.332+0000 7f7aa4da4700 1 -- 172.21.15.33:0/1257974122 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800e750 con 0x561f7ef78860 2024-05-08T21:27:50.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.335+0000 7f7ab609b700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7a9c0a11a0 0x7f7a9c0a3660 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7aac006010 tx=0x7f7aac00b6d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.338+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/1257974122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef78860 msgr2=0x561f7ef78ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.338+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/1257974122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef78860 0x561f7ef78ce0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8003ee0 tx=0x7f7aa8009920 comp rx=0 tx=0).stop 2024-05-08T21:27:50.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.340+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/1257974122 shutdown_connections 2024-05-08T21:27:50.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.340+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/1257974122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef78860 0x561f7ef78ce0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.340+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/1257974122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef77960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.340+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/1257974122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77ea0 0x561f7ef78320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.340+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/1257974122 >> 172.21.15.33:0/1257974122 conn(0x561f7eed5210 msgr2=0x561f7ef71de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.345+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/1257974122 shutdown_connections 2024-05-08T21:27:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.345+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/1257974122 wait complete. 2024-05-08T21:27:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ac44b9b80 1 -- start start 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ac44b9b80 1 Processor -- start 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ac44b9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef96250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ac44b9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef77ea0 0x561f7ef96790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ac44b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef78860 0x561f7ef9d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ac44b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7ee63560 con 0x561f7ef78860 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ac44b9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7ee63200 con 0x561f7ef77540 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ac44b9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7ed4a6a0 con 0x561f7ef77ea0 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ac44b9b80 10 client.0 _ll_register_callbacks cb 0x561f7ee6cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ab689c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef78860 0x561f7ef9d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ab689c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef78860 0x561f7ef9d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54338/0 (socket says 172.21.15.33:54338) 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ab689c700 1 -- 172.21.15.33:0/2600110540 learned_addr learned my addr 172.21.15.33:0/2600110540 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ab589a700 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef77ea0 0x561f7ef96790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.346+0000 7f7ab609b700 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef96250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.347+0000 7f7ab689c700 1 -- 172.21.15.33:0/2600110540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef77ea0 msgr2=0x561f7ef96790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.347+0000 7f7ab689c700 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef77ea0 0x561f7ef96790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.347+0000 7f7ab689c700 1 -- 172.21.15.33:0/2600110540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 msgr2=0x561f7ef96250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.347+0000 7f7ab689c700 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef96250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.347+0000 7f7ab689c700 1 -- 172.21.15.33:0/2600110540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa8003b90 con 0x561f7ef78860 2024-05-08T21:27:50.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.347+0000 7f7ab689c700 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef78860 0x561f7ef9d7d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009f50 tx=0x7f7aa8007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4507a5245db875cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.347+0000 7f7a99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:27:50.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.347+0000 7f7a99ffb700 1 -- 172.21.15.33:0/2600110540 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800e900 con 0x561f7ef78860 2024-05-08T21:27:50.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7a99ffb700 1 -- 172.21.15.33:0/2600110540 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7aa800ea60 con 0x561f7ef78860 2024-05-08T21:27:50.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2600110540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f7ee20270 con 0x561f7ef78860 2024-05-08T21:27:50.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7a99ffb700 1 -- 172.21.15.33:0/2600110540 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aa800ec20 con 0x561f7ef78860 2024-05-08T21:27:50.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7ac44b9b80 10 client.17823 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2600110540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561f7ecf4b60 con 0x561f7ef78860 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7a99ffb700 1 -- 172.21.15.33:0/2600110540 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561f7ee20270 con 0x561f7ef78860 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7a99ffb700 1 -- 172.21.15.33:0/2600110540 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7aa8015c80 con 0x561f7ef78860 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7ac44b9b80 10 client.17823 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7ac44b9b80 10 client.17823 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7ac44b9b80 20 client.17823 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x561f7efa1ff0 0x561f7efa4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7ac44b9b80 4 client.17823 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.348+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2600110540 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f7ef46810 con 0x561f7efa1ff0 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.349+0000 7f7ab609b700 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x561f7efa1ff0 0x561f7efa4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.349+0000 7f7a99ffb700 10 client.17823 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.349+0000 7f7ab609b700 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x561f7efa1ff0 0x561f7efa4410 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c39170ccf4c6db5a server_cookie=7e11b92c02e2d447 in_seq=0 out_seq=0 2024-05-08T21:27:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.350+0000 7f7a99ffb700 1 -- 172.21.15.33:0/2600110540 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561f7ef46810 con 0x561f7efa1ff0 2024-05-08T21:27:50.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.350+0000 7f7a99ffb700 10 client.17823 handle_command_reply: tid=0 2024-05-08T21:27:50.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.350+0000 7f7ac44b9b80 1 client.17823 shutdown 2024-05-08T21:27:50.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.351+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2600110540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef78860 msgr2=0x561f7ef9d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.351+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef78860 0x561f7ef9d7d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009f50 tx=0x7f7aa8007cb0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.351+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2600110540 shutdown_connections 2024-05-08T21:27:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.351+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x561f7efa1ff0 0x561f7efa4410 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.351+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef78860 0x561f7ef9d7d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.351+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef96250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.351+0000 7f7a99ffb700 0 client.17823 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:27:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.351+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/2600110540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef77ea0 0x561f7ef96790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.351+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2600110540 >> 172.21.15.33:0/2600110540 conn(0x561f7eed5210 msgr2=0x561f7ef715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.351+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2600110540 shutdown_connections 2024-05-08T21:27:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.351+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2600110540 wait complete. 2024-05-08T21:27:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.351+0000 7f7ac44b9b80 20 client.17823 trim_cache size 0 max 16384 2024-05-08T21:27:50.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.369+0000 7f7ac44b9b80 1 Processor -- start 2024-05-08T21:27:50.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ac44b9b80 1 -- start start 2024-05-08T21:27:50.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ac44b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77540 0x561f7ef9de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ac44b9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef77ea0 0x561f7ef9d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ac44b9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef78860 0x561f7ef84750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ac44b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7ef73570 con 0x561f7ef77540 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ac44b9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7efa9620 con 0x561f7ef78860 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ac44b9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7ef97440 con 0x561f7ef77ea0 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ab589a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef77ea0 0x561f7ef9d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ab609b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77540 0x561f7ef9de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ab589a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef77ea0 0x561f7ef9d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:41538/0 (socket says 172.21.15.33:41538) 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ab609b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77540 0x561f7ef9de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57354/0 (socket says 172.21.15.33:57354) 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ab589a700 1 -- 172.21.15.33:0/2115962890 learned_addr learned my addr 172.21.15.33:0/2115962890 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ab689c700 1 --2- 172.21.15.33:0/2115962890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef78860 0x561f7ef84750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ab589a700 1 -- 172.21.15.33:0/2115962890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef78860 msgr2=0x561f7ef84750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ab589a700 1 --2- 172.21.15.33:0/2115962890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef78860 0x561f7ef84750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ab589a700 1 -- 172.21.15.33:0/2115962890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77540 msgr2=0x561f7ef9de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.370+0000 7f7ab589a700 1 --2- 172.21.15.33:0/2115962890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77540 0x561f7ef9de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.371+0000 7f7ab589a700 1 -- 172.21.15.33:0/2115962890 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f7ee8d560 con 0x561f7ef77ea0 2024-05-08T21:27:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.371+0000 7f7ab589a700 1 --2- 172.21.15.33:0/2115962890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef77ea0 0x561f7ef9d790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0099d50 tx=0x7f7aa0099720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c417fe56f61963c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.371+0000 7f7aa55a5700 1 -- 172.21.15.33:0/2115962890 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa009a4c0 con 0x561f7ef77ea0 2024-05-08T21:27:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.371+0000 7f7aa55a5700 1 -- 172.21.15.33:0/2115962890 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7aa0099dc0 con 0x561f7ef77ea0 2024-05-08T21:27:50.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.371+0000 7f7aa55a5700 1 -- 172.21.15.33:0/2115962890 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0099510 con 0x561f7ef77ea0 2024-05-08T21:27:50.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.372+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2115962890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef77ea0 msgr2=0x561f7ef9d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.372+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/2115962890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef77ea0 0x561f7ef9d790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0099d50 tx=0x7f7aa0099720 comp rx=0 tx=0).stop 2024-05-08T21:27:50.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.372+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2115962890 shutdown_connections 2024-05-08T21:27:50.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.372+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/2115962890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef78860 0x561f7ef84750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.372+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/2115962890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77540 0x561f7ef9de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.372+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/2115962890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef77ea0 0x561f7ef9d790 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.372+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2115962890 >> 172.21.15.33:0/2115962890 conn(0x561f7eed5210 msgr2=0x561f7ef78c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.372+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2115962890 shutdown_connections 2024-05-08T21:27:50.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.372+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/2115962890 wait complete. 2024-05-08T21:27:50.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.372+0000 7f7ac44b9b80 1 -- start start 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ac44b9b80 1 Processor -- start 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ac44b9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef998a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ac44b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77ea0 0x561f7ef99de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ac44b9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef78860 0x561f7ef9ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ac44b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7ef9d340 con 0x561f7ef77ea0 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ac44b9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7ef9d4b0 con 0x561f7ef77540 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ac44b9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f7ef9d670 con 0x561f7ef78860 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ac44b9b80 10 client.0 _ll_register_callbacks cb 0x561f7ee6cb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ab609b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ab609b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:45956/0 (socket says 172.21.15.33:45956) 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ab609b700 1 -- 172.21.15.33:0/4116949102 learned_addr learned my addr 172.21.15.33:0/4116949102 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ab589a700 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77ea0 0x561f7ef99de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.373+0000 7f7ab689c700 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef78860 0x561f7ef9ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7ab609b700 1 -- 172.21.15.33:0/4116949102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef78860 msgr2=0x561f7ef9ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7ab609b700 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef78860 0x561f7ef9ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7ab609b700 1 -- 172.21.15.33:0/4116949102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77ea0 msgr2=0x561f7ef99de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7ab609b700 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77ea0 0x561f7ef99de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7ab609b700 1 -- 172.21.15.33:0/4116949102 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab019f260 con 0x561f7ef77540 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7ab609b700 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef998a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0098e0 tx=0x7f7aac01a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8af65b11bbd2a1a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7a99ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7a99ffb700 1 -- 172.21.15.33:0/4116949102 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac02b070 con 0x561f7ef77540 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/4116949102 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7aa800f040 con 0x561f7ef77540 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7a99ffb700 1 -- 172.21.15.33:0/4116949102 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7aac003880 con 0x561f7ef77540 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7a99ffb700 1 -- 172.21.15.33:0/4116949102 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aac0059a0 con 0x561f7ef77540 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7ac44b9b80 10 client.27728 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.374+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/4116949102 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x561f7ed4ccd0 con 0x561f7ef77540 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.375+0000 7f7a99ffb700 1 -- 172.21.15.33:0/4116949102 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7aa800f040 con 0x561f7ef77540 2024-05-08T21:27:50.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.375+0000 7f7a99ffb700 1 -- 172.21.15.33:0/4116949102 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7aac00fb10 con 0x561f7ef77540 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.375+0000 7f7ac44b9b80 10 client.27728 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.375+0000 7f7ac44b9b80 10 client.27728 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.375+0000 7f7ac44b9b80 20 client.27728 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.375+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x561f7eff9830 0x561f7effbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.375+0000 7f7ac44b9b80 4 client.27728 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.375+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/4116949102 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561f7eebfec0 con 0x561f7eff9830 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7ab589a700 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x561f7eff9830 0x561f7effbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7ab589a700 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x561f7eff9830 0x561f7effbc50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1270cfcf0ef35367 server_cookie=ae82d84666815813 in_seq=0 out_seq=0 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7a99ffb700 10 client.27728 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7a99ffb700 1 -- 172.21.15.33:0/4116949102 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561f7eebfec0 con 0x561f7eff9830 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7a99ffb700 10 client.27728 handle_command_reply: tid=0 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7ac44b9b80 1 client.27728 shutdown 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/4116949102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 msgr2=0x561f7ef998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef998a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0098e0 tx=0x7f7aac01a050 comp rx=0 tx=0).stop 2024-05-08T21:27:50.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/4116949102 shutdown_connections 2024-05-08T21:27:50.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x561f7eff9830 0x561f7effbc50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f7ef78860 0x561f7ef9ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7ef77540 0x561f7ef998a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7ac44b9b80 1 --2- 172.21.15.33:0/4116949102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f7ef77ea0 0x561f7ef99de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7a99ffb700 0 client.27728 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:27:50.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.376+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/4116949102 >> 172.21.15.33:0/4116949102 conn(0x561f7eed5210 msgr2=0x561f7ef7ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/4116949102 shutdown_connections 2024-05-08T21:27:50.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ac44b9b80 1 -- 172.21.15.33:0/4116949102 wait complete. 2024-05-08T21:27:50.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ac44b9b80 20 client.27728 trim_cache size 0 max 16384 2024-05-08T21:27:50.383 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:27:50.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ab82ff700 1 -- 172.21.15.33:0/896030588 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7a9c0a11a0 msgr2=0x7f7a9c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ab82ff700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7a9c0a11a0 0x7f7a9c0a3660 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7aac006010 tx=0x7f7aac00b6d0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ab82ff700 1 -- 172.21.15.33:0/896030588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ab0100fc0 msgr2=0x7f7ab01972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ab82ff700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ab0100fc0 0x7f7ab01972d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000e9f0 tx=0x7f7aa000ed00 comp rx=0 tx=0).stop 2024-05-08T21:27:50.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ab82ff700 1 -- 172.21.15.33:0/896030588 shutdown_connections 2024-05-08T21:27:50.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ab82ff700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ab0101960 0x7f7ab019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ab82ff700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7a9c0a11a0 0x7f7a9c0a3660 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ab82ff700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ab00ffdd0 0x7f7ab0196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ab82ff700 1 --2- 172.21.15.33:0/896030588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ab0100fc0 0x7f7ab01972d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ab82ff700 1 -- 172.21.15.33:0/896030588 >> 172.21.15.33:0/896030588 conn(0x7f7ab00fb340 msgr2=0x7f7ab0104330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ab82ff700 1 -- 172.21.15.33:0/896030588 shutdown_connections 2024-05-08T21:27:50.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.377+0000 7f7ab82ff700 1 -- 172.21.15.33:0/896030588 wait complete. 2024-05-08T21:27:50.397 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:27:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.594+0000 7f224879d700 1 -- 172.21.15.33:0/2117871290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22401019e0 msgr2=0x7f2240103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.594+0000 7f224879d700 1 --2- 172.21.15.33:0/2117871290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22401019e0 0x7f2240103dd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f223c0098f0 tx=0x7f223c009c00 comp rx=0 tx=0).stop 2024-05-08T21:27:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.596+0000 7f224879d700 1 -- 172.21.15.33:0/2117871290 shutdown_connections 2024-05-08T21:27:50.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.596+0000 7f224879d700 1 --2- 172.21.15.33:0/2117871290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2240106c40 0x7f2240109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.596+0000 7f224879d700 1 --2- 172.21.15.33:0/2117871290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22401019e0 0x7f2240103dd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.596+0000 7f224879d700 1 --2- 172.21.15.33:0/2117871290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2240104310 0x7f2240106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.596+0000 7f224879d700 1 -- 172.21.15.33:0/2117871290 >> 172.21.15.33:0/2117871290 conn(0x7f22400fb2d0 msgr2=0x7f22400fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.597+0000 7f224879d700 1 -- 172.21.15.33:0/2117871290 shutdown_connections 2024-05-08T21:27:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.598+0000 7f224879d700 1 -- 172.21.15.33:0/2117871290 wait complete. 2024-05-08T21:27:50.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.599+0000 7f224879d700 1 Processor -- start 2024-05-08T21:27:50.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.599+0000 7f224879d700 1 -- start start 2024-05-08T21:27:50.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.599+0000 7f224879d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22401019e0 0x7f2240070c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.599+0000 7f224879d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2240104310 0x7f22400711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.599+0000 7f224879d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2240106c40 0x7f2240072190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.599+0000 7f224879d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2240072760 con 0x7f2240106c40 2024-05-08T21:27:50.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.599+0000 7f224879d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22400728d0 con 0x7f2240104310 2024-05-08T21:27:50.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.599+0000 7f224879d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2240072a90 con 0x7f22401019e0 2024-05-08T21:27:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.600+0000 7f2246539700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22401019e0 0x7f2240070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.600+0000 7f2246539700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22401019e0 0x7f2240070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:41574/0 (socket says 172.21.15.33:41574) 2024-05-08T21:27:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.600+0000 7f2246539700 1 -- 172.21.15.33:0/86274549 learned_addr learned my addr 172.21.15.33:0/86274549 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.600+0000 7f2245d38700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2240104310 0x7f22400711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.600+0000 7f2246539700 1 -- 172.21.15.33:0/86274549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2240104310 msgr2=0x7f22400711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.600+0000 7f2246539700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2240104310 0x7f22400711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.600+0000 7f2246539700 1 -- 172.21.15.33:0/86274549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2240106c40 msgr2=0x7f2240072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.600+0000 7f2246539700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2240106c40 0x7f2240072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.600+0000 7f2246539700 1 -- 172.21.15.33:0/86274549 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c009520 con 0x7f22401019e0 2024-05-08T21:27:50.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.600+0000 7f2246539700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22401019e0 0x7f2240070c60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f223c009810 tx=0x7f223c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.601+0000 7f22377fe700 1 -- 172.21.15.33:0/86274549 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c01c070 con 0x7f22401019e0 2024-05-08T21:27:50.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.601+0000 7f22377fe700 1 -- 172.21.15.33:0/86274549 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f223c00f7d0 con 0x7f22401019e0 2024-05-08T21:27:50.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.601+0000 7f22377fe700 1 -- 172.21.15.33:0/86274549 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c003aa0 con 0x7f22401019e0 2024-05-08T21:27:50.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.601+0000 7f224879d700 1 -- 172.21.15.33:0/86274549 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2240072cb0 con 0x7f22401019e0 2024-05-08T21:27:50.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.601+0000 7f224879d700 1 -- 172.21.15.33:0/86274549 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22401a7b00 con 0x7f22401019e0 2024-05-08T21:27:50.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.606+0000 7f2254957b80 1 Processor -- start 2024-05-08T21:27:50.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.607+0000 7f2254957b80 1 -- start start 2024-05-08T21:27:50.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.607+0000 7f2254957b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd6d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.607+0000 7f2254957b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 0x5651bd6d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.607+0000 7f2254957b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6d3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.607+0000 7f2254957b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bd629150 con 0x5651bd6d2ea0 2024-05-08T21:27:50.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.607+0000 7f2254957b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bd626b10 con 0x5651bd6d2540 2024-05-08T21:27:50.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.607+0000 7f2254957b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bd622400 con 0x5651bd6d3860 2024-05-08T21:27:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.607+0000 7f2246d3a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd6d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.607+0000 7f2246d3a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd6d2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:45982/0 (socket says 172.21.15.33:45982) 2024-05-08T21:27:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.607+0000 7f2246d3a700 1 -- 172.21.15.33:0/419576673 learned_addr learned my addr 172.21.15.33:0/419576673 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.608+0000 7f2246d3a700 1 -- 172.21.15.33:0/419576673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 msgr2=0x5651bd6d3ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:27:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.608+0000 7f2246d3a700 1 --2- 172.21.15.33:0/419576673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6d3ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.608+0000 7f2246d3a700 1 -- 172.21.15.33:0/419576673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 msgr2=0x5651bd6d3320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:27:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.608+0000 7f2246d3a700 1 --2- 172.21.15.33:0/419576673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 0x5651bd6d3320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.608+0000 7f2246d3a700 1 -- 172.21.15.33:0/419576673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651bd4a33b0 con 0x5651bd6d2540 2024-05-08T21:27:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.608+0000 7f2246d3a700 1 --2- 172.21.15.33:0/419576673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd6d2960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f223800e400 tx=0x7f223800e710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86950d1dacce389e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.608+0000 7f2234ff9700 1 -- 172.21.15.33:0/419576673 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238018070 con 0x5651bd6d2540 2024-05-08T21:27:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.608+0000 7f2234ff9700 1 -- 172.21.15.33:0/419576673 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f223800f040 con 0x5651bd6d2540 2024-05-08T21:27:50.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.608+0000 7f2234ff9700 1 -- 172.21.15.33:0/419576673 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238004280 con 0x5651bd6d2540 2024-05-08T21:27:50.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.609+0000 7f22377fe700 1 -- 172.21.15.33:0/86274549 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f223c020030 con 0x7f22401019e0 2024-05-08T21:27:50.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.609+0000 7f2254957b80 1 -- 172.21.15.33:0/419576673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 msgr2=0x5651bd6d2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.609+0000 7f2254957b80 1 --2- 172.21.15.33:0/419576673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd6d2960 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f223800e400 tx=0x7f223800e710 comp rx=0 tx=0).stop 2024-05-08T21:27:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.609+0000 7f2254957b80 1 -- 172.21.15.33:0/419576673 shutdown_connections 2024-05-08T21:27:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.609+0000 7f2254957b80 1 --2- 172.21.15.33:0/419576673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6d3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.609+0000 7f2254957b80 1 --2- 172.21.15.33:0/419576673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd6d2960 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.609+0000 7f2254957b80 1 --2- 172.21.15.33:0/419576673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 0x5651bd6d3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.609+0000 7f2254957b80 1 -- 172.21.15.33:0/419576673 >> 172.21.15.33:0/419576673 conn(0x5651bd630210 msgr2=0x5651bd6ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.609+0000 7f22377fe700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f222c0a11a0 0x7f222c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.610+0000 7f2254957b80 1 -- 172.21.15.33:0/419576673 shutdown_connections 2024-05-08T21:27:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.610+0000 7f2245d38700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f222c0a11a0 0x7f222c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.610+0000 7f22377fe700 1 -- 172.21.15.33:0/86274549 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f223c0cdbb0 con 0x7f22401019e0 2024-05-08T21:27:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.610+0000 7f2254957b80 1 -- 172.21.15.33:0/419576673 wait complete. 2024-05-08T21:27:50.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.610+0000 7f2254957b80 1 -- start start 2024-05-08T21:27:50.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.610+0000 7f2254957b80 1 Processor -- start 2024-05-08T21:27:50.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2254957b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd6dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2254957b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 0x5651bd6dc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2254957b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6e1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2254957b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bd5be560 con 0x5651bd6d2ea0 2024-05-08T21:27:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2254957b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bd5be200 con 0x5651bd6d2540 2024-05-08T21:27:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2254957b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bd4a56a0 con 0x5651bd6d3860 2024-05-08T21:27:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2254957b80 10 client.0 _ll_register_callbacks cb 0x5651bd5c7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2246539700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd6dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2246539700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd6dbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:45992/0 (socket says 172.21.15.33:45992) 2024-05-08T21:27:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2246539700 1 -- 172.21.15.33:0/2930919850 learned_addr learned my addr 172.21.15.33:0/2930919850 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2246d3a700 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2245d38700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f222c0a11a0 0x7f222c0a3660 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2230005fd0 tx=0x7f2230009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2246539700 1 -- 172.21.15.33:0/2930919850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 msgr2=0x5651bd6e1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2246539700 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6e1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2246539700 1 -- 172.21.15.33:0/2930919850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 msgr2=0x5651bd6dc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2246539700 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 0x5651bd6dc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2246539700 1 -- 172.21.15.33:0/2930919850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223800e140 con 0x5651bd6d2540 2024-05-08T21:27:50.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2246d3a700 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:27:50.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.611+0000 7f2246539700 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd6dbdd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f223c09aca0 tx=0x7f223c09a900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=935e9f5104e2e31f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.612+0000 7f2229ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:27:50.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.612+0000 7f2229ffb700 1 -- 172.21.15.33:0/2930919850 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c09ad10 con 0x5651bd6d2540 2024-05-08T21:27:50.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.612+0000 7f2229ffb700 1 -- 172.21.15.33:0/2930919850 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f223c09eb40 con 0x5651bd6d2540 2024-05-08T21:27:50.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.612+0000 7f2254957b80 1 -- 172.21.15.33:0/2930919850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651bd57b270 con 0x5651bd6d2540 2024-05-08T21:27:50.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.613+0000 7f2229ffb700 1 -- 172.21.15.33:0/2930919850 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f223c0a3100 con 0x5651bd6d2540 2024-05-08T21:27:50.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.613+0000 7f2254957b80 10 client.27740 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:50.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.613+0000 7f2254957b80 1 -- 172.21.15.33:0/2930919850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5651bd44fb60 con 0x5651bd6d2540 2024-05-08T21:27:50.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.613+0000 7f2229ffb700 1 -- 172.21.15.33:0/2930919850 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f223c0b7080 con 0x5651bd6d2540 2024-05-08T21:27:50.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.613+0000 7f2229ffb700 1 -- 172.21.15.33:0/2930919850 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f223c09bd20 con 0x5651bd6d2540 2024-05-08T21:27:50.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.613+0000 7f2254957b80 10 client.27740 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:50.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.613+0000 7f2254957b80 10 client.27740 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:27:50.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.613+0000 7f2254957b80 20 client.27740 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:50.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.613+0000 7f2254957b80 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5651bd6f8870 0x5651bd6fac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.613+0000 7f2254957b80 4 client.27740 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:50.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.613+0000 7f2254957b80 1 -- 172.21.15.33:0/2930919850 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651bd6a1810 con 0x5651bd6f8870 2024-05-08T21:27:50.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.616+0000 7f2245d38700 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5651bd6f8870 0x5651bd6fac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.616+0000 7f2229ffb700 10 client.27740 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:27:50.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.616+0000 7f2245d38700 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5651bd6f8870 0x5651bd6fac90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6cdbc861ef3d96f server_cookie=b4793894b3016062 in_seq=0 out_seq=0 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2229ffb700 1 -- 172.21.15.33:0/2930919850 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5651bd6a1810 con 0x5651bd6f8870 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2229ffb700 10 client.27740 handle_command_reply: tid=0 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2254957b80 1 client.27740 shutdown 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2254957b80 1 -- 172.21.15.33:0/2930919850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 msgr2=0x5651bd6dbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2254957b80 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd6dbdd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f223c09aca0 tx=0x7f223c09a900 comp rx=0 tx=0).stop 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2254957b80 1 -- 172.21.15.33:0/2930919850 shutdown_connections 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2254957b80 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5651bd6f8870 0x5651bd6fac90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2254957b80 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2229ffb700 0 client.27740 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2254957b80 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd6dbdd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2254957b80 1 --2- 172.21.15.33:0/2930919850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 0x5651bd6dc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2254957b80 1 -- 172.21.15.33:0/2930919850 >> 172.21.15.33:0/2930919850 conn(0x5651bd630210 msgr2=0x5651bd6cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2254957b80 1 -- 172.21.15.33:0/2930919850 shutdown_connections 2024-05-08T21:27:50.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2254957b80 1 -- 172.21.15.33:0/2930919850 wait complete. 2024-05-08T21:27:50.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.617+0000 7f2254957b80 20 client.27740 trim_cache size 0 max 16384 2024-05-08T21:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.634+0000 7f2254957b80 1 Processor -- start 2024-05-08T21:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2254957b80 1 -- start start 2024-05-08T21:27:50.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2254957b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd708af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2254957b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 0x5651bd709030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2254957b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2254957b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bd6d7690 con 0x5651bd6d2ea0 2024-05-08T21:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2254957b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bd6c7800 con 0x5651bd6d2540 2024-05-08T21:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2254957b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bd6ffe40 con 0x5651bd6d3860 2024-05-08T21:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2245d38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 0x5651bd709030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2246539700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd708af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2246539700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd708af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:46004/0 (socket says 172.21.15.33:46004) 2024-05-08T21:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2245d38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 0x5651bd709030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57416/0 (socket says 172.21.15.33:57416) 2024-05-08T21:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2246539700 1 -- 172.21.15.33:0/162903425 learned_addr learned my addr 172.21.15.33:0/162903425 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2246d3a700 1 --2- 172.21.15.33:0/162903425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2245d38700 1 -- 172.21.15.33:0/162903425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 msgr2=0x5651bd6cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2245d38700 1 --2- 172.21.15.33:0/162903425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2245d38700 1 -- 172.21.15.33:0/162903425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 msgr2=0x5651bd708af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2245d38700 1 --2- 172.21.15.33:0/162903425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd708af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.635+0000 7f2245d38700 1 -- 172.21.15.33:0/162903425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651bd5e8560 con 0x5651bd6d2ea0 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.636+0000 7f2246539700 1 --2- 172.21.15.33:0/162903425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd708af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.636+0000 7f2246d3a700 1 --2- 172.21.15.33:0/162903425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.636+0000 7f2245d38700 1 --2- 172.21.15.33:0/162903425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 0x5651bd709030 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x5651bd6fb270 tx=0x7f2230014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8b9b25e4bffe07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.636+0000 7f22357fa700 1 -- 172.21.15.33:0/162903425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230025d00 con 0x5651bd6d2ea0 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.636+0000 7f22357fa700 1 -- 172.21.15.33:0/162903425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2230025040 con 0x5651bd6d2ea0 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.636+0000 7f22357fa700 1 -- 172.21.15.33:0/162903425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230012a90 con 0x5651bd6d2ea0 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.637+0000 7f2254957b80 1 -- 172.21.15.33:0/162903425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 msgr2=0x5651bd709030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.637+0000 7f2254957b80 1 --2- 172.21.15.33:0/162903425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 0x5651bd709030 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x5651bd6fb270 tx=0x7f2230014000 comp rx=0 tx=0).stop 2024-05-08T21:27:50.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.637+0000 7f2254957b80 1 -- 172.21.15.33:0/162903425 shutdown_connections 2024-05-08T21:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.637+0000 7f2254957b80 1 --2- 172.21.15.33:0/162903425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d3860 0x5651bd6cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.637+0000 7f2254957b80 1 --2- 172.21.15.33:0/162903425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd708af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.637+0000 7f2254957b80 1 --2- 172.21.15.33:0/162903425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d2ea0 0x5651bd709030 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.637+0000 7f2254957b80 1 -- 172.21.15.33:0/162903425 >> 172.21.15.33:0/162903425 conn(0x5651bd630210 msgr2=0x5651bd6daf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.637+0000 7f2254957b80 1 -- 172.21.15.33:0/162903425 shutdown_connections 2024-05-08T21:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.637+0000 7f2254957b80 1 -- 172.21.15.33:0/162903425 wait complete. 2024-05-08T21:27:50.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.637+0000 7f2254957b80 1 -- start start 2024-05-08T21:27:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.637+0000 7f2254957b80 1 Processor -- start 2024-05-08T21:27:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2254957b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd7085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2254957b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d2ea0 0x5651bd708b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2254957b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d3860 0x5651bd74dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2254957b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bd74e190 con 0x5651bd6d3860 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2254957b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bd74e300 con 0x5651bd6d2540 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2254957b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651bd74e4c0 con 0x5651bd6d2ea0 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2254957b80 10 client.0 _ll_register_callbacks cb 0x5651bd5c7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2246539700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd7085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2246539700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd7085e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:46018/0 (socket says 172.21.15.33:46018) 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2246539700 1 -- 172.21.15.33:0/2385704542 learned_addr learned my addr 172.21.15.33:0/2385704542 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2246d3a700 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d3860 0x5651bd74dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2245d38700 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d2ea0 0x5651bd708b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2246539700 1 -- 172.21.15.33:0/2385704542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d2ea0 msgr2=0x5651bd708b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2246539700 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d2ea0 0x5651bd708b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2246539700 1 -- 172.21.15.33:0/2385704542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d3860 msgr2=0x5651bd74dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2246539700 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d3860 0x5651bd74dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.638+0000 7f2246539700 1 -- 172.21.15.33:0/2385704542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651bd44fb60 con 0x5651bd6d2540 2024-05-08T21:27:50.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2246539700 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd7085e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f223c09aca0 tx=0x7f223c097180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c510031728390446 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2229ffb700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:27:50.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2229ffb700 1 -- 172.21.15.33:0/2385704542 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c09eb40 con 0x5651bd6d2540 2024-05-08T21:27:50.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2229ffb700 1 -- 172.21.15.33:0/2385704542 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f223c079e40 con 0x5651bd6d2540 2024-05-08T21:27:50.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2254957b80 1 -- 172.21.15.33:0/2385704542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f223800f040 con 0x5651bd6d2540 2024-05-08T21:27:50.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2229ffb700 1 -- 172.21.15.33:0/2385704542 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f223c09ad10 con 0x5651bd6d2540 2024-05-08T21:27:50.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2254957b80 10 client.27746 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:50.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2254957b80 1 -- 172.21.15.33:0/2385704542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5651bd4a33b0 con 0x5651bd6d2540 2024-05-08T21:27:50.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2229ffb700 1 -- 172.21.15.33:0/2385704542 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f223800f040 con 0x5651bd6d2540 2024-05-08T21:27:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2229ffb700 1 -- 172.21.15.33:0/2385704542 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f223c02a060 con 0x5651bd6d2540 2024-05-08T21:27:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2254957b80 10 client.27746 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2254957b80 10 client.27746 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:27:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.639+0000 7f2254957b80 20 client.27746 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.640+0000 7f2254957b80 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5651bd74f9c0 0x5651bd751db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.640+0000 7f2254957b80 4 client.27746 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:27:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.640+0000 7f2254957b80 1 -- 172.21.15.33:0/2385704542 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5651bd61aec0 con 0x5651bd74f9c0 2024-05-08T21:27:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.640+0000 7f2245d38700 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5651bd74f9c0 0x5651bd751db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.640+0000 7f2229ffb700 10 client.27746 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:27:50.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.640+0000 7f2245d38700 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5651bd74f9c0 0x5651bd751db0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=704b747f126e6524 server_cookie=4a84a925ece8aa05 in_seq=0 out_seq=0 2024-05-08T21:27:50.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.640+0000 7f2229ffb700 1 -- 172.21.15.33:0/2385704542 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5651bd61aec0 con 0x5651bd74f9c0 2024-05-08T21:27:50.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.640+0000 7f2229ffb700 10 client.27746 handle_command_reply: tid=0 2024-05-08T21:27:50.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.640+0000 7f2254957b80 1 client.27746 shutdown 2024-05-08T21:27:50.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f2254957b80 1 -- 172.21.15.33:0/2385704542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 msgr2=0x5651bd7085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f2254957b80 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd7085e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f223c09aca0 tx=0x7f223c097180 comp rx=0 tx=0).stop 2024-05-08T21:27:50.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f2254957b80 1 -- 172.21.15.33:0/2385704542 shutdown_connections 2024-05-08T21:27:50.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f2254957b80 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x5651bd74f9c0 0x5651bd751db0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f2254957b80 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5651bd6d3860 0x5651bd74dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f2254957b80 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5651bd6d2540 0x5651bd7085e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f2254957b80 1 --2- 172.21.15.33:0/2385704542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651bd6d2ea0 0x5651bd708b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f2229ffb700 0 client.27746 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:27:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f2254957b80 1 -- 172.21.15.33:0/2385704542 >> 172.21.15.33:0/2385704542 conn(0x5651bd630210 msgr2=0x5651bd6dbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f2254957b80 1 -- 172.21.15.33:0/2385704542 shutdown_connections 2024-05-08T21:27:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f2254957b80 1 -- 172.21.15.33:0/2385704542 wait complete. 2024-05-08T21:27:50.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f2254957b80 20 client.27746 trim_cache size 0 max 16384 2024-05-08T21:27:50.651 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:27:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f224879d700 1 -- 172.21.15.33:0/86274549 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f222c0a11a0 msgr2=0x7f222c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f224879d700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f222c0a11a0 0x7f222c0a3660 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2230005fd0 tx=0x7f2230009310 comp rx=0 tx=0).stop 2024-05-08T21:27:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f224879d700 1 -- 172.21.15.33:0/86274549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22401019e0 msgr2=0x7f2240070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.641+0000 7f224879d700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22401019e0 0x7f2240070c60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f223c009810 tx=0x7f223c00b840 comp rx=0 tx=0).stop 2024-05-08T21:27:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.642+0000 7f224879d700 1 -- 172.21.15.33:0/86274549 shutdown_connections 2024-05-08T21:27:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.642+0000 7f224879d700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2240106c40 0x7f2240072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.642+0000 7f224879d700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22401019e0 0x7f2240070c60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.642+0000 7f224879d700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f222c0a11a0 0x7f222c0a3660 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.642+0000 7f224879d700 1 --2- 172.21.15.33:0/86274549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2240104310 0x7f22400711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.642+0000 7f224879d700 1 -- 172.21.15.33:0/86274549 >> 172.21.15.33:0/86274549 conn(0x7f22400fb2d0 msgr2=0x7f2240103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.642+0000 7f224879d700 1 -- 172.21.15.33:0/86274549 shutdown_connections 2024-05-08T21:27:50.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.642+0000 7f224879d700 1 -- 172.21.15.33:0/86274549 wait complete. 2024-05-08T21:27:50.667 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:50 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4239385084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:27:50.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:27:50.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:50 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/4239385084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:27:50.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:50 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/4239385084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:27:50.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.861+0000 7fa3b1935700 1 -- 172.21.15.33:0/3341287063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ac106c40 msgr2=0x7fa3ac109030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.861+0000 7fa3b1935700 1 --2- 172.21.15.33:0/3341287063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ac106c40 0x7fa3ac109030 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400b0a0 tx=0x7fa3a400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.861+0000 7fa3b1935700 1 -- 172.21.15.33:0/3341287063 shutdown_connections 2024-05-08T21:27:50.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.861+0000 7fa3b1935700 1 --2- 172.21.15.33:0/3341287063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ac106c40 0x7fa3ac109030 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.861+0000 7fa3b1935700 1 --2- 172.21.15.33:0/3341287063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3ac1019e0 0x7fa3ac103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.861+0000 7fa3b1935700 1 --2- 172.21.15.33:0/3341287063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3ac104310 0x7fa3ac106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.861+0000 7fa3b1935700 1 -- 172.21.15.33:0/3341287063 >> 172.21.15.33:0/3341287063 conn(0x7fa3ac0fb330 msgr2=0x7fa3ac0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.861+0000 7fa3b1935700 1 -- 172.21.15.33:0/3341287063 shutdown_connections 2024-05-08T21:27:50.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.861+0000 7fa3b1935700 1 -- 172.21.15.33:0/3341287063 wait complete. 2024-05-08T21:27:50.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.862+0000 7fa3b1935700 1 Processor -- start 2024-05-08T21:27:50.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.862+0000 7fa3b1935700 1 -- start start 2024-05-08T21:27:50.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.863+0000 7fa3b1935700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ac1019e0 0x7fa3ac101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.863+0000 7fa3b1935700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3ac104310 0x7fa3ac0ff990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.863+0000 7fa3b1935700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3ac106c40 0x7fa3ac0ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.863+0000 7fa3b1935700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac19ac70 con 0x7fa3ac1019e0 2024-05-08T21:27:50.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.863+0000 7fa3b1935700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac100410 con 0x7fa3ac106c40 2024-05-08T21:27:50.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.863+0000 7fa3b1935700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac1005d0 con 0x7fa3ac104310 2024-05-08T21:27:50.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.865+0000 7fa3ab7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3ac106c40 0x7fa3ac0ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.865+0000 7fa3ab7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3ac106c40 0x7fa3ac0ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:46046/0 (socket says 172.21.15.33:46046) 2024-05-08T21:27:50.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.865+0000 7fa3ab7fe700 1 -- 172.21.15.33:0/1066846883 learned_addr learned my addr 172.21.15.33:0/1066846883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.868+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ac1019e0 0x7fa3ac101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.869+0000 7fa3aaffd700 1 -- 172.21.15.33:0/1066846883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3ac104310 msgr2=0x7fa3ac0ff990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T21:27:50.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.869+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3ac104310 0x7fa3ac0ff990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.869+0000 7fa3aaffd700 1 -- 172.21.15.33:0/1066846883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3ac106c40 msgr2=0x7fa3ac0ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.869+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3ac106c40 0x7fa3ac0ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.869+0000 7fa3aaffd700 1 -- 172.21.15.33:0/1066846883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a4009d00 con 0x7fa3ac1019e0 2024-05-08T21:27:50.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.869+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ac1019e0 0x7fa3ac101300 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00e9f0 tx=0x7fa39c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.870+0000 7fa3b0933700 1 -- 172.21.15.33:0/1066846883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00eee0 con 0x7fa3ac1019e0 2024-05-08T21:27:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.870+0000 7fa3b0933700 1 -- 172.21.15.33:0/1066846883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa39c00cbd0 con 0x7fa3ac1019e0 2024-05-08T21:27:50.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.870+0000 7fa3b1935700 1 -- 172.21.15.33:0/1066846883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ac1a5620 con 0x7fa3ac1019e0 2024-05-08T21:27:50.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.870+0000 7fa3b1935700 1 -- 172.21.15.33:0/1066846883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ac1a59e0 con 0x7fa3ac1019e0 2024-05-08T21:27:50.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.875+0000 7fa3bdaefb80 1 Processor -- start 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.877+0000 7fa3bdaefb80 1 -- start start 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.877+0000 7fa3bdaefb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adc550 0x564d82adc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.877+0000 7fa3bdaefb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adceb0 0x564d82add330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.877+0000 7fa3bdaefb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82addcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.877+0000 7fa3bdaefb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d82a22150 con 0x564d82adc550 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.877+0000 7fa3bdaefb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d82a1fb10 con 0x564d82add870 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.877+0000 7fa3bdaefb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d82a1b400 con 0x564d82adceb0 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.878+0000 7fa3ab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adc550 0x564d82adc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.878+0000 7fa3aa7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82addcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.878+0000 7fa3ab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adc550 0x564d82adc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57458/0 (socket says 172.21.15.33:57458) 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.878+0000 7fa3ab7fe700 1 -- 172.21.15.33:0/3646235495 learned_addr learned my addr 172.21.15.33:0/3646235495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.878+0000 7fa3aa7fc700 1 -- 172.21.15.33:0/3646235495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adceb0 msgr2=0x564d82add330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.878+0000 7fa3aa7fc700 1 --2- 172.21.15.33:0/3646235495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adceb0 0x564d82add330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.878+0000 7fa3aa7fc700 1 -- 172.21.15.33:0/3646235495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adc550 msgr2=0x564d82adc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.878+0000 7fa3aa7fc700 1 --2- 172.21.15.33:0/3646235495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adc550 0x564d82adc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.878+0000 7fa3aa7fc700 1 -- 172.21.15.33:0/3646235495 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d8289c3b0 con 0x564d82add870 2024-05-08T21:27:50.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.879+0000 7fa3ab7fe700 1 --2- 172.21.15.33:0/3646235495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adc550 0x564d82adc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:50.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.879+0000 7fa3aa7fc700 1 --2- 172.21.15.33:0/3646235495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82addcf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009b90 tx=0x7fa3a0009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83b18edfedd1e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.880+0000 7fa391ffb700 1 -- 172.21.15.33:0/3646235495 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0004230 con 0x564d82add870 2024-05-08T21:27:50.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.880+0000 7fa391ffb700 1 -- 172.21.15.33:0/3646235495 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3a000ed90 con 0x564d82add870 2024-05-08T21:27:50.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.880+0000 7fa391ffb700 1 -- 172.21.15.33:0/3646235495 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0004e70 con 0x564d82add870 2024-05-08T21:27:50.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.881+0000 7fa3b0933700 1 -- 172.21.15.33:0/1066846883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c018680 con 0x7fa3ac1019e0 2024-05-08T21:27:50.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.881+0000 7fa3b0933700 1 -- 172.21.15.33:0/1066846883 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa39c0187e0 con 0x7fa3ac1019e0 2024-05-08T21:27:50.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.881+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3646235495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 msgr2=0x564d82addcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.881+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/3646235495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82addcf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009b90 tx=0x7fa3a0009ea0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.882+0000 7fa3b0933700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa3940a1370 0x7fa3940a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.882+0000 7fa3b0933700 1 -- 172.21.15.33:0/1066846883 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa39c014070 con 0x7fa3ac1019e0 2024-05-08T21:27:50.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.882+0000 7fa3ab7fe700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa3940a1370 0x7fa3940a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.883+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3646235495 shutdown_connections 2024-05-08T21:27:50.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.883+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/3646235495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82addcf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.883+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/3646235495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adc550 0x564d82adc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.883+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/3646235495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adceb0 0x564d82add330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.883+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3646235495 >> 172.21.15.33:0/3646235495 conn(0x564d82a29210 msgr2=0x564d82ac5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.883+0000 7fa3ab7fe700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa3940a1370 0x7fa3940a3830 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4007b90 tx=0x7fa3a400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.883+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3646235495 shutdown_connections 2024-05-08T21:27:50.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.883+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3646235495 wait complete. 2024-05-08T21:27:50.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.884+0000 7fa3bdaefb80 1 -- start start 2024-05-08T21:27:50.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.884+0000 7fa3bdaefb80 1 Processor -- start 2024-05-08T21:27:50.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.884+0000 7fa3bdaefb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adc550 0x564d82ae61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.884+0000 7fa3bdaefb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adceb0 0x564d82ae66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.884+0000 7fa3bdaefb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82aed730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.884+0000 7fa3bdaefb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d829b7560 con 0x564d82adceb0 2024-05-08T21:27:50.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.884+0000 7fa3bdaefb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d829b7200 con 0x564d82add870 2024-05-08T21:27:50.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.884+0000 7fa3bdaefb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d8289e6a0 con 0x564d82adc550 2024-05-08T21:27:50.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.884+0000 7fa3bdaefb80 10 client.0 _ll_register_callbacks cb 0x564d829c0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:50.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.884+0000 7fa3aaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adc550 0x564d82ae61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.885+0000 7fa3aaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adc550 0x564d82ae61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:41630/0 (socket says 172.21.15.33:41630) 2024-05-08T21:27:50.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.885+0000 7fa3aaffd700 1 -- 172.21.15.33:0/3049355822 learned_addr learned my addr 172.21.15.33:0/3049355822 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.885+0000 7fa3ab7fe700 1 --2- 172.21.15.33:0/3049355822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82aed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.885+0000 7fa3aaffd700 1 -- 172.21.15.33:0/3049355822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 msgr2=0x564d82aed730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.885+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/3049355822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82aed730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.885+0000 7fa3aaffd700 1 -- 172.21.15.33:0/3049355822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adceb0 msgr2=0x564d82ae66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.885+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/3049355822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adceb0 0x564d82ae66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.885+0000 7fa3aaffd700 1 -- 172.21.15.33:0/3049355822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ac109c00 con 0x564d82adc550 2024-05-08T21:27:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.885+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/3049355822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adc550 0x564d82ae61b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0979a0 tx=0x7fa39c09b470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d1f13bb3215bac2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.885+0000 7fa38f7f6700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.885+0000 7fa38f7f6700 1 -- 172.21.15.33:0/3049355822 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c09b630 con 0x564d82adc550 2024-05-08T21:27:50.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.885+0000 7fa38f7f6700 1 -- 172.21.15.33:0/3049355822 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa39c09af30 con 0x564d82adc550 2024-05-08T21:27:50.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.886+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3049355822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d82974270 con 0x564d82adc550 2024-05-08T21:27:50.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.886+0000 7fa38f7f6700 1 -- 172.21.15.33:0/3049355822 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa39c09ed60 con 0x564d82adc550 2024-05-08T21:27:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.886+0000 7fa3bdaefb80 10 client.27775 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.886+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3049355822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fa3a0009810 con 0x564d82adc550 2024-05-08T21:27:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.886+0000 7fa38f7f6700 1 -- 172.21.15.33:0/3049355822 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564d82974270 con 0x564d82adc550 2024-05-08T21:27:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.890+0000 7fa38f7f6700 1 -- 172.21.15.33:0/3049355822 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa39c0a2f20 con 0x564d82adc550 2024-05-08T21:27:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.890+0000 7fa3bdaefb80 10 client.27775 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.890+0000 7fa3bdaefb80 10 client.27775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.890+0000 7fa3bdaefb80 20 client.27775 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.890+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/3049355822 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564d82af1f10 0x564d82af4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.890+0000 7fa3bdaefb80 4 client.27775 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:50.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.890+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3049355822 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d82a9a810 con 0x564d82af1f10 2024-05-08T21:27:50.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.890+0000 7fa3aa7fc700 1 --2- 172.21.15.33:0/3049355822 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564d82af1f10 0x564d82af4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.891+0000 7fa3aa7fc700 1 --2- 172.21.15.33:0/3049355822 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564d82af1f10 0x564d82af4350 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b3b7506ec141f43 server_cookie=649b589c13412089 in_seq=0 out_seq=0 2024-05-08T21:27:50.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.891+0000 7fa38f7f6700 10 client.27775 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:50.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.891+0000 7fa38f7f6700 1 -- 172.21.15.33:0/3049355822 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564d82a9a810 con 0x564d82af1f10 2024-05-08T21:27:50.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.891+0000 7fa38f7f6700 10 client.27775 handle_command_reply: tid=0 2024-05-08T21:27:50.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.891+0000 7fa3bdaefb80 1 client.27775 shutdown 2024-05-08T21:27:50.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.894+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3049355822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adc550 msgr2=0x564d82ae61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.894+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/3049355822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adc550 0x564d82ae61b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0979a0 tx=0x7fa39c09b470 comp rx=0 tx=0).stop 2024-05-08T21:27:50.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.894+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3049355822 shutdown_connections 2024-05-08T21:27:50.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.894+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/3049355822 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564d82af1f10 0x564d82af4350 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.894+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/3049355822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82aed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.894+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/3049355822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adc550 0x564d82ae61b0 secure :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0979a0 tx=0x7fa39c09b470 comp rx=0 tx=0).stop 2024-05-08T21:27:50.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.894+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/3049355822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adceb0 0x564d82ae66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.894+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3049355822 >> 172.21.15.33:0/3049355822 conn(0x564d82a29210 msgr2=0x564d82ac55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.894+0000 7fa38f7f6700 0 client.27775 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.894+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3049355822 shutdown_connections 2024-05-08T21:27:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.894+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/3049355822 wait complete. 2024-05-08T21:27:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.894+0000 7fa3bdaefb80 20 client.27775 trim_cache size 0 max 16384 2024-05-08T21:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.910+0000 7fa3bdaefb80 1 Processor -- start 2024-05-08T21:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.910+0000 7fa3bdaefb80 1 -- start start 2024-05-08T21:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3bdaefb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adceb0 0x564d82adc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3bdaefb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82aed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3bdaefb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82addc50 0x564d82ae56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3bdaefb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d82ac0800 con 0x564d82adceb0 2024-05-08T21:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3bdaefb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d82ac7830 con 0x564d82add870 2024-05-08T21:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3bdaefb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d82ae5c30 con 0x564d82addc50 2024-05-08T21:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3aaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adceb0 0x564d82adc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3aaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adceb0 0x564d82adc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57480/0 (socket says 172.21.15.33:57480) 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3aaffd700 1 -- 172.21.15.33:0/172600000 learned_addr learned my addr 172.21.15.33:0/172600000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3ab7fe700 1 --2- 172.21.15.33:0/172600000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82addc50 0x564d82ae56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3aa7fc700 1 --2- 172.21.15.33:0/172600000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82aed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3aaffd700 1 -- 172.21.15.33:0/172600000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82addc50 msgr2=0x564d82ae56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/172600000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82addc50 0x564d82ae56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3aaffd700 1 -- 172.21.15.33:0/172600000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 msgr2=0x564d82aed6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/172600000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82aed6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3aaffd700 1 -- 172.21.15.33:0/172600000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d82848b60 con 0x564d82adceb0 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.911+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/172600000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adceb0 0x564d82adc930 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00e9c0 tx=0x7fa39c09a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f31b62494c1ad3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.912+0000 7fa3927fc700 1 -- 172.21.15.33:0/172600000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c09b630 con 0x564d82adceb0 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.912+0000 7fa3927fc700 1 -- 172.21.15.33:0/172600000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa39c09af30 con 0x564d82adceb0 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.912+0000 7fa3927fc700 1 -- 172.21.15.33:0/172600000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0cb660 con 0x564d82adceb0 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.912+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/172600000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adceb0 msgr2=0x564d82adc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.912+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/172600000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adceb0 0x564d82adc930 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00e9c0 tx=0x7fa39c09a700 comp rx=0 tx=0).stop 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.912+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/172600000 shutdown_connections 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.912+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/172600000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82addc50 0x564d82ae56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.912+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/172600000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82adceb0 0x564d82adc930 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.912+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/172600000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82add870 0x564d82aed6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.912+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/172600000 >> 172.21.15.33:0/172600000 conn(0x564d82a29210 msgr2=0x564d82ac2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.912+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/172600000 shutdown_connections 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.912+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/172600000 wait complete. 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3bdaefb80 1 -- start start 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3bdaefb80 1 Processor -- start 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3bdaefb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adceb0 0x564d82ae91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3bdaefb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82add870 0x564d82ae9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3bdaefb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82addc50 0x564d82aec720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3bdaefb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d82aecc60 con 0x564d82add870 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3bdaefb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d82aecdd0 con 0x564d82addc50 2024-05-08T21:27:50.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3bdaefb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d82aecf90 con 0x564d82adceb0 2024-05-08T21:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3bdaefb80 10 client.0 _ll_register_callbacks cb 0x564d829c0b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3aa7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82add870 0x564d82ae9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3aa7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82add870 0x564d82ae9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57482/0 (socket says 172.21.15.33:57482) 2024-05-08T21:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3aa7fc700 1 -- 172.21.15.33:0/2313335537 learned_addr learned my addr 172.21.15.33:0/2313335537 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.913+0000 7fa3aaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adceb0 0x564d82ae91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa3ab7fe700 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82addc50 0x564d82aec720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa3aaffd700 1 -- 172.21.15.33:0/2313335537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82addc50 msgr2=0x564d82aec720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82addc50 0x564d82aec720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa3aaffd700 1 -- 172.21.15.33:0/2313335537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82add870 msgr2=0x564d82ae9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:50.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82add870 0x564d82ae9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa3aaffd700 1 -- 172.21.15.33:0/2313335537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a0009810 con 0x564d82adceb0 2024-05-08T21:27:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa3ab7fe700 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82addc50 0x564d82aec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T21:27:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa3aa7fc700 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82add870 0x564d82ae9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa3aaffd700 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adceb0 0x564d82ae91c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa39c099720 tx=0x7fa39c0cb870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff0efdcc6638da60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa38f7f6700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa38f7f6700 1 -- 172.21.15.33:0/2313335537 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00ce40 con 0x564d82adceb0 2024-05-08T21:27:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/2313335537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3a000ed90 con 0x564d82adceb0 2024-05-08T21:27:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa38f7f6700 1 -- 172.21.15.33:0/2313335537 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa39c09b630 con 0x564d82adceb0 2024-05-08T21:27:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa38f7f6700 1 -- 172.21.15.33:0/2313335537 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa39c09ed60 con 0x564d82adceb0 2024-05-08T21:27:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa3bdaefb80 10 client.27781 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.914+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/2313335537 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564d829e1560 con 0x564d82adceb0 2024-05-08T21:27:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.915+0000 7fa38f7f6700 1 -- 172.21.15.33:0/2313335537 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3a000ed90 con 0x564d82adceb0 2024-05-08T21:27:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.915+0000 7fa38f7f6700 1 -- 172.21.15.33:0/2313335537 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa39c07f070 con 0x564d82adceb0 2024-05-08T21:27:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.915+0000 7fa3bdaefb80 10 client.27781 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.915+0000 7fa3bdaefb80 10 client.27781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.915+0000 7fa3bdaefb80 20 client.27781 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.915+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564d82b48cd0 0x564d82b4b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.915+0000 7fa3bdaefb80 4 client.27781 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:27:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.915+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/2313335537 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564d82a13ec0 con 0x564d82b48cd0 2024-05-08T21:27:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.915+0000 7fa3aa7fc700 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564d82b48cd0 0x564d82b4b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.915+0000 7fa38f7f6700 10 client.27781 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.915+0000 7fa3aa7fc700 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564d82b48cd0 0x564d82b4b0c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87d14614595cd4a3 server_cookie=fc8afcbe1f865fc0 in_seq=0 out_seq=0 2024-05-08T21:27:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa38f7f6700 1 -- 172.21.15.33:0/2313335537 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564d82a13ec0 con 0x564d82b48cd0 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa38f7f6700 10 client.27781 handle_command_reply: tid=0 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3bdaefb80 1 client.27781 shutdown 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/2313335537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adceb0 msgr2=0x564d82ae91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adceb0 0x564d82ae91c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa39c099720 tx=0x7fa39c0cb870 comp rx=0 tx=0).stop 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/2313335537 shutdown_connections 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564d82b48cd0 0x564d82b4b0c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564d82addc50 0x564d82aec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa38f7f6700 0 client.27781 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564d82adceb0 0x564d82ae91c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3bdaefb80 1 --2- 172.21.15.33:0/2313335537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d82add870 0x564d82ae9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/2313335537 >> 172.21.15.33:0/2313335537 conn(0x564d82a29210 msgr2=0x564d82ae5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/2313335537 shutdown_connections 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3bdaefb80 1 -- 172.21.15.33:0/2313335537 wait complete. 2024-05-08T21:27:50.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3bdaefb80 20 client.27781 trim_cache size 0 max 16384 2024-05-08T21:27:50.927 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:27:50.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3b1935700 1 -- 172.21.15.33:0/1066846883 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa3940a1370 msgr2=0x7fa3940a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3b1935700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa3940a1370 0x7fa3940a3830 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4007b90 tx=0x7fa3a400b700 comp rx=0 tx=0).stop 2024-05-08T21:27:50.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3b1935700 1 -- 172.21.15.33:0/1066846883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ac1019e0 msgr2=0x7fa3ac101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:50.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3b1935700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ac1019e0 0x7fa3ac101300 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00e9f0 tx=0x7fa39c00ed00 comp rx=0 tx=0).stop 2024-05-08T21:27:50.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.916+0000 7fa3b1935700 1 -- 172.21.15.33:0/1066846883 shutdown_connections 2024-05-08T21:27:50.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.917+0000 7fa3b1935700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa3940a1370 0x7fa3940a3830 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.917+0000 7fa3b1935700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3ac106c40 0x7fa3ac0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.917+0000 7fa3b1935700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ac1019e0 0x7fa3ac101300 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.917+0000 7fa3b1935700 1 --2- 172.21.15.33:0/1066846883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3ac104310 0x7fa3ac0ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:50.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.917+0000 7fa3b1935700 1 -- 172.21.15.33:0/1066846883 >> 172.21.15.33:0/1066846883 conn(0x7fa3ac0fb330 msgr2=0x7fa3ac103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:50.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.917+0000 7fa3b1935700 1 -- 172.21.15.33:0/1066846883 shutdown_connections 2024-05-08T21:27:50.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:50.917+0000 7fa3b1935700 1 -- 172.21.15.33:0/1066846883 wait complete. 2024-05-08T21:27:51.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:51 smithi033 ceph-mon[42528]: pgmap v556: 129 pgs: 129 active+clean; 273 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 638 op/s 2024-05-08T21:27:51.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:51 smithi071 ceph-mon[47074]: pgmap v556: 129 pgs: 129 active+clean; 273 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 638 op/s 2024-05-08T21:27:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:51 smithi082 ceph-mon[47512]: pgmap v556: 129 pgs: 129 active+clean; 273 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 638 op/s 2024-05-08T21:27:51.935 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-08T21:27:51.935 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-05-08T21:27:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.127+0000 7fbf2155a700 1 -- 172.21.15.33:0/1619677412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c1018f0 msgr2=0x7fbf1c1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.127+0000 7fbf2155a700 1 --2- 172.21.15.33:0/1619677412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c1018f0 0x7fbf1c1081a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400b0d0 tx=0x7fbf1400b3e0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.127+0000 7fbf2155a700 1 -- 172.21.15.33:0/1619677412 shutdown_connections 2024-05-08T21:27:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.127+0000 7fbf2155a700 1 --2- 172.21.15.33:0/1619677412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c1018f0 0x7fbf1c1081a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.127+0000 7fbf2155a700 1 --2- 172.21.15.33:0/1619677412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf1c0ffd60 0x7fbf1c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.127+0000 7fbf2155a700 1 --2- 172.21.15.33:0/1619677412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf1c100f50 0x7fbf1c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.127+0000 7fbf2155a700 1 -- 172.21.15.33:0/1619677412 >> 172.21.15.33:0/1619677412 conn(0x7fbf1c0fb290 msgr2=0x7fbf1c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:52.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.127+0000 7fbf2155a700 1 -- 172.21.15.33:0/1619677412 shutdown_connections 2024-05-08T21:27:52.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.127+0000 7fbf2155a700 1 -- 172.21.15.33:0/1619677412 wait complete. 2024-05-08T21:27:52.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.128+0000 7fbf2155a700 1 Processor -- start 2024-05-08T21:27:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf2155a700 1 -- start start 2024-05-08T21:27:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf2155a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c0ffd60 0x7fbf1c196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf2155a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf1c100f50 0x7fbf1c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf2155a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf1c1018f0 0x7fbf1c19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf2155a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c19e970 con 0x7fbf1c0ffd60 2024-05-08T21:27:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf2155a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c19eab0 con 0x7fbf1c100f50 2024-05-08T21:27:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf2155a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c19ec40 con 0x7fbf1c1018f0 2024-05-08T21:27:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf1affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c0ffd60 0x7fbf1c196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf1affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c0ffd60 0x7fbf1c196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57508/0 (socket says 172.21.15.33:57508) 2024-05-08T21:27:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf1affd700 1 -- 172.21.15.33:0/456653052 learned_addr learned my addr 172.21.15.33:0/456653052 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf1a7fc700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf1c100f50 0x7fbf1c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf1b7fe700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf1c1018f0 0x7fbf1c19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf1affd700 1 -- 172.21.15.33:0/456653052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf1c1018f0 msgr2=0x7fbf1c19e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf1affd700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf1c1018f0 0x7fbf1c19e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf1affd700 1 -- 172.21.15.33:0/456653052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf1c100f50 msgr2=0x7fbf1c197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf1affd700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf1c100f50 0x7fbf1c197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.129+0000 7fbf1affd700 1 -- 172.21.15.33:0/456653052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009520 con 0x7fbf1c0ffd60 2024-05-08T21:27:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.130+0000 7fbf1affd700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c0ffd60 0x7fbf1c196e40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00e970 tx=0x7fbf0c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.130+0000 7fbf03fff700 1 -- 172.21.15.33:0/456653052 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c009790 con 0x7fbf1c0ffd60 2024-05-08T21:27:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.130+0000 7fbf03fff700 1 -- 172.21.15.33:0/456653052 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf0c00eec0 con 0x7fbf1c0ffd60 2024-05-08T21:27:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.130+0000 7fbf2155a700 1 -- 172.21.15.33:0/456653052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf14009d00 con 0x7fbf1c0ffd60 2024-05-08T21:27:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.130+0000 7fbf2155a700 1 -- 172.21.15.33:0/456653052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c102400 con 0x7fbf1c0ffd60 2024-05-08T21:27:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.130+0000 7fbf03fff700 1 -- 172.21.15.33:0/456653052 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c003c10 con 0x7fbf1c0ffd60 2024-05-08T21:27:52.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.133+0000 7fbf03fff700 1 -- 172.21.15.33:0/456653052 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fbf0c018430 con 0x7fbf1c0ffd60 2024-05-08T21:27:52.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.134+0000 7fbf03fff700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbf040a1120 0x7fbf040a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.134+0000 7fbf03fff700 1 -- 172.21.15.33:0/456653052 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbf0c014070 con 0x7fbf1c0ffd60 2024-05-08T21:27:52.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.134+0000 7fbf1a7fc700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbf040a1120 0x7fbf040a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.134+0000 7fbf2d714b80 1 Processor -- start 2024-05-08T21:27:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf2d714b80 1 -- start start 2024-05-08T21:27:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf2d714b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08feb5c0 0x55cc08feb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf2d714b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08febf20 0x55cc08fec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf2d714b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08fec8e0 0x55cc08fecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf2d714b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08f421d0 con 0x55cc08febf20 2024-05-08T21:27:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf2d714b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08f3fb90 con 0x55cc08fec8e0 2024-05-08T21:27:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf2d714b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08f3b480 con 0x55cc08feb5c0 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf1b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08fec8e0 0x55cc08fecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf1b7fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08fec8e0 0x55cc08fecd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:46106/0 (socket says 172.21.15.33:46106) 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf1b7fe700 1 -- 172.21.15.33:0/76672414 learned_addr learned my addr 172.21.15.33:0/76672414 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf1a7fc700 1 --2- 172.21.15.33:0/76672414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08febf20 0x55cc08fec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf1affd700 1 --2- 172.21.15.33:0/76672414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08feb5c0 0x55cc08feb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf1b7fe700 1 -- 172.21.15.33:0/76672414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08feb5c0 msgr2=0x55cc08feb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf1b7fe700 1 --2- 172.21.15.33:0/76672414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08feb5c0 0x55cc08feb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf1b7fe700 1 -- 172.21.15.33:0/76672414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08febf20 msgr2=0x55cc08fec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf1b7fe700 1 --2- 172.21.15.33:0/76672414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08febf20 0x55cc08fec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.135+0000 7fbf1b7fe700 1 -- 172.21.15.33:0/76672414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc08dbc430 con 0x55cc08fec8e0 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.136+0000 7fbf1a7fc700 1 --2- 172.21.15.33:0/76672414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08febf20 0x55cc08fec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.136+0000 7fbf1a7fc700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbf040a1120 0x7fbf040a35e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009df0 tx=0x7fbf10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.136+0000 7fbf1b7fe700 1 --2- 172.21.15.33:0/76672414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08fec8e0 0x55cc08fecd60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009c30 tx=0x7fbf14009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=601fa28a795344bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.136+0000 7fbf017fa700 1 -- 172.21.15.33:0/76672414 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400e030 con 0x55cc08fec8e0 2024-05-08T21:27:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.136+0000 7fbf017fa700 1 -- 172.21.15.33:0/76672414 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf14009460 con 0x55cc08fec8e0 2024-05-08T21:27:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.136+0000 7fbf017fa700 1 -- 172.21.15.33:0/76672414 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14004c90 con 0x55cc08fec8e0 2024-05-08T21:27:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.137+0000 7fbf2d714b80 1 -- 172.21.15.33:0/76672414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08fec8e0 msgr2=0x55cc08fecd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.137+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/76672414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08fec8e0 0x55cc08fecd60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009c30 tx=0x7fbf14009c60 comp rx=0 tx=0).stop 2024-05-08T21:27:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.137+0000 7fbf2d714b80 1 -- 172.21.15.33:0/76672414 shutdown_connections 2024-05-08T21:27:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.137+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/76672414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08fec8e0 0x55cc08fecd60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.137+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/76672414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08feb5c0 0x55cc08feb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.137+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/76672414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08febf20 0x55cc08fec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.137+0000 7fbf2d714b80 1 -- 172.21.15.33:0/76672414 >> 172.21.15.33:0/76672414 conn(0x55cc08f49290 msgr2=0x55cc08fe5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.137+0000 7fbf2d714b80 1 -- 172.21.15.33:0/76672414 shutdown_connections 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.137+0000 7fbf2d714b80 1 -- 172.21.15.33:0/76672414 wait complete. 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.137+0000 7fbf2d714b80 1 -- start start 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.137+0000 7fbf2d714b80 1 Processor -- start 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf2d714b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08feb5c0 0x55cc08ff4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf2d714b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08febf20 0x55cc08ff5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf2d714b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08fec8e0 0x55cc08ffc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf2d714b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08ed75e0 con 0x55cc08feb5c0 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf2d714b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08ed7280 con 0x55cc08febf20 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf2d714b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08dbe720 con 0x55cc08fec8e0 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf2d714b80 10 client.0 _ll_register_callbacks cb 0x55cc08ee0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf1affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08feb5c0 0x55cc08ff4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf1affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08feb5c0 0x55cc08ff4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57524/0 (socket says 172.21.15.33:57524) 2024-05-08T21:27:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf1affd700 1 -- 172.21.15.33:0/2804429759 learned_addr learned my addr 172.21.15.33:0/2804429759 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf1a7fc700 1 --2- 172.21.15.33:0/2804429759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08febf20 0x55cc08ff5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf1affd700 1 -- 172.21.15.33:0/2804429759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08fec8e0 msgr2=0x55cc08ffc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf1affd700 1 --2- 172.21.15.33:0/2804429759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08fec8e0 0x55cc08ffc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf1affd700 1 -- 172.21.15.33:0/2804429759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08febf20 msgr2=0x55cc08ff5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf1affd700 1 --2- 172.21.15.33:0/2804429759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08febf20 0x55cc08ff5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.138+0000 7fbf1affd700 1 -- 172.21.15.33:0/2804429759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c108af0 con 0x55cc08feb5c0 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.139+0000 7fbf1affd700 1 --2- 172.21.15.33:0/2804429759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08feb5c0 0x55cc08ff4e50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c005950 tx=0x7fbf0c09a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6178a824eaed77a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.139+0000 7fbefeff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.139+0000 7fbefeff5700 1 -- 172.21.15.33:0/2804429759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c09a3e0 con 0x55cc08feb5c0 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.139+0000 7fbefeff5700 1 -- 172.21.15.33:0/2804429759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbf0c099470 con 0x55cc08feb5c0 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.139+0000 7fbf2d714b80 1 -- 172.21.15.33:0/2804429759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc08e942f0 con 0x55cc08feb5c0 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.139+0000 7fbefeff5700 1 -- 172.21.15.33:0/2804429759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf0c099630 con 0x55cc08feb5c0 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.139+0000 7fbf2d714b80 10 client.17865 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.139+0000 7fbf2d714b80 1 -- 172.21.15.33:0/2804429759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55cc08d68be0 con 0x55cc08feb5c0 2024-05-08T21:27:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.139+0000 7fbefeff5700 1 -- 172.21.15.33:0/2804429759 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf0c064020 con 0x55cc08feb5c0 2024-05-08T21:27:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.139+0000 7fbefeff5700 1 -- 172.21.15.33:0/2804429759 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf0c0b8110 con 0x55cc08feb5c0 2024-05-08T21:27:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.140+0000 7fbf2d714b80 10 client.17865 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.140+0000 7fbf2d714b80 10 client.17865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.140+0000 7fbf2d714b80 20 client.17865 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.140+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/2804429759 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cc09000cb0 0x55cc090030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.140+0000 7fbf2d714b80 4 client.17865 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.140+0000 7fbf2d714b80 1 -- 172.21.15.33:0/2804429759 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc08fba890 con 0x55cc09000cb0 2024-05-08T21:27:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.140+0000 7fbf1a7fc700 1 --2- 172.21.15.33:0/2804429759 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cc09000cb0 0x55cc090030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.141+0000 7fbefeff5700 10 client.17865 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.141+0000 7fbf1a7fc700 1 --2- 172.21.15.33:0/2804429759 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cc09000cb0 0x55cc090030d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=356b60cd6393ddff server_cookie=e292936d7fd360b2 in_seq=0 out_seq=0 2024-05-08T21:27:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.141+0000 7fbefeff5700 1 -- 172.21.15.33:0/2804429759 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc08fba890 con 0x55cc09000cb0 2024-05-08T21:27:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.141+0000 7fbefeff5700 10 client.17865 handle_command_reply: tid=0 2024-05-08T21:27:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.142+0000 7fbf2d714b80 1 client.17865 shutdown 2024-05-08T21:27:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.142+0000 7fbf2d714b80 1 -- 172.21.15.33:0/2804429759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08feb5c0 msgr2=0x55cc08ff4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.142+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/2804429759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08feb5c0 0x55cc08ff4e50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c005950 tx=0x7fbf0c09a7b0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.142+0000 7fbf2d714b80 1 -- 172.21.15.33:0/2804429759 shutdown_connections 2024-05-08T21:27:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.142+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/2804429759 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cc09000cb0 0x55cc090030d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.142+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/2804429759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08fec8e0 0x55cc08ffc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.142+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/2804429759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08feb5c0 0x55cc08ff4e50 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.142+0000 7fbefeff5700 0 client.17865 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.142+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/2804429759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08febf20 0x55cc08ff5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.142+0000 7fbf2d714b80 1 -- 172.21.15.33:0/2804429759 >> 172.21.15.33:0/2804429759 conn(0x55cc08f49290 msgr2=0x55cc08fea3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:52.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.142+0000 7fbf2d714b80 1 -- 172.21.15.33:0/2804429759 shutdown_connections 2024-05-08T21:27:52.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.142+0000 7fbf2d714b80 1 -- 172.21.15.33:0/2804429759 wait complete. 2024-05-08T21:27:52.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.143+0000 7fbf2d714b80 20 client.17865 trim_cache size 0 max 16384 2024-05-08T21:27:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf2d714b80 1 Processor -- start 2024-05-08T21:27:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf2d714b80 1 -- start start 2024-05-08T21:27:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf2d714b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08feb5c0 0x55cc08fe5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf2d714b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08febf20 0x55cc08ffc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf2d714b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08fec8e0 0x55cc08fe6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf2d714b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08ffd580 con 0x55cc08fec8e0 2024-05-08T21:27:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf2d714b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08fde160 con 0x55cc08feb5c0 2024-05-08T21:27:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf2d714b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08ff5e60 con 0x55cc08febf20 2024-05-08T21:27:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08fec8e0 0x55cc08fe6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08feb5c0 0x55cc08fe5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1affd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08feb5c0 0x55cc08fe5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:46120/0 (socket says 172.21.15.33:46120) 2024-05-08T21:27:52.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1a7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08febf20 0x55cc08ffc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1affd700 1 -- 172.21.15.33:0/619451442 learned_addr learned my addr 172.21.15.33:0/619451442 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08fec8e0 0x55cc08fe6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57536/0 (socket says 172.21.15.33:57536) 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1b7fe700 1 -- 172.21.15.33:0/619451442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08febf20 msgr2=0x55cc08ffc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1b7fe700 1 --2- 172.21.15.33:0/619451442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08febf20 0x55cc08ffc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1b7fe700 1 -- 172.21.15.33:0/619451442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08feb5c0 msgr2=0x55cc08fe5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1b7fe700 1 --2- 172.21.15.33:0/619451442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08feb5c0 0x55cc08fe5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1b7fe700 1 -- 172.21.15.33:0/619451442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc08f015e0 con 0x55cc08fec8e0 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1a7fc700 1 --2- 172.21.15.33:0/619451442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08febf20 0x55cc08ffc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.168+0000 7fbf1affd700 1 --2- 172.21.15.33:0/619451442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08feb5c0 0x55cc08fe5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.169+0000 7fbf1b7fe700 1 --2- 172.21.15.33:0/619451442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08fec8e0 0x55cc08fe6370 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009cd0 tx=0x7fbf140095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b79b4181857aaf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.169+0000 7fbf01ffb700 1 -- 172.21.15.33:0/619451442 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400e070 con 0x55cc08fec8e0 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.169+0000 7fbf01ffb700 1 -- 172.21.15.33:0/619451442 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf14007730 con 0x55cc08fec8e0 2024-05-08T21:27:52.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.169+0000 7fbf01ffb700 1 -- 172.21.15.33:0/619451442 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14024450 con 0x55cc08fec8e0 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.170+0000 7fbf2d714b80 1 -- 172.21.15.33:0/619451442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08fec8e0 msgr2=0x55cc08fe6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.170+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/619451442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08fec8e0 0x55cc08fe6370 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009cd0 tx=0x7fbf140095d0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.170+0000 7fbf2d714b80 1 -- 172.21.15.33:0/619451442 shutdown_connections 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.170+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/619451442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08fec8e0 0x55cc08fe6370 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.170+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/619451442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08feb5c0 0x55cc08fe5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.170+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/619451442 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08febf20 0x55cc08ffc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.170+0000 7fbf2d714b80 1 -- 172.21.15.33:0/619451442 >> 172.21.15.33:0/619451442 conn(0x55cc08f49290 msgr2=0x55cc08ff2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.170+0000 7fbf2d714b80 1 -- 172.21.15.33:0/619451442 shutdown_connections 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.170+0000 7fbf2d714b80 1 -- 172.21.15.33:0/619451442 wait complete. 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.170+0000 7fbf2d714b80 1 -- start start 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.170+0000 7fbf2d714b80 1 Processor -- start 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf2d714b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08feb5c0 0x55cc08ffbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf2d714b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08febf20 0x55cc08ffc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf2d714b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08fec8e0 0x55cc08ff87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf2d714b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08ff8ef0 con 0x55cc08fec8e0 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf2d714b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08ff9060 con 0x55cc08febf20 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf2d714b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc08ff9220 con 0x55cc08feb5c0 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf2d714b80 10 client.0 _ll_register_callbacks cb 0x55cc08ee0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf1a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08febf20 0x55cc08ffc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf1a7fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08febf20 0x55cc08ffc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:46128/0 (socket says 172.21.15.33:46128) 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf1a7fc700 1 -- 172.21.15.33:0/3995467880 learned_addr learned my addr 172.21.15.33:0/3995467880 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf1a7fc700 1 -- 172.21.15.33:0/3995467880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08feb5c0 msgr2=0x55cc08ffbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf1a7fc700 1 --2- 172.21.15.33:0/3995467880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08feb5c0 0x55cc08ffbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf1a7fc700 1 -- 172.21.15.33:0/3995467880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08fec8e0 msgr2=0x55cc08ff87e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.171+0000 7fbf1a7fc700 1 --2- 172.21.15.33:0/3995467880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08fec8e0 0x55cc08ff87e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.172+0000 7fbf1a7fc700 1 -- 172.21.15.33:0/3995467880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc08d68be0 con 0x55cc08febf20 2024-05-08T21:27:52.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.174+0000 7fbf1a7fc700 1 --2- 172.21.15.33:0/3995467880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08febf20 0x55cc08ffc230 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55cc090039c0 tx=0x7fbf10013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5f79fc7b05b0431 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:52.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.174+0000 7fbefeff5700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:27:52.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.175+0000 7fbefeff5700 1 -- 172.21.15.33:0/3995467880 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1001ca00 con 0x55cc08febf20 2024-05-08T21:27:52.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.175+0000 7fbefeff5700 1 -- 172.21.15.33:0/3995467880 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fbf1001cb60 con 0x55cc08febf20 2024-05-08T21:27:52.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.175+0000 7fbf2d714b80 1 -- 172.21.15.33:0/3995467880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf14009460 con 0x55cc08febf20 2024-05-08T21:27:52.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.176+0000 7fbefeff5700 1 -- 172.21.15.33:0/3995467880 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf1001cd20 con 0x55cc08febf20 2024-05-08T21:27:52.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.176+0000 7fbf2d714b80 10 client.27779 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:52.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.176+0000 7fbf2d714b80 1 -- 172.21.15.33:0/3995467880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7fbf10017070 con 0x55cc08febf20 2024-05-08T21:27:52.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.179+0000 7fbefeff5700 1 -- 172.21.15.33:0/3995467880 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf14009460 con 0x55cc08febf20 2024-05-08T21:27:52.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.179+0000 7fbefeff5700 1 -- 172.21.15.33:0/3995467880 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf1000e6f0 con 0x55cc08febf20 2024-05-08T21:27:52.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.179+0000 7fbf2d714b80 10 client.27779 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:52.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.179+0000 7fbf2d714b80 10 client.27779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:52.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.179+0000 7fbf2d714b80 20 client.27779 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:52.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.179+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/3995467880 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cc090698e0 0x55cc0906bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.179+0000 7fbf2d714b80 4 client.27779 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-08T21:27:52.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.179+0000 7fbf2d714b80 1 -- 172.21.15.33:0/3995467880 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cc08f33f40 con 0x55cc090698e0 2024-05-08T21:27:52.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.181+0000 7fbf1affd700 1 --2- 172.21.15.33:0/3995467880 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cc090698e0 0x55cc0906bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.181+0000 7fbf1affd700 1 --2- 172.21.15.33:0/3995467880 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cc090698e0 0x55cc0906bd20 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94bbd0c01a1efb9b server_cookie=7bb84e92937db6a5 in_seq=0 out_seq=0 2024-05-08T21:27:52.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.181+0000 7fbefeff5700 10 client.27779 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:52.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.182+0000 7fbefeff5700 1 -- 172.21.15.33:0/3995467880 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cc08f33f40 con 0x55cc090698e0 2024-05-08T21:27:52.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.182+0000 7fbefeff5700 10 client.27779 handle_command_reply: tid=0 2024-05-08T21:27:52.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.184+0000 7fbf2d714b80 1 client.27779 shutdown 2024-05-08T21:27:52.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.184+0000 7fbf2d714b80 1 -- 172.21.15.33:0/3995467880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08febf20 msgr2=0x55cc08ffc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.184+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/3995467880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08febf20 0x55cc08ffc230 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55cc090039c0 tx=0x7fbf10013040 comp rx=0 tx=0).stop 2024-05-08T21:27:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.184+0000 7fbf2d714b80 1 -- 172.21.15.33:0/3995467880 shutdown_connections 2024-05-08T21:27:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.184+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/3995467880 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55cc090698e0 0x55cc0906bd20 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.184+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/3995467880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc08fec8e0 0x55cc08ff87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.184+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/3995467880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc08feb5c0 0x55cc08ffbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.184+0000 7fbf2d714b80 1 --2- 172.21.15.33:0/3995467880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc08febf20 0x55cc08ffc230 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.184+0000 7fbf2d714b80 1 -- 172.21.15.33:0/3995467880 >> 172.21.15.33:0/3995467880 conn(0x55cc08f49290 msgr2=0x55cc08fe5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.184+0000 7fbefeff5700 0 client.27779 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:52.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.185+0000 7fbf2d714b80 1 -- 172.21.15.33:0/3995467880 shutdown_connections 2024-05-08T21:27:52.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.187+0000 7fbf2d714b80 1 -- 172.21.15.33:0/3995467880 wait complete. 2024-05-08T21:27:52.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.188+0000 7fbf2d714b80 20 client.27779 trim_cache size 0 max 16384 2024-05-08T21:27:52.190 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:27:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-05-08T21:27:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "a7bcc8a2-090f-400a-8044-2f07f3f18032", 2024-05-08T21:27:52.190 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-05-08T21:27:52.190 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:27:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.188+0000 7fbf2155a700 1 -- 172.21.15.33:0/456653052 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbf040a1120 msgr2=0x7fbf040a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.188+0000 7fbf2155a700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbf040a1120 0x7fbf040a35e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009df0 tx=0x7fbf10009260 comp rx=0 tx=0).stop 2024-05-08T21:27:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.188+0000 7fbf2155a700 1 -- 172.21.15.33:0/456653052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c0ffd60 msgr2=0x7fbf1c196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.188+0000 7fbf2155a700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c0ffd60 0x7fbf1c196e40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00e970 tx=0x7fbf0c00ec80 comp rx=0 tx=0).stop 2024-05-08T21:27:52.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.188+0000 7fbf2155a700 1 -- 172.21.15.33:0/456653052 shutdown_connections 2024-05-08T21:27:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.188+0000 7fbf2155a700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf1c1018f0 0x7fbf1c19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.188+0000 7fbf2155a700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf1c0ffd60 0x7fbf1c196e40 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.188+0000 7fbf2155a700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fbf040a1120 0x7fbf040a35e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.188+0000 7fbf2155a700 1 --2- 172.21.15.33:0/456653052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf1c100f50 0x7fbf1c197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.188+0000 7fbf2155a700 1 -- 172.21.15.33:0/456653052 >> 172.21.15.33:0/456653052 conn(0x7fbf1c0fb290 msgr2=0x7fbf1c068230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.189+0000 7fbf2155a700 1 -- 172.21.15.33:0/456653052 shutdown_connections 2024-05-08T21:27:52.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.189+0000 7fbf2155a700 1 -- 172.21.15.33:0/456653052 wait complete. 2024-05-08T21:27:52.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:27:52.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.477+0000 7f451aefe700 1 -- 172.21.15.33:0/413439795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4514073a00 msgr2=0x7f4514073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.477+0000 7f451aefe700 1 --2- 172.21.15.33:0/413439795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4514073a00 0x7f4514073e80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4500009990 tx=0x7f4500009ca0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.477+0000 7f451aefe700 1 -- 172.21.15.33:0/413439795 shutdown_connections 2024-05-08T21:27:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.477+0000 7f451aefe700 1 --2- 172.21.15.33:0/413439795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f451410d450 0x7f451410f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.477+0000 7f451aefe700 1 --2- 172.21.15.33:0/413439795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4514074fe0 0x7f4514073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.477+0000 7f451aefe700 1 --2- 172.21.15.33:0/413439795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4514073a00 0x7f4514073e80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.477+0000 7f451aefe700 1 -- 172.21.15.33:0/413439795 >> 172.21.15.33:0/413439795 conn(0x7f45140fc3f0 msgr2=0x7f45140fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.477+0000 7f451aefe700 1 -- 172.21.15.33:0/413439795 shutdown_connections 2024-05-08T21:27:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.477+0000 7f451aefe700 1 -- 172.21.15.33:0/413439795 wait complete. 2024-05-08T21:27:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.478+0000 7f451aefe700 1 Processor -- start 2024-05-08T21:27:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.478+0000 7f451aefe700 1 -- start start 2024-05-08T21:27:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.478+0000 7f451aefe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4514073a00 0x7f451406f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.478+0000 7f451aefe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4514074fe0 0x7f451406fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.478+0000 7f451aefe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f451410d450 0x7f45141ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.478+0000 7f451aefe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4514070490 con 0x7f4514074fe0 2024-05-08T21:27:52.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.478+0000 7f451aefe700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4514070600 con 0x7f4514073a00 2024-05-08T21:27:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f451aefe700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45141ad990 con 0x7f451410d450 2024-05-08T21:27:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4518c9a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4514073a00 0x7f451406f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4518c9a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4514073a00 0x7f451406f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:46144/0 (socket says 172.21.15.33:46144) 2024-05-08T21:27:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4518c9a700 1 -- 172.21.15.33:0/1998768213 learned_addr learned my addr 172.21.15.33:0/1998768213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f451949b700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f451410d450 0x7f45141ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4513fff700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4514074fe0 0x7f451406fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4518c9a700 1 -- 172.21.15.33:0/1998768213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f451410d450 msgr2=0x7f45141ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4518c9a700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f451410d450 0x7f45141ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4518c9a700 1 -- 172.21.15.33:0/1998768213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4514074fe0 msgr2=0x7f451406fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4518c9a700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4514074fe0 0x7f451406fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4518c9a700 1 -- 172.21.15.33:0/1998768213 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45000095f0 con 0x7f4514073a00 2024-05-08T21:27:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4518c9a700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4514073a00 0x7f451406f980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f450800e9a0 tx=0x7f450800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4511ffb700 1 -- 172.21.15.33:0/1998768213 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508009790 con 0x7f4514073a00 2024-05-08T21:27:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4511ffb700 1 -- 172.21.15.33:0/1998768213 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4508010430 con 0x7f4514073a00 2024-05-08T21:27:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f4511ffb700 1 -- 172.21.15.33:0/1998768213 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450800cea0 con 0x7f4514073a00 2024-05-08T21:27:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.479+0000 7f451aefe700 1 -- 172.21.15.33:0/1998768213 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45141adc70 con 0x7f4514073a00 2024-05-08T21:27:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.480+0000 7f451aefe700 1 -- 172.21.15.33:0/1998768213 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45141ae130 con 0x7f4514073a00 2024-05-08T21:27:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.483+0000 7f4511ffb700 1 -- 172.21.15.33:0/1998768213 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f45080105a0 con 0x7f4514073a00 2024-05-08T21:27:52.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.485+0000 7f45270b8b80 1 Processor -- start 2024-05-08T21:27:52.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.488+0000 7f4511ffb700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f45040a1120 0x7f45040a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.488+0000 7f4511ffb700 1 -- 172.21.15.33:0/1998768213 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f4508014070 con 0x7f4514073a00 2024-05-08T21:27:52.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.492+0000 7f45270b8b80 1 -- start start 2024-05-08T21:27:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.492+0000 7f45270b8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823b540 0x56482823b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.492+0000 7f45270b8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823bea0 0x56482823c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f45270b8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823c860 0x56482823cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f45270b8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564828192150 con 0x56482823bea0 2024-05-08T21:27:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f45270b8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482818fb10 con 0x56482823b540 2024-05-08T21:27:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f45270b8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482818b400 con 0x56482823c860 2024-05-08T21:27:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f4518c9a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823b540 0x56482823b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f4518c9a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823b540 0x56482823b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:46152/0 (socket says 172.21.15.33:46152) 2024-05-08T21:27:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f4518c9a700 1 -- 172.21.15.33:0/429314036 learned_addr learned my addr 172.21.15.33:0/429314036 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f4518c9a700 1 -- 172.21.15.33:0/429314036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823c860 msgr2=0x56482823cce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:27:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f4518c9a700 1 --2- 172.21.15.33:0/429314036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823c860 0x56482823cce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f4518c9a700 1 -- 172.21.15.33:0/429314036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823bea0 msgr2=0x56482823c320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T21:27:52.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f4518c9a700 1 --2- 172.21.15.33:0/429314036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823bea0 0x56482823c320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f4518c9a700 1 -- 172.21.15.33:0/429314036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56482800c3b0 con 0x56482823b540 2024-05-08T21:27:52.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.493+0000 7f4518c9a700 1 --2- 172.21.15.33:0/429314036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823b540 0x56482823b960 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f45080a1e60 tx=0x7f4508099940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f774aa123ad9fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:52.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.494+0000 7f44feffd700 1 -- 172.21.15.33:0/429314036 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508099f10 con 0x56482823b540 2024-05-08T21:27:52.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.494+0000 7f44feffd700 1 -- 172.21.15.33:0/429314036 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f450809dd40 con 0x56482823b540 2024-05-08T21:27:52.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.494+0000 7f44feffd700 1 -- 172.21.15.33:0/429314036 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45080a1f00 con 0x56482823b540 2024-05-08T21:27:52.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.495+0000 7f45270b8b80 1 -- 172.21.15.33:0/429314036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823b540 msgr2=0x56482823b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.495+0000 7f45270b8b80 1 --2- 172.21.15.33:0/429314036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823b540 0x56482823b960 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f45080a1e60 tx=0x7f4508099940 comp rx=0 tx=0).stop 2024-05-08T21:27:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.498+0000 7f451949b700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f45040a1120 0x7f45040a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.498+0000 7f45270b8b80 1 -- 172.21.15.33:0/429314036 shutdown_connections 2024-05-08T21:27:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.498+0000 7f45270b8b80 1 --2- 172.21.15.33:0/429314036 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823c860 0x56482823cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.498+0000 7f45270b8b80 1 --2- 172.21.15.33:0/429314036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823b540 0x56482823b960 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.498+0000 7f45270b8b80 1 --2- 172.21.15.33:0/429314036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823bea0 0x56482823c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.498+0000 7f45270b8b80 1 -- 172.21.15.33:0/429314036 >> 172.21.15.33:0/429314036 conn(0x564828199210 msgr2=0x564828235de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.499+0000 7f451949b700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f45040a1120 0x7f45040a35e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f450c009ec0 tx=0x7f450c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:52.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.500+0000 7f45270b8b80 1 -- 172.21.15.33:0/429314036 shutdown_connections 2024-05-08T21:27:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.504+0000 7f45270b8b80 1 -- 172.21.15.33:0/429314036 wait complete. 2024-05-08T21:27:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.505+0000 7f45270b8b80 1 -- start start 2024-05-08T21:27:52.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.505+0000 7f45270b8b80 1 Processor -- start 2024-05-08T21:27:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.505+0000 7f45270b8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823b540 0x564828266ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.505+0000 7f45270b8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823bea0 0x564828267430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.506+0000 7f45270b8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823c860 0x56482826e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.506+0000 7f45270b8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564828127560 con 0x56482823c860 2024-05-08T21:27:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.506+0000 7f45270b8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564828127200 con 0x56482823bea0 2024-05-08T21:27:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.506+0000 7f45270b8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482800e6a0 con 0x56482823b540 2024-05-08T21:27:52.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.506+0000 7f45270b8b80 10 client.0 _ll_register_callbacks cb 0x564828130b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:52.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.507+0000 7f4513fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823bea0 0x564828267430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.508+0000 7f451949b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823c860 0x56482826e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.508+0000 7f451949b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823c860 0x56482826e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57550/0 (socket says 172.21.15.33:57550) 2024-05-08T21:27:52.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.508+0000 7f451949b700 1 -- 172.21.15.33:0/1490744925 learned_addr learned my addr 172.21.15.33:0/1490744925 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.510+0000 7f451949b700 1 -- 172.21.15.33:0/1490744925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823b540 msgr2=0x564828266ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:27:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.510+0000 7f451949b700 1 --2- 172.21.15.33:0/1490744925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823b540 0x564828266ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.510+0000 7f451949b700 1 -- 172.21.15.33:0/1490744925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823bea0 msgr2=0x564828267430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.510+0000 7f451949b700 1 --2- 172.21.15.33:0/1490744925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823bea0 0x564828267430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.510+0000 7f451949b700 1 -- 172.21.15.33:0/1490744925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45141ae130 con 0x56482823c860 2024-05-08T21:27:52.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.510+0000 7f451949b700 1 --2- 172.21.15.33:0/1490744925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823c860 0x56482826e3d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f450c0045f0 tx=0x7f450c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955287cc655b4b53 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.511+0000 7f44f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:27:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.511+0000 7f44f7fff700 1 -- 172.21.15.33:0/1490744925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c01c040 con 0x56482823c860 2024-05-08T21:27:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.511+0000 7f44f7fff700 1 -- 172.21.15.33:0/1490744925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f450c00fe80 con 0x56482823c860 2024-05-08T21:27:52.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.511+0000 7f45270b8b80 1 -- 172.21.15.33:0/1490744925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648280e4270 con 0x56482823c860 2024-05-08T21:27:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.512+0000 7f44f7fff700 1 -- 172.21.15.33:0/1490744925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f450c00e040 con 0x56482823c860 2024-05-08T21:27:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.512+0000 7f45270b8b80 10 client.17877 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:52.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.512+0000 7f45270b8b80 1 -- 172.21.15.33:0/1490744925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564827fb8b60 con 0x56482823c860 2024-05-08T21:27:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.512+0000 7f44f7fff700 1 -- 172.21.15.33:0/1490744925 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5648280e4270 con 0x56482823c860 2024-05-08T21:27:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.512+0000 7f44f7fff700 1 -- 172.21.15.33:0/1490744925 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f450c015410 con 0x56482823c860 2024-05-08T21:27:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.512+0000 7f45270b8b80 10 client.17877 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.512+0000 7f45270b8b80 10 client.17877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.512+0000 7f45270b8b80 20 client.17877 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.512+0000 7f45270b8b80 1 --2- 172.21.15.33:0/1490744925 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564828272ce0 0x564828275100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.512+0000 7f45270b8b80 4 client.17877 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:27:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.512+0000 7f45270b8b80 1 -- 172.21.15.33:0/1490744925 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56482820a810 con 0x564828272ce0 2024-05-08T21:27:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.513+0000 7f4518c9a700 1 --2- 172.21.15.33:0/1490744925 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564828272ce0 0x564828275100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.513+0000 7f44f7fff700 10 client.17877 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:52.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.513+0000 7f4518c9a700 1 --2- 172.21.15.33:0/1490744925 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564828272ce0 0x564828275100 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4547140765ccacae server_cookie=4e6152397536881e in_seq=0 out_seq=0 2024-05-08T21:27:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f44f7fff700 1 -- 172.21.15.33:0/1490744925 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56482820a810 con 0x564828272ce0 2024-05-08T21:27:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f44f7fff700 10 client.17877 handle_command_reply: tid=0 2024-05-08T21:27:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f45270b8b80 1 client.17877 shutdown 2024-05-08T21:27:52.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f45270b8b80 1 -- 172.21.15.33:0/1490744925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823c860 msgr2=0x56482826e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f45270b8b80 1 --2- 172.21.15.33:0/1490744925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823c860 0x56482826e3d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f450c0045f0 tx=0x7f450c004ad0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f45270b8b80 1 -- 172.21.15.33:0/1490744925 shutdown_connections 2024-05-08T21:27:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f45270b8b80 1 --2- 172.21.15.33:0/1490744925 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x564828272ce0 0x564828275100 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f45270b8b80 1 --2- 172.21.15.33:0/1490744925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823c860 0x56482826e3d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f45270b8b80 1 --2- 172.21.15.33:0/1490744925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823b540 0x564828266ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f44f7fff700 0 client.17877 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f45270b8b80 1 --2- 172.21.15.33:0/1490744925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823bea0 0x564828267430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f45270b8b80 1 -- 172.21.15.33:0/1490744925 >> 172.21.15.33:0/1490744925 conn(0x564828199210 msgr2=0x5648282355a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f45270b8b80 1 -- 172.21.15.33:0/1490744925 shutdown_connections 2024-05-08T21:27:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f45270b8b80 1 -- 172.21.15.33:0/1490744925 wait complete. 2024-05-08T21:27:52.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.514+0000 7f45270b8b80 20 client.17877 trim_cache size 0 max 16384 2024-05-08T21:27:52.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.539+0000 7f45270b8b80 1 Processor -- start 2024-05-08T21:27:52.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.539+0000 7f45270b8b80 1 -- start start 2024-05-08T21:27:52.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.539+0000 7f45270b8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823b540 0x564828265ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.539+0000 7f45270b8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823bea0 0x56482826e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.539+0000 7f45270b8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823c860 0x564828232210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.539+0000 7f45270b8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564828232dc0 con 0x56482823b540 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.539+0000 7f45270b8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482822e0d0 con 0x56482823c860 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.539+0000 7f45270b8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564828230800 con 0x56482823bea0 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.540+0000 7f451949b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823c860 0x564828232210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.540+0000 7f451949b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823c860 0x564828232210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:46166/0 (socket says 172.21.15.33:46166) 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.540+0000 7f451949b700 1 -- 172.21.15.33:0/66906596 learned_addr learned my addr 172.21.15.33:0/66906596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.540+0000 7f4518c9a700 1 --2- 172.21.15.33:0/66906596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823b540 0x564828265ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.540+0000 7f451949b700 1 -- 172.21.15.33:0/66906596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823bea0 msgr2=0x56482826e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.540+0000 7f451949b700 1 --2- 172.21.15.33:0/66906596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823bea0 0x56482826e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.540+0000 7f451949b700 1 -- 172.21.15.33:0/66906596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823b540 msgr2=0x564828265ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.540+0000 7f451949b700 1 --2- 172.21.15.33:0/66906596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823b540 0x564828265ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.540+0000 7f451949b700 1 -- 172.21.15.33:0/66906596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564828151560 con 0x56482823c860 2024-05-08T21:27:52.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.540+0000 7f4518c9a700 1 --2- 172.21.15.33:0/66906596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823b540 0x564828265ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:27:52.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.541+0000 7f451949b700 1 --2- 172.21.15.33:0/66906596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823c860 0x564828232210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f450c015770 tx=0x7f450c017720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfb34748274f8649 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:52.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.541+0000 7f44ff7fe700 1 -- 172.21.15.33:0/66906596 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c00e040 con 0x56482823c860 2024-05-08T21:27:52.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.541+0000 7f44ff7fe700 1 -- 172.21.15.33:0/66906596 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f450c00f040 con 0x56482823c860 2024-05-08T21:27:52.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.541+0000 7f44ff7fe700 1 -- 172.21.15.33:0/66906596 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c022470 con 0x56482823c860 2024-05-08T21:27:52.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.542+0000 7f45270b8b80 1 -- 172.21.15.33:0/66906596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823c860 msgr2=0x564828232210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.542+0000 7f45270b8b80 1 --2- 172.21.15.33:0/66906596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823c860 0x564828232210 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f450c015770 tx=0x7f450c017720 comp rx=0 tx=0).stop 2024-05-08T21:27:52.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.542+0000 7f45270b8b80 1 -- 172.21.15.33:0/66906596 shutdown_connections 2024-05-08T21:27:52.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.542+0000 7f45270b8b80 1 --2- 172.21.15.33:0/66906596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823c860 0x564828232210 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.542+0000 7f45270b8b80 1 --2- 172.21.15.33:0/66906596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823b540 0x564828265ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.542+0000 7f45270b8b80 1 --2- 172.21.15.33:0/66906596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823bea0 0x56482826e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.542+0000 7f45270b8b80 1 -- 172.21.15.33:0/66906596 >> 172.21.15.33:0/66906596 conn(0x564828199210 msgr2=0x564828242ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:52.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.542+0000 7f45270b8b80 1 -- 172.21.15.33:0/66906596 shutdown_connections 2024-05-08T21:27:52.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.542+0000 7f45270b8b80 1 -- 172.21.15.33:0/66906596 wait complete. 2024-05-08T21:27:52.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.543+0000 7f45270b8b80 1 -- start start 2024-05-08T21:27:52.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.543+0000 7f45270b8b80 1 Processor -- start 2024-05-08T21:27:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f45270b8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823b540 0x564828269b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f45270b8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823bea0 0x56482826a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f45270b8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823c860 0x56482826cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f45270b8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482826d4a0 con 0x56482823c860 2024-05-08T21:27:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f45270b8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482826d610 con 0x56482823bea0 2024-05-08T21:27:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f45270b8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56482826d7d0 con 0x56482823b540 2024-05-08T21:27:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f45270b8b80 10 client.0 _ll_register_callbacks cb 0x564828130b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:27:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f4518c9a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823b540 0x564828269b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f4518c9a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823b540 0x564828269b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:41766/0 (socket says 172.21.15.33:41766) 2024-05-08T21:27:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f4518c9a700 1 -- 172.21.15.33:0/3256320259 learned_addr learned my addr 172.21.15.33:0/3256320259 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:27:52.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f451949b700 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823c860 0x56482826cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f4513fff700 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823bea0 0x56482826a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f4518c9a700 1 -- 172.21.15.33:0/3256320259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823bea0 msgr2=0x56482826a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f4518c9a700 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823bea0 0x56482826a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f4518c9a700 1 -- 172.21.15.33:0/3256320259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823c860 msgr2=0x56482826cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:27:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f4518c9a700 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823c860 0x56482826cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.544+0000 7f4518c9a700 1 -- 172.21.15.33:0/3256320259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564827fb8b60 con 0x56482823b540 2024-05-08T21:27:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.545+0000 7f4518c9a700 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823b540 0x564828269b80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f450800c2d0 tx=0x7f4508098330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9f8932355282ca7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:27:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.545+0000 7f44f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:27:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.545+0000 7f44f7fff700 1 -- 172.21.15.33:0/3256320259 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508099600 con 0x56482823b540 2024-05-08T21:27:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.545+0000 7f44f7fff700 1 -- 172.21.15.33:0/3256320259 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f45080a2390 con 0x56482823b540 2024-05-08T21:27:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.545+0000 7f45270b8b80 1 -- 172.21.15.33:0/3256320259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f450809dd40 con 0x56482823b540 2024-05-08T21:27:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.545+0000 7f44f7fff700 1 -- 172.21.15.33:0/3256320259 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45080b8250 con 0x56482823b540 2024-05-08T21:27:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.545+0000 7f45270b8b80 10 client.27799 fetch_fsmap learned FSMap version 39 2024-05-08T21:27:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.545+0000 7f45270b8b80 1 -- 172.21.15.33:0/3256320259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564828010cd0 con 0x56482823b540 2024-05-08T21:27:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.545+0000 7f44f7fff700 1 -- 172.21.15.33:0/3256320259 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f450809dd40 con 0x56482823b540 2024-05-08T21:27:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.546+0000 7f44f7fff700 1 -- 172.21.15.33:0/3256320259 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 39) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4508094bb0 con 0x56482823b540 2024-05-08T21:27:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.546+0000 7f45270b8b80 10 client.27799 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T21:27:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.546+0000 7f45270b8b80 10 client.27799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:27:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.546+0000 7f45270b8b80 20 client.27799 populate_metadata read hostname 'smithi033' 2024-05-08T21:27:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.546+0000 7f45270b8b80 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5648282c98f0 0x5648282cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:27:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.546+0000 7f45270b8b80 4 client.27799 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:27:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.546+0000 7f45270b8b80 1 -- 172.21.15.33:0/3256320259 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564828183ec0 con 0x5648282c98f0 2024-05-08T21:27:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.546+0000 7f4513fff700 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5648282c98f0 0x5648282cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:27:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.546+0000 7f44f7fff700 10 client.27799 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.546+0000 7f4513fff700 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5648282c98f0 0x5648282cbd10 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcb01355a4cd6799 server_cookie=6780a514bfaae7d6 in_seq=0 out_seq=0 2024-05-08T21:27:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.547+0000 7f44f7fff700 1 -- 172.21.15.33:0/3256320259 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564828183ec0 con 0x5648282c98f0 2024-05-08T21:27:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.547+0000 7f44f7fff700 10 client.27799 handle_command_reply: tid=0 2024-05-08T21:27:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.547+0000 7f45270b8b80 1 client.27799 shutdown 2024-05-08T21:27:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.547+0000 7f45270b8b80 1 -- 172.21.15.33:0/3256320259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823b540 msgr2=0x564828269b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.547+0000 7f45270b8b80 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823b540 0x564828269b80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f450800c2d0 tx=0x7f4508098330 comp rx=0 tx=0).stop 2024-05-08T21:27:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.547+0000 7f45270b8b80 1 -- 172.21.15.33:0/3256320259 shutdown_connections 2024-05-08T21:27:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.547+0000 7f45270b8b80 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x5648282c98f0 0x5648282cbd10 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.548+0000 7f45270b8b80 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482823c860 0x56482826cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.548+0000 7f45270b8b80 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56482823b540 0x564828269b80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.548+0000 7f45270b8b80 1 --2- 172.21.15.33:0/3256320259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56482823bea0 0x56482826a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.548+0000 7f45270b8b80 1 -- 172.21.15.33:0/3256320259 >> 172.21.15.33:0/3256320259 conn(0x564828199210 msgr2=0x564828279a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:52.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.548+0000 7f44f7fff700 0 client.27799 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:27:52.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.548+0000 7f45270b8b80 1 -- 172.21.15.33:0/3256320259 shutdown_connections 2024-05-08T21:27:52.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.548+0000 7f45270b8b80 1 -- 172.21.15.33:0/3256320259 wait complete. 2024-05-08T21:27:52.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.548+0000 7f45270b8b80 20 client.27799 trim_cache size 0 max 16384 2024-05-08T21:27:52.553 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:27:52.553 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-08T21:27:52.553 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-05-08T21:27:52.553 INFO:teuthology.orchestra.run.smithi033.stdout: "a7bcc8a2-090f-400a-8044-2f07f3f18032": { 2024-05-08T21:27:52.554 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-05-08T21:27:52.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "a7bcc8a2-090f-400a-8044-2f07f3f18032", 2024-05-08T21:27:52.554 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-05-08T21:27:52.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:27:52.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:27:52.554 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:27:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.549+0000 7f451aefe700 1 -- 172.21.15.33:0/1998768213 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f45040a1120 msgr2=0x7f45040a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.549+0000 7f451aefe700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f45040a1120 0x7f45040a35e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f450c009ec0 tx=0x7f450c00b040 comp rx=0 tx=0).stop 2024-05-08T21:27:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.549+0000 7f451aefe700 1 -- 172.21.15.33:0/1998768213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4514073a00 msgr2=0x7f451406f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:27:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.549+0000 7f451aefe700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4514073a00 0x7f451406f980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f450800e9a0 tx=0x7f450800ecb0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.549+0000 7f451aefe700 1 -- 172.21.15.33:0/1998768213 shutdown_connections 2024-05-08T21:27:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.549+0000 7f451aefe700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f451410d450 0x7f45141ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.549+0000 7f451aefe700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f45040a1120 0x7f45040a35e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.549+0000 7f451aefe700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4514073a00 0x7f451406f980 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.549+0000 7f451aefe700 1 --2- 172.21.15.33:0/1998768213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4514074fe0 0x7f451406fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:27:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.549+0000 7f451aefe700 1 -- 172.21.15.33:0/1998768213 >> 172.21.15.33:0/1998768213 conn(0x7f45140fc3f0 msgr2=0x7f45141007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:27:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.549+0000 7f451aefe700 1 -- 172.21.15.33:0/1998768213 shutdown_connections 2024-05-08T21:27:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:27:52.549+0000 7f451aefe700 1 -- 172.21.15.33:0/1998768213 wait complete. 2024-05-08T21:27:52.568 INFO:tasks.cephfs.filesystem:scrub status for tag:a7bcc8a2-090f-400a-8044-2f07f3f18032 - {'path': '/', 'tag': 'a7bcc8a2-090f-400a-8044-2f07f3f18032', 'options': 'recursive,force'} 2024-05-08T21:27:53.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub queued for path: / 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: active paths [/] 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: pgmap v557: 129 pgs: 129 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 478 op/s 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:53 smithi033 ceph-mon[42528]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub queued for path: / 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: active paths [/] 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: pgmap v557: 129 pgs: 129 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 478 op/s 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:53 smithi071 ceph-mon[47074]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub queued for path: / 2024-05-08T21:27:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: active paths [/] 2024-05-08T21:27:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: pgmap v557: 129 pgs: 129 active+clean; 263 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 478 op/s 2024-05-08T21:27:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:53.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:53 smithi082 ceph-mon[47512]: scrub summary: idle+waiting paths [/] 2024-05-08T21:27:55.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:55 smithi033 ceph-mon[42528]: pgmap v558: 129 pgs: 129 active+clean; 265 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 598 op/s 2024-05-08T21:27:55.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:55 smithi071 ceph-mon[47074]: pgmap v558: 129 pgs: 129 active+clean; 265 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 598 op/s 2024-05-08T21:27:55.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:55 smithi082 ceph-mon[47512]: pgmap v558: 129 pgs: 129 active+clean; 265 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 598 op/s 2024-05-08T21:27:57.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:57 smithi033 ceph-mon[42528]: pgmap v559: 129 pgs: 129 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 504 op/s 2024-05-08T21:27:57.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:57 smithi071 ceph-mon[47074]: pgmap v559: 129 pgs: 129 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 504 op/s 2024-05-08T21:27:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:57 smithi082 ceph-mon[47512]: pgmap v559: 129 pgs: 129 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 504 op/s 2024-05-08T21:27:59.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:27:59 smithi033 ceph-mon[42528]: pgmap v560: 129 pgs: 129 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 452 op/s 2024-05-08T21:27:59.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:27:59 smithi071 ceph-mon[47074]: pgmap v560: 129 pgs: 129 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 452 op/s 2024-05-08T21:27:59.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:27:59 smithi082 ceph-mon[47512]: pgmap v560: 129 pgs: 129 active+clean; 269 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 452 op/s 2024-05-08T21:28:01.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:01 smithi033 ceph-mon[42528]: pgmap v561: 129 pgs: 129 active+clean; 275 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.7 MiB/s wr, 623 op/s 2024-05-08T21:28:01.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:01 smithi071 ceph-mon[47074]: pgmap v561: 129 pgs: 129 active+clean; 275 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.7 MiB/s wr, 623 op/s 2024-05-08T21:28:01.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:01 smithi082 ceph-mon[47512]: pgmap v561: 129 pgs: 129 active+clean; 275 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.7 MiB/s wr, 623 op/s 2024-05-08T21:28:03.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:03 smithi033 ceph-mon[42528]: pgmap v562: 129 pgs: 129 active+clean; 281 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 470 op/s 2024-05-08T21:28:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:03 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:28:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:03 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:28:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:03 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:28:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:03 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:28:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:03 smithi033 ceph-mon[42528]: scrub summary: idle 2024-05-08T21:28:03.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:03 smithi071 ceph-mon[47074]: pgmap v562: 129 pgs: 129 active+clean; 281 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 470 op/s 2024-05-08T21:28:03.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:03 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:28:03.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:03 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:28:03.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:03 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:28:03.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:03 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:28:03.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:03 smithi071 ceph-mon[47074]: scrub summary: idle 2024-05-08T21:28:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:03 smithi082 ceph-mon[47512]: pgmap v562: 129 pgs: 129 active+clean; 281 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 470 op/s 2024-05-08T21:28:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:03 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:28:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:03 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:28:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:03 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:28:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:03 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:28:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:03 smithi082 ceph-mon[47512]: scrub summary: idle 2024-05-08T21:28:05.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:05 smithi033 ceph-mon[42528]: pgmap v563: 129 pgs: 129 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 612 op/s 2024-05-08T21:28:05.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:05 smithi071 ceph-mon[47074]: pgmap v563: 129 pgs: 129 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 612 op/s 2024-05-08T21:28:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:05 smithi082 ceph-mon[47512]: pgmap v563: 129 pgs: 129 active+clean; 266 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 612 op/s 2024-05-08T21:28:06.535 INFO:tasks.workunit.client.0.smithi033.stdout:All 50000 operations completed A-OK! 2024-05-08T21:28:06.537 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T21:28:06.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T21:28:07.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:07 smithi033 ceph-mon[42528]: pgmap v564: 129 pgs: 129 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 502 op/s 2024-05-08T21:28:07.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:07 smithi071 ceph-mon[47074]: pgmap v564: 129 pgs: 129 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 502 op/s 2024-05-08T21:28:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:07 smithi082 ceph-mon[47512]: pgmap v564: 129 pgs: 129 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 502 op/s 2024-05-08T21:28:09.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:09 smithi033 ceph-mon[42528]: pgmap v565: 129 pgs: 129 active+clean; 258 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 453 op/s 2024-05-08T21:28:09.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:09 smithi071 ceph-mon[47074]: pgmap v565: 129 pgs: 129 active+clean; 258 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 453 op/s 2024-05-08T21:28:09.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:09 smithi082 ceph-mon[47512]: pgmap v565: 129 pgs: 129 active+clean; 258 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 453 op/s 2024-05-08T21:28:11.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:11 smithi033 ceph-mon[42528]: pgmap v566: 129 pgs: 129 active+clean; 259 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 544 op/s 2024-05-08T21:28:11.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:11 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:11.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:11 smithi071 ceph-mon[47074]: pgmap v566: 129 pgs: 129 active+clean; 259 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 544 op/s 2024-05-08T21:28:11.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:11 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:11 smithi082 ceph-mon[47512]: pgmap v566: 129 pgs: 129 active+clean; 259 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.3 MiB/s wr, 544 op/s 2024-05-08T21:28:11.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:11 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:13.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:13 smithi033 ceph-mon[42528]: pgmap v567: 129 pgs: 129 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 381 op/s 2024-05-08T21:28:13.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:13 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:13.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:13 smithi071 ceph-mon[47074]: pgmap v567: 129 pgs: 129 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 381 op/s 2024-05-08T21:28:13.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:13 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:13.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:13 smithi082 ceph-mon[47512]: pgmap v567: 129 pgs: 129 active+clean; 260 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 381 op/s 2024-05-08T21:28:13.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:13 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:14.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:14 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:14.856 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:14 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:14.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:14 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:15.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:15 smithi071 ceph-mon[47074]: pgmap v568: 129 pgs: 129 active+clean; 254 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 392 op/s 2024-05-08T21:28:15.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:15 smithi082 ceph-mon[47512]: pgmap v568: 129 pgs: 129 active+clean; 254 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 392 op/s 2024-05-08T21:28:15.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:15 smithi033 ceph-mon[42528]: pgmap v568: 129 pgs: 129 active+clean; 254 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 392 op/s 2024-05-08T21:28:16.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:16 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:16.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:16 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:16.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:16 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:17.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:17 smithi071 ceph-mon[47074]: pgmap v569: 129 pgs: 129 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 266 op/s 2024-05-08T21:28:17.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:17 smithi082 ceph-mon[47512]: pgmap v569: 129 pgs: 129 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 266 op/s 2024-05-08T21:28:17.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:17 smithi033 ceph-mon[42528]: pgmap v569: 129 pgs: 129 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.2 MiB/s wr, 266 op/s 2024-05-08T21:28:18.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:18 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:18.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:18 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:18.959 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:18 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:19.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:19 smithi071 ceph-mon[47074]: pgmap v570: 129 pgs: 129 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1020 KiB/s wr, 252 op/s 2024-05-08T21:28:19.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:19 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:19 smithi082 ceph-mon[47512]: pgmap v570: 129 pgs: 129 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1020 KiB/s wr, 252 op/s 2024-05-08T21:28:19.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:19 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:19.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:19 smithi033 ceph-mon[42528]: pgmap v570: 129 pgs: 129 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1020 KiB/s wr, 252 op/s 2024-05-08T21:28:19.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:19 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:20.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:20 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:28:20.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:20 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:28:20.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:20 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T21:28:21.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:21 smithi071 ceph-mon[47074]: pgmap v571: 129 pgs: 129 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-05-08T21:28:21.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:21 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:28:21.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:21 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:28:21.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:21 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:28:21.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:21 smithi082 ceph-mon[47512]: pgmap v571: 129 pgs: 129 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-05-08T21:28:21.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:21 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:28:21.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:21 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:28:21.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:21 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:28:21.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:21 smithi033 ceph-mon[42528]: pgmap v571: 129 pgs: 129 active+clean; 255 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 322 op/s 2024-05-08T21:28:21.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:21 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T21:28:21.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:21 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T21:28:21.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:21 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' 2024-05-08T21:28:22.570 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-05-08T21:28:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.727+0000 7f8b7a713700 1 -- 172.21.15.33:0/594468883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b74100f50 msgr2=0x7f8b741013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.727+0000 7f8b7a713700 1 --2- 172.21.15.33:0/594468883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b74100f50 0x7f8b741013b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8b680099c0 tx=0x7f8b68009cd0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.727+0000 7f8b7a713700 1 -- 172.21.15.33:0/594468883 shutdown_connections 2024-05-08T21:28:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.727+0000 7f8b7a713700 1 --2- 172.21.15.33:0/594468883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b741018f0 0x7f8b741081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.727+0000 7f8b7a713700 1 --2- 172.21.15.33:0/594468883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b740ffd60 0x7f8b74100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.727+0000 7f8b7a713700 1 --2- 172.21.15.33:0/594468883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b74100f50 0x7f8b741013b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.727+0000 7f8b7a713700 1 -- 172.21.15.33:0/594468883 >> 172.21.15.33:0/594468883 conn(0x7f8b740fb290 msgr2=0x7f8b740fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:22.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.727+0000 7f8b7a713700 1 -- 172.21.15.33:0/594468883 shutdown_connections 2024-05-08T21:28:22.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.727+0000 7f8b7a713700 1 -- 172.21.15.33:0/594468883 wait complete. 2024-05-08T21:28:22.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.728+0000 7f8b7a713700 1 Processor -- start 2024-05-08T21:28:22.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.728+0000 7f8b7a713700 1 -- start start 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.728+0000 7f8b7a713700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b740ffd60 0x7f8b74196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.728+0000 7f8b7a713700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b74100f50 0x7f8b741972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.728+0000 7f8b7a713700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b741018f0 0x7f8b7419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.728+0000 7f8b7a713700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7419e830 con 0x7f8b74100f50 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.728+0000 7f8b7a713700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7419e9a0 con 0x7f8b741018f0 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.728+0000 7f8b7a713700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7419eb60 con 0x7f8b740ffd60 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b73fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b740ffd60 0x7f8b74196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b74100f50 0x7f8b741972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b73fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b740ffd60 0x7f8b74196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:44728/0 (socket says 172.21.15.33:44728) 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b73fff700 1 -- 172.21.15.33:0/2878711674 learned_addr learned my addr 172.21.15.33:0/2878711674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b78cb0700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b741018f0 0x7f8b7419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b73fff700 1 -- 172.21.15.33:0/2878711674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b741018f0 msgr2=0x7f8b7419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b73fff700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b741018f0 0x7f8b7419e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b73fff700 1 -- 172.21.15.33:0/2878711674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b74100f50 msgr2=0x7f8b741972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:22.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b73fff700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b74100f50 0x7f8b741972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b73fff700 1 -- 172.21.15.33:0/2878711674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b680095f0 con 0x7f8b740ffd60 2024-05-08T21:28:22.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b73fff700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b740ffd60 0x7f8b74196d60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400e970 tx=0x7f8b6400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:22.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b717fa700 1 -- 172.21.15.33:0/2878711674 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64009790 con 0x7f8b740ffd60 2024-05-08T21:28:22.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b717fa700 1 -- 172.21.15.33:0/2878711674 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b6400c7e0 con 0x7f8b740ffd60 2024-05-08T21:28:22.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b717fa700 1 -- 172.21.15.33:0/2878711674 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64003bd0 con 0x7f8b740ffd60 2024-05-08T21:28:22.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.729+0000 7f8b7a713700 1 -- 172.21.15.33:0/2878711674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7419ee40 con 0x7f8b740ffd60 2024-05-08T21:28:22.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.730+0000 7f8b7a713700 1 -- 172.21.15.33:0/2878711674 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7419f210 con 0x7f8b740ffd60 2024-05-08T21:28:22.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.732+0000 7f8b717fa700 1 -- 172.21.15.33:0/2878711674 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8b6400c950 con 0x7f8b740ffd60 2024-05-08T21:28:22.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.732+0000 7f8b717fa700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8b5c0a1120 0x7f8b5c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.733+0000 7f8b737fe700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8b5c0a1120 0x7f8b5c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.733+0000 7f8b717fa700 1 -- 172.21.15.33:0/2878711674 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8b64014070 con 0x7f8b740ffd60 2024-05-08T21:28:22.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.733+0000 7f8b737fe700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8b5c0a1120 0x7f8b5c0a35e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8b68006010 tx=0x7f8b68003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:22.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b868cdb80 1 Processor -- start 2024-05-08T21:28:22.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b868cdb80 1 -- start start 2024-05-08T21:28:22.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b868cdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 0x55be460f6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b868cdb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6ea0 0x55be460f7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b868cdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f7860 0x55be460f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b868cdb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4604d150 con 0x55be460f6ea0 2024-05-08T21:28:22.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b868cdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4604ab10 con 0x55be460f7860 2024-05-08T21:28:22.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b868cdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be46046400 con 0x55be460f6540 2024-05-08T21:28:22.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6ea0 0x55be460f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6ea0 0x55be460f7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44612/0 (socket says 172.21.15.33:44612) 2024-05-08T21:28:22.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b737fe700 1 -- 172.21.15.33:0/3211025866 learned_addr learned my addr 172.21.15.33:0/3211025866 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:22.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b73fff700 1 --2- 172.21.15.33:0/3211025866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 0x55be460f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.734+0000 7f8b78cb0700 1 --2- 172.21.15.33:0/3211025866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f7860 0x55be460f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.735+0000 7f8b73fff700 1 -- 172.21.15.33:0/3211025866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f7860 msgr2=0x55be460f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:22.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.735+0000 7f8b73fff700 1 --2- 172.21.15.33:0/3211025866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f7860 0x55be460f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.735+0000 7f8b73fff700 1 -- 172.21.15.33:0/3211025866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6ea0 msgr2=0x55be460f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:22.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.735+0000 7f8b73fff700 1 --2- 172.21.15.33:0/3211025866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6ea0 0x55be460f7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.735+0000 7f8b73fff700 1 -- 172.21.15.33:0/3211025866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be45ec73b0 con 0x55be460f6540 2024-05-08T21:28:22.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.735+0000 7f8b737fe700 1 --2- 172.21.15.33:0/3211025866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6ea0 0x55be460f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:22.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.735+0000 7f8b73fff700 1 --2- 172.21.15.33:0/3211025866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 0x55be460f6960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8b640ca550 tx=0x7f8b64099830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ebfe91b37c3eca6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:22.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.735+0000 7f8b5a7fc700 1 -- 172.21.15.33:0/3211025866 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b640b1070 con 0x55be460f6540 2024-05-08T21:28:22.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.735+0000 7f8b5a7fc700 1 -- 172.21.15.33:0/3211025866 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b6409a5d0 con 0x55be460f6540 2024-05-08T21:28:22.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.735+0000 7f8b5a7fc700 1 -- 172.21.15.33:0/3211025866 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6401fde0 con 0x55be460f6540 2024-05-08T21:28:22.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.736+0000 7f8b868cdb80 1 -- 172.21.15.33:0/3211025866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 msgr2=0x55be460f6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:22.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.736+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/3211025866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 0x55be460f6960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8b640ca550 tx=0x7f8b64099830 comp rx=0 tx=0).stop 2024-05-08T21:28:22.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.736+0000 7f8b868cdb80 1 -- 172.21.15.33:0/3211025866 shutdown_connections 2024-05-08T21:28:22.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.736+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/3211025866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f7860 0x55be460f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.736+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/3211025866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 0x55be460f6960 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.736+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/3211025866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6ea0 0x55be460f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.736+0000 7f8b868cdb80 1 -- 172.21.15.33:0/3211025866 >> 172.21.15.33:0/3211025866 conn(0x55be46054210 msgr2=0x55be460f0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:22.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.736+0000 7f8b868cdb80 1 -- 172.21.15.33:0/3211025866 shutdown_connections 2024-05-08T21:28:22.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.736+0000 7f8b868cdb80 1 -- 172.21.15.33:0/3211025866 wait complete. 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.736+0000 7f8b868cdb80 1 -- start start 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b868cdb80 1 Processor -- start 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b868cdb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 0x55be460ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b868cdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be46100380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b868cdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f7860 0x55be461073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b868cdb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be45fe2560 con 0x55be460f6540 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b868cdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be45fe2200 con 0x55be460f6ea0 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b868cdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be45ec96a0 con 0x55be460f7860 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b868cdb80 10 client.0 _ll_register_callbacks cb 0x55be45febb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b73fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 0x55be460ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b78cb0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f7860 0x55be461073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b73fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 0x55be460ffe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44626/0 (socket says 172.21.15.33:44626) 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b73fff700 1 -- 172.21.15.33:0/1549271496 learned_addr learned my addr 172.21.15.33:0/1549271496 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:22.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b737fe700 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be46100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b78cb0700 1 -- 172.21.15.33:0/1549271496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 msgr2=0x55be46100380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b78cb0700 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be46100380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b78cb0700 1 -- 172.21.15.33:0/1549271496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 msgr2=0x55be460ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b78cb0700 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 0x55be460ffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b78cb0700 1 -- 172.21.15.33:0/1549271496 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b74108af0 con 0x55be460f7860 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.737+0000 7f8b73fff700 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 0x55be460ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b78cb0700 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f7860 0x55be461073c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55be460f1d40 tx=0x7f8b6c014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4763e34b40306481 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b537fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b537fe700 1 -- 172.21.15.33:0/1549271496 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c00e040 con 0x55be460f7860 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b537fe700 1 -- 172.21.15.33:0/1549271496 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8b6c003a00 con 0x55be460f7860 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1549271496 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be45f9f270 con 0x55be460f7860 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b537fe700 1 -- 172.21.15.33:0/1549271496 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b6c003bc0 con 0x55be460f7860 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b868cdb80 10 client.27811 fetch_fsmap learned FSMap version 45 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1549271496 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f8b7419f210 con 0x55be460f7860 2024-05-08T21:28:22.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b537fe700 1 -- 172.21.15.33:0/1549271496 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55be45f9f270 con 0x55be460f7860 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b537fe700 1 -- 172.21.15.33:0/1549271496 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 45) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f8b6c01c070 con 0x55be460f7860 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b868cdb80 10 client.27811 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b868cdb80 10 client.27811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.738+0000 7f8b868cdb80 20 client.27811 populate_metadata read hostname 'smithi033' 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.739+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55be4610bcb0 0x55be4610e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.739+0000 7f8b868cdb80 4 client.27811 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.739+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1549271496 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be460c5810 con 0x55be4610bcb0 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.739+0000 7f8b73fff700 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55be4610bcb0 0x55be4610e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.739+0000 7f8b73fff700 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55be4610bcb0 0x55be4610e0d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb62274fe4801a86 server_cookie=5c9d65fe50b21caa in_seq=0 out_seq=0 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.739+0000 7f8b537fe700 10 client.27811 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b537fe700 1 -- 172.21.15.33:0/1549271496 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be460c5810 con 0x55be4610bcb0 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b537fe700 10 client.27811 handle_command_reply: tid=0 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b868cdb80 1 client.27811 shutdown 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1549271496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f7860 msgr2=0x55be461073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:22.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f7860 0x55be461073c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55be460f1d40 tx=0x7f8b6c014600 comp rx=0 tx=0).stop 2024-05-08T21:28:22.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1549271496 shutdown_connections 2024-05-08T21:28:22.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55be4610bcb0 0x55be4610e0d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f7860 0x55be461073c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 0x55be460ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b537fe700 0 client.27811 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:28:22.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/1549271496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be46100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1549271496 >> 172.21.15.33:0/1549271496 conn(0x55be46054210 msgr2=0x55be460f05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:22.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1549271496 shutdown_connections 2024-05-08T21:28:22.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1549271496 wait complete. 2024-05-08T21:28:22.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.740+0000 7f8b868cdb80 20 client.27811 trim_cache size 0 max 16384 2024-05-08T21:28:22.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b868cdb80 1 Processor -- start 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b868cdb80 1 -- start start 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b868cdb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 0x55be460f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b868cdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be460f1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b868cdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f7860 0x55be46107380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b868cdb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be460eb800 con 0x55be460f6540 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b868cdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be461132c0 con 0x55be460f6ea0 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b868cdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be460fdac0 con 0x55be460f7860 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b73fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 0x55be460f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b73fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 0x55be460f0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44630/0 (socket says 172.21.15.33:44630) 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b73fff700 1 -- 172.21.15.33:0/3464801402 learned_addr learned my addr 172.21.15.33:0/3464801402 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b737fe700 1 --2- 172.21.15.33:0/3464801402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be460f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.761+0000 7f8b78cb0700 1 --2- 172.21.15.33:0/3464801402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f7860 0x55be46107380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.762+0000 7f8b73fff700 1 -- 172.21.15.33:0/3464801402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f7860 msgr2=0x55be46107380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.762+0000 7f8b73fff700 1 --2- 172.21.15.33:0/3464801402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f7860 0x55be46107380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.762+0000 7f8b73fff700 1 -- 172.21.15.33:0/3464801402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 msgr2=0x55be460f1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.762+0000 7f8b73fff700 1 --2- 172.21.15.33:0/3464801402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be460f1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.762+0000 7f8b73fff700 1 -- 172.21.15.33:0/3464801402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be45e73b60 con 0x55be460f6540 2024-05-08T21:28:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.762+0000 7f8b737fe700 1 --2- 172.21.15.33:0/3464801402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be460f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.762+0000 7f8b78cb0700 1 --2- 172.21.15.33:0/3464801402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f7860 0x55be46107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.762+0000 7f8b73fff700 1 --2- 172.21.15.33:0/3464801402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 0x55be460f0cd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8b64005950 tx=0x7f8b6409a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2bc318a77dfaeb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.762+0000 7f8b5affd700 1 -- 172.21.15.33:0/3464801402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b640b1040 con 0x55be460f6540 2024-05-08T21:28:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.762+0000 7f8b5affd700 1 -- 172.21.15.33:0/3464801402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b64096210 con 0x55be460f6540 2024-05-08T21:28:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.762+0000 7f8b5affd700 1 -- 172.21.15.33:0/3464801402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b640b80d0 con 0x55be460f6540 2024-05-08T21:28:22.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.763+0000 7f8b868cdb80 1 -- 172.21.15.33:0/3464801402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 msgr2=0x55be460f0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:22.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.763+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/3464801402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 0x55be460f0cd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8b64005950 tx=0x7f8b6409a740 comp rx=0 tx=0).stop 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.763+0000 7f8b868cdb80 1 -- 172.21.15.33:0/3464801402 shutdown_connections 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.763+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/3464801402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f7860 0x55be46107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.763+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/3464801402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f6540 0x55be460f0cd0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.763+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/3464801402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be460f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.763+0000 7f8b868cdb80 1 -- 172.21.15.33:0/3464801402 >> 172.21.15.33:0/3464801402 conn(0x55be46054210 msgr2=0x55be460fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.763+0000 7f8b868cdb80 1 -- 172.21.15.33:0/3464801402 shutdown_connections 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.763+0000 7f8b868cdb80 1 -- 172.21.15.33:0/3464801402 wait complete. 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.763+0000 7f8b868cdb80 1 -- start start 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b868cdb80 1 Processor -- start 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b868cdb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 0x55be46103400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b868cdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be46103940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b868cdb80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f7860 0x55be461067d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b868cdb80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be46106d10 con 0x55be460f7860 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b868cdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be46106e80 con 0x55be460f6ea0 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b868cdb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be46106ff0 con 0x55be460f6540 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b868cdb80 10 client.0 _ll_register_callbacks cb 0x55be45febb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b73fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 0x55be46103400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b737fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be46103940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b78cb0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f7860 0x55be461067d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b73fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 0x55be46103400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:44746/0 (socket says 172.21.15.33:44746) 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.764+0000 7f8b73fff700 1 -- 172.21.15.33:0/1358818087 learned_addr learned my addr 172.21.15.33:0/1358818087 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b73fff700 1 -- 172.21.15.33:0/1358818087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 msgr2=0x55be46103940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b73fff700 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be46103940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b73fff700 1 -- 172.21.15.33:0/1358818087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f7860 msgr2=0x55be461067d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b73fff700 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f7860 0x55be461067d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b73fff700 1 -- 172.21.15.33:0/1358818087 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6800bcd0 con 0x55be460f6540 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b737fe700 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be46103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:22.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b78cb0700 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f7860 0x55be461067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b73fff700 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 0x55be46103400 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8b640968f0 tx=0x7f8b640a23f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed33a93e008fba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b537fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b537fe700 1 -- 172.21.15.33:0/1358818087 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b640b1070 con 0x55be460f6540 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b537fe700 1 -- 172.21.15.33:0/1358818087 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8b640b80d0 con 0x55be460f6540 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1358818087 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b6409a5d0 con 0x55be460f6540 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b537fe700 1 -- 172.21.15.33:0/1358818087 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b640b82d0 con 0x55be460f6540 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b868cdb80 10 client.27817 fetch_fsmap learned FSMap version 45 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1358818087 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f8b7419f210 con 0x55be460f6540 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b537fe700 1 -- 172.21.15.33:0/1358818087 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b6409a5d0 con 0x55be460f6540 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.765+0000 7f8b537fe700 1 -- 172.21.15.33:0/1358818087 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 45) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f8b64094ae0 con 0x55be460f6540 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.766+0000 7f8b868cdb80 10 client.27817 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.766+0000 7f8b868cdb80 10 client.27817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.766+0000 7f8b868cdb80 20 client.27817 populate_metadata read hostname 'smithi033' 2024-05-08T21:28:22.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.766+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55be46162df0 0x55be461651e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.766+0000 7f8b868cdb80 4 client.27817 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.766+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1358818087 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55be4603eec0 con 0x55be46162df0 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.766+0000 7f8b737fe700 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55be46162df0 0x55be461651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.766+0000 7f8b537fe700 10 client.27817 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.766+0000 7f8b737fe700 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55be46162df0 0x55be461651e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=456fa249b251d391 server_cookie=e24a3efee7b62b51 in_seq=0 out_seq=0 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.766+0000 7f8b537fe700 1 -- 172.21.15.33:0/1358818087 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55be4603eec0 con 0x55be46162df0 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.766+0000 7f8b537fe700 10 client.27817 handle_command_reply: tid=0 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.766+0000 7f8b868cdb80 1 client.27817 shutdown 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1358818087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 msgr2=0x55be46103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 0x55be46103400 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8b640968f0 tx=0x7f8b640a23f0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1358818087 shutdown_connections 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55be46162df0 0x55be461651e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be460f7860 0x55be461067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b537fe700 0 client.27817 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:28:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be460f6540 0x55be46103400 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b868cdb80 1 --2- 172.21.15.33:0/1358818087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be460f6ea0 0x55be46103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1358818087 >> 172.21.15.33:0/1358818087 conn(0x55be46054210 msgr2=0x55be460fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:22.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1358818087 shutdown_connections 2024-05-08T21:28:22.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b868cdb80 1 -- 172.21.15.33:0/1358818087 wait complete. 2024-05-08T21:28:22.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b868cdb80 20 client.27817 trim_cache size 0 max 16384 2024-05-08T21:28:22.774 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:28:22.774 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-05-08T21:28:22.774 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {} 2024-05-08T21:28:22.774 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:28:22.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b7a713700 1 -- 172.21.15.33:0/2878711674 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8b5c0a1120 msgr2=0x7f8b5c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:22.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b7a713700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8b5c0a1120 0x7f8b5c0a35e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8b68006010 tx=0x7f8b68003680 comp rx=0 tx=0).stop 2024-05-08T21:28:22.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b7a713700 1 -- 172.21.15.33:0/2878711674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b740ffd60 msgr2=0x7f8b74196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:22.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.767+0000 7f8b7a713700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b740ffd60 0x7f8b74196d60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400e970 tx=0x7f8b6400ec80 comp rx=0 tx=0).stop 2024-05-08T21:28:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.768+0000 7f8b7a713700 1 -- 172.21.15.33:0/2878711674 shutdown_connections 2024-05-08T21:28:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.768+0000 7f8b7a713700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8b5c0a1120 0x7f8b5c0a35e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.768+0000 7f8b7a713700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b741018f0 0x7f8b7419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.768+0000 7f8b7a713700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b740ffd60 0x7f8b74196d60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.768+0000 7f8b7a713700 1 --2- 172.21.15.33:0/2878711674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b74100f50 0x7f8b741972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.768+0000 7f8b7a713700 1 -- 172.21.15.33:0/2878711674 >> 172.21.15.33:0/2878711674 conn(0x7f8b740fb290 msgr2=0x7f8b74068210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.768+0000 7f8b7a713700 1 -- 172.21.15.33:0/2878711674 shutdown_connections 2024-05-08T21:28:22.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.768+0000 7f8b7a713700 1 -- 172.21.15.33:0/2878711674 wait complete. 2024-05-08T21:28:22.783 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-08T21:28:22.783 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:28:22.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.928+0000 7f854e099700 1 -- 172.21.15.33:0/863514916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85480ffd80 msgr2=0x7f85481001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:22.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.928+0000 7f854e099700 1 --2- 172.21.15.33:0/863514916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85480ffd80 0x7f85481001a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f85300099c0 tx=0x7f8530009cd0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.928+0000 7f854e099700 1 -- 172.21.15.33:0/863514916 shutdown_connections 2024-05-08T21:28:22.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.928+0000 7f854e099700 1 --2- 172.21.15.33:0/863514916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8548101910 0x7f85481081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.928+0000 7f854e099700 1 --2- 172.21.15.33:0/863514916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85480ffd80 0x7f85481001a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.928+0000 7f854e099700 1 --2- 172.21.15.33:0/863514916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8548100f70 0x7f85481013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.928+0000 7f854e099700 1 -- 172.21.15.33:0/863514916 >> 172.21.15.33:0/863514916 conn(0x7f85480fb310 msgr2=0x7f85480fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:22.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.928+0000 7f854e099700 1 -- 172.21.15.33:0/863514916 shutdown_connections 2024-05-08T21:28:22.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.928+0000 7f854e099700 1 -- 172.21.15.33:0/863514916 wait complete. 2024-05-08T21:28:22.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.929+0000 7f854e099700 1 Processor -- start 2024-05-08T21:28:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.929+0000 7f854e099700 1 -- start start 2024-05-08T21:28:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.929+0000 7f854e099700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85480ffd80 0x7f8548103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f854e099700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8548100f70 0x7f8548102590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f854e099700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8548101910 0x7f8548102ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f854e099700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8548103270 con 0x7f8548101910 2024-05-08T21:28:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f854e099700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85481033e0 con 0x7f85480ffd80 2024-05-08T21:28:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f854e099700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85481a0dc0 con 0x7f8548100f70 2024-05-08T21:28:22.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f85477fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85480ffd80 0x7f8548103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f8547fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8548101910 0x7f8548102ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f85477fe700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85480ffd80 0x7f8548103f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:52918/0 (socket says 172.21.15.33:52918) 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f85477fe700 1 -- 172.21.15.33:0/3871440663 learned_addr learned my addr 172.21.15.33:0/3871440663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f8546ffd700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8548100f70 0x7f8548102590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f85477fe700 1 -- 172.21.15.33:0/3871440663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8548100f70 msgr2=0x7f8548102590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f85477fe700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8548100f70 0x7f8548102590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f85477fe700 1 -- 172.21.15.33:0/3871440663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8548101910 msgr2=0x7f8548102ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f85477fe700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8548101910 0x7f8548102ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f85477fe700 1 -- 172.21.15.33:0/3871440663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8540009c40 con 0x7f85480ffd80 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f8546ffd700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8548100f70 0x7f8548102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f85477fe700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85480ffd80 0x7f8548103f00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8530000c00 tx=0x7f853000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.930+0000 7f8544ff9700 1 -- 172.21.15.33:0/3871440663 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853001d070 con 0x7f85480ffd80 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.931+0000 7f8544ff9700 1 -- 172.21.15.33:0/3871440663 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85300054c0 con 0x7f85480ffd80 2024-05-08T21:28:22.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.931+0000 7f854e099700 1 -- 172.21.15.33:0/3871440663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85300095f0 con 0x7f85480ffd80 2024-05-08T21:28:22.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.931+0000 7f8544ff9700 1 -- 172.21.15.33:0/3871440663 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530004a90 con 0x7f85480ffd80 2024-05-08T21:28:22.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.931+0000 7f854e099700 1 -- 172.21.15.33:0/3871440663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85481a11d0 con 0x7f85480ffd80 2024-05-08T21:28:22.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.932+0000 7f854e099700 1 -- 172.21.15.33:0/3871440663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8548067d80 con 0x7f85480ffd80 2024-05-08T21:28:22.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.933+0000 7f8544ff9700 1 -- 172.21.15.33:0/3871440663 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f853000fb80 con 0x7f85480ffd80 2024-05-08T21:28:22.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.934+0000 7f8544ff9700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f85340a1120 0x7f85340a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:22.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.934+0000 7f8546ffd700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f85340a1120 0x7f85340a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:22.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.934+0000 7f8544ff9700 1 -- 172.21.15.33:0/3871440663 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f85300cdbe0 con 0x7f85480ffd80 2024-05-08T21:28:22.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.934+0000 7f8546ffd700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f85340a1120 0x7f85340a35e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8538011490 tx=0x7f8538009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:22.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:22.939+0000 7f8544ff9700 1 -- 172.21.15.33:0/3871440663 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8530097040 con 0x7f85480ffd80 2024-05-08T21:28:23.049 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-05-08T21:28:23.049 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-08T21:28:23.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.221+0000 7f854e099700 1 -- 172.21.15.33:0/3871440663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f854810b6e0 con 0x7f85480ffd80 2024-05-08T21:28:23.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.222+0000 7f8544ff9700 1 -- 172.21.15.33:0/3871440663 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+10920 (secure 0 0 0) 0x7f853009a660 con 0x7f85480ffd80 2024-05-08T21:28:23.223 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:28:23.223 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:28:18.484549+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:28:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.226+0000 7f854e099700 1 -- 172.21.15.33:0/3871440663 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f85340a1120 msgr2=0x7f85340a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.226+0000 7f854e099700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f85340a1120 0x7f85340a35e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8538011490 tx=0x7f8538009260 comp rx=0 tx=0).stop 2024-05-08T21:28:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.226+0000 7f854e099700 1 -- 172.21.15.33:0/3871440663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85480ffd80 msgr2=0x7f8548103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.226+0000 7f854e099700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85480ffd80 0x7f8548103f00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8530000c00 tx=0x7f853000f690 comp rx=0 tx=0).stop 2024-05-08T21:28:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.226+0000 7f854e099700 1 -- 172.21.15.33:0/3871440663 shutdown_connections 2024-05-08T21:28:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.226+0000 7f854e099700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8548101910 0x7f8548102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.226+0000 7f854e099700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f85340a1120 0x7f85340a35e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.226+0000 7f854e099700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85480ffd80 0x7f8548103f00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.226+0000 7f854e099700 1 --2- 172.21.15.33:0/3871440663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8548100f70 0x7f8548102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.226+0000 7f854e099700 1 -- 172.21.15.33:0/3871440663 >> 172.21.15.33:0/3871440663 conn(0x7f85480fb310 msgr2=0x7f8548068160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.226+0000 7f854e099700 1 -- 172.21.15.33:0/3871440663 shutdown_connections 2024-05-08T21:28:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.226+0000 7f854e099700 1 -- 172.21.15.33:0/3871440663 wait complete. 2024-05-08T21:28:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 45 2024-05-08T21:28:23.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-05-08T21:28:23.363 DEBUG:teuthology.parallel:result is None 2024-05-08T21:28:23.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T21:28:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.387+0000 7f7c810ff700 1 -- 172.21.15.33:0/2155905350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c104310 msgr2=0x7f7c7c106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.387+0000 7f7c810ff700 1 --2- 172.21.15.33:0/2155905350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c104310 0x7f7c7c106700 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009990 tx=0x7f7c70009ca0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.387+0000 7f7c810ff700 1 -- 172.21.15.33:0/2155905350 shutdown_connections 2024-05-08T21:28:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.387+0000 7f7c810ff700 1 --2- 172.21.15.33:0/2155905350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c7c106c40 0x7f7c7c109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.387+0000 7f7c810ff700 1 --2- 172.21.15.33:0/2155905350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c7c1019e0 0x7f7c7c103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.387+0000 7f7c810ff700 1 --2- 172.21.15.33:0/2155905350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c104310 0x7f7c7c106700 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.387+0000 7f7c810ff700 1 -- 172.21.15.33:0/2155905350 >> 172.21.15.33:0/2155905350 conn(0x7f7c7c0fb330 msgr2=0x7f7c7c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.387+0000 7f7c810ff700 1 -- 172.21.15.33:0/2155905350 shutdown_connections 2024-05-08T21:28:23.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.387+0000 7f7c810ff700 1 -- 172.21.15.33:0/2155905350 wait complete. 2024-05-08T21:28:23.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c810ff700 1 Processor -- start 2024-05-08T21:28:23.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c810ff700 1 -- start start 2024-05-08T21:28:23.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c810ff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c7c1019e0 0x7f7c7c196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c810ff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c7c104310 0x7f7c7c197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c810ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c106c40 0x7f7c7c19e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c810ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c19e7e0 con 0x7f7c7c106c40 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c810ff700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c19e950 con 0x7f7c7c104310 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c810ff700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c19eb10 con 0x7f7c7c1019e0 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c7b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c106c40 0x7f7c7c19e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c7b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c106c40 0x7f7c7c19e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44662/0 (socket says 172.21.15.33:44662) 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c7b59e700 1 -- 172.21.15.33:0/2737324489 learned_addr learned my addr 172.21.15.33:0/2737324489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c7a59c700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c7c104310 0x7f7c7c197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.388+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c7c1019e0 0x7f7c7c196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.389+0000 7f7c7b59e700 1 -- 172.21.15.33:0/2737324489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c7c1019e0 msgr2=0x7f7c7c196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.389+0000 7f7c7b59e700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c7c1019e0 0x7f7c7c196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.389+0000 7f7c7b59e700 1 -- 172.21.15.33:0/2737324489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c7c104310 msgr2=0x7f7c7c197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.389+0000 7f7c7b59e700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c7c104310 0x7f7c7c197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.389+0000 7f7c7b59e700 1 -- 172.21.15.33:0/2737324489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c700095f0 con 0x7f7c7c106c40 2024-05-08T21:28:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.389+0000 7f7c7b59e700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c106c40 0x7f7c7c19e210 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400ef30 tx=0x7f7c7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.389+0000 7f7c63fff700 1 -- 172.21.15.33:0/2737324489 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74018070 con 0x7f7c7c106c40 2024-05-08T21:28:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.389+0000 7f7c63fff700 1 -- 172.21.15.33:0/2737324489 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c7400f040 con 0x7f7c7c106c40 2024-05-08T21:28:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.389+0000 7f7c810ff700 1 -- 172.21.15.33:0/2737324489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c7c19ed90 con 0x7f7c7c106c40 2024-05-08T21:28:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.389+0000 7f7c63fff700 1 -- 172.21.15.33:0/2737324489 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7400cbc0 con 0x7f7c7c106c40 2024-05-08T21:28:23.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.389+0000 7f7c810ff700 1 -- 172.21.15.33:0/2737324489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7c19f1c0 con 0x7f7c7c106c40 2024-05-08T21:28:23.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.392+0000 7f7c63fff700 1 -- 172.21.15.33:0/2737324489 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7c7400cd80 con 0x7f7c7c106c40 2024-05-08T21:28:23.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.392+0000 7f7c63fff700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c640a1120 0x7f7c640a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.393+0000 7f7c63fff700 1 -- 172.21.15.33:0/2737324489 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7c740cc760 con 0x7f7c7c106c40 2024-05-08T21:28:23.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.393+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c640a1120 0x7f7c640a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.393+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c640a1120 0x7f7c640a35e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c103150 tx=0x7f7c6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.393+0000 7f7c8d2b9b80 1 Processor -- start 2024-05-08T21:28:23.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.393+0000 7f7c8d2b9b80 1 -- start start 2024-05-08T21:28:23.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c8d2b9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574540 0x55e35b574960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c8d2b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 0x55e35b575320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c8d2b9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b575860 0x55e35b575ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c8d2b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35b4cb150 con 0x55e35b574ea0 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c8d2b9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35b4c8b10 con 0x55e35b574540 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c8d2b9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35b4c4400 con 0x55e35b575860 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c7a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 0x55e35b575320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c7ad9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574540 0x55e35b574960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c7a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 0x55e35b575320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44664/0 (socket says 172.21.15.33:44664) 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c7ad9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574540 0x55e35b574960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:52944/0 (socket says 172.21.15.33:52944) 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c7a59c700 1 -- 172.21.15.33:0/1303398353 learned_addr learned my addr 172.21.15.33:0/1303398353 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c7b59e700 1 --2- 172.21.15.33:0/1303398353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b575860 0x55e35b575ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c7ad9d700 1 -- 172.21.15.33:0/1303398353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b575860 msgr2=0x55e35b575ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/1303398353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b575860 0x55e35b575ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c7ad9d700 1 -- 172.21.15.33:0/1303398353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 msgr2=0x55e35b575320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/1303398353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 0x55e35b575320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c7ad9d700 1 -- 172.21.15.33:0/1303398353 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e35b3453b0 con 0x55e35b574540 2024-05-08T21:28:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.394+0000 7f7c7a59c700 1 --2- 172.21.15.33:0/1303398353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 0x55e35b575320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.395+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/1303398353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574540 0x55e35b574960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c011cc0 tx=0x7f7c6c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc0297bb2c68a356 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.395+0000 7f7c617fa700 1 -- 172.21.15.33:0/1303398353 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c004230 con 0x55e35b574540 2024-05-08T21:28:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.395+0000 7f7c617fa700 1 -- 172.21.15.33:0/1303398353 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c6c021070 con 0x55e35b574540 2024-05-08T21:28:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.395+0000 7f7c617fa700 1 -- 172.21.15.33:0/1303398353 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00ec70 con 0x55e35b574540 2024-05-08T21:28:23.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.396+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1303398353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574540 msgr2=0x55e35b574960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.396+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/1303398353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574540 0x55e35b574960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c011cc0 tx=0x7f7c6c014000 comp rx=0 tx=0).stop 2024-05-08T21:28:23.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.396+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1303398353 shutdown_connections 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.396+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/1303398353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b575860 0x55e35b575ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.396+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/1303398353 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574540 0x55e35b574960 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.396+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/1303398353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 0x55e35b575320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.396+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1303398353 >> 172.21.15.33:0/1303398353 conn(0x55e35b4d2210 msgr2=0x55e35b56ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.396+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1303398353 shutdown_connections 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.396+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1303398353 wait complete. 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.396+0000 7f7c8d2b9b80 1 -- start start 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.396+0000 7f7c8d2b9b80 1 Processor -- start 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c8d2b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574540 0x55e35b57de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c8d2b9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574ea0 0x55e35b57e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c8d2b9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b575860 0x55e35b585390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c8d2b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35b460560 con 0x55e35b574540 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c8d2b9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35b460200 con 0x55e35b575860 2024-05-08T21:28:23.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c8d2b9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35b3476a0 con 0x55e35b574ea0 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c8d2b9b80 10 client.0 _ll_register_callbacks cb 0x55e35b469b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c7ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574540 0x55e35b57de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c7ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574540 0x55e35b57de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44678/0 (socket says 172.21.15.33:44678) 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c7ad9d700 1 -- 172.21.15.33:0/1683353986 learned_addr learned my addr 172.21.15.33:0/1683353986 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c7a59c700 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574ea0 0x55e35b57e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c7b59e700 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b575860 0x55e35b585390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c7a59c700 1 -- 172.21.15.33:0/1683353986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b575860 msgr2=0x55e35b585390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c7a59c700 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b575860 0x55e35b585390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c7a59c700 1 -- 172.21.15.33:0/1683353986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574540 msgr2=0x55e35b57de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c7a59c700 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574540 0x55e35b57de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c7a59c700 1 -- 172.21.15.33:0/1683353986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7c109c00 con 0x55e35b574ea0 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.397+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574540 0x55e35b57de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c7a59c700 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574ea0 0x55e35b57e350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7c70003f60 tx=0x7f7c700053b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cf6930b8c8604d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c5eff5700 1 -- 172.21.15.33:0/1683353986 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7001d070 con 0x55e35b574ea0 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1683353986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e35b41d270 con 0x55e35b574ea0 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c5eff5700 1 -- 172.21.15.33:0/1683353986 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7c70003a90 con 0x55e35b574ea0 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c5eff5700 1 -- 172.21.15.33:0/1683353986 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c70003c50 con 0x55e35b574ea0 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c8d2b9b80 10 client.27835 fetch_fsmap learned FSMap version 45 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1683353986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f7c6c017070 con 0x55e35b574ea0 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c5eff5700 1 -- 172.21.15.33:0/1683353986 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e35b41d270 con 0x55e35b574ea0 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c5eff5700 1 -- 172.21.15.33:0/1683353986 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 45) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f7c70004690 con 0x55e35b574ea0 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c8d2b9b80 10 client.27835 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c8d2b9b80 10 client.27835 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c8d2b9b80 20 client.27835 populate_metadata read hostname 'smithi033' 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e35b589c10 0x55e35b58c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c8d2b9b80 4 client.27835 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.398+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1683353986 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e35b543810 con 0x55e35b589c10 2024-05-08T21:28:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.399+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e35b589c10 0x55e35b58c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.399+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e35b589c10 0x55e35b58c030 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f8f4f8d052266f46 server_cookie=b59072f76698a979 in_seq=0 out_seq=0 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.399+0000 7f7c5eff5700 10 client.27835 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c5eff5700 1 -- 172.21.15.33:0/1683353986 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e35b543810 con 0x55e35b589c10 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c5eff5700 10 client.27835 handle_command_reply: tid=0 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c8d2b9b80 1 client.27835 shutdown 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1683353986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574ea0 msgr2=0x55e35b57e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574ea0 0x55e35b57e350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7c70003f60 tx=0x7f7c700053b0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1683353986 shutdown_connections 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b575860 0x55e35b585390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574540 0x55e35b57de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e35b589c10 0x55e35b58c030 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/1683353986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574ea0 0x55e35b57e350 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1683353986 >> 172.21.15.33:0/1683353986 conn(0x55e35b4d2210 msgr2=0x55e35b56e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c5eff5700 0 client.27835 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:28:23.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1683353986 shutdown_connections 2024-05-08T21:28:23.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/1683353986 wait complete. 2024-05-08T21:28:23.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.400+0000 7f7c8d2b9b80 20 client.27835 trim_cache size 0 max 16384 2024-05-08T21:28:23.407 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T21:28:23.408 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-08T21:28:23.419 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-08T21:28:23.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.414+0000 7f7c8d2b9b80 1 Processor -- start 2024-05-08T21:28:23.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.414+0000 7f7c8d2b9b80 1 -- start start 2024-05-08T21:28:23.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.414+0000 7f7c8d2b9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574540 0x55e35b58c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.414+0000 7f7c8d2b9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574ea0 0x55e35b5852b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.414+0000 7f7c8d2b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b575860 0x55e35b57d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.414+0000 7f7c8d2b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35b56bdc0 con 0x55e35b575860 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.414+0000 7f7c8d2b9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35b570570 con 0x55e35b574ea0 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.414+0000 7f7c8d2b9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35b569800 con 0x55e35b574540 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.414+0000 7f7c7b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b575860 0x55e35b57d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.414+0000 7f7c7b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b575860 0x55e35b57d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44680/0 (socket says 172.21.15.33:44680) 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.414+0000 7f7c7b59e700 1 -- 172.21.15.33:0/235539608 learned_addr learned my addr 172.21.15.33:0/235539608 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.415+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/235539608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574540 0x55e35b58c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.415+0000 7f7c7a59c700 1 --2- 172.21.15.33:0/235539608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574ea0 0x55e35b5852b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.415+0000 7f7c7b59e700 1 -- 172.21.15.33:0/235539608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574540 msgr2=0x55e35b58c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.415+0000 7f7c7b59e700 1 --2- 172.21.15.33:0/235539608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574540 0x55e35b58c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.415+0000 7f7c7b59e700 1 -- 172.21.15.33:0/235539608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574ea0 msgr2=0x55e35b5852b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.415+0000 7f7c7b59e700 1 --2- 172.21.15.33:0/235539608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574ea0 0x55e35b5852b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.415+0000 7f7c7b59e700 1 -- 172.21.15.33:0/235539608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e35b2f1b60 con 0x55e35b575860 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.415+0000 7f7c7a59c700 1 --2- 172.21.15.33:0/235539608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574ea0 0x55e35b5852b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.415+0000 7f7c7b59e700 1 --2- 172.21.15.33:0/235539608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b575860 0x55e35b57d320 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7c740995c0 tx=0x7f7c74095d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1f4886d72acbd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.415+0000 7f7c61ffb700 1 -- 172.21.15.33:0/235539608 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74099fc0 con 0x55e35b575860 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.415+0000 7f7c61ffb700 1 -- 172.21.15.33:0/235539608 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c740998c0 con 0x55e35b575860 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.415+0000 7f7c61ffb700 1 -- 172.21.15.33:0/235539608 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74097a90 con 0x55e35b575860 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.416+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/235539608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b575860 msgr2=0x55e35b57d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.416+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/235539608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b575860 0x55e35b57d320 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7c740995c0 tx=0x7f7c74095d30 comp rx=0 tx=0).stop 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.416+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/235539608 shutdown_connections 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.416+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/235539608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b575860 0x55e35b57d320 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.416+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/235539608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574540 0x55e35b58c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.416+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/235539608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b574ea0 0x55e35b5852b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.416+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/235539608 >> 172.21.15.33:0/235539608 conn(0x55e35b4d2210 msgr2=0x55e35b575c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.416+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/235539608 shutdown_connections 2024-05-08T21:28:23.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.416+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/235539608 wait complete. 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.416+0000 7f7c8d2b9b80 1 -- start start 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.416+0000 7f7c8d2b9b80 1 Processor -- start 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c8d2b9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574540 0x55e35b580910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c8d2b9b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 0x55e35b580e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c8d2b9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b575860 0x55e35b583e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c8d2b9b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35b5843b0 con 0x55e35b574ea0 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c8d2b9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35b584520 con 0x55e35b575860 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c8d2b9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e35b5846e0 con 0x55e35b574540 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c8d2b9b80 10 client.0 _ll_register_callbacks cb 0x55e35b469b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 0x55e35b580e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574540 0x55e35b580910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 0x55e35b580e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44684/0 (socket says 172.21.15.33:44684) 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574540 0x55e35b580910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:44816/0 (socket says 172.21.15.33:44816) 2024-05-08T21:28:23.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7a59c700 1 -- 172.21.15.33:0/125524708 learned_addr learned my addr 172.21.15.33:0/125524708 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7b59e700 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b575860 0x55e35b583e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7a59c700 1 -- 172.21.15.33:0/125524708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574540 msgr2=0x55e35b580910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7a59c700 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574540 0x55e35b580910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7a59c700 1 -- 172.21.15.33:0/125524708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b575860 msgr2=0x55e35b583e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7a59c700 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b575860 0x55e35b583e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7a59c700 1 -- 172.21.15.33:0/125524708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7c19f1c0 con 0x55e35b574ea0 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574540 0x55e35b580910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c7a59c700 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 0x55e35b580e50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7c70005150 tx=0x7f7c70005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a95514077e9e63 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c5eff5700 1 -- 172.21.15.33:0/125524708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7001d040 con 0x55e35b574ea0 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c5eff5700 1 -- 172.21.15.33:0/125524708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f7c70004d10 con 0x55e35b574ea0 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.417+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/125524708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c6c021070 con 0x55e35b574ea0 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.418+0000 7f7c5eff5700 1 -- 172.21.15.33:0/125524708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c70004710 con 0x55e35b574ea0 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.418+0000 7f7c8d2b9b80 10 client.17925 fetch_fsmap learned FSMap version 45 2024-05-08T21:28:23.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.418+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/125524708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55e35b48a560 con 0x55e35b574ea0 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.418+0000 7f7c5eff5700 1 -- 172.21.15.33:0/125524708 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c6c021070 con 0x55e35b574ea0 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.418+0000 7f7c5eff5700 1 -- 172.21.15.33:0/125524708 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f7c700099c0 con 0x55e35b574ea0 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.418+0000 7f7c8d2b9b80 10 client.17925 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.418+0000 7f7c8d2b9b80 10 client.17925 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.418+0000 7f7c8d2b9b80 20 client.17925 populate_metadata read hostname 'smithi033' 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.418+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e35b5e05e0 0x55e35b5e2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.418+0000 7f7c8d2b9b80 4 client.17925 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.418+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/125524708 --> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e35b4bcec0 con 0x55e35b5e05e0 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.418+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e35b5e05e0 0x55e35b5e2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c5eff5700 10 client.17925 ms_handle_connect on v2:172.21.15.82:6832/3382287612 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c7ad9d700 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e35b5e05e0 0x55e35b5e2a00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=220bfe2e7a89eb2a server_cookie=7fabd11bd3ad914c in_seq=0 out_seq=0 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c5eff5700 1 -- 172.21.15.33:0/125524708 <== mds.4 v2:172.21.15.82:6832/3382287612 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e35b4bcec0 con 0x55e35b5e05e0 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c5eff5700 10 client.17925 handle_command_reply: tid=0 2024-05-08T21:28:23.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c8d2b9b80 1 client.17925 shutdown 2024-05-08T21:28:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/125524708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 msgr2=0x55e35b580e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 0x55e35b580e50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7c70005150 tx=0x7f7c70005f10 comp rx=0 tx=0).stop 2024-05-08T21:28:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/125524708 shutdown_connections 2024-05-08T21:28:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e35b575860 0x55e35b583e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.82:6832/3382287612,v1:172.21.15.82:6833/3382287612] conn(0x55e35b5e05e0 0x55e35b5e2a00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e35b574540 0x55e35b580910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c8d2b9b80 1 --2- 172.21.15.33:0/125524708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e35b574ea0 0x55e35b580e50 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c5eff5700 0 client.17925 ms_handle_reset on v2:172.21.15.82:6832/3382287612 2024-05-08T21:28:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/125524708 >> 172.21.15.33:0/125524708 conn(0x55e35b4d2210 msgr2=0x55e35b57d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/125524708 shutdown_connections 2024-05-08T21:28:23.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c8d2b9b80 1 -- 172.21.15.33:0/125524708 wait complete. 2024-05-08T21:28:23.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.419+0000 7f7c8d2b9b80 20 client.17925 trim_cache size 0 max 16384 2024-05-08T21:28:23.432 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:28:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.420+0000 7f7c810ff700 1 -- 172.21.15.33:0/2737324489 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c640a1120 msgr2=0x7f7c640a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.420+0000 7f7c810ff700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c640a1120 0x7f7c640a35e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c103150 tx=0x7f7c6c009260 comp rx=0 tx=0).stop 2024-05-08T21:28:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.420+0000 7f7c810ff700 1 -- 172.21.15.33:0/2737324489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c106c40 msgr2=0x7f7c7c19e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.420+0000 7f7c810ff700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c106c40 0x7f7c7c19e210 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400ef30 tx=0x7f7c7400c3a0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.420+0000 7f7c810ff700 1 -- 172.21.15.33:0/2737324489 shutdown_connections 2024-05-08T21:28:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.420+0000 7f7c810ff700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c7c106c40 0x7f7c7c19e210 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.420+0000 7f7c810ff700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c7c1019e0 0x7f7c7c196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.420+0000 7f7c810ff700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7c640a1120 0x7f7c640a35e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.420+0000 7f7c810ff700 1 --2- 172.21.15.33:0/2737324489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c7c104310 0x7f7c7c197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.420+0000 7f7c810ff700 1 -- 172.21.15.33:0/2737324489 >> 172.21.15.33:0/2737324489 conn(0x7f7c7c0fb330 msgr2=0x7f7c7c103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.420+0000 7f7c810ff700 1 -- 172.21.15.33:0/2737324489 shutdown_connections 2024-05-08T21:28:23.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.420+0000 7f7c810ff700 1 -- 172.21.15.33:0/2737324489 wait complete. 2024-05-08T21:28:23.435 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-05-08T21:28:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.617+0000 7f09941e3700 1 -- 172.21.15.33:0/1715747374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f098c0ff6d0 msgr2=0x7f098c0ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.617+0000 7f09941e3700 1 --2- 172.21.15.33:0/1715747374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f098c0ff6d0 0x7f098c0ffaf0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0988009990 tx=0x7f0988009ca0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.617+0000 7f09941e3700 1 -- 172.21.15.33:0/1715747374 shutdown_connections 2024-05-08T21:28:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.617+0000 7f09941e3700 1 --2- 172.21.15.33:0/1715747374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c1009f0 0x7f098c100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.617+0000 7f09941e3700 1 --2- 172.21.15.33:0/1715747374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f098c0ff6d0 0x7f098c0ffaf0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.617+0000 7f09941e3700 1 --2- 172.21.15.33:0/1715747374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c100030 0x7f098c1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.617+0000 7f09941e3700 1 -- 172.21.15.33:0/1715747374 >> 172.21.15.33:0/1715747374 conn(0x7f098c0fb290 msgr2=0x7f098c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.617+0000 7f09941e3700 1 -- 172.21.15.33:0/1715747374 shutdown_connections 2024-05-08T21:28:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.617+0000 7f09941e3700 1 -- 172.21.15.33:0/1715747374 wait complete. 2024-05-08T21:28:23.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f09941e3700 1 Processor -- start 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f09941e3700 1 -- start start 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f09941e3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c0ff6d0 0x7f098c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f09941e3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c100030 0x7f098c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f09941e3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f098c1009f0 0x7f098c1a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f09941e3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c1a0a50 con 0x7f098c1009f0 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f09941e3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c1a0b90 con 0x7f098c100030 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f09941e3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c1a0d20 con 0x7f098c0ff6d0 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f0992780700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f098c1009f0 0x7f098c1a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f0992780700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f098c1009f0 0x7f098c1a0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44696/0 (socket says 172.21.15.33:44696) 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f0992780700 1 -- 172.21.15.33:0/3871741752 learned_addr learned my addr 172.21.15.33:0/3871741752 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f0991f7f700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c0ff6d0 0x7f098c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.618+0000 7f099177e700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c100030 0x7f098c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.619+0000 7f0992780700 1 -- 172.21.15.33:0/3871741752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c0ff6d0 msgr2=0x7f098c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.619+0000 7f0992780700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c0ff6d0 0x7f098c198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.619+0000 7f0992780700 1 -- 172.21.15.33:0/3871741752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c100030 msgr2=0x7f098c199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.619+0000 7f0992780700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c100030 0x7f098c199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.619+0000 7f0992780700 1 -- 172.21.15.33:0/3871741752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09880095f0 con 0x7f098c1009f0 2024-05-08T21:28:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.619+0000 7f0992780700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f098c1009f0 0x7f098c1a0480 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0984007ee0 tx=0x7f098400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.619+0000 7f0982ffd700 1 -- 172.21.15.33:0/3871741752 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098400dd20 con 0x7f098c1009f0 2024-05-08T21:28:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.619+0000 7f0982ffd700 1 -- 172.21.15.33:0/3871741752 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f098400f040 con 0x7f098c1009f0 2024-05-08T21:28:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.619+0000 7f09941e3700 1 -- 172.21.15.33:0/3871741752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098c1a0fa0 con 0x7f098c1009f0 2024-05-08T21:28:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.619+0000 7f0982ffd700 1 -- 172.21.15.33:0/3871741752 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984004750 con 0x7f098c1009f0 2024-05-08T21:28:23.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.619+0000 7f09941e3700 1 -- 172.21.15.33:0/3871741752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098c1a1430 con 0x7f098c1009f0 2024-05-08T21:28:23.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.621+0000 7f0982ffd700 1 -- 172.21.15.33:0/3871741752 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f0984003a00 con 0x7f098c1009f0 2024-05-08T21:28:23.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.622+0000 7f0982ffd700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f09780a1120 0x7f09780a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.622+0000 7f0982ffd700 1 -- 172.21.15.33:0/3871741752 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f09840cc500 con 0x7f098c1009f0 2024-05-08T21:28:23.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.622+0000 7f0991f7f700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f09780a1120 0x7f09780a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.623+0000 7f0991f7f700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f09780a1120 0x7f09780a35e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0988005910 tx=0x7f098800b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.623+0000 7f09a039db80 1 Processor -- start 2024-05-08T21:28:23.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.623+0000 7f09a039db80 1 -- start start 2024-05-08T21:28:23.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.623+0000 7f09a039db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eed540 0x556287eed960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.623+0000 7f09a039db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eedea0 0x556287eee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.623+0000 7f09a039db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eee860 0x556287eeece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.623+0000 7f09a039db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556287e44150 con 0x556287eed540 2024-05-08T21:28:23.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f09a039db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556287e41b10 con 0x556287eee860 2024-05-08T21:28:23.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f09a039db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556287e3d400 con 0x556287eedea0 2024-05-08T21:28:23.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0991f7f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eed540 0x556287eed960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0992780700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eee860 0x556287eeece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f099177e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eedea0 0x556287eee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0991f7f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eed540 0x556287eed960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44702/0 (socket says 172.21.15.33:44702) 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0991f7f700 1 -- 172.21.15.33:0/3192894809 learned_addr learned my addr 172.21.15.33:0/3192894809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0992780700 1 -- 172.21.15.33:0/3192894809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eedea0 msgr2=0x556287eee320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0992780700 1 --2- 172.21.15.33:0/3192894809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eedea0 0x556287eee320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0992780700 1 -- 172.21.15.33:0/3192894809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eed540 msgr2=0x556287eed960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0992780700 1 --2- 172.21.15.33:0/3192894809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eed540 0x556287eed960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0992780700 1 -- 172.21.15.33:0/3192894809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556287cbe3b0 con 0x556287eee860 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f099177e700 1 --2- 172.21.15.33:0/3192894809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eedea0 0x556287eee320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0992780700 1 --2- 172.21.15.33:0/3192894809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eee860 0x556287eeece0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f09840c9c50 tx=0x7f0984099200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2c62261b11e76f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0977fff700 1 -- 172.21.15.33:0/3192894809 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984003cb0 con 0x556287eee860 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0977fff700 1 -- 172.21.15.33:0/3192894809 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09840cb030 con 0x556287eee860 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.624+0000 7f0977fff700 1 -- 172.21.15.33:0/3192894809 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09840995d0 con 0x556287eee860 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.625+0000 7f09a039db80 1 -- 172.21.15.33:0/3192894809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eee860 msgr2=0x556287eeece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.625+0000 7f09a039db80 1 --2- 172.21.15.33:0/3192894809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eee860 0x556287eeece0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f09840c9c50 tx=0x7f0984099200 comp rx=0 tx=0).stop 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.625+0000 7f09a039db80 1 -- 172.21.15.33:0/3192894809 shutdown_connections 2024-05-08T21:28:23.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.625+0000 7f09a039db80 1 --2- 172.21.15.33:0/3192894809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eee860 0x556287eeece0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.625+0000 7f09a039db80 1 --2- 172.21.15.33:0/3192894809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eed540 0x556287eed960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.625+0000 7f09a039db80 1 --2- 172.21.15.33:0/3192894809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eedea0 0x556287eee320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.625+0000 7f09a039db80 1 -- 172.21.15.33:0/3192894809 >> 172.21.15.33:0/3192894809 conn(0x556287e4b210 msgr2=0x556287ee7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.625+0000 7f09a039db80 1 -- 172.21.15.33:0/3192894809 shutdown_connections 2024-05-08T21:28:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.625+0000 7f09a039db80 1 -- 172.21.15.33:0/3192894809 wait complete. 2024-05-08T21:28:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f09a039db80 1 -- start start 2024-05-08T21:28:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f09a039db80 1 Processor -- start 2024-05-08T21:28:23.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f09a039db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 0x556287ef6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f09a039db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287ef73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f09a039db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eee860 0x556287efe420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f09a039db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556287dd9560 con 0x556287eedea0 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f09a039db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556287dd9200 con 0x556287eed540 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f09a039db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556287cc06a0 con 0x556287eee860 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f09a039db80 10 client.0 _ll_register_callbacks cb 0x556287de2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f099177e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287ef73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f0991f7f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 0x556287ef6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f099177e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287ef73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44708/0 (socket says 172.21.15.33:44708) 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f0991f7f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 0x556287ef6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:52986/0 (socket says 172.21.15.33:52986) 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.626+0000 7f099177e700 1 -- 172.21.15.33:0/1240462870 learned_addr learned my addr 172.21.15.33:0/1240462870 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f0992780700 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eee860 0x556287efe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f099177e700 1 -- 172.21.15.33:0/1240462870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eee860 msgr2=0x556287efe420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f099177e700 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eee860 0x556287efe420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f099177e700 1 -- 172.21.15.33:0/1240462870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 msgr2=0x556287ef6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f099177e700 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 0x556287ef6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f099177e700 1 -- 172.21.15.33:0/1240462870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097c0098e0 con 0x556287eedea0 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f0991f7f700 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 0x556287ef6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f099177e700 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287ef73e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f097c012980 tx=0x7f097c012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2354b73b1040cf98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f09757fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f09757fa700 1 -- 172.21.15.33:0/1240462870 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c017070 con 0x556287eedea0 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f09757fa700 1 -- 172.21.15.33:0/1240462870 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f097c00f7b0 con 0x556287eedea0 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f09a039db80 1 -- 172.21.15.33:0/1240462870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556287d96270 con 0x556287eedea0 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f09757fa700 1 -- 172.21.15.33:0/1240462870 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f097c0122e0 con 0x556287eedea0 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f09a039db80 10 client.17937 fetch_fsmap learned FSMap version 45 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.627+0000 7f09a039db80 1 -- 172.21.15.33:0/1240462870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f098c1a1430 con 0x556287eedea0 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.628+0000 7f09757fa700 1 -- 172.21.15.33:0/1240462870 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556287d96270 con 0x556287eedea0 2024-05-08T21:28:23.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.628+0000 7f09757fa700 1 -- 172.21.15.33:0/1240462870 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f097c010a20 con 0x556287eedea0 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.628+0000 7f09a039db80 10 client.17937 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.628+0000 7f09a039db80 10 client.17937 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.628+0000 7f09a039db80 20 client.17937 populate_metadata read hostname 'smithi033' 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.628+0000 7f09a039db80 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x556287f02d60 0x556287f05180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.628+0000 7f09a039db80 4 client.17937 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.628+0000 7f09a039db80 1 -- 172.21.15.33:0/1240462870 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556287ebc810 con 0x556287f02d60 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.628+0000 7f0991f7f700 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x556287f02d60 0x556287f05180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.628+0000 7f0991f7f700 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x556287f02d60 0x556287f05180 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=efab046acab5cfe8 server_cookie=400baf7dfe2f3b11 in_seq=0 out_seq=0 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.628+0000 7f09757fa700 10 client.17937 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09757fa700 1 -- 172.21.15.33:0/1240462870 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556287ebc810 con 0x556287f02d60 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09757fa700 10 client.17937 handle_command_reply: tid=0 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09a039db80 1 client.17937 shutdown 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09a039db80 1 -- 172.21.15.33:0/1240462870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 msgr2=0x556287ef73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09a039db80 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287ef73e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f097c012980 tx=0x7f097c012c90 comp rx=0 tx=0).stop 2024-05-08T21:28:23.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09a039db80 1 -- 172.21.15.33:0/1240462870 shutdown_connections 2024-05-08T21:28:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09a039db80 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x556287f02d60 0x556287f05180 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09a039db80 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eee860 0x556287efe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09a039db80 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 0x556287ef6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09757fa700 0 client.17937 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:28:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09a039db80 1 --2- 172.21.15.33:0/1240462870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287ef73e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09a039db80 1 -- 172.21.15.33:0/1240462870 >> 172.21.15.33:0/1240462870 conn(0x556287e4b210 msgr2=0x556287ee75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09a039db80 1 -- 172.21.15.33:0/1240462870 shutdown_connections 2024-05-08T21:28:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.629+0000 7f09a039db80 1 -- 172.21.15.33:0/1240462870 wait complete. 2024-05-08T21:28:23.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.630+0000 7f09a039db80 20 client.17937 trim_cache size 0 max 16384 2024-05-08T21:28:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.643+0000 7f09a039db80 1 Processor -- start 2024-05-08T21:28:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.643+0000 7f09a039db80 1 -- start start 2024-05-08T21:28:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f09a039db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eed540 0x556287efead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f09a039db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287ef63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f09a039db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eee860 0x556287efe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f09a039db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556287f00130 con 0x556287eedea0 2024-05-08T21:28:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f09a039db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556287ee4dc0 con 0x556287eee860 2024-05-08T21:28:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f09a039db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556287f0a390 con 0x556287eed540 2024-05-08T21:28:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f099177e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287ef63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f099177e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287ef63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44716/0 (socket says 172.21.15.33:44716) 2024-05-08T21:28:23.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f099177e700 1 -- 172.21.15.33:0/2150216108 learned_addr learned my addr 172.21.15.33:0/2150216108 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f0992780700 1 --2- 172.21.15.33:0/2150216108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eee860 0x556287efe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f0991f7f700 1 --2- 172.21.15.33:0/2150216108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eed540 0x556287efead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f099177e700 1 -- 172.21.15.33:0/2150216108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eed540 msgr2=0x556287efead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f099177e700 1 --2- 172.21.15.33:0/2150216108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eed540 0x556287efead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f099177e700 1 -- 172.21.15.33:0/2150216108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eee860 msgr2=0x556287efe3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f099177e700 1 --2- 172.21.15.33:0/2150216108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eee860 0x556287efe3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f099177e700 1 -- 172.21.15.33:0/2150216108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556287c6ab60 con 0x556287eedea0 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f0992780700 1 --2- 172.21.15.33:0/2150216108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eee860 0x556287efe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f099177e700 1 --2- 172.21.15.33:0/2150216108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287ef63b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f097c0097c0 tx=0x7f097c0096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615cab4e10741aa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f0980ff9700 1 -- 172.21.15.33:0/2150216108 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c017070 con 0x556287eedea0 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.644+0000 7f0980ff9700 1 -- 172.21.15.33:0/2150216108 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f097c018040 con 0x556287eedea0 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.645+0000 7f0980ff9700 1 -- 172.21.15.33:0/2150216108 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c005040 con 0x556287eedea0 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.645+0000 7f09a039db80 1 -- 172.21.15.33:0/2150216108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 msgr2=0x556287ef63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.645+0000 7f09a039db80 1 --2- 172.21.15.33:0/2150216108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287ef63b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f097c0097c0 tx=0x7f097c0096d0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.645+0000 7f09a039db80 1 -- 172.21.15.33:0/2150216108 shutdown_connections 2024-05-08T21:28:23.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.645+0000 7f09a039db80 1 --2- 172.21.15.33:0/2150216108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eee860 0x556287efe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.645+0000 7f09a039db80 1 --2- 172.21.15.33:0/2150216108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eed540 0x556287efead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.645+0000 7f09a039db80 1 --2- 172.21.15.33:0/2150216108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287ef63b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.645+0000 7f09a039db80 1 -- 172.21.15.33:0/2150216108 >> 172.21.15.33:0/2150216108 conn(0x556287e4b210 msgr2=0x556287ee4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.645+0000 7f09a039db80 1 -- 172.21.15.33:0/2150216108 shutdown_connections 2024-05-08T21:28:23.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.645+0000 7f09a039db80 1 -- 172.21.15.33:0/2150216108 wait complete. 2024-05-08T21:28:23.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.645+0000 7f09a039db80 1 -- start start 2024-05-08T21:28:23.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f09a039db80 1 Processor -- start 2024-05-08T21:28:23.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f09a039db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 0x556287ef9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f09a039db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287efa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f09a039db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eee860 0x556287efd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f09a039db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556287efd7f0 con 0x556287eedea0 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f09a039db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556287efd960 con 0x556287eed540 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f09a039db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556287efdb20 con 0x556287eee860 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f09a039db80 10 client.0 _ll_register_callbacks cb 0x556287de2b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f099177e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287efa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f0991f7f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 0x556287ef9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f0991f7f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 0x556287ef9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:52992/0 (socket says 172.21.15.33:52992) 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f099177e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287efa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44730/0 (socket says 172.21.15.33:44730) 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f0991f7f700 1 -- 172.21.15.33:0/2661945941 learned_addr learned my addr 172.21.15.33:0/2661945941 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f0992780700 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eee860 0x556287efd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f0991f7f700 1 -- 172.21.15.33:0/2661945941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eee860 msgr2=0x556287efd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f0991f7f700 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eee860 0x556287efd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f0991f7f700 1 -- 172.21.15.33:0/2661945941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 msgr2=0x556287efa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.646+0000 7f0991f7f700 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287efa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f0991f7f700 1 -- 172.21.15.33:0/2661945941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556287cbe3b0 con 0x556287eed540 2024-05-08T21:28:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f099177e700 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287efa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f0991f7f700 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 0x556287ef9da0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f098800b2e0 tx=0x7f0988009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e42359b20927224 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09757fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:28:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09757fa700 1 -- 172.21.15.33:0/2661945941 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098800b920 con 0x556287eed540 2024-05-08T21:28:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09757fa700 1 -- 172.21.15.33:0/2661945941 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f098800ba80 con 0x556287eed540 2024-05-08T21:28:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09a039db80 1 -- 172.21.15.33:0/2661945941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09840cb030 con 0x556287eed540 2024-05-08T21:28:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09757fa700 1 -- 172.21.15.33:0/2661945941 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f098800bc40 con 0x556287eed540 2024-05-08T21:28:23.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09a039db80 10 client.27857 fetch_fsmap learned FSMap version 45 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09a039db80 1 -- 172.21.15.33:0/2661945941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f098c1a1430 con 0x556287eed540 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09757fa700 1 -- 172.21.15.33:0/2661945941 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09840cb030 con 0x556287eed540 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09757fa700 1 -- 172.21.15.33:0/2661945941 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 45) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f0988004500 con 0x556287eed540 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09a039db80 10 client.27857 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09a039db80 10 client.27857 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09a039db80 20 client.27857 populate_metadata read hostname 'smithi033' 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09a039db80 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x556287f59ea0 0x556287f5c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09a039db80 4 client.27857 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.647+0000 7f09a039db80 1 -- 172.21.15.33:0/2661945941 --> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556287e35ec0 con 0x556287f59ea0 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f099177e700 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x556287f59ea0 0x556287f5c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f099177e700 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x556287f59ea0 0x556287f5c2c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=617ce01c94c909fb server_cookie=985a97dfe5e9ee92 in_seq=0 out_seq=0 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09757fa700 10 client.27857 ms_handle_connect on v2:172.21.15.71:6834/3686264359 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09757fa700 1 -- 172.21.15.33:0/2661945941 <== mds.3 v2:172.21.15.71:6834/3686264359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556287e35ec0 con 0x556287f59ea0 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09757fa700 10 client.27857 handle_command_reply: tid=0 2024-05-08T21:28:23.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09a039db80 1 client.27857 shutdown 2024-05-08T21:28:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09a039db80 1 -- 172.21.15.33:0/2661945941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 msgr2=0x556287ef9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09a039db80 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 0x556287ef9da0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f098800b2e0 tx=0x7f0988009ca0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09a039db80 1 -- 172.21.15.33:0/2661945941 shutdown_connections 2024-05-08T21:28:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09a039db80 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.71:6834/3686264359,v1:172.21.15.71:6835/3686264359] conn(0x556287f59ea0 0x556287f5c2c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09a039db80 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556287eee860 0x556287efd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09757fa700 0 client.27857 ms_handle_reset on v2:172.21.15.71:6834/3686264359 2024-05-08T21:28:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09a039db80 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556287eed540 0x556287ef9da0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09a039db80 1 --2- 172.21.15.33:0/2661945941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556287eedea0 0x556287efa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09a039db80 1 -- 172.21.15.33:0/2661945941 >> 172.21.15.33:0/2661945941 conn(0x556287e4b210 msgr2=0x556287ef4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09a039db80 1 -- 172.21.15.33:0/2661945941 shutdown_connections 2024-05-08T21:28:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.648+0000 7f09a039db80 1 -- 172.21.15.33:0/2661945941 wait complete. 2024-05-08T21:28:23.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09a039db80 20 client.27857 trim_cache size 0 max 16384 2024-05-08T21:28:23.657 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:28:23.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09941e3700 1 -- 172.21.15.33:0/3871741752 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f09780a1120 msgr2=0x7f09780a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09941e3700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f09780a1120 0x7f09780a35e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0988005910 tx=0x7f098800b610 comp rx=0 tx=0).stop 2024-05-08T21:28:23.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09941e3700 1 -- 172.21.15.33:0/3871741752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f098c1009f0 msgr2=0x7f098c1a0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09941e3700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f098c1009f0 0x7f098c1a0480 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0984007ee0 tx=0x7f098400d3b0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09941e3700 1 -- 172.21.15.33:0/3871741752 shutdown_connections 2024-05-08T21:28:23.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09941e3700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f098c1009f0 0x7f098c1a0480 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09941e3700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f098c0ff6d0 0x7f098c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09941e3700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f09780a1120 0x7f09780a35e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09941e3700 1 --2- 172.21.15.33:0/3871741752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f098c100030 0x7f098c199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09941e3700 1 -- 172.21.15.33:0/3871741752 >> 172.21.15.33:0/3871741752 conn(0x7f098c0fb290 msgr2=0x7f098c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09941e3700 1 -- 172.21.15.33:0/3871741752 shutdown_connections 2024-05-08T21:28:23.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.649+0000 7f09941e3700 1 -- 172.21.15.33:0/3871741752 wait complete. 2024-05-08T21:28:23.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:23 smithi033 ceph-mon[42528]: pgmap v572: 129 pgs: 129 active+clean; 257 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 923 KiB/s wr, 269 op/s 2024-05-08T21:28:23.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:23 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3871440663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:28:23.664 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-05-08T21:28:23.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:23 smithi071 ceph-mon[47074]: pgmap v572: 129 pgs: 129 active+clean; 257 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 923 KiB/s wr, 269 op/s 2024-05-08T21:28:23.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:23 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3871440663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:28:23.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:23 smithi082 ceph-mon[47512]: pgmap v572: 129 pgs: 129 active+clean; 257 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 923 KiB/s wr, 269 op/s 2024-05-08T21:28:23.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:23 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3871440663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:28:23.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.882+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4203621210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a74100060 msgr2=0x7f9a741004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.882+0000 7f9a7bae3700 1 --2- 172.21.15.33:0/4203621210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a74100060 0x7f9a741004c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009f90 tx=0x7f9a6400b330 comp rx=0 tx=0).stop 2024-05-08T21:28:23.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.883+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4203621210 shutdown_connections 2024-05-08T21:28:23.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.883+0000 7f9a7bae3700 1 --2- 172.21.15.33:0/4203621210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a74100a00 0x7f9a741082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.883+0000 7f9a7bae3700 1 --2- 172.21.15.33:0/4203621210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a741013c0 0x7f9a741017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.883+0000 7f9a7bae3700 1 --2- 172.21.15.33:0/4203621210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a74100060 0x7f9a741004c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.883+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4203621210 >> 172.21.15.33:0/4203621210 conn(0x7f9a740faad0 msgr2=0x7f9a740fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.883+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4203621210 shutdown_connections 2024-05-08T21:28:23.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.884+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4203621210 wait complete. 2024-05-08T21:28:23.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.885+0000 7f9a7bae3700 1 Processor -- start 2024-05-08T21:28:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.885+0000 7f9a7bae3700 1 -- start start 2024-05-08T21:28:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.885+0000 7f9a7bae3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a74100060 0x7f9a7410a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.885+0000 7f9a7bae3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a74100a00 0x7f9a74108b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.885+0000 7f9a7bae3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a741013c0 0x7f9a74109040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.885+0000 7f9a7bae3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741a70e0 con 0x7f9a741013c0 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.885+0000 7f9a7bae3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741a7250 con 0x7f9a74100a00 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.885+0000 7f9a7bae3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a741a7410 con 0x7f9a74100060 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.885+0000 7f9a7907e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a74100a00 0x7f9a74108b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a7907e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a74100a00 0x7f9a74108b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:53016/0 (socket says 172.21.15.33:53016) 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a7907e700 1 -- 172.21.15.33:0/4116215376 learned_addr learned my addr 172.21.15.33:0/4116215376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a7a080700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a741013c0 0x7f9a74109040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a7987f700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a74100060 0x7f9a7410a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a7907e700 1 -- 172.21.15.33:0/4116215376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a74100060 msgr2=0x7f9a7410a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a7907e700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a74100060 0x7f9a7410a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a7907e700 1 -- 172.21.15.33:0/4116215376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a741013c0 msgr2=0x7f9a74109040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a7907e700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a741013c0 0x7f9a74109040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a7907e700 1 -- 172.21.15.33:0/4116215376 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c0095f0 con 0x7f9a74100a00 2024-05-08T21:28:23.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a7907e700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a74100a00 0x7f9a74108b00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9a64008cb0 tx=0x7f9a64008ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a72ffd700 1 -- 172.21.15.33:0/4116215376 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6400e040 con 0x7f9a74100a00 2024-05-08T21:28:23.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a72ffd700 1 -- 172.21.15.33:0/4116215376 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a6400bc00 con 0x7f9a74100a00 2024-05-08T21:28:23.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.886+0000 7f9a72ffd700 1 -- 172.21.15.33:0/4116215376 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a640052a0 con 0x7f9a74100a00 2024-05-08T21:28:23.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.890+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4116215376 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a64009c40 con 0x7f9a74100a00 2024-05-08T21:28:23.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.890+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4116215376 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a74109820 con 0x7f9a74100a00 2024-05-08T21:28:23.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.897+0000 7f9a87c9db80 1 Processor -- start 2024-05-08T21:28:23.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.897+0000 7f9a87c9db80 1 -- start start 2024-05-08T21:28:23.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.898+0000 7f9a87c9db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46540 0x557d5ae46960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.898+0000 7f9a87c9db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46ea0 0x557d5ae47320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.898+0000 7f9a87c9db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 0x557d5ae47ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.898+0000 7f9a87c9db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5ad9d150 con 0x557d5ae46ea0 2024-05-08T21:28:23.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.898+0000 7f9a87c9db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5ad9ab10 con 0x557d5ae47860 2024-05-08T21:28:23.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.898+0000 7f9a87c9db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5ad96400 con 0x557d5ae46540 2024-05-08T21:28:23.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.898+0000 7f9a7987f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46ea0 0x557d5ae47320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.898+0000 7f9a7987f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46ea0 0x557d5ae47320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44758/0 (socket says 172.21.15.33:44758) 2024-05-08T21:28:23.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.898+0000 7f9a7987f700 1 -- 172.21.15.33:0/2912912236 learned_addr learned my addr 172.21.15.33:0/2912912236 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.900+0000 7f9a7a080700 1 --2- 172.21.15.33:0/2912912236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46540 0x557d5ae46960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.901+0000 7f9a7907e700 1 --2- 172.21.15.33:0/2912912236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 0x557d5ae47ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.902+0000 7f9a7987f700 1 -- 172.21.15.33:0/2912912236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46540 msgr2=0x557d5ae46960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.902+0000 7f9a7987f700 1 --2- 172.21.15.33:0/2912912236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46540 0x557d5ae46960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.902+0000 7f9a7987f700 1 -- 172.21.15.33:0/2912912236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 msgr2=0x557d5ae47ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.902+0000 7f9a7987f700 1 --2- 172.21.15.33:0/2912912236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 0x557d5ae47ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.902+0000 7f9a7987f700 1 -- 172.21.15.33:0/2912912236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d5ac173b0 con 0x557d5ae46ea0 2024-05-08T21:28:23.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.904+0000 7f9a72ffd700 1 -- 172.21.15.33:0/4116215376 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f9a6401d470 con 0x7f9a74100a00 2024-05-08T21:28:23.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.904+0000 7f9a72ffd700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9a5c0a1220 0x7f9a5c0a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.905+0000 7f9a72ffd700 1 -- 172.21.15.33:0/4116215376 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9a640d4b40 con 0x7f9a74100a00 2024-05-08T21:28:23.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.905+0000 7f9a7987f700 1 --2- 172.21.15.33:0/2912912236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46ea0 0x557d5ae47320 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009e50 tx=0x7f9a6c010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7b0a1345b9450e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.905+0000 7f9a7a080700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9a5c0a1220 0x7f9a5c0a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.906+0000 7f9a7a080700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9a5c0a1220 0x7f9a5c0a36e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009d80 tx=0x7f9a68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.908+0000 7f9a5bfff700 1 -- 172.21.15.33:0/2912912236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c010db0 con 0x557d5ae46ea0 2024-05-08T21:28:23.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.908+0000 7f9a5bfff700 1 -- 172.21.15.33:0/2912912236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a6c0128e0 con 0x557d5ae46ea0 2024-05-08T21:28:23.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.908+0000 7f9a5bfff700 1 -- 172.21.15.33:0/2912912236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0117d0 con 0x557d5ae46ea0 2024-05-08T21:28:23.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.909+0000 7f9a87c9db80 1 -- 172.21.15.33:0/2912912236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46ea0 msgr2=0x557d5ae47320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.909+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/2912912236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46ea0 0x557d5ae47320 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009e50 tx=0x7f9a6c010650 comp rx=0 tx=0).stop 2024-05-08T21:28:23.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.909+0000 7f9a87c9db80 1 -- 172.21.15.33:0/2912912236 shutdown_connections 2024-05-08T21:28:23.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.909+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/2912912236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 0x557d5ae47ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.909+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/2912912236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46540 0x557d5ae46960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.909+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/2912912236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46ea0 0x557d5ae47320 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.909+0000 7f9a87c9db80 1 -- 172.21.15.33:0/2912912236 >> 172.21.15.33:0/2912912236 conn(0x557d5ada4210 msgr2=0x557d5ae40de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.913+0000 7f9a87c9db80 1 -- 172.21.15.33:0/2912912236 shutdown_connections 2024-05-08T21:28:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.914+0000 7f9a87c9db80 1 -- 172.21.15.33:0/2912912236 wait complete. 2024-05-08T21:28:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.914+0000 7f9a87c9db80 1 -- start start 2024-05-08T21:28:23.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.914+0000 7f9a87c9db80 1 Processor -- start 2024-05-08T21:28:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.915+0000 7f9a87c9db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae4fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.915+0000 7f9a87c9db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46ea0 0x557d5ae50310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.915+0000 7f9a87c9db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 0x557d5ae57350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.915+0000 7f9a87c9db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5ad32560 con 0x557d5ae46540 2024-05-08T21:28:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.915+0000 7f9a87c9db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5ad32200 con 0x557d5ae47860 2024-05-08T21:28:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.915+0000 7f9a87c9db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5ac196a0 con 0x557d5ae46ea0 2024-05-08T21:28:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.915+0000 7f9a87c9db80 10 client.0 _ll_register_callbacks cb 0x557d5ad3bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:28:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.916+0000 7f9a7987f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae4fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.916+0000 7f9a7987f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae4fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44770/0 (socket says 172.21.15.33:44770) 2024-05-08T21:28:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.916+0000 7f9a7987f700 1 -- 172.21.15.33:0/64768441 learned_addr learned my addr 172.21.15.33:0/64768441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.917+0000 7f9a7987f700 1 -- 172.21.15.33:0/64768441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46ea0 msgr2=0x557d5ae50310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:28:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.917+0000 7f9a7987f700 1 --2- 172.21.15.33:0/64768441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46ea0 0x557d5ae50310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.917+0000 7f9a7987f700 1 -- 172.21.15.33:0/64768441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 msgr2=0x557d5ae57350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T21:28:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.917+0000 7f9a7987f700 1 --2- 172.21.15.33:0/64768441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 0x557d5ae57350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.917+0000 7f9a7987f700 1 -- 172.21.15.33:0/64768441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6c009ab0 con 0x557d5ae46540 2024-05-08T21:28:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.918+0000 7f9a7987f700 1 --2- 172.21.15.33:0/64768441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae4fdd0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c006010 tx=0x7f9a6c012d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a203e919d56c213f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.918+0000 7f9a597fa700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:28:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.918+0000 7f9a597fa700 1 -- 172.21.15.33:0/64768441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c01f070 con 0x557d5ae46540 2024-05-08T21:28:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.918+0000 7f9a597fa700 1 -- 172.21.15.33:0/64768441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9a6c016070 con 0x557d5ae46540 2024-05-08T21:28:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.920+0000 7f9a87c9db80 1 -- 172.21.15.33:0/64768441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d5acef270 con 0x557d5ae46540 2024-05-08T21:28:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.921+0000 7f9a597fa700 1 -- 172.21.15.33:0/64768441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a6c004850 con 0x557d5ae46540 2024-05-08T21:28:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.921+0000 7f9a87c9db80 10 client.17955 fetch_fsmap learned FSMap version 45 2024-05-08T21:28:23.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.921+0000 7f9a87c9db80 1 -- 172.21.15.33:0/64768441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x557d5abc3b60 con 0x557d5ae46540 2024-05-08T21:28:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.922+0000 7f9a597fa700 1 -- 172.21.15.33:0/64768441 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557d5acef270 con 0x557d5ae46540 2024-05-08T21:28:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.922+0000 7f9a597fa700 1 -- 172.21.15.33:0/64768441 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f9a6c005020 con 0x557d5ae46540 2024-05-08T21:28:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.922+0000 7f9a87c9db80 10 client.17955 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T21:28:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.922+0000 7f9a87c9db80 10 client.17955 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:28:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.922+0000 7f9a87c9db80 20 client.17955 populate_metadata read hostname 'smithi033' 2024-05-08T21:28:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.922+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/64768441 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x557d5ae5bad0 0x557d5ae5def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.922+0000 7f9a87c9db80 4 client.17955 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:28:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.922+0000 7f9a87c9db80 1 -- 172.21.15.33:0/64768441 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d5ae15810 con 0x557d5ae5bad0 2024-05-08T21:28:23.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.922+0000 7f9a7907e700 1 --2- 172.21.15.33:0/64768441 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x557d5ae5bad0 0x557d5ae5def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.923+0000 7f9a597fa700 10 client.17955 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:28:23.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.923+0000 7f9a7907e700 1 --2- 172.21.15.33:0/64768441 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x557d5ae5bad0 0x557d5ae5def0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f54e8c361a77f75e server_cookie=b44ae27eef8f3bd6 in_seq=0 out_seq=0 2024-05-08T21:28:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.925+0000 7f9a597fa700 1 -- 172.21.15.33:0/64768441 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d5ae15810 con 0x557d5ae5bad0 2024-05-08T21:28:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.925+0000 7f9a597fa700 10 client.17955 handle_command_reply: tid=0 2024-05-08T21:28:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.925+0000 7f9a87c9db80 1 client.17955 shutdown 2024-05-08T21:28:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.926+0000 7f9a87c9db80 1 -- 172.21.15.33:0/64768441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 msgr2=0x557d5ae4fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.926+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/64768441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae4fdd0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c006010 tx=0x7f9a6c012d30 comp rx=0 tx=0).stop 2024-05-08T21:28:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.926+0000 7f9a87c9db80 1 -- 172.21.15.33:0/64768441 shutdown_connections 2024-05-08T21:28:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.926+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/64768441 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x557d5ae5bad0 0x557d5ae5def0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.926+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/64768441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 0x557d5ae57350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.926+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/64768441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae4fdd0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.926+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/64768441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46ea0 0x557d5ae50310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.926+0000 7f9a87c9db80 1 -- 172.21.15.33:0/64768441 >> 172.21.15.33:0/64768441 conn(0x557d5ada4210 msgr2=0x557d5ae405a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.926+0000 7f9a597fa700 0 client.17955 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:28:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.926+0000 7f9a87c9db80 1 -- 172.21.15.33:0/64768441 shutdown_connections 2024-05-08T21:28:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.927+0000 7f9a87c9db80 1 -- 172.21.15.33:0/64768441 wait complete. 2024-05-08T21:28:23.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.927+0000 7f9a87c9db80 20 client.17955 trim_cache size 0 max 16384 2024-05-08T21:28:23.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.952+0000 7f9a87c9db80 1 Processor -- start 2024-05-08T21:28:23.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.952+0000 7f9a87c9db80 1 -- start start 2024-05-08T21:28:23.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.952+0000 7f9a87c9db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae46920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.952+0000 7f9a87c9db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46ea0 0x557d5ae57310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.952+0000 7f9a87c9db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 0x557d5ae4f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.952+0000 7f9a87c9db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5acef2a0 con 0x557d5ae46540 2024-05-08T21:28:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.952+0000 7f9a87c9db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5ae41b90 con 0x557d5ae47860 2024-05-08T21:28:23.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.952+0000 7f9a87c9db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5ae39040 con 0x557d5ae46ea0 2024-05-08T21:28:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.952+0000 7f9a7987f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae46920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.953+0000 7f9a7987f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae46920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44778/0 (socket says 172.21.15.33:44778) 2024-05-08T21:28:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.953+0000 7f9a7987f700 1 -- 172.21.15.33:0/2092228207 learned_addr learned my addr 172.21.15.33:0/2092228207 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.953+0000 7f9a7a080700 1 --2- 172.21.15.33:0/2092228207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 0x557d5ae4f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.953+0000 7f9a7987f700 1 -- 172.21.15.33:0/2092228207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46ea0 msgr2=0x557d5ae57310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.953+0000 7f9a7987f700 1 --2- 172.21.15.33:0/2092228207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46ea0 0x557d5ae57310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.953+0000 7f9a7987f700 1 -- 172.21.15.33:0/2092228207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 msgr2=0x557d5ae4f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.953+0000 7f9a7987f700 1 --2- 172.21.15.33:0/2092228207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 0x557d5ae4f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.953+0000 7f9a7987f700 1 -- 172.21.15.33:0/2092228207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d5ad5c560 con 0x557d5ae46540 2024-05-08T21:28:23.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.953+0000 7f9a7a080700 1 --2- 172.21.15.33:0/2092228207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 0x557d5ae4f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.953+0000 7f9a7987f700 1 --2- 172.21.15.33:0/2092228207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae46920 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c015040 tx=0x7f9a6c010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81735d0673f5460f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.954+0000 7f9a70da4700 1 -- 172.21.15.33:0/2092228207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c01f040 con 0x557d5ae46540 2024-05-08T21:28:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.954+0000 7f9a70da4700 1 -- 172.21.15.33:0/2092228207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a6c020040 con 0x557d5ae46540 2024-05-08T21:28:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.954+0000 7f9a70da4700 1 -- 172.21.15.33:0/2092228207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c019380 con 0x557d5ae46540 2024-05-08T21:28:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.955+0000 7f9a87c9db80 1 -- 172.21.15.33:0/2092228207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 msgr2=0x557d5ae46920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.955+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/2092228207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae46920 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c015040 tx=0x7f9a6c010650 comp rx=0 tx=0).stop 2024-05-08T21:28:23.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.958+0000 7f9a87c9db80 1 -- 172.21.15.33:0/2092228207 shutdown_connections 2024-05-08T21:28:23.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.958+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/2092228207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae47860 0x557d5ae4f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.958+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/2092228207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae46920 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.958+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/2092228207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae46ea0 0x557d5ae57310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.958+0000 7f9a87c9db80 1 -- 172.21.15.33:0/2092228207 >> 172.21.15.33:0/2092228207 conn(0x557d5ada4210 msgr2=0x557d5ae47c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.958+0000 7f9a87c9db80 1 -- 172.21.15.33:0/2092228207 shutdown_connections 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.958+0000 7f9a87c9db80 1 -- 172.21.15.33:0/2092228207 wait complete. 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.958+0000 7f9a87c9db80 1 -- start start 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.959+0000 7f9a87c9db80 1 Processor -- start 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.959+0000 7f9a87c9db80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae52580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.959+0000 7f9a87c9db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae46ea0 0x557d5ae52ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.959+0000 7f9a87c9db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae47860 0x557d5ae55980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.959+0000 7f9a87c9db80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5ae55ec0 con 0x557d5ae46540 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.959+0000 7f9a87c9db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5ae56030 con 0x557d5ae46ea0 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.959+0000 7f9a87c9db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5ae561f0 con 0x557d5ae47860 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.959+0000 7f9a87c9db80 10 client.0 _ll_register_callbacks cb 0x557d5ad3bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:28:23.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.960+0000 7f9a7a080700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae47860 0x557d5ae55980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.960+0000 7f9a7a080700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae47860 0x557d5ae55980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:44916/0 (socket says 172.21.15.33:44916) 2024-05-08T21:28:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.960+0000 7f9a7a080700 1 -- 172.21.15.33:0/1980280026 learned_addr learned my addr 172.21.15.33:0/1980280026 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.960+0000 7f9a7907e700 1 --2- 172.21.15.33:0/1980280026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae46ea0 0x557d5ae52ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.960+0000 7f9a7907e700 1 -- 172.21.15.33:0/1980280026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae47860 msgr2=0x557d5ae55980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.960+0000 7f9a7907e700 1 --2- 172.21.15.33:0/1980280026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae47860 0x557d5ae55980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.960+0000 7f9a7907e700 1 -- 172.21.15.33:0/1980280026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 msgr2=0x557d5ae52580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.960+0000 7f9a7907e700 1 --2- 172.21.15.33:0/1980280026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae52580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.960+0000 7f9a7907e700 1 -- 172.21.15.33:0/1980280026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a68012c90 con 0x557d5ae46ea0 2024-05-08T21:28:23.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.961+0000 7f9a7907e700 1 --2- 172.21.15.33:0/1980280026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae46ea0 0x557d5ae52ac0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x557d5ae41d40 tx=0x7f9a640a15e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89e38b1ccbf824b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:23.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.964+0000 7f9a597fa700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-05-08T21:28:23.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.964+0000 7f9a597fa700 1 -- 172.21.15.33:0/1980280026 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a640a1bd0 con 0x557d5ae46ea0 2024-05-08T21:28:23.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.964+0000 7f9a597fa700 1 -- 172.21.15.33:0/1980280026 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f9a640a5a00 con 0x557d5ae46ea0 2024-05-08T21:28:23.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.965+0000 7f9a87c9db80 1 -- 172.21.15.33:0/1980280026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d5ae4ede0 con 0x557d5ae46ea0 2024-05-08T21:28:23.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.965+0000 7f9a597fa700 1 -- 172.21.15.33:0/1980280026 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a640aa050 con 0x557d5ae46ea0 2024-05-08T21:28:23.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.967+0000 7f9a87c9db80 10 client.27872 fetch_fsmap learned FSMap version 45 2024-05-08T21:28:23.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.967+0000 7f9a87c9db80 1 -- 172.21.15.33:0/1980280026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x557d5abc3b60 con 0x557d5ae46ea0 2024-05-08T21:28:23.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.975+0000 7f9a597fa700 1 -- 172.21.15.33:0/1980280026 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9a640be070 con 0x557d5ae46ea0 2024-05-08T21:28:23.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.976+0000 7f9a597fa700 1 -- 172.21.15.33:0/1980280026 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 45) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f9a640a2be0 con 0x557d5ae46ea0 2024-05-08T21:28:23.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.976+0000 7f9a87c9db80 10 client.27872 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T21:28:23.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.976+0000 7f9a87c9db80 10 client.27872 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-05-08T21:28:23.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.976+0000 7f9a87c9db80 20 client.27872 populate_metadata read hostname 'smithi033' 2024-05-08T21:28:23.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.976+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/1980280026 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x557d5aed4090 0x557d5aed6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:23.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.976+0000 7f9a87c9db80 4 client.27872 mds_command: new command op to 24499 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:28:23.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.976+0000 7f9a87c9db80 1 -- 172.21.15.33:0/1980280026 --> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f9a6c0128e0 con 0x557d5aed4090 2024-05-08T21:28:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.976+0000 7f9a7987f700 1 --2- 172.21.15.33:0/1980280026 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x557d5aed4090 0x557d5aed6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.976+0000 7f9a7987f700 1 --2- 172.21.15.33:0/1980280026 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x557d5aed4090 0x557d5aed6480 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0e7068b5b251011 server_cookie=7789e5711e46e886 in_seq=0 out_seq=0 2024-05-08T21:28:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.977+0000 7f9a597fa700 10 client.27872 ms_handle_connect on v2:172.21.15.82:6836/2409190198 2024-05-08T21:28:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.977+0000 7f9a597fa700 1 -- 172.21.15.33:0/1980280026 <== mds.2 v2:172.21.15.82:6836/2409190198 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f9a6c0128e0 con 0x557d5aed4090 2024-05-08T21:28:23.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.977+0000 7f9a597fa700 10 client.27872 handle_command_reply: tid=0 2024-05-08T21:28:23.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.977+0000 7f9a87c9db80 1 client.27872 shutdown 2024-05-08T21:28:23.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.981+0000 7f9a87c9db80 1 -- 172.21.15.33:0/1980280026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae46ea0 msgr2=0x557d5ae52ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.981+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/1980280026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae46ea0 0x557d5ae52ac0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x557d5ae41d40 tx=0x7f9a640a15e0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.984+0000 7f9a87c9db80 1 -- 172.21.15.33:0/1980280026 shutdown_connections 2024-05-08T21:28:23.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.984+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/1980280026 >> [v2:172.21.15.82:6836/2409190198,v1:172.21.15.82:6837/2409190198] conn(0x557d5aed4090 0x557d5aed6480 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.984+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/1980280026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557d5ae47860 0x557d5ae55980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.984+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/1980280026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d5ae46540 0x557d5ae52580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.984+0000 7f9a87c9db80 1 --2- 172.21.15.33:0/1980280026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d5ae46ea0 0x557d5ae52ac0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.984+0000 7f9a87c9db80 1 -- 172.21.15.33:0/1980280026 >> 172.21.15.33:0/1980280026 conn(0x557d5ada4210 msgr2=0x557d5ae5dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:23.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.984+0000 7f9a597fa700 0 client.27872 ms_handle_reset on v2:172.21.15.82:6836/2409190198 2024-05-08T21:28:23.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.985+0000 7f9a87c9db80 1 -- 172.21.15.33:0/1980280026 shutdown_connections 2024-05-08T21:28:23.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.985+0000 7f9a87c9db80 1 -- 172.21.15.33:0/1980280026 wait complete. 2024-05-08T21:28:23.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.993+0000 7f9a87c9db80 20 client.27872 trim_cache size 0 max 16384 2024-05-08T21:28:23.995 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:28:23.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.994+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4116215376 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9a5c0a1220 msgr2=0x7f9a5c0a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.994+0000 7f9a7bae3700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9a5c0a1220 0x7f9a5c0a36e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009d80 tx=0x7f9a68009260 comp rx=0 tx=0).stop 2024-05-08T21:28:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.994+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4116215376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a74100a00 msgr2=0x7f9a74108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.994+0000 7f9a7bae3700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a74100a00 0x7f9a74108b00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9a64008cb0 tx=0x7f9a64008ce0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.995+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4116215376 shutdown_connections 2024-05-08T21:28:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.995+0000 7f9a7bae3700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9a741013c0 0x7f9a74109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.995+0000 7f9a7bae3700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a74100060 0x7f9a7410a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.995+0000 7f9a7bae3700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f9a5c0a1220 0x7f9a5c0a36e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.995+0000 7f9a7bae3700 1 --2- 172.21.15.33:0/4116215376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a74100a00 0x7f9a74108b00 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9a64008cb0 tx=0x7f9a64008ce0 comp rx=0 tx=0).stop 2024-05-08T21:28:23.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:23.995+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4116215376 >> 172.21.15.33:0/4116215376 conn(0x7f9a740faad0 msgr2=0x7f9a74104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.001+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4116215376 shutdown_connections 2024-05-08T21:28:24.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.002+0000 7f9a7bae3700 1 -- 172.21.15.33:0/4116215376 wait complete. 2024-05-08T21:28:24.041 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-05-08T21:28:24.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.199+0000 7faf2cc9b700 1 -- 172.21.15.33:0/233834090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf28101920 msgr2=0x7faf281081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.199+0000 7faf2cc9b700 1 --2- 172.21.15.33:0/233834090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf28101920 0x7faf281081d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7faf2000b0d0 tx=0x7faf2000b3e0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.199+0000 7faf2cc9b700 1 -- 172.21.15.33:0/233834090 shutdown_connections 2024-05-08T21:28:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.199+0000 7faf2cc9b700 1 --2- 172.21.15.33:0/233834090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf28101920 0x7faf281081d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.199+0000 7faf2cc9b700 1 --2- 172.21.15.33:0/233834090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf280ffd90 0x7faf281001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.199+0000 7faf2cc9b700 1 --2- 172.21.15.33:0/233834090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf28100f80 0x7faf281013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.199+0000 7faf2cc9b700 1 -- 172.21.15.33:0/233834090 >> 172.21.15.33:0/233834090 conn(0x7faf280fb300 msgr2=0x7faf280fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.199+0000 7faf2cc9b700 1 -- 172.21.15.33:0/233834090 shutdown_connections 2024-05-08T21:28:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.199+0000 7faf2cc9b700 1 -- 172.21.15.33:0/233834090 wait complete. 2024-05-08T21:28:24.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf2cc9b700 1 Processor -- start 2024-05-08T21:28:24.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf2cc9b700 1 -- start start 2024-05-08T21:28:24.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf2cc9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf280ffd90 0x7faf28196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf2cc9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf28100f80 0x7faf28197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf2cc9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf28101920 0x7faf2819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf2cc9b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2819e8f0 con 0x7faf28100f80 2024-05-08T21:28:24.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf2cc9b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2819ea60 con 0x7faf28101920 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf2cc9b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2819ec20 con 0x7faf280ffd90 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf26d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf28101920 0x7faf2819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf25d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf28100f80 0x7faf28197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf26d9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf28101920 0x7faf2819e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.33:53076/0 (socket says 172.21.15.33:53076) 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf26d9d700 1 -- 172.21.15.33:0/2965675924 learned_addr learned my addr 172.21.15.33:0/2965675924 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf2659c700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf280ffd90 0x7faf28196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf26d9d700 1 -- 172.21.15.33:0/2965675924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf280ffd90 msgr2=0x7faf28196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf26d9d700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf280ffd90 0x7faf28196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf26d9d700 1 -- 172.21.15.33:0/2965675924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf28100f80 msgr2=0x7faf28197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf26d9d700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf28100f80 0x7faf28197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf26d9d700 1 -- 172.21.15.33:0/2965675924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf20009d00 con 0x7faf28101920 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.200+0000 7faf2659c700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf280ffd90 0x7faf28196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.201+0000 7faf26d9d700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf28101920 0x7faf2819e320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faf200079c0 tx=0x7faf2000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:24.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.201+0000 7faf177fe700 1 -- 172.21.15.33:0/2965675924 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2000e040 con 0x7faf28101920 2024-05-08T21:28:24.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.201+0000 7faf2cc9b700 1 -- 172.21.15.33:0/2965675924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2819eea0 con 0x7faf28101920 2024-05-08T21:28:24.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.201+0000 7faf177fe700 1 -- 172.21.15.33:0/2965675924 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf20004e60 con 0x7faf28101920 2024-05-08T21:28:24.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.201+0000 7faf177fe700 1 -- 172.21.15.33:0/2965675924 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2001b6b0 con 0x7faf28101920 2024-05-08T21:28:24.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.201+0000 7faf2cc9b700 1 -- 172.21.15.33:0/2965675924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2819f260 con 0x7faf28101920 2024-05-08T21:28:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.204+0000 7faf177fe700 1 -- 172.21.15.33:0/2965675924 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7faf2001d9a0 con 0x7faf28101920 2024-05-08T21:28:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.204+0000 7faf177fe700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faf100a1120 0x7faf100a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.205+0000 7faf2659c700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faf100a1120 0x7faf100a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.205+0000 7faf177fe700 1 -- 172.21.15.33:0/2965675924 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7faf200ce760 con 0x7faf28101920 2024-05-08T21:28:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.205+0000 7faf2659c700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faf100a1120 0x7faf100a35e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7faf18005950 tx=0x7faf180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.205+0000 7faf38e55b80 1 Processor -- start 2024-05-08T21:28:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf38e55b80 1 -- start start 2024-05-08T21:28:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf38e55b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f37429540 0x561f37429960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf38e55b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429ea0 0x561f3742a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf38e55b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f3742a860 0x561f3742ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf38e55b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f37380150 con 0x561f3742a860 2024-05-08T21:28:24.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf38e55b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f3737db10 con 0x561f37429ea0 2024-05-08T21:28:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf38e55b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f37379400 con 0x561f37429540 2024-05-08T21:28:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf26d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f3742a860 0x561f3742ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf2659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f37429540 0x561f37429960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf26d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f3742a860 0x561f3742ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44810/0 (socket says 172.21.15.33:44810) 2024-05-08T21:28:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf26d9d700 1 -- 172.21.15.33:0/3391830171 learned_addr learned my addr 172.21.15.33:0/3391830171 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf25d9b700 1 --2- 172.21.15.33:0/3391830171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429ea0 0x561f3742a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf2659c700 1 -- 172.21.15.33:0/3391830171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429ea0 msgr2=0x561f3742a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf2659c700 1 --2- 172.21.15.33:0/3391830171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429ea0 0x561f3742a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf2659c700 1 -- 172.21.15.33:0/3391830171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f3742a860 msgr2=0x561f3742ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf2659c700 1 --2- 172.21.15.33:0/3391830171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f3742a860 0x561f3742ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf2659c700 1 -- 172.21.15.33:0/3391830171 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f371fa3b0 con 0x561f37429540 2024-05-08T21:28:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.206+0000 7faf26d9d700 1 --2- 172.21.15.33:0/3391830171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f3742a860 0x561f3742ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.207+0000 7faf2659c700 1 --2- 172.21.15.33:0/3391830171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f37429540 0x561f37429960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf28100de0 tx=0x7faf18012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72dd6c86828bf072 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.207+0000 7faf14ff9700 1 -- 172.21.15.33:0/3391830171 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18004220 con 0x561f37429540 2024-05-08T21:28:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.207+0000 7faf14ff9700 1 -- 172.21.15.33:0/3391830171 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf18011d60 con 0x561f37429540 2024-05-08T21:28:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.207+0000 7faf14ff9700 1 -- 172.21.15.33:0/3391830171 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18003d00 con 0x561f37429540 2024-05-08T21:28:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 -- 172.21.15.33:0/3391830171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f37429540 msgr2=0x561f37429960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 --2- 172.21.15.33:0/3391830171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f37429540 0x561f37429960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faf28100de0 tx=0x7faf18012000 comp rx=0 tx=0).stop 2024-05-08T21:28:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 -- 172.21.15.33:0/3391830171 shutdown_connections 2024-05-08T21:28:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 --2- 172.21.15.33:0/3391830171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f3742a860 0x561f3742ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 --2- 172.21.15.33:0/3391830171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f37429540 0x561f37429960 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 --2- 172.21.15.33:0/3391830171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429ea0 0x561f3742a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 -- 172.21.15.33:0/3391830171 >> 172.21.15.33:0/3391830171 conn(0x561f37387210 msgr2=0x561f37423de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 -- 172.21.15.33:0/3391830171 shutdown_connections 2024-05-08T21:28:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 -- 172.21.15.33:0/3391830171 wait complete. 2024-05-08T21:28:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 -- start start 2024-05-08T21:28:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 Processor -- start 2024-05-08T21:28:24.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429540 0x561f37432d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429ea0 0x561f374332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f3742a860 0x561f3743a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f37315560 con 0x561f37429ea0 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f37315200 con 0x561f37429540 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f371fc6a0 con 0x561f3742a860 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.208+0000 7faf38e55b80 10 client.0 _ll_register_callbacks cb 0x561f3731eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf25d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429ea0 0x561f374332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf25d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429ea0 0x561f374332c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44820/0 (socket says 172.21.15.33:44820) 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf25d9b700 1 -- 172.21.15.33:0/3984032744 learned_addr learned my addr 172.21.15.33:0/3984032744 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf26d9d700 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f3742a860 0x561f3743a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf2659c700 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429540 0x561f37432d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf25d9b700 1 -- 172.21.15.33:0/3984032744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f3742a860 msgr2=0x561f3743a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf25d9b700 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f3742a860 0x561f3743a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf25d9b700 1 -- 172.21.15.33:0/3984032744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429540 msgr2=0x561f37432d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf25d9b700 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429540 0x561f37432d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf25d9b700 1 -- 172.21.15.33:0/3984032744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2819f260 con 0x561f37429ea0 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf25d9b700 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429ea0 0x561f374332c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7faf1c012860 tx=0x7faf1c012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8efa6eff7ea5dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf0dffb700 1 -- 172.21.15.33:0/3984032744 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c017070 con 0x561f37429ea0 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf38e55b80 1 -- 172.21.15.33:0/3984032744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f372d2270 con 0x561f37429ea0 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf0dffb700 1 -- 172.21.15.33:0/3984032744 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7faf1c00f520 con 0x561f37429ea0 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.209+0000 7faf0dffb700 1 -- 172.21.15.33:0/3984032744 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf1c0122e0 con 0x561f37429ea0 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.210+0000 7faf38e55b80 10 client.17970 fetch_fsmap learned FSMap version 45 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.210+0000 7faf38e55b80 1 -- 172.21.15.33:0/3984032744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7faf1800be10 con 0x561f37429ea0 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.210+0000 7faf0dffb700 1 -- 172.21.15.33:0/3984032744 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561f372d2270 con 0x561f37429ea0 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.210+0000 7faf0dffb700 1 -- 172.21.15.33:0/3984032744 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) v1 ==== 7518+0+0 (secure 0 0 0) 0x7faf1c0107a0 con 0x561f37429ea0 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.210+0000 7faf38e55b80 10 client.17970 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.210+0000 7faf38e55b80 10 client.17970 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.210+0000 7faf38e55b80 20 client.17970 populate_metadata read hostname 'smithi033' 2024-05-08T21:28:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.210+0000 7faf38e55b80 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561f3743ec40 0x561f37441060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.210+0000 7faf38e55b80 4 client.17970 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.210+0000 7faf38e55b80 1 -- 172.21.15.33:0/3984032744 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f373f8810 con 0x561f3743ec40 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.210+0000 7faf2659c700 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561f3743ec40 0x561f37441060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf2659c700 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561f3743ec40 0x561f37441060 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bc60b7a7f1fb11c server_cookie=f000b009ba4b8790 in_seq=0 out_seq=0 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf0dffb700 10 client.17970 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf0dffb700 1 -- 172.21.15.33:0/3984032744 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561f373f8810 con 0x561f3743ec40 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf0dffb700 10 client.17970 handle_command_reply: tid=0 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf38e55b80 1 client.17970 shutdown 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf38e55b80 1 -- 172.21.15.33:0/3984032744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429ea0 msgr2=0x561f374332c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf38e55b80 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429ea0 0x561f374332c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7faf1c012860 tx=0x7faf1c012b70 comp rx=0 tx=0).stop 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf38e55b80 1 -- 172.21.15.33:0/3984032744 shutdown_connections 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf38e55b80 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561f3743ec40 0x561f37441060 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf38e55b80 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f3742a860 0x561f3743a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf38e55b80 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429540 0x561f37432d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf38e55b80 1 --2- 172.21.15.33:0/3984032744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429ea0 0x561f374332c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf0dffb700 0 client.17970 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:28:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.211+0000 7faf38e55b80 1 -- 172.21.15.33:0/3984032744 >> 172.21.15.33:0/3984032744 conn(0x561f37387210 msgr2=0x561f374235a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.212+0000 7faf38e55b80 1 -- 172.21.15.33:0/3984032744 shutdown_connections 2024-05-08T21:28:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.212+0000 7faf38e55b80 1 -- 172.21.15.33:0/3984032744 wait complete. 2024-05-08T21:28:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.212+0000 7faf38e55b80 20 client.17970 trim_cache size 0 max 16384 2024-05-08T21:28:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf38e55b80 1 Processor -- start 2024-05-08T21:28:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf38e55b80 1 -- start start 2024-05-08T21:28:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf38e55b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 0x561f3743a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf38e55b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429ea0 0x561f3743a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf38e55b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f3742a860 0x561f37432290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf38e55b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f37425570 con 0x561f37429540 2024-05-08T21:28:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf38e55b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f37431d60 con 0x561f37429ea0 2024-05-08T21:28:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf38e55b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f37446250 con 0x561f3742a860 2024-05-08T21:28:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 0x561f3743a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 0x561f3743a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44830/0 (socket says 172.21.15.33:44830) 2024-05-08T21:28:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf2659c700 1 -- 172.21.15.33:0/2406985044 learned_addr learned my addr 172.21.15.33:0/2406985044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:24.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf26d9d700 1 --2- 172.21.15.33:0/2406985044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f3742a860 0x561f37432290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf25d9b700 1 --2- 172.21.15.33:0/2406985044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429ea0 0x561f3743a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf2659c700 1 -- 172.21.15.33:0/2406985044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f3742a860 msgr2=0x561f37432290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf2659c700 1 --2- 172.21.15.33:0/2406985044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f3742a860 0x561f37432290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf2659c700 1 -- 172.21.15.33:0/2406985044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429ea0 msgr2=0x561f3743a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf2659c700 1 --2- 172.21.15.33:0/2406985044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429ea0 0x561f3743a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.226+0000 7faf2659c700 1 -- 172.21.15.33:0/2406985044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f371a6b60 con 0x561f37429540 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf2659c700 1 --2- 172.21.15.33:0/2406985044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 0x561f3743a9b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7faf18011720 tx=0x7faf18004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8face142ff6884d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf157fa700 1 -- 172.21.15.33:0/2406985044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf180118d0 con 0x561f37429540 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf157fa700 1 -- 172.21.15.33:0/2406985044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf18011a30 con 0x561f37429540 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf157fa700 1 -- 172.21.15.33:0/2406985044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1801e450 con 0x561f37429540 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf38e55b80 1 -- 172.21.15.33:0/2406985044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 msgr2=0x561f3743a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf38e55b80 1 --2- 172.21.15.33:0/2406985044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 0x561f3743a9b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7faf18011720 tx=0x7faf18004710 comp rx=0 tx=0).stop 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf38e55b80 1 -- 172.21.15.33:0/2406985044 shutdown_connections 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf38e55b80 1 --2- 172.21.15.33:0/2406985044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f3742a860 0x561f37432290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf38e55b80 1 --2- 172.21.15.33:0/2406985044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 0x561f3743a9b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf38e55b80 1 --2- 172.21.15.33:0/2406985044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f37429ea0 0x561f3743a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf38e55b80 1 -- 172.21.15.33:0/2406985044 >> 172.21.15.33:0/2406985044 conn(0x561f37387210 msgr2=0x561f3742ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf38e55b80 1 -- 172.21.15.33:0/2406985044 shutdown_connections 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.227+0000 7faf38e55b80 1 -- 172.21.15.33:0/2406985044 wait complete. 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf38e55b80 1 -- start start 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf38e55b80 1 Processor -- start 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf38e55b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 0x561f37436450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf38e55b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f37429ea0 0x561f37436990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf38e55b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f3742a860 0x561f37439490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf38e55b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f374399d0 con 0x561f37429540 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf38e55b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f37439b40 con 0x561f3742a860 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf38e55b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f37439d00 con 0x561f37429ea0 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf38e55b80 10 client.0 _ll_register_callbacks cb 0x561f3731eb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 0x561f37436450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf2659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 0x561f37436450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44838/0 (socket says 172.21.15.33:44838) 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf2659c700 1 -- 172.21.15.33:0/384475992 learned_addr learned my addr 172.21.15.33:0/384475992 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:24.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf26d9d700 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f3742a860 0x561f37439490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.228+0000 7faf25d9b700 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f37429ea0 0x561f37436990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf2659c700 1 -- 172.21.15.33:0/384475992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f37429ea0 msgr2=0x561f37436990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf2659c700 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f37429ea0 0x561f37436990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf2659c700 1 -- 172.21.15.33:0/384475992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f3742a860 msgr2=0x561f37439490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf2659c700 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f3742a860 0x561f37439490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf2659c700 1 -- 172.21.15.33:0/384475992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f371fa3b0 con 0x561f37429540 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf2659c700 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 0x561f37436450 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7faf18038040 tx=0x7faf180118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31015c68dcaf26eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf0dffb700 1 -- 172.21.15.33:0/384475992 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18035250 con 0x561f37429540 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf0dffb700 1 -- 172.21.15.33:0/384475992 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7faf180353b0 con 0x561f37429540 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf38e55b80 1 -- 172.21.15.33:0/384475992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf18011d60 con 0x561f37429540 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf0dffb700 1 -- 172.21.15.33:0/384475992 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf1803f450 con 0x561f37429540 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf38e55b80 10 client.17976 fetch_fsmap learned FSMap version 45 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf38e55b80 1 -- 172.21.15.33:0/384475992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x561f3733f560 con 0x561f37429540 2024-05-08T21:28:24.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.229+0000 7faf0dffb700 1 -- 172.21.15.33:0/384475992 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf18011d60 con 0x561f37429540 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf0dffb700 1 -- 172.21.15.33:0/384475992 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) v1 ==== 7518+0+0 (secure 0 0 0) 0x7faf18031070 con 0x561f37429540 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf38e55b80 10 client.17976 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf38e55b80 10 client.17976 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf38e55b80 20 client.17976 populate_metadata read hostname 'smithi033' 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf38e55b80 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561f37495b70 0x561f37497f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf38e55b80 4 client.17976 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf38e55b80 1 -- 172.21.15.33:0/384475992 --> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561f37371ec0 con 0x561f37495b70 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf25d9b700 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561f37495b70 0x561f37497f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf25d9b700 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561f37495b70 0x561f37497f90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e06da47dfae17d1e server_cookie=56f139d49253d4b4 in_seq=0 out_seq=0 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf0dffb700 10 client.17976 ms_handle_connect on v2:172.21.15.71:6836/1668078897 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf0dffb700 1 -- 172.21.15.33:0/384475992 <== mds.1 v2:172.21.15.71:6836/1668078897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561f37371ec0 con 0x561f37495b70 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf0dffb700 10 client.17976 handle_command_reply: tid=0 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.230+0000 7faf38e55b80 1 client.17976 shutdown 2024-05-08T21:28:24.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf38e55b80 1 -- 172.21.15.33:0/384475992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 msgr2=0x561f37436450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf38e55b80 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 0x561f37436450 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7faf18038040 tx=0x7faf180118d0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf38e55b80 1 -- 172.21.15.33:0/384475992 shutdown_connections 2024-05-08T21:28:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf38e55b80 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.71:6836/1668078897,v1:172.21.15.71:6837/1668078897] conn(0x561f37495b70 0x561f37497f90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf38e55b80 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f3742a860 0x561f37439490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf38e55b80 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561f37429540 0x561f37436450 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf0dffb700 0 client.17976 ms_handle_reset on v2:172.21.15.71:6836/1668078897 2024-05-08T21:28:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf38e55b80 1 --2- 172.21.15.33:0/384475992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f37429ea0 0x561f37436990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf38e55b80 1 -- 172.21.15.33:0/384475992 >> 172.21.15.33:0/384475992 conn(0x561f37387210 msgr2=0x561f3742d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf38e55b80 1 -- 172.21.15.33:0/384475992 shutdown_connections 2024-05-08T21:28:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf38e55b80 1 -- 172.21.15.33:0/384475992 wait complete. 2024-05-08T21:28:24.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf38e55b80 20 client.17976 trim_cache size 0 max 16384 2024-05-08T21:28:24.240 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:28:24.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf2cc9b700 1 -- 172.21.15.33:0/2965675924 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faf100a1120 msgr2=0x7faf100a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf2cc9b700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faf100a1120 0x7faf100a35e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7faf18005950 tx=0x7faf180058e0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf2cc9b700 1 -- 172.21.15.33:0/2965675924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf28101920 msgr2=0x7faf2819e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf2cc9b700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf28101920 0x7faf2819e320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faf200079c0 tx=0x7faf2000ccc0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf2cc9b700 1 -- 172.21.15.33:0/2965675924 shutdown_connections 2024-05-08T21:28:24.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf2cc9b700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7faf100a1120 0x7faf100a35e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf2cc9b700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf28101920 0x7faf2819e320 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf2cc9b700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf280ffd90 0x7faf28196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf2cc9b700 1 --2- 172.21.15.33:0/2965675924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf28100f80 0x7faf28197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.231+0000 7faf2cc9b700 1 -- 172.21.15.33:0/2965675924 >> 172.21.15.33:0/2965675924 conn(0x7faf280fb300 msgr2=0x7faf280fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.232+0000 7faf2cc9b700 1 -- 172.21.15.33:0/2965675924 shutdown_connections 2024-05-08T21:28:24.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.232+0000 7faf2cc9b700 1 -- 172.21.15.33:0/2965675924 wait complete. 2024-05-08T21:28:24.246 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-05-08T21:28:24.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.386+0000 7f15053ee700 1 -- 172.21.15.33:0/2539060416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15000ff1b0 msgr2=0x7f15000ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.386+0000 7f15053ee700 1 --2- 172.21.15.33:0/2539060416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15000ff1b0 0x7f15000ff5d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009990 tx=0x7f14f0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.386+0000 7f15053ee700 1 -- 172.21.15.33:0/2539060416 shutdown_connections 2024-05-08T21:28:24.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.386+0000 7f15053ee700 1 --2- 172.21.15.33:0/2539060416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1500102bc0 0x7f1500109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.386+0000 7f15053ee700 1 --2- 172.21.15.33:0/2539060416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15000ff1b0 0x7f15000ff5d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.386+0000 7f15053ee700 1 --2- 172.21.15.33:0/2539060416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1500102220 0x7f1500102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.386+0000 7f15053ee700 1 -- 172.21.15.33:0/2539060416 >> 172.21.15.33:0/2539060416 conn(0x7f15000faae0 msgr2=0x7f15000fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.386+0000 7f15053ee700 1 -- 172.21.15.33:0/2539060416 shutdown_connections 2024-05-08T21:28:24.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.386+0000 7f15053ee700 1 -- 172.21.15.33:0/2539060416 wait complete. 2024-05-08T21:28:24.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.387+0000 7f15053ee700 1 Processor -- start 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.387+0000 7f15053ee700 1 -- start start 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.387+0000 7f15053ee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15000ff1b0 0x7f1500196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.387+0000 7f15053ee700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1500102220 0x7f15001972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.387+0000 7f15053ee700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1500102bc0 0x7f150019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.387+0000 7f15053ee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150019e880 con 0x7f15000ff1b0 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.387+0000 7f15053ee700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150019e9f0 con 0x7f1500102bc0 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.387+0000 7f15053ee700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150019ebb0 con 0x7f1500102220 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15000ff1b0 0x7f1500196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15000ff1b0 0x7f1500196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44856/0 (socket says 172.21.15.33:44856) 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14feffd700 1 -- 172.21.15.33:0/2942600778 learned_addr learned my addr 172.21.15.33:0/2942600778 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1500102220 0x7f15001972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14ff7fe700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1500102bc0 0x7f150019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14fe7fc700 1 -- 172.21.15.33:0/2942600778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1500102bc0 msgr2=0x7f150019e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1500102bc0 0x7f150019e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14fe7fc700 1 -- 172.21.15.33:0/2942600778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15000ff1b0 msgr2=0x7f1500196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15000ff1b0 0x7f1500196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14fe7fc700 1 -- 172.21.15.33:0/2942600778 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e8009520 con 0x7f1500102220 2024-05-08T21:28:24.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14feffd700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15000ff1b0 0x7f1500196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:24.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1500102220 0x7f15001972d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f14e800e9f0 tx=0x7f14e800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:24.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14f7fff700 1 -- 172.21.15.33:0/2942600778 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e800cb70 con 0x7f1500102220 2024-05-08T21:28:24.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f15053ee700 1 -- 172.21.15.33:0/2942600778 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f8009c40 con 0x7f1500102220 2024-05-08T21:28:24.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14f7fff700 1 -- 172.21.15.33:0/2942600778 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14e800ee80 con 0x7f1500102220 2024-05-08T21:28:24.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f14f7fff700 1 -- 172.21.15.33:0/2942600778 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8003c00 con 0x7f1500102220 2024-05-08T21:28:24.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.388+0000 7f15053ee700 1 -- 172.21.15.33:0/2942600778 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f00095f0 con 0x7f1500102220 2024-05-08T21:28:24.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.391+0000 7f14f7fff700 1 -- 172.21.15.33:0/2942600778 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f14e8003d60 con 0x7f1500102220 2024-05-08T21:28:24.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.391+0000 7f14f7fff700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f14ec0a1120 0x7f14ec0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.391+0000 7f14feffd700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f14ec0a1120 0x7f14ec0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.391+0000 7f14f7fff700 1 -- 172.21.15.33:0/2942600778 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f14e8014070 con 0x7f1500102220 2024-05-08T21:28:24.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.392+0000 7f14feffd700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f14ec0a1120 0x7f14ec0a35e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f14f000b2e0 tx=0x7f14f000b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:24.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.392+0000 7f15115a8b80 1 Processor -- start 2024-05-08T21:28:24.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.392+0000 7f15115a8b80 1 -- start start 2024-05-08T21:28:24.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f15115a8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd16960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f15115a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16ea0 0x55977fd17320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f15115a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd17860 0x55977fd17ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f15115a8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fc6d150 con 0x55977fd16540 2024-05-08T21:28:24.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f15115a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fc6ab10 con 0x55977fd17860 2024-05-08T21:28:24.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f15115a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fc66400 con 0x55977fd16ea0 2024-05-08T21:28:24.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd16960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd16960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44862/0 (socket says 172.21.15.33:44862) 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14feffd700 1 -- 172.21.15.33:0/2939845975 learned_addr learned my addr 172.21.15.33:0/2939845975 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14ff7fe700 1 --2- 172.21.15.33:0/2939845975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd17860 0x55977fd17ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/2939845975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16ea0 0x55977fd17320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14feffd700 1 -- 172.21.15.33:0/2939845975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16ea0 msgr2=0x55977fd17320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14feffd700 1 --2- 172.21.15.33:0/2939845975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16ea0 0x55977fd17320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14feffd700 1 -- 172.21.15.33:0/2939845975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd17860 msgr2=0x55977fd17ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14feffd700 1 --2- 172.21.15.33:0/2939845975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd17860 0x55977fd17ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14feffd700 1 -- 172.21.15.33:0/2939845975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55977fae73b0 con 0x55977fd16540 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/2939845975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16ea0 0x55977fd17320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14feffd700 1 --2- 172.21.15.33:0/2939845975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd16960 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f14f0000c00 tx=0x7f14f0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55adce87b4eea9cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14f57fa700 1 -- 172.21.15.33:0/2939845975 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f002b070 con 0x55977fd16540 2024-05-08T21:28:24.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14f57fa700 1 -- 172.21.15.33:0/2939845975 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14f001e040 con 0x55977fd16540 2024-05-08T21:28:24.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.393+0000 7f14f57fa700 1 -- 172.21.15.33:0/2939845975 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f000fcf0 con 0x55977fd16540 2024-05-08T21:28:24.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.394+0000 7f15115a8b80 1 -- 172.21.15.33:0/2939845975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 msgr2=0x55977fd16960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.394+0000 7f15115a8b80 1 --2- 172.21.15.33:0/2939845975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd16960 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f14f0000c00 tx=0x7f14f0020000 comp rx=0 tx=0).stop 2024-05-08T21:28:24.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.394+0000 7f15115a8b80 1 -- 172.21.15.33:0/2939845975 shutdown_connections 2024-05-08T21:28:24.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.394+0000 7f15115a8b80 1 --2- 172.21.15.33:0/2939845975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd17860 0x55977fd17ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.394+0000 7f15115a8b80 1 --2- 172.21.15.33:0/2939845975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd16960 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.394+0000 7f15115a8b80 1 --2- 172.21.15.33:0/2939845975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16ea0 0x55977fd17320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.394+0000 7f15115a8b80 1 -- 172.21.15.33:0/2939845975 >> 172.21.15.33:0/2939845975 conn(0x55977fc74210 msgr2=0x55977fd10de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.394+0000 7f15115a8b80 1 -- 172.21.15.33:0/2939845975 shutdown_connections 2024-05-08T21:28:24.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.394+0000 7f15115a8b80 1 -- 172.21.15.33:0/2939845975 wait complete. 2024-05-08T21:28:24.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f15115a8b80 1 -- start start 2024-05-08T21:28:24.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f15115a8b80 1 Processor -- start 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f15115a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16540 0x55977fd1fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f15115a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd20250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f15115a8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd17860 0x55977fd27290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f15115a8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fc02560 con 0x55977fd17860 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f15115a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fc02200 con 0x55977fd16ea0 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f15115a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fae96a0 con 0x55977fd16540 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f15115a8b80 10 client.0 _ll_register_callbacks cb 0x55977fc0bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f14ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd17860 0x55977fd27290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f14feffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16540 0x55977fd1fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f14feffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16540 0x55977fd1fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:44996/0 (socket says 172.21.15.33:44996) 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f14ff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd17860 0x55977fd27290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44868/0 (socket says 172.21.15.33:44868) 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f14feffd700 1 -- 172.21.15.33:0/3886015046 learned_addr learned my addr 172.21.15.33:0/3886015046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.395+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd20250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f14feffd700 1 -- 172.21.15.33:0/3886015046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 msgr2=0x55977fd20250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f14feffd700 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd20250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f14feffd700 1 -- 172.21.15.33:0/3886015046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd17860 msgr2=0x55977fd27290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f14feffd700 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd17860 0x55977fd27290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f14feffd700 1 -- 172.21.15.33:0/3886015046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f00095f0 con 0x55977fd16540 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd20250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f14feffd700 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16540 0x55977fd1fd10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f14f000b9f0 tx=0x7f14f000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3a97b90a93b6ebf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f14e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f14e67fc700 1 -- 172.21.15.33:0/3886015046 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f002b040 con 0x55977fd16540 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f15115a8b80 1 -- 172.21.15.33:0/3886015046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55977fbbf270 con 0x55977fd16540 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f14e67fc700 1 -- 172.21.15.33:0/3886015046 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f14f0003d10 con 0x55977fd16540 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f14e67fc700 1 -- 172.21.15.33:0/3886015046 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14f0005450 con 0x55977fd16540 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f15115a8b80 10 client.27898 fetch_fsmap learned FSMap version 45 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f15115a8b80 1 -- 172.21.15.33:0/3886015046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f1500109c60 con 0x55977fd16540 2024-05-08T21:28:24.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.396+0000 7f14e67fc700 1 -- 172.21.15.33:0/3886015046 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14f0041080 con 0x55977fd16540 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.397+0000 7f14e67fc700 1 -- 172.21.15.33:0/3886015046 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 45) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f14f0038410 con 0x55977fd16540 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.397+0000 7f15115a8b80 10 client.27898 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.397+0000 7f15115a8b80 10 client.27898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.397+0000 7f15115a8b80 20 client.27898 populate_metadata read hostname 'smithi033' 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.397+0000 7f15115a8b80 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55977fd2b640 0x55977fd2da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.397+0000 7f15115a8b80 4 client.27898 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.397+0000 7f15115a8b80 1 -- 172.21.15.33:0/3886015046 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55977fce5810 con 0x55977fd2b640 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.397+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55977fd2b640 0x55977fd2da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.397+0000 7f14e67fc700 10 client.27898 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.397+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55977fd2b640 0x55977fd2da30 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=372a39798d9e8581 server_cookie=aa1e815cc90675ed in_seq=0 out_seq=0 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f14e67fc700 1 -- 172.21.15.33:0/3886015046 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55977fce5810 con 0x55977fd2b640 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f14e67fc700 10 client.27898 handle_command_reply: tid=0 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f15115a8b80 1 client.27898 shutdown 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f15115a8b80 1 -- 172.21.15.33:0/3886015046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16540 msgr2=0x55977fd1fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f15115a8b80 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16540 0x55977fd1fd10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f14f000b9f0 tx=0x7f14f000bea0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f15115a8b80 1 -- 172.21.15.33:0/3886015046 shutdown_connections 2024-05-08T21:28:24.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f15115a8b80 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55977fd2b640 0x55977fd2da30 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f15115a8b80 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd17860 0x55977fd27290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f15115a8b80 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd16540 0x55977fd1fd10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f14e67fc700 0 client.27898 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:28:24.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f15115a8b80 1 --2- 172.21.15.33:0/3886015046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd20250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f15115a8b80 1 -- 172.21.15.33:0/3886015046 >> 172.21.15.33:0/3886015046 conn(0x55977fc74210 msgr2=0x55977fd105a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f15115a8b80 1 -- 172.21.15.33:0/3886015046 shutdown_connections 2024-05-08T21:28:24.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.398+0000 7f15115a8b80 1 -- 172.21.15.33:0/3886015046 wait complete. 2024-05-08T21:28:24.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.399+0000 7f15115a8b80 20 client.27898 trim_cache size 0 max 16384 2024-05-08T21:28:24.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.412+0000 7f15115a8b80 1 Processor -- start 2024-05-08T21:28:24.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.412+0000 7f15115a8b80 1 -- start start 2024-05-08T21:28:24.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f15115a8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd16920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f15115a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd27940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f15115a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd17860 0x55977fd27250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f15115a8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fd0b800 con 0x55977fd16540 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f15115a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fd32c20 con 0x55977fd16ea0 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f15115a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fd2b3e0 con 0x55977fd17860 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f14feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd16920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f14feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd16920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44870/0 (socket says 172.21.15.33:44870) 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f14feffd700 1 -- 172.21.15.33:0/711941609 learned_addr learned my addr 172.21.15.33:0/711941609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f14ff7fe700 1 --2- 172.21.15.33:0/711941609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd17860 0x55977fd27250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/711941609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd27940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f14feffd700 1 -- 172.21.15.33:0/711941609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd17860 msgr2=0x55977fd27250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f14feffd700 1 --2- 172.21.15.33:0/711941609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd17860 0x55977fd27250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f14feffd700 1 -- 172.21.15.33:0/711941609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 msgr2=0x55977fd27940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f14feffd700 1 --2- 172.21.15.33:0/711941609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd27940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f14feffd700 1 -- 172.21.15.33:0/711941609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55977fa93b60 con 0x55977fd16540 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.413+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/711941609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd27940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:24.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.414+0000 7f14feffd700 1 --2- 172.21.15.33:0/711941609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd16920 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f14f00098e0 tx=0x7f14f000ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204eb01c1adde088 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:24.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.414+0000 7f14f5ffb700 1 -- 172.21.15.33:0/711941609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f002b040 con 0x55977fd16540 2024-05-08T21:28:24.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.414+0000 7f14f5ffb700 1 -- 172.21.15.33:0/711941609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55977fbbf270 con 0x55977fd16540 2024-05-08T21:28:24.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.414+0000 7f14f5ffb700 1 -- 172.21.15.33:0/711941609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f000f6a0 con 0x55977fd16540 2024-05-08T21:28:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.414+0000 7f15115a8b80 1 -- 172.21.15.33:0/711941609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 msgr2=0x55977fd16920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.414+0000 7f15115a8b80 1 --2- 172.21.15.33:0/711941609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd16920 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f14f00098e0 tx=0x7f14f000ffd0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.414+0000 7f15115a8b80 1 -- 172.21.15.33:0/711941609 shutdown_connections 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.414+0000 7f15115a8b80 1 --2- 172.21.15.33:0/711941609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd17860 0x55977fd27250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.414+0000 7f15115a8b80 1 --2- 172.21.15.33:0/711941609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd16920 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.414+0000 7f15115a8b80 1 --2- 172.21.15.33:0/711941609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd27940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.414+0000 7f15115a8b80 1 -- 172.21.15.33:0/711941609 >> 172.21.15.33:0/711941609 conn(0x55977fc74210 msgr2=0x55977fd17c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f15115a8b80 1 -- 172.21.15.33:0/711941609 shutdown_connections 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f15115a8b80 1 -- 172.21.15.33:0/711941609 wait complete. 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f15115a8b80 1 -- start start 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f15115a8b80 1 Processor -- start 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f15115a8b80 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd22a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f15115a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd22fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f15115a8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd17860 0x55977fd25e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f15115a8b80 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fd263a0 con 0x55977fd16540 2024-05-08T21:28:24.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f15115a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fd26510 con 0x55977fd16ea0 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f15115a8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fd266d0 con 0x55977fd17860 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f15115a8b80 10 client.0 _ll_register_callbacks cb 0x55977fc0bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f14feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd22a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f14feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd22a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44876/0 (socket says 172.21.15.33:44876) 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f14feffd700 1 -- 172.21.15.33:0/732903376 learned_addr learned my addr 172.21.15.33:0/732903376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f14ff7fe700 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd17860 0x55977fd25e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.415+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd22fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f14feffd700 1 -- 172.21.15.33:0/732903376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd17860 msgr2=0x55977fd25e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f14feffd700 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd17860 0x55977fd25e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f14feffd700 1 -- 172.21.15.33:0/732903376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 msgr2=0x55977fd22fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f14feffd700 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd22fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f14feffd700 1 -- 172.21.15.33:0/732903376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1500109c60 con 0x55977fd16540 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f14ff7fe700 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd17860 0x55977fd25e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:24.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd22fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f14feffd700 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd22a60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f14f0038740 tx=0x7f14f0005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93333a124388a362 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f14e67fc700 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f14e67fc700 1 -- 172.21.15.33:0/732903376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f002b040 con 0x55977fd16540 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f14e67fc700 1 -- 172.21.15.33:0/732903376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f14f0003d10 con 0x55977fd16540 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f15115a8b80 1 -- 172.21.15.33:0/732903376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14f001e040 con 0x55977fd16540 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f14e67fc700 1 -- 172.21.15.33:0/732903376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14f000f8e0 con 0x55977fd16540 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f15115a8b80 10 client.17994 fetch_fsmap learned FSMap version 45 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.416+0000 7f15115a8b80 1 -- 172.21.15.33:0/732903376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55977fc2c560 con 0x55977fd16540 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.417+0000 7f14e67fc700 1 -- 172.21.15.33:0/732903376 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14f001e040 con 0x55977fd16540 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.417+0000 7f14e67fc700 1 -- 172.21.15.33:0/732903376 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) v1 ==== 7518+0+0 (secure 0 0 0) 0x7f14f0049470 con 0x55977fd16540 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.417+0000 7f15115a8b80 10 client.17994 fetch_fsmap finished waiting for FSMap version 45 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.417+0000 7f15115a8b80 10 client.17994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.417+0000 7f15115a8b80 20 client.17994 populate_metadata read hostname 'smithi033' 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.417+0000 7f15115a8b80 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55977fd81e90 0x55977fd84280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:24.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.417+0000 7f15115a8b80 4 client.17994 mds_command: new command op to 24563 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.417+0000 7f15115a8b80 1 -- 172.21.15.33:0/732903376 --> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55977fc5eec0 con 0x55977fd81e90 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.417+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55977fd81e90 0x55977fd84280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.417+0000 7f14e67fc700 10 client.17994 ms_handle_connect on v2:172.21.15.71:6838/1515211151 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.417+0000 7f14fe7fc700 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55977fd81e90 0x55977fd84280 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3823253b3636f295 server_cookie=5f4eff29796966db in_seq=0 out_seq=0 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f14e67fc700 1 -- 172.21.15.33:0/732903376 <== mds.0 v2:172.21.15.71:6838/1515211151 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55977fc5eec0 con 0x55977fd81e90 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f14e67fc700 10 client.17994 handle_command_reply: tid=0 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15115a8b80 1 client.17994 shutdown 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15115a8b80 1 -- 172.21.15.33:0/732903376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 msgr2=0x55977fd22a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15115a8b80 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd22a60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f14f0038740 tx=0x7f14f0005670 comp rx=0 tx=0).stop 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15115a8b80 1 -- 172.21.15.33:0/732903376 shutdown_connections 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15115a8b80 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.71:6838/1515211151,v1:172.21.15.71:6839/1515211151] conn(0x55977fd81e90 0x55977fd84280 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15115a8b80 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55977fd17860 0x55977fd25e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15115a8b80 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55977fd16540 0x55977fd22a60 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f14e67fc700 0 client.17994 ms_handle_reset on v2:172.21.15.71:6838/1515211151 2024-05-08T21:28:24.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15115a8b80 1 --2- 172.21.15.33:0/732903376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55977fd16ea0 0x55977fd22fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15115a8b80 1 -- 172.21.15.33:0/732903376 >> 172.21.15.33:0/732903376 conn(0x55977fc74210 msgr2=0x55977fd27940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15115a8b80 1 -- 172.21.15.33:0/732903376 shutdown_connections 2024-05-08T21:28:24.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15115a8b80 1 -- 172.21.15.33:0/732903376 wait complete. 2024-05-08T21:28:24.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15115a8b80 20 client.17994 trim_cache size 0 max 16384 2024-05-08T21:28:24.426 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-05-08T21:28:24.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15053ee700 1 -- 172.21.15.33:0/2942600778 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f14ec0a1120 msgr2=0x7f14ec0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15053ee700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f14ec0a1120 0x7f14ec0a35e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f14f000b2e0 tx=0x7f14f000b610 comp rx=0 tx=0).stop 2024-05-08T21:28:24.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15053ee700 1 -- 172.21.15.33:0/2942600778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1500102220 msgr2=0x7f15001972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:24.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15053ee700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1500102220 0x7f15001972d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f14e800e9f0 tx=0x7f14e800ed00 comp rx=0 tx=0).stop 2024-05-08T21:28:24.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15053ee700 1 -- 172.21.15.33:0/2942600778 shutdown_connections 2024-05-08T21:28:24.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15053ee700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f14ec0a1120 0x7f14ec0a35e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15053ee700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1500102bc0 0x7f150019e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15053ee700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15000ff1b0 0x7f1500196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15053ee700 1 --2- 172.21.15.33:0/2942600778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1500102220 0x7f15001972d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:24.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.418+0000 7f15053ee700 1 -- 172.21.15.33:0/2942600778 >> 172.21.15.33:0/2942600778 conn(0x7f15000faae0 msgr2=0x7f1500105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:24.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.419+0000 7f15053ee700 1 -- 172.21.15.33:0/2942600778 shutdown_connections 2024-05-08T21:28:24.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:24.419+0000 7f15053ee700 1 -- 172.21.15.33:0/2942600778 wait complete. 2024-05-08T21:28:25.434 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-05-08T21:28:25.434 INFO:tasks.fwd_scrub:done joining 2024-05-08T21:28:25.434 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-08T21:28:25.445 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:28:25.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.592+0000 7f6617803700 1 -- 172.21.15.33:0/1953721408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100ffd80 msgr2=0x7f66101001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:25.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.592+0000 7f6617803700 1 --2- 172.21.15.33:0/1953721408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100ffd80 0x7f66101001a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f660c0099c0 tx=0x7f660c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:28:25.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.592+0000 7f6617803700 1 -- 172.21.15.33:0/1953721408 shutdown_connections 2024-05-08T21:28:25.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.592+0000 7f6617803700 1 --2- 172.21.15.33:0/1953721408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6610101910 0x7f66101081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:25.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.592+0000 7f6617803700 1 --2- 172.21.15.33:0/1953721408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100ffd80 0x7f66101001a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:25.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.592+0000 7f6617803700 1 --2- 172.21.15.33:0/1953721408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6610100f70 0x7f66101013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:25.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.592+0000 7f6617803700 1 -- 172.21.15.33:0/1953721408 >> 172.21.15.33:0/1953721408 conn(0x7f66100fb310 msgr2=0x7f66100fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:25.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.592+0000 7f6617803700 1 -- 172.21.15.33:0/1953721408 shutdown_connections 2024-05-08T21:28:25.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.592+0000 7f6617803700 1 -- 172.21.15.33:0/1953721408 wait complete. 2024-05-08T21:28:25.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f6617803700 1 Processor -- start 2024-05-08T21:28:25.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f6617803700 1 -- start start 2024-05-08T21:28:25.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f6617803700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100ffd80 0x7f6610195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:25.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f6617803700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6610100f70 0x7f66101961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:25.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f6617803700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6610101910 0x7f661019d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:25.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f6617803700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661019d760 con 0x7f66100ffd80 2024-05-08T21:28:25.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f6617803700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661019d8d0 con 0x7f6610101910 2024-05-08T21:28:25.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f6617803700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661019da90 con 0x7f6610100f70 2024-05-08T21:28:25.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f661559f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100ffd80 0x7f6610195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:25.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f661559f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100ffd80 0x7f6610195c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44890/0 (socket says 172.21.15.33:44890) 2024-05-08T21:28:25.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f661559f700 1 -- 172.21.15.33:0/3611931588 learned_addr learned my addr 172.21.15.33:0/3611931588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:25.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f6615da0700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6610101910 0x7f661019d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:25.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f6614d9e700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6610100f70 0x7f66101961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.593+0000 7f661559f700 1 -- 172.21.15.33:0/3611931588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6610100f70 msgr2=0x7f66101961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.594+0000 7f661559f700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6610100f70 0x7f66101961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.594+0000 7f661559f700 1 -- 172.21.15.33:0/3611931588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6610101910 msgr2=0x7f661019d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.594+0000 7f661559f700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6610101910 0x7f661019d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.594+0000 7f661559f700 1 -- 172.21.15.33:0/3611931588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6600009520 con 0x7f66100ffd80 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.594+0000 7f6614d9e700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6610100f70 0x7f66101961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.594+0000 7f661559f700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100ffd80 0x7f6610195c70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f660c006010 tx=0x7f660c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.594+0000 7f66067fc700 1 -- 172.21.15.33:0/3611931588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c01d070 con 0x7f66100ffd80 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.594+0000 7f66067fc700 1 -- 172.21.15.33:0/3611931588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f660c0054c0 con 0x7f66100ffd80 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.594+0000 7f66067fc700 1 -- 172.21.15.33:0/3611931588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c004bf0 con 0x7f66100ffd80 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.594+0000 7f6617803700 1 -- 172.21.15.33:0/3611931588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c0095f0 con 0x7f66100ffd80 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.594+0000 7f6617803700 1 -- 172.21.15.33:0/3611931588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661019df50 con 0x7f66100ffd80 2024-05-08T21:28:25.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.595+0000 7f6617803700 1 -- 172.21.15.33:0/3611931588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66101072f0 con 0x7f66100ffd80 2024-05-08T21:28:25.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.596+0000 7f66067fc700 1 -- 172.21.15.33:0/3611931588 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f660c004d50 con 0x7f66100ffd80 2024-05-08T21:28:25.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.597+0000 7f66067fc700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65fc0a1120 0x7f65fc0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:25.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.597+0000 7f66067fc700 1 -- 172.21.15.33:0/3611931588 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f660c0cdb00 con 0x7f66100ffd80 2024-05-08T21:28:25.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.597+0000 7f6614d9e700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65fc0a1120 0x7f65fc0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:25.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.597+0000 7f6614d9e700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65fc0a1120 0x7f65fc0a35e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6600009e40 tx=0x7f6600009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:25.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.601+0000 7f66067fc700 1 -- 172.21.15.33:0/3611931588 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f660c096f60 con 0x7f66100ffd80 2024-05-08T21:28:25.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:25 smithi071 ceph-mon[47074]: pgmap v573: 129 pgs: 129 active+clean; 250 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 738 KiB/s wr, 304 op/s 2024-05-08T21:28:25.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.827+0000 7f6617803700 1 -- 172.21.15.33:0/3611931588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f661010b440 con 0x7f66100ffd80 2024-05-08T21:28:25.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.829+0000 7f66067fc700 1 -- 172.21.15.33:0/3611931588 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24561 (secure 0 0 0) 0x7f660c09a580 con 0x7f66100ffd80 2024-05-08T21:28:25.830 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:28:25.831 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":89,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:18:58.419274+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:28:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.831+0000 7f6617803700 1 -- 172.21.15.33:0/3611931588 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65fc0a1120 msgr2=0x7f65fc0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.831+0000 7f6617803700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65fc0a1120 0x7f65fc0a35e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6600009e40 tx=0x7f6600009260 comp rx=0 tx=0).stop 2024-05-08T21:28:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.831+0000 7f6617803700 1 -- 172.21.15.33:0/3611931588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100ffd80 msgr2=0x7f6610195c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.831+0000 7f6617803700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100ffd80 0x7f6610195c70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f660c006010 tx=0x7f660c00f690 comp rx=0 tx=0).stop 2024-05-08T21:28:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.831+0000 7f6617803700 1 -- 172.21.15.33:0/3611931588 shutdown_connections 2024-05-08T21:28:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.831+0000 7f6617803700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65fc0a1120 0x7f65fc0a35e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.831+0000 7f6617803700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6610101910 0x7f661019d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.831+0000 7f6617803700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66100ffd80 0x7f6610195c70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.831+0000 7f6617803700 1 --2- 172.21.15.33:0/3611931588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6610100f70 0x7f66101961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.831+0000 7f6617803700 1 -- 172.21.15.33:0/3611931588 >> 172.21.15.33:0/3611931588 conn(0x7f66100fb310 msgr2=0x7f6610104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.831+0000 7f6617803700 1 -- 172.21.15.33:0/3611931588 shutdown_connections 2024-05-08T21:28:25.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.831+0000 7f6617803700 1 -- 172.21.15.33:0/3611931588 wait complete. 2024-05-08T21:28:25.845 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:25 smithi033 ceph-mon[42528]: pgmap v573: 129 pgs: 129 active+clean; 250 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 738 KiB/s wr, 304 op/s 2024-05-08T21:28:25.847 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:17:01.282803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:17:02.555552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T21:17:05.686277+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T21:28:25.847 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:28:25.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:25 smithi082 ceph-mon[47512]: pgmap v573: 129 pgs: 129 active+clean; 250 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 738 KiB/s wr, 304 op/s 2024-05-08T21:28:26.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.999+0000 7fd9c14d7700 1 -- 172.21.15.33:0/219216675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9bc104230 msgr2=0x7fd9bc106620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.999+0000 7fd9c14d7700 1 --2- 172.21.15.33:0/219216675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9bc104230 0x7fd9bc106620 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009990 tx=0x7fd9a4009ca0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.999+0000 7fd9c14d7700 1 -- 172.21.15.33:0/219216675 shutdown_connections 2024-05-08T21:28:26.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.999+0000 7fd9c14d7700 1 --2- 172.21.15.33:0/219216675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9bc106b60 0x7fd9bc108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.999+0000 7fd9c14d7700 1 --2- 172.21.15.33:0/219216675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9bc101900 0x7fd9bc103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.999+0000 7fd9c14d7700 1 --2- 172.21.15.33:0/219216675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9bc104230 0x7fd9bc106620 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.999+0000 7fd9c14d7700 1 -- 172.21.15.33:0/219216675 >> 172.21.15.33:0/219216675 conn(0x7fd9bc0fb310 msgr2=0x7fd9bc0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.999+0000 7fd9c14d7700 1 -- 172.21.15.33:0/219216675 shutdown_connections 2024-05-08T21:28:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:25.999+0000 7fd9c14d7700 1 -- 172.21.15.33:0/219216675 wait complete. 2024-05-08T21:28:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.000+0000 7fd9c14d7700 1 Processor -- start 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.000+0000 7fd9c14d7700 1 -- start start 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.000+0000 7fd9c14d7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9bc101900 0x7fd9bc196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.000+0000 7fd9c14d7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9bc104230 0x7fd9bc197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.000+0000 7fd9c14d7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9bc106b60 0x7fd9bc19e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.000+0000 7fd9c14d7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc19e7c0 con 0x7fd9bc101900 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.000+0000 7fd9c14d7700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc19e930 con 0x7fd9bc104230 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.000+0000 7fd9c14d7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc19eaf0 con 0x7fd9bc106b60 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9baffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9bc101900 0x7fd9bc196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9baffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9bc101900 0x7fd9bc196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44896/0 (socket says 172.21.15.33:44896) 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9baffd700 1 -- 172.21.15.33:0/1118902755 learned_addr learned my addr 172.21.15.33:0/1118902755 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9bb7fe700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9bc106b60 0x7fd9bc19e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9b27fc700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9bc104230 0x7fd9bc197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:26.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9bb7fe700 1 -- 172.21.15.33:0/1118902755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9bc104230 msgr2=0x7fd9bc197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9bb7fe700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9bc104230 0x7fd9bc197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9bb7fe700 1 -- 172.21.15.33:0/1118902755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9bc101900 msgr2=0x7fd9bc196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9bb7fe700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9bc101900 0x7fd9bc196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9bb7fe700 1 -- 172.21.15.33:0/1118902755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a40095f0 con 0x7fd9bc106b60 2024-05-08T21:28:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9baffd700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9bc101900 0x7fd9bc196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9bb7fe700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9bc106b60 0x7fd9bc19e1f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400efc0 tx=0x7fd9b400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9b8ff9700 1 -- 172.21.15.33:0/1118902755 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4018070 con 0x7fd9bc106b60 2024-05-08T21:28:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9c14d7700 1 -- 172.21.15.33:0/1118902755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc19ed70 con 0x7fd9bc106b60 2024-05-08T21:28:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9b8ff9700 1 -- 172.21.15.33:0/1118902755 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd9b400f040 con 0x7fd9bc106b60 2024-05-08T21:28:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9b8ff9700 1 -- 172.21.15.33:0/1118902755 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400cc50 con 0x7fd9bc106b60 2024-05-08T21:28:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.001+0000 7fd9c14d7700 1 -- 172.21.15.33:0/1118902755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc19f1d0 con 0x7fd9bc106b60 2024-05-08T21:28:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.002+0000 7fd9c14d7700 1 -- 172.21.15.33:0/1118902755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc0fce90 con 0x7fd9bc106b60 2024-05-08T21:28:26.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.004+0000 7fd9b8ff9700 1 -- 172.21.15.33:0/1118902755 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fd9b4003a00 con 0x7fd9bc106b60 2024-05-08T21:28:26.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.005+0000 7fd9b8ff9700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd9a80a1120 0x7fd9a80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:26.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.005+0000 7fd9baffd700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd9a80a1120 0x7fd9a80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:26.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.005+0000 7fd9b8ff9700 1 -- 172.21.15.33:0/1118902755 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fd9b40cd200 con 0x7fd9bc106b60 2024-05-08T21:28:26.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.005+0000 7fd9baffd700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd9a80a1120 0x7fd9a80a35e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc198150 tx=0x7fd9ac00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:26.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.008+0000 7fd9b8ff9700 1 -- 172.21.15.33:0/1118902755 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fd9b4096660 con 0x7fd9bc106b60 2024-05-08T21:28:26.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.225+0000 7fd9c14d7700 1 -- 172.21.15.33:0/1118902755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9bc068160 con 0x7fd9bc106b60 2024-05-08T21:28:26.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.226+0000 7fd9b8ff9700 1 -- 172.21.15.33:0/1118902755 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9b4099c80 con 0x7fd9bc106b60 2024-05-08T21:28:26.227 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:28:26.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.228+0000 7fd9c14d7700 1 -- 172.21.15.33:0/1118902755 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd9a80a1120 msgr2=0x7fd9a80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.228+0000 7fd9c14d7700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd9a80a1120 0x7fd9a80a35e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc198150 tx=0x7fd9ac00b500 comp rx=0 tx=0).stop 2024-05-08T21:28:26.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.228+0000 7fd9c14d7700 1 -- 172.21.15.33:0/1118902755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9bc106b60 msgr2=0x7fd9bc19e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.228+0000 7fd9c14d7700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9bc106b60 0x7fd9bc19e1f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400efc0 tx=0x7fd9b400c3a0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.228+0000 7fd9c14d7700 1 -- 172.21.15.33:0/1118902755 shutdown_connections 2024-05-08T21:28:26.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.228+0000 7fd9c14d7700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9bc106b60 0x7fd9bc19e1f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.228+0000 7fd9c14d7700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9bc101900 0x7fd9bc196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.228+0000 7fd9c14d7700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fd9a80a1120 0x7fd9a80a35e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.228+0000 7fd9c14d7700 1 --2- 172.21.15.33:0/1118902755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9bc104230 0x7fd9bc197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.228+0000 7fd9c14d7700 1 -- 172.21.15.33:0/1118902755 >> 172.21.15.33:0/1118902755 conn(0x7fd9bc0fb310 msgr2=0x7fd9bc103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:26.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.228+0000 7fd9c14d7700 1 -- 172.21.15.33:0/1118902755 shutdown_connections 2024-05-08T21:28:26.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.228+0000 7fd9c14d7700 1 -- 172.21.15.33:0/1118902755 wait complete. 2024-05-08T21:28:26.244 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:28:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.383+0000 7fc9076e1700 1 -- 172.21.15.33:0/3514925371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900102980 msgr2=0x7fc9001092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.383+0000 7fc9076e1700 1 --2- 172.21.15.33:0/3514925371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900102980 0x7fc9001092a0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800b0a0 tx=0x7fc8f800b3b0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.383+0000 7fc9076e1700 1 -- 172.21.15.33:0/3514925371 shutdown_connections 2024-05-08T21:28:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.383+0000 7fc9076e1700 1 --2- 172.21.15.33:0/3514925371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900102980 0x7fc9001092a0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.383+0000 7fc9076e1700 1 --2- 172.21.15.33:0/3514925371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc900100e70 0x7fc900101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.383+0000 7fc9076e1700 1 --2- 172.21.15.33:0/3514925371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc900101fc0 0x7fc900102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.383+0000 7fc9076e1700 1 -- 172.21.15.33:0/3514925371 >> 172.21.15.33:0/3514925371 conn(0x7fc9000fc400 msgr2=0x7fc9000fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.383+0000 7fc9076e1700 1 -- 172.21.15.33:0/3514925371 shutdown_connections 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.384+0000 7fc9076e1700 1 -- 172.21.15.33:0/3514925371 wait complete. 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.384+0000 7fc9076e1700 1 Processor -- start 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.384+0000 7fc9076e1700 1 -- start start 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.384+0000 7fc9076e1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc900100e70 0x7fc900196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.384+0000 7fc9076e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900101fc0 0x7fc9001972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.384+0000 7fc9076e1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc900102980 0x7fc90019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.384+0000 7fc9076e1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90019e8e0 con 0x7fc900101fc0 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.384+0000 7fc9076e1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90019ea20 con 0x7fc900100e70 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.384+0000 7fc9076e1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90019ebb0 con 0x7fc900102980 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.384+0000 7fc904c7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900101fc0 0x7fc9001972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc904c7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900101fc0 0x7fc9001972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44902/0 (socket says 172.21.15.33:44902) 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc904c7c700 1 -- 172.21.15.33:0/2435876011 learned_addr learned my addr 172.21.15.33:0/2435876011 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc905c7e700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc900102980 0x7fc90019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:26.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc90547d700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc900100e70 0x7fc900196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc904c7c700 1 -- 172.21.15.33:0/2435876011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc900102980 msgr2=0x7fc90019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc904c7c700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc900102980 0x7fc90019e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc904c7c700 1 -- 172.21.15.33:0/2435876011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc900100e70 msgr2=0x7fc900196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc904c7c700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc900100e70 0x7fc900196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc904c7c700 1 -- 172.21.15.33:0/2435876011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f8009d00 con 0x7fc900101fc0 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc90547d700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc900100e70 0x7fc900196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc905c7e700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc900102980 0x7fc90019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc904c7c700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900101fc0 0x7fc9001972f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000d950 tx=0x7fc8f000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc8f67fc700 1 -- 172.21.15.33:0/2435876011 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0009b40 con 0x7fc900101fc0 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc8f67fc700 1 -- 172.21.15.33:0/2435876011 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8f0009ca0 con 0x7fc900101fc0 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc9076e1700 1 -- 172.21.15.33:0/2435876011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90019ee90 con 0x7fc900101fc0 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc8f67fc700 1 -- 172.21.15.33:0/2435876011 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000f620 con 0x7fc900101fc0 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.385+0000 7fc9076e1700 1 -- 172.21.15.33:0/2435876011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90019f310 con 0x7fc900101fc0 2024-05-08T21:28:26.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.386+0000 7fc9076e1700 1 -- 172.21.15.33:0/2435876011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc900108290 con 0x7fc900101fc0 2024-05-08T21:28:26.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.387+0000 7fc8f67fc700 1 -- 172.21.15.33:0/2435876011 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc8f0010460 con 0x7fc900101fc0 2024-05-08T21:28:26.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.388+0000 7fc8f67fc700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc8ec0a1120 0x7fc8ec0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.388+0000 7fc8f67fc700 1 -- 172.21.15.33:0/2435876011 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc8f00ccf90 con 0x7fc900101fc0 2024-05-08T21:28:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.388+0000 7fc90547d700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc8ec0a1120 0x7fc8ec0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.389+0000 7fc90547d700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc8ec0a1120 0x7fc8ec0a35e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009cb0 tx=0x7fc8fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.392+0000 7fc8f67fc700 1 -- 172.21.15.33:0/2435876011 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc8f00963a0 con 0x7fc900101fc0 2024-05-08T21:28:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.612+0000 7fc9076e1700 1 -- 172.21.15.33:0/2435876011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc90010c520 con 0x7fc900101fc0 2024-05-08T21:28:26.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.613+0000 7fc8f67fc700 1 -- 172.21.15.33:0/2435876011 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8f0020020 con 0x7fc900101fc0 2024-05-08T21:28:26.614 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:28:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.615+0000 7fc9076e1700 1 -- 172.21.15.33:0/2435876011 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc8ec0a1120 msgr2=0x7fc8ec0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.615+0000 7fc9076e1700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc8ec0a1120 0x7fc8ec0a35e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009cb0 tx=0x7fc8fc009260 comp rx=0 tx=0).stop 2024-05-08T21:28:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.615+0000 7fc9076e1700 1 -- 172.21.15.33:0/2435876011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900101fc0 msgr2=0x7fc9001972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.615+0000 7fc9076e1700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900101fc0 0x7fc9001972f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000d950 tx=0x7fc8f000dd10 comp rx=0 tx=0).stop 2024-05-08T21:28:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.615+0000 7fc9076e1700 1 -- 172.21.15.33:0/2435876011 shutdown_connections 2024-05-08T21:28:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.615+0000 7fc9076e1700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc900102980 0x7fc90019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.615+0000 7fc9076e1700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc8ec0a1120 0x7fc8ec0a35e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.615+0000 7fc9076e1700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc900100e70 0x7fc900196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.615+0000 7fc9076e1700 1 --2- 172.21.15.33:0/2435876011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc900101fc0 0x7fc9001972f0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.615+0000 7fc9076e1700 1 -- 172.21.15.33:0/2435876011 >> 172.21.15.33:0/2435876011 conn(0x7fc9000fc400 msgr2=0x7fc9001053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.615+0000 7fc9076e1700 1 -- 172.21.15.33:0/2435876011 shutdown_connections 2024-05-08T21:28:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.615+0000 7fc9076e1700 1 -- 172.21.15.33:0/2435876011 wait complete. 2024-05-08T21:28:26.631 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:28:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.770+0000 7f13fcf9e700 1 -- 172.21.15.33:0/2811434742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f8102020 msgr2=0x7f13f8102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.770+0000 7f13fcf9e700 1 --2- 172.21.15.33:0/2811434742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f8102020 0x7f13f8102480 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009990 tx=0x7f13e8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.770+0000 7f13fcf9e700 1 -- 172.21.15.33:0/2811434742 shutdown_connections 2024-05-08T21:28:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.770+0000 7f13fcf9e700 1 --2- 172.21.15.33:0/2811434742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f81029c0 0x7f13f8109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.770+0000 7f13fcf9e700 1 --2- 172.21.15.33:0/2811434742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f80fefb0 0x7f13f80ff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.770+0000 7f13fcf9e700 1 --2- 172.21.15.33:0/2811434742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f8102020 0x7f13f8102480 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.770+0000 7f13fcf9e700 1 -- 172.21.15.33:0/2811434742 >> 172.21.15.33:0/2811434742 conn(0x7f13f80fa8c0 msgr2=0x7f13f80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.770+0000 7f13fcf9e700 1 -- 172.21.15.33:0/2811434742 shutdown_connections 2024-05-08T21:28:26.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.770+0000 7f13fcf9e700 1 -- 172.21.15.33:0/2811434742 wait complete. 2024-05-08T21:28:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.771+0000 7f13fcf9e700 1 Processor -- start 2024-05-08T21:28:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.771+0000 7f13fcf9e700 1 -- start start 2024-05-08T21:28:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13fcf9e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f80fefb0 0x7f13f8101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13fcf9e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f8102020 0x7f13f80fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13fcf9e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f81029c0 0x7f13f8100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13fcf9e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f8100cb0 con 0x7f13f80fefb0 2024-05-08T21:28:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13fcf9e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f8100e20 con 0x7f13f81029c0 2024-05-08T21:28:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13fcf9e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f81a0ae0 con 0x7f13f8102020 2024-05-08T21:28:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f80fefb0 0x7f13f8101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f80fefb0 0x7f13f8101940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44918/0 (socket says 172.21.15.33:44918) 2024-05-08T21:28:26.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f77fe700 1 -- 172.21.15.33:0/1176841386 learned_addr learned my addr 172.21.15.33:0/1176841386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f7fff700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f81029c0 0x7f13f8100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f6ffd700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f8102020 0x7f13f80fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f77fe700 1 -- 172.21.15.33:0/1176841386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f8102020 msgr2=0x7f13f80fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f77fe700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f8102020 0x7f13f80fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f77fe700 1 -- 172.21.15.33:0/1176841386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f81029c0 msgr2=0x7f13f8100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f77fe700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f81029c0 0x7f13f8100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f77fe700 1 -- 172.21.15.33:0/1176841386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e80095f0 con 0x7f13f80fefb0 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f6ffd700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f8102020 0x7f13f80fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f77fe700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f80fefb0 0x7f13f8101940 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00d980 tx=0x7f13ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.772+0000 7f13f4ff9700 1 -- 172.21.15.33:0/1176841386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec004020 con 0x7f13f80fefb0 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.773+0000 7f13f4ff9700 1 -- 172.21.15.33:0/1176841386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f13ec0096d0 con 0x7f13f80fefb0 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.773+0000 7f13fcf9e700 1 -- 172.21.15.33:0/1176841386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f81a0d70 con 0x7f13f80fefb0 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.773+0000 7f13f4ff9700 1 -- 172.21.15.33:0/1176841386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec003a00 con 0x7f13f80fefb0 2024-05-08T21:28:26.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.773+0000 7f13fcf9e700 1 -- 172.21.15.33:0/1176841386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f81a11d0 con 0x7f13f80fefb0 2024-05-08T21:28:26.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.774+0000 7f13fcf9e700 1 -- 172.21.15.33:0/1176841386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f8108250 con 0x7f13f80fefb0 2024-05-08T21:28:26.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.775+0000 7f13f4ff9700 1 -- 172.21.15.33:0/1176841386 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f13ec009840 con 0x7f13f80fefb0 2024-05-08T21:28:26.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.775+0000 7f13f4ff9700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13e00a11c0 0x7f13e00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:26.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.776+0000 7f13f6ffd700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13e00a11c0 0x7f13e00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:26.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.776+0000 7f13f4ff9700 1 -- 172.21.15.33:0/1176841386 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f13ec0cccf0 con 0x7f13f80fefb0 2024-05-08T21:28:26.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.776+0000 7f13f6ffd700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13e00a11c0 0x7f13e00a3680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f13e8003780 tx=0x7f13e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:26.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.780+0000 7f13f4ff9700 1 -- 172.21.15.33:0/1176841386 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f13ec0960c0 con 0x7f13f80fefb0 2024-05-08T21:28:26.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:26 smithi071 ceph-mon[47074]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:26.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:26 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3611931588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:28:26.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:26 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1118902755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:28:26.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:26 smithi082 ceph-mon[47512]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:26.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:26 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3611931588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:28:26.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:26 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1118902755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:28:26.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:26 smithi033 ceph-mon[42528]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T21:28:26.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:26 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3611931588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:28:26.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:26 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1118902755' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:28:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.996+0000 7f13fcf9e700 1 -- 172.21.15.33:0/1176841386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13f810c3a0 con 0x7f13f80fefb0 2024-05-08T21:28:26.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.997+0000 7f13f4ff9700 1 -- 172.21.15.33:0/1176841386 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13ec0996e0 con 0x7f13f80fefb0 2024-05-08T21:28:26.998 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T21:28:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.999+0000 7f13fcf9e700 1 -- 172.21.15.33:0/1176841386 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13e00a11c0 msgr2=0x7f13e00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.999+0000 7f13fcf9e700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13e00a11c0 0x7f13e00a3680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f13e8003780 tx=0x7f13e8005fd0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.999+0000 7f13fcf9e700 1 -- 172.21.15.33:0/1176841386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f80fefb0 msgr2=0x7f13f8101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.999+0000 7f13fcf9e700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f80fefb0 0x7f13f8101940 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00d980 tx=0x7f13ec00dc90 comp rx=0 tx=0).stop 2024-05-08T21:28:27.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.999+0000 7f13fcf9e700 1 -- 172.21.15.33:0/1176841386 shutdown_connections 2024-05-08T21:28:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.999+0000 7f13fcf9e700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f13e00a11c0 0x7f13e00a3680 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.999+0000 7f13fcf9e700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13f81029c0 0x7f13f8100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.999+0000 7f13fcf9e700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13f80fefb0 0x7f13f8101940 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.999+0000 7f13fcf9e700 1 --2- 172.21.15.33:0/1176841386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f8102020 0x7f13f80fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.999+0000 7f13fcf9e700 1 -- 172.21.15.33:0/1176841386 >> 172.21.15.33:0/1176841386 conn(0x7f13f80fa8c0 msgr2=0x7f13f8105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.999+0000 7f13fcf9e700 1 -- 172.21.15.33:0/1176841386 shutdown_connections 2024-05-08T21:28:27.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:26.999+0000 7f13fcf9e700 1 -- 172.21.15.33:0/1176841386 wait complete. 2024-05-08T21:28:27.015 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T21:28:27.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.155+0000 7f8b11715700 1 -- 172.21.15.33:0/609315479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b0c1009c0 msgr2=0x7f8b0c108280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:27.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.155+0000 7f8b11715700 1 --2- 172.21.15.33:0/609315479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b0c1009c0 0x7f8b0c108280 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8b0400b0a0 tx=0x7f8b0400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.155+0000 7f8b11715700 1 -- 172.21.15.33:0/609315479 shutdown_connections 2024-05-08T21:28:27.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.155+0000 7f8b11715700 1 --2- 172.21.15.33:0/609315479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b0c1009c0 0x7f8b0c108280 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.155+0000 7f8b11715700 1 --2- 172.21.15.33:0/609315479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b0c1012f0 0x7f8b0c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.155+0000 7f8b11715700 1 --2- 172.21.15.33:0/609315479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b0c100020 0x7f8b0c100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.155+0000 7f8b11715700 1 -- 172.21.15.33:0/609315479 >> 172.21.15.33:0/609315479 conn(0x7f8b0c0faa50 msgr2=0x7f8b0c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.155+0000 7f8b11715700 1 -- 172.21.15.33:0/609315479 shutdown_connections 2024-05-08T21:28:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.155+0000 7f8b11715700 1 -- 172.21.15.33:0/609315479 wait complete. 2024-05-08T21:28:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.156+0000 7f8b11715700 1 Processor -- start 2024-05-08T21:28:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.156+0000 7f8b11715700 1 -- start start 2024-05-08T21:28:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.156+0000 7f8b11715700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b0c100020 0x7f8b0c196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.156+0000 7f8b11715700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b0c1009c0 0x7f8b0c197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.156+0000 7f8b11715700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b0c1012f0 0x7f8b0c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.156+0000 7f8b11715700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c19e8f0 con 0x7f8b0c1012f0 2024-05-08T21:28:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.156+0000 7f8b11715700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c19ea30 con 0x7f8b0c1009c0 2024-05-08T21:28:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.156+0000 7f8b11715700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c19ebc0 con 0x7f8b0c100020 2024-05-08T21:28:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.156+0000 7f8b0b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b0c1012f0 0x7f8b0c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b0b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b0c1012f0 0x7f8b0c19e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44932/0 (socket says 172.21.15.33:44932) 2024-05-08T21:28:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b027fc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b0c1009c0 0x7f8b0c197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b0b7fe700 1 -- 172.21.15.33:0/155930257 learned_addr learned my addr 172.21.15.33:0/155930257 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b0affd700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b0c100020 0x7f8b0c196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b0b7fe700 1 -- 172.21.15.33:0/155930257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b0c100020 msgr2=0x7f8b0c196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b0b7fe700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b0c100020 0x7f8b0c196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b0b7fe700 1 -- 172.21.15.33:0/155930257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b0c1009c0 msgr2=0x7f8b0c197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b0b7fe700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b0c1009c0 0x7f8b0c197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b0b7fe700 1 -- 172.21.15.33:0/155930257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b04009d00 con 0x7f8b0c1012f0 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b027fc700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b0c1009c0 0x7f8b0c197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b0b7fe700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b0c1012f0 0x7f8b0c19e320 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009820 tx=0x7f8b0400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b08ff9700 1 -- 172.21.15.33:0/155930257 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0400e040 con 0x7f8b0c1012f0 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b08ff9700 1 -- 172.21.15.33:0/155930257 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b04004c50 con 0x7f8b0c1012f0 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b11715700 1 -- 172.21.15.33:0/155930257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c19ee40 con 0x7f8b0c1012f0 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b08ff9700 1 -- 172.21.15.33:0/155930257 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b040052a0 con 0x7f8b0c1012f0 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.157+0000 7f8b11715700 1 -- 172.21.15.33:0/155930257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c19f270 con 0x7f8b0c1012f0 2024-05-08T21:28:27.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.159+0000 7f8b11715700 1 -- 172.21.15.33:0/155930257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0c1072e0 con 0x7f8b0c1012f0 2024-05-08T21:28:27.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.160+0000 7f8b08ff9700 1 -- 172.21.15.33:0/155930257 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f8b0401d470 con 0x7f8b0c1012f0 2024-05-08T21:28:27.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.161+0000 7f8b08ff9700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8af80a11a0 0x7f8af80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.161+0000 7f8b08ff9700 1 -- 172.21.15.33:0/155930257 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8b0401de40 con 0x7f8b0c1012f0 2024-05-08T21:28:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.161+0000 7f8b0affd700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8af80a11a0 0x7f8af80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:27.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.161+0000 7f8b0affd700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8af80a11a0 0x7f8af80a3660 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c067ee0 tx=0x7f8afc007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:27.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.165+0000 7f8b08ff9700 1 -- 172.21.15.33:0/155930257 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8b04097a90 con 0x7f8b0c1012f0 2024-05-08T21:28:27.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.386+0000 7f8b11715700 1 -- 172.21.15.33:0/155930257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8b0c10c4d0 con 0x7f8b0c1012f0 2024-05-08T21:28:27.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.386+0000 7f8b08ff9700 1 -- 172.21.15.33:0/155930257 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8b0409b0b0 con 0x7f8b0c1012f0 2024-05-08T21:28:27.388 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:28:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.389+0000 7f8b11715700 1 -- 172.21.15.33:0/155930257 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8af80a11a0 msgr2=0x7f8af80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.389+0000 7f8b11715700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8af80a11a0 0x7f8af80a3660 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c067ee0 tx=0x7f8afc007400 comp rx=0 tx=0).stop 2024-05-08T21:28:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.389+0000 7f8b11715700 1 -- 172.21.15.33:0/155930257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b0c1012f0 msgr2=0x7f8b0c19e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.389+0000 7f8b11715700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b0c1012f0 0x7f8b0c19e320 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009820 tx=0x7f8b0400bab0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.389+0000 7f8b11715700 1 -- 172.21.15.33:0/155930257 shutdown_connections 2024-05-08T21:28:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.389+0000 7f8b11715700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b0c1012f0 0x7f8b0c19e320 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.389+0000 7f8b11715700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b0c100020 0x7f8b0c196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.389+0000 7f8b11715700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f8af80a11a0 0x7f8af80a3660 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.389+0000 7f8b11715700 1 --2- 172.21.15.33:0/155930257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b0c1009c0 0x7f8b0c197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.389+0000 7f8b11715700 1 -- 172.21.15.33:0/155930257 >> 172.21.15.33:0/155930257 conn(0x7f8b0c0faa50 msgr2=0x7f8b0c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.389+0000 7f8b11715700 1 -- 172.21.15.33:0/155930257 shutdown_connections 2024-05-08T21:28:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.389+0000 7f8b11715700 1 -- 172.21.15.33:0/155930257 wait complete. 2024-05-08T21:28:27.405 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-08T21:28:27.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.543+0000 7f21951b6700 1 -- 172.21.15.33:0/1951613144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21900ffd80 msgr2=0x7f21901001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:27.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.543+0000 7f21951b6700 1 --2- 172.21.15.33:0/1951613144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21900ffd80 0x7f21901001a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f21800099c0 tx=0x7f2180009cd0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.543+0000 7f21951b6700 1 -- 172.21.15.33:0/1951613144 shutdown_connections 2024-05-08T21:28:27.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.543+0000 7f21951b6700 1 --2- 172.21.15.33:0/1951613144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2190101910 0x7f21901081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.543+0000 7f21951b6700 1 --2- 172.21.15.33:0/1951613144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21900ffd80 0x7f21901001a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.544+0000 7f21951b6700 1 --2- 172.21.15.33:0/1951613144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2190100f70 0x7f21901013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.544+0000 7f21951b6700 1 -- 172.21.15.33:0/1951613144 >> 172.21.15.33:0/1951613144 conn(0x7f21900fb310 msgr2=0x7f21900fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:27.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.544+0000 7f21951b6700 1 -- 172.21.15.33:0/1951613144 shutdown_connections 2024-05-08T21:28:27.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.544+0000 7f21951b6700 1 -- 172.21.15.33:0/1951613144 wait complete. 2024-05-08T21:28:27.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.544+0000 7f21951b6700 1 Processor -- start 2024-05-08T21:28:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f21951b6700 1 -- start start 2024-05-08T21:28:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f21951b6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21900ffd80 0x7f2190103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f21951b6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190100f70 0x7f2190102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f21951b6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2190101910 0x7f2190102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f21951b6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2190102f90 con 0x7f2190100f70 2024-05-08T21:28:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f21951b6700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2190103100 con 0x7f2190101910 2024-05-08T21:28:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f21951b6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21901a0e60 con 0x7f21900ffd80 2024-05-08T21:28:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f218659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190100f70 0x7f2190102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f218659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190100f70 0x7f2190102350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44942/0 (socket says 172.21.15.33:44942) 2024-05-08T21:28:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f218659c700 1 -- 172.21.15.33:0/344936400 learned_addr learned my addr 172.21.15.33:0/344936400 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:27.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f218ed9d700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21900ffd80 0x7f2190103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f218f59e700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2190101910 0x7f2190102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f218659c700 1 -- 172.21.15.33:0/344936400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21900ffd80 msgr2=0x7f2190103c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f218659c700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21900ffd80 0x7f2190103c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f218659c700 1 -- 172.21.15.33:0/344936400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2190101910 msgr2=0x7f2190102890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f218659c700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2190101910 0x7f2190102890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f218659c700 1 -- 172.21.15.33:0/344936400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21800095f0 con 0x7f2190100f70 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.545+0000 7f218f59e700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2190101910 0x7f2190102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.546+0000 7f218659c700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190100f70 0x7f2190102350 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f217800e9a0 tx=0x7f217800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.546+0000 7f218cd99700 1 -- 172.21.15.33:0/344936400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178009790 con 0x7f2190100f70 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.546+0000 7f21951b6700 1 -- 172.21.15.33:0/344936400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21901a10c0 con 0x7f2190100f70 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.546+0000 7f218cd99700 1 -- 172.21.15.33:0/344936400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f217800cbd0 con 0x7f2190100f70 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.546+0000 7f218cd99700 1 -- 172.21.15.33:0/344936400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2178018680 con 0x7f2190100f70 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.546+0000 7f21951b6700 1 -- 172.21.15.33:0/344936400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21901a15d0 con 0x7f2190100f70 2024-05-08T21:28:27.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.547+0000 7f21951b6700 1 -- 172.21.15.33:0/344936400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2190067d10 con 0x7f2190100f70 2024-05-08T21:28:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.548+0000 7f218cd99700 1 -- 172.21.15.33:0/344936400 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f217800cd40 con 0x7f2190100f70 2024-05-08T21:28:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.549+0000 7f218cd99700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f217c0a1120 0x7f217c0a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.549+0000 7f218ed9d700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f217c0a1120 0x7f217c0a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:27.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.549+0000 7f218cd99700 1 -- 172.21.15.33:0/344936400 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f2178014070 con 0x7f2190100f70 2024-05-08T21:28:27.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.549+0000 7f218ed9d700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f217c0a1120 0x7f217c0a35e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2180009990 tx=0x7f2180003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.554+0000 7f218cd99700 1 -- 172.21.15.33:0/344936400 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2178052d60 con 0x7f2190100f70 2024-05-08T21:28:27.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:27 smithi033 ceph-mon[42528]: pgmap v574: 129 pgs: 129 active+clean; 240 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 767 KiB/s wr, 301 op/s 2024-05-08T21:28:27.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:27 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2435876011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:28:27.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:27 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1176841386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:28:27.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:27 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/155930257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:28:27.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:27 smithi071 ceph-mon[47074]: pgmap v574: 129 pgs: 129 active+clean; 240 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 767 KiB/s wr, 301 op/s 2024-05-08T21:28:27.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:27 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2435876011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:28:27.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:27 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1176841386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:28:27.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:27 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/155930257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:28:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.842+0000 7f21951b6700 1 -- 172.21.15.33:0/344936400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f219010b490 con 0x7f2190100f70 2024-05-08T21:28:27.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:27 smithi082 ceph-mon[47512]: pgmap v574: 129 pgs: 129 active+clean; 240 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 767 KiB/s wr, 301 op/s 2024-05-08T21:28:27.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:27 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2435876011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:28:27.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:27 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1176841386' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:28:27.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:27 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/155930257' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:28:27.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.853+0000 7f218cd99700 1 -- 172.21.15.33:0/344936400 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733410 (secure 0 0 0) 0x7f21780962e0 con 0x7f2190100f70 2024-05-08T21:28:27.865 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:28:27.865 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:28:27.865 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 18, 2024-05-08T21:28:27.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.865 INFO:teuthology.orchestra.run.smithi033.stdout: "active_gid": 14152, 2024-05-08T21:28:27.865 INFO:teuthology.orchestra.run.smithi033.stdout: "active_name": "x", 2024-05-08T21:28:27.865 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addrs": { 2024-05-08T21:28:27.865 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T21:28:27.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:27.865 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T21:28:27.866 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6800", 2024-05-08T21:28:27.866 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2288189532 2024-05-08T21:28:27.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.866 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:27.866 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-05-08T21:28:27.866 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6801", 2024-05-08T21:28:27.866 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2288189532 2024-05-08T21:28:27.866 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:27.866 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addr": "172.21.15.33:6801/2288189532", 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: "active_change": "2024-05-08T21:11:56.047173+0000", 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: "active_mgr_features": 4540138322906710015, 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: "available": true, 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: "standbys": [ 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 24115, 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "z", 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540138322906710015, 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-05-08T21:28:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.868 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.869 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:27.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:27.869 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.869 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.869 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.869 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.869 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.869 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.869 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:27.869 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:27.869 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.870 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.871 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.872 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:27.872 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.872 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.872 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:27.872 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:27.872 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:27.872 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:27.872 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:27.872 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.873 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.875 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-05-08T21:28:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-05-08T21:28:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-05-08T21:28:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.876 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.877 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.878 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.879 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-05-08T21:28:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-05-08T21:28:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-05-08T21:28:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.881 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:27.882 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T21:28:27.883 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T21:28:27.884 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T21:28:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T21:28:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T21:28:27.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T21:28:27.888 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:27.891 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.892 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:27.893 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.895 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap" 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-05-08T21:28:27.897 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T21:28:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T21:28:27.899 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T21:28:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-05-08T21:28:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-05-08T21:28:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.901 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-05-08T21:28:27.902 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-05-08T21:28:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-05-08T21:28:27.904 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.905 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-05-08T21:28:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-05-08T21:28:27.908 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-05-08T21:28:27.909 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T21:28:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-05-08T21:28:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-05-08T21:28:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-05-08T21:28:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.915 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.919 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.921 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.922 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.926 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.927 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.929 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-05-08T21:28:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.931 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-05-08T21:28:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-05-08T21:28:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-05-08T21:28:27.932 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-05-08T21:28:27.933 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T21:28:27.934 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.935 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.937 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.938 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:27.939 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T21:28:27.941 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T21:28:27.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-05-08T21:28:27.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.944 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.945 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-05-08T21:28:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T21:28:27.948 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.949 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.950 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T21:28:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-05-08T21:28:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-05-08T21:28:27.958 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.959 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-05-08T21:28:27.960 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.964 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.967 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.968 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T21:28:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.974 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.975 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.976 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.977 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.980 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.981 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.982 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-05-08T21:28:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-05-08T21:28:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.986 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.987 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.988 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.989 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.992 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.996 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:27.999 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T21:28:28.001 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T21:28:28.003 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T21:28:28.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T21:28:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T21:28:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T21:28:28.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T21:28:28.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T21:28:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T21:28:28.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.011 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.013 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.014 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.015 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.016 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.019 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.020 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-05-08T21:28:28.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.024 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.025 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.026 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.028 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-08T21:28:28.029 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.030 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.031 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.032 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.033 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.034 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-05-08T21:28:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.040 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.041 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.042 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.045 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.046 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.047 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-08T21:28:28.049 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-05-08T21:28:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.051 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-05-08T21:28:28.052 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-05-08T21:28:28.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-05-08T21:28:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.055 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.056 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.056 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-05-08T21:28:28.056 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-05-08T21:28:28.056 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.056 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.056 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T21:28:28.056 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.056 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.056 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.056 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.056 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.057 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.061 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-05-08T21:28:28.062 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.064 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.065 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T21:28:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.068 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.069 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.074 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.076 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.078 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.081 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.082 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.083 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.084 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.085 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.091 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T21:28:28.093 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-05-08T21:28:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-05-08T21:28:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.095 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.096 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.097 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.100 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.101 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.103 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.104 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.105 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.106 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.110 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-05-08T21:28:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.114 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.115 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.117 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.118 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.120 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.122 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.123 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.124 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.126 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.127 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.128 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.130 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.131 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.133 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.134 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.135 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T21:28:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.138 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.139 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.140 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.141 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.142 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-05-08T21:28:28.144 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.145 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.146 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.147 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.150 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.151 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.152 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.154 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.156 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-08T21:28:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-08T21:28:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-05-08T21:28:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.157 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.158 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.159 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-08T21:28:28.160 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.161 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.163 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.164 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.165 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.166 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.166 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.166 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.166 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.166 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.166 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.166 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.166 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-05-08T21:28:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-05-08T21:28:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-05-08T21:28:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-05-08T21:28:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-05-08T21:28:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.191 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.192 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-05-08T21:28:28.193 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.195 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.199 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-05-08T21:28:28.204 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-05-08T21:28:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-05-08T21:28:28.206 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.207 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.211 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-05-08T21:28:28.212 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.213 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.214 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.216 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.217 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.218 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.220 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-05-08T21:28:28.223 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.224 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.229 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.230 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.231 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.232 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.235 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-05-08T21:28:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.238 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.240 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.241 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.242 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.245 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.247 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.248 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.249 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.251 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.252 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-05-08T21:28:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-05-08T21:28:28.254 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.255 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.256 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.257 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.258 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.258 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.258 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.258 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.258 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.258 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.258 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.258 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.258 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.258 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.258 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.259 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.260 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.264 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.267 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T21:28:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.269 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-05-08T21:28:28.270 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.270 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.270 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.270 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-05-08T21:28:28.270 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-05-08T21:28:28.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:28.270 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.270 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.270 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.271 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.274 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.276 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.277 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.278 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.280 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 24119, 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "y", 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540138322906710015, 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-05-08T21:28:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.282 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.286 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.289 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.291 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.292 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.293 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.294 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.295 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.296 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.297 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.297 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.297 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.297 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.297 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T21:28:28.297 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.297 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.297 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-05-08T21:28:28.297 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-05-08T21:28:28.298 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-05-08T21:28:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.300 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.302 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.304 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.305 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.306 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.309 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-05-08T21:28:28.310 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap" 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.311 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.312 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T21:28:28.313 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T21:28:28.314 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-05-08T21:28:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.319 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.320 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.321 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T21:28:28.322 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.322 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.322 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.322 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-05-08T21:28:28.322 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-05-08T21:28:28.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.322 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.322 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.322 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.322 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T21:28:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T21:28:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T21:28:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T21:28:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T21:28:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T21:28:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T21:28:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T21:28:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.338 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.339 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T21:28:28.340 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.341 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-05-08T21:28:28.344 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-05-08T21:28:28.345 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-05-08T21:28:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.347 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.349 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.351 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.352 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.353 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.357 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.358 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-05-08T21:28:28.359 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.361 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.362 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T21:28:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.365 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.366 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-05-08T21:28:28.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-05-08T21:28:28.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-05-08T21:28:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-05-08T21:28:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.371 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.372 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-05-08T21:28:28.378 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-05-08T21:28:28.379 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.381 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.382 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.384 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.385 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.388 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.389 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.390 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T21:28:28.391 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T21:28:28.392 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T21:28:28.393 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T21:28:28.394 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.396 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.398 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.399 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.400 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.401 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.402 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.403 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.405 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.406 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.407 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.408 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.410 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.412 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T21:28:28.416 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T21:28:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T21:28:28.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T21:28:28.419 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T21:28:28.420 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T21:28:28.421 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T21:28:28.422 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-05-08T21:28:28.423 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.424 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.427 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.428 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.429 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.430 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.431 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.432 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.433 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.434 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.436 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.437 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-08T21:28:28.440 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.441 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.442 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.444 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.445 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.451 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-05-08T21:28:28.452 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-05-08T21:28:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T21:28:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.454 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.455 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.459 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.460 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.461 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-05-08T21:28:28.464 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.465 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.466 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.468 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.469 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.470 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-05-08T21:28:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-05-08T21:28:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-05-08T21:28:28.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.480 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.481 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.482 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.486 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.489 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.490 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.491 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.495 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-05-08T21:28:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.497 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.498 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.502 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.503 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.504 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-05-08T21:28:28.505 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.506 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.508 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.509 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.510 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.511 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.511 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.511 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.511 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.511 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.512 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.513 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.519 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.521 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.522 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-05-08T21:28:28.523 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.525 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.526 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.528 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.531 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.535 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.539 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.541 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.542 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.543 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-08T21:28:28.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-05-08T21:28:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-08T21:28:28.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.556 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-05-08T21:28:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.558 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T21:28:28.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.560 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.563 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T21:28:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-05-08T21:28:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-08T21:28:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-08T21:28:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-05-08T21:28:28.569 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.570 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.572 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.573 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.576 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.582 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.582 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-05-08T21:28:28.582 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-05-08T21:28:28.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.582 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.582 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.582 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-05-08T21:28:28.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-08T21:28:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.585 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.586 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.588 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.591 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.592 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.593 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.595 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.596 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.596 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.596 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.596 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.596 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.596 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.596 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.596 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.596 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.597 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.597 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.597 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.597 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.597 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.597 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.608 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.610 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-05-08T21:28:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-05-08T21:28:28.614 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.617 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.621 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.622 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.626 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.627 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.629 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.632 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.634 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.635 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.636 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.638 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.640 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.641 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.643 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.644 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-05-08T21:28:28.646 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.648 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.649 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.654 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.655 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.660 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.663 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-05-08T21:28:28.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-05-08T21:28:28.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-05-08T21:28:28.667 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.670 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.670 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.670 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.670 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.670 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.670 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.670 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.670 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.671 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.675 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.676 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.678 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.680 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-05-08T21:28:28.681 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-05-08T21:28:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.689 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.691 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: "modules": [ 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard", 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: "iostat", 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: "nfs", 2024-05-08T21:28:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: "restful" 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.696 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.700 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.702 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-05-08T21:28:28.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-05-08T21:28:28.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-05-08T21:28:28.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T21:28:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T21:28:28.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T21:28:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.710 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-05-08T21:28:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-05-08T21:28:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T21:28:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T21:28:28.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T21:28:28.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.712 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.712 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-05-08T21:28:28.712 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-05-08T21:28:28.712 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.712 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.712 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.712 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-05-08T21:28:28.712 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-05-08T21:28:28.712 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.713 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.718 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.718 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.718 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.718 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.718 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.718 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.718 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.718 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap" 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.722 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.724 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-05-08T21:28:28.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T21:28:28.729 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T21:28:28.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.731 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.733 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-05-08T21:28:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-05-08T21:28:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T21:28:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.736 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.737 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.739 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-05-08T21:28:28.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-05-08T21:28:28.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-05-08T21:28:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T21:28:28.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T21:28:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-05-08T21:28:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-05-08T21:28:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-05-08T21:28:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.748 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T21:28:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-08T21:28:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-05-08T21:28:28.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-05-08T21:28:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-05-08T21:28:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-05-08T21:28:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.750 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.751 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-05-08T21:28:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-05-08T21:28:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T21:28:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-05-08T21:28:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T21:28:28.755 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T21:28:28.757 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.758 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T21:28:28.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.760 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.764 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.765 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.767 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T21:28:28.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.767 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.767 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.767 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.767 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.767 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.767 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.768 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.771 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-05-08T21:28:28.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-05-08T21:28:28.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.776 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T21:28:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.778 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.779 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T21:28:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-05-08T21:28:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-05-08T21:28:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.780 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.781 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.782 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.783 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T21:28:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.786 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.788 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.789 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.790 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.792 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.793 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.796 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.798 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.799 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T21:28:28.799 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.799 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.799 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-05-08T21:28:28.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-05-08T21:28:28.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.799 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.799 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.799 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-05-08T21:28:28.799 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.800 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.801 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T21:28:28.802 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T21:28:28.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.802 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.802 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.802 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.802 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.802 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.802 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.802 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.802 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.802 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.807 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T21:28:28.810 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T21:28:28.811 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T21:28:28.812 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T21:28:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T21:28:28.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.815 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.816 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.819 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.821 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.822 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.827 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.829 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.830 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.834 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T21:28:28.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T21:28:28.836 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T21:28:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T21:28:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.840 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.841 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.843 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.844 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.851 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-05-08T21:28:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.855 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-05-08T21:28:28.857 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-08T21:28:28.858 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.863 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.864 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.865 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.866 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.869 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-05-08T21:28:28.871 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.872 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.873 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.877 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.879 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-05-08T21:28:28.880 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-05-08T21:28:28.885 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T21:28:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.887 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.889 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-05-08T21:28:28.892 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-05-08T21:28:28.892 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.892 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.892 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.892 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.892 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.892 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.892 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.892 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-05-08T21:28:28.892 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.892 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-05-08T21:28:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T21:28:28.894 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.896 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.897 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.901 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.902 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.904 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.907 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-05-08T21:28:28.908 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.911 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.912 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.914 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.916 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.917 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.920 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-05-08T21:28:28.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-05-08T21:28:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.931 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.932 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.934 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.935 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.941 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.942 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.944 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.947 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.949 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.956 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.958 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.961 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.963 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:28.965 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.966 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.969 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.971 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.972 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.974 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.976 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.977 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:28.978 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.979 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.979 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.979 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T21:28:28.979 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T21:28:28.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.979 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.979 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.979 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.979 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.980 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.984 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.985 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.986 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.987 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.989 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.992 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.993 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.995 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:28.996 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:28.997 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-05-08T21:28:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.000 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:29.001 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.002 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.003 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.004 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.005 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.010 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.011 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.012 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.012 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.012 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.012 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.012 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.012 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.012 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.012 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.012 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.013 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.013 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.013 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.013 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.013 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.013 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.013 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:29.013 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:29.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.013 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.014 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.014 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.014 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.014 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.014 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.014 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.014 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.014 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.015 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:29.015 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:29.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.015 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.015 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.015 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:29.015 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.015 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.015 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.015 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.016 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.016 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.016 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.016 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.016 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.016 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.016 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.016 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.016 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.016 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.018 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:29.019 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.021 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.024 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.025 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.026 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.027 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.029 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.032 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.034 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.035 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.038 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.039 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.039 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.039 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-05-08T21:28:29.039 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-05-08T21:28:29.039 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.039 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.039 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.039 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.039 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.039 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.040 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.041 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.041 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.041 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.041 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.041 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.041 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.041 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.041 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-05-08T21:28:29.041 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T21:28:29.042 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.043 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.043 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.043 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-05-08T21:28:29.043 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-05-08T21:28:29.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.043 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.043 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.043 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.045 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.045 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.045 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.045 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.045 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.045 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.045 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.045 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.045 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.045 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.045 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.046 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.047 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.047 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.047 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.047 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.047 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:29.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:29.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.047 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.047 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.047 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:29.048 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.048 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.048 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.048 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.048 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.048 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.048 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.048 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.048 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.048 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.048 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.048 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.049 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.049 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:29.049 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:29.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.049 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.049 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.050 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.050 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.050 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.050 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.050 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.050 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.050 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.050 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-05-08T21:28:29.051 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:29.051 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:29.051 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:29.051 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:29.051 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:29.051 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.051 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.051 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.051 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.051 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.053 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.054 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.055 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:29.056 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.057 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.059 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-05-08T21:28:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.062 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.064 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.065 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.069 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.070 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.071 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.073 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.075 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.076 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.077 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.077 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T21:28:29.077 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.077 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.077 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.077 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.077 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.077 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.077 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-05-08T21:28:29.077 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-05-08T21:28:29.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.078 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.078 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.078 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.078 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.078 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.078 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.078 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.078 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.078 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.079 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.080 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.080 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-05-08T21:28:29.080 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-05-08T21:28:29.080 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:29.080 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.081 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.082 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.082 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.082 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.082 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.082 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.082 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.082 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.082 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.082 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-05-08T21:28:29.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-05-08T21:28:29.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.083 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.083 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.083 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.083 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.083 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.083 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.083 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.084 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.084 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.084 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.084 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:29.084 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:29.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.084 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.084 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.084 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.086 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.086 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:29.086 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:29.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.086 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.086 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.086 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.087 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.087 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.087 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.087 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.087 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.087 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:29.087 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:29.087 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.088 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.088 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:29.088 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.088 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.088 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.088 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.088 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.088 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.088 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.089 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.089 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.089 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.089 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.089 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.089 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.089 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:29.089 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:29.090 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.090 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.090 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.090 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.090 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.090 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.090 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.090 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.090 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.091 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.091 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.091 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-05-08T21:28:29.091 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-05-08T21:28:29.091 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.091 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.091 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.091 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.091 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.091 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.091 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.093 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.094 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.097 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.098 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:29.099 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.100 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.101 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.102 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:29.103 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-05-08T21:28:29.104 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-05-08T21:28:29.105 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-05-08T21:28:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.108 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.110 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.111 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.113 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.113 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.113 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.113 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.113 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.113 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.113 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.113 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.113 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-05-08T21:28:29.113 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-05-08T21:28:29.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.114 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.114 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.114 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-05-08T21:28:29.114 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.114 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.114 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-05-08T21:28:29.114 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-05-08T21:28:29.114 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-05-08T21:28:29.114 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-05-08T21:28:29.114 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-05-08T21:28:29.115 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-05-08T21:28:29.116 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.116 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.116 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.116 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.116 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.116 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.116 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.117 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-05-08T21:28:29.118 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-05-08T21:28:29.118 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-05-08T21:28:29.118 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.118 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.118 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-05-08T21:28:29.118 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.118 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.118 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.118 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.118 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-05-08T21:28:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-05-08T21:28:29.120 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-05-08T21:28:29.120 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-05-08T21:28:29.120 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-05-08T21:28:29.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.120 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.120 INFO:teuthology.orchestra.run.smithi033.stdout: "services": { 2024-05-08T21:28:29.120 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard": "https://172.21.15.33:8443/" 2024-05-08T21:28:29.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.120 INFO:teuthology.orchestra.run.smithi033.stdout: "always_on_modules": { 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus": [ 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific": [ 2024-05-08T21:28:29.121 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-08T21:28:29.122 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-08T21:28:29.122 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-08T21:28:29.122 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-08T21:28:29.122 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-08T21:28:29.122 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-08T21:28:29.122 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-08T21:28:29.122 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-08T21:28:29.122 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-08T21:28:29.122 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-08T21:28:29.122 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.123 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy": [ 2024-05-08T21:28:29.123 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-08T21:28:29.123 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-08T21:28:29.123 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-08T21:28:29.123 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-08T21:28:29.123 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-08T21:28:29.123 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-08T21:28:29.123 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-08T21:28:29.123 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-08T21:28:29.123 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-08T21:28:29.123 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-08T21:28:29.124 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-05-08T21:28:29.124 INFO:teuthology.orchestra.run.smithi033.stdout: "reef": [ 2024-05-08T21:28:29.124 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-05-08T21:28:29.124 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-05-08T21:28:29.124 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-05-08T21:28:29.124 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-05-08T21:28:29.124 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-05-08T21:28:29.124 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-05-08T21:28:29.124 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-05-08T21:28:29.124 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-05-08T21:28:29.124 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-05-08T21:28:29.124 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-05-08T21:28:29.125 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T21:28:29.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.125 INFO:teuthology.orchestra.run.smithi033.stdout: "last_failure_osd_epoch": 3, 2024-05-08T21:28:29.125 INFO:teuthology.orchestra.run.smithi033.stdout: "active_clients": [ 2024-05-08T21:28:29.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.125 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-05-08T21:28:29.125 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T21:28:29.125 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T21:28:29.125 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-08T21:28:29.125 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 4017296868 2024-05-08T21:28:29.126 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.126 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T21:28:29.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.126 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "libcephsqlite", 2024-05-08T21:28:29.126 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T21:28:29.126 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.126 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T21:28:29.126 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-08T21:28:29.126 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1465921313 2024-05-08T21:28:29.126 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1419901208 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-05-08T21:28:29.127 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-05-08T21:28:29.128 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-05-08T21:28:29.128 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-05-08T21:28:29.128 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-05-08T21:28:29.128 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2546944913 2024-05-08T21:28:29.128 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.128 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T21:28:29.128 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:29.128 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-05-08T21:28:29.128 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:28:29.129 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:28 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/344936400' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T21:28:29.130 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:28 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/344936400' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T21:28:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.859+0000 7f21951b6700 1 -- 172.21.15.33:0/344936400 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f217c0a1120 msgr2=0x7f217c0a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.859+0000 7f21951b6700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f217c0a1120 0x7f217c0a35e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2180009990 tx=0x7f2180003680 comp rx=0 tx=0).stop 2024-05-08T21:28:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.859+0000 7f21951b6700 1 -- 172.21.15.33:0/344936400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190100f70 msgr2=0x7f2190102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.859+0000 7f21951b6700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190100f70 0x7f2190102350 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f217800e9a0 tx=0x7f217800ecb0 comp rx=0 tx=0).stop 2024-05-08T21:28:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.859+0000 7f21951b6700 1 -- 172.21.15.33:0/344936400 shutdown_connections 2024-05-08T21:28:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.859+0000 7f21951b6700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f217c0a1120 0x7f217c0a35e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.859+0000 7f21951b6700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2190101910 0x7f2190102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.859+0000 7f21951b6700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21900ffd80 0x7f2190103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.859+0000 7f21951b6700 1 --2- 172.21.15.33:0/344936400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190100f70 0x7f2190102350 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.859+0000 7f21951b6700 1 -- 172.21.15.33:0/344936400 >> 172.21.15.33:0/344936400 conn(0x7f21900fb310 msgr2=0x7f2190068250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.859+0000 7f21951b6700 1 -- 172.21.15.33:0/344936400 shutdown_connections 2024-05-08T21:28:29.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:27.859+0000 7f21951b6700 1 -- 172.21.15.33:0/344936400 wait complete. 2024-05-08T21:28:29.137 DEBUG:tasks.check_counter:Getting stats from a 2024-05-08T21:28:29.138 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-08T21:28:29.138 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:28 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/344936400' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T21:28:29.392 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:28:29.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:29 smithi071 ceph-mon[47074]: pgmap v575: 129 pgs: 129 active+clean; 236 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 613 KiB/s wr, 264 op/s 2024-05-08T21:28:29.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:29 smithi082 ceph-mon[47512]: pgmap v575: 129 pgs: 129 active+clean; 236 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 613 KiB/s wr, 264 op/s 2024-05-08T21:28:29.852 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:29 smithi033 ceph-mon[42528]: pgmap v575: 129 pgs: 129 active+clean; 236 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 613 KiB/s wr, 264 op/s 2024-05-08T21:28:30.445 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:28:30.445 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T21:28:30.445 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 723, 2024-05-08T21:28:30.445 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 722, 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 12822605, 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 223747, 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 7, 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.089278137, 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.030332062, 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.026863532, 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.023293985, 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 721, 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.051835944, 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000071894 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.446 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 723, 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 702, 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 25885004, 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 265343, 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 4, 2024-05-08T21:28:30.447 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.087734381, 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.022562626, 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.045107702, 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.017954789, 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 701, 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.025486950, 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000036357 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.448 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 590144, 2024-05-08T21:28:30.449 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 107920 2024-05-08T21:28:30.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.449 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T21:28:30.449 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 833, 2024-05-08T21:28:30.449 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 974, 2024-05-08T21:28:30.449 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 14948896, 2024-05-08T21:28:30.449 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 574187, 2024-05-08T21:28:30.449 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 6, 2024-05-08T21:28:30.449 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 5, 2024-05-08T21:28:30.449 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.085578875, 2024-05-08T21:28:30.449 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.025016632, 2024-05-08T21:28:30.450 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.030333218, 2024-05-08T21:28:30.450 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.023853141, 2024-05-08T21:28:30.450 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:30.450 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 972, 2024-05-08T21:28:30.451 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.065635391, 2024-05-08T21:28:30.451 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000067526 2024-05-08T21:28:30.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.451 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:30.451 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.451 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.451 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 315424 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 1, 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-MDSRank": { 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3419, 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.094126279, 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000027530 2024-05-08T21:28:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-PurgeQueue": { 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: "request": 0, 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: "reply": 0, 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: "reply_latency": { 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.454 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.454 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_reply": 0, 2024-05-08T21:28:30.454 INFO:teuthology.orchestra.run.smithi033.stdout: "forward": 0, 2024-05-08T21:28:30.454 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_complete": 0, 2024-05-08T21:28:30.454 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_keys": 0, 2024-05-08T21:28:30.454 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_commit": 0, 2024-05-08T21:28:30.454 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_split": 0, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_merge": 0, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes": 1410, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_top": 0, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_bottom": 1410, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pin_tail": 0, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pinned": 1410, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_expired": 0, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_with_caps": 0, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": 0, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "subtrees": 1, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse": 0, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_hit": 0, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_forward": 0, 2024-05-08T21:28:30.455 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_discover": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_dir_fetch": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_remote_ino": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_lock": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "load_cent": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "q": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "exported": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "exported_inodes": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "imported": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "imported_inodes": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_dir_fetch": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_backtrace_fetch": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_peer_discover": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rfiles": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rbytes": 0, 2024-05-08T21:28:30.456 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rsnaps": 0, 2024-05-08T21:28:30.457 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T21:28:30.457 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_set_tag": 0, 2024-05-08T21:28:30.457 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T21:28:30.457 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_inotable_repaired": 0, 2024-05-08T21:28:30.457 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_inodes": 0, 2024-05-08T21:28:30.457 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T21:28:30.457 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T21:28:30.457 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_file_inodes": 0, 2024-05-08T21:28:30.457 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_inode_file_caps": 0, 2024-05-08T21:28:30.457 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T21:28:30.457 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_grant": 0, 2024-05-08T21:28:30.458 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T21:28:30.458 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T21:28:30.458 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T21:28:30.458 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps": 0, 2024-05-08T21:28:30.458 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps_dirty": 0, 2024-05-08T21:28:30.458 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_cap_release": 0, 2024-05-08T21:28:30.459 INFO:teuthology.orchestra.run.smithi033.stdout: "process_request_cap_release": 0 2024-05-08T21:28:30.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.459 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache": { 2024-05-08T21:28:30.459 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update": 0, 2024-05-08T21:28:30.459 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update_receipt": 0, 2024-05-08T21:28:30.459 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_try_discover": 0, 2024-05-08T21:28:30.459 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_send_discover": 0, 2024-05-08T21:28:30.459 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_handle_discover": 0, 2024-05-08T21:28:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays": 0, 2024-05-08T21:28:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_delayed": 0, 2024-05-08T21:28:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_enqueuing": 0, 2024-05-08T21:28:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_created": 0, 2024-05-08T21:28:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_enqueued": 0, 2024-05-08T21:28:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_reintegrated": 0, 2024-05-08T21:28:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_migrated": 0, 2024-05-08T21:28:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_processing": 0, 2024-05-08T21:28:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_enqueued": 0, 2024-05-08T21:28:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_prioritized": 0, 2024-05-08T21:28:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_started": 0, 2024-05-08T21:28:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_completed": 0, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_exportdir": 0, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_flush": 0, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragmentdir": 0, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragstats": 0, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_inodestats": 0 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log": { 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "evadd": 0, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "evex": 0, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "evtrm": 0, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "ev": 73086, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "evexg": 0, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "evexd": 0, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "segadd": 0, 2024-05-08T21:28:30.461 INFO:teuthology.orchestra.run.smithi033.stdout: "segex": 0, 2024-05-08T21:28:30.462 INFO:teuthology.orchestra.run.smithi033.stdout: "segtrm": 0, 2024-05-08T21:28:30.462 INFO:teuthology.orchestra.run.smithi033.stdout: "seg": 87, 2024-05-08T21:28:30.462 INFO:teuthology.orchestra.run.smithi033.stdout: "segexg": 0, 2024-05-08T21:28:30.462 INFO:teuthology.orchestra.run.smithi033.stdout: "segexd": 0, 2024-05-08T21:28:30.462 INFO:teuthology.orchestra.run.smithi033.stdout: "expos": 4194304, 2024-05-08T21:28:30.462 INFO:teuthology.orchestra.run.smithi033.stdout: "wrpos": 56676410, 2024-05-08T21:28:30.462 INFO:teuthology.orchestra.run.smithi033.stdout: "rdpos": 56674086, 2024-05-08T21:28:30.462 INFO:teuthology.orchestra.run.smithi033.stdout: "jlat": { 2024-05-08T21:28:30.462 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.462 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "replayed": 73086 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_mem": { 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "ino": 13, 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "ino+": 1781, 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "ino-": 1768, 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "dir": 11, 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "dir+": 109, 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "dir-": 98, 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "dn": 1410, 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "dn+": 2928, 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "dn-": 1518, 2024-05-08T21:28:30.463 INFO:teuthology.orchestra.run.smithi033.stdout: "cap": 0, 2024-05-08T21:28:30.464 INFO:teuthology.orchestra.run.smithi033.stdout: "cap+": 0, 2024-05-08T21:28:30.464 INFO:teuthology.orchestra.run.smithi033.stdout: "cap-": 0, 2024-05-08T21:28:30.464 INFO:teuthology.orchestra.run.smithi033.stdout: "rss": 54948, 2024-05-08T21:28:30.464 INFO:teuthology.orchestra.run.smithi033.stdout: "heap": 207156 2024-05-08T21:28:30.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.464 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_server": { 2024-05-08T21:28:30.464 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_client_request": 0, 2024-05-08T21:28:30.464 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_server_request": 0, 2024-05-08T21:28:30.464 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_request": 0, 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_session": 0, 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_peer_request": 0, 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "req_create_latency": { 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getattr_latency": { 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getfilelock_latency": { 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.465 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "req_link_latency": { 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookup_latency": { 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookuphash_latency": { 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.467 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.467 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupino_latency": { 2024-05-08T21:28:30.467 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.467 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.467 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.467 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupname_latency": { 2024-05-08T21:28:30.467 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.467 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.467 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupparent_latency": { 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupsnap_latency": { 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lssnap_latency": { 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.468 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mkdir_latency": { 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mknod_latency": { 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mksnap_latency": { 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.469 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: "req_open_latency": { 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: "req_readdir_latency": { 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rename_latency": { 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.470 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "req_renamesnap_latency": { 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "req_snapdiff_latency": { 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmdir_latency": { 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmsnap_latency": { 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmxattr_latency": { 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setattr_latency": { 2024-05-08T21:28:30.472 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setdirlayout_latency": { 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setfilelock_latency": { 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setlayout_latency": { 2024-05-08T21:28:30.473 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setxattr_latency": { 2024-05-08T21:28:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: "req_symlink_latency": { 2024-05-08T21:28:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: "req_unlink_latency": { 2024-05-08T21:28:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_revoke_eviction": 0, 2024-05-08T21:28:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_acquisition_throttle": 0, 2024-05-08T21:28:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getvxattr_latency": { 2024-05-08T21:28:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_sessions": { 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: "session_count": 1, 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: "session_add": 1, 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: "session_remove": 0, 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_open": 1, 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_stale": 0, 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: "total_load": 0, 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: "average_load": 0, 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: "avg_session_uptime": 2108, 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: "mdthresh_evicted": 0 2024-05-08T21:28:30.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T21:28:30.477 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T21:28:30.478 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 222487, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 68, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-05-08T21:28:30.479 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 9080, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 174, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 1035400, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 5768, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-05-08T21:28:30.480 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-05-08T21:28:30.481 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-05-08T21:28:30.481 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-05-08T21:28:30.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.481 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter": { 2024-05-08T21:28:30.481 INFO:teuthology.orchestra.run.smithi033.stdout: "op_active": 0, 2024-05-08T21:28:30.481 INFO:teuthology.orchestra.run.smithi033.stdout: "op_laggy": 0, 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send": 2056, 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send_bytes": 21, 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "op_resend": 3, 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "op_reply": 2056, 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "op_latency": { 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2056, 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 1.399978062, 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000680923 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "op_inflight": 0, 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "oplen_avg": { 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2056, 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 2057 2024-05-08T21:28:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "op": 2056, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "op_r": 2056, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "op_w": 0, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "op_rmw": 0, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "op_pg": 0, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_stat": 1284, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_create": 0, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_read": 770, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_write": 0, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writefull": 0, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writesame": 0, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_append": 0, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_zero": 0, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_truncate": 0, 2024-05-08T21:28:30.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_delete": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_mapext": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_sparse_read": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_clonerange": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_getxattr": 1, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_setxattr": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_cmpxattr": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_rmxattr": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_resetxattrs": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_call": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_watch": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_notify": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls_filter": 0, 2024-05-08T21:28:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_other": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_active": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_send": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_resend": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_ping": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_active": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_send": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_resend": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_active": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_send": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_resend": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_active": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_send": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_resend": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "command_active": 0, 2024-05-08T21:28:30.485 INFO:teuthology.orchestra.run.smithi033.stdout: "command_send": 0, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "command_resend": 0, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "map_epoch": 89, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "map_full": 0, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "map_inc": 4, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_sessions": 9, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_open": 9, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_close": 0, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_laggy": 0, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_wr": 0, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_rd": 2, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_del": 0 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "oft": { 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_objs": 0, 2024-05-08T21:28:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_kv_pairs": 0, 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_updates": 0, 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_removes": 0 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "osd-slow-ops": { 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_ops_count": 0 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "purge_queue": { 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops": 0, 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing": 0, 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_high_water": 0, 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed": 0, 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_item_in_journal": 0 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.487 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 2275, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 53437709, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 2275, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 2275, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 53437709, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.488 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_bytes": { 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 2056, 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 52485459, 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 2056, 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 52485459, 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:30.489 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_ops": { 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 1024, 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 2056, 2024-05-08T21:28:30.490 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 2056, 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 2056, 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 2056, 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle": { 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:30.491 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle-0x55b54a3b40a0": { 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-05-08T21:28:30.492 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:30.493 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:28:30.817 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-08T21:28:30.817 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-08T21:28:30.817 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-08T21:28:30.817 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-05-08T21:28:30.817 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-05-08T21:28:30.817 DEBUG:tasks.check_counter:Getting stats from d 2024-05-08T21:28:30.817 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-08T21:28:31.075 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:28:31.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:31 smithi071 ceph-mon[47074]: pgmap v576: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 615 KiB/s wr, 252 op/s 2024-05-08T21:28:31.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:31 smithi082 ceph-mon[47512]: pgmap v576: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 615 KiB/s wr, 252 op/s 2024-05-08T21:28:31.859 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:31 smithi033 ceph-mon[42528]: pgmap v576: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 615 KiB/s wr, 252 op/s 2024-05-08T21:28:32.078 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 1567, 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 1568, 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 17233154, 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 487033, 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 6, 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 2, 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.152980519, 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.041867453, 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.050964144, 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.046767520, 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:32.079 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 1566, 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.069364165, 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000044293 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 317, 2024-05-08T21:28:32.080 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 293, 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 4825310, 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 139273, 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.043309942, 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.013654219, 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.023036240, 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.007173954, 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 292, 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.016388184, 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000056123 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:32.081 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 607200, 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 110288 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 368, 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 510, 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 393645, 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 430861, 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-05-08T21:28:32.082 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 4, 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.046794431, 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.017223689, 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.010702120, 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.014402887, 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 509, 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.033553637, 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000065920 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 317968 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 1, 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-MDSRank": { 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3388, 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.099200263, 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000029279 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-PurgeQueue": { 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "request": 0, 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "reply": 0, 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "reply_latency": { 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.085 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_reply": 0, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "forward": 0, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_complete": 0, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_keys": 0, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_commit": 0, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_split": 0, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_merge": 0, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes": 2923, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_top": 21, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_bottom": 2902, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pin_tail": 0, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pinned": 2893, 2024-05-08T21:28:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_expired": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_with_caps": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "subtrees": 1, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_hit": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_forward": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_discover": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_dir_fetch": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_remote_ino": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_lock": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "load_cent": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "q": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "exported": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "exported_inodes": 0, 2024-05-08T21:28:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "imported": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "imported_inodes": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_dir_fetch": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_backtrace_fetch": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_peer_discover": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rfiles": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rbytes": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rsnaps": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_set_tag": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_inotable_repaired": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_inodes": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T21:28:32.088 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_file_inodes": 0, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_inode_file_caps": 0, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_grant": 0, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps": 0, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps_dirty": 0, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_cap_release": 0, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "process_request_cap_release": 0 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache": { 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update": 0, 2024-05-08T21:28:32.089 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update_receipt": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_try_discover": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_send_discover": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_handle_discover": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_delayed": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_enqueuing": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_created": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_enqueued": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_reintegrated": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_migrated": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_processing": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_enqueued": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_prioritized": 0, 2024-05-08T21:28:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_started": 0, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_completed": 0, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_exportdir": 0, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_flush": 0, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragmentdir": 0, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragstats": 0, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_inodestats": 0 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log": { 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "evadd": 0, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "evex": 0, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "evtrm": 0, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "ev": 25830, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "evexg": 0, 2024-05-08T21:28:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "evexd": 0, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "segadd": 0, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "segex": 0, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "segtrm": 0, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "seg": 29, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "segexg": 0, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "segexd": 0, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "expos": 4194304, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "wrpos": 25466869, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "rdpos": 25466270, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "jlat": { 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.092 INFO:teuthology.orchestra.run.smithi033.stdout: "replayed": 25830 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_mem": { 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "ino": 22, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "ino+": 1640, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "ino-": 1618, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "dir": 153, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "dir+": 227, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "dir-": 74, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "dn": 2850, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "dn+": 3859, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "dn-": 1009, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "cap": 0, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "cap+": 0, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "cap-": 0, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "rss": 53488, 2024-05-08T21:28:32.093 INFO:teuthology.orchestra.run.smithi033.stdout: "heap": 207156 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_server": { 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_client_request": 0, 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_server_request": 0, 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_request": 0, 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_session": 0, 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_peer_request": 0, 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "req_create_latency": { 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getattr_latency": { 2024-05-08T21:28:32.094 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getfilelock_latency": { 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "req_link_latency": { 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookup_latency": { 2024-05-08T21:28:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookuphash_latency": { 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupino_latency": { 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupname_latency": { 2024-05-08T21:28:32.096 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupparent_latency": { 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupsnap_latency": { 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lssnap_latency": { 2024-05-08T21:28:32.097 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mkdir_latency": { 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mknod_latency": { 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mksnap_latency": { 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "req_open_latency": { 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "req_readdir_latency": { 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rename_latency": { 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "req_renamesnap_latency": { 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "req_snapdiff_latency": { 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmdir_latency": { 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmsnap_latency": { 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmxattr_latency": { 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setattr_latency": { 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setdirlayout_latency": { 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setfilelock_latency": { 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setlayout_latency": { 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setxattr_latency": { 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "req_symlink_latency": { 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.103 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "req_unlink_latency": { 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_revoke_eviction": 0, 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_acquisition_throttle": 0, 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getvxattr_latency": { 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_sessions": { 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "session_count": 1, 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "session_add": 1, 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "session_remove": 0, 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_open": 1, 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_stale": 0, 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "total_load": 0, 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "average_load": 0, 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "avg_session_uptime": 2110, 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "mdthresh_evicted": 0 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T21:28:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-05-08T21:28:32.106 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-05-08T21:28:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 199278, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 57, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 9080, 2024-05-08T21:28:32.108 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 174, 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 2323996, 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 12949, 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter": { 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "op_active": 0, 2024-05-08T21:28:32.109 INFO:teuthology.orchestra.run.smithi033.stdout: "op_laggy": 0, 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send": 2022, 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send_bytes": 21, 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: "op_resend": 3, 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: "op_reply": 2022, 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: "op_latency": { 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2022, 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 1.216980955, 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000601869 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: "op_inflight": 0, 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: "oplen_avg": { 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2022, 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 2023 2024-05-08T21:28:32.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "op": 2022, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "op_r": 2022, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "op_w": 0, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "op_rmw": 0, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "op_pg": 0, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_stat": 1242, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_create": 0, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_read": 778, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_write": 0, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writefull": 0, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writesame": 0, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_append": 0, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_zero": 0, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_truncate": 0, 2024-05-08T21:28:32.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_delete": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_mapext": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_sparse_read": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_clonerange": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_getxattr": 1, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_setxattr": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_cmpxattr": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_rmxattr": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_resetxattrs": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_call": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_watch": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_notify": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls": 0, 2024-05-08T21:28:32.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls_filter": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_other": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_active": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_send": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_resend": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_ping": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_active": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_send": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_resend": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_active": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_send": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_resend": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_active": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_send": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_resend": 0, 2024-05-08T21:28:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "command_active": 0, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "command_send": 0, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "command_resend": 0, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "map_epoch": 89, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "map_full": 0, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "map_inc": 4, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_sessions": 6, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_open": 6, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_close": 0, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_laggy": 0, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_wr": 0, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_rd": 2, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_del": 0 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "oft": { 2024-05-08T21:28:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_objs": 0, 2024-05-08T21:28:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_kv_pairs": 0, 2024-05-08T21:28:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_updates": 0, 2024-05-08T21:28:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_removes": 0 2024-05-08T21:28:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "osd-slow-ops": { 2024-05-08T21:28:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_ops_count": 0 2024-05-08T21:28:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "purge_queue": { 2024-05-08T21:28:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops": 0, 2024-05-08T21:28:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T21:28:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing": 0, 2024-05-08T21:28:32.115 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_high_water": 0, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed": 0, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_item_in_journal": 0 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 2248, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 22235408, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 2248, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 2248, 2024-05-08T21:28:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 22235408, 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_bytes": { 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:32.117 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 2022, 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 21272721, 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 2022, 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 21272721, 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_ops": { 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 1024, 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:32.118 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 2022, 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 2022, 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 2022, 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 2022, 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle": { 2024-05-08T21:28:32.119 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle-0x55a756a14140": { 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:32.122 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:32.122 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:28:32.457 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-05-08T21:28:32.457 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-05-08T21:28:32.457 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-05-08T21:28:32.457 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-05-08T21:28:32.457 INFO:tasks.check_counter:Daemon mds.d mds.dir_split=0 2024-05-08T21:28:32.457 DEBUG:tasks.check_counter:Getting stats from g 2024-05-08T21:28:32.458 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-08T21:28:32.719 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:28:33.727 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:28:33.728 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T21:28:33.728 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 196, 2024-05-08T21:28:33.728 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 191, 2024-05-08T21:28:33.728 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 467200, 2024-05-08T21:28:33.728 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 108016, 2024-05-08T21:28:33.728 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-05-08T21:28:33.728 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-05-08T21:28:33.728 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.033266251, 2024-05-08T21:28:33.728 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.012270696, 2024-05-08T21:28:33.728 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.015813940, 2024-05-08T21:28:33.728 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004517797, 2024-05-08T21:28:33.729 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:33.729 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 191, 2024-05-08T21:28:33.729 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010719326, 2024-05-08T21:28:33.729 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000056122 2024-05-08T21:28:33.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:33.729 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:33.729 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:33.729 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:33.729 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:33.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:33.730 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 467200, 2024-05-08T21:28:33.730 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 108016 2024-05-08T21:28:33.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:33.730 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T21:28:33.730 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 1, 2024-05-08T21:28:33.730 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 143, 2024-05-08T21:28:33.730 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 160, 2024-05-08T21:28:33.730 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 101248, 2024-05-08T21:28:33.730 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 3, 2024-05-08T21:28:33.730 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 2, 2024-05-08T21:28:33.730 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.011231479, 2024-05-08T21:28:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.008163527, 2024-05-08T21:28:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001583909, 2024-05-08T21:28:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T21:28:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 141, 2024-05-08T21:28:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010763773, 2024-05-08T21:28:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000076338 2024-05-08T21:28:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:33.731 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:33.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:33.732 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T21:28:33.732 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 101248 2024-05-08T21:28:33.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:33.732 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T21:28:33.732 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-05-08T21:28:33.732 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-05-08T21:28:33.732 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-05-08T21:28:33.732 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.000726465, 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000628735, 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:33.733 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:33.734 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:33.734 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:33.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:33.734 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T21:28:33.734 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T21:28:33.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:33.734 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-05-08T21:28:33.734 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-05-08T21:28:33.734 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-05-08T21:28:33.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-05-08T21:28:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-05-08T21:28:33.736 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T21:28:33.737 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T21:28:33.737 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-05-08T21:28:33.737 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-05-08T21:28:33.737 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-05-08T21:28:33.737 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-05-08T21:28:33.737 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T21:28:33.737 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-05-08T21:28:33.737 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T21:28:33.737 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-05-08T21:28:33.737 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148984, 2024-05-08T21:28:33.737 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 30, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-05-08T21:28:33.738 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-05-08T21:28:33.739 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-05-08T21:28:33.739 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-05-08T21:28:33.739 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-05-08T21:28:33.739 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-05-08T21:28:33.739 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-05-08T21:28:33.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:33.739 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T21:28:33.739 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:33.739 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-08T21:28:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 193, 2024-05-08T21:28:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 437083, 2024-05-08T21:28:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 193, 2024-05-08T21:28:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-08T21:28:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-08T21:28:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 193, 2024-05-08T21:28:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 437083, 2024-05-08T21:28:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:33.741 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:33.741 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:33.741 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:33.741 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:28:33.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:33 smithi071 ceph-mon[47074]: pgmap v577: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 440 KiB/s wr, 174 op/s 2024-05-08T21:28:33.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:33 smithi082 ceph-mon[47512]: pgmap v577: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 440 KiB/s wr, 174 op/s 2024-05-08T21:28:33.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:33 smithi033 ceph-mon[42528]: pgmap v577: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 440 KiB/s wr, 174 op/s 2024-05-08T21:28:34.465 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-08T21:28:34.466 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-08T21:28:34.466 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-05-08T21:28:34.466 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-05-08T21:28:34.466 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-05-08T21:28:34.466 DEBUG:tasks.check_counter:Getting stats from j 2024-05-08T21:28:34.466 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-08T21:28:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T21:28:35.761 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-05-08T21:28:35.761 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T21:28:35.761 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 910, 2024-05-08T21:28:35.761 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 886, 2024-05-08T21:28:35.761 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 5030138, 2024-05-08T21:28:35.761 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 323915, 2024-05-08T21:28:35.761 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 6, 2024-05-08T21:28:35.761 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 2, 2024-05-08T21:28:35.761 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.111232221, 2024-05-08T21:28:35.761 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.034952176, 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.039669036, 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.029753635, 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 885, 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.048335802, 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000054616 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 607472, 2024-05-08T21:28:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 110880 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 702, 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 845, 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 21154018, 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 536547, 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 6, 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 5, 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.089696552, 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.026095233, 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.035901179, 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.020684183, 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 844, 2024-05-08T21:28:35.763 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.039294056, 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000046556 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 319712 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 656, 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 658, 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 8601578, 2024-05-08T21:28:35.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 204017, 2024-05-08T21:28:35.765 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-05-08T21:28:35.765 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 4, 2024-05-08T21:28:35.765 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.067836653, 2024-05-08T21:28:35.765 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.017283769, 2024-05-08T21:28:35.765 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.022011872, 2024-05-08T21:28:35.765 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.022738062, 2024-05-08T21:28:35.765 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:35.765 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 656, 2024-05-08T21:28:35.765 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.032390980, 2024-05-08T21:28:35.765 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000049376 2024-05-08T21:28:35.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 1, 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.766 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-MDSRank": { 2024-05-08T21:28:35.767 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-05-08T21:28:35.767 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-05-08T21:28:35.767 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3375, 2024-05-08T21:28:35.767 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.102799611, 2024-05-08T21:28:35.767 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000030459 2024-05-08T21:28:35.767 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:35.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.767 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-PurgeQueue": { 2024-05-08T21:28:35.767 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-05-08T21:28:35.767 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-05-08T21:28:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-05-08T21:28:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: "request": 0, 2024-05-08T21:28:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: "reply": 0, 2024-05-08T21:28:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: "reply_latency": { 2024-05-08T21:28:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_reply": 0, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "forward": 0, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_complete": 0, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_keys": 0, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_commit": 0, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_split": 0, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_merge": 0, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes": 1464, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_top": 0, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_bottom": 1464, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pin_tail": 0, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pinned": 1464, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_expired": 0, 2024-05-08T21:28:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_with_caps": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "subtrees": 1, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_hit": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_forward": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_discover": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_dir_fetch": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_remote_ino": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_lock": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "load_cent": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "q": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "exported": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "exported_inodes": 0, 2024-05-08T21:28:35.770 INFO:teuthology.orchestra.run.smithi033.stdout: "imported": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "imported_inodes": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_dir_fetch": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_backtrace_fetch": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_peer_discover": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rfiles": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rbytes": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rsnaps": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_set_tag": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_inotable_repaired": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_inodes": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T21:28:35.771 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_file_inodes": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_inode_file_caps": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_grant": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps_dirty": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_cap_release": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "process_request_cap_release": 0 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache": { 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update_receipt": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_try_discover": 0, 2024-05-08T21:28:35.772 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_send_discover": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_handle_discover": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_delayed": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_enqueuing": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_created": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_enqueued": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_reintegrated": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_migrated": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_processing": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_enqueued": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_prioritized": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_started": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_completed": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T21:28:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_exportdir": 0, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_flush": 0, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragmentdir": 0, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragstats": 0, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_inodestats": 0 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log": { 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "evadd": 0, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "evex": 0, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "evtrm": 0, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "ev": 47123, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "evexg": 0, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "evexd": 0, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "segadd": 0, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "segex": 0, 2024-05-08T21:28:35.774 INFO:teuthology.orchestra.run.smithi033.stdout: "segtrm": 0, 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: "seg": 55, 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: "segexg": 0, 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: "segexd": 0, 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: "expos": 4194304, 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: "wrpos": 37796177, 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: "rdpos": 37795039, 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: "jlat": { 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: "replayed": 47123 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.775 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_mem": { 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "ino": 15, 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "ino+": 1476, 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "ino-": 1461, 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "dir": 13, 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "dir+": 96, 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "dir-": 83, 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "dn": 1464, 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "dn+": 2717, 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "dn-": 1253, 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "cap": 0, 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "cap+": 0, 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "cap-": 0, 2024-05-08T21:28:35.776 INFO:teuthology.orchestra.run.smithi033.stdout: "rss": 52120, 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "heap": 207156 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_server": { 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_client_request": 0, 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_server_request": 0, 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_request": 0, 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_session": 0, 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_peer_request": 0, 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "req_create_latency": { 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getattr_latency": { 2024-05-08T21:28:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.778 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.778 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.778 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getfilelock_latency": { 2024-05-08T21:28:35.778 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.778 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.778 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "req_link_latency": { 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookup_latency": { 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookuphash_latency": { 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupino_latency": { 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupname_latency": { 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupparent_latency": { 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.780 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupsnap_latency": { 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lssnap_latency": { 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mkdir_latency": { 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.781 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mknod_latency": { 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mksnap_latency": { 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: "req_open_latency": { 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.782 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: "req_readdir_latency": { 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rename_latency": { 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: "req_renamesnap_latency": { 2024-05-08T21:28:35.783 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "req_snapdiff_latency": { 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmdir_latency": { 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmsnap_latency": { 2024-05-08T21:28:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmxattr_latency": { 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setattr_latency": { 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.786 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setdirlayout_latency": { 2024-05-08T21:28:35.786 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.786 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.786 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.786 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setfilelock_latency": { 2024-05-08T21:28:35.786 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.786 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setlayout_latency": { 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setxattr_latency": { 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: "req_symlink_latency": { 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: "req_unlink_latency": { 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_revoke_eviction": 0, 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_acquisition_throttle": 0, 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getvxattr_latency": { 2024-05-08T21:28:35.788 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_sessions": { 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "session_count": 1, 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "session_add": 2, 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "session_remove": 1, 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_open": 2, 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_stale": 0, 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "total_load": 0, 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "average_load": 0, 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "avg_session_uptime": 3567, 2024-05-08T21:28:35.789 INFO:teuthology.orchestra.run.smithi033.stdout: "mdthresh_evicted": 0 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T21:28:35.790 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-05-08T21:28:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 215438, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 65, 2024-05-08T21:28:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 9080, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 174, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-05-08T21:28:35.793 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 1080513, 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 6223, 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter": { 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "op_active": 0, 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "op_laggy": 0, 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send": 2038, 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send_bytes": 21, 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "op_resend": 3, 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "op_reply": 2038, 2024-05-08T21:28:35.794 INFO:teuthology.orchestra.run.smithi033.stdout: "op_latency": { 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2038, 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 1.241980531, 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000609411 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: "op_inflight": 0, 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: "oplen_avg": { 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2038, 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 2039 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: "op": 2038, 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: "op_r": 2038, 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: "op_w": 0, 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: "op_rmw": 0, 2024-05-08T21:28:35.795 INFO:teuthology.orchestra.run.smithi033.stdout: "op_pg": 0, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_stat": 1258, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_create": 0, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_read": 778, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_write": 0, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writefull": 0, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writesame": 0, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_append": 0, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_zero": 0, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_truncate": 0, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_delete": 0, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_mapext": 0, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_sparse_read": 0, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_clonerange": 0, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_getxattr": 1, 2024-05-08T21:28:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_setxattr": 0, 2024-05-08T21:28:35.797 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_cmpxattr": 0, 2024-05-08T21:28:35.797 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_rmxattr": 0, 2024-05-08T21:28:35.797 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_resetxattrs": 0, 2024-05-08T21:28:35.797 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_call": 0, 2024-05-08T21:28:35.797 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_watch": 0, 2024-05-08T21:28:35.797 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_notify": 0, 2024-05-08T21:28:35.797 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T21:28:35.797 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls": 0, 2024-05-08T21:28:35.797 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls_filter": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_other": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_active": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_send": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_resend": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_ping": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_active": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_send": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_resend": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_active": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_send": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_resend": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_active": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_send": 0, 2024-05-08T21:28:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_resend": 0, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "command_active": 0, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "command_send": 0, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "command_resend": 0, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "map_epoch": 89, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "map_full": 0, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "map_inc": 4, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_sessions": 8, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_open": 8, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_close": 0, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_laggy": 0, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_wr": 0, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_rd": 2, 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_del": 0 2024-05-08T21:28:35.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: "oft": { 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_objs": 0, 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_kv_pairs": 0, 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_updates": 0, 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_removes": 0 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: "osd-slow-ops": { 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_ops_count": 0 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: "purge_queue": { 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops": 0, 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing": 0, 2024-05-08T21:28:35.800 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_high_water": 0, 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed": 0, 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_item_in_journal": 0 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 2264, 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 34567541, 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 2264, 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-08T21:28:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 2264, 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 34567541, 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_bytes": { 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-05-08T21:28:35.802 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 2038, 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 33601878, 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 2038, 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 33601878, 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_ops": { 2024-05-08T21:28:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 1024, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 2038, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 2038, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 2038, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 2038, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle": { 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle-0x55f8ec2d00a0": { 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:35.806 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-05-08T21:28:35.807 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-05-08T21:28:35.807 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-05-08T21:28:35.807 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-05-08T21:28:35.807 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-05-08T21:28:35.807 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-05-08T21:28:35.807 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-05-08T21:28:35.807 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-05-08T21:28:35.807 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:35.807 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-05-08T21:28:35.807 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-05-08T21:28:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:35 smithi033 ceph-mon[42528]: pgmap v578: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 220 KiB/s wr, 125 op/s 2024-05-08T21:28:35.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:35 smithi071 ceph-mon[47074]: pgmap v578: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 220 KiB/s wr, 125 op/s 2024-05-08T21:28:35.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:35 smithi082 ceph-mon[47512]: pgmap v578: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 220 KiB/s wr, 125 op/s 2024-05-08T21:28:36.133 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2024-05-08T21:28:36.135 INFO:tasks.check_counter:Daemon mds.j mds.imported=0 2024-05-08T21:28:36.135 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update=0 2024-05-08T21:28:36.135 INFO:tasks.check_counter:Daemon mds.j mds_cache.dir_update_receipt=0 2024-05-08T21:28:36.136 INFO:tasks.check_counter:Daemon mds.j mds.dir_split=0 2024-05-08T21:28:36.137 DEBUG:tasks.check_counter:Getting stats from b 2024-05-08T21:28:36.137 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-08T21:28:36.399 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:28:37.714 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:37 smithi071 ceph-mon[47074]: pgmap v579: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 182 KiB/s wr, 70 op/s 2024-05-08T21:28:37.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:37 smithi082 ceph-mon[47512]: pgmap v579: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 182 KiB/s wr, 70 op/s 2024-05-08T21:28:37.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:37 smithi033 ceph-mon[42528]: pgmap v579: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 182 KiB/s wr, 70 op/s 2024-05-08T21:28:38.160 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-05-08T21:28:38.160 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T21:28:38.160 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 55746, 2024-05-08T21:28:38.160 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 67519, 2024-05-08T21:28:38.160 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 23256324, 2024-05-08T21:28:38.160 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 156995108, 2024-05-08T21:28:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 10, 2024-05-08T21:28:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 4, 2024-05-08T21:28:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 3.447727710, 2024-05-08T21:28:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 1.327439687, 2024-05-08T21:28:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 1.634455629, 2024-05-08T21:28:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.560933598, 2024-05-08T21:28:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 67520, 2024-05-08T21:28:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2.936467679, 2024-05-08T21:28:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000043490 2024-05-08T21:28:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 73420, 2024-05-08T21:28:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.012147833, 2024-05-08T21:28:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000165 2024-05-08T21:28:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-05-08T21:28:38.163 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T21:28:38.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.163 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T21:28:38.163 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 73469, 2024-05-08T21:28:38.163 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 124423, 2024-05-08T21:28:38.163 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 22511517, 2024-05-08T21:28:38.165 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 81598696, 2024-05-08T21:28:38.165 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 151, 2024-05-08T21:28:38.165 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 10, 2024-05-08T21:28:38.165 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 4.860768561, 2024-05-08T21:28:38.165 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 1.858047907, 2024-05-08T21:28:38.165 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 2.603618229, 2024-05-08T21:28:38.165 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.564115146, 2024-05-08T21:28:38.166 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:38.166 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 124419, 2024-05-08T21:28:38.166 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 6.701813875, 2024-05-08T21:28:38.167 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000053864 2024-05-08T21:28:38.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.167 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:38.168 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 136895, 2024-05-08T21:28:38.168 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.022331099, 2024-05-08T21:28:38.168 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000163 2024-05-08T21:28:38.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.168 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 58000, 2024-05-08T21:28:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 366720 2024-05-08T21:28:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T21:28:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 55667, 2024-05-08T21:28:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 69241, 2024-05-08T21:28:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 23293095, 2024-05-08T21:28:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 117804708, 2024-05-08T21:28:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 12, 2024-05-08T21:28:38.170 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 7, 2024-05-08T21:28:38.170 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 3.205898110, 2024-05-08T21:28:38.170 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 1.138874201, 2024-05-08T21:28:38.170 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 1.563674331, 2024-05-08T21:28:38.171 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.609801429, 2024-05-08T21:28:38.171 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:38.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 69240, 2024-05-08T21:28:38.171 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 4.244999232, 2024-05-08T21:28:38.171 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000061308 2024-05-08T21:28:38.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.174 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:38.174 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 69015, 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.011488523, 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000166 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 639568, 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 136032 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-08T21:28:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 102768, 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 27.741616803, 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000269944 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2162, 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.234062889, 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000108262 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-05-08T21:28:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 28589, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 24475, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 24475, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 95.855713978, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003916474 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 4114, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 194, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 22, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 33, 2024-05-08T21:28:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 4, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 9, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 20, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 26, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 4708, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 2, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 2, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 4, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 38753, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 31525, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 4082, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 54, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-05-08T21:28:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 2, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 558, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 181, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 4550, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 2, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 7, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 2562, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 4, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 234, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 0, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 2562, 2024-05-08T21:28:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 297, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 17, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 280, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 2265, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 13872, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 15191, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 8501, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 19488, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 13739, 2024-05-08T21:28:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 55, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 16954 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 291, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 345, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 194, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 321, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 880, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 1, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 1049, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 1046, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 2, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-05-08T21:28:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 17, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 213, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 22, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 43873, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-05-08T21:28:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 43873, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 60, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 60, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 98321448, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 30902, 2024-05-08T21:28:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 60.790910027, 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001967216 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 34, 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 5214, 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 5180, 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 27, 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 450, 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 423, 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 33, 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 5980, 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 5947, 2024-05-08T21:28:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 2, 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 5138, 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 5136, 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 93836, 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 207156 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 52660, 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 3934, 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 28589, 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 201, 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 7866, 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-05-08T21:28:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 5038, 2024-05-08T21:28:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 16.717616562, 2024-05-08T21:28:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003318304 2024-05-08T21:28:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-05-08T21:28:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1189, 2024-05-08T21:28:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.631063872, 2024-05-08T21:28:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000530751 2024-05-08T21:28:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-05-08T21:28:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.004192556, 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002096278 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 7321, 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 37.086628643, 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005065787 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 82, 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.083970233, 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001024027 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 110, 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.062829117, 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000571173 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 74, 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.421409268, 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005694719 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 63, 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.132158039, 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002097746 2024-05-08T21:28:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 24, 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.163166122, 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.006798588 2024-05-08T21:28:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 9562, 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 38.432410493, 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.004019285 2024-05-08T21:28:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 7, 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.003444491, 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000492070 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-05-08T21:28:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 8, 2024-05-08T21:28:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.020706925, 2024-05-08T21:28:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002588365 2024-05-08T21:28:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-05-08T21:28:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 5, 2024-05-08T21:28:38.198 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.003679613, 2024-05-08T21:28:38.198 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000735922 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 990, 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2.092438044, 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002113573 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-05-08T21:28:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 1, 2024-05-08T21:28:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 3, 2024-05-08T21:28:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 2, 2024-05-08T21:28:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 1, 2024-05-08T21:28:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-05-08T21:28:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 1889, 2024-05-08T21:28:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 1889, 2024-05-08T21:28:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 590, 2024-05-08T21:28:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T21:28:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-05-08T21:28:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 299071, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 123, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-05-08T21:28:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9080, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 174, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 137357, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 945, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-05-08T21:28:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-05-08T21:28:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-05-08T21:28:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-05-08T21:28:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-05-08T21:28:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 76496, 2024-05-08T21:28:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 96624339, 2024-05-08T21:28:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 37, 2024-05-08T21:28:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 76496, 2024-05-08T21:28:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-05-08T21:28:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 76496, 2024-05-08T21:28:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 395.144288505, 2024-05-08T21:28:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005165554 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 76496, 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 76765 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 76496, 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 2685, 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 73811, 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-05-08T21:28:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 11, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 106, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 30991, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 98, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 6, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 2653, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 2579, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 22, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-05-08T21:28:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 39793, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-05-08T21:28:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 89, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 9, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 12, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-05-08T21:28:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 468, 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 0, 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 38 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-05-08T21:28:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 33, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 6, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 1046, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 184878, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 52397472, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 184878, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-08T21:28:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 184878, 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 52397472, 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-08T21:28:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 76496, 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 96738225, 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 76496, 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 96738225, 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 76496, 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 76496, 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 76496, 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 76496, 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 1046, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 113886, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 1046, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 89, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 113886, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x561f6e902000": { 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 43873, 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 94127144, 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 43873, 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 30902, 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 94127144, 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:38.217 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:38.217 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:38.217 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:38.217 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:38.217 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-05-08T21:28:38.483 INFO:tasks.check_counter:Daemon mds.b mds.exported=181 2024-05-08T21:28:38.483 INFO:tasks.check_counter:Daemon mds.b mds.imported=2 2024-05-08T21:28:38.483 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=291 2024-05-08T21:28:38.483 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=345 2024-05-08T21:28:38.483 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-05-08T21:28:38.483 DEBUG:tasks.check_counter:Getting stats from e 2024-05-08T21:28:38.484 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-08T21:28:38.704 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:28:39.670 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-05-08T21:28:39.670 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 15332, 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 19752, 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 5251604, 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 16253794, 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 15, 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.939033751, 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.354584336, 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.562780836, 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.103798386, 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 19751, 2024-05-08T21:28:39.671 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.108759261, 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000056136 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 23757, 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.003500694, 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000147 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 618944, 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 127440 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 8676, 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 8821, 2024-05-08T21:28:39.672 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 2201042, 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 19900966, 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 18, 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.523753620, 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.176788567, 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.117642828, 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.194112393, 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 8819, 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.842718191, 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000095557 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 19, 2024-05-08T21:28:39.673 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000002614, 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000137 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2096, 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 323760 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 26701, 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 19887, 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 51914899, 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 18596495, 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 36, 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 1.471032578, 2024-05-08T21:28:39.674 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.610147757, 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 1.020949419, 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.078675855, 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 19886, 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.877291868, 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000044116 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 39536, 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.005465409, 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000138 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T21:28:39.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 10646, 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 4.990324483, 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000468751 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-08T21:28:39.676 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2964, 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.429386276, 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000144867 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 9203, 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 7190, 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 7190, 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 5.983675072, 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000832221 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.677 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 2013, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 11, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 26, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 4, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 8, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 14, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 17, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 384, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 0, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 0, 2024-05-08T21:28:39.678 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 4, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 17205, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 6229, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 2011, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 86, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 949, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 47, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 581, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 47, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 1179, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-05-08T21:28:39.679 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 11865, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 3568, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 64348743, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 0, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 11865, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 224, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 224, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 11641, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 135, 2024-05-08T21:28:39.680 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 1857, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 1030, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 449, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 65, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 3257 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 192, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 508, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 283, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 429, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 263, 2024-05-08T21:28:39.681 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 2, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 1440, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 1438, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 47, 2024-05-08T21:28:39.682 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 47123, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 47123, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 55, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-05-08T21:28:39.683 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 55, 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 37796177, 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 990, 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2.990591301, 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003020799 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 27, 2024-05-08T21:28:39.684 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 1535, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 1508, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 152, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 318, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 166, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 458, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 2921, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 2463, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 0, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 1448, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 1448, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 65196, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 198964 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-05-08T21:28:39.685 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 11836, 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 17, 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 9203, 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 12, 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 826, 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 373, 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.209089832, 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003241527 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 96, 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.028459772, 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000296455 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.686 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 5191, 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.536970456, 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000296083 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.687 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.688 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.689 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3, 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.005305121, 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001768373 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000602756, 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000301378 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 85, 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.063050027, 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000741765 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 20, 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.249211174, 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.012460558 2024-05-08T21:28:39.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.694 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1420, 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2.890985934, 2024-05-08T21:28:39.695 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002035905 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 1, 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 2, 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 1, 2024-05-08T21:28:39.696 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 1, 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 1842, 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 1842, 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 592, 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-05-08T21:28:39.697 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T21:28:39.698 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-05-08T21:28:39.699 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 315779, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 123, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9080, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 174, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-05-08T21:28:39.700 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 553167, 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 5675, 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 16117, 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 33882411, 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 27, 2024-05-08T21:28:39.701 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 16117, 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 16117, 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 30.475483671, 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001890890 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 16117, 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 16181 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 16117, 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 11991, 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 4126, 2024-05-08T21:28:39.702 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 10, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 126, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 1116, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 82, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 2878, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-05-08T21:28:39.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 11865, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 20, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-05-08T21:28:39.704 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 89, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-05-08T21:28:39.705 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 4, 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 12, 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 61, 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 0, 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 23 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.706 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 10, 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 6, 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 1438, 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:39.707 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 50705, 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 54786981, 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 50705, 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 50705, 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 54786981, 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:39.708 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 16117, 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 34039009, 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 16117, 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 34039009, 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:39.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 16117, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 16117, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 16117, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 16117, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.710 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 1438, 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 156598, 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 1438, 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 126, 2024-05-08T21:28:39.711 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 156598, 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x555b60a4a000": { 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 47123, 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 33601873, 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 47123, 2024-05-08T21:28:39.712 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-08T21:28:39.713 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-08T21:28:39.713 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 990, 2024-05-08T21:28:39.713 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 33601873, 2024-05-08T21:28:39.713 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:39.713 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:39.713 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:39.713 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:39.713 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:39.713 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:39.713 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-05-08T21:28:39.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:39 smithi071 ceph-mon[47074]: pgmap v580: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 33 op/s 2024-05-08T21:28:39.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:39 smithi082 ceph-mon[47512]: pgmap v580: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 33 op/s 2024-05-08T21:28:39.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:39 smithi033 ceph-mon[42528]: pgmap v580: 129 pgs: 129 active+clean; 224 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 33 op/s 2024-05-08T21:28:39.991 INFO:tasks.check_counter:Daemon mds.e mds.exported=47 2024-05-08T21:28:39.991 INFO:tasks.check_counter:Daemon mds.e mds.imported=47 2024-05-08T21:28:39.991 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=192 2024-05-08T21:28:39.991 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=508 2024-05-08T21:28:39.991 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-05-08T21:28:39.991 DEBUG:tasks.check_counter:Getting stats from h 2024-05-08T21:28:39.992 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-08T21:28:40.213 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:28:41.184 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-05-08T21:28:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T21:28:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 27836, 2024-05-08T21:28:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 22792, 2024-05-08T21:28:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 50976089, 2024-05-08T21:28:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 25064063, 2024-05-08T21:28:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 11, 2024-05-08T21:28:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 4, 2024-05-08T21:28:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 1.540177884, 2024-05-08T21:28:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.634952597, 2024-05-08T21:28:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.998768942, 2024-05-08T21:28:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.119901687, 2024-05-08T21:28:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 22791, 2024-05-08T21:28:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.001582370, 2024-05-08T21:28:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000043946 2024-05-08T21:28:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 39792, 2024-05-08T21:28:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.005958740, 2024-05-08T21:28:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000149 2024-05-08T21:28:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-05-08T21:28:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T21:28:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T21:28:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 14704, 2024-05-08T21:28:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 19188, 2024-05-08T21:28:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 4403518, 2024-05-08T21:28:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 9164835, 2024-05-08T21:28:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 49, 2024-05-08T21:28:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 9, 2024-05-08T21:28:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.871015880, 2024-05-08T21:28:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.329182276, 2024-05-08T21:28:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.533989103, 2024-05-08T21:28:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.086144693, 2024-05-08T21:28:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 19184, 2024-05-08T21:28:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.051568058, 2024-05-08T21:28:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000054814 2024-05-08T21:28:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 23685, 2024-05-08T21:28:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.003580938, 2024-05-08T21:28:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000151 2024-05-08T21:28:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T21:28:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 323696 2024-05-08T21:28:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T21:28:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 11242, 2024-05-08T21:28:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 12010, 2024-05-08T21:28:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 3919692, 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 9409402, 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 8, 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.621459975, 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.214644386, 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.229578219, 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.173316391, 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 12011, 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.077537036, 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000089712 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 6443, 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.001060424, 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000164 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 628048, 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 129472 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-08T21:28:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 11358, 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 4.589471424, 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000404073 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3447, 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.249456132, 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000072369 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 9093, 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 6660, 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 6660, 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 11.590031721, 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001740245 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 2433, 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 11, 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-05-08T21:28:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 650, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 158, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 72, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 420, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 423, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 61, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 2, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 2, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 4, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 17203, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 8075, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 2247, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 52, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-05-08T21:28:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 3, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 1069, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 2, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 7, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 181, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 2125, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 11915, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 3568, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 64348743, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 0, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 11915, 2024-05-08T21:28:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 528, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 528, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 11387, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 4, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 207, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 2113, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 3, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 1620, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 455, 2024-05-08T21:28:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 73, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 3707 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 304, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 400, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 214, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 382, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 623, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 9, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 1615, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 1606, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-05-08T21:28:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 2, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 25831, 2024-05-08T21:28:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 25831, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 29, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 29, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 25468520, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-05-08T21:28:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1397, 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 5.138913863, 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003678535 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 30, 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 1663, 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 1633, 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 162, 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 288, 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 126, 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 436, 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 4019, 2024-05-08T21:28:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 3583, 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 2, 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 1621, 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 1619, 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 70728, 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 198964 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 12665, 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 71, 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 9093, 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 8, 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 1293, 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 420, 2024-05-08T21:28:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.110239186, 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002643426 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 240, 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.064801177, 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000270004 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-05-08T21:28:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4278, 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2.134607466, 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000498973 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-05-08T21:28:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.004179718, 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002089859 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-05-08T21:28:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 10, 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.011018446, 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001101844 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 75, 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.078625253, 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001048336 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 18, 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 4.281140449, 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.237841136 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 49, 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.761779808, 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.015546526 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1, 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.001072114, 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001072114 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-05-08T21:28:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-05-08T21:28:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1, 2024-05-08T21:28:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000324012, 2024-05-08T21:28:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000324012 2024-05-08T21:28:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-05-08T21:28:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-05-08T21:28:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-05-08T21:28:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-05-08T21:28:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.002068826, 2024-05-08T21:28:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001034413 2024-05-08T21:28:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1564, 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 3.140175266, 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002007784 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-05-08T21:28:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 1, 2024-05-08T21:28:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 1, 2024-05-08T21:28:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 0, 2024-05-08T21:28:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 1, 2024-05-08T21:28:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-05-08T21:28:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 2042, 2024-05-08T21:28:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 2042, 2024-05-08T21:28:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 592, 2024-05-08T21:28:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-05-08T21:28:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-05-08T21:28:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T21:28:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T21:28:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-05-08T21:28:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-05-08T21:28:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-05-08T21:28:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-05-08T21:28:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T21:28:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T21:28:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T21:28:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-05-08T21:28:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T21:28:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-05-08T21:28:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 309199, 2024-05-08T21:28:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 131, 2024-05-08T21:28:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-05-08T21:28:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-05-08T21:28:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-05-08T21:28:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-05-08T21:28:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-05-08T21:28:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-05-08T21:28:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9080, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 174, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 791537, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 6742, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-05-08T21:28:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 16981, 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 21578285, 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 32, 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 16981, 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 16981, 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 32.464449709, 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001911810 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-05-08T21:28:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 16981, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 17054 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 16981, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 12066, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 4915, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 10, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 151, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 1548, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 99, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-05-08T21:28:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 3213, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 11915, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 20, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-05-08T21:28:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T21:28:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-05-08T21:28:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-05-08T21:28:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 3, 2024-05-08T21:28:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-05-08T21:28:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-05-08T21:28:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-05-08T21:28:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-05-08T21:28:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-05-08T21:28:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-05-08T21:28:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-05-08T21:28:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-05-08T21:28:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-05-08T21:28:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-05-08T21:28:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-05-08T21:28:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-05-08T21:28:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-05-08T21:28:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-05-08T21:28:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-05-08T21:28:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-05-08T21:28:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 89, 2024-05-08T21:28:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-05-08T21:28:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 4, 2024-05-08T21:28:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 12, 2024-05-08T21:28:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-05-08T21:28:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-05-08T21:28:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-05-08T21:28:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 70, 2024-05-08T21:28:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 0, 2024-05-08T21:28:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 25 2024-05-08T21:28:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-05-08T21:28:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-05-08T21:28:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-05-08T21:28:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-05-08T21:28:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-05-08T21:28:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-05-08T21:28:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-05-08T21:28:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-05-08T21:28:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-05-08T21:28:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 17, 2024-05-08T21:28:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-05-08T21:28:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 7, 2024-05-08T21:28:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 1606, 2024-05-08T21:28:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-05-08T21:28:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T21:28:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 53778, 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 54442011, 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 53778, 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 53778, 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 54442011, 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 16981, 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 21753027, 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 16981, 2024-05-08T21:28:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 21753027, 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 16981, 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 16981, 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 16981, 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 16981, 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-08T21:28:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 1606, 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 174742, 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 1606, 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 151, 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 174742, 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x5636c235a000": { 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 25831, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 21274216, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 25831, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 1397, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 21274216, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:41.233 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:41.233 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:41.233 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:41.233 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-05-08T21:28:41.556 INFO:tasks.check_counter:Daemon mds.h mds.exported=2 2024-05-08T21:28:41.557 INFO:tasks.check_counter:Daemon mds.h mds.imported=181 2024-05-08T21:28:41.557 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=304 2024-05-08T21:28:41.557 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=400 2024-05-08T21:28:41.557 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-05-08T21:28:41.557 DEBUG:tasks.check_counter:Getting stats from k 2024-05-08T21:28:41.557 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-08T21:28:41.778 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T21:28:41.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:41 smithi071 ceph-mon[47074]: pgmap v581: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 26 op/s 2024-05-08T21:28:41.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:41 smithi082 ceph-mon[47512]: pgmap v581: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 26 op/s 2024-05-08T21:28:41.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:41 smithi033 ceph-mon[42528]: pgmap v581: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 KiB/s wr, 26 op/s 2024-05-08T21:28:42.782 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-05-08T21:28:42.782 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 324, 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 324, 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 12671195, 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 100032, 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 7, 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.033838919, 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.010266476, 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.012679939, 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.007158798, 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 323, 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.016238066, 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000050272 2024-05-08T21:28:42.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 596, 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 743, 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 12730168, 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 506234, 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 5, 2024-05-08T21:28:42.784 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 4, 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.058942885, 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.016529138, 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.019076297, 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.017811014, 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 741, 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.025381736, 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000034253 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.785 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 321184 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 1355, 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1331, 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 8367776, 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 462897, 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.119142962, 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.035578152, 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.042291976, 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.034188992, 2024-05-08T21:28:42.786 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:42.787 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1330, 2024-05-08T21:28:42.787 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.056746942, 2024-05-08T21:28:42.787 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000042666 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 608912, 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 112400 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.788 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-05-08T21:28:42.789 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-08T21:28:42.789 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-08T21:28:42.789 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3430, 2024-05-08T21:28:42.789 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.084147793, 2024-05-08T21:28:42.789 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000024532 2024-05-08T21:28:42.789 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:42.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.789 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-05-08T21:28:42.789 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 0, 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 0, 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.790 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 0, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 0, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 1387, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 0, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 1387, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 0, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 1387, 2024-05-08T21:28:42.791 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 0, 2024-05-08T21:28:42.792 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 0, 2024-05-08T21:28:42.792 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 0, 2024-05-08T21:28:42.792 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 1, 2024-05-08T21:28:42.792 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 0, 2024-05-08T21:28:42.792 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 0, 2024-05-08T21:28:42.792 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 0, 2024-05-08T21:28:42.792 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 0, 2024-05-08T21:28:42.792 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 0, 2024-05-08T21:28:42.793 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 0, 2024-05-08T21:28:42.794 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T21:28:42.794 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-05-08T21:28:42.794 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T21:28:42.794 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-05-08T21:28:42.794 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-05-08T21:28:42.794 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T21:28:42.794 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T21:28:42.794 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-05-08T21:28:42.794 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-05-08T21:28:42.795 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T21:28:42.795 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 0, 2024-05-08T21:28:42.795 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T21:28:42.796 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T21:28:42.796 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T21:28:42.796 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 0, 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 0, 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 0, 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 0 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 0, 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 0, 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 0, 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 0, 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 0, 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-05-08T21:28:42.797 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-05-08T21:28:42.798 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 0, 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 46616, 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 0, 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 51, 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-05-08T21:28:42.799 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 36775060, 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 36774461, 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 46616 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 13, 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 1411, 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 1398, 2024-05-08T21:28:42.800 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 11, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 75, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 64, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 1387, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 2544, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 1157, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 0, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 0, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 0, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 54500, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 207156 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 0, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 0, 2024-05-08T21:28:42.801 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 0, 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 0, 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 0, 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-05-08T21:28:42.802 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.803 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.805 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-05-08T21:28:42.806 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.808 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.809 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-05-08T21:28:42.809 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.809 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.809 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.809 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.809 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-05-08T21:28:42.809 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.809 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.810 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.812 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 1, 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 1, 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 0, 2024-05-08T21:28:42.813 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 1, 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 0, 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 0, 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 2097, 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-05-08T21:28:42.814 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T21:28:42.815 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-05-08T21:28:42.816 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 206367, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 61, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9080, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 174, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T21:28:42.817 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 1019208, 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 5676, 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 2042, 2024-05-08T21:28:42.818 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 21, 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 3, 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 2042, 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2042, 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.178979922, 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000577365 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2042, 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 2043 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 2042, 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 2042, 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 0, 2024-05-08T21:28:42.819 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 1269, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 771, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-05-08T21:28:42.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 1, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-05-08T21:28:42.821 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 89, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-05-08T21:28:42.822 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 4, 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 7, 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 7, 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 0, 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 2, 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 0 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 0, 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.823 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 0, 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 0, 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 2271, 2024-05-08T21:28:42.824 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 33550173, 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 2271, 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2271, 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 33550173, 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:42.825 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 2042, 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 32582181, 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2042, 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 32582181, 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:42.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 2042, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 2042, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 2042, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 2042, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.827 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-05-08T21:28:42.828 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x55c4d3154140": { 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-05-08T21:28:42.829 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-05-08T21:28:42.830 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-05-08T21:28:42.830 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-05-08T21:28:42.830 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-05-08T21:28:42.830 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-05-08T21:28:42.830 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-05-08T21:28:42.830 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-05-08T21:28:42.830 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:42.830 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-05-08T21:28:42.830 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-05-08T21:28:43.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:43 smithi071 ceph-mon[47074]: pgmap v582: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-08T21:28:43.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:43 smithi082 ceph-mon[47512]: pgmap v582: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-08T21:28:43.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:43 smithi033 ceph-mon[42528]: pgmap v582: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-08T21:28:44.045 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-08T21:28:44.045 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-08T21:28:44.045 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-05-08T21:28:44.045 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-05-08T21:28:44.045 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-05-08T21:28:44.046 DEBUG:tasks.check_counter:Getting stats from c 2024-05-08T21:28:44.046 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-08T21:28:44.294 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:28:44.766 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:44 smithi082 ceph-mon[47512]: pgmap v583: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-08T21:28:44.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:44 smithi033 ceph-mon[42528]: pgmap v583: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-08T21:28:45.069 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:44 smithi071 ceph-mon[47074]: pgmap v583: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-08T21:28:45.352 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-08T21:28:45.352 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T21:28:45.352 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 15106, 2024-05-08T21:28:45.352 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 19290, 2024-05-08T21:28:45.352 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 4413935, 2024-05-08T21:28:45.352 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 22543275, 2024-05-08T21:28:45.352 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 15, 2024-05-08T21:28:45.352 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-05-08T21:28:45.352 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.928887572, 2024-05-08T21:28:45.352 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.348422985, 2024-05-08T21:28:45.353 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.487625754, 2024-05-08T21:28:45.353 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.147518433, 2024-05-08T21:28:45.353 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:45.353 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 19288, 2024-05-08T21:28:45.354 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.108642536, 2024-05-08T21:28:45.354 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000057478 2024-05-08T21:28:45.354 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.354 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:45.354 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 19225, 2024-05-08T21:28:45.354 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003061884, 2024-05-08T21:28:45.354 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000159 2024-05-08T21:28:45.354 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.354 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T21:28:45.354 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T21:28:45.354 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.355 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T21:28:45.355 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 7090, 2024-05-08T21:28:45.355 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 7236, 2024-05-08T21:28:45.355 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1799853, 2024-05-08T21:28:45.355 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 15264842, 2024-05-08T21:28:45.356 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 17, 2024-05-08T21:28:45.356 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 7, 2024-05-08T21:28:45.356 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.447118561, 2024-05-08T21:28:45.356 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.159420309, 2024-05-08T21:28:45.356 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.098470105, 2024-05-08T21:28:45.356 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.159758707, 2024-05-08T21:28:45.356 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:45.356 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 7233, 2024-05-08T21:28:45.356 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.793367800, 2024-05-08T21:28:45.357 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000109687 2024-05-08T21:28:45.357 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.357 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:45.357 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6, 2024-05-08T21:28:45.357 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000759, 2024-05-08T21:28:45.357 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000126 2024-05-08T21:28:45.357 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.357 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-05-08T21:28:45.357 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 324464 2024-05-08T21:28:45.357 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.358 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T21:28:45.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 24401, 2024-05-08T21:28:45.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 17366, 2024-05-08T21:28:45.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 51203387, 2024-05-08T21:28:45.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 13669597, 2024-05-08T21:28:45.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 35, 2024-05-08T21:28:45.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 8, 2024-05-08T21:28:45.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 1.215285041, 2024-05-08T21:28:45.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.457704902, 2024-05-08T21:28:45.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.954599718, 2024-05-08T21:28:45.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.038564139, 2024-05-08T21:28:45.359 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:45.359 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 17367, 2024-05-08T21:28:45.359 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.449545273, 2024-05-08T21:28:45.359 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000025885 2024-05-08T21:28:45.359 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.359 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:45.359 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 38355, 2024-05-08T21:28:45.359 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.005735141, 2024-05-08T21:28:45.359 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000149 2024-05-08T21:28:45.359 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.359 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 623616, 2024-05-08T21:28:45.360 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 134688 2024-05-08T21:28:45.360 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.360 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-05-08T21:28:45.360 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-05-08T21:28:45.360 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-05-08T21:28:45.360 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.360 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-05-08T21:28:45.360 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-05-08T21:28:45.360 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-05-08T21:28:45.360 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 9552, 2024-05-08T21:28:45.361 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 4.369333119, 2024-05-08T21:28:45.361 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000457425 2024-05-08T21:28:45.361 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:45.361 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.361 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-05-08T21:28:45.361 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-05-08T21:28:45.361 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-05-08T21:28:45.361 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2852, 2024-05-08T21:28:45.361 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.190294794, 2024-05-08T21:28:45.361 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000066723 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 7167, 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 4860, 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4860, 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 5.433743788, 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001118054 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 2307, 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-05-08T21:28:45.362 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-05-08T21:28:45.363 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 11, 2024-05-08T21:28:45.363 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-05-08T21:28:45.363 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-05-08T21:28:45.363 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 19, 2024-05-08T21:28:45.363 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 3, 2024-05-08T21:28:45.363 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 5, 2024-05-08T21:28:45.363 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 11, 2024-05-08T21:28:45.363 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 14, 2024-05-08T21:28:45.363 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 265, 2024-05-08T21:28:45.363 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 1, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 1, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 5, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 13296, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 6033, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 2307, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 106, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 819, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 47, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 111, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 47, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 1115, 2024-05-08T21:28:45.364 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 10680, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 3568, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 64348743, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 10680, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 10680, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-05-08T21:28:45.365 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 90, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 1771, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 609, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 442, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 66, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 3140 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 121, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 582, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 327, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 475, 2024-05-08T21:28:45.366 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 129, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 1377, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 1377, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 47, 2024-05-08T21:28:45.367 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 46616, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 46616, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 51, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 51, 2024-05-08T21:28:45.368 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 36775060, 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1056, 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.431010690, 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003249063 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 25, 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 1490, 2024-05-08T21:28:45.369 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 1465, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 24, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 321, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 297, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 19, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 2708, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 2689, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 1, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 1389, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 1388, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 65540, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 198964 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 10061, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 11, 2024-05-08T21:28:45.370 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 7167, 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 8, 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 917, 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 386, 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.347898353, 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003491964 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000262208, 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000262208 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.371 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3032, 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.026490211, 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000338552 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.372 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.373 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.374 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.375 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 64, 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.038784201, 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000606003 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.376 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.377 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.378 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.379 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1377, 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.020308815, 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002193397 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.380 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 1, 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 0, 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 1635, 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 1635, 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 595, 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.381 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T21:28:45.382 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T21:28:45.383 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 300753, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 126, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-05-08T21:28:45.384 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-05-08T21:28:45.385 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-05-08T21:28:45.385 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9080, 2024-05-08T21:28:45.385 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 174, 2024-05-08T21:28:45.385 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T21:28:45.385 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-05-08T21:28:45.385 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-05-08T21:28:45.385 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 250989, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 1833, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 14871, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 32845500, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 27, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 14871, 2024-05-08T21:28:45.386 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-05-08T21:28:45.387 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 14871, 2024-05-08T21:28:45.387 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 30.286431380, 2024-05-08T21:28:45.387 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002036610 2024-05-08T21:28:45.387 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.387 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-05-08T21:28:45.387 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-05-08T21:28:45.387 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 14871, 2024-05-08T21:28:45.387 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 14942 2024-05-08T21:28:45.387 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.387 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 14871, 2024-05-08T21:28:45.387 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 10796, 2024-05-08T21:28:45.387 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 4075, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 10, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 116, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 1172, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 78, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 2773, 2024-05-08T21:28:45.388 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 10680, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 20, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-05-08T21:28:45.389 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-05-08T21:28:45.390 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 89, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 4, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 12, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 12, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 69, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 24 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-05-08T21:28:45.391 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 10, 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 5, 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 1377, 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T21:28:45.392 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 46593, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 53206897, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 46593, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 46593, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 53206897, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.393 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 14871, 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 32995593, 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 14871, 2024-05-08T21:28:45.394 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 32995593, 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:45.395 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 14871, 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 14871, 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 14871, 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 14871, 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:45.396 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 1377, 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 150093, 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 1377, 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 116, 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 150093, 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x561bc8260000": { 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:45.397 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 46616, 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 32580756, 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 46616, 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 1056, 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 32580756, 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:45.398 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:45.399 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-08T21:28:45.770 INFO:tasks.check_counter:Daemon mds.c mds.exported=47 2024-05-08T21:28:45.770 INFO:tasks.check_counter:Daemon mds.c mds.imported=47 2024-05-08T21:28:45.770 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=121 2024-05-08T21:28:45.770 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=582 2024-05-08T21:28:45.770 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-05-08T21:28:45.770 DEBUG:tasks.check_counter:Getting stats from f 2024-05-08T21:28:45.771 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-08T21:28:45.991 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:28:47.250 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-08T21:28:47.250 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T21:28:47.250 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1483, 2024-05-08T21:28:47.250 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 1485, 2024-05-08T21:28:47.250 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 35841881, 2024-05-08T21:28:47.250 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 461223, 2024-05-08T21:28:47.250 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 8, 2024-05-08T21:28:47.250 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-05-08T21:28:47.250 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.135422001, 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.034406805, 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.049457695, 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.039505338, 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1483, 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.052630504, 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000035489 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T21:28:47.251 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 721, 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 695, 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 34292333, 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 264842, 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 5, 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.071118902, 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.019094494, 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.036894456, 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.014041209, 2024-05-08T21:28:47.252 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 695, 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.024434335, 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000035157 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 612032, 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 113008 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T21:28:47.253 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 551, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 697, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 25305699, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 495060, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.061409786, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.019079329, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.023052392, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.013992377, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 695, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.025433703, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000036595 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.254 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 324320 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 4285, 2024-05-08T21:28:47.255 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.092891399, 2024-05-08T21:28:47.256 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000021678 2024-05-08T21:28:47.256 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:47.256 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.256 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-05-08T21:28:47.256 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-05-08T21:28:47.256 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-05-08T21:28:47.256 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.256 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.256 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.256 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:47.256 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.256 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-05-08T21:28:47.257 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 0, 2024-05-08T21:28:47.257 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 0, 2024-05-08T21:28:47.257 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-05-08T21:28:47.257 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.257 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.257 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.257 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.257 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 1082, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 1082, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 0, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 1082, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 0, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 0, 2024-05-08T21:28:47.258 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 0, 2024-05-08T21:28:47.259 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 2, 2024-05-08T21:28:47.259 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 0, 2024-05-08T21:28:47.259 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 0, 2024-05-08T21:28:47.259 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2024-05-08T21:28:47.259 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2024-05-08T21:28:47.259 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-05-08T21:28:47.259 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-05-08T21:28:47.259 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-05-08T21:28:47.259 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 4, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 234, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T21:28:47.260 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-05-08T21:28:47.261 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-05-08T21:28:47.261 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T21:28:47.261 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T21:28:47.261 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2024-05-08T21:28:47.261 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-05-08T21:28:47.261 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T21:28:47.261 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2024-05-08T21:28:47.261 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T21:28:47.261 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T21:28:47.261 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T21:28:47.261 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2024-05-08T21:28:47.262 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2024-05-08T21:28:47.263 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-05-08T21:28:47.263 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-05-08T21:28:47.263 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-05-08T21:28:47.263 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-05-08T21:28:47.263 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-05-08T21:28:47.263 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-05-08T21:28:47.263 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-05-08T21:28:47.263 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T21:28:47.264 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2024-05-08T21:28:47.264 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-05-08T21:28:47.264 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-05-08T21:28:47.264 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-05-08T21:28:47.264 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-05-08T21:28:47.264 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.264 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-05-08T21:28:47.264 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 0, 2024-05-08T21:28:47.264 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-05-08T21:28:47.264 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-05-08T21:28:47.264 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 43873, 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 60, 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 98321448, 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 98317147, 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-05-08T21:28:47.265 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.266 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.266 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.266 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.266 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 43873 2024-05-08T21:28:47.266 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.266 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-05-08T21:28:47.266 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 23, 2024-05-08T21:28:47.266 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 5137, 2024-05-08T21:28:47.266 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 5114, 2024-05-08T21:28:47.266 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 18, 2024-05-08T21:28:47.266 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 269, 2024-05-08T21:28:47.267 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 251, 2024-05-08T21:28:47.267 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 1082, 2024-05-08T21:28:47.267 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 5675, 2024-05-08T21:28:47.267 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 4593, 2024-05-08T21:28:47.267 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 0, 2024-05-08T21:28:47.267 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 0, 2024-05-08T21:28:47.267 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2024-05-08T21:28:47.267 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 58284, 2024-05-08T21:28:47.267 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 207156 2024-05-08T21:28:47.268 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.268 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-05-08T21:28:47.268 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 0, 2024-05-08T21:28:47.268 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2024-05-08T21:28:47.268 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 0, 2024-05-08T21:28:47.268 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 0, 2024-05-08T21:28:47.268 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2024-05-08T21:28:47.268 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-05-08T21:28:47.268 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.268 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.268 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.269 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.269 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-05-08T21:28:47.269 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.269 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.269 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.269 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.269 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-05-08T21:28:47.269 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.269 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.269 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.269 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.270 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.271 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.272 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.273 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.274 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.275 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.276 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-05-08T21:28:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 3, 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 2, 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 2, 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 0, 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 0, 2024-05-08T21:28:47.279 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 3518, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T21:28:47.280 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-05-08T21:28:47.281 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 231391, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 75, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-05-08T21:28:47.282 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9080, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 174, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 834877, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 4688, 2024-05-08T21:28:47.283 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 2520, 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 26, 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 4, 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 2520, 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2520, 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.607969554, 2024-05-08T21:28:47.284 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000638083 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2520, 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2521 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 2520, 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 2520, 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 0, 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 1360, 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2024-05-08T21:28:47.285 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 1157, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 0, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 0, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 0, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 2, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2024-05-08T21:28:47.286 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-05-08T21:28:47.287 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 89, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 4, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 10, 2024-05-08T21:28:47.288 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 10, 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 0, 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 2, 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-05-08T21:28:47.289 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 2751, 2024-05-08T21:28:47.290 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 95177701, 2024-05-08T21:28:47.291 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:47.291 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 2751, 2024-05-08T21:28:47.291 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-08T21:28:47.291 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-08T21:28:47.291 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2751, 2024-05-08T21:28:47.291 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 95177701, 2024-05-08T21:28:47.291 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:47.291 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.291 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.291 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.291 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:47.291 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2520, 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 94132031, 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2520, 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 94132031, 2024-05-08T21:28:47.292 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:47.293 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 2520, 2024-05-08T21:28:47.294 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 2520, 2024-05-08T21:28:47.294 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2520, 2024-05-08T21:28:47.294 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 2520, 2024-05-08T21:28:47.294 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:47.294 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.294 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.294 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.294 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:47.294 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.294 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.295 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x55705cc821e0": { 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:47.296 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:47.297 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:47.297 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:47.297 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:47.298 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-08T21:28:47.628 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-08T21:28:47.628 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-08T21:28:47.628 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-08T21:28:47.629 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-08T21:28:47.629 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-05-08T21:28:47.629 DEBUG:tasks.check_counter:Getting stats from i 2024-05-08T21:28:47.629 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-08T21:28:47.655 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:47 smithi082 ceph-mon[47512]: pgmap v584: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 13 op/s 2024-05-08T21:28:47.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:47 smithi033 ceph-mon[42528]: pgmap v584: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 13 op/s 2024-05-08T21:28:47.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:47 smithi071 ceph-mon[47074]: pgmap v584: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 13 op/s 2024-05-08T21:28:47.849 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:28:48.881 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-08T21:28:48.882 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T21:28:48.882 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 200, 2024-05-08T21:28:48.882 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 195, 2024-05-08T21:28:48.882 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 468288, 2024-05-08T21:28:48.882 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 110384, 2024-05-08T21:28:48.883 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2024-05-08T21:28:48.883 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2024-05-08T21:28:48.883 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.028317136, 2024-05-08T21:28:48.883 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.008018456, 2024-05-08T21:28:48.883 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.014405217, 2024-05-08T21:28:48.883 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.004900778, 2024-05-08T21:28:48.884 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:48.884 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 195, 2024-05-08T21:28:48.884 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.010860121, 2024-05-08T21:28:48.884 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000055692 2024-05-08T21:28:48.884 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:48.884 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:48.884 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:48.884 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:48.884 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:48.884 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:48.884 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 468288, 2024-05-08T21:28:48.885 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 110384 2024-05-08T21:28:48.885 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:48.885 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T21:28:48.885 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1, 2024-05-08T21:28:48.885 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 147, 2024-05-08T21:28:48.885 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 160, 2024-05-08T21:28:48.885 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 101952, 2024-05-08T21:28:48.885 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 3, 2024-05-08T21:28:48.885 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2024-05-08T21:28:48.886 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.007551635, 2024-05-08T21:28:48.886 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.005214765, 2024-05-08T21:28:48.886 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000943679, 2024-05-08T21:28:48.886 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T21:28:48.886 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:48.886 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 145, 2024-05-08T21:28:48.886 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.006579722, 2024-05-08T21:28:48.886 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000045377 2024-05-08T21:28:48.887 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:48.887 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:48.887 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:48.887 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:48.887 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:48.887 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:48.887 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T21:28:48.887 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 101952 2024-05-08T21:28:48.887 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:48.887 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T21:28:48.887 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2024-05-08T21:28:48.887 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.000706937, 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000627432, 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:48.888 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T21:28:48.889 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-05-08T21:28:48.890 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-05-08T21:28:48.891 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 149998, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 30, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-05-08T21:28:48.892 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:48.893 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 197, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 437603, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 197, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 197, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 437603, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:48.894 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:48.895 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:48.895 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-08T21:28:49.294 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.i 2024-05-08T21:28:49.294 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.i 2024-05-08T21:28:49.294 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.i 2024-05-08T21:28:49.294 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.i 2024-05-08T21:28:49.295 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.i 2024-05-08T21:28:49.295 DEBUG:tasks.check_counter:Getting stats from l 2024-05-08T21:28:49.295 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d21e4f961318b56f0d915caa2490c3711bf13c34 shell --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-08T21:28:49.556 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T21:28:49.681 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:49 smithi082 ceph-mon[47512]: pgmap v585: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-08T21:28:49.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:49 smithi033 ceph-mon[42528]: pgmap v585: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-08T21:28:49.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:49 smithi071 ceph-mon[47074]: pgmap v585: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-08T21:28:50.597 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-05-08T21:28:50.597 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T21:28:50.597 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 14286, 2024-05-08T21:28:50.597 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 19171, 2024-05-08T21:28:50.597 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 4492613, 2024-05-08T21:28:50.597 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 32902475, 2024-05-08T21:28:50.597 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 16, 2024-05-08T21:28:50.597 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 6, 2024-05-08T21:28:50.597 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.947879557, 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.384365516, 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.543282822, 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.102363930, 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 19168, 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.716162653, 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000037362 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 23163, 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003608179, 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000155 2024-05-08T21:28:50.598 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 28759, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 22165, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 52531633, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 17366197, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 11, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 6, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 1.426015573, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.512203337, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.990405763, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.137496020, 2024-05-08T21:28:50.599 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 22166, 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.046046489, 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000047191 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 37895, 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.005517586, 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000145 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 614208, 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 134896 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 6678, 2024-05-08T21:28:50.600 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 6336, 2024-05-08T21:28:50.601 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 2376921, 2024-05-08T21:28:50.601 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 25829850, 2024-05-08T21:28:50.601 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 41, 2024-05-08T21:28:50.601 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 10, 2024-05-08T21:28:50.601 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.434312106, 2024-05-08T21:28:50.601 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.172801110, 2024-05-08T21:28:50.601 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.141851780, 2024-05-08T21:28:50.601 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.114735941, 2024-05-08T21:28:50.601 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T21:28:50.601 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6334, 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.622966725, 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000098352 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3260, 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000500661, 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000153 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 326352 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2024-05-08T21:28:50.602 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 9969, 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 5.594593167, 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000561199 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2824, 2024-05-08T21:28:50.603 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.191618062, 2024-05-08T21:28:50.604 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000067853 2024-05-08T21:28:50.604 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:50.604 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.604 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2024-05-08T21:28:50.604 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 8213, 2024-05-08T21:28:50.604 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 6025, 2024-05-08T21:28:50.604 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2024-05-08T21:28:50.604 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6025, 2024-05-08T21:28:50.604 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 7.216750523, 2024-05-08T21:28:50.605 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001197800 2024-05-08T21:28:50.605 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.605 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2024-05-08T21:28:50.605 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 2188, 2024-05-08T21:28:50.605 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 11, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 21, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 3, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 7, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 11, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 14, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 836, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 0, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 0, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 6, 2024-05-08T21:28:50.606 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 16071, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 6002, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 2187, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 101, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 75, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 605, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 75, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 1428, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 10269, 2024-05-08T21:28:50.607 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 3568, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 64348743, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 10269, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 10269, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 66, 2024-05-08T21:28:50.608 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 1987, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 1009, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 676, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 72, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 3378 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 142, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 540, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 307, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 450, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 164, 2024-05-08T21:28:50.609 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 1400, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 1400, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 89, 2024-05-08T21:28:50.610 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 73086, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 73086, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 87, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2024-05-08T21:28:50.611 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 87, 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 56676410, 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1411, 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.264917882, 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002313903 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 28, 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 1858, 2024-05-08T21:28:50.612 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 1830, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 27, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 318, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 291, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 21, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 3208, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 3187, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 0, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 1765, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 1765, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 65756, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 198964 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 11791, 2024-05-08T21:28:50.613 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 12, 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 8213, 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 33, 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 1250, 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 578, 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2.252514305, 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003897083 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2024-05-08T21:28:50.614 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.615 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.615 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.615 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.615 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2024-05-08T21:28:50.615 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.615 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.615 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.615 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.615 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2024-05-08T21:28:50.615 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3976, 2024-05-08T21:28:50.615 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.503491102, 2024-05-08T21:28:50.616 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000378141 2024-05-08T21:28:50.616 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.616 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2024-05-08T21:28:50.616 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.616 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.616 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.616 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.616 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2024-05-08T21:28:50.616 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.616 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.616 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2024-05-08T21:28:50.617 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.618 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.618 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.618 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.618 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2024-05-08T21:28:50.618 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.618 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.618 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.618 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.618 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2024-05-08T21:28:50.618 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.618 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.619 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.619 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.619 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2024-05-08T21:28:50.619 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.619 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.619 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.619 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.619 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2024-05-08T21:28:50.619 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.619 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.619 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.619 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.620 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2024-05-08T21:28:50.620 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2024-05-08T21:28:50.620 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000796938, 2024-05-08T21:28:50.620 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000796938 2024-05-08T21:28:50.620 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.620 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2024-05-08T21:28:50.620 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 70, 2024-05-08T21:28:50.620 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.046206828, 2024-05-08T21:28:50.620 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000660097 2024-05-08T21:28:50.620 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.620 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2024-05-08T21:28:50.621 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.621 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.621 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.621 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.621 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2024-05-08T21:28:50.621 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.621 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.621 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.621 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.621 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2024-05-08T21:28:50.621 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.621 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.622 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.622 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.622 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2024-05-08T21:28:50.622 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.622 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.622 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.622 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.622 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2024-05-08T21:28:50.622 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.622 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.622 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2024-05-08T21:28:50.623 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.624 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.624 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.624 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.624 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2024-05-08T21:28:50.624 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.624 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.624 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.624 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.624 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2024-05-08T21:28:50.624 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.624 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.624 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2024-05-08T21:28:50.625 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1400, 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.413741350, 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002438386 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2024-05-08T21:28:50.626 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2024-05-08T21:28:50.627 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 1, 2024-05-08T21:28:50.627 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 0, 2024-05-08T21:28:50.627 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2024-05-08T21:28:50.627 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2024-05-08T21:28:50.627 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 1572, 2024-05-08T21:28:50.627 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 1572, 2024-05-08T21:28:50.627 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 600, 2024-05-08T21:28:50.627 INFO:teuthology.orchestra.run.smithi082.stdout: "mdthresh_evicted": 0 2024-05-08T21:28:50.627 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.627 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2024-05-08T21:28:50.627 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2024-05-08T21:28:50.627 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T21:28:50.628 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T21:28:50.629 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 283197, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 117, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2024-05-08T21:28:50.630 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2024-05-08T21:28:50.631 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2024-05-08T21:28:50.631 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2024-05-08T21:28:50.631 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2024-05-08T21:28:50.631 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2024-05-08T21:28:50.631 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2024-05-08T21:28:50.631 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 9080, 2024-05-08T21:28:50.631 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 174, 2024-05-08T21:28:50.631 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T21:28:50.631 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2024-05-08T21:28:50.631 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2024-05-08T21:28:50.631 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2024-05-08T21:28:50.631 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 173461, 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 1315, 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 14888, 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 52763010, 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 33, 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 14888, 2024-05-08T21:28:50.632 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2024-05-08T21:28:50.633 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 14888, 2024-05-08T21:28:50.633 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 28.087471897, 2024-05-08T21:28:50.633 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001886584 2024-05-08T21:28:50.633 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.633 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2024-05-08T21:28:50.633 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2024-05-08T21:28:50.633 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 14888, 2024-05-08T21:28:50.633 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 14956 2024-05-08T21:28:50.633 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.633 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 14888, 2024-05-08T21:28:50.633 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 10395, 2024-05-08T21:28:50.633 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 4493, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 10, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 126, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 1537, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 82, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 2824, 2024-05-08T21:28:50.634 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2024-05-08T21:28:50.635 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2024-05-08T21:28:50.635 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2024-05-08T21:28:50.635 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 10269, 2024-05-08T21:28:50.635 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 20, 2024-05-08T21:28:50.635 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2024-05-08T21:28:50.635 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2024-05-08T21:28:50.635 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2024-05-08T21:28:50.635 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2024-05-08T21:28:50.635 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2024-05-08T21:28:50.636 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 89, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 4, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 12, 2024-05-08T21:28:50.637 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 12, 2024-05-08T21:28:50.638 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2024-05-08T21:28:50.638 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2024-05-08T21:28:50.638 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 65, 2024-05-08T21:28:50.638 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2024-05-08T21:28:50.638 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 23 2024-05-08T21:28:50.638 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.638 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2024-05-08T21:28:50.638 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2024-05-08T21:28:50.638 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2024-05-08T21:28:50.638 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2024-05-08T21:28:50.638 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2024-05-08T21:28:50.638 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 10, 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 5, 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 1400, 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:50.639 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 49719, 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 54909170, 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 49719, 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 49719, 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 54909170, 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.640 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.641 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:50.641 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.641 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2024-05-08T21:28:50.641 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:50.641 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2024-05-08T21:28:50.641 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:50.641 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-05-08T21:28:50.641 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-05-08T21:28:50.641 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:50.641 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:50.641 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 14888, 2024-05-08T21:28:50.641 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 52915610, 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 14888, 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 52915610, 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:50.642 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 14888, 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 14888, 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 14888, 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 14888, 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:50.643 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 1400, 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 152600, 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 1400, 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 126, 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 152600, 2024-05-08T21:28:50.644 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x55b6ae662000": { 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 73086, 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 52482106, 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2024-05-08T21:28:50.645 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 73086, 2024-05-08T21:28:50.646 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2024-05-08T21:28:50.646 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2024-05-08T21:28:50.646 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 1411, 2024-05-08T21:28:50.646 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 52482106, 2024-05-08T21:28:50.646 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2024-05-08T21:28:50.646 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2024-05-08T21:28:50.646 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2024-05-08T21:28:50.646 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2024-05-08T21:28:50.646 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:50.646 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-05-08T21:28:50.646 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-05-08T21:28:51.018 INFO:tasks.check_counter:Daemon mds.l mds.exported=75 2024-05-08T21:28:51.018 INFO:tasks.check_counter:Daemon mds.l mds.imported=75 2024-05-08T21:28:51.019 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=142 2024-05-08T21:28:51.019 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=540 2024-05-08T21:28:51.019 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-05-08T21:28:51.019 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-08T21:28:51.029 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-08T21:28:51.029 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:28:51.029 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T21:28:51.056 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:28:51.056 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T21:28:51.120 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:28:51.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.313+0000 7f35bdf62700 1 -- 172.21.15.33:0/1580041518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35b80ffd80 msgr2=0x7f35b81001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.313+0000 7f35bdf62700 1 --2- 172.21.15.33:0/1580041518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35b80ffd80 0x7f35b81001a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f35a80099c0 tx=0x7f35a8009cd0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.313+0000 7f35bdf62700 1 -- 172.21.15.33:0/1580041518 shutdown_connections 2024-05-08T21:28:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.313+0000 7f35bdf62700 1 --2- 172.21.15.33:0/1580041518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35b8101910 0x7f35b81081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.313+0000 7f35bdf62700 1 --2- 172.21.15.33:0/1580041518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35b80ffd80 0x7f35b81001a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.313+0000 7f35bdf62700 1 --2- 172.21.15.33:0/1580041518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8100f70 0x7f35b81013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.313+0000 7f35bdf62700 1 -- 172.21.15.33:0/1580041518 >> 172.21.15.33:0/1580041518 conn(0x7f35b80fb310 msgr2=0x7f35b80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.313+0000 7f35bdf62700 1 -- 172.21.15.33:0/1580041518 shutdown_connections 2024-05-08T21:28:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.313+0000 7f35bdf62700 1 -- 172.21.15.33:0/1580041518 wait complete. 2024-05-08T21:28:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.313+0000 7f35bdf62700 1 Processor -- start 2024-05-08T21:28:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35bdf62700 1 -- start start 2024-05-08T21:28:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35bdf62700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35b80ffd80 0x7f35b8196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35bdf62700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35b8100f70 0x7f35b8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35bdf62700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101910 0x7f35b819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35bdf62700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b819e8e0 con 0x7f35b80ffd80 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35bdf62700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b819ea20 con 0x7f35b8101910 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35bdf62700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b819ebb0 con 0x7f35b8100f70 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35b80ffd80 0x7f35b8196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35b77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35b80ffd80 0x7f35b8196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42282/0 (socket says 172.21.15.33:42282) 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35b77fe700 1 -- 172.21.15.33:0/2606439299 learned_addr learned my addr 172.21.15.33:0/2606439299 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35b6ffd700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35b8100f70 0x7f35b8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35b7fff700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101910 0x7f35b819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35b77fe700 1 -- 172.21.15.33:0/2606439299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35b8100f70 msgr2=0x7f35b8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35b77fe700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35b8100f70 0x7f35b8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35b77fe700 1 -- 172.21.15.33:0/2606439299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101910 msgr2=0x7f35b819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35b77fe700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101910 0x7f35b819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.314+0000 7f35b77fe700 1 -- 172.21.15.33:0/2606439299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a80095f0 con 0x7f35b80ffd80 2024-05-08T21:28:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.315+0000 7f35b77fe700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35b80ffd80 0x7f35b8196e00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009990 tx=0x7f35a800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.315+0000 7f35b4ff9700 1 -- 172.21.15.33:0/2606439299 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a801d070 con 0x7f35b80ffd80 2024-05-08T21:28:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.315+0000 7f35b4ff9700 1 -- 172.21.15.33:0/2606439299 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35a800bd10 con 0x7f35b80ffd80 2024-05-08T21:28:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.315+0000 7f35bdf62700 1 -- 172.21.15.33:0/2606439299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b819ee30 con 0x7f35b80ffd80 2024-05-08T21:28:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.315+0000 7f35b4ff9700 1 -- 172.21.15.33:0/2606439299 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8017650 con 0x7f35b80ffd80 2024-05-08T21:28:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.315+0000 7f35bdf62700 1 -- 172.21.15.33:0/2606439299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b819f1a0 con 0x7f35b80ffd80 2024-05-08T21:28:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.316+0000 7f35bdf62700 1 -- 172.21.15.33:0/2606439299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b8067d80 con 0x7f35b80ffd80 2024-05-08T21:28:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.317+0000 7f35b4ff9700 1 -- 172.21.15.33:0/2606439299 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f35a8022470 con 0x7f35b80ffd80 2024-05-08T21:28:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.318+0000 7f35b4ff9700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f35a00a1120 0x7f35a00a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.318+0000 7f35b6ffd700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f35a00a1120 0x7f35a00a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.318+0000 7f35b4ff9700 1 -- 172.21.15.33:0/2606439299 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f35a80ce1b0 con 0x7f35b80ffd80 2024-05-08T21:28:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.319+0000 7f35b6ffd700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f35a00a1120 0x7f35a00a35e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009c60 tx=0x7f35ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:51.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.322+0000 7f35b4ff9700 1 -- 172.21.15.33:0/2606439299 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f35a8097650 con 0x7f35b80ffd80 2024-05-08T21:28:51.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:51 smithi033 ceph-mon[42528]: pgmap v586: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-08T21:28:51.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.545+0000 7f35bdf62700 1 -- 172.21.15.33:0/2606439299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35b810b490 con 0x7f35b80ffd80 2024-05-08T21:28:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.547+0000 7f35b4ff9700 1 -- 172.21.15.33:0/2606439299 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24561 (secure 0 0 0) 0x7f35a809ac70 con 0x7f35b80ffd80 2024-05-08T21:28:51.549 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:28:51.549 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":89,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:18:58.419274+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:28:51.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.550+0000 7f35bdf62700 1 -- 172.21.15.33:0/2606439299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f35a00a1120 msgr2=0x7f35a00a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.550+0000 7f35bdf62700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f35a00a1120 0x7f35a00a35e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009c60 tx=0x7f35ac009260 comp rx=0 tx=0).stop 2024-05-08T21:28:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.550+0000 7f35bdf62700 1 -- 172.21.15.33:0/2606439299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35b80ffd80 msgr2=0x7f35b8196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.550+0000 7f35bdf62700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35b80ffd80 0x7f35b8196e00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009990 tx=0x7f35a800fa00 comp rx=0 tx=0).stop 2024-05-08T21:28:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.550+0000 7f35bdf62700 1 -- 172.21.15.33:0/2606439299 shutdown_connections 2024-05-08T21:28:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.550+0000 7f35bdf62700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f35a00a1120 0x7f35a00a35e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.550+0000 7f35bdf62700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35b8101910 0x7f35b819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.550+0000 7f35bdf62700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35b80ffd80 0x7f35b8196e00 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.550+0000 7f35bdf62700 1 --2- 172.21.15.33:0/2606439299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35b8100f70 0x7f35b8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.550+0000 7f35bdf62700 1 -- 172.21.15.33:0/2606439299 >> 172.21.15.33:0/2606439299 conn(0x7f35b80fb310 msgr2=0x7f35b80681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.550+0000 7f35bdf62700 1 -- 172.21.15.33:0/2606439299 shutdown_connections 2024-05-08T21:28:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.550+0000 7f35bdf62700 1 -- 172.21.15.33:0/2606439299 wait complete. 2024-05-08T21:28:51.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:17:01.282803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:17:02.555552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T21:17:05.686277+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T21:28:51.567 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:28:51.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.708+0000 7f1ec3056700 1 -- 172.21.15.33:0/893663213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc101fc0 msgr2=0x7f1ebc102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:51.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.708+0000 7f1ec3056700 1 --2- 172.21.15.33:0/893663213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc101fc0 0x7f1ebc102440 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009990 tx=0x7f1eb0009ca0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.708+0000 7f1ec3056700 1 -- 172.21.15.33:0/893663213 shutdown_connections 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.708+0000 7f1ec3056700 1 --2- 172.21.15.33:0/893663213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ebc102980 0x7f1ebc1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.708+0000 7f1ec3056700 1 --2- 172.21.15.33:0/893663213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc100e70 0x7f1ebc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.708+0000 7f1ec3056700 1 --2- 172.21.15.33:0/893663213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc101fc0 0x7f1ebc102440 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.708+0000 7f1ec3056700 1 -- 172.21.15.33:0/893663213 >> 172.21.15.33:0/893663213 conn(0x7f1ebc0fc400 msgr2=0x7f1ebc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.708+0000 7f1ec3056700 1 -- 172.21.15.33:0/893663213 shutdown_connections 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.708+0000 7f1ec3056700 1 -- 172.21.15.33:0/893663213 wait complete. 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.708+0000 7f1ec3056700 1 Processor -- start 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.708+0000 7f1ec3056700 1 -- start start 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec3056700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ebc100e70 0x7f1ebc196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec3056700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc101fc0 0x7f1ebc197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec3056700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc102980 0x7f1ebc19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec3056700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc19e820 con 0x7f1ebc102980 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec3056700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc19e990 con 0x7f1ebc101fc0 2024-05-08T21:28:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec3056700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc19eb50 con 0x7f1ebc100e70 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec15f3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc102980 0x7f1ebc19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ebbfff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc101fc0 0x7f1ebc197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec15f3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc102980 0x7f1ebc19e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42304/0 (socket says 172.21.15.33:42304) 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec15f3700 1 -- 172.21.15.33:0/3706336970 learned_addr learned my addr 172.21.15.33:0/3706336970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec0df2700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ebc100e70 0x7f1ebc196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec15f3700 1 -- 172.21.15.33:0/3706336970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ebc100e70 msgr2=0x7f1ebc196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec15f3700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ebc100e70 0x7f1ebc196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec15f3700 1 -- 172.21.15.33:0/3706336970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc101fc0 msgr2=0x7f1ebc197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec15f3700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc101fc0 0x7f1ebc197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec15f3700 1 -- 172.21.15.33:0/3706336970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00095f0 con 0x7f1ebc102980 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1ec15f3700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc102980 0x7f1ebc19e250 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400ef30 tx=0x7f1eb400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.709+0000 7f1eb9ffb700 1 -- 172.21.15.33:0/3706336970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4018070 con 0x7f1ebc102980 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.710+0000 7f1eb9ffb700 1 -- 172.21.15.33:0/3706336970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1eb400f040 con 0x7f1ebc102980 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.710+0000 7f1ec3056700 1 -- 172.21.15.33:0/3706336970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc19edd0 con 0x7f1ebc102980 2024-05-08T21:28:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.710+0000 7f1eb9ffb700 1 -- 172.21.15.33:0/3706336970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400cbc0 con 0x7f1ebc102980 2024-05-08T21:28:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.710+0000 7f1ec3056700 1 -- 172.21.15.33:0/3706336970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc19f200 con 0x7f1ebc102980 2024-05-08T21:28:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.711+0000 7f1ec3056700 1 -- 172.21.15.33:0/3706336970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc108330 con 0x7f1ebc102980 2024-05-08T21:28:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.712+0000 7f1eb9ffb700 1 -- 172.21.15.33:0/3706336970 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f1eb4007500 con 0x7f1ebc102980 2024-05-08T21:28:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.712+0000 7f1eb9ffb700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1ea40a1120 0x7f1ea40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.713+0000 7f1eb9ffb700 1 -- 172.21.15.33:0/3706336970 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f1eb40cc7a0 con 0x7f1ebc102980 2024-05-08T21:28:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.713+0000 7f1ec0df2700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1ea40a1120 0x7f1ea40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.713+0000 7f1ec0df2700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1ea40a1120 0x7f1ea40a35e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0fe080 tx=0x7f1eac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:51.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.718+0000 7f1eb9ffb700 1 -- 172.21.15.33:0/3706336970 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f1eb4095c00 con 0x7f1ebc102980 2024-05-08T21:28:51.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:51 smithi071 ceph-mon[47074]: pgmap v586: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-08T21:28:51.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:51 smithi082 ceph-mon[47512]: pgmap v586: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-08T21:28:51.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.935+0000 7f1ec3056700 1 -- 172.21.15.33:0/3706336970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ebc10c750 con 0x7f1ebc102980 2024-05-08T21:28:51.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.936+0000 7f1eb9ffb700 1 -- 172.21.15.33:0/3706336970 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1eb4099220 con 0x7f1ebc102980 2024-05-08T21:28:51.937 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:28:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.938+0000 7f1ec3056700 1 -- 172.21.15.33:0/3706336970 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1ea40a1120 msgr2=0x7f1ea40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.938+0000 7f1ec3056700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1ea40a1120 0x7f1ea40a35e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0fe080 tx=0x7f1eac009260 comp rx=0 tx=0).stop 2024-05-08T21:28:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.938+0000 7f1ec3056700 1 -- 172.21.15.33:0/3706336970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc102980 msgr2=0x7f1ebc19e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.938+0000 7f1ec3056700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc102980 0x7f1ebc19e250 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400ef30 tx=0x7f1eb400c3a0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.938+0000 7f1ec3056700 1 -- 172.21.15.33:0/3706336970 shutdown_connections 2024-05-08T21:28:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.938+0000 7f1ec3056700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ebc102980 0x7f1ebc19e250 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.938+0000 7f1ec3056700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ebc100e70 0x7f1ebc196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.938+0000 7f1ec3056700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f1ea40a1120 0x7f1ea40a35e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.938+0000 7f1ec3056700 1 --2- 172.21.15.33:0/3706336970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ebc101fc0 0x7f1ebc197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.938+0000 7f1ec3056700 1 -- 172.21.15.33:0/3706336970 >> 172.21.15.33:0/3706336970 conn(0x7f1ebc0fc400 msgr2=0x7f1ebc105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.938+0000 7f1ec3056700 1 -- 172.21.15.33:0/3706336970 shutdown_connections 2024-05-08T21:28:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:51.938+0000 7f1ec3056700 1 -- 172.21.15.33:0/3706336970 wait complete. 2024-05-08T21:28:51.954 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:28:52.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.093+0000 7f6b0017d700 1 -- 172.21.15.33:0/95639571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8101fc0 msgr2=0x7f6af8102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.093+0000 7f6b0017d700 1 --2- 172.21.15.33:0/95639571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8101fc0 0x7f6af8102440 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009990 tx=0x7f6ae8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.093+0000 7f6b0017d700 1 -- 172.21.15.33:0/95639571 shutdown_connections 2024-05-08T21:28:52.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.093+0000 7f6b0017d700 1 --2- 172.21.15.33:0/95639571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af8102980 0x7f6af81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.093+0000 7f6b0017d700 1 --2- 172.21.15.33:0/95639571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6af8100e70 0x7f6af8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.093+0000 7f6b0017d700 1 --2- 172.21.15.33:0/95639571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8101fc0 0x7f6af8102440 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.093+0000 7f6b0017d700 1 -- 172.21.15.33:0/95639571 >> 172.21.15.33:0/95639571 conn(0x7f6af80fc400 msgr2=0x7f6af80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.093+0000 7f6b0017d700 1 -- 172.21.15.33:0/95639571 shutdown_connections 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.093+0000 7f6b0017d700 1 -- 172.21.15.33:0/95639571 wait complete. 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.093+0000 7f6b0017d700 1 Processor -- start 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6b0017d700 1 -- start start 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6b0017d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6af8100e70 0x7f6af8074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6b0017d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8101fc0 0x7f6af8073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6b0017d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af8102980 0x7f6af80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6b0017d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af8074070 con 0x7f6af8101fc0 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6b0017d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80741e0 con 0x7f6af8100e70 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6b0017d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af81a0d70 con 0x7f6af8102980 2024-05-08T21:28:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6afd718700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8101fc0 0x7f6af8073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6afd718700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8101fc0 0x7f6af8073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42314/0 (socket says 172.21.15.33:42314) 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6afd718700 1 -- 172.21.15.33:0/127994014 learned_addr learned my addr 172.21.15.33:0/127994014 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6afe71a700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af8102980 0x7f6af80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6afdf19700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6af8100e70 0x7f6af8074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6afd718700 1 -- 172.21.15.33:0/127994014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af8102980 msgr2=0x7f6af80738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6afd718700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af8102980 0x7f6af80738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6afd718700 1 -- 172.21.15.33:0/127994014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6af8100e70 msgr2=0x7f6af8074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6afd718700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6af8100e70 0x7f6af8074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6afd718700 1 -- 172.21.15.33:0/127994014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae80095f0 con 0x7f6af8101fc0 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.094+0000 7f6afd718700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8101fc0 0x7f6af8073390 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8006010 tx=0x7f6ae8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.095+0000 7f6aeeffd700 1 -- 172.21.15.33:0/127994014 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae801d070 con 0x7f6af8101fc0 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.095+0000 7f6aeeffd700 1 -- 172.21.15.33:0/127994014 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ae800bd50 con 0x7f6af8101fc0 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.095+0000 7f6b0017d700 1 -- 172.21.15.33:0/127994014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af81a0fa0 con 0x7f6af8101fc0 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.095+0000 7f6aeeffd700 1 -- 172.21.15.33:0/127994014 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae800fd50 con 0x7f6af8101fc0 2024-05-08T21:28:52.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.095+0000 7f6b0017d700 1 -- 172.21.15.33:0/127994014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af81a1360 con 0x7f6af8101fc0 2024-05-08T21:28:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.096+0000 7f6b0017d700 1 -- 172.21.15.33:0/127994014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af81082d0 con 0x7f6af8101fc0 2024-05-08T21:28:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.097+0000 7f6aeeffd700 1 -- 172.21.15.33:0/127994014 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6ae800b8c0 con 0x7f6af8101fc0 2024-05-08T21:28:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.098+0000 7f6aeeffd700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6ae40a11a0 0x7f6ae40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.098+0000 7f6aeeffd700 1 -- 172.21.15.33:0/127994014 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f6ae80ce270 con 0x7f6af8101fc0 2024-05-08T21:28:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.098+0000 7f6afdf19700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6ae40a11a0 0x7f6ae40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.098+0000 7f6afdf19700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6ae40a11a0 0x7f6ae40a3660 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009c60 tx=0x7f6af4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:52.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.102+0000 7f6aeeffd700 1 -- 172.21.15.33:0/127994014 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6ae80976d0 con 0x7f6af8101fc0 2024-05-08T21:28:52.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.319+0000 7f6b0017d700 1 -- 172.21.15.33:0/127994014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6af810c680 con 0x7f6af8101fc0 2024-05-08T21:28:52.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.320+0000 7f6aeeffd700 1 -- 172.21.15.33:0/127994014 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6ae809acf0 con 0x7f6af8101fc0 2024-05-08T21:28:52.323 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:28:52.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.322+0000 7f6b0017d700 1 -- 172.21.15.33:0/127994014 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6ae40a11a0 msgr2=0x7f6ae40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.322+0000 7f6b0017d700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6ae40a11a0 0x7f6ae40a3660 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6af4009c60 tx=0x7f6af4009260 comp rx=0 tx=0).stop 2024-05-08T21:28:52.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.322+0000 7f6b0017d700 1 -- 172.21.15.33:0/127994014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8101fc0 msgr2=0x7f6af8073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.322+0000 7f6b0017d700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8101fc0 0x7f6af8073390 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8006010 tx=0x7f6ae8004730 comp rx=0 tx=0).stop 2024-05-08T21:28:52.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.322+0000 7f6b0017d700 1 -- 172.21.15.33:0/127994014 shutdown_connections 2024-05-08T21:28:52.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.322+0000 7f6b0017d700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af8102980 0x7f6af80738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.322+0000 7f6b0017d700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6ae40a11a0 0x7f6ae40a3660 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.322+0000 7f6b0017d700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6af8100e70 0x7f6af8074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.322+0000 7f6b0017d700 1 --2- 172.21.15.33:0/127994014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8101fc0 0x7f6af8073390 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.322+0000 7f6b0017d700 1 -- 172.21.15.33:0/127994014 >> 172.21.15.33:0/127994014 conn(0x7f6af80fc400 msgr2=0x7f6af8105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:52.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.322+0000 7f6b0017d700 1 -- 172.21.15.33:0/127994014 shutdown_connections 2024-05-08T21:28:52.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.322+0000 7f6b0017d700 1 -- 172.21.15.33:0/127994014 wait complete. 2024-05-08T21:28:52.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:28:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.477+0000 7fa231c2f700 1 -- 172.21.15.33:0/3322898225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c100f80 msgr2=0x7fa22c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.477+0000 7fa231c2f700 1 --2- 172.21.15.33:0/3322898225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c100f80 0x7fa22c1013e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa2140099c0 tx=0x7fa214009cd0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.477+0000 7fa231c2f700 1 -- 172.21.15.33:0/3322898225 shutdown_connections 2024-05-08T21:28:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.477+0000 7fa231c2f700 1 --2- 172.21.15.33:0/3322898225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c101920 0x7fa22c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.477+0000 7fa231c2f700 1 --2- 172.21.15.33:0/3322898225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c0ffd90 0x7fa22c1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.477+0000 7fa231c2f700 1 --2- 172.21.15.33:0/3322898225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c100f80 0x7fa22c1013e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.477+0000 7fa231c2f700 1 -- 172.21.15.33:0/3322898225 >> 172.21.15.33:0/3322898225 conn(0x7fa22c0fb300 msgr2=0x7fa22c0fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.478+0000 7fa231c2f700 1 -- 172.21.15.33:0/3322898225 shutdown_connections 2024-05-08T21:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.478+0000 7fa231c2f700 1 -- 172.21.15.33:0/3322898225 wait complete. 2024-05-08T21:28:52.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.478+0000 7fa231c2f700 1 Processor -- start 2024-05-08T21:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.478+0000 7fa231c2f700 1 -- start start 2024-05-08T21:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa231c2f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c0ffd90 0x7fa22c196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa231c2f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c100f80 0x7fa22c197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa231c2f700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c101920 0x7fa22c19e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa231c2f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c19e8a0 con 0x7fa22c100f80 2024-05-08T21:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa231c2f700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c19e9e0 con 0x7fa22c101920 2024-05-08T21:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa231c2f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c19eba0 con 0x7fa22c0ffd90 2024-05-08T21:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c100f80 0x7fa22c197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c100f80 0x7fa22c197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42336/0 (socket says 172.21.15.33:42336) 2024-05-08T21:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22affd700 1 -- 172.21.15.33:0/1788303823 learned_addr learned my addr 172.21.15.33:0/1788303823 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22b7fe700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c0ffd90 0x7fa22c196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22bfff700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c101920 0x7fa22c19e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22b7fe700 1 -- 172.21.15.33:0/1788303823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c101920 msgr2=0x7fa22c19e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22b7fe700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c101920 0x7fa22c19e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22b7fe700 1 -- 172.21.15.33:0/1788303823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c100f80 msgr2=0x7fa22c197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22b7fe700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c100f80 0x7fa22c197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22b7fe700 1 -- 172.21.15.33:0/1788303823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2140095f0 con 0x7fa22c0ffd90 2024-05-08T21:28:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22affd700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c100f80 0x7fa22c197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:28:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22bfff700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c101920 0x7fa22c19e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.479+0000 7fa22b7fe700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c0ffd90 0x7fa22c196e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00e9f0 tx=0x7fa21c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.480+0000 7fa228ff9700 1 -- 172.21.15.33:0/1788303823 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c00eee0 con 0x7fa22c0ffd90 2024-05-08T21:28:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.480+0000 7fa228ff9700 1 -- 172.21.15.33:0/1788303823 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa21c0044b0 con 0x7fa22c0ffd90 2024-05-08T21:28:52.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.480+0000 7fa231c2f700 1 -- 172.21.15.33:0/1788303823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22c19ee80 con 0x7fa22c0ffd90 2024-05-08T21:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.480+0000 7fa228ff9700 1 -- 172.21.15.33:0/1788303823 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c003bb0 con 0x7fa22c0ffd90 2024-05-08T21:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.480+0000 7fa231c2f700 1 -- 172.21.15.33:0/1788303823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c19f2a0 con 0x7fa22c0ffd90 2024-05-08T21:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.481+0000 7fa231c2f700 1 -- 172.21.15.33:0/1788303823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22c0674f0 con 0x7fa22c0ffd90 2024-05-08T21:28:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.482+0000 7fa228ff9700 1 -- 172.21.15.33:0/1788303823 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa21c004870 con 0x7fa22c0ffd90 2024-05-08T21:28:52.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.482+0000 7fa228ff9700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa2180a1120 0x7fa2180a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:52.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.483+0000 7fa22affd700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa2180a1120 0x7fa2180a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:52.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.483+0000 7fa228ff9700 1 -- 172.21.15.33:0/1788303823 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa21c014070 con 0x7fa22c0ffd90 2024-05-08T21:28:52.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.483+0000 7fa22affd700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa2180a1120 0x7fa2180a35e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa214006010 tx=0x7fa21401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:52.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.486+0000 7fa228ff9700 1 -- 172.21.15.33:0/1788303823 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa21c096850 con 0x7fa22c0ffd90 2024-05-08T21:28:52.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:52 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2606439299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:28:52.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:52 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3706336970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:28:52.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:52 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/127994014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:28:52.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.705+0000 7fa231c2f700 1 -- 172.21.15.33:0/1788303823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa22c0674f0 con 0x7fa22c0ffd90 2024-05-08T21:28:52.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.706+0000 7fa228ff9700 1 -- 172.21.15.33:0/1788303823 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa22c0674f0 con 0x7fa22c0ffd90 2024-05-08T21:28:52.708 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T21:28:52.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.708+0000 7fa231c2f700 1 -- 172.21.15.33:0/1788303823 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa2180a1120 msgr2=0x7fa2180a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.708+0000 7fa231c2f700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa2180a1120 0x7fa2180a35e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa214006010 tx=0x7fa21401a040 comp rx=0 tx=0).stop 2024-05-08T21:28:52.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.708+0000 7fa231c2f700 1 -- 172.21.15.33:0/1788303823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c0ffd90 msgr2=0x7fa22c196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.708+0000 7fa231c2f700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c0ffd90 0x7fa22c196e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00e9f0 tx=0x7fa21c00ed00 comp rx=0 tx=0).stop 2024-05-08T21:28:52.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.708+0000 7fa231c2f700 1 -- 172.21.15.33:0/1788303823 shutdown_connections 2024-05-08T21:28:52.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.708+0000 7fa231c2f700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa2180a1120 0x7fa2180a35e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.708+0000 7fa231c2f700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa22c101920 0x7fa22c19e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.708+0000 7fa231c2f700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa22c0ffd90 0x7fa22c196e10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.708+0000 7fa231c2f700 1 --2- 172.21.15.33:0/1788303823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa22c100f80 0x7fa22c197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.708+0000 7fa231c2f700 1 -- 172.21.15.33:0/1788303823 >> 172.21.15.33:0/1788303823 conn(0x7fa22c0fb300 msgr2=0x7fa22c0690a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:52.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.709+0000 7fa231c2f700 1 -- 172.21.15.33:0/1788303823 shutdown_connections 2024-05-08T21:28:52.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.709+0000 7fa231c2f700 1 -- 172.21.15.33:0/1788303823 wait complete. 2024-05-08T21:28:52.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T21:28:52.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:52 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2606439299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:28:52.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:52 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3706336970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:28:52.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:52 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/127994014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:28:52.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:52 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2606439299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:28:52.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:52 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3706336970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:28:52.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:52 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/127994014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:28:52.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.864+0000 7f6b21bb9700 1 -- 172.21.15.33:0/2522029992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b1c106b40 msgr2=0x7f6b1c108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.864+0000 7f6b21bb9700 1 --2- 172.21.15.33:0/2522029992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b1c106b40 0x7f6b1c108fb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400b0a0 tx=0x7f6b1400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.864+0000 7f6b21bb9700 1 -- 172.21.15.33:0/2522029992 shutdown_connections 2024-05-08T21:28:52.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.864+0000 7f6b21bb9700 1 --2- 172.21.15.33:0/2522029992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b1c106b40 0x7f6b1c108fb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.864+0000 7f6b21bb9700 1 --2- 172.21.15.33:0/2522029992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b1c1018e0 0x7f6b1c103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.864+0000 7f6b21bb9700 1 --2- 172.21.15.33:0/2522029992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1c104210 0x7f6b1c106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.864+0000 7f6b21bb9700 1 -- 172.21.15.33:0/2522029992 >> 172.21.15.33:0/2522029992 conn(0x7f6b1c0fb290 msgr2=0x7f6b1c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:52.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.864+0000 7f6b21bb9700 1 -- 172.21.15.33:0/2522029992 shutdown_connections 2024-05-08T21:28:52.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.864+0000 7f6b21bb9700 1 -- 172.21.15.33:0/2522029992 wait complete. 2024-05-08T21:28:52.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.865+0000 7f6b21bb9700 1 Processor -- start 2024-05-08T21:28:52.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.865+0000 7f6b21bb9700 1 -- start start 2024-05-08T21:28:52.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.865+0000 7f6b21bb9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1c1018e0 0x7f6b1c101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:52.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.865+0000 7f6b21bb9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b1c104210 0x7f6b1c0ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:52.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.865+0000 7f6b21bb9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b1c106b40 0x7f6b1c0ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:52.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.865+0000 7f6b21bb9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c100570 con 0x7f6b1c1018e0 2024-05-08T21:28:52.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.865+0000 7f6b21bb9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1006e0 con 0x7f6b1c106b40 2024-05-08T21:28:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.865+0000 7f6b21bb9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c1a0d90 con 0x7f6b1c104210 2024-05-08T21:28:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b1b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1c1018e0 0x7f6b1c101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b1affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b1c104210 0x7f6b1c0ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b1b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1c1018e0 0x7f6b1c101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42354/0 (socket says 172.21.15.33:42354) 2024-05-08T21:28:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b1affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b1c104210 0x7f6b1c0ff890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.33:54964/0 (socket says 172.21.15.33:54964) 2024-05-08T21:28:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b1b7fe700 1 -- 172.21.15.33:0/3807050039 learned_addr learned my addr 172.21.15.33:0/3807050039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b1bfff700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b1c106b40 0x7f6b1c0ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b1affd700 1 -- 172.21.15.33:0/3807050039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b1c106b40 msgr2=0x7f6b1c0ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b1affd700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b1c106b40 0x7f6b1c0ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b1affd700 1 -- 172.21.15.33:0/3807050039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1c1018e0 msgr2=0x7f6b1c101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b1affd700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1c1018e0 0x7f6b1c101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:52.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b1affd700 1 -- 172.21.15.33:0/3807050039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b04009520 con 0x7f6b1c104210 2024-05-08T21:28:52.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b1affd700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b1c104210 0x7f6b1c0ff890 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e9f0 tx=0x7f6b0c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:52.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b18ff9700 1 -- 172.21.15.33:0/3807050039 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00cb70 con 0x7f6b1c104210 2024-05-08T21:28:52.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b21bb9700 1 -- 172.21.15.33:0/3807050039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c009520 con 0x7f6b1c104210 2024-05-08T21:28:52.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b18ff9700 1 -- 172.21.15.33:0/3807050039 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b0c00ee90 con 0x7f6b1c104210 2024-05-08T21:28:52.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b18ff9700 1 -- 172.21.15.33:0/3807050039 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c003bc0 con 0x7f6b1c104210 2024-05-08T21:28:52.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.866+0000 7f6b21bb9700 1 -- 172.21.15.33:0/3807050039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b14009d00 con 0x7f6b1c104210 2024-05-08T21:28:52.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.867+0000 7f6b21bb9700 1 -- 172.21.15.33:0/3807050039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c0fce10 con 0x7f6b1c104210 2024-05-08T21:28:52.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.869+0000 7f6b18ff9700 1 -- 172.21.15.33:0/3807050039 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f6b0c003e20 con 0x7f6b1c104210 2024-05-08T21:28:52.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.870+0000 7f6b18ff9700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b080a1120 0x7f6b080a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:52.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.870+0000 7f6b18ff9700 1 -- 172.21.15.33:0/3807050039 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f6b0c014070 con 0x7f6b1c104210 2024-05-08T21:28:52.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.870+0000 7f6b1b7fe700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b080a1120 0x7f6b080a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:52.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.870+0000 7f6b1b7fe700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b080a1120 0x7f6b080a35e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009ef0 tx=0x7f6b04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:52.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:52.873+0000 7f6b18ff9700 1 -- 172.21.15.33:0/3807050039 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f6b0c0964a0 con 0x7f6b1c104210 2024-05-08T21:28:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.092+0000 7f6b21bb9700 1 -- 172.21.15.33:0/3807050039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6b1c10c4a0 con 0x7f6b1c104210 2024-05-08T21:28:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.092+0000 7f6b18ff9700 1 -- 172.21.15.33:0/3807050039 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6b0c099ac0 con 0x7f6b1c104210 2024-05-08T21:28:53.094 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:28:53.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.095+0000 7f6b21bb9700 1 -- 172.21.15.33:0/3807050039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b080a1120 msgr2=0x7f6b080a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:53.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.095+0000 7f6b21bb9700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b080a1120 0x7f6b080a35e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009ef0 tx=0x7f6b04009260 comp rx=0 tx=0).stop 2024-05-08T21:28:53.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.095+0000 7f6b21bb9700 1 -- 172.21.15.33:0/3807050039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b1c104210 msgr2=0x7f6b1c0ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:53.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.095+0000 7f6b21bb9700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b1c104210 0x7f6b1c0ff890 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00e9f0 tx=0x7f6b0c00ed00 comp rx=0 tx=0).stop 2024-05-08T21:28:53.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.095+0000 7f6b21bb9700 1 -- 172.21.15.33:0/3807050039 shutdown_connections 2024-05-08T21:28:53.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.095+0000 7f6b21bb9700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f6b080a1120 0x7f6b080a35e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.095+0000 7f6b21bb9700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b1c106b40 0x7f6b1c0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.095+0000 7f6b21bb9700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b1c1018e0 0x7f6b1c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.095+0000 7f6b21bb9700 1 --2- 172.21.15.33:0/3807050039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b1c104210 0x7f6b1c0ff890 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.095+0000 7f6b21bb9700 1 -- 172.21.15.33:0/3807050039 >> 172.21.15.33:0/3807050039 conn(0x7f6b1c0fb290 msgr2=0x7f6b1c102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:53.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.095+0000 7f6b21bb9700 1 -- 172.21.15.33:0/3807050039 shutdown_connections 2024-05-08T21:28:53.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.095+0000 7f6b21bb9700 1 -- 172.21.15.33:0/3807050039 wait complete. 2024-05-08T21:28:53.112 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T21:28:53.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.248+0000 7f2a9d890700 1 -- 172.21.15.33:0/1924979781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a98102990 msgr2=0x7f2a981092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:53.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.248+0000 7f2a9d890700 1 --2- 172.21.15.33:0/1924979781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a98102990 0x7f2a981092b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000b0a0 tx=0x7f2a9000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.248+0000 7f2a9d890700 1 -- 172.21.15.33:0/1924979781 shutdown_connections 2024-05-08T21:28:53.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.248+0000 7f2a9d890700 1 --2- 172.21.15.33:0/1924979781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a98102990 0x7f2a981092b0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.248+0000 7f2a9d890700 1 --2- 172.21.15.33:0/1924979781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a98100e80 0x7f2a981012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.248+0000 7f2a9d890700 1 --2- 172.21.15.33:0/1924979781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a98101fd0 0x7f2a98102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.248+0000 7f2a9d890700 1 -- 172.21.15.33:0/1924979781 >> 172.21.15.33:0/1924979781 conn(0x7f2a980fc3f0 msgr2=0x7f2a980fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.248+0000 7f2a9d890700 1 -- 172.21.15.33:0/1924979781 shutdown_connections 2024-05-08T21:28:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.248+0000 7f2a9d890700 1 -- 172.21.15.33:0/1924979781 wait complete. 2024-05-08T21:28:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.249+0000 7f2a9d890700 1 Processor -- start 2024-05-08T21:28:53.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.249+0000 7f2a9d890700 1 -- start start 2024-05-08T21:28:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.249+0000 7f2a9d890700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a98100e80 0x7f2a98074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.249+0000 7f2a9d890700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a98101fd0 0x7f2a98073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:53.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.249+0000 7f2a9d890700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a98102990 0x7f2a980738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.249+0000 7f2a9d890700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a98074070 con 0x7f2a98102990 2024-05-08T21:28:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.249+0000 7f2a9d890700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980741e0 con 0x7f2a98100e80 2024-05-08T21:28:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.249+0000 7f2a9d890700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a981a0e20 con 0x7f2a98101fd0 2024-05-08T21:28:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.249+0000 7f2a977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a98102990 0x7f2a980738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.249+0000 7f2a977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a98102990 0x7f2a980738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42370/0 (socket says 172.21.15.33:42370) 2024-05-08T21:28:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.249+0000 7f2a977fe700 1 -- 172.21.15.33:0/2427126519 learned_addr learned my addr 172.21.15.33:0/2427126519 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a8e7fc700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a98101fd0 0x7f2a98073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a96ffd700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a98100e80 0x7f2a98074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a977fe700 1 -- 172.21.15.33:0/2427126519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a98101fd0 msgr2=0x7f2a98073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:53.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a977fe700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a98101fd0 0x7f2a98073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a977fe700 1 -- 172.21.15.33:0/2427126519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a98100e80 msgr2=0x7f2a98074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a977fe700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a98100e80 0x7f2a98074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a977fe700 1 -- 172.21.15.33:0/2427126519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a90009d00 con 0x7f2a98102990 2024-05-08T21:28:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a977fe700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a98102990 0x7f2a980738d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009840 tx=0x7f2a9000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a9c88e700 1 -- 172.21.15.33:0/2427126519 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9000e040 con 0x7f2a98102990 2024-05-08T21:28:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a9c88e700 1 -- 172.21.15.33:0/2427126519 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a90004c50 con 0x7f2a98102990 2024-05-08T21:28:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a9d890700 1 -- 172.21.15.33:0/2427126519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a981a1050 con 0x7f2a98102990 2024-05-08T21:28:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a9c88e700 1 -- 172.21.15.33:0/2427126519 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a900052a0 con 0x7f2a98102990 2024-05-08T21:28:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.250+0000 7f2a9d890700 1 -- 172.21.15.33:0/2427126519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a981a1470 con 0x7f2a98102990 2024-05-08T21:28:53.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.251+0000 7f2a9d890700 1 -- 172.21.15.33:0/2427126519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a981082c0 con 0x7f2a98102990 2024-05-08T21:28:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.253+0000 7f2a9c88e700 1 -- 172.21.15.33:0/2427126519 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2a9001d470 con 0x7f2a98102990 2024-05-08T21:28:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.253+0000 7f2a9c88e700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2a840a11a0 0x7f2a840a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.253+0000 7f2a9c88e700 1 -- 172.21.15.33:0/2427126519 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f2a900ce870 con 0x7f2a98102990 2024-05-08T21:28:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.254+0000 7f2a96ffd700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2a840a11a0 0x7f2a840a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:53.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.254+0000 7f2a96ffd700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2a840a11a0 0x7f2a840a3660 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009c60 tx=0x7f2a88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:53.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.257+0000 7f2a9c88e700 1 -- 172.21.15.33:0/2427126519 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2a90097cd0 con 0x7f2a98102990 2024-05-08T21:28:53.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.493+0000 7f2a9d890700 1 -- 172.21.15.33:0/2427126519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2a9810c640 con 0x7f2a98102990 2024-05-08T21:28:53.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.493+0000 7f2a9c88e700 1 -- 172.21.15.33:0/2427126519 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+592 (secure 0 0 0) 0x7f2a9009b2f0 con 0x7f2a98102990 2024-05-08T21:28:53.495 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3156563197 2024-05-09T21:11:56.047045+0000 2024-05-08T21:28:53.495 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3413344235 2024-05-09T21:11:56.047045+0000 2024-05-08T21:28:53.495 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3413344235 2024-05-09T21:11:56.047045+0000 2024-05-08T21:28:53.495 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1762147139 2024-05-09T21:11:56.047045+0000 2024-05-08T21:28:53.495 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2047278949 2024-05-09T21:11:32.156884+0000 2024-05-08T21:28:53.495 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1002276836 2024-05-09T21:11:32.156884+0000 2024-05-08T21:28:53.495 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3475366293 2024-05-09T21:11:32.156884+0000 2024-05-08T21:28:53.495 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3475366293 2024-05-09T21:11:32.156884+0000 2024-05-08T21:28:53.496 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1708721255 2024-05-09T21:11:32.156884+0000 2024-05-08T21:28:53.496 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3864964868 2024-05-09T21:11:56.047045+0000 2024-05-08T21:28:53.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.496+0000 7f2a9d890700 1 -- 172.21.15.33:0/2427126519 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2a840a11a0 msgr2=0x7f2a840a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:53.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.496+0000 7f2a9d890700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2a840a11a0 0x7f2a840a3660 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009c60 tx=0x7f2a88009260 comp rx=0 tx=0).stop 2024-05-08T21:28:53.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.496+0000 7f2a9d890700 1 -- 172.21.15.33:0/2427126519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a98102990 msgr2=0x7f2a980738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:53.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.496+0000 7f2a9d890700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a98102990 0x7f2a980738d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009840 tx=0x7f2a9000ba50 comp rx=0 tx=0).stop 2024-05-08T21:28:53.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.496+0000 7f2a9d890700 1 -- 172.21.15.33:0/2427126519 shutdown_connections 2024-05-08T21:28:53.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.496+0000 7f2a9d890700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a98102990 0x7f2a980738d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.496+0000 7f2a9d890700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2a840a11a0 0x7f2a840a3660 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.496+0000 7f2a9d890700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a98100e80 0x7f2a98074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.496+0000 7f2a9d890700 1 --2- 172.21.15.33:0/2427126519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a98101fd0 0x7f2a98073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.496+0000 7f2a9d890700 1 -- 172.21.15.33:0/2427126519 >> 172.21.15.33:0/2427126519 conn(0x7f2a980fc3f0 msgr2=0x7f2a98105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:53.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.496+0000 7f2a9d890700 1 -- 172.21.15.33:0/2427126519 shutdown_connections 2024-05-08T21:28:53.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.496+0000 7f2a9d890700 1 -- 172.21.15.33:0/2427126519 wait complete. 2024-05-08T21:28:53.499 INFO:teuthology.orchestra.run.smithi033.stderr:listed 10 entries 2024-05-08T21:28:53.513 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-05-08T21:28:53.513 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T21:28:53.534 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:28:53.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:53 smithi033 ceph-mon[42528]: pgmap v587: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-08T21:28:53.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:53 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1788303823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:28:53.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:53 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3807050039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:28:53.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.716+0000 7f8199423700 1 -- 172.21.15.33:0/3634237865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8194106b60 msgr2=0x7f8194108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:53.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.716+0000 7f8199423700 1 --2- 172.21.15.33:0/3634237865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8194106b60 0x7f8194108fd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f818c00b0a0 tx=0x7f818c00b3b0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.716+0000 7f8199423700 1 -- 172.21.15.33:0/3634237865 shutdown_connections 2024-05-08T21:28:53.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.716+0000 7f8199423700 1 --2- 172.21.15.33:0/3634237865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8194106b60 0x7f8194108fd0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.716+0000 7f8199423700 1 --2- 172.21.15.33:0/3634237865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8194101900 0x7f8194103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.716+0000 7f8199423700 1 --2- 172.21.15.33:0/3634237865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8194104230 0x7f8194106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.716+0000 7f8199423700 1 -- 172.21.15.33:0/3634237865 >> 172.21.15.33:0/3634237865 conn(0x7f81940fb310 msgr2=0x7f81940fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:53.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.716+0000 7f8199423700 1 -- 172.21.15.33:0/3634237865 shutdown_connections 2024-05-08T21:28:53.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.716+0000 7f8199423700 1 -- 172.21.15.33:0/3634237865 wait complete. 2024-05-08T21:28:53.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f8199423700 1 Processor -- start 2024-05-08T21:28:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f8199423700 1 -- start start 2024-05-08T21:28:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f8199423700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8194101900 0x7f8194196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f8199423700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8194104230 0x7f8194197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f8199423700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8194106b60 0x7f819419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f8199423700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819419e940 con 0x7f8194101900 2024-05-08T21:28:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f8199423700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819419ea80 con 0x7f8194106b60 2024-05-08T21:28:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f8199423700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819419ec10 con 0x7f8194104230 2024-05-08T21:28:53.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f8192ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8194101900 0x7f8194196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:53.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f8192ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8194101900 0x7f8194196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42390/0 (socket says 172.21.15.33:42390) 2024-05-08T21:28:53.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f8192ffd700 1 -- 172.21.15.33:0/822663916 learned_addr learned my addr 172.21.15.33:0/822663916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:53.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f81927fc700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8194104230 0x7f8194197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:53.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.717+0000 7f81937fe700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8194106b60 0x7f819419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.718+0000 7f8192ffd700 1 -- 172.21.15.33:0/822663916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8194104230 msgr2=0x7f8194197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.718+0000 7f8192ffd700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8194104230 0x7f8194197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.718+0000 7f8192ffd700 1 -- 172.21.15.33:0/822663916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8194106b60 msgr2=0x7f819419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.718+0000 7f8192ffd700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8194106b60 0x7f819419e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.718+0000 7f8192ffd700 1 -- 172.21.15.33:0/822663916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c009d00 con 0x7f8194101900 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.718+0000 7f81927fc700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8194104230 0x7f8194197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.718+0000 7f8192ffd700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8194101900 0x7f8194196e10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f818400e9a0 tx=0x7f818400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.718+0000 7f817bfff700 1 -- 172.21.15.33:0/822663916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184009790 con 0x7f8194101900 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.718+0000 7f817bfff700 1 -- 172.21.15.33:0/822663916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f818400cbd0 con 0x7f8194101900 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.718+0000 7f8199423700 1 -- 172.21.15.33:0/822663916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819419eef0 con 0x7f8194101900 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.718+0000 7f817bfff700 1 -- 172.21.15.33:0/822663916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184018680 con 0x7f8194101900 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.718+0000 7f8199423700 1 -- 172.21.15.33:0/822663916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819419f320 con 0x7f8194101900 2024-05-08T21:28:53.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.719+0000 7f8199423700 1 -- 172.21.15.33:0/822663916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81940fce90 con 0x7f8194101900 2024-05-08T21:28:53.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.720+0000 7f817bfff700 1 -- 172.21.15.33:0/822663916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f818400cd40 con 0x7f8194101900 2024-05-08T21:28:53.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.721+0000 7f817bfff700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f817c0a11a0 0x7f817c0a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:53.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.721+0000 7f81927fc700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f817c0a11a0 0x7f817c0a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:53.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.721+0000 7f817bfff700 1 -- 172.21.15.33:0/822663916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f8184014070 con 0x7f8194101900 2024-05-08T21:28:53.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.721+0000 7f81927fc700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f817c0a11a0 0x7f817c0a3660 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8188009cb0 tx=0x7f8188009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:53.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.725+0000 7f817bfff700 1 -- 172.21.15.33:0/822663916 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f8184096500 con 0x7f8194101900 2024-05-08T21:28:53.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:53 smithi071 ceph-mon[47074]: pgmap v587: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-08T21:28:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:53 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1788303823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:28:53.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:53 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3807050039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:28:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:53 smithi082 ceph-mon[47512]: pgmap v587: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-08T21:28:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:53 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1788303823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:28:53.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:53 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3807050039' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:28:53.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.945+0000 7f8199423700 1 -- 172.21.15.33:0/822663916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8194068160 con 0x7f8194101900 2024-05-08T21:28:53.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.947+0000 7f817bfff700 1 -- 172.21.15.33:0/822663916 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24561 (secure 0 0 0) 0x7f8184026070 con 0x7f8194101900 2024-05-08T21:28:53.949 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:28:53.949 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":89,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:18:58.419274+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:28:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.949+0000 7f8199423700 1 -- 172.21.15.33:0/822663916 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f817c0a11a0 msgr2=0x7f817c0a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.949+0000 7f8199423700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f817c0a11a0 0x7f817c0a3660 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8188009cb0 tx=0x7f8188009260 comp rx=0 tx=0).stop 2024-05-08T21:28:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.949+0000 7f8199423700 1 -- 172.21.15.33:0/822663916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8194101900 msgr2=0x7f8194196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.949+0000 7f8199423700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8194101900 0x7f8194196e10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f818400e9a0 tx=0x7f818400ed60 comp rx=0 tx=0).stop 2024-05-08T21:28:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.950+0000 7f8199423700 1 -- 172.21.15.33:0/822663916 shutdown_connections 2024-05-08T21:28:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.950+0000 7f8199423700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f817c0a11a0 0x7f817c0a3660 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.950+0000 7f8199423700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8194106b60 0x7f819419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.950+0000 7f8199423700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8194101900 0x7f8194196e10 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.950+0000 7f8199423700 1 --2- 172.21.15.33:0/822663916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8194104230 0x7f8194197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.950+0000 7f8199423700 1 -- 172.21.15.33:0/822663916 >> 172.21.15.33:0/822663916 conn(0x7f81940fb310 msgr2=0x7f8194102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.950+0000 7f8199423700 1 -- 172.21.15.33:0/822663916 shutdown_connections 2024-05-08T21:28:53.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:53.950+0000 7f8199423700 1 -- 172.21.15.33:0/822663916 wait complete. 2024-05-08T21:28:53.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T21:13:48.583196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T21:17:01.282803+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T21:17:02.555552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T21:17:05.686277+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T21:28:53.966 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T21:28:54.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.105+0000 7fcdf3783700 1 -- 172.21.15.33:0/1656824006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdec1009f0 msgr2=0x7fcdec100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:54.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.105+0000 7fcdf3783700 1 --2- 172.21.15.33:0/1656824006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdec1009f0 0x7fcdec100e70 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcde400b0a0 tx=0x7fcde400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.105+0000 7fcdf3783700 1 -- 172.21.15.33:0/1656824006 shutdown_connections 2024-05-08T21:28:54.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.105+0000 7fcdf3783700 1 --2- 172.21.15.33:0/1656824006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdec1009f0 0x7fcdec100e70 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.105+0000 7fcdf3783700 1 --2- 172.21.15.33:0/1656824006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec0ff6d0 0x7fcdec0ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.105+0000 7fcdf3783700 1 --2- 172.21.15.33:0/1656824006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdec100030 0x7fcdec1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.105+0000 7fcdf3783700 1 -- 172.21.15.33:0/1656824006 >> 172.21.15.33:0/1656824006 conn(0x7fcdec0fb290 msgr2=0x7fcdec0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.105+0000 7fcdf3783700 1 -- 172.21.15.33:0/1656824006 shutdown_connections 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.105+0000 7fcdf3783700 1 -- 172.21.15.33:0/1656824006 wait complete. 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.105+0000 7fcdf3783700 1 Processor -- start 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf3783700 1 -- start start 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf3783700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec0ff6d0 0x7fcdec199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf3783700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdec100030 0x7fcdec199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf3783700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdec1009f0 0x7fcdec1a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf3783700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec1a0b10 con 0x7fcdec100030 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf3783700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec1a0c50 con 0x7fcdec0ff6d0 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf3783700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec1a0de0 con 0x7fcdec1009f0 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf0d1e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdec100030 0x7fcdec199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf0d1e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdec100030 0x7fcdec199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42416/0 (socket says 172.21.15.33:42416) 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf0d1e700 1 -- 172.21.15.33:0/1255973884 learned_addr learned my addr 172.21.15.33:0/1255973884 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf1d20700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdec1009f0 0x7fcdec1a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf151f700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec0ff6d0 0x7fcdec199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf0d1e700 1 -- 172.21.15.33:0/1255973884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdec1009f0 msgr2=0x7fcdec1a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf0d1e700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdec1009f0 0x7fcdec1a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf0d1e700 1 -- 172.21.15.33:0/1255973884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec0ff6d0 msgr2=0x7fcdec199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf0d1e700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec0ff6d0 0x7fcdec199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf0d1e700 1 -- 172.21.15.33:0/1255973884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde4009d00 con 0x7fcdec100030 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf151f700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec0ff6d0 0x7fcdec199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.106+0000 7fcdf0d1e700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdec100030 0x7fcdec199570 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00e9a0 tx=0x7fcddc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.107+0000 7fcde27fc700 1 -- 172.21.15.33:0/1255973884 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc009790 con 0x7fcdec100030 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.107+0000 7fcde27fc700 1 -- 172.21.15.33:0/1255973884 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcddc00cbd0 con 0x7fcdec100030 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.107+0000 7fcdf3783700 1 -- 172.21.15.33:0/1255973884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdec1a10c0 con 0x7fcdec100030 2024-05-08T21:28:54.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.107+0000 7fcde27fc700 1 -- 172.21.15.33:0/1255973884 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc018680 con 0x7fcdec100030 2024-05-08T21:28:54.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.107+0000 7fcdf3783700 1 -- 172.21.15.33:0/1255973884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec1a15d0 con 0x7fcdec100030 2024-05-08T21:28:54.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.108+0000 7fcdf3783700 1 -- 172.21.15.33:0/1255973884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdec10a4b0 con 0x7fcdec100030 2024-05-08T21:28:54.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.109+0000 7fcde27fc700 1 -- 172.21.15.33:0/1255973884 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fcddc00cd40 con 0x7fcdec100030 2024-05-08T21:28:54.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.110+0000 7fcde27fc700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcdd80a1120 0x7fcdd80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:54.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.110+0000 7fcdf151f700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcdd80a1120 0x7fcdd80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:54.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.110+0000 7fcde27fc700 1 -- 172.21.15.33:0/1255973884 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fcddc014070 con 0x7fcdec100030 2024-05-08T21:28:54.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.110+0000 7fcdf151f700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcdd80a1120 0x7fcdd80a35e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009cb0 tx=0x7fcde8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:54.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.115+0000 7fcde27fc700 1 -- 172.21.15.33:0/1255973884 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fcddc052d60 con 0x7fcdec100030 2024-05-08T21:28:54.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.333+0000 7fcdf3783700 1 -- 172.21.15.33:0/1255973884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcdec10e8d0 con 0x7fcdec100030 2024-05-08T21:28:54.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.334+0000 7fcde27fc700 1 -- 172.21.15.33:0/1255973884 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcddc0962e0 con 0x7fcdec100030 2024-05-08T21:28:54.335 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-05-08T21:28:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.336+0000 7fcdf3783700 1 -- 172.21.15.33:0/1255973884 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcdd80a1120 msgr2=0x7fcdd80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.336+0000 7fcdf3783700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcdd80a1120 0x7fcdd80a35e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009cb0 tx=0x7fcde8009260 comp rx=0 tx=0).stop 2024-05-08T21:28:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.336+0000 7fcdf3783700 1 -- 172.21.15.33:0/1255973884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdec100030 msgr2=0x7fcdec199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.336+0000 7fcdf3783700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdec100030 0x7fcdec199570 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00e9a0 tx=0x7fcddc00ecb0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.336+0000 7fcdf3783700 1 -- 172.21.15.33:0/1255973884 shutdown_connections 2024-05-08T21:28:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.336+0000 7fcdf3783700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdec1009f0 0x7fcdec1a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.336+0000 7fcdf3783700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fcdd80a1120 0x7fcdd80a35e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.336+0000 7fcdf3783700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdec0ff6d0 0x7fcdec199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.336+0000 7fcdf3783700 1 --2- 172.21.15.33:0/1255973884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdec100030 0x7fcdec199570 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.336+0000 7fcdf3783700 1 -- 172.21.15.33:0/1255973884 >> 172.21.15.33:0/1255973884 conn(0x7fcdec0fb290 msgr2=0x7fcdec0fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.336+0000 7fcdf3783700 1 -- 172.21.15.33:0/1255973884 shutdown_connections 2024-05-08T21:28:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.336+0000 7fcdf3783700 1 -- 172.21.15.33:0/1255973884 wait complete. 2024-05-08T21:28:54.351 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T21:28:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.492+0000 7f15a13db700 1 -- 172.21.15.33:0/2067557130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f159c0ffd80 msgr2=0x7f159c1001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.492+0000 7f15a13db700 1 --2- 172.21.15.33:0/2067557130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f159c0ffd80 0x7f159c1001a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f158c0099c0 tx=0x7f158c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.492+0000 7f15a13db700 1 -- 172.21.15.33:0/2067557130 shutdown_connections 2024-05-08T21:28:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.492+0000 7f15a13db700 1 --2- 172.21.15.33:0/2067557130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f159c101910 0x7f159c1081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.492+0000 7f15a13db700 1 --2- 172.21.15.33:0/2067557130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f159c0ffd80 0x7f159c1001a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.492+0000 7f15a13db700 1 --2- 172.21.15.33:0/2067557130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f159c100f70 0x7f159c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.492+0000 7f15a13db700 1 -- 172.21.15.33:0/2067557130 >> 172.21.15.33:0/2067557130 conn(0x7f159c0fb310 msgr2=0x7f159c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.492+0000 7f15a13db700 1 -- 172.21.15.33:0/2067557130 shutdown_connections 2024-05-08T21:28:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.492+0000 7f15a13db700 1 -- 172.21.15.33:0/2067557130 wait complete. 2024-05-08T21:28:54.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f15a13db700 1 Processor -- start 2024-05-08T21:28:54.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f15a13db700 1 -- start start 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f15a13db700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f159c0ffd80 0x7f159c196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f15a13db700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f159c100f70 0x7f159c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f15a13db700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f159c101910 0x7f159c19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f15a13db700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c19e860 con 0x7f159c0ffd80 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f15a13db700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c19e9d0 con 0x7f159c100f70 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f15a13db700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159c19eb90 con 0x7f159c101910 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f159affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f159c0ffd80 0x7f159c196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f159affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f159c0ffd80 0x7f159c196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42442/0 (socket says 172.21.15.33:42442) 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f159affd700 1 -- 172.21.15.33:0/3992338660 learned_addr learned my addr 172.21.15.33:0/3992338660 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f159a7fc700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f159c100f70 0x7f159c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.493+0000 7f159b7fe700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f159c101910 0x7f159c19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.494+0000 7f159affd700 1 -- 172.21.15.33:0/3992338660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f159c101910 msgr2=0x7f159c19e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.494+0000 7f159affd700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f159c101910 0x7f159c19e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.494+0000 7f159affd700 1 -- 172.21.15.33:0/3992338660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f159c100f70 msgr2=0x7f159c1972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.494+0000 7f159affd700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f159c100f70 0x7f159c1972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.494+0000 7f159affd700 1 -- 172.21.15.33:0/3992338660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c0095f0 con 0x7f159c0ffd80 2024-05-08T21:28:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.494+0000 7f159affd700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f159c0ffd80 0x7f159c196d70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f158c006010 tx=0x7f158c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.494+0000 7f1583fff700 1 -- 172.21.15.33:0/3992338660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c01d070 con 0x7f159c0ffd80 2024-05-08T21:28:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.494+0000 7f1583fff700 1 -- 172.21.15.33:0/3992338660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f158c0054c0 con 0x7f159c0ffd80 2024-05-08T21:28:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.494+0000 7f15a13db700 1 -- 172.21.15.33:0/3992338660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159c19ee10 con 0x7f159c0ffd80 2024-05-08T21:28:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.494+0000 7f1583fff700 1 -- 172.21.15.33:0/3992338660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c004bf0 con 0x7f159c0ffd80 2024-05-08T21:28:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.494+0000 7f15a13db700 1 -- 172.21.15.33:0/3992338660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159c19f180 con 0x7f159c0ffd80 2024-05-08T21:28:54.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.495+0000 7f15a13db700 1 -- 172.21.15.33:0/3992338660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159c068160 con 0x7f159c0ffd80 2024-05-08T21:28:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.496+0000 7f1583fff700 1 -- 172.21.15.33:0/3992338660 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f158c004d50 con 0x7f159c0ffd80 2024-05-08T21:28:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.497+0000 7f1583fff700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15840a1120 0x7f15840a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.497+0000 7f1583fff700 1 -- 172.21.15.33:0/3992338660 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f158c0cda80 con 0x7f159c0ffd80 2024-05-08T21:28:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.497+0000 7f159a7fc700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15840a1120 0x7f15840a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:54.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.498+0000 7f159a7fc700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15840a1120 0x7f15840a35e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1590009c60 tx=0x7f1590009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.501+0000 7f1583fff700 1 -- 172.21.15.33:0/3992338660 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f158c096f20 con 0x7f159c0ffd80 2024-05-08T21:28:54.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:54 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2427126519' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T21:28:54.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:54 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/822663916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:28:54.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:54 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1255973884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:28:54.956 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:54 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2427126519' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T21:28:54.956 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:54 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/822663916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:28:54.956 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:54 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1255973884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:28:54.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:54 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2427126519' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T21:28:54.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:54 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/822663916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:28:54.956 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:54 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1255973884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T21:28:54.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.718+0000 7f15a13db700 1 -- 172.21.15.33:0/3992338660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f159c10b690 con 0x7f159c0ffd80 2024-05-08T21:28:54.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.718+0000 7f1583fff700 1 -- 172.21.15.33:0/3992338660 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f158c09a540 con 0x7f159c0ffd80 2024-05-08T21:28:54.957 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:28:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.721+0000 7f15a13db700 1 -- 172.21.15.33:0/3992338660 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15840a1120 msgr2=0x7f15840a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.721+0000 7f15a13db700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15840a1120 0x7f15840a35e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1590009c60 tx=0x7f1590009260 comp rx=0 tx=0).stop 2024-05-08T21:28:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.721+0000 7f15a13db700 1 -- 172.21.15.33:0/3992338660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f159c0ffd80 msgr2=0x7f159c196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.721+0000 7f15a13db700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f159c0ffd80 0x7f159c196d70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f158c006010 tx=0x7f158c00f690 comp rx=0 tx=0).stop 2024-05-08T21:28:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.721+0000 7f15a13db700 1 -- 172.21.15.33:0/3992338660 shutdown_connections 2024-05-08T21:28:54.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.721+0000 7f15a13db700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f159c101910 0x7f159c19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.721+0000 7f15a13db700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f159c0ffd80 0x7f159c196d70 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.721+0000 7f15a13db700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f15840a1120 0x7f15840a35e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.721+0000 7f15a13db700 1 --2- 172.21.15.33:0/3992338660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f159c100f70 0x7f159c1972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.721+0000 7f15a13db700 1 -- 172.21.15.33:0/3992338660 >> 172.21.15.33:0/3992338660 conn(0x7f159c0fb310 msgr2=0x7f159c069540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.721+0000 7f15a13db700 1 -- 172.21.15.33:0/3992338660 shutdown_connections 2024-05-08T21:28:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:54.721+0000 7f15a13db700 1 -- 172.21.15.33:0/3992338660 wait complete. 2024-05-08T21:28:54.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T21:28:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.109+0000 7ff5135ca700 1 -- 172.21.15.33:0/456340094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff50c102990 msgr2=0x7ff50c1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.109+0000 7ff5135ca700 1 --2- 172.21.15.33:0/456340094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff50c102990 0x7ff50c1092b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff50400b0a0 tx=0x7ff50400b3b0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.109+0000 7ff5135ca700 1 -- 172.21.15.33:0/456340094 shutdown_connections 2024-05-08T21:28:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.109+0000 7ff5135ca700 1 --2- 172.21.15.33:0/456340094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff50c102990 0x7ff50c1092b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.109+0000 7ff5135ca700 1 --2- 172.21.15.33:0/456340094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff50c100e80 0x7ff50c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.109+0000 7ff5135ca700 1 --2- 172.21.15.33:0/456340094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff50c101fd0 0x7ff50c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.109+0000 7ff5135ca700 1 -- 172.21.15.33:0/456340094 >> 172.21.15.33:0/456340094 conn(0x7ff50c0fc3d0 msgr2=0x7ff50c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.109+0000 7ff5135ca700 1 -- 172.21.15.33:0/456340094 shutdown_connections 2024-05-08T21:28:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.109+0000 7ff5135ca700 1 -- 172.21.15.33:0/456340094 wait complete. 2024-05-08T21:28:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.109+0000 7ff5135ca700 1 Processor -- start 2024-05-08T21:28:55.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff5135ca700 1 -- start start 2024-05-08T21:28:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff5135ca700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff50c100e80 0x7ff50c074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff5135ca700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff50c101fd0 0x7ff50c073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff5135ca700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff50c102990 0x7ff50c0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff5135ca700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c074070 con 0x7ff50c101fd0 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff5135ca700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0741e0 con 0x7ff50c102990 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff5135ca700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c1a0df0 con 0x7ff50c100e80 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff510b65700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff50c101fd0 0x7ff50c073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff510b65700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff50c101fd0 0x7ff50c073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42454/0 (socket says 172.21.15.33:42454) 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff510b65700 1 -- 172.21.15.33:0/528648428 learned_addr learned my addr 172.21.15.33:0/528648428 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff511b67700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff50c102990 0x7ff50c0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff511366700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff50c100e80 0x7ff50c074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff510b65700 1 -- 172.21.15.33:0/528648428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff50c100e80 msgr2=0x7ff50c074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff510b65700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff50c100e80 0x7ff50c074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff510b65700 1 -- 172.21.15.33:0/528648428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff50c102990 msgr2=0x7ff50c0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff510b65700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff50c102990 0x7ff50c0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff510b65700 1 -- 172.21.15.33:0/528648428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff504009d00 con 0x7ff50c101fd0 2024-05-08T21:28:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff511366700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff50c100e80 0x7ff50c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:28:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.110+0000 7ff510b65700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff50c101fd0 0x7ff50c073390 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e9a0 tx=0x7ff50800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.111+0000 7ff5027fc700 1 -- 172.21.15.33:0/528648428 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508009790 con 0x7ff50c101fd0 2024-05-08T21:28:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.111+0000 7ff5027fc700 1 -- 172.21.15.33:0/528648428 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff508010430 con 0x7ff50c101fd0 2024-05-08T21:28:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.111+0000 7ff5135ca700 1 -- 172.21.15.33:0/528648428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c1a1080 con 0x7ff50c101fd0 2024-05-08T21:28:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.111+0000 7ff5027fc700 1 -- 172.21.15.33:0/528648428 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508018680 con 0x7ff50c101fd0 2024-05-08T21:28:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.111+0000 7ff5135ca700 1 -- 172.21.15.33:0/528648428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c1a1590 con 0x7ff50c101fd0 2024-05-08T21:28:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.112+0000 7ff5135ca700 1 -- 172.21.15.33:0/528648428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50c1082f0 con 0x7ff50c101fd0 2024-05-08T21:28:55.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.113+0000 7ff5027fc700 1 -- 172.21.15.33:0/528648428 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7ff5080105a0 con 0x7ff50c101fd0 2024-05-08T21:28:55.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.114+0000 7ff5027fc700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4f80a1120 0x7ff4f80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:55.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.114+0000 7ff5027fc700 1 -- 172.21.15.33:0/528648428 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7ff508014070 con 0x7ff50c101fd0 2024-05-08T21:28:55.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.114+0000 7ff511366700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4f80a1120 0x7ff4f80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:55.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.114+0000 7ff511366700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4f80a1120 0x7ff4f80a35e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009cb0 tx=0x7ff4fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:55.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.119+0000 7ff5027fc700 1 -- 172.21.15.33:0/528648428 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7ff508096520 con 0x7ff50c101fd0 2024-05-08T21:28:55.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.342+0000 7ff5135ca700 1 -- 172.21.15.33:0/528648428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff50c10c530 con 0x7ff50c101fd0 2024-05-08T21:28:55.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.343+0000 7ff5027fc700 1 -- 172.21.15.33:0/528648428 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5080260a0 con 0x7ff50c101fd0 2024-05-08T21:28:55.344 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-05-08T21:28:55.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.345+0000 7ff5135ca700 1 -- 172.21.15.33:0/528648428 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4f80a1120 msgr2=0x7ff4f80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.345+0000 7ff5135ca700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4f80a1120 0x7ff4f80a35e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009cb0 tx=0x7ff4fc009260 comp rx=0 tx=0).stop 2024-05-08T21:28:55.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.345+0000 7ff5135ca700 1 -- 172.21.15.33:0/528648428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff50c101fd0 msgr2=0x7ff50c073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.345+0000 7ff5135ca700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff50c101fd0 0x7ff50c073390 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff50800e9a0 tx=0x7ff50800ecb0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.345+0000 7ff5135ca700 1 -- 172.21.15.33:0/528648428 shutdown_connections 2024-05-08T21:28:55.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.345+0000 7ff5135ca700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7ff4f80a1120 0x7ff4f80a35e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.345+0000 7ff5135ca700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff50c102990 0x7ff50c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.345+0000 7ff5135ca700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff50c100e80 0x7ff50c074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.345+0000 7ff5135ca700 1 --2- 172.21.15.33:0/528648428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff50c101fd0 0x7ff50c073390 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.345+0000 7ff5135ca700 1 -- 172.21.15.33:0/528648428 >> 172.21.15.33:0/528648428 conn(0x7ff50c0fc3d0 msgr2=0x7ff50c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:55.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.345+0000 7ff5135ca700 1 -- 172.21.15.33:0/528648428 shutdown_connections 2024-05-08T21:28:55.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.345+0000 7ff5135ca700 1 -- 172.21.15.33:0/528648428 wait complete. 2024-05-08T21:28:55.361 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T21:28:55.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.501+0000 7f623ae57700 1 -- 172.21.15.33:0/4068787352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6234101920 msgr2=0x7f62341081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.501+0000 7f623ae57700 1 --2- 172.21.15.33:0/4068787352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6234101920 0x7f62341081d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f622c00b0d0 tx=0x7f622c00b3e0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.501+0000 7f623ae57700 1 -- 172.21.15.33:0/4068787352 shutdown_connections 2024-05-08T21:28:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.501+0000 7f623ae57700 1 --2- 172.21.15.33:0/4068787352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6234101920 0x7f62341081d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.501+0000 7f623ae57700 1 --2- 172.21.15.33:0/4068787352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62340ffd90 0x7f62341001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.501+0000 7f623ae57700 1 --2- 172.21.15.33:0/4068787352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6234100f80 0x7f62341013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.501+0000 7f623ae57700 1 -- 172.21.15.33:0/4068787352 >> 172.21.15.33:0/4068787352 conn(0x7f62340fb300 msgr2=0x7f62340fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.501+0000 7f623ae57700 1 -- 172.21.15.33:0/4068787352 shutdown_connections 2024-05-08T21:28:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.501+0000 7f623ae57700 1 -- 172.21.15.33:0/4068787352 wait complete. 2024-05-08T21:28:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f623ae57700 1 Processor -- start 2024-05-08T21:28:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f623ae57700 1 -- start start 2024-05-08T21:28:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f623ae57700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62340ffd90 0x7f6234196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:55.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f623ae57700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6234100f80 0x7f6234197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f623ae57700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6234101920 0x7f623419e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f623ae57700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623419e960 con 0x7f62340ffd90 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f623ae57700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623419eaa0 con 0x7f6234100f80 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f623ae57700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623419ec30 con 0x7f6234101920 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f6238bf3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62340ffd90 0x7f6234196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f6238bf3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62340ffd90 0x7f6234196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42470/0 (socket says 172.21.15.33:42470) 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f6238bf3700 1 -- 172.21.15.33:0/4038294554 learned_addr learned my addr 172.21.15.33:0/4038294554 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f62393f4700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6234101920 0x7f623419e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.502+0000 7f6233fff700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6234100f80 0x7f6234197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.503+0000 7f6238bf3700 1 -- 172.21.15.33:0/4038294554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6234101920 msgr2=0x7f623419e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.503+0000 7f6238bf3700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6234101920 0x7f623419e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.503+0000 7f6238bf3700 1 -- 172.21.15.33:0/4038294554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6234100f80 msgr2=0x7f6234197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.503+0000 7f6238bf3700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6234100f80 0x7f6234197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.503+0000 7f6238bf3700 1 -- 172.21.15.33:0/4038294554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c009d00 con 0x7f62340ffd90 2024-05-08T21:28:55.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.503+0000 7f6238bf3700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62340ffd90 0x7f6234196e30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f622400d980 tx=0x7f622400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:55.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.503+0000 7f6231ffb700 1 -- 172.21.15.33:0/4038294554 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224004020 con 0x7f62340ffd90 2024-05-08T21:28:55.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.503+0000 7f6231ffb700 1 -- 172.21.15.33:0/4038294554 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f622400f690 con 0x7f62340ffd90 2024-05-08T21:28:55.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.503+0000 7f623ae57700 1 -- 172.21.15.33:0/4038294554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623419ef10 con 0x7f62340ffd90 2024-05-08T21:28:55.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.503+0000 7f6231ffb700 1 -- 172.21.15.33:0/4038294554 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62240039e0 con 0x7f62340ffd90 2024-05-08T21:28:55.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.503+0000 7f623ae57700 1 -- 172.21.15.33:0/4038294554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623419f330 con 0x7f62340ffd90 2024-05-08T21:28:55.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.504+0000 7f623ae57700 1 -- 172.21.15.33:0/4038294554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62340674f0 con 0x7f62340ffd90 2024-05-08T21:28:55.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.505+0000 7f6231ffb700 1 -- 172.21.15.33:0/4038294554 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f622400f800 con 0x7f62340ffd90 2024-05-08T21:28:55.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.506+0000 7f6231ffb700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f62200a1120 0x7f62200a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:55.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.506+0000 7f6233fff700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f62200a1120 0x7f62200a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:55.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.506+0000 7f6231ffb700 1 -- 172.21.15.33:0/4038294554 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f62240cc870 con 0x7f62340ffd90 2024-05-08T21:28:55.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.507+0000 7f6233fff700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f62200a1120 0x7f62200a35e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f621c005950 tx=0x7f621c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:55.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.511+0000 7f6231ffb700 1 -- 172.21.15.33:0/4038294554 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f62340674f0 con 0x7f62340ffd90 2024-05-08T21:28:55.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:55 smithi033 ceph-mon[42528]: pgmap v588: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-05-08T21:28:55.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:55 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3992338660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:28:55.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:55 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/528648428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:28:55.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.731+0000 7f623ae57700 1 -- 172.21.15.33:0/4038294554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6234067480 con 0x7f62340ffd90 2024-05-08T21:28:55.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.732+0000 7f6231ffb700 1 -- 172.21.15.33:0/4038294554 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6224095c40 con 0x7f62340ffd90 2024-05-08T21:28:55.733 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-05-08T21:28:55.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.734+0000 7f623ae57700 1 -- 172.21.15.33:0/4038294554 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f62200a1120 msgr2=0x7f62200a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.734+0000 7f623ae57700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f62200a1120 0x7f62200a35e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f621c005950 tx=0x7f621c00b410 comp rx=0 tx=0).stop 2024-05-08T21:28:55.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.734+0000 7f623ae57700 1 -- 172.21.15.33:0/4038294554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62340ffd90 msgr2=0x7f6234196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.734+0000 7f623ae57700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62340ffd90 0x7f6234196e30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f622400d980 tx=0x7f622400dc90 comp rx=0 tx=0).stop 2024-05-08T21:28:55.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.734+0000 7f623ae57700 1 -- 172.21.15.33:0/4038294554 shutdown_connections 2024-05-08T21:28:55.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.734+0000 7f623ae57700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6234101920 0x7f623419e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.734+0000 7f623ae57700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62340ffd90 0x7f6234196e30 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.734+0000 7f623ae57700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f62200a1120 0x7f62200a35e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.734+0000 7f623ae57700 1 --2- 172.21.15.33:0/4038294554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6234100f80 0x7f6234197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.734+0000 7f623ae57700 1 -- 172.21.15.33:0/4038294554 >> 172.21.15.33:0/4038294554 conn(0x7f62340fb300 msgr2=0x7f6234104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:55.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.734+0000 7f623ae57700 1 -- 172.21.15.33:0/4038294554 shutdown_connections 2024-05-08T21:28:55.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.734+0000 7f623ae57700 1 -- 172.21.15.33:0/4038294554 wait complete. 2024-05-08T21:28:55.750 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-08T21:28:55.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:55 smithi071 ceph-mon[47074]: pgmap v588: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-05-08T21:28:55.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:55 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3992338660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:28:55.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:55 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/528648428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:28:55.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:55 smithi082 ceph-mon[47512]: pgmap v588: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-05-08T21:28:55.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:55 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3992338660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T21:28:55.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:55 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/528648428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T21:28:55.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.888+0000 7f5b43c3a700 1 -- 172.21.15.33:0/1667516722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b3c0ffd60 msgr2=0x7f5b3c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.888+0000 7f5b43c3a700 1 --2- 172.21.15.33:0/1667516722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b3c0ffd60 0x7f5b3c100180 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5b380099c0 tx=0x7f5b38009cd0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.888+0000 7f5b43c3a700 1 -- 172.21.15.33:0/1667516722 shutdown_connections 2024-05-08T21:28:55.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.888+0000 7f5b43c3a700 1 --2- 172.21.15.33:0/1667516722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c1018f0 0x7f5b3c1081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.888+0000 7f5b43c3a700 1 --2- 172.21.15.33:0/1667516722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b3c0ffd60 0x7f5b3c100180 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.888+0000 7f5b43c3a700 1 --2- 172.21.15.33:0/1667516722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c100f50 0x7f5b3c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.888+0000 7f5b43c3a700 1 -- 172.21.15.33:0/1667516722 >> 172.21.15.33:0/1667516722 conn(0x7f5b3c0fb290 msgr2=0x7f5b3c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.888+0000 7f5b43c3a700 1 -- 172.21.15.33:0/1667516722 shutdown_connections 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.888+0000 7f5b43c3a700 1 -- 172.21.15.33:0/1667516722 wait complete. 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.888+0000 7f5b43c3a700 1 Processor -- start 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.888+0000 7f5b43c3a700 1 -- start start 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b43c3a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c0ffd60 0x7f5b3c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b43c3a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c100f50 0x7f5b3c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b43c3a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b3c1018f0 0x7f5b3c1a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b43c3a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c1a0a40 con 0x7f5b3c1018f0 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b43c3a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c1a0b80 con 0x7f5b3c0ffd60 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b43c3a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c1a0d10 con 0x7f5b3c100f50 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b421d7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b3c1018f0 0x7f5b3c1a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b419d6700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c0ffd60 0x7f5b3c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:55.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b421d7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b3c1018f0 0x7f5b3c1a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42488/0 (socket says 172.21.15.33:42488) 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b421d7700 1 -- 172.21.15.33:0/888931841 learned_addr learned my addr 172.21.15.33:0/888931841 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b411d5700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c100f50 0x7f5b3c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b421d7700 1 -- 172.21.15.33:0/888931841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c100f50 msgr2=0x7f5b3c199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b421d7700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c100f50 0x7f5b3c199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b421d7700 1 -- 172.21.15.33:0/888931841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c0ffd60 msgr2=0x7f5b3c198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b421d7700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c0ffd60 0x7f5b3c198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b421d7700 1 -- 172.21.15.33:0/888931841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b380095f0 con 0x7f5b3c1018f0 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.889+0000 7f5b421d7700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b3c1018f0 0x7f5b3c1a0470 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5b34007ee0 tx=0x7f5b3400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.890+0000 7f5b32ffd700 1 -- 172.21.15.33:0/888931841 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34017070 con 0x7f5b3c1018f0 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.890+0000 7f5b32ffd700 1 -- 172.21.15.33:0/888931841 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b3400f040 con 0x7f5b3c1018f0 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.890+0000 7f5b43c3a700 1 -- 172.21.15.33:0/888931841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3c1a0ff0 con 0x7f5b3c1018f0 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.890+0000 7f5b32ffd700 1 -- 172.21.15.33:0/888931841 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34005570 con 0x7f5b3c1018f0 2024-05-08T21:28:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.890+0000 7f5b43c3a700 1 -- 172.21.15.33:0/888931841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3c1a1420 con 0x7f5b3c1018f0 2024-05-08T21:28:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.891+0000 7f5b43c3a700 1 -- 172.21.15.33:0/888931841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3c067d10 con 0x7f5b3c1018f0 2024-05-08T21:28:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.892+0000 7f5b32ffd700 1 -- 172.21.15.33:0/888931841 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f5b3400db20 con 0x7f5b3c1018f0 2024-05-08T21:28:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.893+0000 7f5b32ffd700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5b280a1120 0x7f5b280a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:28:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.893+0000 7f5b419d6700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5b280a1120 0x7f5b280a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:28:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.893+0000 7f5b32ffd700 1 -- 172.21.15.33:0/888931841 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f5b340cca10 con 0x7f5b3c1018f0 2024-05-08T21:28:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.893+0000 7f5b419d6700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5b280a1120 0x7f5b280a35e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5b38006010 tx=0x7f5b38000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:28:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:55.898+0000 7f5b32ffd700 1 -- 172.21.15.33:0/888931841 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f5b34095e20 con 0x7f5b3c1018f0 2024-05-08T21:28:56.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.135+0000 7f5b43c3a700 1 -- 172.21.15.33:0/888931841 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5b3c10b670 con 0x7f5b3c1018f0 2024-05-08T21:28:56.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.135+0000 7f5b32ffd700 1 -- 172.21.15.33:0/888931841 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+592 (secure 0 0 0) 0x7f5b34099440 con 0x7f5b3c1018f0 2024-05-08T21:28:56.137 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3156563197 2024-05-09T21:11:56.047045+0000 2024-05-08T21:28:56.137 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3413344235 2024-05-09T21:11:56.047045+0000 2024-05-08T21:28:56.137 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3413344235 2024-05-09T21:11:56.047045+0000 2024-05-08T21:28:56.137 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1762147139 2024-05-09T21:11:56.047045+0000 2024-05-08T21:28:56.137 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2047278949 2024-05-09T21:11:32.156884+0000 2024-05-08T21:28:56.137 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1002276836 2024-05-09T21:11:32.156884+0000 2024-05-08T21:28:56.137 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3475366293 2024-05-09T21:11:32.156884+0000 2024-05-08T21:28:56.137 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3475366293 2024-05-09T21:11:32.156884+0000 2024-05-08T21:28:56.137 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1708721255 2024-05-09T21:11:32.156884+0000 2024-05-08T21:28:56.137 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3864964868 2024-05-09T21:11:56.047045+0000 2024-05-08T21:28:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.138+0000 7f5b43c3a700 1 -- 172.21.15.33:0/888931841 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5b280a1120 msgr2=0x7f5b280a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.138+0000 7f5b43c3a700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5b280a1120 0x7f5b280a35e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5b38006010 tx=0x7f5b38000bc0 comp rx=0 tx=0).stop 2024-05-08T21:28:56.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.138+0000 7f5b43c3a700 1 -- 172.21.15.33:0/888931841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b3c1018f0 msgr2=0x7f5b3c1a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:28:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.138+0000 7f5b43c3a700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b3c1018f0 0x7f5b3c1a0470 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5b34007ee0 tx=0x7f5b3400d3b0 comp rx=0 tx=0).stop 2024-05-08T21:28:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.138+0000 7f5b43c3a700 1 -- 172.21.15.33:0/888931841 shutdown_connections 2024-05-08T21:28:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.138+0000 7f5b43c3a700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b3c1018f0 0x7f5b3c1a0470 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.138+0000 7f5b43c3a700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f5b280a1120 0x7f5b280a35e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.138+0000 7f5b43c3a700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c0ffd60 0x7f5b3c198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.138+0000 7f5b43c3a700 1 --2- 172.21.15.33:0/888931841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b3c100f50 0x7f5b3c199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:28:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.138+0000 7f5b43c3a700 1 -- 172.21.15.33:0/888931841 >> 172.21.15.33:0/888931841 conn(0x7f5b3c0fb290 msgr2=0x7f5b3c068190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:28:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.138+0000 7f5b43c3a700 1 -- 172.21.15.33:0/888931841 shutdown_connections 2024-05-08T21:28:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:28:56.138+0000 7f5b43c3a700 1 -- 172.21.15.33:0/888931841 wait complete. 2024-05-08T21:28:56.140 INFO:teuthology.orchestra.run.smithi033.stderr:listed 10 entries 2024-05-08T21:28:56.153 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-05-08T21:28:56.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:28:56.154 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-08T21:28:56.199 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[108675]: fuse finished with error 0 and tester_r 0 2024-05-08T21:28:56.201 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T21:28:56.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:56 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/4038294554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:28:56.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:56 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/888931841' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T21:28:56.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:56 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/4038294554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:28:56.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:56 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/888931841' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T21:28:56.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:56 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/4038294554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T21:28:56.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:56 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/888931841' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T21:28:57.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:57 smithi033 ceph-mon[42528]: pgmap v589: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-08T21:28:57.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:57 smithi071 ceph-mon[47074]: pgmap v589: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-08T21:28:57.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:57 smithi082 ceph-mon[47512]: pgmap v589: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 13 op/s 2024-05-08T21:28:59.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:28:59 smithi033 ceph-mon[42528]: pgmap v590: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-08T21:28:59.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:28:59 smithi071 ceph-mon[47074]: pgmap v590: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-08T21:28:59.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:28:59 smithi082 ceph-mon[47512]: pgmap v590: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-08T21:29:01.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:01 smithi033 ceph-mon[42528]: pgmap v591: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-05-08T21:29:01.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:01 smithi071 ceph-mon[47074]: pgmap v591: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-05-08T21:29:01.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:01 smithi082 ceph-mon[47512]: pgmap v591: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-05-08T21:29:02.302 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T21:29:02.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:29:02.303 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T21:29:02.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:29:02.330 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-05-08T21:29:02.392 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T21:29:02.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:29:02.393 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T21:29:02.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T21:29:02.479 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-05-08T21:29:02.554 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-08T21:29:02.565 INFO:tasks.cephadm:Teardown begin 2024-05-08T21:29:02.565 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T21:29:02.565 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:29:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.705+0000 7fa08afe1700 1 -- 172.21.15.33:0/350115681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa084101fd0 msgr2=0x7fa084102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.705+0000 7fa08afe1700 1 --2- 172.21.15.33:0/350115681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa084101fd0 0x7fa084102450 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa070009990 tx=0x7fa070009ca0 comp rx=0 tx=0).stop 2024-05-08T21:29:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.705+0000 7fa08afe1700 1 -- 172.21.15.33:0/350115681 shutdown_connections 2024-05-08T21:29:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.705+0000 7fa08afe1700 1 --2- 172.21.15.33:0/350115681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa084102990 0x7fa0841092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.705+0000 7fa08afe1700 1 --2- 172.21.15.33:0/350115681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa084100e80 0x7fa0841012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.705+0000 7fa08afe1700 1 --2- 172.21.15.33:0/350115681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa084101fd0 0x7fa084102450 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.705+0000 7fa08afe1700 1 -- 172.21.15.33:0/350115681 >> 172.21.15.33:0/350115681 conn(0x7fa0840fc410 msgr2=0x7fa0840fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.705+0000 7fa08afe1700 1 -- 172.21.15.33:0/350115681 shutdown_connections 2024-05-08T21:29:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.705+0000 7fa08afe1700 1 -- 172.21.15.33:0/350115681 wait complete. 2024-05-08T21:29:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa08afe1700 1 Processor -- start 2024-05-08T21:29:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa08afe1700 1 -- start start 2024-05-08T21:29:02.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa08afe1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa084100e80 0x7fa084074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa08afe1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa084101fd0 0x7fa084073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa08afe1700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa084102990 0x7fa0840738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa08afe1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa084074070 con 0x7fa084100e80 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa08afe1700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0840741e0 con 0x7fa084102990 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa08afe1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0841a0d40 con 0x7fa084101fd0 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa088d7d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa084100e80 0x7fa084074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa088d7d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa084100e80 0x7fa084074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40736/0 (socket says 172.21.15.33:40736) 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa088d7d700 1 -- 172.21.15.33:0/315874602 learned_addr learned my addr 172.21.15.33:0/315874602 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa08957e700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa084102990 0x7fa0840738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa083fff700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa084101fd0 0x7fa084073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa088d7d700 1 -- 172.21.15.33:0/315874602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa084101fd0 msgr2=0x7fa084073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa088d7d700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa084101fd0 0x7fa084073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa088d7d700 1 -- 172.21.15.33:0/315874602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa084102990 msgr2=0x7fa0840738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa088d7d700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa084102990 0x7fa0840738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.706+0000 7fa088d7d700 1 -- 172.21.15.33:0/315874602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0700095f0 con 0x7fa084100e80 2024-05-08T21:29:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.707+0000 7fa088d7d700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa084100e80 0x7fa084074d00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa07800e9a0 tx=0x7fa07800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.707+0000 7fa081ffb700 1 -- 172.21.15.33:0/315874602 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa078009790 con 0x7fa084100e80 2024-05-08T21:29:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.707+0000 7fa081ffb700 1 -- 172.21.15.33:0/315874602 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa078010430 con 0x7fa084100e80 2024-05-08T21:29:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.707+0000 7fa08afe1700 1 -- 172.21.15.33:0/315874602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0841a0fd0 con 0x7fa084100e80 2024-05-08T21:29:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.707+0000 7fa081ffb700 1 -- 172.21.15.33:0/315874602 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa078018680 con 0x7fa084100e80 2024-05-08T21:29:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.707+0000 7fa08afe1700 1 -- 172.21.15.33:0/315874602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0841a13f0 con 0x7fa084100e80 2024-05-08T21:29:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.708+0000 7fa08afe1700 1 -- 172.21.15.33:0/315874602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa084108330 con 0x7fa084100e80 2024-05-08T21:29:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.710+0000 7fa081ffb700 1 -- 172.21.15.33:0/315874602 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fa0780105a0 con 0x7fa084100e80 2024-05-08T21:29:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.710+0000 7fa081ffb700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa0740a1120 0x7fa0740a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.710+0000 7fa081ffb700 1 -- 172.21.15.33:0/315874602 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fa078014070 con 0x7fa084100e80 2024-05-08T21:29:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.710+0000 7fa083fff700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa0740a1120 0x7fa0740a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:02.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.711+0000 7fa083fff700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa0740a1120 0x7fa0740a35e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa07000b610 tx=0x7fa07000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:02.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.715+0000 7fa081ffb700 1 -- 172.21.15.33:0/315874602 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fa0780965f0 con 0x7fa084100e80 2024-05-08T21:29:02.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.988+0000 7fa08afe1700 1 -- 172.21.15.33:0/315874602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa08410c810 con 0x7fa084100e80 2024-05-08T21:29:02.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.989+0000 7fa081ffb700 1 -- 172.21.15.33:0/315874602 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10921 (secure 0 0 0) 0x7fa08410c810 con 0x7fa084100e80 2024-05-08T21:29:02.990 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:29:02.990 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:28:25.506048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:29:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.991+0000 7fa08afe1700 1 -- 172.21.15.33:0/315874602 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa0740a1120 msgr2=0x7fa0740a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.991+0000 7fa08afe1700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa0740a1120 0x7fa0740a35e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa07000b610 tx=0x7fa07000b260 comp rx=0 tx=0).stop 2024-05-08T21:29:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.991+0000 7fa08afe1700 1 -- 172.21.15.33:0/315874602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa084100e80 msgr2=0x7fa084074d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.991+0000 7fa08afe1700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa084100e80 0x7fa084074d00 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa07800e9a0 tx=0x7fa07800ed60 comp rx=0 tx=0).stop 2024-05-08T21:29:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.991+0000 7fa08afe1700 1 -- 172.21.15.33:0/315874602 shutdown_connections 2024-05-08T21:29:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.991+0000 7fa08afe1700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fa0740a1120 0x7fa0740a35e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.991+0000 7fa08afe1700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa084102990 0x7fa0840738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.991+0000 7fa08afe1700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa084100e80 0x7fa084074d00 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:02.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.991+0000 7fa08afe1700 1 --2- 172.21.15.33:0/315874602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa084101fd0 0x7fa084073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.991+0000 7fa08afe1700 1 -- 172.21.15.33:0/315874602 >> 172.21.15.33:0/315874602 conn(0x7fa0840fc410 msgr2=0x7fa084105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.991+0000 7fa08afe1700 1 -- 172.21.15.33:0/315874602 shutdown_connections 2024-05-08T21:29:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:02.991+0000 7fa08afe1700 1 -- 172.21.15.33:0/315874602 wait complete. 2024-05-08T21:29:02.994 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 46 2024-05-08T21:29:03.008 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-08T21:29:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.149+0000 7f657bcd9700 1 -- 172.21.15.33:0/645746313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6574100f70 msgr2=0x7f65741013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.149+0000 7f657bcd9700 1 --2- 172.21.15.33:0/645746313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6574100f70 0x7f65741013d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f65640099c0 tx=0x7f6564009cd0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.149+0000 7f657bcd9700 1 -- 172.21.15.33:0/645746313 shutdown_connections 2024-05-08T21:29:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.149+0000 7f657bcd9700 1 --2- 172.21.15.33:0/645746313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6574101910 0x7f65741081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.149+0000 7f657bcd9700 1 --2- 172.21.15.33:0/645746313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65740ffd80 0x7f65741001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.149+0000 7f657bcd9700 1 --2- 172.21.15.33:0/645746313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6574100f70 0x7f65741013d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.149+0000 7f657bcd9700 1 -- 172.21.15.33:0/645746313 >> 172.21.15.33:0/645746313 conn(0x7f65740fb310 msgr2=0x7f65740fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.149+0000 7f657bcd9700 1 -- 172.21.15.33:0/645746313 shutdown_connections 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.149+0000 7f657bcd9700 1 -- 172.21.15.33:0/645746313 wait complete. 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.150+0000 7f657bcd9700 1 Processor -- start 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.150+0000 7f657bcd9700 1 -- start start 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.150+0000 7f657bcd9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65740ffd80 0x7f6574103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.150+0000 7f657bcd9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6574100f70 0x7f6574102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.150+0000 7f657bcd9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6574101910 0x7f6574102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.150+0000 7f657bcd9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6574103030 con 0x7f6574101910 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.150+0000 7f657bcd9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65741031a0 con 0x7f65740ffd80 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.150+0000 7f657bcd9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65741a0d10 con 0x7f6574100f70 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f657a276700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6574101910 0x7f6574102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f657a276700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6574101910 0x7f6574102890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40754/0 (socket says 172.21.15.33:40754) 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f657a276700 1 -- 172.21.15.33:0/2962258561 learned_addr learned my addr 172.21.15.33:0/2962258561 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f6579a75700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65740ffd80 0x7f6574103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f6579274700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6574100f70 0x7f6574102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f657a276700 1 -- 172.21.15.33:0/2962258561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6574100f70 msgr2=0x7f6574102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f657a276700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6574100f70 0x7f6574102350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f657a276700 1 -- 172.21.15.33:0/2962258561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65740ffd80 msgr2=0x7f6574103cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f657a276700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65740ffd80 0x7f6574103cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f657a276700 1 -- 172.21.15.33:0/2962258561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65640095f0 con 0x7f6574101910 2024-05-08T21:29:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f6579274700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6574100f70 0x7f6574102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:29:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f657a276700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6574101910 0x7f6574102890 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f656c00ef30 tx=0x7f656c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f656affd700 1 -- 172.21.15.33:0/2962258561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c018070 con 0x7f6574101910 2024-05-08T21:29:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f656affd700 1 -- 172.21.15.33:0/2962258561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f656c00f040 con 0x7f6574101910 2024-05-08T21:29:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f657bcd9700 1 -- 172.21.15.33:0/2962258561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65741a0f40 con 0x7f6574101910 2024-05-08T21:29:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f656affd700 1 -- 172.21.15.33:0/2962258561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c00cc10 con 0x7f6574101910 2024-05-08T21:29:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.151+0000 7f657bcd9700 1 -- 172.21.15.33:0/2962258561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65741a13c0 con 0x7f6574101910 2024-05-08T21:29:03.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.153+0000 7f657bcd9700 1 -- 172.21.15.33:0/2962258561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6574067d10 con 0x7f6574101910 2024-05-08T21:29:03.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.154+0000 7f656affd700 1 -- 172.21.15.33:0/2962258561 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f656c007500 con 0x7f6574101910 2024-05-08T21:29:03.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.155+0000 7f656affd700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65600a11a0 0x7f65600a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:03.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.155+0000 7f6579a75700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65600a11a0 0x7f65600a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:03.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.155+0000 7f656affd700 1 -- 172.21.15.33:0/2962258561 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f656c0cc6a0 con 0x7f6574101910 2024-05-08T21:29:03.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.155+0000 7f6579a75700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65600a11a0 0x7f65600a3660 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6574100dd0 tx=0x7f6570007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.160+0000 7f656affd700 1 -- 172.21.15.33:0/2962258561 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f656c095b40 con 0x7f6574101910 2024-05-08T21:29:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.437+0000 7f657bcd9700 1 -- 172.21.15.33:0/2962258561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f657410b440 con 0x7f6574101910 2024-05-08T21:29:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.438+0000 7f656affd700 1 -- 172.21.15.33:0/2962258561 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10921 (secure 0 0 0) 0x7f656c099160 con 0x7f6574101910 2024-05-08T21:29:03.439 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:29:03.439 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14610,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/1299124477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":1299124477},{"type":"v1","addr":"172.21.15.82:6839","nonce":1299124477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/2620722111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":2620722111},{"type":"v1","addr":"172.21.15.33:6839","nonce":2620722111}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T21:17:03.219607+0000","modified":"2024-05-08T21:28:25.506048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24563,"mds_1":24521,"mds_2":24499,"mds_3":24497,"mds_4":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/2132539478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":2132539478},{"type":"v1","addr":"172.21.15.33:6841","nonce":2132539478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24451":{"gid":24451,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6833/3382287612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3382287612},{"type":"v1","addr":"172.21.15.82:6833","nonce":3382287612}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24460":{"gid":24460,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/82548780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":82548780},{"type":"v1","addr":"172.21.15.33:6835","nonce":82548780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/497942273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":497942273},{"type":"v1","addr":"172.21.15.71:6833","nonce":497942273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24472":{"gid":24472,"name":"d","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/362093971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":362093971},{"type":"v1","addr":"172.21.15.33:6837","nonce":362093971}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24487":{"gid":24487,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/1877231014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1877231014},{"type":"v1","addr":"172.21.15.82:6835","nonce":1877231014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6835/3686264359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3686264359},{"type":"v1","addr":"172.21.15.71:6835","nonce":3686264359}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24499":{"gid":24499,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.82:6837/2409190198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2409190198},{"type":"v1","addr":"172.21.15.82:6837","nonce":2409190198}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6837/1668078897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":1668078897},{"type":"v1","addr":"172.21.15.71:6837","nonce":1668078897}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24563":{"gid":24563,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1515211151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1515211151},{"type":"v1","addr":"172.21.15.71:6839","nonce":1515211151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-05-08T21:29:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.440+0000 7f657bcd9700 1 -- 172.21.15.33:0/2962258561 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65600a11a0 msgr2=0x7f65600a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.440+0000 7f657bcd9700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65600a11a0 0x7f65600a3660 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6574100dd0 tx=0x7f6570007400 comp rx=0 tx=0).stop 2024-05-08T21:29:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.440+0000 7f657bcd9700 1 -- 172.21.15.33:0/2962258561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6574101910 msgr2=0x7f6574102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.440+0000 7f657bcd9700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6574101910 0x7f6574102890 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f656c00ef30 tx=0x7f656c00c3a0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.440+0000 7f657bcd9700 1 -- 172.21.15.33:0/2962258561 shutdown_connections 2024-05-08T21:29:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.440+0000 7f657bcd9700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6574101910 0x7f6574102890 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.440+0000 7f657bcd9700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f65600a11a0 0x7f65600a3660 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.440+0000 7f657bcd9700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65740ffd80 0x7f6574103cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.440+0000 7f657bcd9700 1 --2- 172.21.15.33:0/2962258561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6574100f70 0x7f6574102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.440+0000 7f657bcd9700 1 -- 172.21.15.33:0/2962258561 >> 172.21.15.33:0/2962258561 conn(0x7f65740fb310 msgr2=0x7f6574068160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:03.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.441+0000 7f657bcd9700 1 -- 172.21.15.33:0/2962258561 shutdown_connections 2024-05-08T21:29:03.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.441+0000 7f657bcd9700 1 -- 172.21.15.33:0/2962258561 wait complete. 2024-05-08T21:29:03.443 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 46 2024-05-08T21:29:03.456 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T21:29:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.597+0000 7fc2d71c2700 1 -- 172.21.15.33:0/2851820125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d00ffd80 msgr2=0x7fc2d01001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.597+0000 7fc2d71c2700 1 --2- 172.21.15.33:0/2851820125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d00ffd80 0x7fc2d01001a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0099c0 tx=0x7fc2cc009cd0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.597+0000 7fc2d71c2700 1 -- 172.21.15.33:0/2851820125 shutdown_connections 2024-05-08T21:29:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.597+0000 7fc2d71c2700 1 --2- 172.21.15.33:0/2851820125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d0101910 0x7fc2d01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.597+0000 7fc2d71c2700 1 --2- 172.21.15.33:0/2851820125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d00ffd80 0x7fc2d01001a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.597+0000 7fc2d71c2700 1 --2- 172.21.15.33:0/2851820125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d0100f70 0x7fc2d01013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.597+0000 7fc2d71c2700 1 -- 172.21.15.33:0/2851820125 >> 172.21.15.33:0/2851820125 conn(0x7fc2d00fb310 msgr2=0x7fc2d00fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.597+0000 7fc2d71c2700 1 -- 172.21.15.33:0/2851820125 shutdown_connections 2024-05-08T21:29:03.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.597+0000 7fc2d71c2700 1 -- 172.21.15.33:0/2851820125 wait complete. 2024-05-08T21:29:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.598+0000 7fc2d71c2700 1 Processor -- start 2024-05-08T21:29:03.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.598+0000 7fc2d71c2700 1 -- start start 2024-05-08T21:29:03.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.598+0000 7fc2d71c2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d00ffd80 0x7fc2d0195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:03.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.598+0000 7fc2d71c2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d0100f70 0x7fc2d01961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:03.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.598+0000 7fc2d71c2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d0101910 0x7fc2d019d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:03.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.598+0000 7fc2d71c2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d019d760 con 0x7fc2d0101910 2024-05-08T21:29:03.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.598+0000 7fc2d71c2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d019d8d0 con 0x7fc2d0100f70 2024-05-08T21:29:03.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.598+0000 7fc2d71c2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d019da90 con 0x7fc2d00ffd80 2024-05-08T21:29:03.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2d575f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d0101910 0x7fc2d019d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:03.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2d575f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d0101910 0x7fc2d019d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40776/0 (socket says 172.21.15.33:40776) 2024-05-08T21:29:03.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2d575f700 1 -- 172.21.15.33:0/270773993 learned_addr learned my addr 172.21.15.33:0/270773993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:29:03.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2d4f5e700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d00ffd80 0x7fc2d0195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2c7fff700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d0100f70 0x7fc2d01961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2d575f700 1 -- 172.21.15.33:0/270773993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d00ffd80 msgr2=0x7fc2d0195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2d575f700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d00ffd80 0x7fc2d0195c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2d575f700 1 -- 172.21.15.33:0/270773993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d0100f70 msgr2=0x7fc2d01961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2d575f700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d0100f70 0x7fc2d01961b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2d575f700 1 -- 172.21.15.33:0/270773993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc0095f0 con 0x7fc2d0101910 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2c7fff700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d0100f70 0x7fc2d01961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2d575f700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d0101910 0x7fc2d019d190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8007ee0 tx=0x7fc2c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2c5ffb700 1 -- 172.21.15.33:0/270773993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8017070 con 0x7fc2d0101910 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2c5ffb700 1 -- 172.21.15.33:0/270773993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2c800f040 con 0x7fc2d0101910 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.599+0000 7fc2d71c2700 1 -- 172.21.15.33:0/270773993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d019dd70 con 0x7fc2d0101910 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.600+0000 7fc2c5ffb700 1 -- 172.21.15.33:0/270773993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8005570 con 0x7fc2d0101910 2024-05-08T21:29:03.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.600+0000 7fc2d71c2700 1 -- 172.21.15.33:0/270773993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d019e1a0 con 0x7fc2d0101910 2024-05-08T21:29:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.601+0000 7fc2d71c2700 1 -- 172.21.15.33:0/270773993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d01072f0 con 0x7fc2d0101910 2024-05-08T21:29:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.602+0000 7fc2c5ffb700 1 -- 172.21.15.33:0/270773993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7fc2c800db20 con 0x7fc2d0101910 2024-05-08T21:29:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.603+0000 7fc2c5ffb700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2b80a1120 0x7fc2b80a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.603+0000 7fc2d4f5e700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2b80a1120 0x7fc2b80a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.603+0000 7fc2c5ffb700 1 -- 172.21.15.33:0/270773993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7fc2c80cc420 con 0x7fc2d0101910 2024-05-08T21:29:03.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.604+0000 7fc2d4f5e700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2b80a1120 0x7fc2b80a35e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc006010 tx=0x7fc2cc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:03.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.608+0000 7fc2c5ffb700 1 -- 172.21.15.33:0/270773993 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7fc2c80957f0 con 0x7fc2d0101910 2024-05-08T21:29:03.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:03 smithi033 ceph-mon[42528]: pgmap v592: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-08T21:29:03.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:03 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/315874602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:29:03.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:03 smithi071 ceph-mon[47074]: pgmap v592: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-08T21:29:03.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:03 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/315874602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:29:03.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.830+0000 7fc2d71c2700 1 -- 172.21.15.33:0/270773993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2d0067d10 con 0x7fc2d0101910 2024-05-08T21:29:03.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.832+0000 7fc2c5ffb700 1 -- 172.21.15.33:0/270773993 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24561 (secure 0 0 0) 0x7fc2c8098e10 con 0x7fc2d0101910 2024-05-08T21:29:03.834 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-05-08T21:29:03.834 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":89,"fsid":"53ae3a5a-0d7f-11ef-bc97-c7b262605968","created":"2024-05-08T21:11:07.208389+0000","modified":"2024-05-08T21:18:58.419274+0000","last_up_change":"2024-05-08T21:16:28.205385+0000","last_in_change":"2024-05-08T21:16:12.473872+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T21:13:48.583196+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T21:17:01.282803+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T21:17:02.555552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T21:17:05.686277+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a37bcd68-6698-4832-8f60-78c1e15a7208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6803","nonce":1942383393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6805","nonce":1942383393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6809","nonce":1942383393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1942383393},{"type":"v1","addr":"172.21.15.33:6807","nonce":1942383393}]},"public_addr":"172.21.15.33:6803/1942383393","cluster_addr":"172.21.15.33:6805/1942383393","heartbeat_back_addr":"172.21.15.33:6809/1942383393","heartbeat_front_addr":"172.21.15.33:6807/1942383393","state":["exists","up"]},{"osd":1,"uuid":"f789dc73-b4d8-49f3-9a6f-83388b7245da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6801","nonce":2306412237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6803","nonce":2306412237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6807","nonce":2306412237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2306412237},{"type":"v1","addr":"172.21.15.71:6805","nonce":2306412237}]},"public_addr":"172.21.15.71:6801/2306412237","cluster_addr":"172.21.15.71:6803/2306412237","heartbeat_back_addr":"172.21.15.71:6807/2306412237","heartbeat_front_addr":"172.21.15.71:6805/2306412237","state":["exists","up"]},{"osd":2,"uuid":"b6c73166-7e6f-41d9-a8ad-713e7ddceb63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6801","nonce":3057197540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6803","nonce":3057197540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6807","nonce":3057197540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":3057197540},{"type":"v1","addr":"172.21.15.82:6805","nonce":3057197540}]},"public_addr":"172.21.15.82:6801/3057197540","cluster_addr":"172.21.15.82:6803/3057197540","heartbeat_back_addr":"172.21.15.82:6807/3057197540","heartbeat_front_addr":"172.21.15.82:6805/3057197540","state":["exists","up"]},{"osd":3,"uuid":"a1b61b4d-bbc1-448a-856a-5be6034cb981","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6811","nonce":2632370799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6813","nonce":2632370799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6817","nonce":2632370799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2632370799},{"type":"v1","addr":"172.21.15.33:6815","nonce":2632370799}]},"public_addr":"172.21.15.33:6811/2632370799","cluster_addr":"172.21.15.33:6813/2632370799","heartbeat_back_addr":"172.21.15.33:6817/2632370799","heartbeat_front_addr":"172.21.15.33:6815/2632370799","state":["exists","up"]},{"osd":4,"uuid":"21a8fee3-514c-4bea-b364-566a8664ee55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6809","nonce":4006373764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6811","nonce":4006373764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6815","nonce":4006373764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":4006373764},{"type":"v1","addr":"172.21.15.71:6813","nonce":4006373764}]},"public_addr":"172.21.15.71:6809/4006373764","cluster_addr":"172.21.15.71:6811/4006373764","heartbeat_back_addr":"172.21.15.71:6815/4006373764","heartbeat_front_addr":"172.21.15.71:6813/4006373764","state":["exists","up"]},{"osd":5,"uuid":"7669410f-f67f-41a0-9b5b-3cc04a4ab45b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6809","nonce":1310343719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6811","nonce":1310343719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6815","nonce":1310343719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1310343719},{"type":"v1","addr":"172.21.15.82:6813","nonce":1310343719}]},"public_addr":"172.21.15.82:6809/1310343719","cluster_addr":"172.21.15.82:6811/1310343719","heartbeat_back_addr":"172.21.15.82:6815/1310343719","heartbeat_front_addr":"172.21.15.82:6813/1310343719","state":["exists","up"]},{"osd":6,"uuid":"b7ff5abb-f264-461b-8f31-afd52b29bd59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6819","nonce":3543778506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6821","nonce":3543778506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6825","nonce":3543778506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3543778506},{"type":"v1","addr":"172.21.15.33:6823","nonce":3543778506}]},"public_addr":"172.21.15.33:6819/3543778506","cluster_addr":"172.21.15.33:6821/3543778506","heartbeat_back_addr":"172.21.15.33:6825/3543778506","heartbeat_front_addr":"172.21.15.33:6823/3543778506","state":["exists","up"]},{"osd":7,"uuid":"c1de3175-dfdd-40ad-a3d3-52a45dec6b2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6817","nonce":1213981909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6819","nonce":1213981909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6823","nonce":1213981909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1213981909},{"type":"v1","addr":"172.21.15.71:6821","nonce":1213981909}]},"public_addr":"172.21.15.71:6817/1213981909","cluster_addr":"172.21.15.71:6819/1213981909","heartbeat_back_addr":"172.21.15.71:6823/1213981909","heartbeat_front_addr":"172.21.15.71:6821/1213981909","state":["exists","up"]},{"osd":8,"uuid":"4a7620db-cbad-473d-a5fd-ab506474d661","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6817","nonce":1451701395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6819","nonce":1451701395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6823","nonce":1451701395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1451701395},{"type":"v1","addr":"172.21.15.82:6821","nonce":1451701395}]},"public_addr":"172.21.15.82:6817/1451701395","cluster_addr":"172.21.15.82:6819/1451701395","heartbeat_back_addr":"172.21.15.82:6823/1451701395","heartbeat_front_addr":"172.21.15.82:6821/1451701395","state":["exists","up"]},{"osd":9,"uuid":"227000e2-dc41-45c2-84f6-4a4345b83328","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6827","nonce":4123781180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6829","nonce":4123781180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6833","nonce":4123781180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":4123781180},{"type":"v1","addr":"172.21.15.33:6831","nonce":4123781180}]},"public_addr":"172.21.15.33:6827/4123781180","cluster_addr":"172.21.15.33:6829/4123781180","heartbeat_back_addr":"172.21.15.33:6833/4123781180","heartbeat_front_addr":"172.21.15.33:6831/4123781180","state":["exists","up"]},{"osd":10,"uuid":"768308bc-54b7-43ff-af95-87a481d38bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6825","nonce":3245386576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6827","nonce":3245386576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6831","nonce":3245386576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":3245386576},{"type":"v1","addr":"172.21.15.71:6829","nonce":3245386576}]},"public_addr":"172.21.15.71:6825/3245386576","cluster_addr":"172.21.15.71:6827/3245386576","heartbeat_back_addr":"172.21.15.71:6831/3245386576","heartbeat_front_addr":"172.21.15.71:6829/3245386576","state":["exists","up"]},{"osd":11,"uuid":"760e78e6-d9db-44c1-bcbd-dfe68bfbfe6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6825","nonce":1998522386}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6827","nonce":1998522386}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6831","nonce":1998522386}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1998522386},{"type":"v1","addr":"172.21.15.82:6829","nonce":1998522386}]},"public_addr":"172.21.15.82:6825/1998522386","cluster_addr":"172.21.15.82:6827/1998522386","heartbeat_back_addr":"172.21.15.82:6831/1998522386","heartbeat_front_addr":"172.21.15.82:6829/1998522386","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:09.819653+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:27.497507+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:13:44.856034+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:03.833476+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:21.073417+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:38.313849+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:14:57.025668+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:14.794214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:32.304191+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:15:51.684182+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:09.485242+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T21:16:27.180839+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.2a","mappings":[{"from":3,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/3156563197":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6801/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:6800/3413344235":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/1762147139":"2024-05-09T21:11:56.047045+0000","172.21.15.33:0/2047278949":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1002276836":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6800/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:6801/3475366293":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/1708721255":"2024-05-09T21:11:32.156884+0000","172.21.15.33:0/3864964868":"2024-05-09T21:11:56.047045+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T21:29:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.835+0000 7fc2d71c2700 1 -- 172.21.15.33:0/270773993 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2b80a1120 msgr2=0x7fc2b80a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.835+0000 7fc2d71c2700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2b80a1120 0x7fc2b80a35e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc006010 tx=0x7fc2cc000bc0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.835+0000 7fc2d71c2700 1 -- 172.21.15.33:0/270773993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d0101910 msgr2=0x7fc2d019d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.835+0000 7fc2d71c2700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d0101910 0x7fc2d019d190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8007ee0 tx=0x7fc2c800d3b0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.835+0000 7fc2d71c2700 1 -- 172.21.15.33:0/270773993 shutdown_connections 2024-05-08T21:29:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.835+0000 7fc2d71c2700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2d0101910 0x7fc2d019d190 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.835+0000 7fc2d71c2700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2d00ffd80 0x7fc2d0195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.835+0000 7fc2d71c2700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7fc2b80a1120 0x7fc2b80a35e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.835+0000 7fc2d71c2700 1 --2- 172.21.15.33:0/270773993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d0100f70 0x7fc2d01961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.835+0000 7fc2d71c2700 1 -- 172.21.15.33:0/270773993 >> 172.21.15.33:0/270773993 conn(0x7fc2d00fb310 msgr2=0x7fc2d0104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.835+0000 7fc2d71c2700 1 -- 172.21.15.33:0/270773993 shutdown_connections 2024-05-08T21:29:03.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.835+0000 7fc2d71c2700 1 -- 172.21.15.33:0/270773993 wait complete. 2024-05-08T21:29:03.851 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-08T21:29:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:03 smithi082 ceph-mon[47512]: pgmap v592: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-05-08T21:29:03.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:03 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/315874602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:29:03.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.991+0000 7f0266128700 1 -- 172.21.15.33:0/3522370376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0260104210 msgr2=0x7f0260106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.991+0000 7f0266128700 1 --2- 172.21.15.33:0/3522370376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0260104210 0x7f0260106600 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0254009990 tx=0x7f0254009ca0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.991+0000 7f0266128700 1 -- 172.21.15.33:0/3522370376 shutdown_connections 2024-05-08T21:29:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.991+0000 7f0266128700 1 --2- 172.21.15.33:0/3522370376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0260106b40 0x7f0260108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.991+0000 7f0266128700 1 --2- 172.21.15.33:0/3522370376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02601018e0 0x7f0260103cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.991+0000 7f0266128700 1 --2- 172.21.15.33:0/3522370376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0260104210 0x7f0260106600 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.991+0000 7f0266128700 1 -- 172.21.15.33:0/3522370376 >> 172.21.15.33:0/3522370376 conn(0x7f02600fb290 msgr2=0x7f02600fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.991+0000 7f0266128700 1 -- 172.21.15.33:0/3522370376 shutdown_connections 2024-05-08T21:29:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.991+0000 7f0266128700 1 -- 172.21.15.33:0/3522370376 wait complete. 2024-05-08T21:29:03.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.992+0000 7f0266128700 1 Processor -- start 2024-05-08T21:29:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.992+0000 7f0266128700 1 -- start start 2024-05-08T21:29:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.992+0000 7f0266128700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02601018e0 0x7f0260196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.992+0000 7f0266128700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0260104210 0x7f0260197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.992+0000 7f0266128700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0260106b40 0x7f026019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.992+0000 7f0266128700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026019e830 con 0x7f0260104210 2024-05-08T21:29:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.992+0000 7f0266128700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026019e9a0 con 0x7f02601018e0 2024-05-08T21:29:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.992+0000 7f0266128700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026019eb60 con 0x7f0260106b40 2024-05-08T21:29:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.992+0000 7f025effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0260104210 0x7f0260197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.992+0000 7f025effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0260104210 0x7f0260197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40790/0 (socket says 172.21.15.33:40790) 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.992+0000 7f025effd700 1 -- 172.21.15.33:0/1724166859 learned_addr learned my addr 172.21.15.33:0/1724166859 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025ffff700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0260106b40 0x7f026019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025f7fe700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02601018e0 0x7f0260196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025effd700 1 -- 172.21.15.33:0/1724166859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0260106b40 msgr2=0x7f026019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025effd700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0260106b40 0x7f026019e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025effd700 1 -- 172.21.15.33:0/1724166859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02601018e0 msgr2=0x7f0260196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025effd700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02601018e0 0x7f0260196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025effd700 1 -- 172.21.15.33:0/1724166859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02540095f0 con 0x7f0260104210 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025ffff700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0260106b40 0x7f026019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025f7fe700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02601018e0 0x7f0260196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025effd700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0260104210 0x7f0260197280 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0254004710 tx=0x7f0254009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025cff9700 1 -- 172.21.15.33:0/1724166859 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025401d070 con 0x7f0260104210 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025cff9700 1 -- 172.21.15.33:0/1724166859 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f025400f460 con 0x7f0260104210 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f025cff9700 1 -- 172.21.15.33:0/1724166859 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254021670 con 0x7f0260104210 2024-05-08T21:29:03.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f0266128700 1 -- 172.21.15.33:0/1724166859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026019ede0 con 0x7f0260104210 2024-05-08T21:29:03.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.993+0000 7f0266128700 1 -- 172.21.15.33:0/1724166859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026019f1b0 con 0x7f0260104210 2024-05-08T21:29:03.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.995+0000 7f0266128700 1 -- 172.21.15.33:0/1724166859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02600fce10 con 0x7f0260104210 2024-05-08T21:29:03.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.996+0000 7f025cff9700 1 -- 172.21.15.33:0/1724166859 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f025400f5d0 con 0x7f0260104210 2024-05-08T21:29:03.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.996+0000 7f025cff9700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02480a1120 0x7f02480a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:03.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.996+0000 7f025cff9700 1 -- 172.21.15.33:0/1724166859 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9136+0+0 (secure 0 0 0) 0x7f02540ce0a0 con 0x7f0260104210 2024-05-08T21:29:03.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.996+0000 7f025f7fe700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02480a1120 0x7f02480a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:04.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:03.997+0000 7f025f7fe700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02480a1120 0x7f02480a35e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f02601981c0 tx=0x7f025000b690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:04.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.001+0000 7f025cff9700 1 -- 172.21.15.33:0/1724166859 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f0254097540 con 0x7f0260104210 2024-05-08T21:29:04.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.296+0000 7f0266128700 1 -- 172.21.15.33:0/1724166859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0260067d10 con 0x7f0260104210 2024-05-08T21:29:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.311+0000 7f025cff9700 1 -- 172.21.15.33:0/1724166859 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7f025409ab60 con 0x7f0260104210 2024-05-08T21:29:04.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.318+0000 7f0266128700 1 -- 172.21.15.33:0/1724166859 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02480a1120 msgr2=0x7f02480a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:04.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.318+0000 7f0266128700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02480a1120 0x7f02480a35e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f02601981c0 tx=0x7f025000b690 comp rx=0 tx=0).stop 2024-05-08T21:29:04.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.318+0000 7f0266128700 1 -- 172.21.15.33:0/1724166859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0260104210 msgr2=0x7f0260197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:04.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.318+0000 7f0266128700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0260104210 0x7f0260197280 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0254004710 tx=0x7f0254009580 comp rx=0 tx=0).stop 2024-05-08T21:29:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.321+0000 7f0266128700 1 -- 172.21.15.33:0/1724166859 shutdown_connections 2024-05-08T21:29:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.321+0000 7f0266128700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0260106b40 0x7f026019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.321+0000 7f0266128700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f02480a1120 0x7f02480a35e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.321+0000 7f0266128700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02601018e0 0x7f0260196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.321+0000 7f0266128700 1 --2- 172.21.15.33:0/1724166859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0260104210 0x7f0260197280 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:04.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.321+0000 7f0266128700 1 -- 172.21.15.33:0/1724166859 >> 172.21.15.33:0/1724166859 conn(0x7f02600fb290 msgr2=0x7f02601023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.322+0000 7f0266128700 1 -- 172.21.15.33:0/1724166859 shutdown_connections 2024-05-08T21:29:04.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.322+0000 7f0266128700 1 -- 172.21.15.33:0/1724166859 wait complete. 2024-05-08T21:29:04.324 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T21:29:04.346 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T21:29:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.503+0000 7f7a88fc9700 1 -- 172.21.15.33:0/1334422204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a84100f50 msgr2=0x7f7a841013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.503+0000 7f7a88fc9700 1 --2- 172.21.15.33:0/1334422204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a84100f50 0x7f7a841013b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0099c0 tx=0x7f7a6c009cd0 comp rx=0 tx=0).stop 2024-05-08T21:29:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.503+0000 7f7a88fc9700 1 -- 172.21.15.33:0/1334422204 shutdown_connections 2024-05-08T21:29:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.503+0000 7f7a88fc9700 1 --2- 172.21.15.33:0/1334422204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a841018f0 0x7f7a841081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.503+0000 7f7a88fc9700 1 --2- 172.21.15.33:0/1334422204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a840ffd60 0x7f7a84100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.503+0000 7f7a88fc9700 1 --2- 172.21.15.33:0/1334422204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a84100f50 0x7f7a841013b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.503+0000 7f7a88fc9700 1 -- 172.21.15.33:0/1334422204 >> 172.21.15.33:0/1334422204 conn(0x7f7a840fb290 msgr2=0x7f7a840fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.503+0000 7f7a88fc9700 1 -- 172.21.15.33:0/1334422204 shutdown_connections 2024-05-08T21:29:04.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.503+0000 7f7a88fc9700 1 -- 172.21.15.33:0/1334422204 wait complete. 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.504+0000 7f7a88fc9700 1 Processor -- start 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.505+0000 7f7a88fc9700 1 -- start start 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.505+0000 7f7a88fc9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a840ffd60 0x7f7a84195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.505+0000 7f7a88fc9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a84100f50 0x7f7a84196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.505+0000 7f7a88fc9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a841018f0 0x7f7a8419d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.505+0000 7f7a88fc9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8419d720 con 0x7f7a841018f0 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.505+0000 7f7a88fc9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8419d860 con 0x7f7a84100f50 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.505+0000 7f7a88fc9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8419d9f0 con 0x7f7a840ffd60 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.505+0000 7f7a82d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a841018f0 0x7f7a8419d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.505+0000 7f7a82d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a841018f0 0x7f7a8419d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40862/0 (socket says 172.21.15.33:40862) 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.505+0000 7f7a82d9d700 1 -- 172.21.15.33:0/2417972970 learned_addr learned my addr 172.21.15.33:0/2417972970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.505+0000 7f7a8259c700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a840ffd60 0x7f7a84195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:04.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.505+0000 7f7a79d9b700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a84100f50 0x7f7a84196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.506+0000 7f7a82d9d700 1 -- 172.21.15.33:0/2417972970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a840ffd60 msgr2=0x7f7a84195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.506+0000 7f7a82d9d700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a840ffd60 0x7f7a84195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.506+0000 7f7a82d9d700 1 -- 172.21.15.33:0/2417972970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a84100f50 msgr2=0x7f7a84196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.506+0000 7f7a82d9d700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a84100f50 0x7f7a84196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.506+0000 7f7a82d9d700 1 -- 172.21.15.33:0/2417972970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c0095f0 con 0x7f7a841018f0 2024-05-08T21:29:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.506+0000 7f7a8259c700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a840ffd60 0x7f7a84195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T21:29:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.506+0000 7f7a82d9d700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a841018f0 0x7f7a8419d150 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c007ee0 tx=0x7f7a7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.506+0000 7f7a7bfff700 1 -- 172.21.15.33:0/2417972970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c017070 con 0x7f7a841018f0 2024-05-08T21:29:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.506+0000 7f7a88fc9700 1 -- 172.21.15.33:0/2417972970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8419dcd0 con 0x7f7a841018f0 2024-05-08T21:29:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.506+0000 7f7a7bfff700 1 -- 172.21.15.33:0/2417972970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7a7c00f040 con 0x7f7a841018f0 2024-05-08T21:29:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.506+0000 7f7a7bfff700 1 -- 172.21.15.33:0/2417972970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c005570 con 0x7f7a841018f0 2024-05-08T21:29:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.506+0000 7f7a88fc9700 1 -- 172.21.15.33:0/2417972970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8419e190 con 0x7f7a841018f0 2024-05-08T21:29:04.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.507+0000 7f7a88fc9700 1 -- 172.21.15.33:0/2417972970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a84107230 con 0x7f7a841018f0 2024-05-08T21:29:04.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.508+0000 7f7a7bfff700 1 -- 172.21.15.33:0/2417972970 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f7a7c00db20 con 0x7f7a841018f0 2024-05-08T21:29:04.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.509+0000 7f7a7bfff700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7a700a1120 0x7f7a700a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:04.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.509+0000 7f7a8259c700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7a700a1120 0x7f7a700a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:04.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.509+0000 7f7a7bfff700 1 -- 172.21.15.33:0/2417972970 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9566+0+0 (secure 0 0 0) 0x7f7a7c0cc5c0 con 0x7f7a841018f0 2024-05-08T21:29:04.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.510+0000 7f7a8259c700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7a700a1120 0x7f7a700a35e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7a84100db0 tx=0x7f7a7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:04.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.514+0000 7f7a7bfff700 1 -- 172.21.15.33:0/2417972970 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f7a7c0957c0 con 0x7f7a841018f0 2024-05-08T21:29:04.706 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 08 21:29:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-a[93434]: ignoring --setuser ceph since I am not root 2024-05-08T21:29:04.707 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 08 21:29:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-a[93434]: ignoring --setgroup ceph since I am not root 2024-05-08T21:29:04.707 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 08 21:29:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-a[93434]: starting mds.a at 2024-05-08T21:29:04.707 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:29:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-j[92140]: ignoring --setuser ceph since I am not root 2024-05-08T21:29:04.707 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:29:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-j[92140]: ignoring --setgroup ceph since I am not root 2024-05-08T21:29:04.707 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:29:04 smithi033 ceph-mds[92163]: -- 172.21.15.33:0/4198075303 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5652985761a0 con 0x56529854d000 2024-05-08T21:29:04.707 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:29:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-j[92140]: starting mds.j at 2024-05-08T21:29:04.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-a[42503]: 2024-05-08T21:29:04.297+0000 7f9147c1a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T21:29:04.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:04 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2962258561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:29:04.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:04 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/270773993' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:29:04.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:04 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1724166859' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T21:29:04.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:04 smithi033 ceph-mon[42528]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T21:29:04.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:04 smithi033 ceph-mon[42528]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T21:29:04.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:04 smithi033 ceph-mon[42528]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T21:29:04.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:04 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/1724166859' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T21:29:04.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:04 smithi033 ceph-mon[42528]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T21:29:04.708 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:29:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-d[92529]: ignoring --setuser ceph since I am not root 2024-05-08T21:29:04.708 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:29:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-d[92529]: ignoring --setgroup ceph since I am not root 2024-05-08T21:29:04.709 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:29:04 smithi033 ceph-mds[92552]: -- 172.21.15.33:0/3736247666 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55843a5f61a0 con 0x55843a5ccc00 2024-05-08T21:29:04.709 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:29:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-d[92529]: starting mds.d at 2024-05-08T21:29:04.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:04.798+0000 7f7a88fc9700 1 -- 172.21.15.33:0/2417972970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7a84067d10 con 0x7f7a841018f0 2024-05-08T21:29:04.819 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:29:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-b[83724]: ignoring --setuser ceph since I am not root 2024-05-08T21:29:04.819 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:29:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-b[83724]: ignoring --setgroup ceph since I am not root 2024-05-08T21:29:04.819 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:29:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-b[83724]: starting mds.b at 2024-05-08T21:29:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:04 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2962258561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:29:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:04 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/270773993' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:29:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:04 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1724166859' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T21:29:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:04 smithi071 ceph-mon[47074]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T21:29:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:04 smithi071 ceph-mon[47074]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T21:29:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:04 smithi071 ceph-mon[47074]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T21:29:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:04 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/1724166859' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T21:29:04.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:04 smithi071 ceph-mon[47074]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T21:29:04.821 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:29:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-e[83040]: ignoring --setuser ceph since I am not root 2024-05-08T21:29:04.821 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:29:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-e[83040]: ignoring --setgroup ceph since I am not root 2024-05-08T21:29:04.821 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:29:04 smithi071 ceph-mds[83063]: -- 172.21.15.71:0/3395485452 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c4e5d401a0 con 0x55c4e5d16c00 2024-05-08T21:29:04.821 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:29:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-e[83040]: starting mds.e at 2024-05-08T21:29:04.821 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:29:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-h[83382]: ignoring --setuser ceph since I am not root 2024-05-08T21:29:04.821 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:29:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-h[83382]: ignoring --setgroup ceph since I am not root 2024-05-08T21:29:04.821 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:29:04 smithi071 ceph-mds[83405]: -- 172.21.15.71:0/1620448032 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ac41bc51e0 con 0x55ac41b8f000 2024-05-08T21:29:04.821 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:29:04 smithi071 ceph-mds[83405]: -- 172.21.15.71:0/1620448032 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ac41bb81a0 con 0x55ac41b8f000 2024-05-08T21:29:04.821 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:29:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-h[83382]: starting mds.h at 2024-05-08T21:29:04.822 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:29:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-k[82698]: ignoring --setuser ceph since I am not root 2024-05-08T21:29:04.822 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:29:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-k[82698]: ignoring --setgroup ceph since I am not root 2024-05-08T21:29:04.822 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:29:04 smithi071 ceph-mds[82721]: -- 172.21.15.71:0/1398465637 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a9b80cc1a0 con 0x55a9b80a2c00 2024-05-08T21:29:04.822 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:29:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-k[82698]: starting mds.k at 2024-05-08T21:29:04.852 INFO:journalctl@ceph.mds.f.smithi082.stdout:May 08 21:29:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-f[84325]: ignoring --setuser ceph since I am not root 2024-05-08T21:29:04.852 INFO:journalctl@ceph.mds.f.smithi082.stdout:May 08 21:29:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-f[84325]: ignoring --setgroup ceph since I am not root 2024-05-08T21:29:04.852 INFO:journalctl@ceph.mds.f.smithi082.stdout:May 08 21:29:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-f[84325]: starting mds.f at 2024-05-08T21:29:04.852 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:29:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-l[83975]: ignoring --setuser ceph since I am not root 2024-05-08T21:29:04.852 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:29:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-l[83975]: ignoring --setgroup ceph since I am not root 2024-05-08T21:29:04.852 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:29:04 smithi082 ceph-mds[83998]: -- 172.21.15.82:0/2432648841 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a349bb21a0 con 0x55a349b89000 2024-05-08T21:29:04.852 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:29:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-l[83975]: starting mds.l at 2024-05-08T21:29:04.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:04 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2962258561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T21:29:04.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:04 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/270773993' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T21:29:04.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:04 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1724166859' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T21:29:04.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:04 smithi082 ceph-mon[47512]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T21:29:04.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:04 smithi082 ceph-mon[47512]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T21:29:04.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:04 smithi082 ceph-mon[47512]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T21:29:04.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:04 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/1724166859' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T21:29:04.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:04 smithi082 ceph-mon[47512]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T21:29:04.853 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:29:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-c[84671]: ignoring --setuser ceph since I am not root 2024-05-08T21:29:04.853 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:29:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-c[84671]: ignoring --setgroup ceph since I am not root 2024-05-08T21:29:04.853 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:29:04 smithi082 ceph-mds[84694]: -- 172.21.15.82:0/889148457 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561c00f4a1a0 con 0x561c00f20c00 2024-05-08T21:29:04.854 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:29:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-c[84671]: starting mds.c at 2024-05-08T21:29:05.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.316+0000 7f7a7bfff700 1 -- 172.21.15.33:0/2417972970 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7a7c098de0 con 0x7f7a841018f0 2024-05-08T21:29:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.321+0000 7f7a88fc9700 1 -- 172.21.15.33:0/2417972970 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7a700a1120 msgr2=0x7f7a700a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.321+0000 7f7a88fc9700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7a700a1120 0x7f7a700a35e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7a84100db0 tx=0x7f7a7400b410 comp rx=0 tx=0).stop 2024-05-08T21:29:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.321+0000 7f7a88fc9700 1 -- 172.21.15.33:0/2417972970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a841018f0 msgr2=0x7f7a8419d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.321+0000 7f7a88fc9700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a841018f0 0x7f7a8419d150 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c007ee0 tx=0x7f7a7c00d3b0 comp rx=0 tx=0).stop 2024-05-08T21:29:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.321+0000 7f7a88fc9700 1 -- 172.21.15.33:0/2417972970 shutdown_connections 2024-05-08T21:29:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.321+0000 7f7a88fc9700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a841018f0 0x7f7a8419d150 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.321+0000 7f7a88fc9700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a840ffd60 0x7f7a84195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.321+0000 7f7a88fc9700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f7a700a1120 0x7f7a700a35e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.321+0000 7f7a88fc9700 1 --2- 172.21.15.33:0/2417972970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a84100f50 0x7f7a84196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.321+0000 7f7a88fc9700 1 -- 172.21.15.33:0/2417972970 >> 172.21.15.33:0/2417972970 conn(0x7f7a840fb290 msgr2=0x7f7a84104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.324+0000 7f7a88fc9700 1 -- 172.21.15.33:0/2417972970 shutdown_connections 2024-05-08T21:29:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.324+0000 7f7a88fc9700 1 -- 172.21.15.33:0/2417972970 wait complete. 2024-05-08T21:29:05.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T21:29:05.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.488+0000 7f39ad316700 1 -- 172.21.15.33:0/365134843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8106b80 msgr2=0x7f39a8108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:05.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.488+0000 7f39ad316700 1 --2- 172.21.15.33:0/365134843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8106b80 0x7f39a8108ff0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f39a000b0a0 tx=0x7f39a000b3b0 comp rx=0 tx=0).stop 2024-05-08T21:29:05.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.488+0000 7f39ad316700 1 -- 172.21.15.33:0/365134843 shutdown_connections 2024-05-08T21:29:05.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.488+0000 7f39ad316700 1 --2- 172.21.15.33:0/365134843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8106b80 0x7f39a8108ff0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:05.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.488+0000 7f39ad316700 1 --2- 172.21.15.33:0/365134843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39a8101920 0x7f39a8103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:05.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.488+0000 7f39ad316700 1 --2- 172.21.15.33:0/365134843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a8104250 0x7f39a8106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:05.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.489+0000 7f39ad316700 1 -- 172.21.15.33:0/365134843 >> 172.21.15.33:0/365134843 conn(0x7f39a80fb330 msgr2=0x7f39a80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:05.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.489+0000 7f39ad316700 1 -- 172.21.15.33:0/365134843 shutdown_connections 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.489+0000 7f39ad316700 1 -- 172.21.15.33:0/365134843 wait complete. 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.489+0000 7f39ad316700 1 Processor -- start 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.489+0000 7f39ad316700 1 -- start start 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f39ad316700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39a8101920 0x7f39a8101240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f39ad316700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8104250 0x7f39a80ff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f39ad316700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a8106b80 0x7f39a80ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f39ad316700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a81005b0 con 0x7f39a8104250 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f39ad316700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a8100720 con 0x7f39a8106b80 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f39ad316700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a81a0e30 con 0x7f39a8101920 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f399e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8104250 0x7f39a80ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f399e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8104250 0x7f39a80ff8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40890/0 (socket says 172.21.15.33:40890) 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f399e7fc700 1 -- 172.21.15.33:0/383191032 learned_addr learned my addr 172.21.15.33:0/383191032 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f39a77fe700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a8106b80 0x7f39a80ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:05.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f39a6ffd700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39a8101920 0x7f39a8101240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:05.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f399e7fc700 1 -- 172.21.15.33:0/383191032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39a8101920 msgr2=0x7f39a8101240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:05.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f399e7fc700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39a8101920 0x7f39a8101240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:05.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f399e7fc700 1 -- 172.21.15.33:0/383191032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a8106b80 msgr2=0x7f39a80ffe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:05.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f399e7fc700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a8106b80 0x7f39a80ffe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:05.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f399e7fc700 1 -- 172.21.15.33:0/383191032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a0009d00 con 0x7f39a8104250 2024-05-08T21:29:05.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f399e7fc700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8104250 0x7f39a80ff8d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f399000e9a0 tx=0x7f399000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:05.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.490+0000 7f39a4ff9700 1 -- 172.21.15.33:0/383191032 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990009790 con 0x7f39a8104250 2024-05-08T21:29:05.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.491+0000 7f39ad316700 1 -- 172.21.15.33:0/383191032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a81a10c0 con 0x7f39a8104250 2024-05-08T21:29:05.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.491+0000 7f39a4ff9700 1 -- 172.21.15.33:0/383191032 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3990010430 con 0x7f39a8104250 2024-05-08T21:29:05.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.491+0000 7f39a4ff9700 1 -- 172.21.15.33:0/383191032 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990018680 con 0x7f39a8104250 2024-05-08T21:29:05.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.491+0000 7f39ad316700 1 -- 172.21.15.33:0/383191032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a81a15d0 con 0x7f39a8104250 2024-05-08T21:29:05.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.492+0000 7f39ad316700 1 -- 172.21.15.33:0/383191032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a80fceb0 con 0x7f39a8104250 2024-05-08T21:29:05.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.493+0000 7f39a4ff9700 1 -- 172.21.15.33:0/383191032 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f39900105a0 con 0x7f39a8104250 2024-05-08T21:29:05.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.493+0000 7f39a4ff9700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39940a1120 0x7f39940a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:05.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.494+0000 7f39a4ff9700 1 -- 172.21.15.33:0/383191032 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9566+0+0 (secure 0 0 0) 0x7f3990014070 con 0x7f39a8104250 2024-05-08T21:29:05.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.494+0000 7f39a6ffd700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39940a1120 0x7f39940a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:05.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.494+0000 7f39a6ffd700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39940a1120 0x7f39940a35e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3998009bf0 tx=0x7f3998009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:05.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.498+0000 7f39a4ff9700 1 -- 172.21.15.33:0/383191032 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f3990096520 con 0x7f39a8104250 2024-05-08T21:29:05.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: pgmap v594: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2417972970' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: Cluster is now healthy 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.71:6836/3582033139,v1:172.21.15.71:6837/3582033139] up:boot 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.71:6838/3887110983,v1:172.21.15.71:6839/3887110983] up:boot 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.82:6836/2614578257,v1:172.21.15.82:6837/2614578257] up:boot 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.82:6832/2151653857,v1:172.21.15.82:6833/2151653857] up:boot 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.71:6833/2131930998,v1:172.21.15.71:6835/2131930998] up:boot 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.71:6832/2256906565,v1:172.21.15.71:6834/2256906565] up:boot 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.82:6834/1711694179,v1:172.21.15.82:6835/1711694179] up:boot 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.33:6834/4031866552,v1:172.21.15.33:6835/4031866552] up:boot 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.33:6836/2383037772,v1:172.21.15.33:6840/2383037772] up:boot 2024-05-08T21:29:05.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: mds.? [v2:172.21.15.33:6837/2234128676,v1:172.21.15.33:6841/2234128676] up:boot 2024-05-08T21:29:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/2417972970' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T21:29:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: fsmap 12 up:standby 2024-05-08T21:29:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T21:29:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T21:29:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T21:29:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T21:29:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T21:29:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T21:29:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T21:29:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T21:29:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T21:29:05.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:05 smithi033 ceph-mon[42528]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T21:29:05.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:05.713+0000 7f39ad316700 1 -- 172.21.15.33:0/383191032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f39a8068160 con 0x7f39a8104250 2024-05-08T21:29:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: pgmap v594: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-08T21:29:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2417972970' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:29:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T21:29:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T21:29:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: Cluster is now healthy 2024-05-08T21:29:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.71:6836/3582033139,v1:172.21.15.71:6837/3582033139] up:boot 2024-05-08T21:29:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.71:6838/3887110983,v1:172.21.15.71:6839/3887110983] up:boot 2024-05-08T21:29:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.82:6836/2614578257,v1:172.21.15.82:6837/2614578257] up:boot 2024-05-08T21:29:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.82:6832/2151653857,v1:172.21.15.82:6833/2151653857] up:boot 2024-05-08T21:29:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.71:6833/2131930998,v1:172.21.15.71:6835/2131930998] up:boot 2024-05-08T21:29:05.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.71:6832/2256906565,v1:172.21.15.71:6834/2256906565] up:boot 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.82:6834/1711694179,v1:172.21.15.82:6835/1711694179] up:boot 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.33:6834/4031866552,v1:172.21.15.33:6835/4031866552] up:boot 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.33:6836/2383037772,v1:172.21.15.33:6840/2383037772] up:boot 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: mds.? [v2:172.21.15.33:6837/2234128676,v1:172.21.15.33:6841/2234128676] up:boot 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/2417972970' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: fsmap 12 up:standby 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T21:29:05.821 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T21:29:05.822 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:05 smithi071 ceph-mon[47074]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T21:29:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: pgmap v594: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-05-08T21:29:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2417972970' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T21:29:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T21:29:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T21:29:05.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: Cluster is now healthy 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.71:6836/3582033139,v1:172.21.15.71:6837/3582033139] up:boot 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.71:6838/3887110983,v1:172.21.15.71:6839/3887110983] up:boot 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.82:6836/2614578257,v1:172.21.15.82:6837/2614578257] up:boot 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.82:6832/2151653857,v1:172.21.15.82:6833/2151653857] up:boot 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.71:6833/2131930998,v1:172.21.15.71:6835/2131930998] up:boot 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.71:6832/2256906565,v1:172.21.15.71:6834/2256906565] up:boot 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.82:6834/1711694179,v1:172.21.15.82:6835/1711694179] up:boot 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.33:6834/4031866552,v1:172.21.15.33:6835/4031866552] up:boot 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.33:6836/2383037772,v1:172.21.15.33:6840/2383037772] up:boot 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: mds.? [v2:172.21.15.33:6837/2234128676,v1:172.21.15.33:6841/2234128676] up:boot 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/2417972970' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: fsmap 12 up:standby 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T21:29:05.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T21:29:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T21:29:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T21:29:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T21:29:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T21:29:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T21:29:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T21:29:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T21:29:05.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:05 smithi082 ceph-mon[47512]: from='mgr.14152 172.21.15.33:0/3866876127' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T21:29:06.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.446+0000 7f39a4ff9700 1 -- 172.21.15.33:0/383191032 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 181+0+0 (secure 0 0 0) 0x7f39900260d0 con 0x7f39a8104250 2024-05-08T21:29:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.450+0000 7f39ad316700 1 -- 172.21.15.33:0/383191032 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39940a1120 msgr2=0x7f39940a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.450+0000 7f39ad316700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39940a1120 0x7f39940a35e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3998009bf0 tx=0x7f3998009260 comp rx=0 tx=0).stop 2024-05-08T21:29:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.450+0000 7f39ad316700 1 -- 172.21.15.33:0/383191032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8104250 msgr2=0x7f39a80ff8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.450+0000 7f39ad316700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8104250 0x7f39a80ff8d0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f399000e9a0 tx=0x7f399000ecb0 comp rx=0 tx=0).stop 2024-05-08T21:29:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.450+0000 7f39ad316700 1 -- 172.21.15.33:0/383191032 shutdown_connections 2024-05-08T21:29:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.450+0000 7f39ad316700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f39940a1120 0x7f39940a35e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.450+0000 7f39ad316700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a8106b80 0x7f39a80ffe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.450+0000 7f39ad316700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39a8101920 0x7f39a8101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.450+0000 7f39ad316700 1 --2- 172.21.15.33:0/383191032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a8104250 0x7f39a80ff8d0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.450+0000 7f39ad316700 1 -- 172.21.15.33:0/383191032 >> 172.21.15.33:0/383191032 conn(0x7f39a80fb330 msgr2=0x7f39a81023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.450+0000 7f39ad316700 1 -- 172.21.15.33:0/383191032 shutdown_connections 2024-05-08T21:29:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.450+0000 7f39ad316700 1 -- 172.21.15.33:0/383191032 wait complete. 2024-05-08T21:29:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2024-05-08T21:29:06.468 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T21:29:06.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.613+0000 7f68c36a3700 1 -- 172.21.15.33:0/3186676164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68bc1018f0 msgr2=0x7f68bc1081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:06.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.613+0000 7f68c36a3700 1 --2- 172.21.15.33:0/3186676164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68bc1018f0 0x7f68bc1081a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f68b400b0d0 tx=0x7f68b400b3e0 comp rx=0 tx=0).stop 2024-05-08T21:29:06.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.613+0000 7f68c36a3700 1 -- 172.21.15.33:0/3186676164 shutdown_connections 2024-05-08T21:29:06.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.613+0000 7f68c36a3700 1 --2- 172.21.15.33:0/3186676164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68bc1018f0 0x7f68bc1081a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:06.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.613+0000 7f68c36a3700 1 --2- 172.21.15.33:0/3186676164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68bc0ffd60 0x7f68bc100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:06.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.613+0000 7f68c36a3700 1 --2- 172.21.15.33:0/3186676164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68bc100f50 0x7f68bc1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:06.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.613+0000 7f68c36a3700 1 -- 172.21.15.33:0/3186676164 >> 172.21.15.33:0/3186676164 conn(0x7f68bc0fb290 msgr2=0x7f68bc0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:06.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.613+0000 7f68c36a3700 1 -- 172.21.15.33:0/3186676164 shutdown_connections 2024-05-08T21:29:06.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.613+0000 7f68c36a3700 1 -- 172.21.15.33:0/3186676164 wait complete. 2024-05-08T21:29:06.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c36a3700 1 Processor -- start 2024-05-08T21:29:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c36a3700 1 -- start start 2024-05-08T21:29:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c36a3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68bc0ffd60 0x7f68bc196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c36a3700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68bc100f50 0x7f68bc197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c36a3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68bc1018f0 0x7f68bc19e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c36a3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc19e970 con 0x7f68bc0ffd60 2024-05-08T21:29:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c36a3700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc19eab0 con 0x7f68bc100f50 2024-05-08T21:29:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c36a3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc19ec40 con 0x7f68bc1018f0 2024-05-08T21:29:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c143f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68bc0ffd60 0x7f68bc196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c143f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68bc0ffd60 0x7f68bc196e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40908/0 (socket says 172.21.15.33:40908) 2024-05-08T21:29:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c143f700 1 -- 172.21.15.33:0/923375320 learned_addr learned my addr 172.21.15.33:0/923375320 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:29:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c0c3e700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68bc100f50 0x7f68bc197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c1c40700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68bc1018f0 0x7f68bc19e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c143f700 1 -- 172.21.15.33:0/923375320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68bc1018f0 msgr2=0x7f68bc19e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c143f700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68bc1018f0 0x7f68bc19e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c143f700 1 -- 172.21.15.33:0/923375320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68bc100f50 msgr2=0x7f68bc197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c143f700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68bc100f50 0x7f68bc197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c143f700 1 -- 172.21.15.33:0/923375320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b4009d00 con 0x7f68bc0ffd60 2024-05-08T21:29:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c1c40700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68bc1018f0 0x7f68bc19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T21:29:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.614+0000 7f68c143f700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68bc0ffd60 0x7f68bc196e40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f68b800d980 tx=0x7f68b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.615+0000 7f68b27fc700 1 -- 172.21.15.33:0/923375320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8004020 con 0x7f68bc0ffd60 2024-05-08T21:29:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.615+0000 7f68b27fc700 1 -- 172.21.15.33:0/923375320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f68b8009730 con 0x7f68bc0ffd60 2024-05-08T21:29:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.615+0000 7f68c36a3700 1 -- 172.21.15.33:0/923375320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68bc19ef20 con 0x7f68bc0ffd60 2024-05-08T21:29:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.615+0000 7f68b27fc700 1 -- 172.21.15.33:0/923375320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8003a00 con 0x7f68bc0ffd60 2024-05-08T21:29:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.615+0000 7f68c36a3700 1 -- 172.21.15.33:0/923375320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68bc19f350 con 0x7f68bc0ffd60 2024-05-08T21:29:06.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.616+0000 7f68c36a3700 1 -- 172.21.15.33:0/923375320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68bc067d10 con 0x7f68bc0ffd60 2024-05-08T21:29:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.617+0000 7f68b27fc700 1 -- 172.21.15.33:0/923375320 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f68b80098a0 con 0x7f68bc0ffd60 2024-05-08T21:29:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.617+0000 7f68b27fc700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f68a80a11a0 0x7f68a80a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.618+0000 7f68c0c3e700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f68a80a11a0 0x7f68a80a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.618+0000 7f68b27fc700 1 -- 172.21.15.33:0/923375320 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9012+0+0 (secure 0 0 0) 0x7f68b80ccc30 con 0x7f68bc0ffd60 2024-05-08T21:29:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.618+0000 7f68c0c3e700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f68a80a11a0 0x7f68a80a3660 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009c60 tx=0x7f68ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:06.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.622+0000 7f68b27fc700 1 -- 172.21.15.33:0/923375320 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f68b80960d0 con 0x7f68bc0ffd60 2024-05-08T21:29:06.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:06 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/383191032' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T21:29:06.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:06 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/383191032' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T21:29:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:06.842+0000 7f68c36a3700 1 -- 172.21.15.33:0/923375320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f68bc10b3e0 con 0x7f68bc0ffd60 2024-05-08T21:29:06.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:06 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/383191032' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T21:29:07.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.449+0000 7f68b27fc700 1 -- 172.21.15.33:0/923375320 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 169+0+0 (secure 0 0 0) 0x7f68b80170d0 con 0x7f68bc0ffd60 2024-05-08T21:29:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.454+0000 7f68c36a3700 1 -- 172.21.15.33:0/923375320 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f68a80a11a0 msgr2=0x7f68a80a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:07.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.454+0000 7f68c36a3700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f68a80a11a0 0x7f68a80a3660 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009c60 tx=0x7f68ac009260 comp rx=0 tx=0).stop 2024-05-08T21:29:07.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.454+0000 7f68c36a3700 1 -- 172.21.15.33:0/923375320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68bc0ffd60 msgr2=0x7f68bc196e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:07.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.454+0000 7f68c36a3700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68bc0ffd60 0x7f68bc196e40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f68b800d980 tx=0x7f68b800dc90 comp rx=0 tx=0).stop 2024-05-08T21:29:07.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.457+0000 7f68c36a3700 1 -- 172.21.15.33:0/923375320 shutdown_connections 2024-05-08T21:29:07.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.457+0000 7f68c36a3700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68bc1018f0 0x7f68bc19e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:07.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.457+0000 7f68c36a3700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68bc0ffd60 0x7f68bc196e40 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:07.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.457+0000 7f68c36a3700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f68a80a11a0 0x7f68a80a3660 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:07.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.457+0000 7f68c36a3700 1 --2- 172.21.15.33:0/923375320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68bc100f50 0x7f68bc197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:07.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.457+0000 7f68c36a3700 1 -- 172.21.15.33:0/923375320 >> 172.21.15.33:0/923375320 conn(0x7f68bc0fb290 msgr2=0x7f68bc0681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:07.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.457+0000 7f68c36a3700 1 -- 172.21.15.33:0/923375320 shutdown_connections 2024-05-08T21:29:07.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.457+0000 7f68c36a3700 1 -- 172.21.15.33:0/923375320 wait complete. 2024-05-08T21:29:07.460 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2024-05-08T21:29:07.479 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-08T21:29:07.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.640+0000 7f2cef3dc700 1 -- 172.21.15.33:0/370287384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8101fd0 msgr2=0x7f2ce8102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:07.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.640+0000 7f2cef3dc700 1 --2- 172.21.15.33:0/370287384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8101fd0 0x7f2ce8102450 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009990 tx=0x7f2cd8009ca0 comp rx=0 tx=0).stop 2024-05-08T21:29:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.640+0000 7f2cef3dc700 1 -- 172.21.15.33:0/370287384 shutdown_connections 2024-05-08T21:29:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.640+0000 7f2cef3dc700 1 --2- 172.21.15.33:0/370287384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ce8102990 0x7f2ce81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.640+0000 7f2cef3dc700 1 --2- 172.21.15.33:0/370287384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ce8100e80 0x7f2ce81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.640+0000 7f2cef3dc700 1 --2- 172.21.15.33:0/370287384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8101fd0 0x7f2ce8102450 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.640+0000 7f2cef3dc700 1 -- 172.21.15.33:0/370287384 >> 172.21.15.33:0/370287384 conn(0x7f2ce80fc3f0 msgr2=0x7f2ce80fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.640+0000 7f2cef3dc700 1 -- 172.21.15.33:0/370287384 shutdown_connections 2024-05-08T21:29:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.640+0000 7f2cef3dc700 1 -- 172.21.15.33:0/370287384 wait complete. 2024-05-08T21:29:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2cef3dc700 1 Processor -- start 2024-05-08T21:29:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2cef3dc700 1 -- start start 2024-05-08T21:29:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2cef3dc700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ce8100e80 0x7f2ce8074d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:07.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2cef3dc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ce8101fd0 0x7f2ce8073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2cef3dc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8102990 0x7f2ce80738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2cef3dc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8074070 con 0x7f2ce8102990 2024-05-08T21:29:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2cef3dc700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80741e0 con 0x7f2ce8100e80 2024-05-08T21:29:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2cef3dc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce81a0d50 con 0x7f2ce8101fd0 2024-05-08T21:29:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2ced979700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8102990 0x7f2ce80738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2ced979700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8102990 0x7f2ce80738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40922/0 (socket says 172.21.15.33:40922) 2024-05-08T21:29:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2ced979700 1 -- 172.21.15.33:0/3394882491 learned_addr learned my addr 172.21.15.33:0/3394882491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-05-08T21:29:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2cec977700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ce8101fd0 0x7f2ce8073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.641+0000 7f2ced178700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ce8100e80 0x7f2ce8074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.642+0000 7f2ced979700 1 -- 172.21.15.33:0/3394882491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ce8101fd0 msgr2=0x7f2ce8073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:07.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.642+0000 7f2ced979700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ce8101fd0 0x7f2ce8073390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.642+0000 7f2ced979700 1 -- 172.21.15.33:0/3394882491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ce8100e80 msgr2=0x7f2ce8074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.642+0000 7f2ced979700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ce8100e80 0x7f2ce8074d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.642+0000 7f2ced979700 1 -- 172.21.15.33:0/3394882491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd80095f0 con 0x7f2ce8102990 2024-05-08T21:29:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.642+0000 7f2ced979700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8102990 0x7f2ce80738d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0007ee0 tx=0x7f2ce000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.642+0000 7f2cde7fc700 1 -- 172.21.15.33:0/3394882491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce000dcc0 con 0x7f2ce8102990 2024-05-08T21:29:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.642+0000 7f2cde7fc700 1 -- 172.21.15.33:0/3394882491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ce000f040 con 0x7f2ce8102990 2024-05-08T21:29:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.642+0000 7f2cef3dc700 1 -- 172.21.15.33:0/3394882491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce81a0fe0 con 0x7f2ce8102990 2024-05-08T21:29:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.642+0000 7f2cde7fc700 1 -- 172.21.15.33:0/3394882491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce00046e0 con 0x7f2ce8102990 2024-05-08T21:29:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.642+0000 7f2cef3dc700 1 -- 172.21.15.33:0/3394882491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce81a1410 con 0x7f2ce8102990 2024-05-08T21:29:07.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.643+0000 7f2cef3dc700 1 -- 172.21.15.33:0/3394882491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce8108340 con 0x7f2ce8102990 2024-05-08T21:29:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.644+0000 7f2cde7fc700 1 -- 172.21.15.33:0/3394882491 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135873+0+0 (secure 0 0 0) 0x7f2ce0003a00 con 0x7f2ce8102990 2024-05-08T21:29:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.645+0000 7f2cde7fc700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2cd40a1120 0x7f2cd40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T21:29:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.645+0000 7f2cde7fc700 1 -- 172.21.15.33:0/3394882491 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8424+0+0 (secure 0 0 0) 0x7f2ce00cc960 con 0x7f2ce8102990 2024-05-08T21:29:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.645+0000 7f2ced178700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2cd40a1120 0x7f2cd40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T21:29:07.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.646+0000 7f2ced178700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2cd40a1120 0x7f2cd40a35e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4007c60 tx=0x7f2ce40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T21:29:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.649+0000 7f2cde7fc700 1 -- 172.21.15.33:0/3394882491 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183622 (secure 0 0 0) 0x7f2ce0095fd0 con 0x7f2ce8102990 2024-05-08T21:29:07.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:07 smithi071 ceph-mon[47074]: pgmap v595: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-08T21:29:07.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:07 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/383191032' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T21:29:07.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:07 smithi071 ceph-mon[47074]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T21:29:07.820 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:07 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/923375320' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T21:29:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:07 smithi082 ceph-mon[47512]: pgmap v595: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-08T21:29:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:07 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/383191032' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T21:29:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:07 smithi082 ceph-mon[47512]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T21:29:07.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:07 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/923375320' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T21:29:07.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:07.868+0000 7f2cef3dc700 1 -- 172.21.15.33:0/3394882491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2ce8068160 con 0x7f2ce8102990 2024-05-08T21:29:07.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:07 smithi033 ceph-mon[42528]: pgmap v595: 129 pgs: 129 active+clean; 225 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-08T21:29:07.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:07 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/383191032' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T21:29:07.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:07 smithi033 ceph-mon[42528]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T21:29:07.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:07 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/923375320' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T21:29:08.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.467+0000 7f2cde7fc700 1 -- 172.21.15.33:0/3394882491 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v93) v1 ==== 178+0+0 (secure 0 0 0) 0x7f2ce00230d0 con 0x7f2ce8102990 2024-05-08T21:29:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.474+0000 7f2cef3dc700 1 -- 172.21.15.33:0/3394882491 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2cd40a1120 msgr2=0x7f2cd40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.474+0000 7f2cef3dc700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2cd40a1120 0x7f2cd40a35e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4007c60 tx=0x7f2ce40073d0 comp rx=0 tx=0).stop 2024-05-08T21:29:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.474+0000 7f2cef3dc700 1 -- 172.21.15.33:0/3394882491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8102990 msgr2=0x7f2ce80738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T21:29:08.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.474+0000 7f2cef3dc700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8102990 0x7f2ce80738d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0007ee0 tx=0x7f2ce000d3b0 comp rx=0 tx=0).stop 2024-05-08T21:29:08.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.482+0000 7f2cef3dc700 1 -- 172.21.15.33:0/3394882491 shutdown_connections 2024-05-08T21:29:08.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.482+0000 7f2cef3dc700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8102990 0x7f2ce80738d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:08.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.482+0000 7f2cef3dc700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.33:6800/2288189532,v1:172.21.15.33:6801/2288189532] conn(0x7f2cd40a1120 0x7f2cd40a35e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:08.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.482+0000 7f2cef3dc700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ce8100e80 0x7f2ce8074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:08.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.482+0000 7f2cef3dc700 1 --2- 172.21.15.33:0/3394882491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ce8101fd0 0x7f2ce8073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T21:29:08.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.482+0000 7f2cef3dc700 1 -- 172.21.15.33:0/3394882491 >> 172.21.15.33:0/3394882491 conn(0x7f2ce80fc3f0 msgr2=0x7f2ce8105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T21:29:08.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.484+0000 7f2cef3dc700 1 -- 172.21.15.33:0/3394882491 shutdown_connections 2024-05-08T21:29:08.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-05-08T21:29:08.484+0000 7f2cef3dc700 1 -- 172.21.15.33:0/3394882491 wait complete. 2024-05-08T21:29:08.486 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data_ec' removed 2024-05-08T21:29:08.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:29:08.539 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:29:08.573 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:29:08.608 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-08T21:29:08.608 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T21:29:08.628 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T21:29:08.646 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T21:29:08.666 INFO:tasks.cephadm:Stopping all daemons... 2024-05-08T21:29:08.667 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-08T21:29:08.667 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.a 2024-05-08T21:29:08.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:08 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/923375320' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T21:29:08.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:08 smithi071 ceph-mon[47074]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T21:29:08.819 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:08 smithi071 ceph-mon[47074]: from='client.? 172.21.15.33:0/3394882491' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T21:29:08.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:08 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/923375320' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T21:29:08.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:08 smithi082 ceph-mon[47512]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T21:29:08.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:08 smithi082 ceph-mon[47512]: from='client.? 172.21.15.33:0/3394882491' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T21:29:08.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:08 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/923375320' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T21:29:08.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:08 smithi033 ceph-mon[42528]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T21:29:08.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:08 smithi033 ceph-mon[42528]: from='client.? 172.21.15.33:0/3394882491' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T21:29:08.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:08 smithi033 systemd[1]: Stopping Ceph mon.a for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:09.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:08 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-a[42503]: 2024-05-08T21:29:08.907+0000 7f914d425700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:29:09.207 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:08 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-a[42503]: 2024-05-08T21:29:08.907+0000 7f914d425700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-08T21:29:09.609 INFO:journalctl@ceph.mon.a.smithi033.stdout:May 08 21:29:09 smithi033 bash[140441]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-a 2024-05-08T21:29:09.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.a.service' 2024-05-08T21:29:09.676 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:29:09.676 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-08T21:29:09.676 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-08T21:29:09.677 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.b 2024-05-08T21:29:10.069 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:09 smithi071 systemd[1]: Stopping Ceph mon.b for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:10.069 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:09 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-b[47051]: 2024-05-08T21:29:09.922+0000 7f87631a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:29:10.069 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:09 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-b[47051]: 2024-05-08T21:29:09.922+0000 7f87631a4700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-08T21:29:10.449 INFO:journalctl@ceph.mon.b.smithi071.stdout:May 08 21:29:10 smithi071 bash[88015]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-b 2024-05-08T21:29:10.456 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.b.service' 2024-05-08T21:29:10.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:29:10.509 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-08T21:29:10.509 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-08T21:29:10.509 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.c 2024-05-08T21:29:10.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:10 smithi082 systemd[1]: Stopping Ceph mon.c for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:10.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:10 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-c[47490]: 2024-05-08T21:29:10.779+0000 7fb15cf53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:29:10.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:10 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-c[47490]: 2024-05-08T21:29:10.779+0000 7fb15cf53700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-08T21:29:11.418 INFO:journalctl@ceph.mon.c.smithi082.stdout:May 08 21:29:11 smithi082 bash[89649]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mon-c 2024-05-08T21:29:11.424 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mon.c.service' 2024-05-08T21:29:11.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:29:11.478 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-08T21:29:11.478 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-08T21:29:11.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.x 2024-05-08T21:29:11.763 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:29:11 smithi033 systemd[1]: Stopping Ceph mgr.x for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:11.763 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:29:11 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:29:11.731+0000 7ff7aa921700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:29:11.763 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:29:11 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x[42777]: 2024-05-08T21:29:11.731+0000 7ff7aa921700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-08T21:29:12.178 INFO:journalctl@ceph.mgr.x.smithi033.stdout:May 08 21:29:11 smithi033 bash[140580]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-x 2024-05-08T21:29:12.185 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-08T21:29:12.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:29:12.227 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-08T21:29:12.227 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-08T21:29:12.228 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.y 2024-05-08T21:29:12.569 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:29:12 smithi071 systemd[1]: Stopping Ceph mgr.y for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:12.911 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:29:12 smithi071 bash[88166]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-y 2024-05-08T21:29:12.911 INFO:journalctl@ceph.mgr.y.smithi071.stdout:May 08 21:29:12 smithi071 systemd[1]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-08T21:29:12.917 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-08T21:29:12.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:29:12.953 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-08T21:29:12.953 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-08T21:29:12.954 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.z 2024-05-08T21:29:13.310 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:29:12 smithi082 systemd[1]: Stopping Ceph mgr.z for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:13.602 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:29:13 smithi082 bash[89798]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mgr-z 2024-05-08T21:29:13.602 INFO:journalctl@ceph.mgr.z.smithi082.stdout:May 08 21:29:13 smithi082 systemd[1]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-08T21:29:13.698 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-08T21:29:13.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:29:13.735 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-08T21:29:13.736 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-08T21:29:13.736 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.0 2024-05-08T21:29:14.207 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:29:13 smithi033 systemd[1]: Stopping Ceph osd.0 for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:14.207 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:29:13 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-0[60243]: 2024-05-08T21:29:13.972+0000 7f0963c38700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:29:14.207 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:29:13 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-0[60243]: 2024-05-08T21:29:13.972+0000 7f0963c38700 -1 osd.0 93 *** Got signal Terminated *** 2024-05-08T21:29:14.207 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:29:13 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-0[60243]: 2024-05-08T21:29:13.972+0000 7f0963c38700 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T21:29:19.956 INFO:journalctl@ceph.osd.0.smithi033.stdout:May 08 21:29:19 smithi033 bash[140724]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-0 2024-05-08T21:29:20.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.0.service' 2024-05-08T21:29:21.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:29:21.050 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-08T21:29:21.050 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-08T21:29:21.050 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.1 2024-05-08T21:29:21.569 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:29:21 smithi071 systemd[1]: Stopping Ceph osd.1 for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:21.569 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:29:21 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-1[52049]: 2024-05-08T21:29:21.286+0000 7fcd43abe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:29:21.570 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:29:21 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-1[52049]: 2024-05-08T21:29:21.286+0000 7fcd43abe700 -1 osd.1 93 *** Got signal Terminated *** 2024-05-08T21:29:21.570 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:29:21 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-1[52049]: 2024-05-08T21:29:21.286+0000 7fcd43abe700 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T21:29:27.320 INFO:journalctl@ceph.osd.1.smithi071.stdout:May 08 21:29:27 smithi071 bash[88304]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-1 2024-05-08T21:29:28.266 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.1.service' 2024-05-08T21:29:28.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:29:28.307 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-08T21:29:28.307 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-08T21:29:28.307 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.2 2024-05-08T21:29:28.602 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:29:28 smithi082 systemd[1]: Stopping Ceph osd.2 for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:28.602 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:29:28 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-2[53308]: 2024-05-08T21:29:28.548+0000 7fbdfd583700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:29:28.602 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:29:28 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-2[53308]: 2024-05-08T21:29:28.548+0000 7fbdfd583700 -1 osd.2 93 *** Got signal Terminated *** 2024-05-08T21:29:28.602 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:29:28 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-2[53308]: 2024-05-08T21:29:28.548+0000 7fbdfd583700 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T21:29:34.603 INFO:journalctl@ceph.osd.2.smithi082.stdout:May 08 21:29:34 smithi082 bash[89944]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-2 2024-05-08T21:29:35.738 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.2.service' 2024-05-08T21:29:35.778 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:29:35.778 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-08T21:29:35.779 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-08T21:29:35.779 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.3 2024-05-08T21:29:36.207 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 08 21:29:35 smithi033 systemd[1]: Stopping Ceph osd.3 for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:36.207 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 08 21:29:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-3[68970]: 2024-05-08T21:29:36.035+0000 7fd6f339c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:29:36.207 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 08 21:29:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-3[68970]: 2024-05-08T21:29:36.035+0000 7fd6f339c700 -1 osd.3 93 *** Got signal Terminated *** 2024-05-08T21:29:36.207 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 08 21:29:36 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-3[68970]: 2024-05-08T21:29:36.035+0000 7fd6f339c700 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T21:29:40.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:39 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:39.765+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:40.956 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 08 21:29:40 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-3[68970]: 2024-05-08T21:29:40.594+0000 7fd6eb98a700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:14.910496+0000 front 2024-05-08T21:29:14.910524+0000 (oldest deadline 2024-05-08T21:29:40.210791+0000) 2024-05-08T21:29:40.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:40 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:40.767+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:42.207 INFO:journalctl@ceph.osd.3.smithi033.stdout:May 08 21:29:41 smithi033 bash[140994]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-3 2024-05-08T21:29:42.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:41 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:41.799+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:42.569 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:42 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:42.303+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:42.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:42 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:42.266+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:42.956 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:42 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:42.794+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:42.991 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.3.service' 2024-05-08T21:29:43.044 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:29:43.045 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-08T21:29:43.045 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-08T21:29:43.045 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.4 2024-05-08T21:29:43.570 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 08 21:29:43 smithi071 systemd[1]: Stopping Ceph osd.4 for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:43.570 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 08 21:29:43 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-4[59968]: 2024-05-08T21:29:43.298+0000 7f30168ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:29:43.570 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 08 21:29:43 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-4[59968]: 2024-05-08T21:29:43.298+0000 7f30168ea700 -1 osd.4 93 *** Got signal Terminated *** 2024-05-08T21:29:43.570 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 08 21:29:43 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-4[59968]: 2024-05-08T21:29:43.298+0000 7f30168ea700 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T21:29:43.570 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:43 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:43.283+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:43.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:43 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:43.239+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:44.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:43 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:43.839+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:44.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:44 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:44.265+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:44.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:44 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:44.322+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:45.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:44 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:44.839+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:45.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:45 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:45.246+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:45.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:45 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:45.284+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:45.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:45 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:45.324+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:46.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:45 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:45.852+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:46.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:46 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:46.257+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:46.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:46 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:46.289+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:46.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:46 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:46.354+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:47.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:46 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:46.822+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:47.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:47 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:47.276+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:47.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:47 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:47.237+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:47.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:47 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:47.237+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.686048+0000 front 2024-05-08T21:29:23.685896+0000 (oldest deadline 2024-05-08T21:29:47.185914+0000) 2024-05-08T21:29:47.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:47 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:47.359+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:47.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:47 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:47.359+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:48.102 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:47 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:47.760+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:29:48.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:47 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:47.778+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:48.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:48 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:48.238+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:48.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:48 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:48.278+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:48.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:48 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:48.278+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.686048+0000 front 2024-05-08T21:29:23.685896+0000 (oldest deadline 2024-05-08T21:29:47.185914+0000) 2024-05-08T21:29:48.633 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:48 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:48.373+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:48.633 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:48 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:48.373+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:49.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:48 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:48.631+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:29:49.070 INFO:journalctl@ceph.osd.4.smithi071.stdout:May 08 21:29:48 smithi071 bash[88575]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-4 2024-05-08T21:29:49.081 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:48 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:48.778+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:49.102 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:48 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:48.799+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:29:49.456 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:49 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:49.079+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:29:49.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:49 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:49.194+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:49.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:49 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:49.278+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:49.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:49 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:49.278+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.686048+0000 front 2024-05-08T21:29:23.685896+0000 (oldest deadline 2024-05-08T21:29:47.185914+0000) 2024-05-08T21:29:49.672 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:49 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:49.420+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:49.672 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:49 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:49.420+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:50.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:49 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:49.670+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:29:50.079 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:49 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:49.790+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:50.102 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:49 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:49.768+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:29:50.110 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.4.service' 2024-05-08T21:29:50.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:29:50.152 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-08T21:29:50.153 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-08T21:29:50.153 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.5 2024-05-08T21:29:50.452 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:50 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:50.189+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:50.452 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:50 smithi082 systemd[1]: Stopping Ceph osd.5 for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:50.452 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:50 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:50.261+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:50.452 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:50 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:50.261+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.686048+0000 front 2024-05-08T21:29:23.685896+0000 (oldest deadline 2024-05-08T21:29:47.185914+0000) 2024-05-08T21:29:50.453 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:50 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:50.450+0000 7fc5bfccb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:29:50.453 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:50 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:50.450+0000 7fc5bfccb700 -1 osd.5 93 *** Got signal Terminated *** 2024-05-08T21:29:50.453 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:50 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:50.450+0000 7fc5bfccb700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T21:29:50.456 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:50 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:50.077+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:29:50.820 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:50 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:50.627+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:29:50.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:50 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:50.400+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:50.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:50 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:50.400+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:51.102 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:50 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:50.805+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:29:51.206 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:51 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:51.035+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:29:51.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:50 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:50.836+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:51.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:50 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:50.836+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:29:51.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:51 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:51.220+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:51.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:51 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:51.247+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:51.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:51 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:51.247+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.686048+0000 front 2024-05-08T21:29:23.685896+0000 (oldest deadline 2024-05-08T21:29:47.185914+0000) 2024-05-08T21:29:51.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:51 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:51.627+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:29:51.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:51 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:51.392+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:51.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:51 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:51.392+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:52.206 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:52 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:52.007+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:29:52.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:51 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:51.876+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:52.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:51 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:51.876+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:29:52.247 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:51 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:51.851+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:29:52.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:52 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:52.246+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:52.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:52 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:52.268+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:52.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:52 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:52.268+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.686048+0000 front 2024-05-08T21:29:23.685896+0000 (oldest deadline 2024-05-08T21:29:47.185914+0000) 2024-05-08T21:29:52.677 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:52 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:52.386+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:52.677 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:52 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:52.386+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:53.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:52 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:52.676+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:29:53.206 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:53 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:53.010+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:29:53.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:52 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:52.862+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:53.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:52 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:52.862+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:29:53.249 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:52 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:52.892+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:29:53.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:53 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:53.270+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:53.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:53 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:53.248+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:53.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:53 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:53.248+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.686048+0000 front 2024-05-08T21:29:23.685896+0000 (oldest deadline 2024-05-08T21:29:47.185914+0000) 2024-05-08T21:29:53.692 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:53 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:53.388+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:53.693 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:53 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:53.388+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:54.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:53 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:53.691+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:29:54.207 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:54 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:54.032+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:29:54.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:53 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:53.843+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:54.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:53 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:53.843+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:29:54.252 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:53 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:53.893+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:29:54.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:54 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:54.251+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:54.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:54 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:54.251+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.686048+0000 front 2024-05-08T21:29:23.685896+0000 (oldest deadline 2024-05-08T21:29:47.185914+0000) 2024-05-08T21:29:54.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:54 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:54.281+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:54.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:54 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:54.281+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:29:54.719 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:54 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:54.350+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:54.719 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:54 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:54.350+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:55.053 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:54 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:54.795+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:55.053 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:54 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:54.795+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:29:55.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:54 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:54.718+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:29:55.237 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:54 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:54.874+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:29:55.456 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:55 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:55.052+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:29:55.456 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:55 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:55.052+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:28.085305+0000 front 2024-05-08T21:29:28.085336+0000 (oldest deadline 2024-05-08T21:29:54.485951+0000) 2024-05-08T21:29:55.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:55 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:55.254+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.984117+0000 front 2024-05-08T21:29:17.984218+0000 (oldest deadline 2024-05-08T21:29:42.084068+0000) 2024-05-08T21:29:55.602 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:55 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5[61860]: 2024-05-08T21:29:55.254+0000 7fc5b82b9700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.686048+0000 front 2024-05-08T21:29:23.685896+0000 (oldest deadline 2024-05-08T21:29:47.185914+0000) 2024-05-08T21:29:55.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:55 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:55.236+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:55.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:55 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:55.236+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:29:55.748 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:55 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:55.399+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:55.748 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:55 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:55.399+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:56.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:55 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:55.746+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:29:56.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:55 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:55.746+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:29:56.206 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:56 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:56.008+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:29:56.206 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:56 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:56.008+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:28.085305+0000 front 2024-05-08T21:29:28.085336+0000 (oldest deadline 2024-05-08T21:29:54.485951+0000) 2024-05-08T21:29:56.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:55 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:55.832+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:56.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:55 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:55.832+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:29:56.261 INFO:journalctl@ceph.osd.5.smithi082.stdout:May 08 21:29:56 smithi082 bash[90206]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-5 2024-05-08T21:29:56.261 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:55 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:55.899+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:29:56.562 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:56 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:56.260+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:56.562 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:56 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:56.260+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:29:56.775 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:56 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:56.437+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:56.776 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:56 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:56.437+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:57.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:56 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:56.774+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:29:57.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:56 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:56.774+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:29:57.206 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:56 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:56.983+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:29:57.207 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:56 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:56.983+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:28.085305+0000 front 2024-05-08T21:29:28.085336+0000 (oldest deadline 2024-05-08T21:29:54.485951+0000) 2024-05-08T21:29:57.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:56 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:56.795+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:57.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:56 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:56.795+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:29:57.275 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:56 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:56.936+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:29:57.275 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:56 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:56.936+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:29:57.282 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.5.service' 2024-05-08T21:29:57.323 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:29:57.324 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-08T21:29:57.324 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-08T21:29:57.324 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.6 2024-05-08T21:29:57.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:57 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:57.286+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:57.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:57 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:57.286+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:29:57.706 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:57 smithi033 systemd[1]: Stopping Ceph osd.6 for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:29:57.707 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:57 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:57.574+0000 7f41672f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:29:57.707 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:57 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:57.574+0000 7f41672f9700 -1 osd.6 93 *** Got signal Terminated *** 2024-05-08T21:29:57.707 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:57 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:57.574+0000 7f41672f9700 -1 osd.6 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T21:29:57.731 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:57 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:57.390+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:57.731 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:57 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:57.390+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:58.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:57 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:57.730+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:29:58.070 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:57 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:57.730+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:29:58.207 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:58 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:58.012+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:29:58.207 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:58 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:58.012+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:28.085305+0000 front 2024-05-08T21:29:28.085336+0000 (oldest deadline 2024-05-08T21:29:54.485951+0000) 2024-05-08T21:29:58.208 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:57 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:57.784+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:58.208 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:57 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:57.784+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:29:58.276 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:57 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:57.960+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:29:58.276 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:57 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:57.960+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:29:58.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:58 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:58.275+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:58.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:58 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:58.275+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:29:58.731 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:58 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:58.391+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:58.733 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:58 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:58.391+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:59.052 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:58 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:58.784+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:29:59.052 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:58 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:58.784+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:29:59.053 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:58 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:58.784+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:29:59.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:58 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:58.730+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:29:59.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:58 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:58.730+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:29:59.289 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:58 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:58.994+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:29:59.290 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:29:58 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:29:58.994+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:29:59.456 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:59 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:59.051+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:29:59.456 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:29:59 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:29:59.051+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:28.085305+0000 front 2024-05-08T21:29:28.085336+0000 (oldest deadline 2024-05-08T21:29:54.485951+0000) 2024-05-08T21:29:59.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:59 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:59.288+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:29:59.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:29:59 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:29:59.288+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:29:59.742 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:59 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:59.344+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:29:59.742 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:59 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:59.344+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:29:59.742 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:29:59 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:29:59.344+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:00.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:59 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:59.741+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:30:00.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:29:59 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:29:59.741+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:30:00.094 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:59 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:59.813+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:00.094 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:59 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:59.813+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:00.094 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:29:59 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:29:59.813+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:00.302 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:00 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:00.031+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:00.302 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:00 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:00.031+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:00.456 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:30:00 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:30:00.092+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:30:00.456 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:30:00 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:30:00.092+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:28.085305+0000 front 2024-05-08T21:29:28.085336+0000 (oldest deadline 2024-05-08T21:29:54.485951+0000) 2024-05-08T21:30:00.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:00 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:00.301+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:00.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:00 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:00.301+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:00.732 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:00 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:00.376+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:00.732 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:00 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:00.376+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:00.732 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:00 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:00.376+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:01.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:00 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:00.731+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:30:01.070 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:00 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:00.731+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:30:01.206 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:30:01 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:30:01.107+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:30:01.207 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:30:01 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:30:01.107+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:28.085305+0000 front 2024-05-08T21:29:28.085336+0000 (oldest deadline 2024-05-08T21:29:54.485951+0000) 2024-05-08T21:30:01.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:00 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:00.858+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:01.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:00 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:00.858+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:01.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:00 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:00.858+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:01.319 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:00 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:00.983+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:01.319 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:00 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:00.983+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:01.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:01 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:01.298+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:01.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:01 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:01.298+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:01.714 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:01 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:01.380+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:01.714 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:01 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:01.380+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:01.715 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:01 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:01.380+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:02.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:01 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:01.713+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:30:02.070 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:01 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:01.713+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:30:02.144 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:01 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:01.835+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:02.144 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:01 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:01.835+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:02.145 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:01 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:01.835+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:02.252 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:01 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:01.934+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:02.252 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:01 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:01.934+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:02.456 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:30:02 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:30:02.143+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.984693+0000 front 2024-05-08T21:29:23.984727+0000 (oldest deadline 2024-05-08T21:29:48.084796+0000) 2024-05-08T21:30:02.456 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:30:02 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6[76970]: 2024-05-08T21:30:02.143+0000 7f41638ef700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:28.085305+0000 front 2024-05-08T21:29:28.085336+0000 (oldest deadline 2024-05-08T21:29:54.485951+0000) 2024-05-08T21:30:02.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:02 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:02.250+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:02.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:02 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:02.250+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:02.679 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:02 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:02.346+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:02.679 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:02 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:02.346+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:02.679 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:02 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:02.346+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:03.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:02 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:02.678+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:30:03.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:02 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:02.678+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:30:03.196 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:02 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:02.842+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:03.196 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:02 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:02.842+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:03.196 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:02 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:02.842+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:03.196 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:02 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:02.842+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:03.296 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:02 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:02.917+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:03.296 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:02 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:02.917+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:03.456 INFO:journalctl@ceph.osd.6.smithi033.stdout:May 08 21:30:03 smithi033 bash[141249]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-6 2024-05-08T21:30:03.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:03 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:03.295+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:03.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:03 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:03.295+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:03.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:03 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:03.295+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:03.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:03 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:03.631+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:30:03.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:03 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:03.631+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:30:03.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:03 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:03.384+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:03.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:03 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:03.384+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:03.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:03 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:03.384+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:03.956 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:03 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:03.872+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:03.956 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:03 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:03.872+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:03.956 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:03 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:03.872+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:03.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:03 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:03.872+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:04.337 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:03 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:03.915+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:04.337 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:03 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:03.915+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:04.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.6.service' 2024-05-08T21:30:04.492 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:30:04.493 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-08T21:30:04.493 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-08T21:30:04.493 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.7 2024-05-08T21:30:04.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:04.335+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:04.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:04.335+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:04.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:04.336+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:04.729 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:04.431+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:04.729 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:04.431+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:04.729 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:04.431+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:04.729 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:04 smithi071 systemd[1]: Stopping Ceph osd.7 for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:30:04.729 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:04.648+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:30:04.729 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:04.648+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:30:04.729 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:04.727+0000 7f968261a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:30:05.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:04.727+0000 7f968261a700 -1 osd.7 93 *** Got signal Terminated *** 2024-05-08T21:30:05.069 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:04 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:04.727+0000 7f968261a700 -1 osd.7 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T21:30:05.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:04.903+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:05.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:04.903+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:05.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:04.903+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:05.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:04 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:04.903+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:05.313 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:04.869+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:05.313 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:04.869+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:05.313 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:04 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:04.869+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:40.159320+0000 front 2024-05-08T21:29:40.159460+0000 (oldest deadline 2024-05-08T21:30:04.259349+0000) 2024-05-08T21:30:05.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:05 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:05.312+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:05.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:05 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:05.312+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:05.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:05 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:05.312+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:05.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:05 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:05.683+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:30:05.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:05 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:05.683+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:30:05.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:05 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:05.683+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.326259+0000 front 2024-05-08T21:29:38.326312+0000 (oldest deadline 2024-05-08T21:30:04.726819+0000) 2024-05-08T21:30:05.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:05 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:05.447+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:05.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:05 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:05.447+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:05.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:05 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:05.447+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:06.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:05 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:05.863+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:06.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:05 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:05.863+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:06.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:05 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:05.863+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:06.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:05 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:05.863+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:06.348 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:05 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:05.865+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:06.348 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:05 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:05.865+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:06.348 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:05 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:05.865+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:40.159320+0000 front 2024-05-08T21:29:40.159460+0000 (oldest deadline 2024-05-08T21:30:04.259349+0000) 2024-05-08T21:30:06.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:06 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:06.347+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:06.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:06 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:06.347+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:06.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:06 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:06.347+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:06.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:06 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:06.645+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:30:06.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:06 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:06.645+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:30:06.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:06 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:06.645+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.326259+0000 front 2024-05-08T21:29:38.326312+0000 (oldest deadline 2024-05-08T21:30:04.726819+0000) 2024-05-08T21:30:06.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:06 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:06.467+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:06.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:06 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:06.467+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:06.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:06 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:06.467+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:07.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:06 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:06.828+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:07.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:06 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:06.828+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:07.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:06 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:06.828+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:07.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:06 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:06.828+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:07.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:06 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:06.871+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:07.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:06 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:06.871+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:07.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:06 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:06.871+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:40.159320+0000 front 2024-05-08T21:29:40.159460+0000 (oldest deadline 2024-05-08T21:30:04.259349+0000) 2024-05-08T21:30:07.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:07 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:07.687+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:30:07.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:07 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:07.687+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:30:07.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:07 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:07.687+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.326259+0000 front 2024-05-08T21:29:38.326312+0000 (oldest deadline 2024-05-08T21:30:04.726819+0000) 2024-05-08T21:30:07.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:07 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:07.482+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:07.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:07 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:07.482+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:07.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:07 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:07.482+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:07.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:07 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:07.482+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:07.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:07 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:07.358+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:07.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:07 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:07.358+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:07.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:07 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:07.358+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:08.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:07 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:07.811+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:08.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:07 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:07.811+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:08.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:07 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:07.811+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:08.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:07 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:07.811+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:08.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:07 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:07.874+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:08.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:07 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:07.874+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:08.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:07 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:07.874+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:40.159320+0000 front 2024-05-08T21:29:40.159460+0000 (oldest deadline 2024-05-08T21:30:04.259349+0000) 2024-05-08T21:30:08.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:08 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:08.711+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:24.324222+0000 front 2024-05-08T21:29:24.324240+0000 (oldest deadline 2024-05-08T21:29:47.824336+0000) 2024-05-08T21:30:08.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:08 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:08.711+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.925159+0000 front 2024-05-08T21:29:31.925195+0000 (oldest deadline 2024-05-08T21:29:55.425083+0000) 2024-05-08T21:30:08.820 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:08 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7[68055]: 2024-05-08T21:30:08.711+0000 7f967ec10700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.326259+0000 front 2024-05-08T21:29:38.326312+0000 (oldest deadline 2024-05-08T21:30:04.726819+0000) 2024-05-08T21:30:08.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:08 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:08.516+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:08.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:08 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:08.516+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:08.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:08 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:08.516+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:08.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:08 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:08.516+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:08.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:08 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:08.387+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:08.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:08 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:08.387+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:08.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:08 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:08.387+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:09.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:08 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:08.826+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:09.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:08 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:08.826+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:09.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:08 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:08.826+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:09.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:08 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:08.826+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:09.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:08 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:08.872+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:09.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:08 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:08.872+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:09.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:08 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:08.872+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:40.159320+0000 front 2024-05-08T21:29:40.159460+0000 (oldest deadline 2024-05-08T21:30:04.259349+0000) 2024-05-08T21:30:09.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:09 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:09.518+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:09.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:09 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:09.518+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:09.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:09 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:09.518+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:09.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:09 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:09.518+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:09.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:09 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:09.379+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:09.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:09 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:09.379+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:09.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:09 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:09.379+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:10.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:09 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:09.823+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:10.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:09 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:09.823+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:10.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:09 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:09.823+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:10.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:09 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:09.823+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:10.347 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:09 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:09.898+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:10.347 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:09 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:09.898+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:10.347 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:09 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:09.898+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:40.159320+0000 front 2024-05-08T21:29:40.159460+0000 (oldest deadline 2024-05-08T21:30:04.259349+0000) 2024-05-08T21:30:10.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:10 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:10.346+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:10.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:10 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:10.346+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:10.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:10 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:10.346+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:10.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:May 08 21:30:10 smithi071 bash[88833]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-7 2024-05-08T21:30:10.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:10 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:10.512+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:10.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:10 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:10.512+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:10.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:10 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:10.512+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:10.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:10 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:10.512+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:10.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:10 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:10.512+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:11.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:10 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:10.819+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:11.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:10 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:10.819+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:11.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:10 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:10.819+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:11.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:10 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:10.819+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:11.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:10 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:10.819+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:11.304 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:10 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:10.913+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:11.304 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:10 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:10.913+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:11.304 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:10 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:10.913+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:40.159320+0000 front 2024-05-08T21:29:40.159460+0000 (oldest deadline 2024-05-08T21:30:04.259349+0000) 2024-05-08T21:30:11.570 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:11 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:11.502+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:11.570 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:11 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:11.502+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:11.570 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:11 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:11.502+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:11.570 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:11 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:11.502+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:11.570 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:11 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:11.502+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:11.598 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.7.service' 2024-05-08T21:30:11.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:11 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:11.302+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:11.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:11 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:11.302+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:11.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:11 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:11.302+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:11.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:11 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:11.302+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:11.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:30:11.639 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-08T21:30:11.639 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-08T21:30:11.640 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.8 2024-05-08T21:30:11.938 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:11 smithi082 systemd[1]: Stopping Ceph osd.8 for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:30:11.938 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:11 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:11.874+0000 7f485b6d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:30:11.938 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:11 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:11.874+0000 7f485b6d2700 -1 osd.8 93 *** Got signal Terminated *** 2024-05-08T21:30:11.938 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:11 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:11.874+0000 7f485b6d2700 -1 osd.8 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T21:30:12.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:11 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:11.854+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:12.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:11 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:11.854+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:12.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:11 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:11.854+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:12.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:11 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:11.854+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:12.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:11 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:11.854+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:12.312 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:11 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:11.937+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:12.312 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:11 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:11.937+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:12.312 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:11 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:11.937+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:40.159320+0000 front 2024-05-08T21:29:40.159460+0000 (oldest deadline 2024-05-08T21:30:04.259349+0000) 2024-05-08T21:30:12.312 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:11 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:11.937+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.960311+0000 front 2024-05-08T21:29:45.960370+0000 (oldest deadline 2024-05-08T21:30:11.860333+0000) 2024-05-08T21:30:12.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:12 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:12.312+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:12.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:12 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:12.312+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:12.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:12 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:12.312+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:12.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:12 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:12.312+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:12.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:12 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:12.507+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:12.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:12 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:12.507+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:12.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:12 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:12.507+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:12.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:12 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:12.507+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:12.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:12 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:12.507+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:13.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:12 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:12.854+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:13.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:12 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:12.854+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:13.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:12 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:12.854+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:13.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:12 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:12.854+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:13.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:12 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:12.854+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:13.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:12 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:12.965+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:13.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:12 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:12.965+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:13.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:12 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:12.965+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:40.159320+0000 front 2024-05-08T21:29:40.159460+0000 (oldest deadline 2024-05-08T21:30:04.259349+0000) 2024-05-08T21:30:13.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:12 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:12.965+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.960311+0000 front 2024-05-08T21:29:45.960370+0000 (oldest deadline 2024-05-08T21:30:11.860333+0000) 2024-05-08T21:30:13.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:13 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:13.527+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:13.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:13 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:13.527+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:13.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:13 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:13.527+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:13.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:13 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:13.527+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:13.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:13 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:13.527+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:13.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:13 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:13.353+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:13.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:13 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:13.353+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:13.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:13 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:13.353+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:13.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:13 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:13.353+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:14.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:13 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:13.849+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:14.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:13 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:13.849+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:14.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:13 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:13.849+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:14.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:13 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:13.849+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:14.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:13 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:13.849+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:14.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:13 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:13.921+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:14.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:13 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:13.921+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:14.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:13 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:13.921+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:40.159320+0000 front 2024-05-08T21:29:40.159460+0000 (oldest deadline 2024-05-08T21:30:04.259349+0000) 2024-05-08T21:30:14.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:13 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:13.921+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.960311+0000 front 2024-05-08T21:29:45.960370+0000 (oldest deadline 2024-05-08T21:30:11.860333+0000) 2024-05-08T21:30:14.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:14 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:14.502+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:14.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:14 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:14.502+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:14.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:14 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:14.502+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:14.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:14 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:14.502+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:14.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:14 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:14.502+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:14.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:14 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:14.359+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:14.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:14 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:14.359+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:14.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:14 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:14.359+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:14.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:14 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:14.359+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:15.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:14 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:14.860+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:15.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:14 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:14.860+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:15.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:14 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:14.860+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:15.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:14 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:14.860+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:15.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:14 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:14.860+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:15.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:14 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:14.919+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:15.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:14 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:14.919+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:15.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:14 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:14.919+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:40.159320+0000 front 2024-05-08T21:29:40.159460+0000 (oldest deadline 2024-05-08T21:30:04.259349+0000) 2024-05-08T21:30:15.352 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:14 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:14.919+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.960311+0000 front 2024-05-08T21:29:45.960370+0000 (oldest deadline 2024-05-08T21:30:11.860333+0000) 2024-05-08T21:30:15.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:15 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:15.509+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:15.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:15 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:15.509+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:15.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:15 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:15.509+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:15.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:15 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:15.509+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:15.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:15 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:15.509+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:15.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:15 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:15.363+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:15.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:15 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:15.363+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:15.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:15 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:15.363+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:15.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:15 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:15.363+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:16.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:15 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:15.893+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:16.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:15 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:15.893+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:16.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:15 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:15.893+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:16.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:15 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:15.893+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:16.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:15 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:15.893+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:16.344 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:15 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:15.908+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:23.356310+0000 front 2024-05-08T21:29:23.356289+0000 (oldest deadline 2024-05-08T21:29:46.856084+0000) 2024-05-08T21:30:16.344 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:15 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:15.908+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.558058+0000 front 2024-05-08T21:29:32.558141+0000 (oldest deadline 2024-05-08T21:29:56.058145+0000) 2024-05-08T21:30:16.344 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:15 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:15.908+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:40.159320+0000 front 2024-05-08T21:29:40.159460+0000 (oldest deadline 2024-05-08T21:30:04.259349+0000) 2024-05-08T21:30:16.344 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:15 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8[69972]: 2024-05-08T21:30:15.908+0000 7f4857cc8700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.960311+0000 front 2024-05-08T21:29:45.960370+0000 (oldest deadline 2024-05-08T21:30:11.860333+0000) 2024-05-08T21:30:16.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:16 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:16.343+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:16.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:16 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:16.343+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:16.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:16 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:16.343+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:16.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:16 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:16.343+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:16.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:16 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:16.472+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:16.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:16 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:16.472+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:16.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:16 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:16.472+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:16.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:16 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:16.472+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:16.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:16 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:16.472+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:17.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:16 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:16.938+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:17.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:16 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:16.938+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:17.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:16 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:16.938+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:17.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:16 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:16.938+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:17.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:16 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:16.938+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:17.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:16 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:16.938+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:50.642967+0000 front 2024-05-08T21:29:50.642981+0000 (oldest deadline 2024-05-08T21:30:16.543068+0000) 2024-05-08T21:30:17.602 INFO:journalctl@ceph.osd.8.smithi082.stdout:May 08 21:30:17 smithi082 bash[90465]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-8 2024-05-08T21:30:17.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:17 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:17.350+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:17.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:17 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:17.350+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:17.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:17 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:17.350+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:17.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:17 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:17.350+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:17.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:17 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:17.350+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:17.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:17 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:17.423+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:17.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:17 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:17.423+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:17.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:17 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:17.423+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:17.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:17 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:17.423+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:17.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:17 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:17.423+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:18.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:17 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:17.902+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:18.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:17 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:17.902+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:18.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:17 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:17.902+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:18.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:17 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:17.902+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:18.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:17 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:17.902+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:18.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:17 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:17.902+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:50.642967+0000 front 2024-05-08T21:29:50.642981+0000 (oldest deadline 2024-05-08T21:30:16.543068+0000) 2024-05-08T21:30:18.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:18 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:18.378+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:18.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:18 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:18.378+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:18.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:18 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:18.378+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:18.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:18 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:18.378+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:18.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:18 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:18.378+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:18.620 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.8.service' 2024-05-08T21:30:18.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:30:18.662 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-08T21:30:18.662 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-08T21:30:18.662 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.9 2024-05-08T21:30:18.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:18 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:18.411+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:18.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:18 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:18.411+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:18.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:18 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:18.411+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:18.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:18 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:18.411+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:18.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:18 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:18.411+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:18.956 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:18 smithi033 systemd[1]: Stopping Ceph osd.9 for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:30:18.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:18 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:18.920+0000 7fcf320b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:30:18.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:18 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:18.920+0000 7fcf320b5700 -1 osd.9 93 *** Got signal Terminated *** 2024-05-08T21:30:18.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:18 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:18.920+0000 7fcf320b5700 -1 osd.9 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T21:30:18.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:18 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:18.940+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:18.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:18 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:18.940+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:18.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:18 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:18.940+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:18.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:18 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:18.940+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:18.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:18 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:18.940+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:18.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:18 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:18.940+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:50.642967+0000 front 2024-05-08T21:29:50.642981+0000 (oldest deadline 2024-05-08T21:30:16.543068+0000) 2024-05-08T21:30:19.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:19 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:19.366+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:19.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:19 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:19.366+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:19.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:19 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:19.366+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:19.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:19 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:19.366+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:19.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:19 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:19.366+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:19.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:19 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:19.423+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:19.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:19 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:19.423+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:19.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:19 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:19.423+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:19.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:19 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:19.423+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:19.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:19 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:19.423+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:20.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:19 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:19.939+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:20.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:19 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:19.939+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:20.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:19 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:19.939+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:20.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:19 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:19.939+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:20.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:19 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:19.939+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:20.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:19 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:19.939+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:50.642967+0000 front 2024-05-08T21:29:50.642981+0000 (oldest deadline 2024-05-08T21:30:16.543068+0000) 2024-05-08T21:30:20.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:20 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:20.378+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:20.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:20 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:20.378+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:20.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:20 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:20.378+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:20.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:20 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:20.378+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:20.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:20 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:20.378+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:20.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:20 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:20.441+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:20.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:20 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:20.441+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:20.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:20 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:20.441+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:20.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:20 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:20.441+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:20.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:20 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:20.441+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:21.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:20 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:20.905+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:21.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:20 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:20.905+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:21.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:20 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:20.905+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:21.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:20 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:20.905+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:21.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:20 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:20.905+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:21.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:20 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:20.905+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:50.642967+0000 front 2024-05-08T21:29:50.642981+0000 (oldest deadline 2024-05-08T21:30:16.543068+0000) 2024-05-08T21:30:21.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:21 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:21.348+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:21.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:21 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:21.348+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:21.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:21 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:21.348+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:21.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:21 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:21.348+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:21.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:21 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:21.348+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:21.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:21 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:21.348+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:54.799264+0000 front 2024-05-08T21:29:54.799385+0000 (oldest deadline 2024-05-08T21:30:20.699179+0000) 2024-05-08T21:30:21.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:21 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:21.397+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:21.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:21 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:21.397+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:21.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:21 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:21.397+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:21.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:21 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:21.397+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:21.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:21 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:21.397+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:22.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:21 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:21.880+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:22.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:21 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:21.880+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:22.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:21 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:21.880+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:22.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:21 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:21.880+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:22.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:21 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:21.880+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:22.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:21 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:21.880+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:50.642967+0000 front 2024-05-08T21:29:50.642981+0000 (oldest deadline 2024-05-08T21:30:16.543068+0000) 2024-05-08T21:30:22.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:22 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:22.332+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:22.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:22 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:22.332+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:22.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:22 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:22.332+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:22.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:22 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:22.332+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:22.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:22 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:22.332+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:22.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:22 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:22.332+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:54.799264+0000 front 2024-05-08T21:29:54.799385+0000 (oldest deadline 2024-05-08T21:30:20.699179+0000) 2024-05-08T21:30:22.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:22 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:22.374+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:22.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:22 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:22.374+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:22.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:22 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:22.374+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:22.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:22 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:22.374+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:22.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:22 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:22.374+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:23.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:22 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:22.886+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:23.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:22 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:22.886+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:23.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:22 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:22.886+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:23.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:22 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:22.886+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:23.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:22 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:22.886+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:23.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:22 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:22.886+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:50.642967+0000 front 2024-05-08T21:29:50.642981+0000 (oldest deadline 2024-05-08T21:30:16.543068+0000) 2024-05-08T21:30:23.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:23 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:23.368+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:23.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:23 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:23.368+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:23.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:23 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:23.368+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:23.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:23 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:23.368+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:23.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:23 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:23.368+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:23.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:23 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:23.368+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:54.799264+0000 front 2024-05-08T21:29:54.799385+0000 (oldest deadline 2024-05-08T21:30:20.699179+0000) 2024-05-08T21:30:23.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:23 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:23.404+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:23.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:23 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:23.404+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:23.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:23 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:23.404+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:23.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:23 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:23.404+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:23.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:23 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:23.404+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:24.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:23 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:23.847+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:17.134884+0000 front 2024-05-08T21:29:17.135103+0000 (oldest deadline 2024-05-08T21:29:39.435098+0000) 2024-05-08T21:30:24.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:23 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:23.847+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:25.038004+0000 front 2024-05-08T21:29:25.037903+0000 (oldest deadline 2024-05-08T21:29:50.338030+0000) 2024-05-08T21:30:24.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:23 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:23.847+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:33.239366+0000 front 2024-05-08T21:29:33.239269+0000 (oldest deadline 2024-05-08T21:29:57.939354+0000) 2024-05-08T21:30:24.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:23 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:23.847+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.440452+0000 front 2024-05-08T21:29:38.440467+0000 (oldest deadline 2024-05-08T21:30:02.540524+0000) 2024-05-08T21:30:24.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:23 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:23.847+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:46.542268+0000 front 2024-05-08T21:29:46.542401+0000 (oldest deadline 2024-05-08T21:30:10.642459+0000) 2024-05-08T21:30:24.207 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:23 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9[85725]: 2024-05-08T21:30:23.847+0000 7fcf2e6ab700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:50.642967+0000 front 2024-05-08T21:29:50.642981+0000 (oldest deadline 2024-05-08T21:30:16.543068+0000) 2024-05-08T21:30:24.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:24 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:24.412+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:24.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:24 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:24.412+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:24.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:24 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:24.412+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:24.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:24 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:24.412+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:24.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:24 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:24.412+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:24.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:24 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:24.412+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:54.799264+0000 front 2024-05-08T21:29:54.799385+0000 (oldest deadline 2024-05-08T21:30:20.699179+0000) 2024-05-08T21:30:24.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:24 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:24.412+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:01.800472+0000 front 2024-05-08T21:30:01.800488+0000 (oldest deadline 2024-05-08T21:30:23.400772+0000) 2024-05-08T21:30:24.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:24 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:24.409+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:24.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:24 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:24.409+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:24.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:24 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:24.409+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:24.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:24 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:24.409+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:24.853 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:24 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:24.409+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:24.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:May 08 21:30:24 smithi033 bash[141513]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-9 2024-05-08T21:30:25.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:25 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:25.370+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:25.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:25 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:25.370+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:25.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:25 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:25.370+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:25.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:25 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:25.370+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:25.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:25 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:25.370+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:25.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:25 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:25.370+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:54.799264+0000 front 2024-05-08T21:29:54.799385+0000 (oldest deadline 2024-05-08T21:30:20.699179+0000) 2024-05-08T21:30:25.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:25 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:25.370+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:01.800472+0000 front 2024-05-08T21:30:01.800488+0000 (oldest deadline 2024-05-08T21:30:23.400772+0000) 2024-05-08T21:30:25.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:25 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:25.367+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:25.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:25 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:25.367+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:25.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:25 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:25.367+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:25.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:25 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:25.367+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:25.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:25 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:25.367+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:25.936 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.9.service' 2024-05-08T21:30:25.987 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:30:25.988 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-08T21:30:25.988 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-08T21:30:25.988 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.10 2024-05-08T21:30:26.319 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:26 smithi071 systemd[1]: Stopping Ceph osd.10 for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:30:26.319 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:26 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:26.231+0000 7fb4386cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:30:26.319 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:26 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:26.231+0000 7fb4386cf700 -1 osd.10 93 *** Got signal Terminated *** 2024-05-08T21:30:26.320 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:26 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:26.231+0000 7fb4386cf700 -1 osd.10 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T21:30:26.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:26 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:26.345+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:26.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:26 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:26.345+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:26.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:26 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:26.345+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:26.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:26 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:26.345+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:26.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:26 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:26.345+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:26.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:26 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:26.403+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:26.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:26 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:26.403+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:26.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:26 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:26.403+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:26.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:26 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:26.403+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:26.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:26 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:26.403+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:26.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:26 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:26.403+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:54.799264+0000 front 2024-05-08T21:29:54.799385+0000 (oldest deadline 2024-05-08T21:30:20.699179+0000) 2024-05-08T21:30:26.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:26 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:26.403+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:01.800472+0000 front 2024-05-08T21:30:01.800488+0000 (oldest deadline 2024-05-08T21:30:23.400772+0000) 2024-05-08T21:30:27.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:27 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:27.429+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:27.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:27 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:27.429+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:27.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:27 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:27.429+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:27.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:27 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:27.429+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:27.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:27 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:27.429+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:27.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:27 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:27.429+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:54.799264+0000 front 2024-05-08T21:29:54.799385+0000 (oldest deadline 2024-05-08T21:30:20.699179+0000) 2024-05-08T21:30:27.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:27 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:27.429+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:01.800472+0000 front 2024-05-08T21:30:01.800488+0000 (oldest deadline 2024-05-08T21:30:23.400772+0000) 2024-05-08T21:30:27.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:27 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:27.372+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:27.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:27 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:27.372+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:27.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:27 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:27.372+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:27.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:27 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:27.372+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:27.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:27 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:27.372+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:28.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:28 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:28.327+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:28.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:28 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:28.327+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:28.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:28 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:28.327+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:28.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:28 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:28.327+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:28.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:28 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:28.327+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:28.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:28 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:28.327+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:00.770654+0000 front 2024-05-08T21:30:00.770696+0000 (oldest deadline 2024-05-08T21:30:27.771150+0000) 2024-05-08T21:30:28.821 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:28 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:28.382+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:28.821 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:28 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:28.382+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:28.821 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:28 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:28.382+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:28.821 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:28 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:28.382+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:28.821 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:28 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:28.382+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:28.822 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:28 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:28.382+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:54.799264+0000 front 2024-05-08T21:29:54.799385+0000 (oldest deadline 2024-05-08T21:30:20.699179+0000) 2024-05-08T21:30:28.822 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:28 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:28.382+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:01.800472+0000 front 2024-05-08T21:30:01.800488+0000 (oldest deadline 2024-05-08T21:30:23.400772+0000) 2024-05-08T21:30:29.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:29 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:29.347+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:29.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:29 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:29.347+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:29.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:29 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:29.347+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:29.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:29 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:29.347+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:29.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:29 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:29.347+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:29.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:29 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:29.347+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:54.799264+0000 front 2024-05-08T21:29:54.799385+0000 (oldest deadline 2024-05-08T21:30:20.699179+0000) 2024-05-08T21:30:29.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:29 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:29.347+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:01.800472+0000 front 2024-05-08T21:30:01.800488+0000 (oldest deadline 2024-05-08T21:30:23.400772+0000) 2024-05-08T21:30:29.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:29 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:29.367+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:29.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:29 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:29.367+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:29.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:29 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:29.367+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:29.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:29 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:29.367+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:29.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:29 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:29.367+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:29.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:29 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:29.367+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:00.770654+0000 front 2024-05-08T21:30:00.770696+0000 (oldest deadline 2024-05-08T21:30:27.771150+0000) 2024-05-08T21:30:30.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:30 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:30.373+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.191836+0000 front 2024-05-08T21:29:18.191881+0000 (oldest deadline 2024-05-08T21:29:41.691770+0000) 2024-05-08T21:30:30.819 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:30 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:30.373+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-05-08T21:29:22.792876+0000 front 2024-05-08T21:29:22.792796+0000 (oldest deadline 2024-05-08T21:29:46.893182+0000) 2024-05-08T21:30:30.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:30 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:30.373+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:32.694807+0000 front 2024-05-08T21:29:32.694789+0000 (oldest deadline 2024-05-08T21:29:58.594935+0000) 2024-05-08T21:30:30.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:30 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:30.373+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:39.696177+0000 front 2024-05-08T21:29:39.696271+0000 (oldest deadline 2024-05-08T21:30:07.296735+0000) 2024-05-08T21:30:30.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:30 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:30.373+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:47.297189+0000 front 2024-05-08T21:29:47.297213+0000 (oldest deadline 2024-05-08T21:30:10.197400+0000) 2024-05-08T21:30:30.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:30 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:30.373+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:54.799264+0000 front 2024-05-08T21:29:54.799385+0000 (oldest deadline 2024-05-08T21:30:20.699179+0000) 2024-05-08T21:30:30.820 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:30 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10[76708]: 2024-05-08T21:30:30.374+0000 7fb430cbd700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:01.800472+0000 front 2024-05-08T21:30:01.800488+0000 (oldest deadline 2024-05-08T21:30:23.400772+0000) 2024-05-08T21:30:30.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:30 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:30.377+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:30.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:30 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:30.377+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:30.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:30 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:30.377+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:30.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:30 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:30.377+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:30.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:30 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:30.377+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:30.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:30 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:30.377+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:00.770654+0000 front 2024-05-08T21:30:00.770696+0000 (oldest deadline 2024-05-08T21:30:27.771150+0000) 2024-05-08T21:30:31.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:31 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:31.341+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:31.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:31 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:31.341+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:31.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:31 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:31.341+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:31.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:31 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:31.341+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:31.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:31 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:31.341+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:31.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:31 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:31.341+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:00.770654+0000 front 2024-05-08T21:30:00.770696+0000 (oldest deadline 2024-05-08T21:30:27.771150+0000) 2024-05-08T21:30:31.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:31 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:31.341+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-08T21:30:08.872163+0000 front 2024-05-08T21:30:08.872240+0000 (oldest deadline 2024-05-08T21:30:31.172036+0000) 2024-05-08T21:30:32.069 INFO:journalctl@ceph.osd.10.smithi071.stdout:May 08 21:30:31 smithi071 bash[89089]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-10 2024-05-08T21:30:32.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:32 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:32.381+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:32.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:32 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:32.381+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:32.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:32 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:32.381+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:32.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:32 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:32.381+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:32.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:32 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:32.381+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:32.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:32 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:32.381+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:00.770654+0000 front 2024-05-08T21:30:00.770696+0000 (oldest deadline 2024-05-08T21:30:27.771150+0000) 2024-05-08T21:30:32.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:32 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:32.381+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-08T21:30:08.872163+0000 front 2024-05-08T21:30:08.872240+0000 (oldest deadline 2024-05-08T21:30:31.172036+0000) 2024-05-08T21:30:33.103 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.10.service' 2024-05-08T21:30:33.143 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:30:33.144 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-08T21:30:33.144 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-08T21:30:33.144 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.11 2024-05-08T21:30:33.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:33 smithi082 systemd[1]: Stopping Ceph osd.11 for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:30:33.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:33 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:33.385+0000 7fe5afcee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:30:33.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:33 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:33.385+0000 7fe5afcee700 -1 osd.11 93 *** Got signal Terminated *** 2024-05-08T21:30:33.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:33 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:33.385+0000 7fe5afcee700 -1 osd.11 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T21:30:33.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:33 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:33.396+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:33.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:33 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:33.396+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:33.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:33 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:33.396+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:33.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:33 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:33.396+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:33.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:33 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:33.396+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:33.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:33 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:33.396+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:00.770654+0000 front 2024-05-08T21:30:00.770696+0000 (oldest deadline 2024-05-08T21:30:27.771150+0000) 2024-05-08T21:30:33.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:33 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:33.396+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-08T21:30:08.872163+0000 front 2024-05-08T21:30:08.872240+0000 (oldest deadline 2024-05-08T21:30:31.172036+0000) 2024-05-08T21:30:34.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:34 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:34.365+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:34.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:34 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:34.365+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:34.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:34 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:34.365+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:34.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:34 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:34.365+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:34.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:34 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:34.365+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:34.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:34 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:34.365+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:00.770654+0000 front 2024-05-08T21:30:00.770696+0000 (oldest deadline 2024-05-08T21:30:27.771150+0000) 2024-05-08T21:30:34.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:34 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:34.365+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-08T21:30:08.872163+0000 front 2024-05-08T21:30:08.872240+0000 (oldest deadline 2024-05-08T21:30:31.172036+0000) 2024-05-08T21:30:35.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:35 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:35.356+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:35.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:35 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:35.356+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:35.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:35 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:35.356+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:35.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:35 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:35.356+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:35.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:35 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:35.356+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:35.852 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:35 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:35.356+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:00.770654+0000 front 2024-05-08T21:30:00.770696+0000 (oldest deadline 2024-05-08T21:30:27.771150+0000) 2024-05-08T21:30:35.853 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:35 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:35.356+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-08T21:30:08.872163+0000 front 2024-05-08T21:30:08.872240+0000 (oldest deadline 2024-05-08T21:30:31.172036+0000) 2024-05-08T21:30:36.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:36 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:36.323+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:36.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:36 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:36.323+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:36.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:36 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:36.323+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:36.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:36 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:36.323+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:36.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:36 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:36.323+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:36.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:36 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:36.323+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:00.770654+0000 front 2024-05-08T21:30:00.770696+0000 (oldest deadline 2024-05-08T21:30:27.771150+0000) 2024-05-08T21:30:36.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:36 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:36.323+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-08T21:30:08.872163+0000 front 2024-05-08T21:30:08.872240+0000 (oldest deadline 2024-05-08T21:30:31.172036+0000) 2024-05-08T21:30:37.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:37 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:37.297+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:37.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:37 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:37.297+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:37.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:37 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:37.297+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:37.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:37 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:37.297+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:37.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:37 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:37.297+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:37.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:37 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:37.297+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:00.770654+0000 front 2024-05-08T21:30:00.770696+0000 (oldest deadline 2024-05-08T21:30:27.771150+0000) 2024-05-08T21:30:37.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:37 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:37.297+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-08T21:30:08.872163+0000 front 2024-05-08T21:30:08.872240+0000 (oldest deadline 2024-05-08T21:30:31.172036+0000) 2024-05-08T21:30:38.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:38 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:38.317+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-05-08T21:29:18.962553+0000 front 2024-05-08T21:29:18.962404+0000 (oldest deadline 2024-05-08T21:29:44.862370+0000) 2024-05-08T21:30:38.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:38 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:38.317+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2024-05-08T21:29:31.864124+0000 front 2024-05-08T21:29:31.864144+0000 (oldest deadline 2024-05-08T21:29:54.164331+0000) 2024-05-08T21:30:38.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:38 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:38.317+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-05-08T21:29:38.865545+0000 front 2024-05-08T21:29:38.865570+0000 (oldest deadline 2024-05-08T21:30:02.965683+0000) 2024-05-08T21:30:38.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:38 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:38.317+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-05-08T21:29:45.866820+0000 front 2024-05-08T21:29:45.866841+0000 (oldest deadline 2024-05-08T21:30:11.166848+0000) 2024-05-08T21:30:38.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:38 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:38.317+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2024-05-08T21:29:51.668099+0000 front 2024-05-08T21:29:51.668033+0000 (oldest deadline 2024-05-08T21:30:16.868706+0000) 2024-05-08T21:30:38.602 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:38 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:38.317+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-05-08T21:30:00.770654+0000 front 2024-05-08T21:30:00.770696+0000 (oldest deadline 2024-05-08T21:30:27.771150+0000) 2024-05-08T21:30:38.603 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:38 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11[78598]: 2024-05-08T21:30:38.317+0000 7fe5ac2e4700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-05-08T21:30:08.872163+0000 front 2024-05-08T21:30:08.872240+0000 (oldest deadline 2024-05-08T21:30:31.172036+0000) 2024-05-08T21:30:39.352 INFO:journalctl@ceph.osd.11.smithi082.stdout:May 08 21:30:39 smithi082 bash[90719]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-osd-11 2024-05-08T21:30:40.141 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@osd.11.service' 2024-05-08T21:30:40.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:30:40.182 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-08T21:30:40.182 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-08T21:30:40.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.a 2024-05-08T21:30:40.706 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 08 21:30:40 smithi033 systemd[1]: Stopping Ceph mds.a for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:30:40.707 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 08 21:30:40 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-a[93434]: 2024-05-08T21:30:40.442+0000 7fec54c32700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:30:40.707 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 08 21:30:40 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-a[93434]: 2024-05-08T21:30:40.442+0000 7fec54c32700 -1 mds.a *** got signal Terminated *** 2024-05-08T21:30:44.956 INFO:journalctl@ceph.mds.a.smithi033.stdout:May 08 21:30:44 smithi033 bash[141771]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-a 2024-05-08T21:30:44.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.a.service' 2024-05-08T21:30:45.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:30:45.018 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-08T21:30:45.018 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-08T21:30:45.018 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.d 2024-05-08T21:30:45.300 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:30:45 smithi033 systemd[1]: Stopping Ceph mds.d for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:30:45.706 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:30:45 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-d[92529]: 2024-05-08T21:30:45.298+0000 7f931722d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:30:45.706 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:30:45 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-d[92529]: 2024-05-08T21:30:45.298+0000 7f931722d700 -1 mds.d *** got signal Terminated *** 2024-05-08T21:30:48.953 INFO:journalctl@ceph.mds.d.smithi033.stdout:May 08 21:30:48 smithi033 bash[141909]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-d 2024-05-08T21:30:48.960 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.d.service' 2024-05-08T21:30:48.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:30:48.998 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-08T21:30:48.998 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-08T21:30:48.999 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.g 2024-05-08T21:30:49.284 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 08 21:30:49 smithi033 systemd[1]: Stopping Ceph mds.g for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:30:49.706 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 08 21:30:49 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-g[93049]: 2024-05-08T21:30:49.282+0000 7f1bff952700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:30:49.706 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 08 21:30:49 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-g[93049]: 2024-05-08T21:30:49.282+0000 7f1bff952700 -1 mds.g *** got signal Terminated *** 2024-05-08T21:30:53.706 INFO:journalctl@ceph.mds.g.smithi033.stdout:May 08 21:30:53 smithi033 bash[142047]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-g 2024-05-08T21:30:53.720 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.g.service' 2024-05-08T21:30:53.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:30:53.759 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-08T21:30:53.759 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-08T21:30:53.759 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.j 2024-05-08T21:30:54.066 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:30:53 smithi033 systemd[1]: Stopping Ceph mds.j for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:30:54.456 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:30:54 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-j[92140]: 2024-05-08T21:30:54.063+0000 7fdad730a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:30:54.457 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:30:54 smithi033 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-j[92140]: 2024-05-08T21:30:54.063+0000 7fdad730a700 -1 mds.j *** got signal Terminated *** 2024-05-08T21:30:56.899 INFO:journalctl@ceph.mds.j.smithi033.stdout:May 08 21:30:56 smithi033 bash[142183]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-j 2024-05-08T21:30:56.905 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.j.service' 2024-05-08T21:30:56.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:30:56.944 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-08T21:30:56.944 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-08T21:30:56.944 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.b 2024-05-08T21:30:57.319 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:30:56 smithi071 systemd[1]: Stopping Ceph mds.b for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:30:57.319 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:30:57 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-b[83724]: 2024-05-08T21:30:57.190+0000 7f265459e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:30:57.319 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:30:57 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-b[83724]: 2024-05-08T21:30:57.190+0000 7f265459e700 -1 mds.b *** got signal Terminated *** 2024-05-08T21:31:00.821 INFO:journalctl@ceph.mds.b.smithi071.stdout:May 08 21:31:00 smithi071 bash[89355]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-b 2024-05-08T21:31:00.926 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.b.service' 2024-05-08T21:31:00.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:31:00.963 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-08T21:31:00.963 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-08T21:31:00.963 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.e 2024-05-08T21:31:01.241 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:31:01 smithi071 systemd[1]: Stopping Ceph mds.e for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:31:01.571 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:31:01 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-e[83040]: 2024-05-08T21:31:01.239+0000 7f89ddd50700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:31:01.571 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:31:01 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-e[83040]: 2024-05-08T21:31:01.239+0000 7f89ddd50700 -1 mds.e *** got signal Terminated *** 2024-05-08T21:31:04.819 INFO:journalctl@ceph.mds.e.smithi071.stdout:May 08 21:31:04 smithi071 bash[89494]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-e 2024-05-08T21:31:04.869 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.e.service' 2024-05-08T21:31:04.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:31:04.903 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-08T21:31:04.904 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-08T21:31:04.904 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.h 2024-05-08T21:31:05.165 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:31:04 smithi071 systemd[1]: Stopping Ceph mds.h for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:31:05.569 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:31:05 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-h[83382]: 2024-05-08T21:31:05.163+0000 7fdbbb929700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:31:05.569 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:31:05 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-h[83382]: 2024-05-08T21:31:05.163+0000 7fdbbb929700 -1 mds.h *** got signal Terminated *** 2024-05-08T21:31:08.819 INFO:journalctl@ceph.mds.h.smithi071.stdout:May 08 21:31:08 smithi071 bash[89632]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-h 2024-05-08T21:31:08.854 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.h.service' 2024-05-08T21:31:08.887 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:31:08.887 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-08T21:31:08.887 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-08T21:31:08.887 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.k 2024-05-08T21:31:09.138 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:31:08 smithi071 systemd[1]: Stopping Ceph mds.k for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:31:09.569 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:31:09 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-k[82698]: 2024-05-08T21:31:09.137+0000 7fae02735700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:31:09.569 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:31:09 smithi071 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-k[82698]: 2024-05-08T21:31:09.137+0000 7fae02735700 -1 mds.k *** got signal Terminated *** 2024-05-08T21:31:12.819 INFO:journalctl@ceph.mds.k.smithi071.stdout:May 08 21:31:12 smithi071 bash[89772]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-k 2024-05-08T21:31:12.884 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.k.service' 2024-05-08T21:31:12.919 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:31:12.919 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-08T21:31:12.919 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-08T21:31:12.919 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.c 2024-05-08T21:31:13.352 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:31:12 smithi082 systemd[1]: Stopping Ceph mds.c for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:31:13.352 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:31:13 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-c[84671]: 2024-05-08T21:31:13.160+0000 7ff805c1b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:31:13.352 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:31:13 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-c[84671]: 2024-05-08T21:31:13.160+0000 7ff805c1b700 -1 mds.c *** got signal Terminated *** 2024-05-08T21:31:16.852 INFO:journalctl@ceph.mds.c.smithi082.stdout:May 08 21:31:16 smithi082 bash[90983]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-c 2024-05-08T21:31:16.918 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.c.service' 2024-05-08T21:31:16.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:31:16.954 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-08T21:31:16.954 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-08T21:31:16.954 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.f 2024-05-08T21:31:17.211 INFO:journalctl@ceph.mds.f.smithi082.stdout:May 08 21:31:17 smithi082 systemd[1]: Stopping Ceph mds.f for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:31:17.602 INFO:journalctl@ceph.mds.f.smithi082.stdout:May 08 21:31:17 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-f[84325]: 2024-05-08T21:31:17.208+0000 7f877c99c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:31:17.602 INFO:journalctl@ceph.mds.f.smithi082.stdout:May 08 21:31:17 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-f[84325]: 2024-05-08T21:31:17.208+0000 7f877c99c700 -1 mds.f *** got signal Terminated *** 2024-05-08T21:31:20.797 INFO:journalctl@ceph.mds.f.smithi082.stdout:May 08 21:31:20 smithi082 bash[91123]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-f 2024-05-08T21:31:20.803 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.f.service' 2024-05-08T21:31:20.836 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:31:20.836 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-08T21:31:20.836 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-08T21:31:20.836 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.i 2024-05-08T21:31:21.092 INFO:journalctl@ceph.mds.i.smithi082.stdout:May 08 21:31:20 smithi082 systemd[1]: Stopping Ceph mds.i for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:31:21.352 INFO:journalctl@ceph.mds.i.smithi082.stdout:May 08 21:31:21 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-i[85015]: 2024-05-08T21:31:21.091+0000 7efdd6542700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:31:21.352 INFO:journalctl@ceph.mds.i.smithi082.stdout:May 08 21:31:21 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-i[85015]: 2024-05-08T21:31:21.091+0000 7efdd6542700 -1 mds.i *** got signal Terminated *** 2024-05-08T21:31:24.352 INFO:journalctl@ceph.mds.i.smithi082.stdout:May 08 21:31:24 smithi082 bash[91262]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-i 2024-05-08T21:31:24.360 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.i.service' 2024-05-08T21:31:24.394 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:31:24.394 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-08T21:31:24.394 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-08T21:31:24.395 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.l 2024-05-08T21:31:24.649 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:31:24 smithi082 systemd[1]: Stopping Ceph mds.l for 53ae3a5a-0d7f-11ef-bc97-c7b262605968... 2024-05-08T21:31:25.102 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:31:24 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-l[83975]: 2024-05-08T21:31:24.648+0000 7f4e5d853700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T21:31:25.102 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:31:24 smithi082 ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-l[83975]: 2024-05-08T21:31:24.648+0000 7f4e5d853700 -1 mds.l *** got signal Terminated *** 2024-05-08T21:31:28.852 INFO:journalctl@ceph.mds.l.smithi082.stdout:May 08 21:31:28 smithi082 bash[91401]: ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968-mds-l 2024-05-08T21:31:28.879 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-53ae3a5a-0d7f-11ef-bc97-c7b262605968@mds.l.service' 2024-05-08T21:31:28.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T21:31:28.913 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-08T21:31:28.914 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T21:31:29.119 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:31:30.695 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T21:31:30.880 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:31:32.266 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T21:31:32.455 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:31:33.839 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:31:33.879 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:31:33.919 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T21:31:33.950 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-08T21:31:33.952 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035/remote/smithi033/crash 2024-05-08T21:31:33.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/crash -- . 2024-05-08T21:31:34.000 INFO:teuthology.orchestra.run.smithi033.stderr:tar: /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T21:31:34.000 INFO:teuthology.orchestra.run.smithi033.stderr:tar: Error is not recoverable: exiting now 2024-05-08T21:31:34.002 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035/remote/smithi071/crash 2024-05-08T21:31:34.003 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/crash -- . 2024-05-08T21:31:34.043 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T21:31:34.043 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-05-08T21:31:34.045 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035/remote/smithi082/crash 2024-05-08T21:31:34.046 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/crash -- . 2024-05-08T21:31:34.080 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T21:31:34.080 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now 2024-05-08T21:31:34.081 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-08T21:31:34.081 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-08T21:31:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:grep: /var/log/ceph/53ae3a5a-0d7f-11ef-bc97-c7b262605968/ceph.log: No such file or directory 2024-05-08T21:31:34.126 INFO:tasks.cephadm:Compressing logs... 2024-05-08T21:31:34.126 DEBUG:teuthology.orchestra.run.smithi033:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T21:31:34.169 DEBUG:teuthology.orchestra.run.smithi071:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T21:31:34.172 DEBUG:teuthology.orchestra.run.smithi082:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T21:31:34.199 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T21:31:34.201 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T21:31:34.201 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-08T21:34:07.855 INFO:tasks.cephadm:Archiving logs... 2024-05-08T21:34:07.855 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035/remote/smithi033/log 2024-05-08T21:34:07.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T21:34:24.270 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035/remote/smithi071/log 2024-05-08T21:34:24.270 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T21:34:44.620 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035/remote/smithi082/log 2024-05-08T21:34:44.621 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T21:35:04.819 INFO:tasks.cephadm:Removing cluster... 2024-05-08T21:35:04.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 --force 2024-05-08T21:35:05.018 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:35:05.404 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 --force 2024-05-08T21:35:05.592 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:35:05.987 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 53ae3a5a-0d7f-11ef-bc97-c7b262605968 --force 2024-05-08T21:35:06.175 INFO:teuthology.orchestra.run.smithi082.stdout:Deleting cluster with fsid: 53ae3a5a-0d7f-11ef-bc97-c7b262605968 2024-05-08T21:35:06.569 INFO:tasks.cephadm:Removing cephadm ... 2024-05-08T21:35:06.569 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T21:35:06.592 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T21:35:06.611 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T21:35:06.629 INFO:tasks.cephadm:Teardown complete 2024-05-08T21:35:06.629 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T21:35:06.640 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T21:35:06.641 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T21:35:06.643 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T21:35:06.654 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T21:35:06.773 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T21:35:06.782 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T21:35:06.782 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T21:35:06.785 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T21:35:06.789 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T21:35:06.803 INFO:teuthology.orchestra.run.smithi071.stderr:bash: ntpq: command not found 2024-05-08T21:35:06.804 INFO:teuthology.orchestra.run.smithi033.stderr:bash: ntpq: command not found 2024-05-08T21:35:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:bash: ntpq: command not found 2024-05-08T21:35:06.809 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T21:35:06.809 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-05-08T21:35:06.809 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 +1783us[+1799us] +/- 68ms 2024-05-08T21:35:06.809 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 63 -1573us[-1557us] +/- 74ms 2024-05-08T21:35:06.809 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 -424us[ -424us] +/- 93ms 2024-05-08T21:35:06.809 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:35:06.810 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T21:35:06.810 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-05-08T21:35:06.810 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +1759us[+1762us] +/- 68ms 2024-05-08T21:35:06.810 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -1596us[-1593us] +/- 74ms 2024-05-08T21:35:06.810 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 -438us[ -434us] +/- 93ms 2024-05-08T21:35:06.810 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:35:06.814 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T21:35:06.815 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-05-08T21:35:06.815 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 +1773us[+1790us] +/- 68ms 2024-05-08T21:35:06.815 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -1581us[-1564us] +/- 74ms 2024-05-08T21:35:06.815 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 -416us[ -399us] +/- 93ms 2024-05-08T21:35:06.815 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T21:35:06.815 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T21:35:06.826 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T21:35:06.826 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T21:35:06.837 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T21:35:06.942 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T21:35:07.009 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T21:35:07.076 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T21:35:07.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:35:07.115 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T21:35:07.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:35:07.146 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-05-08T21:35:07.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:35:07.176 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T21:35:07.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T21:35:07.198 INFO:teuthology.task.internal:Duration was 2409.074649 seconds 2024-05-08T21:35:07.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T21:35:07.209 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T21:35:07.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T21:35:07.212 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T21:35:07.214 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T21:35:07.259 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T21:35:07.263 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T21:35:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T21:35:07.581 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T21:35:07.582 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-05-08T21:35:07.582 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T21:35:07.611 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-05-08T21:35:07.612 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T21:35:07.633 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-05-08T21:35:07.633 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T21:35:07.657 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T21:35:07.657 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T21:35:07.660 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T21:35:07.675 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T21:35:07.748 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T21:35:07.748 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T21:35:07.750 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T21:35:07.753 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T21:35:07.878 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T21:35:07.889 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T21:35:07.890 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T21:35:07.920 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T21:35:07.947 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T21:35:07.976 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T21:35:07.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T21:35:07.990 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T21:35:07.993 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T21:35:08.017 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-05-08T21:35:08.020 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-05-08T21:35:08.042 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-05-08T21:35:08.067 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T21:35:08.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:35:08.100 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T21:35:08.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:35:08.118 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T21:35:08.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T21:35:08.136 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T21:35:08.148 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T21:35:08.150 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035/remote/smithi033 2024-05-08T21:35:08.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T21:35:08.203 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035/remote/smithi071 2024-05-08T21:35:08.204 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T21:35:08.251 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_19:22:21-fs-wip-pdonnell-testing-20240508.150423-reef-distro-default-smithi/7698035/remote/smithi082 2024-05-08T21:35:08.252 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T21:35:08.294 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T21:35:08.294 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T21:35:08.297 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T21:35:08.299 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T21:35:08.344 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T21:35:08.358 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T21:35:08.358 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T21:35:08.369 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T21:35:08.369 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T21:35:08.371 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T21:35:08.374 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T21:35:08.388 INFO:teuthology.orchestra.run.smithi071.stdout: 262179 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 21:35 /home/ubuntu/cephtest 2024-05-08T21:35:08.391 INFO:teuthology.orchestra.run.smithi033.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 21:35 /home/ubuntu/cephtest 2024-05-08T21:35:08.401 INFO:teuthology.orchestra.run.smithi082.stdout: 262179 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 21:35 /home/ubuntu/cephtest 2024-05-08T21:35:08.402 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T21:35:08.415 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T21:35:08.443 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/fsx}} duration: 2409.0746490955353 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-08T21:35:08.446 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T21:35:08.520 INFO:teuthology.run:pass